Nowadays, there is a rapid growth in the number of video cameras at public and private sector because of the monitoring and security purposes. As video surveillance using Closed Circuit Television (CCTV) is in boom nowadays, it has got more research attention due to increased global security concerns. This rapidly growing data can be used to automatically detect the anomalous activities which are going around in our surrounding. Anomalous activity is something that deviates from its normal nature or something that opposes the normal events. This research mainly focuses on detecting anomalous activities in crowded scenes by using video data. Automatically detecting the anomalous activity without using the handcrafted feature has become the need of the hour. This paper contains a survey of different approaches used for anomaly detection in the past. Different incremental and transfer learning approaches are discussed in this paper and it was found that incremental learning has not been used for video-based anomalous activity detection.