Human activity detection has become a prime research area in computer vision problem. There are many applications where human activity detection can be incorporated such as analyzing the video data which will help to monitor data generated by closed-circuit television (CCTV) cameras. In this paper, a system is proposed which uses convolutional neural network along with incremental learning to detect different type human activities in videos. The proposed system is evaluated on benchmark dataset, KTH dataset. © 2022 IEEE.