Human activity recognition framework is facing many challenges and issues that are promoting the development of a new activity recognition system to improve accuracy under realistic conditions. Therefore, efficient human activity detection is proposed in this paper. At first, the input video is converted into frames then the keyframes are selected using a structural similarity measure. Next, the local and global features are selected from the video frames. Finally, the selected features are fed to the classifier for identifying human activity. In the proposed method, the traditional deep learning algorithm is improved by applying optimization techniques. Adaptive Monarch Butterfly Optimization algorithm (AMBO) is used to improve the performance of the traditional deep learning algorithm. Monarch Butterfly Optimization Algorithm (MBO) is a population-based natural inspired algorithm. It mimics the foraging and the social behavior of the butterflies. The optimal parameters are selected for activity detection. The proposed method is computed based on accuracy, sensitivity, and specificity and implemented using the MATLAB platform. The experimental results show activity recognition accuracy improved to 96%. © 2021, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.