Deep learning is current buzz word in domain of computer vision. In this work, a method for human action recognition based on a variation of General Convolutional Neural Network (GCNN), called Scaled CNN (SCNN) is proposed. In GCNN, weights of the network are updated in every epoch of training to minimize the classification error. In SCNN, the weighs are first computed using gradient descent algorithm as in GCNN, and then multiplied by scaling factor. Scaling factor is calculated using statistical measures, mean and standard deviation of the frames. Since statistical measures vary from video to video, scaling factor adapts to these changes. As the statistical information from the frames is directly used to alter the weights, it results in minimizing the error faster as compared to GCNN. Results of the proposed method prove that higher accuracy can be achieved with less number of epochs if scaling is used.