Extracting keyframes for action recognition purposes is a challenging task, as compression is to be achieved without losing the impression of the action. Insufficient or wrong keyframes can lead to confusion in the type of action. Proposed two-level key frame extraction algorithm uses an adaptive threshold technique to identify most dissimilar frames as keyframes. At the first level, global features based on intensity histogram and at second level local features computed from wavelet decomposition are used as similarity measures. A new performance parameter, Compression Ratio-Normalized Fidelity (CRNF) is introduced for evaluating the keyframe extraction algorithm. Average CRNF of 0.81 is achieved by the proposed algorithm, as compared 0.37 achieved by existing histogram-based method for action recognition dataset. CRNF value of 0.95 is achieved for Open Video project dataset, which is more than existing methods. Qualitative results further prove the effectiveness of the proposed algorithm.