The visual quality of an encoded frame is related to the bits it consumes. To maintain stable quality throughout the video sequence, one needs to consider the distribution of the bit budget for each frame. Within the constraints of bit rate, in order to achieve high visual quality, number of parameters can be considered which include amount of motion present in the video, human visual system, bit allocation, quantisation parameter assignment, motion estimation and mode selection. This paper presents a method, which jointly considers the human visual systems and quality of video to be encoded for rate control. Improvement in performance is obtained as compared to TM5 algorithm used in MPEG2. The proposed technique evaluates the perceptual distortion sensitivity on a macroblock basis, and allocates fewer bits to regions permitting large perceptual distortions for rate reduction. © 2007 IEEE.