Communication is the most important aspect in today’s world. Being social helps us to grow andunderstand people. But communication is only possible when there are two or more people who have the necessarymeans for communication. Today’s technology has made communication possible over very long distances, but thebasic need for communication is the power of hearing and speech. This is unfortunate in the case of some people (i.e.deaf and dumb). As we know that the deaf and dumb communicate using sign languages, we can use technology tomake the most of this ability for communication. In this paper, we have proposed a dynamic gesture recognition systemwhich will identify the letters of the English Language and convert them to text and speech. The major steps involvedare Video Acquisition, Pre-processing, Color Conversion, Color Localization, Template Processing and GestureDetection. This system will help in making communication between deaf and dumb people and normal people possible.