India is worldwide well known for exporting fruits, having a massive significance in the world. Global food security is essential for the durable production of fruits as well as for a remarkable reduction in pre and post-harvest waste. Harvesting and estimating the ripeness of fruits by a human is an expensive, laborious and time-consuming task. Ripeness estimation is carried out on single fruit like orange, apple, tomato, banana, papaya and etc., using color features. By taking into account increasing productivity of grapes and there is need to focus on ripeness estimation of grapes at the correct time. In this paper, we proposed a methodology that classifies grapes image into ripen and unripen category. A local breed of grape 'Sonaka' was examined during the harvest season from January to March 2019. The images were separated into two ripen categories, e.g. unripen and ripen according to the color and shape of grapes. This image was subjected to a classification model like Convolutional Neural Network (CNN) and support vector machine (SVM). Color features such as RGB and HSV and morphological features such as the shape of grapes were chosen as features for this classification model. The validation result shows that the CNN model achieves higher classification accuracy with 79.49% than the SVM classifier having 69%. © 2019 IEEE.