Determining the image sentiment is a tedious task for classification algorithms, owing to complexities in the raw images as well as the intangible nature of human sentiments. Classifying image sentiments is an evergreen research area, especially in social data analytics. In current times, it is a common practice for majority people to precise their feelings on the web by substituting text with the upload of images via a multiplicity of social media sites like Facebook, Instagram, Twitter as well as any other platform. To identify the emotions from visual cues, some visual features as well as image processing techniques are used. Several existing systems have already introduced emotion detection using machine learning techniques, but the traditional feature extraction strategies do not achieve the required accuracy on random objects. In the entire process, normalization of image, feature extraction, and feature selection are important tasks in the train module. This work articulates the newest developments in the field of image sentiment employing deep learning techniques. Also, the use of conventional machine learning techniques is compared along with deep learning algorithms. It is indicative that a combination of fast recurrent neural networks and CNN may produce high accuracy with minimum time complexity. It is noted from the survey that existing researchers reflect CNN provides around 96.50% average accuracy for sentiment classification on the flicker image dataset. © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd 2021.