The past few decades have witnessed an increase in dietary ailments, majorly caused due to unhealthy food habits. According to experts, mobile-based diet monitoring and assessment systems can capture real-time images of various food items as well as analyze the nutritional content present in it and can be very convenient to use and assist in improving food habits. This can help people lead a healthier lifestyle. This proposed model provides an innovative system that can automatically estimate various food attributes like the nutrients and ingredients by classifying the food image that is given as input. The approach involves the use of different types of deep learning models for accurate food item identification. Apart from image recognition and analysis, the food ingredients, nutrients and attributes are obtained and estimated by extracting words which are semantically related from a large collection of text, accumulated over the Internet. Experimentation has been performed with the Food-101 dataset. The proposed system assists the user to obtain the nutritional value of the food item in real-time which is effective and simple to use. The proposed system also provides supporting features such as food logging, calorie tracking and healthy recipe recommendations for self-monitoring of the user. © 2021, The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.