A plant disease of any kind has a significant impact on the yield and quality of a harvest. The tomato is an important crop with a high commercial value on the global market. Early disease detection is critical for a successful crop yield. Plant disease has recently been the subject of a slew of studies. This research proposes the use of leaf pictures to classify tomato illnesses. Deep learning outperforms machine learning in a number of ways, and this is an important step forward in terms of categorization accuracy and the global reach of applications. ResNet50 and Xception architecture were used in this paper for comparison. Three different optimizers Adam, Nadam, and RMSProp have been used with learning rates of 0.0001, 0.002, and 0.04. All tests were done with data that was available to the public. Rotation, zooming, height shifting, breadth shifting, and other data augmentation techniques are used. Xception Architecture with Adam optimizer and Learning rate of 0.0001 gives greater accuracy, recall, precision, and F-score values of 99% on average when compared to Nadam and RMSProp optimizers and learning rates. These values are higher than those obtained with any other combination of learning rates and optimizers. © 2023, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.