The brain tumor is a cluster of the abnormal tissues, and it is essential to categorize brain tumors for treatment using Magnetic Resonance Imaging (MRI). The segmentation of tumors from brain MRI is understood to be complicated and also crucial tasks. It can be further use in surgery, medical preparation, and assessments. In addition to this, the brain MRI classification is also essential. The enhancement of machine learning and technology will aid radiologists in diagnosing tumors without taking invasive steps. In this paper, the method to detect a brain tumor and classification has been present. Brain tumor detection processes through pre-processing, skull stripping, and tumor segmentation. It is employing a thresholding method followed by morphological operations. The number of training image influences the feature extracted by the CNN, then CNN models overfit after some epoch. Hence, deep learning CNN with transfer learning techniques has evolved. The tumorous brain MRI is classified using CNN based AlexNet architecture. Further, the malignant brain tumor is classified using GooLeNet transfer learning architecture. The performance of this approach is evaluated by precision, recall, F-measure, and accuracy metrics.