This work provides a novel segmentation and classification of diseases from the GI tract using WCE images. First, the input WCE images are collected from the KID Atlas data set, and they are preprocessed using filtering and contrast enhancement techniques. Then, the modified U-Net with parameter tuning by the Deer Hunting with Distance-based Solution Update (DH-DSU) is adopted for performing the lesion or abnormality segmentation. From the segmented images, the feature extraction is employed by a set of approaches such as ‘Gray Level Co-occurrence Matrix (GLCM), Scale-Invariant Feature Transform (SIFT), Histogram of Oriented Gradients (HOG), Local Binary Pattern (LBP) and Canny edge detection’. These features are concatenated and given to the improved Deep Neural Network (DNN) with DH-DSU-based hidden neuron optimisation. This provides the classification outcome in terms of three diseases of the GI tract such as inflammatory, polypoid and ulcer. Through the performance analysis, MCC of DH-DSU-IDNN is 8.49%, 1.73%, 18.46% and 17.23% more than DHOA-IDNN, WOA-IDNN, GWO-IDNN and PSO-IDNN, respectively. At the end, the offered approach has been validated on collected data sets, and the achievement of findings is adequate when compared with the baseline approaches. © 2022 Informa UK Limited, trading as Taylor & Francis Group.