Today ovarian cancer is second most perilous cause of cancer deaths in women after breast cancer. In this work, we have developed system which acquires ultrasound images and using image processing and machine learning algorithms accurately classify benign and malignant tumors in ovarian cancer. This technique denoise image using wavelet transform, grey level texture features extracted using GLCM(grey level co-occurrence algorithm), extracted features will be trained through SVM(Support vector machine) and selected non-redundant features selected through Relief-F will be further train and test through SVM for output. Proposed technique was validated by 60 malignant and 60 benign images of patients. On evaluating classifier for 14-texture descriptors give 74% and relief-F gives 82% accuracy. After selecting 6 features from 14 features it will give accuracy 86% and relief-F gives 92% accuracy. Thus, the features are significant for result and preliminary results depict that the proposed technique can be reliable for ovarian tumor classification as this system is fully automated, advantageous and cost-effective too. © 2015 IEEE.