Classification of imbalance data is the major challenge to the community these days. Machine learning algorithms can evolve a one-sided classifier when data is imbalance. The vital challenge in imbalance dataset problem is that sometimes the minority (tiny) classes are more useful, but standard classifiers tend to be biased toward the majority (huge) classes and ignore the tiny ones. In this paper we compared the existing methods to handle imbalance dataset and provide a new hybrid approach which will improve the accuracy of classifier on imbalanced data. © 2016 IEEE.