Past decade has shown an overwhelming use of social media. Inspite of multi-fold benefits of using social media platforms, it is being misused due to increasing prevalence of inappropriate content in form of hate speech and abusive language shared on these platforms. Hate speech content is one such form of content that has shown dramatic increase in recent years. Automated techniques like AI and NLP have reported significant performance in detection of hate speech content. In order to extract the textual properties from online content that help in the detection of hate speech, feature extraction and representation using NLP are essential. There is an increasing trend in academia and industry to the use of pretrained neural network language models for hate speech and offensive content detection. This paper presents the performance of important feature extraction techniques: TD-IDF, GloVe, FastText and BERT for binary classification and multiclass classification of hate speech with Convolutional Neural Network (CNN) architecture as the base network for all the techniques. © 2022 IEEE.