Sentiment analysis of trending tweets using Spark NLP and deep learning: a benchmark study of CNN vs transformer models

Abstract

This study investigates the implementation of different sentiment analysis models, exploring their theoretical foundations, robust evaluation criteria, and significant findings, and integrating natural language processing methods to preprocess data collected from social media, the main source of information. The goal of this study is to present notable advancements in sentiment classification by implementing innovative models, including Convolutional Neural Networks and Transformer-based architectures such as BERT, DistilBERT, and RoBERTa, with Spark NLP for preprocessing. The tasks to be performed include: collecting the dataset from Twitter/X focusing particularly on trending topics that appear daily; performing preprocessing steps using Spark NLP, a prominent and scalable NLP library built on Apache Spark to handle scalable and distributed processing of textual data; assigning the polarity for each tweet by applying VADER lexicon-based tool; pretraining both TextCNN and Transformer-based models on lexicon-based VADER labels under identical parameters; fine-tuning the models on manually annotated tweets; comparing their effectiveness, evaluating their strengths, weaknesses, and overall performance in sentiment classification that can guide model selection in resource-constrained settings. The methods used include Spark NLP pipelines, lexicon-based weak labeling, two-phase supervised learning on Convolutional Neural Networks (TextCNN), and Transformer-based models (BERT, DistilBERT, RoBERTa) with early stopping and learning-rate warm-up strategies, and finally the comparative evaluation metrics. The dataset used in this work (12,422 tweets, including 3,475 manually labeled tweets) is not large. The use of Apache Spark enables distributed data processing and supports scalability for larger datasets. The results show that the Transformer-based models outperformed TextCNN model in terms of classification accuracy and robustness. RoBERTa achieved the highest accuracy (85%), followed by BERT and DistilBERT (84%) and TextCNN (72%). DistilBERT balances the predictive performance and computational efficiency well. Conclusions. The scientific novelty of this study focuses particularly on the integration of Apache Spark NLP preprocessing, followed by a benchmarking of CNN and Transformer-based models trained on trending tweets in a hybrid two-phase learning strategy that combined lexicon-based weak supervision with human annotation under the same experimental conditions to provide methodological insights and select appropriate model in our sentiment classification experiments.