An Empirical Study of Federated BERT for Decentralized Twitter Sentiment Analysis

Abstract

Twitter/x has become a key platform for analyzing public opinion on a large scale; however, traditional centralized approaches raise significant concerns regarding privacy and data governance. To address these challenges, this paper presents an empirical study of a federated learning approach based on a BERT model for decentralized sentiment analysis at the tweet level. This study focuses on evaluating the effectiveness of transformer-based models under realistic non-independent and identically distributed (non-IID) data distributions across distributed clients. The proposed approach enables collaborative model training without sharing raw tweet data, thereby preserving user privacy while leveraging knowledge from multiple sources. The model is evaluated over 100 communication rounds using the Sentiment140 dataset, distributed among four clients with heterogeneous data distributions. Experimental results demonstrate stable convergence and robust performance, with an accuracy of 95.00%, an F1 score of 95.00%, and a PR-AUC of 96.76%. It should be noted that the federated model performs within 1.2% of a centralized baseline, indicating minimal performance degradation despite data sharing constraints.