Enhancing Sentiment-Driven Recommender Systems With LLM-Based Feature Engineering: A Case Study in Drug Review Analysis

Abstract

Sentiment analysis is vital for evaluating user feedback in drug reviews because understanding patient experiences leads to more personalized treatment recommendations by providing insights into the real-world effectiveness and tolerability of medications, which are often overlooked in clinical trials. This study evaluates the effectiveness of word-level and sentence-level embeddings for feature extraction in sentiment analysis. These embeddings are used in sequential models (Bi-LSTM, CNN) and non-sequential models (Random Forest, DNN, ExtraTreesClassifier). The Random Forest model with LLM2Vec achieves the best performance, with 0.93 accuracy, F1-scores of 0.95 (positive) and 0.88 (negative), and precision scores of 0.93 (positive) and 0.94 (negative). This approach detects subtle negative feedback often missed by standard models. To capture social consensus in patient feedback, we introduce Adaptive Confidence-Weighted Scoring. This method leverages social validation as an implicit confidence signal, enabling sentiment scores to reflect both individual experiences and community agreement. It enhances trust and interpretability while standardizing sentiment polarity on a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$[-5, +5]$ </tex-math></inline-formula> scale. Clinically validated drug-related information, scraped from trusted sources, is encoded using the Llama-3.2-3B-Instruct model to extract context-aware representations. Features derived from this external medical knowledge improve semantic understanding, ensure grounding and safety, and enhance robustness even when reviews are sparse or noisy. The final vectors are employed within a cosine similarity framework to recommend relevant drugs, aligning recommendations with user satisfaction. This work demonstrates how LLM-based feature engineering can advance clinically valid, patient-aligned healthcare recommender systems informed by real-world feedback.