Improving critical infrastructure security through hybrid embeddings for vulnerability classification

Abstract

The growing prevalence of vulnerabilities in embedded devices poses a significant risk to critical infrastructure. While deep learning has advanced vulnerability classification, its effectiveness is often hindered by limitations in word representation. Traditional word embeddings struggle with out-of-vocabulary (OOV) words common in domain-specific reports, while pre-trained language models (PLMs), despite their contextual power, may lack specialized domain knowledge. To address these challenges, we propose a novel Two-Stream hybrid embedding architecture that combines Vuln2Vec, a custom domain-specific word embedding, with a large pre-trained language model (PLM) using a learnable weighted feature fusion. Our approach leverages the rich domain-specific vocabulary of Vuln2Vec to understand specialized terminology, while the PLM captures broader contextual relationships and effectively handles OOV words. We validate our method through rigorous experiments, including ablation studies and comparative analyses on vulnerability databases such as the National Vulnerability Database (NVD), the Chinese Vulnerability Database (CNNVD), and a challenging manually collected dataset. Our experiments demonstrate that the proposed hybrid embedding method achieves a state-of-the-art F1-score of 94.25% and an accuracy of 94.88% on the challenging test dataset, validating the superiority of fusing specialized and general-purpose knowledge for this critical task.