Leveraging ChatGPT for Enhancing Arabic NLP: Application for Semantic Role Labeling and Cross-Lingual Annotation Projection

Abstract

Semantic role labeling involves assigning semantic roles to sentence arguments, providing rich information for various NLP tasks and applications. Annotated corpora with semantic roles are a critical factor in improving the performance of semantic-based models. Besides, Arabic as a low resourced language, have to pay more attention to alternative methods to build such annotated corpora. To this end, two traditional methods have been intensively used, namely, manual annotation and crowed-resourced annotation. The former is highly precise but it demands substantial training and extensive resources, while, the latter, reduce human effort but often results in lower-quality annotations. Recently, Large language model (LLM) based conversational systems like ChatGPT have emerged as a promising tool for text annotation across various NLP tasks. In this paper, we leverage ChatGPT for two main sub-tasks in Arabic language processing. (1) Creating an Arabic annotated resource with emotional semantic roles from an English corpus, using cross-lingual annotation projection approach. (2) Annotating the Arabic corpus of emotional sentences with emotion categories and semantic roles. Furthermore, we evaluate ChatGPT’s potential for translating English sentences into Arabic. From the perspective of generalization, we test the performance of open-LLMs, specifically, mBERT, and mBART for the same tasks. The evaluation process includes assessing the impact of sentence complexity on the performance of ChatGPT, and open-LLMs in semantic role labeling, and cross-lingual annotation projection. We compared the obtained zero-shot annotation accuracy with that of human base annotations, where the GPT results achieved an accuracy of 0.94 for cross-lingual projection and 0.76 in semantic role labelling, While the open-LLMs achieved notable accuracies of 0.72, and 0.38 respectively.