Training an LSTM-based Seq2Seq Model on a Moroccan Biscript Lexicon

Abstract

In today’s increasingly multilingual world, where diverse scripts coexist, training natural language processing (NLP) models on biscript lexicons holds immense value. This approach allows for a deeper understanding and more accurate analysis of text data by encompassing multiple writing systems. It enhances language processing tasks like translation and sentiment analysis by capturing nuances and contextual cues specific to each script. Additionally, it promotes cross-cultural communication, inclusivity, and collaborative knowledge sharing. By training models on biscript lexicons, we can achieve more accurate translation, language generation, and language understanding across different scripts. This paper aims to evaluate the performance of a Sequence-to-Sequence (Seq2Seq) model based on Long Short-Term Memory (LSTM) architecture. The evaluation focuses on a biscript lexicon within the context of the Moroccan language. While Seq2Seq models have shown promise in various natural language processing tasks, their application to biscript lexicons remains relatively unexplored. Our approach involves constructing a Moroccan biscript lexicon where each entry is represented in both Arabizi and Arabic writing scripts. Subsequently, we will train our LSTM-based Seq2Seq model using various optimization techniques to enhance its effectiveness and accuracy.