Semantic-Driven Automated Comment Generation for Java Methods Using Fine-Tuned CodeT5

Abstract

While pre-trained models on natural language processing, such as BERT and GPT, were proven to be versatile in their application over programming languages by becoming an important booster to tasks involving code, most of the existing methods have limitations in that they only consider either encoder-only or decoder-only architectures, which are suboptimal for generation or understanding tasks. Besides, most of them treat code fragments that resemble human language text without considering the characteristics of programming languages, such as token-type information. In this paper, we fine-tune CodeT5 to a particular task of generating automated comments in Java. Capitalizing on the capacity of CodeT5 to wrap up semantic meaning for Java code, our fine-tuned model delivers comments which are both correct and contextually relevant, enhancing code documentation and understanding. Through experimental results, we have shown that model performance is pretty good while generating comments for Java methods, facilitating efficient and reliable code documentation.