How do autonomous software agents learn? A systematic literature review

Abstract

Modern artificial intelligence approaches powered by artificial neural networks (ANNs) enable many computational tasks by learning from data rather than from hand-coded rules. Currently, autonomous agents (AAs), whether virtual or physical, are of great interest because they can perceive an environment and take actions to achieve goals without direct human control. Also, with the growing adoption of large language models (LLMs), agents are increasingly used in environments where the agent needs to receive commands. This article reviews how AAs learn and make decisions, with a focus on the role of artificial neural networks (ANNs) in producing autonomous behavior. A literature review was conducted to map the reported learning mechanisms, neural networks employed, decision-making methods, and evaluation protocols used to assess agents and their networks. Results show that reinforcement-learning approaches are the dominant paradigm for producing autonomy in AAs. From 569 records initially identified, 42 articles met the inclusion criteria; among those, reinforcement learning represented the largest share of learning approaches (33.3%), most evaluations were conducted in simulation (29 of 42 articles, 69.04%), and recurrent and convolutional networks were among the most frequently used network families (21.4% and 16.7%, respectively). As a contribution to the conceptual framework in AAs, this study mapped that the agents’ learning can be set in three categories: aspect-based (the agent must learn to solve a specific task), preventive (the agent must assess risks to avoid causing harm to the environment or other agents), and progressive (the agent must learn to achieve multiple objectives).