AFP assistant: a retrieval-augmented generation and large language model-powered multilingual polio chatbot for low-resource language communities

Abstract

Acute flaccid paralysis (AFP) surveillance in Ethiopia remains limited due to hard-to-reach populations, restricted healthcare access, low public awareness, and persistent underreporting within community-based surveillance (CBS) systems. Contributing to this are inadequate training of community reporters, communication barriers, and cultural beliefs and misinformation that further hinder timely case detection. Here, AFP Assistant, a multilingual large language model powered chatbot, is developed to strengthen CBS through improved health communication and real-time reporting. The system supports Amharic, Afaan Oromo, and English to enhance accessibility and inclusivity. A curated dataset derived from literature, guidelines, and community interviews was translated and validated by experts to ensure clinical and cultural accuracy. The system integrates supervised fine-tuning with retrieval-augmented generation (RAG) to deliver accurate, context-aware responses. Trained on 468 question-answer pairs using pretrained Gemini 2.5-flash, the model achieved an accuracy of 0.90 and a loss of 0.31. The RAG framework improved retrieval relevance, grounding, and response efficiency. Evaluation through in-house testing and human annotation using a structured rubric demonstrated robust performance across tasks and languages, while user feedback confirmed usability and relevance. These findings highlight the potential of AI-driven chatbots to enhance AFP surveillance and equitable health communication in low-resource settings.