Enhanced Human-Robot System for Underwater Divers' Hand Gesture Recognition

Abstract

This paper introduces a human-robot system for improving the communication between divers and autonomous underwater vehicles (AUVs) in underwater environments. The goal is to enhance the accuracy of recognition of AUVs to divers' hand gestures, which would enable exploration in challenging conditions, and enhance diver experiences. The system consists of two main modules: localization and classification module, and compiler module. The first module is used to detect and localize the hand gesture per frame. Considering the embedded devices used in AUVs, the lite architectures in YOLOv7 and YOLOv8 are chosen to this task. Whereas, the second module receives the recognized gesture from first module as tokens. These tokens are parsed to identify the required task after its validation using the deterministic finite automaton (DFA). The system can efficiently detect and interpret sequences of gestures made by divers, which improving human-robot communication. The localization and classification is trained on CADDY dataset, whereas the DFA validates the Caddian language. The proposed system achieves 0.991 mAP for detection and 99.5% accuracy for classification using Yolov8n. The model size is 6.2 MB. Also, this system outperforms the state-of-art techniques.