ATMS-KD: Adaptive temperature and mixed sample knowledge distillation for a lightweight residual CNN in agricultural embedded systems

Abstract

Agricultural embedded systems require efficient deep learning models for real-time crop monitoring and maturity assessment. In this study, we propose Adaptive Temperature and Mixed-Sample Knowledge Distillation (ATMS-KD), a novel framework for developing lightweight CNN models suitable for resource-constrained agricultural environments. The framework combines adaptive temperature scheduling with mixed-sample augmentation to transfer knowledge from a MobileNetV3 Large teacher model (5.7M parameters) to lightweight residual CNN students. We evaluated three student configurations: compact (0.75× width, 1.3M parameters), standard (1.0× width, 2.4M parameters), and enhanced (1.25× width, 3.8M parameters). The dataset consisted of images of Rosa damascena collected from agricultural fields under diverse environmental conditions in the Dades Oasis, southeastern Morocco, under diverse environmental conditions. The experimental evaluation of the Damascena rose maturity classification dataset showed significant improvements over the direct training methods. All student models achieved validation accuracies exceeding 96.7% with ATMS-KD compared to 95-96% with direct training. The framework outperformed 11 established knowledge distillation methods, achieving 97.11% accuracy with the compact model while maintaining the lowest inference latency of 72.19 ms. The knowledge retention remained above 99% for all configurations, which shows that the knowledge transfer worked well regardless of the size of the student model. The compact model also delivered excellent computational efficiency, reaching a throughput of about 13.9 samples per second, making it suitable for embedded agricultural systems. The qualitative results further confirmed that the model performed reliably under different real-world field conditions, including changes in lighting, background complexity, and other environmental factors. Overall, the proposed framework offers a practical and efficient way to deploy accurate deep learning models in agricultural embedded systems, where computing resources are limited.