article · Sensors
Large language models provide powerful performance for complex natural language tasks, but their high computational demands and power consumption prevent direct deployment on resource-constrained hardware. Tiny language models, also known as BabyLMs, address this limitation by offering compact alternatives designed for smartphones, Internet of Things systems, and embedded platforms. These smaller models rely on core compression and optimisation methods, including knowledge distillation, pruning, and quantization, to balance speed, efficiency, and low energy use. While these compact architectures show promise across automation and control environments, they encounter distinct operational challenges. These hurdles include balancing reduced model size against accuracy, managing limited generalisation across varied tasks, and addressing ethical concerns during implementation. Ongoing developments point towards hybrid compression approaches, application-tailored adaptations, and context-aware models engineered to meet hardware-specific operational constraints.
Bringing advanced language processing to small devices allows intelligent capabilities to operate locally without relying on remote cloud computing. Learning to compress these models efficiently enables faster, low-power functionality in everyday electronics. This shift is critical for delivering responsive automation across environments such as medical care and industrial operations, where immediate processing and strict energy limits are vital.
The reviewed approaches could enable on-device intelligence for industrial automation, healthcare monitoring, and connected consumer electronics. Primary end users include embedded systems engineers and Internet of Things developers seeking local language processing without cloud dependencies. Because this work is an analytical survey covering existing techniques, potential applications, and open challenges, the technology remains at an early stage of development and requires further hardware-specific testing before commercial deployment.
AI-generated from the published abstract. Always read the original work before citing.
Large Language Models (LLMs), like GPT and BERT, have significantly advanced Natural Language Processing (NLP), enabling high performance on complex tasks. However, their size and computational needs make LLMs unsuitable for deployment on resource-constrained devices, where efficiency, speed, and low power consumption are critical. Tiny Language Models (TLMs), also known as BabyLMs, offer compact alternatives by using advanced compression and optimization techniques to function effectively on devices such as smartphones, Internet of Things (IoT) systems, and embedded platforms. This paper provides a comprehensive survey of TLM architectures and methodologies, including key techniques such as knowledge distillation, quantization, and pruning. Additionally, it explores potential and emerging applications of TLMs in automation and control, covering areas such as edge computing, IoT, industrial automation, and healthcare. The survey discusses challenges unique to TLMs, such as trade-offs between model size and accuracy, limited generalization, and ethical considerations in deployment. Future research directions are also proposed, focusing on hybrid compression techniques, application-specific adaptations, and context-aware TLMs optimized for hardware-specific constraints. This paper aims to serve as a foundational resource for advancing TLMs capabilities across diverse real-world applications.
This page summarises published work. The authoritative version sits with the publisher.
DOI: 10.3390/s25051318
Is something wrong with this record? Report it or request removal.
Discussion
Have you built on this work, tried to replicate it, or seen it applied in practice? Share what you know. Verified researchers and MARATTO™ domain experts can open a discussion, and any member can reply. Contributions are reviewed before they appear.
No discussion yet. Open the first thread.
New to MARATTO™? Create a free account.
