Patient smart home monitoring using vision neural network transformers

Abstract

Image captioning is a task that involves generating natural language descriptions of the content of an image, and has the potential to support healthcare providers in monitoring patient conditions and routines at home. The ability to remotely monitor patients can provide valuable information to healthcare providers, allowing them to identify changes in patient behavior and facilitate timely interventions. In this study, we examine the usability of transformer neural networks for image caption generation from surveillance camera footage taken at regular intervals of one minute. Our objective is to develop and evaluate a transformer neural network model for generating captions of patient behavior, trained and evaluated on the Common Objects in Context (COCO) dataset. Our study provides a proof-of-concept for the potential of transformer neural networks in image captioning for remote monitoring of patient behavior. By generating natural language descriptions of patient behavior, healthcare providers can obtain valuable insights into patient routines and conditions, allowing them to monitor patients remotely and identify changes in behavior that may require intervention. Furthermore, our study highlights the potential for transformer neural networks to support healthcare providers in identifying patterns and trends in patient behavior over time.