Dataset Preprocessing Leveraging Vision Language Models: Storytelling for Accident Detection and Description

Abstract

In recent years, the Vision-Language Models (VLMs) have significantly advanced the field of storytelling applications. Visual storytelling involves the use of Generative AI models to transform a sequence of video frames into a semantics-preserving story-like text. A critical aspect of developing these models lies in the preparation of datasets, which directly impacts their efficacy and robustness. This paper explores a comprehensive methodology for dataset preparation tailored to the detection and description of car accidents in video surveillance footage. This research explains the necessity of data collection, annotation, and preprocessing to transform a sequence of frames in a video into a semantics-preserving story-like text. We describe the steps, tools and challenges of this task.