A two stage framework integrating base deep models with multiscale residual attention fusion for complex temporal dynamics

Abstract

This work introduces a two-stage deep learning architecture designed to enhance the modeling of complex temporal dynamics in real-world time-series data. In the first stage, a Hybrid Transformer–CNN model, pretrained to extract robust temporal representations, is employed as the base learner. This component captures global trends, long-range dependencies, and general temporal regularities. To further enrich these representations, a novel multi-feature meta-learning layer is integrated to dynamically reweight extracted features across heterogeneous temporal conditions. In the second stage, we propose a novel Multi-Scale Residual Attention Fusion (MSRAF) network that operates on the residual errors produced by the base model. MSRAF is specifically designed to learn nonlinear residual structures, capture multi-scale temporal dependencies, and model local irregularities that conventional sequence models often fail to represent. By jointly leveraging deep temporal encoding and multi-scale residual refinement, the framework achieves consistent improvements over standard Transformer and CNN-based baselines, particularly in settings characterized by non-stationarity, noise, and irregular fluctuations. The results demonstrate that this two-stage learning strategy effectively decomposes forecasting complexity: the base model addresses general temporal patterns, while MSRAF focuses on fine-grained error correction. This separation of responsibilities significantly enhances accuracy and stability across diverse forecasting horizons and data regimes.