Lambda Asterisk: Context-Aware Query Routing for Dual-Serving-Layer Architectures

Abstract

The massive volumes of data generated by sources such as IoT sensors and social media platforms make big data a major research domain in academia and industry. Lambda and Kappa are the foundational architectures of big data. Modern Lambda is an example of a Lambda extension that has evolved to address emerging operational requirements. It differs from the classical Lambda by having two serving layers: one for batch views and the other for real-time views. Our paper addresses the following ambiguity: for any incoming query, which serving layer should be used? Existing architectures handle this by systematically merging results from both layers, a costly and often unnecessary operation that increases latency and resource consumption. To address this limitation, this paper proposes Lambda Asterisk, a new extension with a dedicated orchestration and routing layer capable of performing contextaware, metadata-driven query routing and a governance layer to enforce access control and data privacy constraints.