Heterogeneous stacking ensembles with SHAP-derived diversity weighting for high-frequency virtual sensing of riverine nutrients across contrasting catchments

Abstract

High-frequency monitoring of dissolved nutrients is essential for managing riverine pollution, yet direct in-situ sensors for species such as total reactive phosphorus (TRP), total phosphorus (TP), nitrate‑nitrogen (NO₃ − -N), and ammonium‑nitrogen (NH₄ + -N) remain costly, maintenance-intensive, and prone to biofouling, leading to persistent data gaps, motivating virtual sensing approaches that predict these quantities from cheaper surrogate measurements. However, most existing studies rely on single algorithms or homogeneous ensembles composed of models with similar inductive biases, limiting predictive diversity and performance. Here we present a nine-model heterogeneous stacking ensemble that spans five inductive-bias families: kernel/linear, kernel/RBF, distance, neural, and tree methods. The framework is evaluated on the full hourly time-series from two contrasting catchments in southern England, both characterized by lowland, responsive hydrology: Enborne (rural agricultural) and The Cut (urban-impacted). We introduce SHAP engagement entropy as a model-agnostic diagnostic of sensor utilization breadth, and derive ensemble weights from a convex combination of cross-validated R 2 and entropy. Stacking is implemented via a Ridge meta-learner trained on out-of-fold predictions using inner five-fold cross-validation (CV). Temporal block CV is employed as the primary evaluation scheme to respect riverine autocorrelation structure. Under shuffled k-fold CV, stacking achieves R 2 = 0.960 (Enborne TRP), 0.966 (Enborne NO₃ − ), 0.850 (The Cut TRP), 0.848 (The Cut TP), and 0.940 (The Cut NH₄ + ), with consistent gains over the best individual model on four of five tasks. Under temporal block CV, stacking yields R 2 = 0.730 and 0.927 for Enborne TRP and NO₃ − respectively, and 0.486–0.539 for The Cut phosphorus targets, with the stacking meta-learner systematically outperforming all individual models across both evaluation protocols. The Krogh-Vedelsby ambiguity of the heterogeneous pool exceeds that of a tree-only ensemble by 2.2–4.2×, formally proving the diversity advantage. A sensor-dropout experiment confirms that models with higher engagement entropy degrade more gracefully under individual sensor loss.