The Persistence of Market Efficiency: Why Neither Traditional ML Nor Agentic AI Can Beat Buy-and-Hold

Abstract

Predicting excess returns in liquid equity markets under realistic risk constraints remains a central challenge in financial machine learning and a practical test of the Efficient Market Hypothesis (EMH). This paper investigates whether modern predictive systems, ranging from traditional machine learning to agentic Large Language Model (LLM) frameworks, can outperform a passive buy-and-hold strategy on the S&P 500 when volatility is explicitly constrained. Using the Hull Tactical Asset Allocation dataset, we conduct a two-phase empirical study. In the first phase, we evaluate gradient-boosting models with feature selection and Bayesian hyperparameter optimization. In the second phase, we assess a multi-agent LLM framework designed to combine heterogeneous reasoning strategies under a unified riskmanagement layer. All experiments are conducted using rigorous cross-validation and a volatility-penalized Sharpe-based evaluation metric. Across both phases, no approach consistently exceeds the buy-and-hold baseline. Traditional machine learning models converge toward highly regularized, near-constant predictions, while agentic LLM systems achieve performance comparable to the market benchmark, with substantial variance across regimes.