A Bayesian Hierarchical Cox Model with Elastic Net Regularization for Improved Survival Prediction and Feature Selection

Abstract

In recent years, the growing availability of large-scale data across a wide range of disciplines has created new opportunities for developing models that improve the predictive accuracy of statistical models. Although techniques such as regularization and Bayesian hierarchical methods are commonly used for building predictive models, substantial challenges remain, particularly when dealing with high-dimensional datasets that contain considerable noise. In this study, we propose a Bayesian hierarchical model that employs a spike-and-slab hierarchical elastic net prior that regularizes the Cox Proportional Hazards (Cox-PH) model. The method combines Bayesian modeling with the regularized partial log-likelihood of the Cox-PH framework, incorporating an Elastic Net penalty to estimate the joint posterior distribution under a hierarchical elastic net prior. We compute this posterior using an Expectation–Maximization Cyclic Coordinate Descent Algorithm (EM-CCDA), which streamlines feature selection and enhances overall predictive performance. We evaluate the algorithm’s performance through Monte Carlo simulations and apply it to three real-world datasets, comparing the results with those from established classical and Bayesian survival analysis approaches. The findings demonstrate notable gains in both feature selection and predictive accuracy, highlighting the model’s strong ability to predict patient survival and identify relevant genes in real biological datasets.