Sensitivity evaluation of machine learning-based calibrated transportation mode choice models: A case study of Alexandria City, Egypt

Abstract

Intelligent methods including Machine Learning (ML) techniques have been increasingly employed in transportation mode choice modeling, which is more complex than other demand models, since it has to reliably and accurately reflect a wide range of related categorical and continuous variables, concerning the travelers, transportation system, and trip characteristics. ML techniques can capture such complex relationships. So that, they can provide a more nuanced understanding of the travelers’ decision process. Most research studies focused mainly on the evaluation of the model accuracy, where little has been done to evaluate the models’ performance toward transport attributes. This research aims to calibrate the transportation choice models using ML techniques, then conduct a comparison with the Multinomial Logit (MNL) model to identify the impact on the model accuracy and performance and quantify to what extent the ML models are sensitive to transport policies compared to the traditional MNL model. To this end, eight ML classifiers were examined. As a case study, the models were calibrated to reflect the choice behavior of trip makers in Alexandria City, Egypt. The models were successfully calibrated with satisfying accuracy; however, the ML models have better calibration results in terms of predictability, outperforming the MNL model, where the GBDT classifier records the best prediction accuracy. Finally, the sensitivity analysis test was performed to quantify the elasticity of the models to transport policies. The results show the ML models’ structure is more comprehensively and accurately built than the MNL model providing better indicative and reliable sensitivity analysis results.