Enhancing Diagnostic Accuracy in Breast Cancer and Diabetes Prediction Using Probability-Based Correction and Machine Learning

Abstract

In this study, we present a unified framework that approaches the accuracy of breast cancer diagnosis as well as diabetes prediction through the introduction of a novel probability-based correction method coupled with various machine learning (ML) techniques. Through the application of the Breast Cancer Wisconsin Diagnostic Dataset and the PIMA Indian Diabetes Dataset, we compare the performance of various ML classifiers. We show that, despite high baseline accuracy of ML models (eg, Multilayer Perceptron (MLP) and Logistic Regression), high-risk misclassifications close to decision boundaries still persist. We tackle this issue by proposing a probability-based corrective mechanism based on moted Optimal Stopping Theory and the Generalized Secretary Problem. The results demonstrate that this correction method can significantly improve diagnostic performance, resulting in 100 % accuracy for the prediction of cancer and a 6 % increase in diabetes prediction accuracy, thus being a great booster for clinical decision-making.