Feature-Importance-Weighted Distance Metrics for Enhanced k-Nearest Neighbors Classification in Medical Diagnosis

Abstract

Abstract Traditional k-Nearest Neighbors (KNN) classification assumes equal feature importance, which limits its effectiveness in domains where features exhibit inherently different levels of relevance. This study introduces a systematic feature importance weighted distance framework that integrates Random Forest derived importance scores into classical distance metrics. Ten distance measures, namely Euclidean, Manhattan, Cosine, Lorentzian, Canberra, Squared Chord, Chi Square, Whittaker’s Index of Association Distance, Motyka, and Hassanat, are adapted by embedding normalized feature importance weights, allowing informative features to contribute more strongly to similarity computation while reducing the influence of less relevant ones. Comprehensive evaluation across seven publicly available datasets, including five medical and two non-medical datasets, demonstrates consistent performance improvements. Using stratified 5 fold cross validation repeated over ten random seeds, the proposed weighted variants achieve an average accuracy gain of approximately 5.01 percentage points across all metrics, datasets, and values of k . Performance is improved or maintained in 91.1% of all tested configurations spanning five values of k ( $$k \in \{1,3,5,7,9\}$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mi>k</mml:mi> <mml:mo>∈</mml:mo> <mml:mo>{</mml:mo> <mml:mn>1</mml:mn> <mml:mo>,</mml:mo> <mml:mn>3</mml:mn> <mml:mo>,</mml:mo> <mml:mn>5</mml:mn> <mml:mo>,</mml:mo> <mml:mn>7</mml:mn> <mml:mo>,</mml:mo> <mml:mn>9</mml:mn> <mml:mo>}</mml:mo> </mml:mrow> </mml:math> ). Wilcoxon signed rank tests confirm statistical significance ( $$p &lt; 0.05$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mi>p</mml:mi> <mml:mo>&lt;</mml:mo> <mml:mn>0.05</mml:mn> </mml:mrow> </mml:math> ) in 78.0% of configurations, while sensitivity analysis indicates stable behavior across different Random Forest hyperparameter settings. Overall, the proposed framework provides an interpretable and effective mechanism for incorporating feature relevance into similarity based classification.