Towards Closing The Major Barrier of Adoption of Blackbox Models in The Medical Arena Based on Human-Centered XAI Design.

Abstract

The opacity of black-box models presents a significant obstacle to their acceptance in the medical field. To improve their adoption, it is crucial to identify the stakeholders who need explanations of these models and to develop effective methods for providing these explanations. This paper aims to identify the key actors/stakeholders in the medical field who require explanations of black-box models to enhance their adoption. Through a comprehensive literature review, we identify physicians, patients, regulatory bodies, ethicists, and legal professionals etc. as the primary actors with information needs regarding the workings and rationale of black-box models. Physicians require explanations to validate predictions against their clinical expertise, while patients seek transparency to understand the basis of recommendations. Regulatory bodies focus on compliance and ethical considerations, while ethicists and legal professionals evaluate fairness and accountability. By providing tailored explanations to these actors, trust can be fostered, informed decision-making facilitated, ethical concerns addressed, regulatory compliance ensured, and effective communication established. This research highlights the information needs of various stakeholders, proposes two frameworks—Human-Centered XAI Design and a workflow for black-box model research—and emphasizes the importance of explanations in enhancing the adoption of black-box models in the medical field.