Only practical knowledge or knowing the algorithm? Meanings and necessities of eXplainable Artificial Intelligence in care

By C.M. van Leersum

Artificial Intelligence, as part of technologies to support daily practices, has the potential to improve personalized care and ageing in place. The internal machineries of AI systems often remain hidden as a black-box. Interest in eXplainable AI (XAI) originate from this black-boxing. XAI should assist users in understanding the underlying logic of the decision-making process, and in identifying mistakes. It is unknown how various stakeholders understand AI, and what value do they see in XAI. The meaning and value of XAI in care settings was investigated with the use of scenario-based interviews. ‘What is XAI?’ in the worlds of different stakeholders and the different enactments of XAI that become visible in their practices. Preliminary findings show that XAI sounds simple, but seems more difficult in practice. Stakeholders express different meanings and necessities of XAI. This varies from knowledge of algorithms or data specific knowledge towards practical understanding. In care of older adults, trust and willingness to use AI are essential. The needed level of explainability differs according to different stakeholders.