Modelling the Impact of AI Systems for Clinical Decision Support

EasyChair Preprint 5177

Abstract

Many AI (or ML) systems have been proposed for clinical decision support, providing diagnosis, prognosis or treatment recommendation. It is well known that the impact of these systems varies, with benefits such as improved decision making or time saved set against any potential harm introduced by the AI. The main technique proposed to evaluate impact is an ‘Impact Study’, a form of trial of a completed system. Vital though such studies are, they require at least a prototype system to be deployed which can be expensive. Meanwhile, the merits of AI predictors are mainly argued using accuracy measures, such as a confusion matrix or an AUC. We argue that the impact of a proposed AI system should be modelled during its development, to justify the expense of an Impact Study. We show that an Influence Diagram can be used for this and provide a small set of models for generic AI systems, with two main findings. First, we show that the way that an AI predictor is used – primarily how it interacts with clinical decision makers – is at least as important as its predictive accuracy. Indeed, we show that two different uses of the same predictor vary in impact without any change in its accuracy. Secondly, we show that the proposed use of an AI predictor also determines the information needed to model its potential impact. Some information is always needed on the decision accuracy of existing clinical decision makers, but the form and extent of this varies.

Keyphrases: Artificial Intelligence, Clinical Decision Support, impact analysis, influence diagram

@booklet{EasyChair:5177,
  author    = {Mariana Neves and William Marsh},
  title     = {Modelling the Impact of AI Systems for Clinical Decision Support},
  howpublished = {EasyChair Preprint 5177},
  year      = {EasyChair, 2021}}