)
)
Identifying and understanding significant change due to drift when assessing AI models in healthcare: a narrative review
Artificial intelligence (AI) as a Medical Device (AIaMD) is a medical device that incorporates AI, specifically machine learning (ML) approaches. While they may offer significant advantages in healthcare, AIaMDs present unique characteristics that challenge existing regulatory frameworks developed for dealing with traditional medical devices.
One challenge is that the underlying characteristics of health data used to train AIaMD are likely to change over time, potentially impacting performance of devices and assumptions underpinning the initial authorisation.5 Changes in health data could happen for several reasons, for example, due to the launch of new technologies on the market, which improve measurement accuracy or changes in population characteristics (eg, age, diet and genetics) that may be harder to forecast as the COVID-19 pandemic has shown. This phenomenon is known as drift.
This paper presents the consensus view from an expert working group (EWG) involving regulatory experts, clinicians, data scientists, industry representatives and health service AI implementation experts, hosted by the UK Medicines and Healthcare products Regulatory Agency (MHRA) to understand approaches to detecting and assessing significant changes in AI models’ behaviour as well as the nature of an observed drift and its regulatory implications. We discuss three distinct subtypes of drift from a statistical perspective (covariate, target and concept drift) and highlight potential causes in the real world that could lead to significant shifts in performance requiring actions such as model recalibration or retraining on new data. We also discuss the regulatory perspective on risk assessment and the key features of drift (such as speed and severity) that must be considered to properly address interventions and guarantee the release of safe medical products onto the market.
We further consider how regulatory approaches (such as predetermined change control plans (PCCPs) required under medical device regulations, data protection regulations and the European Union (EU) AI Act) can provide mechanisms to manage drift responsively and fairly. By framing drift not only as a technical issue but also as a regulatory and ethical one, we argue for a total product lifecycle (TPLC) model that enables continuous model monitoring, recalibration and governance across the entire deployment journey.