Ensuring fairness of AI in healthcare requires cross-disciplinary collaboration

While AI has demonstrated potential for healthcare insights, concerns around bias remain. “A fair model is expected to perform equally well across subgroups like age, gender and race. However, differences in performance may have underlying clinical reasons and may not necessarily indicate unfairness,” explained first author Ms Liu Mingxuan, a PhD candidate in the Quantitative Biology and Medicine (Biostatistics & Health Data Science) Programme and Centre for Quantitative Medicine (CQM) at Duke-NUS.

“Focusing on equity — that is, recognising factors like race, gender, etc., and adjusting the AI algorithm or its application to make sure more vulnerable groups get the care they need — rather than complete equality, is likely a more reasonable approach for clinical AI,” said Dr Ning Yilin, Research Fellow with CQM and a co-first-author of the paper. “Patient preferences and prognosis are also crucial considerations, as equal treatment does not always mean fair treatment. An example of this is age, which frequently factors into treatment decisions and outcomes.”

The paper highlights key misalignments between AI fairness research and clinical needs. “Various metrics exist to measure model fairness, but choosing suitable ones for healthcare is difficult as they can conflict. Trade-offs are often inevitable,” commented Associate Professor Liu Nan also from Duke-NUS’ CQM, senior and corresponding author of the paper.

He added, “Differences detected between groups are frequently treated as biases to be mitigated in AI research. However, in the medical context, we must discern between meaningful differences and true biases requiring correction.”

The authors emphasise the need to evaluate which attributes are considered ‘sensitive’ for each application. They say that actively engaging clinicians is vital for developing useful and fair AI models.

“Variables like race and ethnicity need careful handling as they may represent systemic biases or biological differences,” said Assoc Prof Liu. “Clinicians can provide context, determine if differences are justified, and guide models towards equitable decisions.”

Overall, the authors argue that pursuing fair AI for healthcare requires collaboration between experts in AI, medicine, ethics and beyond.

“Achieving fairness in the use of AI in healthcare is an important but highly complex issue. Despite extensive developments in fair AI methodologies, it remains challenging to translate them into actual clinical practice due to the nature of healthcare — which involves biological, ethical and social considerations. In order to advance AI practices to benefit patient care, clinicians, AI and industry experts need to work together and take active steps towards addressing fairness in AI,” said co-author Associate Professor Daniel Ting, Director of SingHealth’s AI Office and Associate Professor from the SingHealth Duke-NUS Ophthalmology & Visual Sciences Academic Clinical Programme. He is also Senior Consultant at the Singapore National Eye Centre and Head of AI & Digital Innovation at the Singapore Eye Research Institute (SERI).

“This paper highlights the complexities of translating AI fairness techniques into ethical clinical applications. It represents our collective commitment to developing AI that augments clinicians with trustworthy insights to provide quality and equitable care enhanced by technology,” remarked co-author Clinical Associate Professor Lionel Cheng Tim-Ee, Chief Data & Digital Officer, Clinical Director (AI) Future Health System Department, and Senior Consultant, Department of Diagnostic Radiology at Singapore General Hospital (SGH).

“Clinicians must be actively engaged in iterative communication with AI developers to ensure models align with medical ethics and context,” stressed senior co-author Professor Marcus Ong, Director of the Health Services & Systems Research (HSSR) Programme at Duke-NUS, who is also Senior Consultant at SGH’s Department of Emergency Medicine. “Good intentions alone cannot guarantee fair AI unless we have collective oversight from diverse experts, considering all social and ethical nuances. Pursuing equitable and unbiased AI to improve healthcare will require open, cross-disciplinary dialogues.”

The perspective published in npj Digital Medicine represents an international collaboration between researchers from institutions across Singapore, Belgium, and the United States. Authors from across the SingHealth Duke-NUS Academic Medical Centre (including Duke-NUS, SingHealth, SGH, Singapore Eye Research Institute and Singapore National Eye Centre) worked together with experts from the University of Antwerp in Belgium as well as Weill Cornell Medicine, Massachusetts Institute of Technology, Beth Israel Deaconess Medical Center and Harvard T.H. Chan School of Public Health in the United States.

Professor Patrick Tan, Senior Vice-Dean for Research at Duke-NUS, commented, “This global cooperation exemplifies the cross-disciplinary dialogues required to advance fair AI techniques for enhancing healthcare. We hope this collaborative effort spanning Singapore, Europe, and the US provides valuable perspectives to inspire further multinational partnerships towards equitable and unbiased AI.”