SLSG INSIGHTS

Human-Centered AI in Health: Why Trust Must Be Designed In

Artificial intelligence is becoming increasingly relevant across healthcare, but technical capability alone is not enough. In health-related environments, trust must be designed into the technology from the beginning.

Artificial intelligence has the potential to support healthcare across areas such as data analysis, pattern recognition, workflow support, monitoring and decision-support systems.

Yet healthcare is different from many other technology environments. The information involved can be deeply personal, the decisions can carry significant consequences, and users may include patients, clinicians, caregivers and institutions with very different responsibilities.

For that reason, the question is not simply whether an AI system can perform a task. The more important question is whether that system can be designed, governed and used responsibly.

From intelligent systems to responsible systems

AI can help identify patterns in large or complex datasets, organize information and support people in making sense of signals that might otherwise be difficult to process.

But identifying a pattern is not the same as making a clinical judgment.

This distinction is fundamental to responsible health technology.

The World Health Organization has emphasized that artificial intelligence used in health should place ethics, human rights, safety and accountability at the heart of its design, deployment and use. ^[1]

Regulatory considerations extend well beyond the algorithm itself. WHO highlights areas including transparency, risk management, intended use, validation, data quality, privacy, system performance and stakeholder engagement. ^[2]

In practical terms, responsible AI requires questions such as:

• Is the system being used for an appropriate and clearly defined purpose?
• Is the underlying data appropriate for that use?
• Can professionals understand the limitations of the technology?
• Is meaningful human oversight preserved?
• Can system performance be monitored after deployment?
• Is responsibility clear when technology informs an important decision?

Privacy should not be added at the end

Health and wellbeing technologies may involve some of the most personal forms of information a digital system can process.

Depending on the application, this may include physiological measurements, movement information, behavioral patterns, health history, wearable-device data or information generated through connected sensors.

Privacy therefore needs to influence architecture from the earliest stages of product development.

Privacy-aware design can include principles such as:

• Collecting only information that has a clear purpose
• Applying appropriate consent mechanisms
• Limiting access according to role and responsibility
• Protecting information during storage and transmission
• Defining appropriate retention and deletion practices
• Considering whether some processing can occur locally rather than centrally

A system that collects more information is not automatically a more intelligent system.

In some cases, good technology is defined as much by what it deliberately chooses not to collect as by what it is technically capable of collecting.

Human oversight is part of the architecture

Human-centered AI does not mean placing a person somewhere near an automated system and calling the result responsible.

Human oversight must be meaningful.

Systems should be designed around the role, expertise and responsibilities of the people who actually use them.

In healthcare, that can mean determining which information should be presented to a professional, how uncertainty is communicated, when an alert deserves attention and when an automated output requires further review.

The goal is not to automate human responsibility away.

It is to use technology to improve visibility, consistency and access to useful information while keeping appropriate judgment in human hands.

Trust also depends on practical usability

Even technically sophisticated systems can fail if they do not fit the environments in which people are expected to use them.

Healthcare professionals already operate within complex workflows. Patients have different levels of digital confidence. Institutions have security, governance and operational requirements.

Good digital-health products therefore need to integrate technological, organizational and human considerations.

WHO's Global Strategy on Digital Health emphasizes that sustainable digital-health initiatives depend on the integration of technological, organizational, human and other resources rather than technology operating in isolation. ^[3]