AI in Healthcare

Basis

These views come from three places.

Clinically — from a career in surgery, and from governance work: national cancer oversight, guideline development, and the hospital and patient safety structures that decide how technology is adopted into practice. That is the side of the table where the difficult questions get asked, and it is where a good deal of my working life has been spent.

Technically — from building and running these systems directly rather than commissioning them: automated workflows, agents configured with their own tools and defined knowledge sources, retrieval over private document collections, and the practical business of measuring output. That means testing for hallucination, ungrounded claims and silent error rather than assuming their absence.

Currently — from actively developing the engineering fundamentals, including Python, because the useful position is not to specify what a system should do but to understand what it actually does.

Where it is being deployed

Clinical documentation

Scribing, summarising and correspondence.

Imaging and histopathology

Interpretation and prioritisation of studies.

Triage and referral

Ranking and routing of referrals.

Risk prediction

Deterioration, readmission and outcome modelling.

Decision support

Guideline retrieval and treatment prompts.

Patient information

Explaining conditions in plain language.

Operational

Scheduling, coding and waiting-list management.

What it does well

Pattern recognition at a scale and consistency no individual can match. Relentless availability, at any hour, without fatigue. And the removal of administrative burden that currently falls on clinicians and takes time away from patients — which, on its own, may prove to be the largest benefit of all.

Where it fails

It fabricates confidently

Output that reads as authoritative but has no basis in fact or source.

It is ungrounded

Answers assembled from statistical memory rather than from anything verifiable.

It inherits bias

Models reflect the data they were trained on, and perform worse for the groups least represented in it.

It degrades quietly

Performance drifts as practice, coding and populations move away from what it learned.

It invites deference

Under time pressure, clinicians defer to a confident answer. That is a failure of the working environment, not the software.

None of these are reasons to reject the technology. They are the reason it has to be evaluated against clinical reality rather than a demonstration.

Ethical and safety concerns

Accountability

When a system contributes to harm, who answers for it — the clinician, the developer, or the deploying organisation?

Consent and transparency

Patients should know when AI is involved in their care, and be able to ask about it.

Data

What is used to train these systems, under what governance, and with what safeguards for confidentiality.

Equity

A tool that works less well for some patients widens the gap it was meant to close.

Deskilling

If the judgement is never exercised, it will not be there when the system is wrong.

Governance in the United Kingdom

The frameworks already exist, and they are more developed than most people assume:

• Clinical risk management — DCB0129 and DCB0160, the standards covering both those who build health software and those who deploy it.
• Assessment — DTAC, the Digital Technology Assessment Criteria applied to technology entering NHS care.
• Device regulation — the MHRA regulates software as a medical device, and runs the AI Airlock sandbox for AI-enabled devices.
• Policy — the National Commission into the Regulation of AI in Healthcare, whose recommendations are still being settled.
• In practice — the Clinical Safety Officer, the named individual accountable for clinical risk when a system is deployed.

The gap is not regulation. It is the number of people who understand both the machinery and the clinic well enough to apply it.

Interest

This is an active and developing area of interest — practical, technical and regulatory — and one I expect to work in alongside clinical practice. Enquiries are welcome at healthai@saboorkhan.co.uk.