Klinik yapay zekâyı kullanmadan önce değerlendirmek

Bu yazı şu anda yalnızca İngilizce olarak mevcuttur. İncelenmiş Türkçe çevirisi daha sonra eklenecektir.

Most clinical AI projects have a moment when the numbers look good. The area under the curve is high, the confusion matrix is tidy, and someone asks when it can go live. That moment is where careful evaluation starts, not where it ends.

As the authors of the DECIDE-AI guideline put it, a growing number of AI decision-support systems show promising performance in preclinical, in-silico evaluation, but few have yet shown real benefit to patient care. The gap between the two is where most of the work lies.

Four gates, not one

We find it useful to think of evaluation as four gates. Each one asks a different question, and passing one tells you little about the next.

1. Does the model work on data it has never seen? This is the familiar question: discrimination, calibration, and performance across subgroups, measured on held-out and ideally external data. For prediction models, TRIPOD+AI (2024) sets out what a transparent report should include, from how missing data were handled to how the sample size was justified. If a vendor can’t answer TRIPOD+AI’s questions about their model, that is itself an answer.

2. Does it work here? Populations, coding practices, devices, and workflows differ between sites. A triage model trained in one hospital network may be well calibrated there and poorly calibrated in yours. Local retrospective validation on your own data, before any prospective use, is the minimum. Calibration deserves particular attention: a model can rank patients correctly while systematically over- or under-estimating their risk.

3. Does it work in the hands of the people who will use it? This is the stage DECIDE-AI was written for: early, small-scale clinical evaluation that looks at actual performance in context, at safety, and at human factors. Do clinicians understand the output? Do they over-rely on it, or ignore it? What happens to workflow when the tool is slow, unavailable, or wrong? These questions can only be answered with real users, under supervision, before scaling up.

4. Does it improve outcomes that matter? The final gate is comparative evaluation, ideally randomised, against usual care. CONSORT-AI extends the standard trial reporting checklist with AI-specific items: the version of the algorithm, how inputs were acquired, how outputs were used, and how errors were analysed. Not every tool needs a trial before any use, but claims of clinical benefit should be backed by evidence of this kind.

What we look for at each gate

Gate	Core question	Evidence we expect
Technical	Does it generalise?	External validation, calibration, subgroup analysis (TRIPOD+AI)
Local	Does it fit our data?	Retrospective validation on local data; drift checks
Human factors	Does it work with our people?	Supervised early use, error analysis, usability (DECIDE-AI)
Impact	Does it help patients or staff?	Comparative evaluation against usual care (CONSORT-AI)

Evaluating generative systems

Language-model features, such as drafting notes or summarising case files, don’t fit neatly into the metrics built for classifiers. There is no single right answer for a clinical note. We still think the same gates apply, with adapted measures:

• Faithfulness: does every statement in the output have a source in the input? Unsupported statements should be counted, not just noticed.
• Omission: what clinically important content was left out? Omissions are harder to spot than errors, so they need structured review.
• Critical-field accuracy: medications, doses, laterality, dates, and allergies should be checked separately, because their error cost is higher.
• Editing burden: how much do clinicians change before signing, and does that go down, or stay stable while they stop looking?

That last point matters. A falling edit rate can mean the tool is improving, or it can mean reviewers are skimming. You can only tell the difference by occasionally auditing signed outputs against sources.

Evaluation doesn’t end at go-live

Performance can drift as populations, coding practices, or upstream systems change. Evaluation should therefore include monitoring after deployment: periodic recalibration checks, tracking of override rates, and a clear route for clinicians to report problems. WHO’s 2021 guidance on AI for health makes a similar point: responsibility and accountability continue through the life of a system, not just up to its approval.

For predictive tools, we also think the system should say when it is uncertain, and sometimes decline to give an answer at all. We cover this in showing uncertainty in clinical AI.

A practical starting checklist

• Is there a written intended use, including who uses the output and what decision it informs?
• Has the model been validated on data from your institution?
• Is calibration reported, not just discrimination?
• Has performance been checked across relevant subgroups?
• Has a small supervised pilot looked at how clinicians actually use the output?
• Is there a monitoring plan, with named owners, for after go-live?

None of this is new. The reporting guidelines exist, and they were written by the research community with exactly this problem in mind. The work is in applying them consistently, including when the first numbers look good.

Kaynaklar

1. Vasey B, Nagendran M, Campbell B, et al. Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. Nat Med. 2022;28(5):924–933. doi.org/10.1038/s41591-022-01772-9
2. Liu X, Cruz Rivera S, Moher D, et al. Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension. Nat Med. 2020;26(9):1364–1374. doi.org/10.1038/s41591-020-1034-x
3. Collins GS, Moons KGM, Dhiman P, et al. TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ. 2024;385:e078378. doi.org/10.1136/bmj-2023-078378
4. World Health Organization. Ethics and governance of artificial intelligence for health: WHO guidance. Geneva: WHO; 2021. www.who.int/publications/i/item/9789240029200
5. Kompa B, Snoek J, Beam AL. Second opinion needed: communicating uncertainty in medical machine learning. NPJ Digit Med. 2021;4(1):4. doi.org/10.1038/s41746-020-00367-3