Invisible Ink on Medical Reports: When AI Watermarks Endanger Patients
> Paper: Marking the Wrong Symptoms: Evaluating LLM Watermarks in Medical Texts > Authors: Melanie Rieff, Robin Staab, Thibaud Gloaguen, Stefan Hegselmann, Martin Vechev (ETH Zurich) > arXiv: 2607.20462 > Venues: FM4LS @ ICML 2026, AI4GOOD @ ICML 2026
---
The Hospital Emergency Room, 3 A.M.
Late at night in the emergency room, a physician on duty reviews a chart summary generated by the hospital's newly deployed AI assistant. The system has just tagged it "watermark verification passed" — meaning the text genuinely came from the AI, not a human forger. Reassured, the physician prescribes medication based on the summary's diagnostic suggestions.
Three days later, the patient is rushed back with an adverse drug reaction. The investigation shows the AI had replaced a critical medical term — "pulmonary embolism" — with the vague "pulmonary abnormality" during generation: a side effect of the watermarking process. And the "verification passed" badge is exactly what lowered the physician's guard.
This is not science fiction. It is the real risk revealed by an ETH Zurich research team in a pioneering study. When AI watermarks — "safety locks" designed to trace and verify AI-generated content — meet medicine, a field where every word matters, things take an unexpected turn.
---
Chapter 1: What Is Watermarking and Why Do We Need It?
Imagine a painter hiding a nearly invisible mark in a corner of a masterpiece — only she knows it's there, only she can verify authenticity. That is the traditional watermark concept.
In the AI world, watermarks serve a similar mission. As LLMs like ChatGPT generate more text, an urgent question arises: how do we distinguish human-written from AI-generated text? This concerns academic integrity, information authenticity, and traceability.
LLM watermarking works subtly: during generation, the algorithm nudges certain token choices. When choosing between "excellent" and "outstanding," the watermark gently tips the scales toward one. The bias is imperceptible to human readers, but a detector holding the "key" can read the hidden signature: "this text is from an AI."
Sounds perfect — an invisible guardian preserving traceability without hurting user experience. It shows great promise in news writing, customer service, and creative writing.
But problems arise in medicine.
---
Chapter 2: Why Medical Texts Tolerate No Error
If a waiter misremembers "less salt" as "less chili," the dish just tastes a bit off — easily fixed. But in medicine, such "minor deviations" can cost lives.
Medical text has a unique property: extremely high density of low-entropy vocabulary — words that are nearly impossible to substitute. In general text, "excellent" and "outstanding" are interchangeable. In medicine, "myocardial infarction" and "myocardial injury" differ by two characters yet point to entirely different pathologies and treatments.
The ETH Zurich team noted that nearly all existing watermark evaluations use general-text benchmarks — fluency, coherence, human-preference alignment. These metrics suffice for ordinary scenarios but can mask fatal problems in medicine: like measuring a painting with a ruler.
---
Chapter 3: What the Study Did
Rieff and colleagues conducted an exhaustive evaluation: 5 mainstream watermarking schemes across 11 LLMs and 7 vision-language models, on multiple unimodal and multimodal clinical reasoning tasks.
The five schemes:
1. KGW (Kirchenbauer et al.): the classic logit-bias method splitting the vocabulary into "red" and "green" lists 2. SWEET: an improved variant targeting low-entropy tokens 3. DiPmark: a "distribution-preserving" watermark 4. EXP-edit: a rank-based sampling modification 5. LogRank & DetectGPT: post-hoc detection methods
The key innovation lies not in which watermarks were tested, but in how their impact was assessed. The team designed a human-expert-validated evaluation pipeline auditing three dimensions:
- Medical reasoning quality: does watermarking break the model's clinical reasoning chain?
- Terminological precision: are medical terms accurately preserved?
- Induced hallucination: does watermarking make the model invent nonexistent medical concepts?
---
Chapter 4: The Alarming Findings
Finding 1: Lexical Corruption. Watermark insertion subtly shifts key medical terms — not obvious typos, but semantic drift. "Acute myocardial infarction" may be distorted into "acute myocardial injury." To laypeople these sound similar; to clinicians, the former demands immediate coronary intervention, the latter further diagnostic work-up. In "branch point" tests, when the watermark first acts after a clinical prefix, the model's answer direction deviates significantly.
Finding 2: Hallucinated Terminology. More chilling: watermarks not only alter existing terms but fabricate nonexistent medical concepts. In one recorded case, on a question about post-streptococcal glomerulonephritis (PSGN), the watermarked model began describing features of focal segmental glomerulosclerosis (FSGS) — an entirely different disease. The misattribution was not random but a systematic bias induced by the watermark.
Finding 3: Misattribution or Omission of Image Findings. In multimodal tasks — models interpreting both text and medical images — the harm amplifies. A model may attribute a finding to the wrong pathology or omit it entirely. In radiology, a missed lung nodule can mean a delayed cancer diagnosis; a wrong attribution can send treatment down an entirely wrong path.
---
Chapter 5: Why General Benchmarks Mask These Failures
The team identified a deeper problem: the evaluation system itself has systematic blind spots.
Aggregate metrics — overall fluency scores, preference ratings — are like measuring an earthquake with a thermometer. "Fluency dropped only 2%" sounds fine, until you learn that 2% occurred precisely on the most critical medical terms. It's like rating a car "A+ for safety" based on top speed, fuel economy, and seat comfort — while never testing emergency braking.
This masking is not accidental but systematic: watermark impact is uneven. It doesn't degrade text quality randomly; it concentrates on low-entropy, "obvious" tokens — exactly the words carrying the most critical information in medical text.
---
Chapter 6: Technical Deep Dive — How Watermarks "Poison" Medical Reasoning
Basic mechanism. Taking KGW as an example: at each token, a hash of the previous token splits the vocabulary into a "red list" (avoided) and "green list" (preferred). Green-list logits are boosted (typically by 1–2), steering selection.
In general text this is imperceptible: vocabularies are huge (50k–100k tokens), green lists cover half, synonyms abound, and human language is redundant.
Medical text breaks these assumptions:
1. Irreplaceability of terms: medical terms are near-univocal. "Pulmonary embolism" cannot become "pulmonary thrombus" or "lung vessel blockage" without changing meaning. When watermarks force the model off its optimal choice, it may land on a semantically close but non-equivalent term.
2. The low-entropy trap: many words in medical text are "obvious" to a well-trained medical LLM — near-certain choices. To embed its mark there, the watermark must apply stronger bias, producing more severe distortion.
3. Cascading errors in reasoning chains: clinical reasoning is a chain. One early terminological error dominoes into the entire downstream reasoning — an inaccurate symptom term leads to differential diagnosis from a false premise.
---
Chapter 7: What It Means and Where We Go From Here
For medical AI developers: if your product generates medical text, watermarking is not a "default-on" feature. Before deployment, rigorous domain-specific evaluation is mandatory — not whether the text is fluent, but whether every medical term is accurate and every inference reliable.
For regulators: approval processes for medical AI may need dedicated assessment of traceability techniques like watermarking. A "watermark verified" badge should not automatically equate to "safe and trustworthy" — especially in medicine.
For the research community: the study points to an overlooked direction: domain-adaptive watermarking — algorithms that identify and protect critical domain vocabulary: diagnostic terms in medicine, legal citations in law, technical parameters in engineering.
The team's core conclusion: domain-specific evaluation should be a prerequisite for safely deploying watermarked models in medical settings. Current general benchmarks, however technically sound, can conceal clinically consequential failures in high-risk domains like medicine.
---
Epilogue: A Story About Trust
Back to that late-night emergency room. The physician's doubts were soothed by a green checkmark, and a decision was made on trust — trust that the badge meant "this text is reliable."
But this study tells us: a watermark verifies provenance, not quality; identity, not accuracy. Like a passport proving who you are, not whether you're a good person.
As AI enters high-stakes decision-making, we need finer-grained, domain-aware evaluation frameworks. Watermarking itself is not wrong — it provides valuable traceability. But applied to medicine, law, or finance, where error tolerance is near zero, we must look past aggregate metrics and see their shadows.
After all, when lives are at stake, there is no "close enough."
---
References
1. Rieff, M., Staab, R., Gloaguen, T., Hegselmann, S., & Vechev, M. (2026). *Marking the Wrong Symptoms: Evaluating LLM Watermarks in Medical Texts*. arXiv:2607.20462. FM4LS @ ICML 2026. 2. Kirchenbauer, J., et al. (2023). A Watermark for Large Language Models. *ICML*. 3. Hastuti, R.P., et al. (2025). Factuality beyond coherence: Evaluating LLM watermarking methods for medical texts. *EMNLP 2025 Findings*. 4. Kamenica, E., & Gentzkow, M. (2011). Bayesian Persuasion. *American Economic Review*. 5. Kuditipudi, R., et al. (2024). Robust Distortion-free Watermarks for Language Models. *TMLR*.