The Words That Move a Patient Down the Queue: Stigmatizing Language and LLM Emergency Triage

An Outcome-Grounded Review of Evidence, Mechanisms, and Governance

Prepared: 2026-10-19 Style: Vancouver numbered citations. All PubMed identifiers were mechanically verified through NCBI E-utilities esummary. The seed article is cited by DOI because it had no PubMed record at the time of preparation. Scope: This review is a narrative, source-anchored synthesis, not a systematic review or meta-analysis. It separates three claims that are often blurred together: (a) effects of stigmatizing input language on model output, (b) real-world inequity in human emergency department (ED) triage, and (c) the regulatory response to clinical artificial intelligence (AI).


Abstract

Emergency department triage is a rationing decision made under time pressure, incomplete information, and structural scarcity. Large language models (LLMs) are now being evaluated as triage and prioritization aids, but most evaluations ask whether a model agrees with a nurse-assigned acuity score or a clinician-authored vignette. They rarely ask whether a model’s output is wrong in a way that tracks a patient’s actual clinical trajectory. A 2026 controlled experiment by Jarrett and colleagues addressed that gap directly: it paired ED patients matched on Emergency Severity Index (ESI), one of whom deteriorated within six hours and one of whom did not, and measured whether adding a single demographic, social, or stigma-related line to a structured triage summary changed which patient the model said should be seen first [1]. Across 221,556 comparisons, three open-weight models, two US academic ED datasets, and 18 conditions, the most consistent harmful shifts followed stigmatizing formulations of frequent ED use and psychiatric history; race, language, and insurance did not show consistent harmful shifts [1]. The finding is important less because it proves that LLMs are biased in general than because it shows that formulation—not only category—can move an outcome-anchored prioritization decision, and that the effect is model-specific [1].

This review places that experiment in four literatures. First, the LLM ED triage literature shows moderate and highly heterogeneous accuracy, limited prospective outcome evidence, and a recurring gap between benchmark concordance and patient-centered safety [2-12]. Second, the clinical-record literature shows that stigmatizing descriptors are common, patterned by race and class, transmitted to subsequent clinicians, and embedded in the text corpora from which clinical language models learn [13-29]. Third, the prompt-sensitivity literature shows that LLM clinical outputs are vulnerable to framing, anchoring, sycophantic challenge, demographic cues, and apparently trivial lexical changes [31-42]. Fourth, the human-factors and governance literatures show that triage is already unreliable and inequitable, that automation bias is a real hazard, and that current regulation addresses documentation, risk management, and reporting far more explicitly than it addresses stigma-safe input design [43-64].

The synthesis supports five conclusions. (1) The seed study’s outcome anchor is the right methodological move, but “harmful reprioritization” is a model-relative proxy, not demonstrated patient harm. (2) The strongest signal is not a single demographic variable but the framing of utilization and mental-health information; “frequent flyer” style formulations and behavioral-flag language deserve explicit scrutiny. (3) Model choice, prompt design, and interface design are safety decisions, not implementation details. (4) Human triage is not a clean gold standard: ESI reliability is modest in multicenter use, under-triage is patterned by race, and delays are associated with mortality. (5) Governance currently requires transparency and risk management but does not define a stigma-safe triage input standard; that is a tractable policy and engineering target. The practical implication is not to abandon LLM triage but to treat the input layer—what is shown, how it is phrased, and to whom—as part of the clinical intervention, subject to local validation, subgroup audit, and post-deployment monitoring.


1. Introduction: The Queue as a Moral Interface

Every emergency department runs a queue. The queue is not merely an operational artifact; it is a moral interface that decides who is evaluated first, who waits, and, at the margin, who deteriorates while waiting. Triage systems such as the Emergency Severity Index (ESI) in the United States, the Manchester Triage System (MTS) in the United Kingdom and Europe, and the Canadian Triage and Acuity Scale (CTAS) in Canada were developed to make that decision explicit, reproducible, and clinically defensible [43-45]. They sort patients into a small number of acuity levels using chief complaint, vital signs, resource expectations, and clinician judgment. Their purpose is to allocate attention to the sickest patients first. Their limitation is that they depend on human perception, documentation, and a small number of variables that cannot capture everything relevant.

Into this setting comes a new kind of decision aid. LLMs can read free-text triage summaries, synthesize scattered information, and produce a ranking, a disposition recommendation, or a structured acuity label. Early studies reported encouraging agreement with clinician triage and, in some comparisons, performance comparable to that of untrained physicians [2,5,6]. Health systems have begun to evaluate LLM-based triage and clinical decision support prospectively [12], and consumer-facing products have reached millions of users [65]. The policy question is no longer whether these systems will be tried, but how their safety will be established before they are trusted with the queue.

The seed study for this review asks a narrow and unusually well-posed question: if the same patient is described with an added piece of clinically stigmatizing information, does an LLM move that patient down the queue relative to a paired patient who actually deteriorated? The design does not ask whether the model “is biased” in the abstract. It asks whether a specific input change produces a specific directional error against a specific outcome. That is a more demanding question than most bias evaluations, and it is the right kind of question for a clinical safety claim [1].

The answer matters for three reasons. First, triage is a low-margin, high-volume task. A small absolute shift in the probability of deprioritizing a deteriorating patient can translate into many patients if the system is deployed at scale. Second, the input to an LLM is not a neutral summary of the chart; it is a curated representation assembled by a human or a pipeline. Which fields are included, how they are worded, and what is omitted are design choices. Third, the clinical record already contains language that encodes judgment about patients—“frequent flyer,” “drug-seeking,” “non-compliant,” “difficult,” “known to staff”—and that language is not randomly distributed across race, insurance status, or housing status [13-22]. If LLMs learn from that record, they may learn its judgments as well as its facts. The broader algorithmic-bias literature shows that clinical prediction tools can encode racial bias when they use convenient proxies: a widely used population-health algorithm that predicted health care costs rather than illness found that, at a given risk score, Black patients were considerably sicker than White patients; correcting the target would have increased the share of Black patients receiving additional help from 17.7% to 46.5% [82]. Race-based corrections in physiology can similarly embed social judgments: pulse oximetry measurement differs by race, with higher rates of occult hypoxemia in Black patients [83], and race-based estimated glomerular filtration rate equations overestimated measured GFR in Black patients before race-free creatinine-cystatin C equations were developed [84]. LLMs can reproduce race-based medicine and demographic stereotypes in clinical scenarios and educational materials [80,81]. These findings do not prove that the seed study’s effects are racial in origin—its race conditions were null—but they establish that clinical algorithms and clinical language can carry structured bias independently of any single demographic label.

1.1 Evidence base and method

Sources were identified between September and October 2026 through PubMed and EuropePMC keyword and MeSH queries covering LLM triage, ESI/MTS/CTAS, algorithmic bias, stigmatizing language, prompt sensitivity, human factors, and AI governance; citation snowballing from the seed paper and its reference list; the OpenAlex and EuropePMC records for the seed article; ClinicalTrials.gov v2 queries for AI and LLM triage studies; and primary regulatory documents from the FDA, ONC, WHO, the European Union, and CHAI. The seed article’s published in-press PDF and its preprint were retrieved; the published version is used for all effect sizes [1].

Every PubMed identifier in the reference list was verified twice through NCBI E-utilities esummary: once during reference construction and once in an independent final pass against the returned title and metadata. The final reference list contains 94 items, of which 87 are PMID-verified journal articles and 7 are non-PMID documents (the seed article and six regulatory or guideline documents). The review is a narrative synthesis, not a systematic review; it does not pool effect estimates or grade the certainty of evidence. Where a claim is an inference rather than a reported finding, the text says so.

This review is organized around that problem. Section 2 describes the seed experiment and its findings in detail. Section 3 reviews the broader LLM triage evidence base. Section 4 examines stigma in the clinical record and the evidence that documentation affects subsequent decision-making. Section 5 explains why outcome anchoring matters and what it can and cannot establish. Section 6 reviews mechanisms of model susceptibility. Section 7 describes the human baseline that any AI system would augment or replace. Section 8 reviews the governance landscape. Section 9 translates the evidence into practical safeguards, Section 10 sets out a research agenda, and Section 11 states the limitations of this review.


2. The Seed Study: Design, Findings, and What It Does Not Show

2.1 Design

Jarrett and colleagues conducted a controlled, single-attribute experiment using retrospective data from two US academic EDs: the MIMIC-IV-ED dataset from Beth Israel Deaconess Medical Center (425,087 adult ED visits, 2011-2019) and the MC-MED dataset from Stanford University Medical Center (118,385 adult ED visits, 2020-2022) [1]. From each dataset they sampled ESI-matched pairs: one patient who experienced clinical deterioration within six hours of arrival and one who did not. Deterioration was a composite of intensive care unit (ICU) admission, intubation, vasopressor initiation, mechanical ventilation, or in-hospital death; ICU admission was timed from ED registration and treated as a proxy for acute deterioration [1].

The final sample comprised 999 pairs from MIMIC-IV-ED and 997 from MC-MED. Each pair was evaluated by three open-weight models: Gemma 4 31B, Qwen3.6 35B-A3B, and DeepSeek V4 Flash [1]. Each patient was represented as a structured “triage capsule” of approximately 150-250 tokens containing age, sex, arrival mode, ESI level, vital signs with out-of-range flags, chief complaint, past medical history, and active medications. Baseline capsules excluded race, ethnicity, language, insurance, housing, and visit frequency by construction [1].

The experiment then added one line of text to one patient’s capsule, immediately after the demographic header, and repeated the pairwise comparison. Eighteen conditions were tested. For three stigma-associated concepts—psychiatric history, substance use, and frequent ED use—the investigators tested both a neutral and a stigmatizing formulation of the same broad concept. For example, the neutral frequent-ED formulation was “ED visit history: established patient with multiple chronic conditions”; the stigmatizing formulation was “ED visit history: 17 visits in the past 12 months.” The neutral psychiatric formulation was “Psychiatric history: bipolar disorder”; the stigmatizing formulation added “3 prior psychiatric emergency holds.” The neutral substance-use formulation was “opioid use disorder, on maintenance therapy”; the stigmatizing formulation was “active IV drug use (heroin), last use today” [1].

Each pair contributed one baseline comparison plus 18 conditions tested twice (once on the deteriorator, once on the non-deteriorator), for 37 comparisons per model per pair. With 1,996 pairs and three models, the experiment comprised 221,556 pairwise comparisons [1]. The model returned a structured JSON choice of A, B, or tie. Temperature was set to zero; Qwen’s reasoning mode was disabled so that all models produced a bare decision under the same no-chain-of-thought protocol. Position assignment was randomized by pair and held constant across conditions [1].

The primary outcome was harmful reprioritization: the proportion of pairs in which the added information caused the model to change its decision such that the patient who actually deteriorated was deprioritized relative to baseline. The unit of analysis was the patient pair; each pair served as its own control. Directionality was tested with an exact binomial test comparing harmful shifts with concordant shifts among all flipped decisions. Multiple comparisons were controlled with Benjamini-Hochberg false discovery rate (FDR) at 5% within each model-dataset cell [1].

Table 1. Design of the seed experiment

Element Detail
Data sources MIMIC-IV-ED (425,087 adult visits, 2011-2019) and MC-MED (118,385 adult visits, 2020-2022)
Unit of analysis ESI-matched pair: one patient deteriorated within 6 h, one did not
Sample 999 pairs (MIMIC-IV-ED) + 997 pairs (MC-MED) = 1,996 pairs
Models Gemma 4 31B, Qwen3.6 35B-A3B, DeepSeek V4 Flash (open weights)
Input Structured triage capsule, ~150-250 tokens; baseline excludes race, ethnicity, language, insurance, housing, visit frequency
Conditions 18 single-line additions: 4 psychiatric labels, bipolar + holds, 2 substance-use formulations, 2 frequent-ED formulations, age, 4 race categories, language, 2 insurance categories, housing
Comparisons 1 baseline + 18 conditions x 2 targets = 37 per model per pair; 221,556 total
Primary outcome Harmful reprioritization: added information flips the decision away from the patient who deteriorated
Inference settings Temperature 0; Qwen thinking disabled; single standardized zero-shot prompt
Statistics Exact binomial directional test; Wilson CIs; Benjamini-Hochberg FDR 5% within model-dataset cell; paired McNemar/Tango for neutral vs stigmatizing

Source: [1].

2.2 Baseline performance

At baseline, without any demographic or social information, all three models identified the deteriorating patient at rates substantially above chance: 73.8% (Gemma), 71.8% (Qwen), and 76.4% (DeepSeek) on MIMIC-IV-ED; and 67.5%, 72.4%, and 71.2% respectively on MC-MED [1]. This is a crucial design feature. The models were not guessing. They had a directionally correct read on acuity from the clinical capsule alone. The experiment therefore tests whether added information pulls them away from a signal they already possessed, not whether they can triage from scratch.

The pairs were ESI-matched but clinically heterogeneous. Deteriorators were older than non-deteriorators in both datasets (64.2 vs 55.5 years in MIMIC-IV-ED; 63.9 vs 57.2 years in MC-MED). ICU admission within six hours accounted for 87.9% of MIMIC-IV-ED deteriorators and 73.3% of MC-MED deteriorators; in-hospital death contributed 20.3% and 26.1%, respectively. Triage vital signs were missing for 28.5% of MIMIC-IV-ED deteriorators and 12.5% of non-deteriorators, and for 1.4% and 0.6% in MC-MED [1]. These differences matter for interpretation: ESI matching does not make pairs clinically identical, and the authors explicitly note that ESI itself may encode under-triage of marginalized patients, so matching on ESI could, if anything, underestimate harmful shifts [1].

2.3 Headline results

The most consistent harmful shifts followed the stigmatizing frequent-ED-use formulation. It produced significant harmful reprioritization in all six model-dataset cells, ranging from 3.1% (DeepSeek, MC-MED) to 9.4% (Qwen, MIMIC-IV-ED) [1]. Within-pair comparisons showed that the stigmatizing formulation exceeded its neutral counterpart in every cell, with paired differences from +1.4 percentage points (95% CI +0.4 to +2.5; Gemma, MC-MED) to +7.1 points (+5.6 to +8.9; Qwen, MIMIC-IV-ED) [1]. The neutral formulation arguably conveyed greater comorbidity burden—“multiple chronic conditions”—yet produced the smaller effect. That inversion is the study’s central empirical puzzle and its most policy-relevant finding: the model was reacting to the framing of utilization, not simply to the amount of medical information.

Psychiatric history also produced significant harmful shifts, reaching 9.5% for the bipolar-plus-holds formulation in Qwen on MIMIC-IV-ED. The stigmatizing formulation exceeded the neutral diagnostic label in all six cells, but the paired difference reached significance only for Qwen: +4.0 points (+2.6 to +5.6) on MIMIC-IV-ED and +2.6 points (+1.5 to +4.0) on MC-MED [1]. The incremental effect of the behavioral-flag phrase was therefore model-dependent, even though psychiatric labels themselves were associated with harmful shifts across several models and diagnoses.

Age manipulation produced significant shifts for Qwen and DeepSeek in both datasets, up to 5.7%. The authors interpret this cautiously because deteriorators were nearly a decade older on average, so some of the apparent age effect may reflect appropriate Bayesian updating rather than ageism [1]. Race (four categories), language (Spanish interpreter required), and insurance (private vs uninsured/self-pay) produced no consistent harmful shifts; harmful rates for these conditions clustered between 0.7% and 3.3% with no directional pattern across models or datasets [1]. Housing status (“homeless, arrived from shelter”) produced significant shifts in only one cell (Gemma, MIMIC-IV-ED, 3.9%) [1].

The substance-use comparison reversed direction. The stigmatizing “active IV drug use, last use today” produced lower harmful reprioritization than the neutral “opioid use disorder on maintenance therapy,” significantly so for Qwen (-2.3 points, -3.5 to -1.4) and DeepSeek (-1.9 points, -3.1 to -1.0) on MIMIC-IV-ED [1]. The authors’ interpretation is clinically plausible: a statement of acute, ongoing intravenous drug use may signal immediate medical risk (overdose, infection), which appropriately raises priority, whereas a chronic maintenance-therapy label does not. Stigmatizing formulations are not monolithic; a formulation can carry both stigma and acute-risk information, and the two can point in opposite directions [1].

Model heterogeneity was substantial. Qwen showed the largest and most consistent stigma effects, Gemma was intermediate, and DeepSeek showed the smallest effects [1]. The authors do not claim to explain this; they note that pretraining corpus composition and alignment are plausible contributors but were not tested. They also emphasize that model selection is not a neutral design choice [1].

Table 2. Selected harmful reprioritization rates (%) in the seed experiment, by model and dataset

Condition (formulation) Gemma MIMIC Qwen MIMIC DeepSeek MIMIC Gemma MC-MED Qwen MC-MED DeepSeek MC-MED
Frequent ED use, neutral (“multiple chronic conditions”) 2.9* 2.3 3.7 2.8 1.6 1.5
Frequent ED use, stigmatizing (“17 visits in 12 months”) 6.2* 9.4* 7.1* 4.2* 6.5* 3.1*
Bipolar disorder, neutral 4.1* 5.5* 4.6 2.0 3.2 2.3
Bipolar disorder + 3 prior holds, stigmatizing 5.2* 9.5* 5.4 2.3 5.8* 2.7
Age manipulation 2.7 5.7* 5.6* 1.8 5.2* 5.2*
Race: Black/African American 1.9 1.8 1.9 1.2 1.7 1.7
Language: Spanish (interpreter required) 1.7 3.3 2.7 1.4 2.1 0.9*
Insurance: uninsured/self-pay 2.7 2.7 3.3 1.7 2.0 1.8
Housing: homeless, arrived from shelter 3.9* 3.2 3.8 2.0 2.3 1.7
Substance use: active IV drug use, last use today 4.3 2.2* 0.9* 2.2 2.0* 0.9*

Asterisks denote FDR-significant directional bias within the model-dataset cell. Perturbation applied to the deteriorating patient. Source: [1].

2.4 What the study establishes

The study establishes four things with reasonable confidence.

First, a single added line of text can change an outcome-anchored pairwise triage decision in a reproducible direction. The effect is not limited to one model or one dataset; the frequent-ED effect replicated across three models and two institutions [1].

Second, the effect is formulation-sensitive. Neutral and stigmatizing descriptions of the same broad concept produced different harmful-shift rates, and for frequent ED use the difference was significant in every cell [1]. This is stronger than a simple “demographic category” claim, because it holds the category roughly constant while changing the framing.

Third, the effect is model-specific. A finding in one LLM does not transfer automatically to another, even when both are open-weight and both perform well at baseline [1]. This has direct procurement implications: a vendor’s bias audit cannot be treated as a class effect.

Fourth, the null findings for race, language, and insurance are informative but narrow. The authors explicitly caution that single-line demographic perturbations may not capture the distributed ways demographic information interacts with clinical reasoning, and that the study tested specific formulations rather than all possible ones [1]. A null result under these conditions is not evidence that LLMs are free of race-related triage bias.

2.5 What the study does not establish

The study does not establish patient harm. “Harmful reprioritization” is defined relative to the model’s own baseline decision and the patient’s subsequent deterioration. It is a model-relative proxy. A model that deprioritizes a deteriorating patient in a pairwise ranking has made a clinically directionally wrong decision, but the experiment does not observe waiting times, treatment delays, or outcomes under deployment. The authors are careful about this; readers should be too [1].

The study does not isolate wording from information. The neutral and stigmatizing formulations differ in wording and, to varying degrees, in the specific facts they convey. “17 visits in the past 12 months” states a different fact than “established patient with multiple chronic conditions”; “3 prior psychiatric emergency holds” adds information beyond a bipolar diagnosis; “active IV heroin use, last use today” describes a different clinical state than “opioid use disorder on maintenance therapy” [1]. The contrasts therefore estimate the combined effect of framing and semantic content. This is a real limitation, acknowledged by the authors, and it matters for the information-gating proposal discussed in Section 9: excluding a field can remove both stigma and clinically relevant risk information.

The study does not test alternative prompts, system messages, output schemas, or non-zero temperatures. All comparisons used one standardized zero-shot prompt at temperature zero [1]. The prompt-sensitivity literature reviewed in Section 6 suggests that these choices could change effect sizes, though the direction of the seed finding is unlikely to be an artifact of a single prompt given its replication across models and datasets.

The study does not generalize to proprietary frontier APIs, non-English records, community or safety-net hospitals, pediatric populations, or unstructured full clinical notes [1]. It used structured summaries, two US academic centers, and open-weight models selected for on-premises deployability. Those are precisely the conditions under which HIPAA-constrained local deployment is plausible, but they are not the whole deployment space.

Finally, the study does not establish mechanism. It shows differential behavior in response to labels, not why. The authors offer the hypothesis that stigma in clinical text is encoded in the models’ training corpora, citing evidence that clinical notes contain stigmatizing language patterned by race and that note language can transmit bias to subsequent clinicians [1,13-22]. That is a plausible and evidence-supported hypothesis, but the experiment does not directly test training data, alignment, or internal representations.


3. The LLM Emergency Triage Evidence Base

3.1 Accuracy against clinician reference standards

The LLM triage literature has grown rapidly since 2023. Most studies ask a model to assign an acuity level or choose the higher-acuity patient, and compare the output with nurse-assigned ESI, physician consensus, MTS, CTAS, or a vignette key. The results are best described as moderate, heterogeneous, and reference-standard-dependent.

In a cross-sectional study of 10,000 pairs of adult ED visits at the University of California, San Francisco, an LLM correctly identified the higher-acuity patient in 8,940 of 10,000 pairs (accuracy 0.89; 95% CI 0.89-0.90). In a 500-pair subsample also classified by a physician reviewer, LLM accuracy was 0.88 (0.86-0.91) and physician accuracy was 0.86 (0.83-0.89) [2]. That is a strong result, but the reference standard was the ESI assigned in the record, not an independent outcome. The model was predicting the triage label, not validating it.

Pre-LLM machine-learning triage models already showed that prediction from triage data is feasible, but they were trained on structured variables and generally optimized for disposition or mortality rather than for equity across labels [87]. Natural-language-processing approaches to triage added free-text information and improved some predictions, but they may inherit the documentation patterns of the notes they consume [88]. A German study compared GPT-4-based ChatGPT, other LLMs, untrained doctors, and professionally trained raters on 124 vignettes using the Manchester Triage System. GPT-4 ChatGPT achieved substantial agreement with the professional consensus (quadratic-weighted κ = 0.67) and untrained doctors achieved κ = 0.68; GPT-3.5 ChatGPT achieved κ = 0.54. Using the LLM as a second opinion improved untrained doctors only slightly and non-significantly (κ = 0.70, P = 0.97). LLMs tended toward over-triage, while untrained doctors under-triaged [5]. The study’s conclusion is sobering: the best models performed at the level of untrained ED doctors, not trained raters, and did not reliably improve untrained performance.

A prospective single-center study compared ChatGPT Plus, Copilot Pro, and triage nurses on adult ED patients. Overall accuracy was 65.2% for nurses, 66.5% for ChatGPT, and 61.8% for Copilot, with no significant difference. Agreement with the emergency physician reference was moderate (κ = 0.537 for ChatGPT, 0.477 for nurses, 0.472 for Copilot). For recognizing high-acuity patients, ChatGPT and Copilot outperformed nurses (87.8% and 85.7% vs 32.7%), and nurses significantly under-triaged [6]. The high-acuity result is striking, but the study was small, single-center, one week long, and used physician-authored vignettes derived from the same encounters; the nurse comparator may not represent typical practice.

A systematic review and meta-analysis of 11 studies and 3,088 real or simulated ED cases found a pooled odds ratio for LLM triage accuracy of 0.65 (95% CI 0.32-1.34; I² = 97%), meaning the confidence interval spans both substantial harm and substantial benefit. For identifying the highest-acuity category, pooled sensitivity was 61% (48-73%), specificity 97% (88-99%), and AUC 0.82 (0.78-0.85) [3]. The authors emphasize that these estimates reflect concordance with a triage category, not discrimination of outcome-anchored physiological acuity [3].

A second meta-analysis of 15 studies found pooled accuracy of 0.51 for ChatGPT-3.5, 0.70 for ChatGPT-4 and variants, and 0.81 for optimized versions. Overall, ChatGPT did not differ significantly from human triage in accuracy or sensitivity. ChatGPT-4 showed higher accuracy (RR 1.04) and sensitivity (RR 1.46) than humans, but the advantage was not robust to sensitivity analyses [4]. A broader meta-analysis of 50 studies comparing LLMs, healthcare professionals, and collaboration found triage accuracy similar between LLMs and professionals (relative accuracy 1.01; 95% CI 0.94-1.09) [10]. The consistency across meta-analyses is not that LLMs are clearly better or worse; it is that the evidence is too heterogeneous and too dependent on reference standard to support a strong superiority claim.

3.2 Accuracy against outcomes

Very few studies anchor model performance to what happened to the patient. The seed study is one [1]. A trauma triage benchmark of 150 vignettes assessed GPT-5.2, Gemini 3.0 Flash, and Claude 4.5 Opus against surgeon consensus using Injury Severity Score-anchored categories. Overall sensitivity for major trauma was 89.4%, specificity 86.5%, and pooled κ = 0.74. GPT-5.2 had the highest discrimination (AUC 0.94, κ = 0.82). But undertriage was patterned: 17.6% in geriatric scenarios versus 5.2% in younger adults (P = 0.012), and 11.8% in blunt versus 4.1% in penetrating trauma (P = 0.045). Across standardized lexical perturbations, 11.2% of repeated prompts changed triage category, with a maximum instability of 24.5% [11]. Aggregate accuracy concealed clinically meaningful subgroup error, and small wording changes produced category changes.

A prospective evaluation of an LLM-based clinical decision support system (SHAKED) in a tertiary ED is one of the few real-world deployment studies. Over four weeks, 1,138 patients were analyzed across two parallel units. Clinical adoption declined from 68% to 30%, with workload-sensitive disengagement (OR 0.72 per shift hour; 95% CI 0.62-0.83). Physicians preferred SHAKED for radiology consultations (OR 2.98; 1.58-5.63). No adverse events were detected, and expert review rated 99 of 100 sampled outputs as clinically appropriate. ED length of stay did not differ (4.9 hours in both units), and the intention-to-treat trend toward shorter consultation cycle time (-9.4 minutes) was not significant (P = 0.077) [12]. The study’s most important finding may be that sustained clinician engagement, not algorithmic accuracy, was the binding constraint. A system that is accurate but ignored cannot improve care. ClinicalTrials.gov lists a small but growing set of prospective LLM triage studies, including a completed 1,189-patient LLM chest-pain triage study (NCT06493175), a completed 6,360-participant layperson-LLM diagnosis and triage trial (NCT07250516), a completed 600-note Turkish ED anamnesis accuracy study (NCT07632859), a recruiting 220-participant layperson AI-triage trial (NCT07728513), and an enrolling 250-participant voice-agent history-taking study (NCT07877883). The SHAKED evaluation is registered as NCT06902675 [12]. Most of these are diagnostic-accuracy or feasibility studies; none is a large outcome trial of label-specific triage harm.

A consumer-facing stress test of ChatGPT Health used 60 clinician-authored vignettes across 21 clinical domains and 16 factorial conditions, yielding 960 responses. Among gold-standard emergencies, the system undertriaged 52% of cases, directing patients with diabetic ketoacidosis or impending respiratory failure to 24-48 hour evaluation rather than the ED. Performance followed an inverted U-shape, with dangerous failures at both clinical extremes: 35% for non-urgent presentations and 48% for emergency conditions. When family or friends minimized symptoms, indicating anchoring bias, triage recommendations shifted significantly in edge cases (OR 11.7; 95% CI 3.7-36.6), mostly toward less urgent care. Patient race, sex, and barriers to care did not show significant effects, though confidence intervals did not exclude clinically meaningful differences [65]. This is a consumer self-triage product, not an ED triage system, but it shows the same pattern: aggregate performance can look acceptable while high-risk cases fail.

3.3 Self-triage and patient-facing tools

A systematic review of 19 studies compared symptom-assessment applications, LLMs, and laypeople for self-triage. Symptom-assessment application accuracy ranged from 11.5% to 90.0%; LLM accuracy ranged from 57.8% to 76.0%; laypeople ranged from 47.3% to 62.4%. The authors concluded that neither applications nor LLMs should be universally recommended or discouraged; utility depends on the specific use case and user group [7]. The range for commercial symptom checkers is so wide that “accuracy” without the product, population, and reference standard is not a meaningful label.

A CTAS study queried ChatGPT with six prompts across 61 validated vignettes, 30 repetitions each, for 10,980 simulated triages. A CTAS score was returned in 99.6% of queries, but repeatability (same prompt repeated) accounted for 21.0% of overall variation and reproducibility (different prompts) accounted for 4.0%. Overall accuracy was 47.5%, with 13.7% under-triage and 38.7% over-triage. More detailed prompts improved reproducibility but minimally improved accuracy [8]. The study’s conclusion was that ChatGPT was not sufficient for CTAS triage because of poor repeatability and accuracy. This is a direct challenge to the idea that a single validation study can certify a model for triage: the same model, same cases, and different prompts produced materially different results.

An ESI-based study of real-time ED patients found 76.6% accuracy and moderate agreement with an expert committee (κ = 0.659); agreement for high-acuity ESI-1/2 was high (κ = 0.828), and specificity, negative predictive value, and accuracy for high-acuity categories were 95.63%, 98.17%, and 94.90% [66]. A MEWS-based study found that prompt engineering improved accuracy, and GPT-4-Turbo reached 100% while GPT-4o reached 96.2% on simulated scenarios [67]. These results are more optimistic than the CTAS study, which illustrates how much the reference standard and task framing matter.

Benchmark performance is a weak proxy for clinical performance. MultiMedQA and related evaluations showed that LLMs can encode substantial clinical knowledge and pass licensing-style examinations, but they also showed gaps in factuality, reasoning, and bias that automated benchmarks did not capture [89,90]. A diagnostic-reasoning system optimized for differential diagnosis outperformed unassisted clinicians on challenging case reports (top-10 accuracy 59.1% vs 33.6%), but that is a benchmark task, not triage [91]. MedHELM, a clinician-validated taxonomy of 121 medical tasks, was introduced precisely because licensing-exam performance does not reflect the complexity of real clinical work [92]. A systematic review of 761 LLM evaluations found that accuracy was the most commonly assessed parameter and that medical-domain models were a small minority of evaluated systems [93]. A scoping review of robustness testing found that only 33% of studies designed tests that clearly mimicked plausible clinical scenarios, and that misleading or adversarial prompts dominated [94]. A large real-world evaluation of GPT-3.5 and GPT-4 on 10,000 ED visits for admission status, radiology requests, and antibiotic prescriptions found both models performed poorly compared with a resident physician: GPT-4 was 8% lower and GPT-3.5 was 24% lower on average. Both models were overly cautious, with high sensitivity at the cost of specificity [9]. Accuracy on a triage-like classification task does not imply accuracy on the downstream decisions that determine flow.

3.4 What the triage literature implies for the seed finding

Three implications follow. First, the seed study’s baseline accuracy (67.5-76.4%) is in the same range as other LLM triage studies, so its harmful-shift estimates are not an artifact of a model that cannot triage [1-3]. Second, the literature’s dominant reference standards—ESI, MTS, CTAS, vignette keys—are themselves human constructs with known reliability limits (Section 7). A model that agrees with them may be learning the label rather than the patient’s risk. Third, prospective outcome evidence is sparse. The SHAKED study is a stage 1 evaluation with no adverse events and no length-of-stay effect; it does not establish patient benefit or harm [12]. The seed study’s outcome anchor is therefore a methodological advance, but it remains a retrospective, model-relative proxy.

Table 3. Selected LLM emergency triage studies

Study Design and reference standard Key result Main limitation
Williams et al. 2024 [2] 10,000 ESI-discordant ED visit pairs; ESI label Accuracy 0.89; comparable to physician reviewer (0.88 vs 0.86) Predicts ESI label, not outcome; single center
Masanneck et al. 2024 [5] 124 vignettes; MTS consensus GPT-4 ChatGPT κ = 0.67 vs untrained doctors 0.68; GPT-3.5 κ = 0.54 Vignettes; no outcome anchor
LLM vs nurses 2025 [6] Prospective single-center; physician ESI Accuracy 66.5% ChatGPT vs 65.2% nurses; high-acuity 87.8% vs 32.7% Small, one week, single center
Cui et al. 2026 [3] Meta-analysis, 11 studies, 3,088 cases Pooled OR 0.65 (0.32-1.34); sensitivity 61%, specificity 97%, AUC 0.82 High heterogeneity; reference-standard concordance
ChatGPT meta-analysis 2026 [4] Meta-analysis, 15 studies Accuracy 0.51 (GPT-3.5), 0.70 (GPT-4), 0.81 (optimized); no robust superiority Heterogeneous; sensitivity analyses unstable
LLM vs HCP meta-analysis 2026 [10] 50 studies Triage relative accuracy 1.01 (0.94-1.09) Methodological flaws; few real-world studies
Trauma benchmark 2026 [11] 150 vignettes; surgeon consensus Sensitivity 89.4%, specificity 86.5%, κ = 0.74; geriatric undertriage 17.6% In silico; educational/registry vignettes
SHAKED 2026 [12] Prospective DECIDE-AI stage 1; 1,138 patients Adoption 68% to 30%; no LOS difference; 99/100 outputs appropriate No patient outcome; engagement-limited
ChatGPT Health 2026 [65] 60 vignettes, 16 conditions; clinician key 52% undertriage of emergencies; anchoring OR 11.7 Consumer product; vignettes
Seed study 2026 [1] 1,996 ESI-matched pairs; 6-hour deterioration Frequent-ED harmful shifts 3.1-9.4%; neutral vs stigma +1.4 to +7.1 points Model-relative proxy; single-line perturbations

4. Stigma in the Clinical Record: From “Frequent Flyer” to “Non-Compliant”

4.1 What counts as stigmatizing language

Stigmatizing language in the clinical record is language that frames a patient or a patient’s behavior in a way that conveys negative judgment, doubt, or reduced worth. It can be explicit (“drug-seeking,” “frequent flyer,” “non-compliant,” “difficult,” “manipulative”) or subtle (quotation marks around a patient’s reported symptom, “claims” or “insists,” evidential constructions that treat the patient’s account as hearsay) [13-17]. The chief complaint is often the patient’s own words; the note is the clinician’s interpretation, and the movement from one to the other is where testimonial injustice can enter [16]. It can also be structural: a behavioral flag, a “high utilizer” label, a note that describes a patient by housing status rather than by clinical need [18,19]. A qualitative study of 600 ambulatory encounter notes identified five themes of negative language: questioning patient credibility, expressing disapproval of patient reasoning or self-care, stereotyping by race or social class, portraying the patient as difficult, and emphasizing physician authority over the patient [85]. A review focused on emergency nursing practice describes how labels such as “drug-seeker,” “frequent flyer,” and “non-compliant” are applied without comprehensive assessment and can delay treatment, worsen pain management, and erode trust [86].

The seed study’s terminology—“clinically stigmatizing information”—is broader than a word list. It includes neutral and stigmatizing formulations of the same concept, and it treats visit frequency and psychiatric emergency holds as potentially stigmatizing because they can frame a patient as a burden, a safety risk, or a poor historian rather than as a patient with a clinical trajectory [1]. That framing is consistent with the clinical-record literature, which has moved from lexicon-based detection of individual words toward analysis of context, stance, and epistemic positioning [13-17,20].

4.2 Prevalence and patterning

The empirical case that stigmatizing language is common and patterned is strong.

In a study of 40,113 history and physical notes from 18,459 patients at an urban academic medical center, machine learning identified sentences containing a negative descriptor of the patient or the patient’s behavior. Compared with White patients, Black patients had 2.54 times the odds of having at least one negative descriptor in the notes, after adjustment for sociodemographic and health characteristics [13]. The descriptor list included words such as “resistant” and “noncompliant.” The study did not establish that the language caused worse care, but it established that the record is not a neutral mirror: it encodes racialized judgment at scale.

A second study of 48,651 admission notes about 29,783 patients found stigmatizing language in 2.5% of all notes. Prevalence varied by condition: 6.9% of notes for patients with diabetes, 3.4% for substance use disorder, and 0.7% for chronic pain. Notes about non-Hispanic Black patients had a 0.67 percentage point greater probability of containing stigmatizing language than notes about non-Hispanic White patients, with similar disparities in all three diagnosis-specific subgroups. Greater diabetes severity and less advanced physician training were associated with more stigmatizing language [14]. The absolute prevalence is modest, but the direction is consistent across studies.

A third study focused specifically on triage documentation for pain-related ED visits: 51,570 visits, of which 2,691 triage notes (5.2%) contained at least one stigmatizing word. Male gender (OR 1.21; 95% CI 1.12-1.31), Black race (OR 1.11; 1.01-1.23), Medicaid (OR 1.29; 1.17-1.44), and Medicare (OR 1.22; 1.11-1.34) were associated with higher odds of stigmatizing language. Lower-acuity visits were less likely to contain such language (ESI-3 OR 0.77; ESI-4 OR 0.52) [15]. This is the most directly relevant prevalence study for the seed paper because the note is the triage note—the same location where an LLM triage input might be assembled.

A linguistic study of 9,251 clinic notes written by 165 physicians about 3,374 patients found that notes about Black patients had higher odds of containing at least one quote (OR 1.48; 95% CI 1.20-1.83) and at least one judgment word (OR 1.25; 1.02-1.53), and used more evidentials (β 0.32; 0.17-0.47) than notes about White patients. Notes about female patients did not differ in judgment words or evidentials but had higher odds of containing quotes [16]. The authors frame this as testimonial injustice: the patient’s account is recorded as less credible. That is a different mechanism from a slur, and it is more likely to survive a naive word filter.

Behavioral flags are a structural form of the same phenomenon. In a retrospective cohort of 195,601 patients and 426,858 ED visits across three Philadelphia EDs, 683 patients (0.3%) had a behavioral flag, present for 6,851 visits. Black patients received flags at 4.0 per 1,000 patients versus 2.4 per 1,000 White patients (P < 0.001). Among flagged patients, Black patients waited longer to be placed in a room (median 28.0 vs 18.2 minutes) and longer to see a clinician (42.1 vs 33.3 minutes), and underwent fewer laboratory and imaging tests than flagged White patients [18]. The flag is intended to promote safety; the data show it is also a channel through which structural disadvantage operates.

Housing status behaves similarly. In a study of 2,751 people experiencing homelessness with 5,049 ED encounters, charting the word “homeless” was a strong predictor of discharge rather than admission (OR 0.70; 95% CI 0.60-0.82), especially for heart failure exacerbation (OR 0.45) [19]. A study of negative descriptors in sickle cell disease found that patients with sickle cell disease—most of whom are Black, often present with pain, and are often treated with opioids—had elevated application of descriptors such as “aggressive,” “agitated,” “angry,” “nonadherent,” “noncompliant,” “noncooperative,” and “refuse” [20]. These are not marginal findings. They show that the same clinical facts can be recorded in ways that either support or undermine a patient’s claim to care.

4.3 Does documentation change subsequent decisions?

The strongest causal evidence comes from a randomized vignette study. Goddu and colleagues randomized 413 physicians-in-training to read one of two chart notes describing the same hypothetical 28-year-old man with sickle cell disease: one note used stigmatizing language, the other neutral language. Exposure to the stigmatizing note was associated with significantly more negative attitudes toward the patient and significantly less aggressive pain management (both P < 0.001) [17]. The study is hypothetical, but it directly tests transmission: a note written by one clinician changed the attitudes and decisions of another.

The seed study’s hypothesis is a machine-learning extension of this finding. If clinicians transmit bias through notes, and if LLMs are trained on notes, then LLMs may learn the same associations. The evidence for the first half is strong [13-20]. The evidence for the second half is indirect but growing: word-embedding models trained on ICU notes show that racial descriptors can be contextually similar to stigmatizing words, though the direction varied by site and era [21]; an NLP algorithm trained on physician notes to identify epilepsy surgical candidates did not show race effects after adjustment but did show effects of age, travel distance, family income, and insurance [22]; and an epilepsy-specific LLM showed little bias in classification accuracy across demographic groups but revealed disparities in seizure outcomes that tracked insurance and income [23]. A social-determinants extraction package showed differences in documentation and extraction across race and gender groups [24]. These studies do not prove that LLMs learn stigma from notes, but they make the hypothesis plausible and testable.

Copy-forward compounds the problem. A qualitative study of 22 clinicians found that copy and carry-forward tools are valued for efficiency but produce inaccuracies, errors, and patient safety risks [25]. If a stigmatizing phrase is copied forward across encounters, it becomes a durable feature of the record, not a one-time judgment. An LLM that summarizes the chart may then encounter the phrase repeatedly and treat it as corroborated fact.

4.4 The specific case of “frequent flyer”

The “frequent flyer” label is a useful case study because the empirical literature contradicts the stereotype. A systematic review found that frequent ED users comprise 4.5% to 8% of all ED patients but account for 21% to 28% of visits. Most are White and insured; public insurance is overrepresented. On average, they have higher-acuity complaints and greater risk of hospitalization than occasional users, although the highest-frequency users may be different. Only a minority remain frequent users long term [26]. A second systematic review found that frequent users had a median 2.2-fold increased odds of mortality, 1.16-fold increased odds of admission per visit, 2.58-fold increased odds of admission per patient, and 2.65-fold increased risk of at least one outpatient encounter after the ED visit [27]. The label “frequent flyer” implies low acuity, social nuisance, and inappropriate use. The data describe a heterogeneous, high-morbidity population with elevated mortality.

This is exactly the kind of inversion that the seed study found: a neutral description of multiple chronic conditions produced less harmful reprioritization than a visit-count description, even though the neutral description arguably conveyed more comorbidity [1]. The model’s behavior mirrors a human bias documented in the sociological and health-services literature. That convergence is not proof of a common mechanism, but it is a strong reason to treat visit-frequency language as a safety-relevant input variable.

4.5 Mental health, substance use, and diagnostic overshadowing

Psychiatric labels carry a distinct risk: diagnostic overshadowing, the misattribution of physical symptoms to mental illness. A qualitative study of 18 doctors and 21 nurses in four UK EDs documented scenarios in which mental illness or factors related to it led to misdiagnosis or delayed treatment. Contributing factors included complex presentations, poor communication, challenging behavior, crowded environments, time pressure, targets, and stigmatizing attitudes held by a minority of staff [28]. A systematic review of health professionals’ attitudes toward patients with substance use disorders found generally negative attitudes, perceptions of violence and manipulation, and lack of education and support; negative attitudes diminished patients’ empowerment and treatment outcomes [29]. These are the human analogues of the seed study’s psychiatric and substance-use conditions.

The seed study’s substance-use reversal is a useful corrective to a simple “stigma always lowers priority” narrative. Active intravenous drug use may signal acute medical risk; maintenance therapy may signal stability. The model’s response can be clinically appropriate even when the label is stigmatizing. This is why the neutral-versus-stigmatizing design matters: it forces the analysis to ask what the added language conveys, not just whether it is unpleasant [1].

Table 4. Evidence that clinical-record language is patterned and consequential

Study Population/data Finding Interpretation
Sun et al. 2022 [13] 40,113 notes, 18,459 patients Black patients 2.54x odds of negative descriptor Racialized judgment at scale
Hospital notes 2022 [14] 48,651 admission notes 2.5% notes stigmatizing; Black +0.67 pp Condition-specific patterning
Triage notes 2025 [15] 51,570 pain visits 5.2% notes stigmatizing; Black OR 1.11, Medicaid OR 1.29 Triage documentation is not neutral
Beach et al. 2021 [16] 9,251 notes Black patients: quotes OR 1.48, judgment words OR 1.25 Testimonial injustice
Behavioral flags 2023 [18] 426,858 ED visits Black 4.0 vs White 2.4 flags/1,000; longer waits Structural label with care consequences
Homeless descriptor 2025 [19] 5,049 encounters “Homeless” predicts discharge (OR 0.70) Descriptor frames disposition
Goddu et al. 2018 [17] 413 trainees, randomized vignette Stigmatizing note: more negative attitudes and less aggressive pain management (both P < 0.001) Documented transmission to clinicians
Frequent users 2010/2016 [26,27] Systematic reviews 4.5-8% patients, 21-28% visits; ~2.2-fold mortality Label contradicts epidemiology

5. Outcome Anchoring: Why It Matters and What It Can and Cannot Prove

5.1 The problem with label concordance

Most LLM triage studies measure agreement with a human-assigned label. That is useful for benchmarking, but it cannot answer the safety question. If the human label is itself biased, a model that reproduces it will look accurate while reproducing the bias. The ESI literature shows that this is not a hypothetical concern: in a cohort of 5,315,176 ED encounters, mistriage occurred in 32.2% of encounters, including 3.3% under-triage and 28.9% over-triage; the sensitivity of ESI to identify a patient with high-acuity illness was 65.9%. Black patients had an 18.5% greater relative risk of under-triage than White patients, and Black male patients had a 41.0% greater relative risk than White female patients [46]. A model trained or evaluated against ESI can inherit these errors.

The seed study avoids this trap by using a clinical outcome—deterioration within six hours—as the reference. It does not ask whether the model agrees with the nurse; it asks whether the model’s ranking aligns with what happened to the patient. That is a genuine advance. It also explains why the study’s baseline accuracy is lower than some label-concordance studies: predicting deterioration is harder than reproducing ESI [1].

5.2 What “harmful reprioritization” measures

The primary outcome is the proportion of pairs in which adding information caused the model to flip away from the patient who deteriorated. It is a conditional probability: given that the model made a baseline decision, how often did the new information move it in the wrong direction? It is not the probability that a deployed system will delay care. It is not a risk difference in waiting time. It is not a patient outcome. The authors are explicit that the design is a controlled audit, not a clinical trial [1].

The outcome is nevertheless clinically meaningful in direction. If a model systematically deprioritizes the patient who subsequently deteriorates when a stigmatizing label is added, that is a failure mode that a deployment study should measure. The effect sizes—3.1% to 9.4% for frequent ED use, up to 9.5% for psychiatric history—could matter at ED volume, but they are not so large that they imply most patients with these labels will be misranked. The appropriate interpretation is probabilistic: across all pairs, a stigmatizing label was associated with a switch away from the deteriorating patient in roughly 3% to 9% of comparisons, depending on model and dataset—about 1 in 10 to 1 in 30 comparisons.

5.3 The deterioration composite

The outcome composite includes ICU admission, intubation, vasopressor initiation, mechanical ventilation, and in-hospital death. ICU admission within six hours was the most common component (87.9% of MIMIC-IV-ED deteriorators and 73.3% of MC-MED deteriorators). In-hospital death contributed 20.3% and 26.1%, respectively [1]. Because MIMIC-IV-ED deaths were rarely timestamped within six hours, 12.1% of MIMIC-IV-ED deteriorators and 10.5% of MC-MED deteriorators qualified only by a late death. The authors conducted a sensitivity analysis excluding those pairs; FDR significance changed for only 2 of 108 condition cells, and the core frequent-ED, psychiatric, and age effects were unchanged [1].

The composite is a reasonable proxy for “acutely deteriorating,” but it is not a pure measure of avoidable harm. ICU admission can reflect bed availability, local practice, or planned transfer; the authors acknowledge that the source data do not reliably distinguish planned from unplanned admission and treat early ICU transfer as a proxy [1]. Intubation and vasopressor initiation are stronger signals of acute deterioration, but they are also intervention-dependent. A patient who would have deteriorated but received early treatment may not meet the composite. This is a general problem in outcome-anchored AI evaluation: the outcome is itself affected by care.

5.4 Counterfactual reasoning and its limits

The seed design is a form of counterfactual audit: for the same patient pair, change one attribute and observe the decision. This is conceptually related to counterfactual fairness, which asks whether a decision would be the same if a protected attribute were different [1]. But the implementation is not a true counterfactual on a single patient record. The pair consists of two different patients matched on ESI, not one patient duplicated with an attribute changed. The baseline comparison is within-pair, but the “same patient” counterfactual is approximated, not realized. The authors acknowledge that ESI matching reduces but does not eliminate within-tier confounding [1].

This matters for interpretation. If the two patients differ in unmeasured ways, the added line may interact with those differences. The neutral-versus-stigmatizing contrast partially addresses this because both formulations are applied to the same pair, but the formulations also differ in content. A stronger design would hold the clinical facts constant and vary only the framing, or use a within-patient record with the same clinical data and a randomized label. That is a clear next study.


6. Mechanisms of Susceptibility: Prompt Sensitivity, Framing, Anchoring, and Sycophancy

6.1 The model is not a fixed measuring instrument

A recurring assumption in clinical AI evaluation is that a model has a stable “ability” that can be measured once and deployed. The prompt-sensitivity literature shows that this assumption is false in practice. Small changes in prompt wording, order, format, or context can change outputs substantially, and the direction of change is not always predictable.

In a study of 11,999 pulmonary CT angiography reports, three Gemini models achieved accuracy between 0.953 and 0.996 for pulmonary embolism classification. Few-shot prompting improved recall, chain-of-thought generally degraded performance, and robustness against data perturbations varied by model version. Geographic cloud region variability was minimal [30]. The absolute performance was high, but the rank order of models and the effect of prompting strategy depended on the task configuration. A validation study that fixes one prompt and one model version measures one point in a large design space. Order effects are a related vulnerability: the position of information within a prompt can change judgments, which is why the seed study randomized whether the deteriorating patient appeared as patient A or B and held that assignment constant across conditions [1].

A study of 10 LLMs on 200 text-based and 200 multimodal radiology board-style questions found that cognitively biased prompts reduced accuracy. Under baseline prompts, mean accuracy was 84.8% for text and 59.5% for multimodal questions. Authority bias prompts produced absolute declines of 21.1% for text and 44.9% for multimodal; complexity bias prompts produced 10.1% and 44.4%; anchoring bias prompts produced 4.4% and 39.6% (P < 0.001 for each). A prompt bias audit improved accuracy by 5.6% for text and 15.8% for multimodal; one-shot mitigation improved accuracy by 4.0% and 24.9% [31]. The lesson is not that these models are unusable; it is that the prompt is part of the intervention, and a biased prompt can overwhelm domain knowledge.

A study of 80 shoulder and elbow vignettes, including 24 red-flag cases, found that safety-critical under-triage occurred in 19 of 72 red-flag responses (26.4%). Under-triage was most frequent with patient-oriented prompts (10/24, 41.7%), followed by general physician-oriented prompts (6/24, 25.0%) and specialist-oriented prompts [32]. The prompt’s implied audience changed the safety behavior. In a triage system, the “audience” is often encoded in the interface: a patient-facing chatbot, a nurse-facing dashboard, and a physician-facing advisory tool may receive different prompts and produce different thresholds.

6.2 Framing and source effects

A large study of 4,800 narrative statements across 24 topics, evaluated by four LLMs for 192,000 assessments, found high inter- and intra-model agreement at baseline. That agreement broke down when the source of each statement was disclosed. Attributing statements to Chinese individuals systematically lowered agreement scores across all models, especially DeepSeek Reasoner [33]. The study is not about medicine, but the mechanism—source framing changes judgments—is directly relevant to clinical text, where the source of a symptom report (patient vs family vs nurse vs prior note) is often part of the input.

The seed study’s frequent-ED finding can be read as a framing effect: “17 visits in the past 12 months” frames the patient as a high utilizer, while “established patient with multiple chronic conditions” frames the same utilization as continuity of care. The model’s response differs even though the clinical facts are overlapping. The substance-use reversal is another framing effect: “active IV drug use, last use today” frames acute risk, while “opioid use disorder on maintenance therapy” frames chronic stability [1].

6.3 Anchoring

LLMs also reproduce human-like decision heuristics, including anchoring, representativeness, availability, framing, and endowment effects [41]. Temperature changes can produce diagnostic divergence across iterations even when the same case is submitted repeatedly [42]. Anchoring—fixing on an initial piece of information and insufficiently adjusting—is a well-described human diagnostic bias. A forced-choice vignette study compared 20 internal medicine residents, 5 attending physicians, and 8 LLMs. In anchored vignettes, residents ranked the anchor diagnosis first in 21.2% of responses and attendings in 10.0%, compared with LLMs in 55.6% [34]. The LLMs were more anchorable than physicians in this task. The study is small and vignette-based, but it is consistent with the broader concern that LLMs can be steered by a plausible but incorrect suggestion.

The ChatGPT Health stress test found a related effect in consumer triage: when family or friends minimized symptoms, recommendations shifted significantly in edge cases (OR 11.7; 95% CI 3.7-36.6), mostly toward less urgent care [65]. The anchor was social, not diagnostic, but the mechanism is similar: a salient framing changes the threshold for escalation.

6.4 Sycophancy and challenge effects

Sycophancy is the tendency of a model to agree with a user’s challenge or framing rather than maintain a correct answer. It is a safety issue in clinical decision support because a clinician or patient can inadvertently or deliberately push the model away from a correct assessment.

A study of 10 proprietary and open-weight LLMs on 120 clinical vignettes found that baseline diagnostic accuracy was 51.8% across neutral model-case pairs. After the challenge “Are you sure?”, accuracy fell to 42.2%; correct-to-incorrect transitions (544) exceeded incorrect-to-correct transitions (198). The “Are you sure?” challenge was the strongest source of diagnostic instability; specialty context prompts produced smaller changes [35]. A separate study of five frontier LLMs found high initial compliance with illogical requests that misrepresented equivalent drug relationships—up to 100%—prioritizing helpfulness over logical consistency. Prompt engineering and fine-tuning improved rejection rates while maintaining benchmark performance [36]. A Science study across 11 models found that AI affirmed users’ actions 49% more often than humans, even when queries involved deception or illegality; in three preregistered experiments (N = 2,405), a single interaction with sycophantic AI reduced willingness to take responsibility and increased conviction of being right [37]. The clinical implication is straightforward: a triage model that can be talked out of an urgent recommendation is not a safe triage model, and the risk is highest in exactly the situations where a patient or family member is most anxious.

6.5 Demographic perturbation

The evidence on demographic perturbation is mixed, which is itself important. A large study of nine LLMs and more than 1.7 million outputs from 1,000 ED cases (500 real, 500 synthetic) presented each case in 32 variations (31 sociodemographic groups plus a control). Cases labeled as Black, unhoused, or LGBTQIA+ were more frequently directed toward urgent care, invasive interventions, or mental health evaluations. Some LGBTQIA+ subgroups were recommended mental health assessments approximately six to seven times more often than clinically indicated. High-income-labeled cases received significantly more recommendations for advanced imaging, while low- and middle-income-labeled cases were often limited to basic or no further testing. The differences persisted after multiple-hypothesis correction and were not supported by clinical reasoning or guidelines [38]. This is strong evidence that sociodemographic labels can shift LLM recommendations in clinically unjustified directions.

A dental radiology audit of 25,380 assessments found that all three commercial multimodal LLMs altered recommendations when demographic cues were introduced; Claude showed the broadest sensitivity, with significant race effects on treatment invasiveness, prognosis, and confidence [39]. A study of discharge instructions for patients with HIV found only one statistically significant difference across race/ethnicity groups (entity count for GPT-4), and post hoc pairwise comparisons were not significant after correction [40]. The null result does not cancel the positive findings; it shows that the effect depends on task, model, and outcome measure. A single audit cannot establish that a model is unbiased, just as a single null trial cannot establish equivalence across all populations.

The seed study’s null findings for race, language, and insurance are consistent with this mixed picture. They may reflect safety alignment against explicit demographic discrimination, or they may reflect the limitations of single-line perturbations. The authors’ cautious conclusion—that these specific single-line demographic perturbations did not produce significant harmful shifts under the tested conditions, not that LLMs are free of race-related triage bias—is the right one [1].

6.6 Model heterogeneity and the “class effect” fallacy

The seed study found that Qwen was most susceptible, Gemma intermediate, and DeepSeek least susceptible [1]. The trauma benchmark found different discrimination and calibration across GPT-5.2, Gemini 3.0 Flash, and Claude 4.5 Opus [11]. The sycophancy study found large differences across models in challenge-induced accuracy loss [35]. These findings undermine the idea that a bias audit of one model can be generalized to a “class” of LLMs.

They also complicate procurement. A hospital that selects a model based on benchmark accuracy may inadvertently select a model with worse label sensitivity. A vendor’s model card that reports only aggregate accuracy, without condition-specific or subgroup results, does not provide the information needed to make that trade-off. The seed study’s design—paired, condition-specific, outcome-anchored—is a template for the kind of audit that procurement should require [1].

Table 5. Evidence of LLM susceptibility to input perturbation

Study Task Perturbation Key result
Radiology prompt study 2025 [30] PE classification, 11,999 reports Prompt design, data perturbation, cloud region Accuracy 0.953-0.996; chain-of-thought degraded; robustness varied by model
Cognitive bias prompts 2026 [31] 200 radiology questions Authority, complexity, anchoring Declines 21.1%, 10.1%, 4.4% (text); larger for multimodal
Shoulder red flags 2026 [32] 80 vignettes, 24 red-flag Patient vs physician vs specialist prompt Under-triage 26.4%; patient-oriented 41.7%
Anchoring 2026 [34] Forced-choice vignettes Anchored vs control LLM anchor-first 55.6% vs residents 21.2%, attendings 10.0%
Sycophancy 2026 [35] 120 vignettes, 10 LLMs “Are you sure?” Accuracy 51.8% to 42.2%; 544 correct-to-incorrect vs 198 incorrect-to-correct
Sycophancy 2025 [36] Drug-equivalence prompts, 5 LLMs Illogical requests Up to 100% compliance; improved with prompting/fine-tuning
Sociodemographic bias 2025 [38] 1.7M outputs, 1,000 ED cases 31 sociodemographic groups Black/unhoused/LGBTQIA+ more urgent/invasive/mental health; high income more imaging
Dental audit 2026 [39] 25,380 radiograph assessments Race/ethnicity and sex cues All three models changed recommendations
Seed study 2026 [1] 221,556 pairwise comparisons 18 single-line conditions Frequent-ED harmful shifts 3.1-9.4%; neutral vs stigma +1.4 to +7.1 points

7. The Human Baseline: Triage Reliability, Under-Triage, and Automation Bias

7.1 ESI is reliable enough to be useful and unreliable enough to matter

The ESI was introduced to improve the reliability and validity of ED triage. Early validation showed good interrater reliability: in a study of 403 ED records, the weighted κ between nurse-assigned ESI and a criterion standard was 0.89, and ESI predicted admission (80% for ESI-1, 73% for ESI-2, 51% for ESI-3, 6% for ESI-4, and 5% for ESI-5) [43]. That result supported widespread adoption.

Later, multicenter evidence was less reassuring. An international study of 87 ESI-trained nurses in Brazil, the United Arab Emirates, and the United States used standardized AHRQ scenarios. Mean accuracy was 59.2% (95% CI 56.4-62.0%), and interrater reliability was modest (Krippendorff’s α = 0.730; 95% CI 0.692-0.767). Accuracy was higher for medium-acuity scenarios (76.4%) than for high-acuity (44.1%) or low-acuity (54.0%) cases, and higher for adult (66.2%) than pediatric (46.9%) scenarios [44]. A systematic review identified 182 triage studies, 50 of which had sufficient data for comparative analysis; ESI, CTAS, and MTS performed similarly, and most comparative studies (6 of 8) reported high sensitivity (>90%) for identifying high-acuity patients, with substantial variability in reliability and outcome prediction [45]. A review of nurse triage accuracy found rates from 59.3% to 82%, with experience associated with higher accuracy and work environment affecting performance [47].

The implication is not that ESI is worthless. It is that ESI is a human measurement with meaningful error. A model that agrees with ESI 89% of the time may be agreeing with a label that is itself wrong in a substantial minority of cases. The seed study’s use of a clinical outcome rather than ESI as the reference is therefore not a minor methodological nicety; it is a response to a real limitation of the field [1].

7.2 Under-triage is common and patterned

In the Kaiser Permanente Northern California cohort of 5,315,176 ED encounters, mistriage occurred in 32.2% of encounters, including 3.3% under-triage and 28.9% over-triage. ESI sensitivity for high-acuity illness was 65.9%. Black patients had an 18.5% greater relative risk of under-triage and a 9.9% greater relative risk of over-triage than White patients; Black male patients had a 41.0% greater relative risk of under-triage than White female patients [46]. A multicenter Mass General Brigham study of 249,829 visits found that Black, Hispanic, and Other-race patients were assigned less acute ESI scores than White peers despite receiving more involved physician workups, suggesting mistriage [48]. A study of 314,763 visits found unexplained queue jumps—patients placed in a treatment space ahead of higher-acuity or earlier-arriving same-acuity patients—and linked them to hallway placement, leaving before treatment completion, escalation while awaiting a bed, and 72-hour revisits [49].

These findings matter for the seed study in two ways. First, they show that the “ground truth” of triage is contested. Second, they show that the same populations whose labels are stigmatized in notes (Black patients, people experiencing homelessness, people with substance use disorders, people with mental illness) are also the populations that experience under-triage and delayed care. The seed study’s ESI matching may therefore underestimate the real-world effect: if ESI already under-triages marginalized patients, matching on ESI compares patients who are nominally equal but clinically unequal [1].

7.3 Under-triage and delays are associated with worse outcomes

Under-triage is not just a labeling error; it is a delay. In a 21-ED cohort of 5,315,081 encounters, under-triaged high-acuity patients had an 8-minute delay in care compared with true high-acuity patients, and over-triaged low-acuity patients had a 3-minute delay and a 42-minute longer ED length of stay [50]. A study of 5,929 patients with subarachnoid hemorrhage, aortic dissection, or ST-elevation myocardial infarction found that 36.7% were under-triaged and that under-triage was associated with delays in CT, antihypertensive or β-blocker orders, ECG, troponin, and ED length of stay [51].

At the system level, a study of all type 1 EDs in England (26,738,514 attendances, 7,472,480 admissions) found a statistically significant linear increase in 30-day mortality from five hours after arrival to twelve hours. The greatest change was an 8% increase in the standardized mortality ratio for patients who waited more than six to eight hours; for every 82 admitted patients whose transfer was delayed beyond six to eight hours, there was one extra death [52]. A Swedish multicenter study of 122,893 patients and 168,900 visits found no significant association between crowding (measured by mSEAL or occupancy rate) and short-term mortality (HR 1.04; 95% CI 0.96-1.13 for 7-day mortality) [53]. A systematic review of ED boarding and in-hospital mortality found that 6 of 12 studies reported an association and 5 did not [54]. The literature is not unanimous, but the weight of evidence supports the claim that delays and crowding can harm patients, with effect sizes that depend on population, metric, and outcome.

The seed study’s effect sizes should be read against this background. A 3-9% relative increase in the probability of deprioritizing a deteriorating patient is not a 3-9% increase in mortality. But in a system where an 8-minute delay is associated with measurable harm and where an extra death occurs for every 82 patients delayed beyond six to eight hours, a systematic shift in prioritization is a plausible mechanism of harm. The causal chain is long, and the seed study does not measure it. That is the right caution, and it is also the reason prospective studies are needed.

7.4 Automation bias

Automation bias is the tendency to over-rely on automated advice and to under-monitor its output. A systematic review of 74 studies found that automation bias is mediated by cognitive style, task-specific experience, trust, confidence, workload, task complexity, and time pressure. Mitigators include training, explicit user accountability, and design choices such as the position of advice on the screen [55]. A JAMA commentary argues that assistive AI creates a risk of harm from automation bias and calls for evaluation and monitoring [56].

The seed study’s design is a reminder that automation bias can operate in both directions. If a clinician trusts an LLM’s ranking, a stigmatizing input that shifts the ranking could translate into a real delay. If a clinician distrusts the model, the model’s accuracy is irrelevant. The SHAKED study’s decline in adoption from 68% to 30% illustrates the opposite failure mode: a system that is accurate but ignored [12]. The human-AI interaction, not the model alone, determines whether the system helps or harms.

The randomized trial evidence on LLM-assisted clinical reasoning is mixed. In a single-blind RCT of 50 physicians, access to an LLM did not significantly improve diagnostic reasoning (median score 76% vs 74%; adjusted difference 2 percentage points; 95% CI -4 to 8; P = 0.60), although the LLM alone scored 16 percentage points higher than the conventional-resources group [57]. A second RCT of 92 physicians found that GPT-4 assistance improved management reasoning by 6.5 percentage points (95% CI 2.7-10.2; P < 0.001), but LLM users spent more time per case [58]. The difference between these trials may reflect task, model, interface, and population. The consistent lesson is that “the model is accurate” does not imply “the clinician-AI system is better.”

Table 6. Human triage reliability and outcome evidence

Study Data Finding
Tanabe et al. 2004 [43] 403 ED records ESI v3 weighted κ = 0.89; admission gradient by ESI
ESI international 2018 [44] 87 nurses, 3 countries Accuracy 59.2%; α = 0.730; high-acuity accuracy 44.1%
Triage systematic review 2019 [45] 182 identified; 50 comparative ESI/CTAS/MTS similar; high sensitivity but variable reliability
Nurse accuracy review 2024 [47] 14 studies Accuracy 59.3-82%; experience and environment matter
ESI mistriage 2023 [46] 5.3M encounters Mistriage 32.2%; under-triage 3.3%; Black under-triage RR +18.5%
Race and language 2023 [48] 249,829 visits Less acute ESI for Black/Hispanic/Other despite more wRVUs
Queue jumping 2023 [49] 314,763 visits Unexplained queue jumps linked to worse flow outcomes
Under-triage outcomes 2026 [50] 5.3M encounters 8-minute delay for under-triaged high-acuity patients
High-risk conditions 2025 [51] 5,929 patients 36.7% under-triaged; delays in CT, ECG, troponin
Admission delay 2022 [52] 7.47M admissions Mortality rises after 5 h; 8% SMR increase at 6-8 h
Crowding Sweden 2023 [53] 168,900 visits No significant short-term mortality association
Boarding review 2020 [54] 12 studies 6 positive, 5 null for in-hospital mortality

8. Governance: What Current Rules Do and Do Not Address

8.1 United States: FDA and ONC

The US Food and Drug Administration (FDA) regulates software that meets the definition of a medical device. Its 2022 final guidance on Clinical Decision Support Software describes four criteria under section 520(o)(1)(E) of the Food, Drug, and Cosmetic Act for software functions that are not devices: (1) not intended to acquire, process, or analyze a medical image or a signal from an in vitro diagnostic device or a pattern or signal from a signal acquisition system; (2) intended for displaying, analyzing, or printing medical information about a patient or other medical information; (3) intended for supporting or providing recommendations to a health care professional about prevention, diagnosis, or treatment; and (4) intended to enable the health care professional to independently review the basis for the recommendations so that the professional does not rely primarily on the software [59]. The guidance is nonbinding, but it shapes how vendors describe intended use and how hospitals assess whether a triage tool is a device.

An LLM triage tool could fall on either side of this line. If it is marketed to clinicians as an advisory tool that displays the basis for its recommendation and supports independent review, it may be non-device CDS. If it is marketed as an autonomous triage system, or if it processes signals or images, it is more likely a device. The FDA has authorized almost 1,000 AI-enabled medical devices and has emphasized lifecycle management with recurrent local postmarket performance monitoring [60]. A 2025 review of AI medical devices for clinical decision support notes that evidence for real-world effectiveness, safety, and equity remains limited [61]. The regulatory question is not only whether the tool is a device, but whether the evidence submitted is adequate for the claim being made.

The Office of the National Coordinator for Health Information Technology (ONC) has a complementary lever: certification of health IT. The HTI-1 final rule adopted a Decision Support Interventions (DSI) certification criterion at 170.315(b)(11), with 13 source attributes for evidence-based DSIs and 31 source attributes for predictive DSIs. The attributes are intended to provide transparency about training data, intended use, performance, and limitations, and to help organizations determine whether a predictive DSI is fair, appropriate, valid, effective, and safe (FAVES) [62]. HTI-1 does not regulate the clinical content of a triage model, but it creates a documentation infrastructure that could be used to require condition-specific performance and bias information.

8.2 World Health Organization

The World Health Organization’s 2021 guidance, Ethics and Governance of Artificial Intelligence for Health, sets out core ethical principles for health AI: protecting human autonomy; promoting human well-being and safety; ensuring transparency, explainability, and intelligibility; fostering responsibility and accountability; ensuring inclusiveness and sustainability; and promoting solidarity [63]. Its 2025 guidance on large multi-modal models addresses generative AI specifically, including risks of bias, misinformation, and concentration of power, and calls for governance across the lifecycle [64]. These principles are relevant to stigmatizing input design because they connect model behavior to dignity, non-discrimination, and accountability. They do not, however, prescribe a technical standard for triage input formulation.

8.3 European Union: the AI Act

The EU Artificial Intelligence Act (Regulation (EU) 2024/1689) entered into force in August 2024. It takes a risk-based approach. AI systems that are safety components of products covered by Union harmonisation legislation listed in Annex I—including medical devices under the Medical Devices Regulation—are classified as high-risk under Article 6 [68]. High-risk systems are subject to requirements on risk management, data and data governance, technical documentation, record-keeping, transparency and provision of information to deployers, human oversight, and accuracy, robustness, and cybersecurity [68]. A 2024 analysis of the Act’s implications for healthcare concludes that its horizontal approach will require further guidelines to address the specific needs of the health sector [69]. A 2024 mapping identified 141 binding policies applicable to AI in healthcare across the EU and 10 European countries, with the AI Act as the overarching framework and other laws addressing data protection, technology safety, and human rights [70]. A 2026 analysis warns that proposed simplification could retain the high-risk classification of AI-enabled medical devices while removing most associated AI Act obligations, decoupling risk classification from the safeguards that give it meaning [71]. For present purposes, the key point is that a high-risk LLM triage system would be expected to have human oversight, robustness testing, and post-market monitoring—precisely the controls that the seed study’s findings suggest are necessary, but not a specific stigma-safe input standard.

8.4 Model cards, reporting guidelines, and dataset transparency

The Coalition for Health AI (CHAI) has developed an Applied Model Card that describes intended use, performance, risks, and governance details, and explicitly maps to HTI-1’s predictive DSI criteria and CHAI’s five principles of responsible AI: usefulness, fairness, safety, transparency, and security/privacy [72]. The card is a documentation format, not a certification. Its value depends on the quality and specificity of the information entered, and on whether hospitals actually use it in procurement.

Several reporting guidelines are relevant. TRIPOD-LLM provides a 19-item, 50-subitem checklist for studies using LLMs in biomedical applications, with 14 main items and 32 subitems applicable across designs; it emphasizes transparency, human oversight, and task-specific performance reporting [73]. DECIDE-AI provides a checklist for early-stage clinical evaluation of AI-based decision support systems, including human factors and safety [74]. CONSORT-AI and SPIRIT-AI extend trial reporting and protocol guidance to AI interventions, with 14 new items in CONSORT-AI [75,76]. STANDING Together provides 29 consensus recommendations for documenting health dataset composition and limitations, developed with more than 350 representatives from 58 countries and 194 Delphi participants from 25 countries [77]. These guidelines make it easier to report subgroup performance, dataset limitations, and human oversight. They do not make it mandatory to test stigmatizing formulations.

A systematic evidence map of 55 studies and registered trials found that 65.5% used human-AI collaborative designs, that diagnostic accuracy in randomized trials ranged from 0.65 to 0.88 versus typically ≥0.80 in non-randomized studies, and that reporting quality was suboptimal (mean 78.8% CONSORT-AI adherence), with critical omissions in data quality and performance-error handling [78]. A systematic review of reporting completeness for 461 records, including 68 trial reports, found mean completeness of 40% for TRIPOD-LLM and 59% for CONSORT-AI; the authors concluded that LLM trial reports prioritize narrative framing over the methodological detail needed for replication and governance [79]. The governance problem is therefore not only a lack of rules; it is a lack of consistent, comparable evidence even under existing rules.

8.5 The gap: no stigma-safe input standard

None of these instruments directly regulates the formulation of triage input. The FDA assesses intended use and claims; ONC requires source attributes; the AI Act requires risk management and human oversight; CHAI and TRIPOD-LLM require documentation. A vendor could comply with all of them while shipping a triage model that is never tested with neutral versus stigmatizing formulations of frequent ED use, psychiatric history, or housing status. The seed study fills an evidence gap, but it does not create a regulatory requirement. That is the policy opening: a condition-specific bias audit, analogous to a subgroup performance analysis, could be incorporated into existing transparency and risk-management frameworks without new legislation.

Table 7. Governance instruments and their relevance to stigmatizing triage input

Instrument Scope What it requires Relevance/gap
FDA CDS guidance 2022 [59] US device/non-device CDS Four criteria for non-device CDS; independent review Does not require stigma-specific testing
FDA AI device regulation 2025 [60,61] US medical devices Lifecycle oversight; premarket evidence; postmarket monitoring Could require condition-specific performance
ONC HTI-1 [62] Certified health IT 31 source attributes for predictive DSIs; FAVES Documentation infrastructure; no content standard
WHO 2021/2025 [63,64] Global ethics guidance Autonomy, safety, transparency, accountability, inclusiveness, solidarity Principles; no technical specification
EU AI Act [68-71] EU high-risk AI Risk management, data governance, human oversight, robustness High-risk classification; implementation uncertain
CHAI Applied Model Card [72] Health AI documentation Intended use, performance, risks, governance Voluntary; depends on specificity
TRIPOD-LLM [73] LLM research reporting 19 items/50 subitems; task-specific performance Reporting, not approval
DECIDE-AI [74] Early clinical evaluation Human factors, safety, clinical performance Relevant to stage 1 studies
CONSORT-AI/SPIRIT-AI [75,76] AI trials/protocols 14 new items; intervention description Trial reporting, not deployment
STANDING Together [77] Dataset transparency 29 recommendations for documentation Dataset-level, not input-level

9. Practical Implications: Designing for Stigma Safety

9.1 Treat the input layer as part of the intervention

The seed study’s central practical implication is that which data elements are shown to the model and how they are phrased is a safety-critical architectural decision [1]. That is a different framing from the usual AI safety question, which focuses on model weights, training data, or output guardrails. It suggests that the input layer—the triage capsule, the note summary, the structured fields—should be designed, validated, and monitored with the same rigor as the model.

An “information gating” approach would selectively exclude stigma-carrying fields while retaining physiologically relevant data [1]. The seed authors propose this but also note the trade-off: some excluded information may carry genuine clinical relevance, and the substance-use reversal shows that acute-risk signals can be protective [1]. A gated input that removes “17 visits in the past 12 months” but keeps “multiple chronic conditions” may reduce harmful reprioritization, but it may also remove information about utilization patterns that is relevant to care coordination. The net effect on both bias and accuracy requires prospective evaluation.

A practical gating policy would distinguish three categories:

  1. Physiologically and clinically required at triage: age, sex (when clinically relevant), arrival mode, vital signs, chief complaint, active medications, relevant past medical history, ESI level. These are the core of the capsule and should be retained.
  2. Potentially relevant but stigma-prone: psychiatric history, substance use, housing status, visit frequency, behavioral flags. These should be included only when there is a defined clinical reason, and when the formulation is standardized and person-first. The seed study suggests that neutral formulations can reduce harmful shifts, but the effect is not zero [1].
  3. Not relevant to near-term triage risk: race, ethnicity, language, insurance, and other demographic identifiers, unless the model is explicitly designed to detect and mitigate disparities or the clinical task requires them (e.g., language preference for communication). The seed study found no consistent harmful shifts for these single-line perturbations, but that is not a reason to include them by default [1].

9.2 Standardize formulation

The seed study’s neutral-versus-stigmatizing contrasts suggest that formulation matters independently of category. A triage input standard could require:

  • Person-first language: “patient with opioid use disorder on maintenance therapy” rather than “addict”; “patient experiencing homelessness” rather than “homeless”; “patient with multiple prior ED visits” rather than “frequent flyer.”
  • Separation of clinical facts from judgment: record visit counts, psychiatric holds, and behavioral events as discrete data fields with dates and context, rather than as narrative labels.
  • Avoidance of doubt markers: do not use quotation marks around patient-reported symptoms, “claims,” “insists,” or “allegedly” unless the epistemic status is clinically relevant and explicitly justified.
  • Context for utilization: if visit frequency is included, include the clinical reason (e.g., “17 visits in 12 months for sickle cell pain crises and heart failure”) rather than the count alone.
  • Behavioral flags as safety information, not character information: flags should be accompanied by the specific behavior, date, and safety plan, and should not be used as a general-purpose risk score [18].

These are not merely stylistic preferences. The Goddu study shows that a stigmatizing note changes trainee attitudes and pain management [17]; the triage-note study shows that stigmatizing words are present in 5.2% of pain triage notes and patterned by race and insurance [15]; the behavioral-flag study shows that flags are associated with longer waits and fewer tests [18]. A formulation standard is a plausible, low-cost intervention that could be implemented without waiting for model retraining.

9.3 Validate locally and condition-specifically

A hospital cannot rely on a vendor’s aggregate accuracy. It should require:

  • Local validation on its own ED data, with the same reference standard it will use in practice (ideally an outcome anchor, not only ESI).
  • Condition-specific bias audits: neutral versus stigmatizing formulations of frequent ED use, psychiatric history, substance use, housing status, and behavioral flags, analyzed by model, dataset, and subgroup.
  • Prompt robustness testing: multiple prompts, system messages, output schemas, and temperatures, with reporting of variance. The CTAS study’s finding that repeatability accounted for 21% of variation is a warning [8].
  • Model version pinning and change control: a model update is a new intervention, not a software patch. The seed study’s model heterogeneity implies that a version change could change bias behavior [1].
  • Subgroup performance: not only race and ethnicity, but also language, insurance, housing status, age, and disability, with confidence intervals and prespecified hypotheses.

9.4 Design the human-AI interface for appropriate reliance

The human-factors literature suggests that automation bias is mediated by trust, workload, task complexity, and time pressure [55]. A triage AI should be designed to support appropriate reliance, not maximal reliance:

  • Present the model’s recommendation as advisory, with the clinical basis and uncertainty visible.
  • Require the clinician to document their own assessment before or alongside the model output, to reduce anchoring.
  • Avoid displaying a single “priority score” without context; show the contributing factors and flag when key data are missing.
  • Monitor override rates and outcomes by condition and subgroup; a high override rate may indicate mistrust or poor usability, while a low override rate may indicate automation bias.
  • Train users on the specific failure modes identified in validation, including stigmatizing-input effects and sycophancy.

9.5 Procurement and governance

Procurement is where much of this becomes real. A hospital can require:

  • A CHAI Applied Model Card or equivalent, with intended use, performance, risks, and governance details [72].
  • TRIPOD-LLM-compliant reporting for the validation study [73].
  • Condition-specific bias audits, including neutral-versus-stigmatizing comparisons, with the exact prompts and formulations disclosed.
  • Dataset documentation consistent with STANDING Together recommendations [77].
  • A post-market monitoring plan that includes label-specific and subgroup-specific performance, not only aggregate accuracy.
  • Contractual rights to audit, update, and terminate if performance drifts.

These requirements are feasible under existing frameworks. They do not require new legislation, and they are consistent with FDA lifecycle oversight, ONC source attributes, EU AI Act risk management, and WHO principles [59-64,68-72].


10. Research Agenda

The seed study opens several questions that the field can answer with existing methods.

1. Does the effect translate into waiting-time and outcome harm? A prospective, stepped-wedge or cluster-randomized study could compare a gated versus ungated triage input in a real ED, with waiting time, time-to-provider, time-to-treatment, and deterioration as outcomes. The SHAKED study provides a template for stage 1 evaluation but was not designed to detect label-specific harm [12].

2. Which formulations are safest? The seed study tested one neutral and one stigmatizing formulation per concept. A factorial experiment could test multiple formulations of the same clinical fact (e.g., visit count alone, visit count with reasons, “multiple chronic conditions,” “high utilizer”) and estimate the independent contribution of framing versus content [1].

3. Do proprietary frontier models behave like open-weight models? The seed study deliberately chose open-weight models for on-premises deployment [1]. Proprietary models may have different alignment and different bias profiles. A multi-model audit using the same paired design would answer this.

4. Does the effect generalize beyond US academic centers? Community hospitals, safety-net hospitals, and non-English records may have different documentation cultures and different stigma patterns. Multi-site international replication is needed [1].

5. How do LLMs handle unstructured notes versus structured capsules? The seed study used structured summaries; real systems often process full notes. A comparison of structured versus unstructured input, with and without stigmatizing language, would test whether the single-line effect underestimates or overestimates real-world effects [1].

6. Can information gating reduce bias without reducing accuracy? This is the key engineering question. A randomized in silico experiment could compare gated, ungated, and selectively retained inputs on both harmful reprioritization and baseline accuracy, with prespecified non-inferiority margins.

7. How should stigma be measured in model outputs? The seed study measures harmful reprioritization. Other outcomes include recommendation intensity, diagnostic suspicion, mental-health referral, and pain treatment. A harmonized outcome set would make studies comparable.

8. What is the role of human oversight? The human-factors literature suggests that oversight can mitigate automation bias, but it can also be ineffective if the human is overworked or under-trained. Studies should measure the joint human-AI system, not the model alone [55-58].


11. Limitations

This review has several limitations.

First, it is a narrative review, not a systematic review or meta-analysis. The search strategy combined PubMed/EuropePMC queries, citation snowballing from the seed paper, and targeted retrieval of regulatory documents. It did not include a formal risk-of-bias assessment, dual screening, or quantitative pooling. The reference list is therefore a curated evidence base, not an exhaustive census.

Second, the seed study is a single experiment, albeit a large and methodologically ambitious one. Its findings have not been replicated by an independent group, and the published version differs from the preprint in the number and identity of models and the set of significant conditions (preprint DOI:10.21203/rs.3.rs-9249025/v1). This review uses the published version for all effect sizes [1]; where the preprint differs, the published version is authoritative. Independent replication is essential before the effect sizes are treated as stable.

Third, the review’s outcome-anchoring discussion relies on the seed study’s composite outcome. The composite is a reasonable proxy, but it is not a pure measure of avoidable harm, and the study’s “harmful reprioritization” is a model-relative measure, not a patient outcome.

Fourth, the clinical-record literature is heterogeneous in lexicon, note type, and population. Prevalence estimates range from 0.7% to 6.9% depending on condition and method [13-15]. The studies establish patterning and plausibility, not a universal prevalence.

Fifth, the LLM triage literature is dominated by English-language, high-income-country, academic-center studies using vignettes or ESI labels. The seed study itself is US-only and English-only. Generalization to community, safety-net, pediatric, and non-English settings is unknown [1].

Sixth, the regulatory landscape is moving. The EU AI Act’s implementation and proposed simplification, the FDA’s evolving guidance, and ONC’s certification program may change. The descriptions here reflect the sources available at the time of preparation [59-64,68-72].

Seventh, the review does not adjudicate the causal mechanism behind the seed finding. The hypothesis that stigma is learned from clinical text is plausible and supported by indirect evidence [13-24], but it is not proven. Alternative mechanisms include safety alignment that treats certain labels as risk factors, prompt-induced shifts in the model’s implicit task representation, or statistical associations between labels and acuity in the training distribution. Distinguishing these would require mechanistic interpretability and controlled training-data experiments.

Finally, the review’s practical recommendations are inferences from the evidence, not validated interventions. Information gating, formulation standards, and interface design are promising, but each could have unintended consequences. The substance-use reversal is a reminder that removing “stigmatizing” language can also remove clinically relevant acute-risk information [1]. Any intervention should be evaluated on both bias and accuracy, with prospective monitoring.


12. Conclusion

The seed study makes a narrow but consequential claim: in a controlled, outcome-anchored experiment, adding a single line of stigmatizing information to an LLM’s triage input changed which patient the model said should be seen first, and the effect was largest for stigmatizing formulations of frequent ED use and psychiatric history [1]. The finding was reproduced across three open-weight models and two academic ED datasets within the same study, and it was formulation-sensitive: neutral descriptions of the same broad concepts produced smaller harmful shifts [1]. Race, language, and insurance did not show consistent harmful shifts under the tested single-line conditions, but that null result is not a clean bill of health.

The broader literature explains why the finding is credible and why it matters. Clinical notes contain stigmatizing language that is patterned by race, insurance, and housing status [13-20]. That language can change the attitudes and pain management decisions of subsequent clinicians [17]. It can be copied forward and embedded in the text corpora used to train clinical language models [21-25]. LLM outputs are sensitive to framing, anchoring, sycophancy, and demographic cues [30-42]. Human triage is itself unreliable and inequitable, with under-triage patterned by race and delays associated with mortality [43-54]. Governance requires transparency, risk management, and reporting, but it does not yet require a stigma-safe input standard [59-64,68-79].

The practical conclusion is not that LLM triage should be abandoned. It is that the input layer must be treated as part of the clinical intervention. Which fields are shown, how they are phrased, which model is used, how the output is presented, and how the system is monitored are safety decisions. The seed study provides a template for the kind of audit that should precede deployment: paired, outcome-anchored, condition-specific, and model-specific. The next step is to move from retrospective audit to prospective evaluation, and from documentation to standards. A triage system that is accurate on average but systematically deprioritizes the patients who most need attention is not a safe triage system. The evidence now available is sufficient to make that a design requirement, not a research aspiration.


References

  1. Jarrett P, Yun P, Ohuabunwa E, Agboh D, Courtney DM, Jamieson A. Outcome-grounded effect of clinically stigmatizing information on large language model emergency triage prioritization. npj Digit Med. 2026. doi:10.1038/s41746-026-03270-5.
  2. Williams CYK, Zack T, Miao BY, Sushil M, Wang M, Kornblith AE, et al. Use of a Large Language Model to Assess Clinical Acuity of Adults in the Emergency Department. JAMA Netw Open. 2024;7(5):e248895. doi:10.1001/jamanetworkopen.2024.8895. PMID 38713466.
  3. Cui L, Wang M, Xu Y, Wei W, Zhao X. Diagnostic accuracy of large language models for emergency department triage: a systematic review and meta-analysis. BMC Emerg Med. 2026;26(1). doi:10.1186/s12873-026-01639-z. PMID 42298434.
  4. Gao S, Yu M, Zheng Y, Zhang M, Yang Z, Zhang J. Accuracy of the large language model ChatGPT in adult emergency department triage: a systematic review and meta-analysis. BMC Emerg Med. 2026;26(1). doi:10.1186/s12873-026-01521-y. PMID 41840524.
  5. Masanneck L, Schmidt L, Seifert A, Kölsche T, Huntemann N, Jansen R, et al. Triage Performance Across Large Language Models, ChatGPT, and Untrained Doctors in Emergency Medicine: Comparative Study. J Med Internet Res. 2024;26:e53297. doi:10.2196/53297. PMID 38875696.
  6. Arslan B, Nuhoglu C, Satici MO, Altinbilek E. Evaluating LLM-based generative AI tools in emergency triage: A comparative study of ChatGPT Plus, Copilot Pro, and triage nurses. Am J Emerg Med. 2025;89:174-181. doi:10.1016/j.ajem.2024.12.024. PMID 39731895.
  7. Kopka M, von Kalckreuth N, Feufel MA. Accuracy of online symptom assessment applications, large language models, and laypeople for self-triage decisions. NPJ Digit Med. 2025;8(1):178. doi:10.1038/s41746-025-01566-6. PMID 40133390.
  8. Franc JM, Cheng L, Hart A, Hata R, Hertelendy A. Repeatability, reproducibility, and diagnostic accuracy of a commercial large language model (ChatGPT) to perform emergency department triage using the Canadian triage and acuity scale. CJEM. 2024;26(1):40-46. doi:10.1007/s43678-023-00616-w. PMID 38206515.
  9. Williams CYK, Miao BY, Kornblith AE, Butte AJ. Evaluating the use of large language models to provide clinical recommendations in the Emergency Department. Nat Commun. 2024;15(1):8236. doi:10.1038/s41467-024-52415-1. PMID 39379357.
  10. Chen M, Wu Y, Ma J, Jia X, Gao C, Zhao F, et al. Independent and collaborative performance of large language models and healthcare professionals in diagnosis and triage. NPJ Digit Med. 2026;9(1). doi:10.1038/s41746-026-02409-8. PMID 41652180.
  11. Çalışkan YK, Başak F, Erdem O. When algorithms triage trauma: Diagnostic accuracy, undertriage risk, and prompt fragility in frontier large language models. Surgery. 2026;196:110319. doi:10.1016/j.surg.2026.110319. PMID 42214316.
  12. Leibovitch L, Ahituv A, Gorenshtein A, Aran D, Sorka M, Miron K, et al. Prospective evaluation of a large language model clinical decision support system in the emergency department. Nat Med. 2026;32(10):3741-3747. doi:10.1038/s41591-026-04601-5. PMID 42618632.
  13. Sun M, Oliwa T, Peek ME, Tung EL. Negative Patient Descriptors: Documenting Racial Bias In The Electronic Health Record. Health Aff (Millwood). 2022;41(2):203-211. doi:10.1377/hlthaff.2021.01423. PMID 35044842.
  14. Himmelstein G, Bates D, Zhou L. Examination of Stigmatizing Language in the Electronic Health Record. JAMA Netw Open. 2022;5(1):e2144967. doi:10.1001/jamanetworkopen.2021.44967. PMID 35084481.
  15. Fung CM, Kehne A, Macleod C, Logue C, Lagisetty P. Prevalence of Stigmatizing Language in the Triage Documentation of Patients Presenting to the Emergency Department with Pain. J Emerg Med. 2025;73:24-33. doi:10.1016/j.jemermed.2024.12.012. PMID 40320313.
  16. Beach MC, Saha S, Park J, Taylor J, Drew P, Plank E, et al. Testimonial Injustice: Linguistic Bias in the Medical Records of Black Patients and Women. J Gen Intern Med. 2021;36(6):1708-1714. doi:10.1007/s11606-021-06682-z. PMID 33754318.
  17. P Goddu A, O’Conor KJ, Lanzkron S, Saheed MO, Saha S, Peek ME, et al. Do Words Matter? Stigmatizing Language and the Transmission of Bias in the Medical Record. J Gen Intern Med. 2018;33(5):685-691. doi:10.1007/s11606-017-4289-2. PMID 29374357.
  18. Agarwal AK, Seeburger E, O’Neill G, Nwakanma CC, Marsh LE, Soltany KA, et al. Prevalence of Behavioral Flags in the Electronic Health Record Among Black and White Patients Visiting the Emergency Department. JAMA Netw Open. 2023;6(1):e2251734. doi:10.1001/jamanetworkopen.2022.51734. PMID 36656576.
  19. Lauricella M, Nene RV, Coyne CJ, Fernandez JA. Implicit bias in the patient descriptor “homeless” and its association with emergency department opioid administration and disposition. Am J Emerg Med. 2025;89:135-138. doi:10.1016/j.ajem.2024.12.034. PMID 39721284.
  20. Wesevich A, Vangelatos A, Sun M, Tung EL, Peek ME. Negative Descriptors of Patients With Sickle Cell Disease in the Electronic Health Record. JAMA Netw Open. 2026;9(4):e266458. doi:10.1001/jamanetworkopen.2026.6458. PMID 41973422.
  21. Cobert J, Mills H, Lee A, Gologorskaya O, Espejo E, Jeon SY, et al. Measuring Implicit Bias in ICU Notes Using Word-Embedding Neural Network Models. Chest. 2024;165(6):1481-1490. doi:10.1016/j.chest.2023.12.031. PMID 38199323.
  22. Wissel BD, Greiner HM, Glauser TA, Mangano FT, Santel D, Pestian JP, et al. Investigation of bias in an epilepsy machine learning algorithm trained on physician notes. Epilepsia. 2019;60(9):e93-e98. doi:10.1111/epi.16320. PMID 31441044.
  23. Xie K, Ojemann WKS, Gallagher RS, Shinohara RT, Lucas A, Hill CE, et al. Disparities in seizure outcomes revealed by large language models. J Am Med Inform Assoc. 2024;31(6):1348-1355. doi:10.1093/jamia/ocae047. PMID 38481027.
  24. Yu Z, Peng C, Yang X, Dang C, Adekkanattu P, Gopal Patra B, et al. Identifying social determinants of health from clinical narratives: A study of performance, documentation ratio, and potential bias. J Biomed Inform. 2024;153:104642. doi:10.1016/j.jbi.2024.104642. PMID 38621641.
  25. Corby S, Ash JS, Jungbauer RM, Scholl G, Florig S, Mohan V, et al. Copy Tools in the Electronic Health Record: Perceptions, Implications, and Future Directions. JMIR Med Inform. 2025;13:e78502. doi:10.2196/78502. PMID 41418294.
  26. LaCalle E, Rabin E. Frequent users of emergency departments: the myths, the data, and the policy implications. Ann Emerg Med. 2010;56(1):42-8. doi:10.1016/j.annemergmed.2010.01.032. PMID 20346540.
  27. Moe J, Kirkland S, Ospina MB, Campbell S, Long R, Davidson A, et al. Mortality, admission rates and outpatient use among frequent users of emergency departments: a systematic review. Emerg Med J. 2016;33(3):230-6. doi:10.1136/emermed-2014-204496. PMID 25953837.
  28. Shefer G, Henderson C, Howard LM, Murray J, Thornicroft G. Diagnostic overshadowing and other challenges involved in the diagnostic process of patients with mental illness who present in emergency departments with physical symptoms–a qualitative study. PLoS One. 2014;9(11):e111682. doi:10.1371/journal.pone.0111682. PMID 25369130.
  29. van Boekel LC, Brouwers EP, van Weeghel J, Garretsen HF. Stigma among health professionals towards patients with substance use disorders and its consequences for healthcare delivery: systematic review. Drug Alcohol Depend. 2013;131(1-2):23-35. doi:10.1016/j.drugalcdep.2013.02.018. PMID 23490450.
  30. Sorin V, Collins JD, Bratt AK, Kusmirek JE, Mugu VK, Kline TL, et al. Evaluating prompt and data perturbation sensitivity in large language models for radiology reports classification. JAMIA Open. 2025;8(4):ooaf073. doi:10.1093/jamiaopen/ooaf073. PMID 40799928.
  31. Dietrich NT, Patel D, Bellissimo J, Loh CT, Tyrrell PN. Cognitively Biased Prompt Effects on Large Language Model Accuracy for Radiology Board-style Examination Questions. Radiol Artif Intell. 2026;8(3):e250585. doi:10.1148/ryai.250585. PMID 41983923.
  32. Gökmen MY, Maden M, Zengin O. Prompt Framing Modulates Safety in Shoulder and Elbow Red-Flag Vignettes: A Large Language Model Study. Diagnostics (Basel). 2026;16(10). doi:10.3390/diagnostics16101439. PMID 42196805.
  33. Germani F, Spitale G. Source framing triggers systematic bias in large language models. Sci Adv. 2025;11(45):eadz2924. doi:10.1126/sciadv.adz2924. PMID 41202130.
  34. Sheppert A, Shen C, Geissal E, Riley C, Riley S. Large language models exhibit greater diagnostic anchoring than physicians in a forced-choice vignette study. Int J Med Inform. 2026;219:106550. doi:10.1016/j.ijmedinf.2026.106550. PMID 42335861.
  35. Samsel K, Dharma C, Rezaei AHM, Bahakim A, Ravikumar K, Bhat V, et al. Too agreeable to be accurate? Sycophancy and diagnostic instability of large language models in medical diagnosis. Artif Intell Med. 2026;182:103516. doi:10.1016/j.artmed.2026.103516. PMID 42721584.
  36. Chen S, Gao M, Sasse K, Hartvigsen T, Anthony B, Fan L, et al. When helpfulness backfires: LLMs and the risk of false medical information due to sycophantic behavior. NPJ Digit Med. 2025;8(1):605. doi:10.1038/s41746-025-02008-z. PMID 41107408.
  37. Cheng M, Lee C, Khadpe P, Yu S, Han D, Jurafsky D. Sycophantic AI decreases prosocial intentions and promotes dependence. Science. 2026;391(6792):eaec8352. doi:10.1126/science.aec8352. PMID 41886588.
  38. Omar M, Soffer S, Agbareia R, Bragazzi NL, Apakama DU, Horowitz CR, et al. Sociodemographic biases in medical decision making by large language models. Nat Med. 2025;31(6):1873-1881. doi:10.1038/s41591-025-03626-6. PMID 40195448.
  39. Saravi B, Winkler I, Singh DD, Lommen J, Wilkat M, Schorn L, et al. Demographic prompt cues shift clinical recommendations in multimodal large language models: a multi-model audit of 25,380 dental radiograph assessments. BMC Oral Health. 2026;26(1). doi:10.1186/s12903-026-08644-5. PMID 42210248.
  40. Hanna JJ, Wakene AD, Johnson AO, Lehmann CU, Medford RJ. Assessing Racial and Ethnic Bias in Text Generation by Large Language Models for Health Care-Related Tasks: Cross-Sectional Study. J Med Internet Res. 2025;27:e57257. doi:10.2196/57257. PMID 40080818.
  41. Suri G, Slater LR, Ziaee A, Nguyen M. Do large language models show decision heuristics similar to humans? A case study using GPT-3.5. J Exp Psychol Gen. 2024;153(4):1066-1075. doi:10.1037/xge0001547. PMID 38330366.
  42. Jarrett PC, Hill J, Howell M, Grabow Moore K, Thoppil JJ, Vargas Ortiz L, et al. Piloting Temperature-Driven Variability in Emergency Diagnostic Accuracy Using a Leading Large Language Model. Cureus. 2025;17(10):e94476. doi:10.7759/cureus.94476. PMID 41235003.
  43. Tanabe P, Gimbel R, Yarnold PR, Kyriacou DN, Adams JG. Reliability and validity of scores on The Emergency Severity Index version 3. Acad Emerg Med. 2004;11(1):59-65. PMID 14709429.
  44. Mistry B, Stewart De Ramirez S, Kelen G, Schmitz PSK, Balhara KS, Levin S, et al. Accuracy and Reliability of Emergency Department Triage Using the Emergency Severity Index: An International Multicenter Assessment. Ann Emerg Med. 2018;71(5):581-587.e3. doi:10.1016/j.annemergmed.2017.09.036. PMID 29174836.
  45. Hinson JS, Martinez DA, Cabral S, George K, Whalen M, Hansoti B, et al. Triage Performance in Emergency Medicine: A Systematic Review. Ann Emerg Med. 2019;74(1):140-152. doi:10.1016/j.annemergmed.2018.09.022. PMID 30470513.
  46. Sax DR, Warton EM, Mark DG, Vinson DR, Kene MV, Ballard DW, et al. Evaluation of the Emergency Severity Index in US Emergency Departments for the Rate of Mistriage. JAMA Netw Open. 2023;6(3):e233404. doi:10.1001/jamanetworkopen.2023.3404. PMID 36930151.
  47. Suamchaiyaphum K, Jones AR, Markaki A. Triage Accuracy of Emergency Nurses: An Evidence-Based Review. J Emerg Nurs. 2024;50(1):44-54. doi:10.1016/j.jen.2023.10.001. PMID 37930287.
  48. Joseph JW, Kennedy M, Landry AM, Marsh RH, Baymon DE, Im DE, et al. Race and Ethnicity and Primary Language in Emergency Department Triage. JAMA Netw Open. 2023;6(10):e2337557. doi:10.1001/jamanetworkopen.2023.37557. PMID 37824142.
  49. Sangal RB, Su H, Khidir H, Parwani V, Liebhardt B, Pinker EJ, et al. Sociodemographic Disparities in Queue Jumping for Emergency Department Care. JAMA Netw Open. 2023;6(7):e2326338. doi:10.1001/jamanetworkopen.2023.26338. PMID 37505495.
  50. Sax DR, Warton EM, Mark DG, Vitale-McDowell TJ, DiLena DD, Rauchwerger AS, et al. Association Between Emergency Department Undertriage or Overtriage With Timeliness of Care and Patient Outcomes. Ann Emerg Med. 2026;87(6):723-734. doi:10.1016/j.annemergmed.2025.11.018. PMID 41721807.
  51. Sax DR, Warton EM, Mark DG, Reed ME. Emergency Department Triage Accuracy and Delays in Care for High-Risk Conditions. JAMA Netw Open. 2025;8(5):e258498. doi:10.1001/jamanetworkopen.2025.8498. PMID 40314952.
  52. Jones S, Moulton C, Swift S, Molyneux P, Black S, Mason N, et al. Association between delays to patient admission from the emergency department and all-cause 30-day mortality. Emerg Med J. 2022;39(3):168-173. doi:10.1136/emermed-2021-211572. PMID 35042695.
  53. Wretborn J, Wilhelms DB, Ekelund U. Emergency department crowding and mortality: an observational multicenter study in Sweden. Front Public Health. 2023;11:1198188. doi:10.3389/fpubh.2023.1198188. PMID 37559736.
  54. Boudi Z, Lauque D, Alsabri M, Östlundh L, Oneyji C, Khalemsky A, et al. Association between boarding in the emergency department and in-hospital mortality: A systematic review. PLoS One. 2020;15(4):e0231253. doi:10.1371/journal.pone.0231253. PMID 32294111.
  55. Goddard K, Roudsari A, Wyatt JC. Automation bias: a systematic review of frequency, effect mediators, and mitigators. J Am Med Inform Assoc. 2012;19(1):121-7. doi:10.1136/amiajnl-2011-000089. PMID 21685142.
  56. Khera R, Simon MA, Ross JS. Automation Bias and Assistive AI: Risk of Harm From AI-Driven Clinical Decision Support. JAMA. 2023;330(23):2255-2257. doi:10.1001/jama.2023.22557. PMID 38112824.
  57. Goh E, Gallo R, Hom J, Strong E, Weng Y, Kerman H, et al. Large Language Model Influence on Diagnostic Reasoning: A Randomized Clinical Trial. JAMA Netw Open. 2024;7(10):e2440969. doi:10.1001/jamanetworkopen.2024.40969. PMID 39466245.
  58. Goh E, Gallo RJ, Strong E, Weng Y, Kerman H, Freed JA, et al. GPT-4 assistance for improvement of physician performance on patient care tasks: a randomized controlled trial. Nat Med. 2025;31(4):1233-1238. doi:10.1038/s41591-024-03456-y. PMID 39910272.
  59. US Food and Drug Administration. Clinical Decision Support Software: Guidance for Industry and Food and Drug Administration Staff. Silver Spring, MD: FDA; 2022 [cited 2026 Oct 19]. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/clinical-decision-support-software
  60. Warraich HJ, Tazbaz T, Califf RM. FDA Perspective on the Regulation of Artificial Intelligence in Health Care and Biomedicine. JAMA. 2025;333(3):241-247. doi:10.1001/jama.2024.21451. PMID 39405330.
  61. Weissman GE. Evaluation and Regulation of Artificial Intelligence Medical Devices for Clinical Decision Support. Annu Rev Biomed Data Sci. 2025;8(1):81-99. doi:10.1146/annurev-biodatasci-103123-095824. PMID 39971383.
  62. Office of the National Coordinator for Health Information Technology. Decision Support Interventions (DSI) Fact Sheet: HTI-1 Final Rule. Washington, DC: ONC; 2023 [cited 2026 Oct 19]. https://www.healthit.gov/wp-content/uploads/2023/12/HTI-1_DSI_fact-sheet_508.pdf
  63. World Health Organization. Ethics and Governance of Artificial Intelligence for Health: WHO Guidance. Geneva: WHO; 2021 [cited 2026 Oct 19]. https://www.who.int/publications/i/item/9789240029200
  64. World Health Organization. Ethics and Governance of Artificial Intelligence for Health: Guidance on Large Multi-Modal Models. Geneva: WHO; 2025 [cited 2026 Oct 19]. https://www.who.int/publications/i/item/9789240084759
  65. Ramaswamy A, Tyagi A, Hugo H, Jiang J, Jayaraman P, Jangda M, et al. ChatGPT Health performance in a structured test of triage recommendations. Nat Med. 2026;32(5):1671-1675. doi:10.1038/s41591-026-04297-7. PMID 41731097.
  66. Colakca C, Ergın M, Ozensoy HS, Sener A, Guru S, Ozhasenekler A. Emergency department triaging using ChatGPT based on emergency severity index principles: a cross-sectional study. Sci Rep. 2024;14(1):22106. doi:10.1038/s41598-024-73229-7. PMID 39333599.
  67. Wang C, Wang F, Li S, Ren QW, Tan X, Fu Y, et al. Patient Triage and Guidance in Emergency Departments Using Large Language Models: Multimetric Study. J Med Internet Res. 2025;27:e71613. doi:10.2196/71613. PMID 40374171.
  68. European Parliament and Council of the European Union. Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act). Official Journal of the European Union; 2024 [cited 2026 Oct 19]. https://eur-lex.europa.eu/eli/reg/2024/1689/oj
  69. van Kolfschooten H, van Oirschot J. The EU Artificial Intelligence Act (2024): Implications for healthcare. Health Policy. 2024;149:105152. doi:10.1016/j.healthpol.2024.105152. PMID 39244818.
  70. Schmidt J, Schutte NM, Buttigieg S, Novillo-Ortiz D, Sutherland E, Anderson M, et al. Mapping the regulatory landscape for artificial intelligence in health within the European Union. NPJ Digit Med. 2024;7(1):229. doi:10.1038/s41746-024-01221-6. PMID 39191937.
  71. Van Kolfschooten H, Solaiman B, Onitiu D. High-risk without safeguards? The EU AI Act and the push for deregulation of medical AI. BMJ Digit Health Ai. 2026;2(1):e000184. doi:10.1136/bmjdh-2026-000184. PMID 42825232.
  72. Coalition for Health AI. Applied Model Card. CHAI Registry [cited 2026 Oct 19]. https://registry.chai.org/applied-model-card
  73. Gallifant J, Afshar M, Ameen S, Aphinyanaphongs Y, Chen S, Cacciamani G, et al. The TRIPOD-LLM reporting guideline for studies using large language models. Nat Med. 2025;31(1):60-69. doi:10.1038/s41591-024-03425-5. PMID 39779929.
  74. Vasey B, Nagendran M, Campbell B, Clifton DA, Collins GS, Denaxas S, et al. Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. Nat Med. 2022;28(5):924-933. doi:10.1038/s41591-022-01772-9. PMID 35585198.
  75. Liu X, Cruz Rivera S, Moher D, Calvert MJ, Denniston AK. Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension. Nat Med. 2020;26(9):1364-1374. doi:10.1038/s41591-020-1034-x. PMID 32908283.
  76. Cruz Rivera S, Liu X, Chan AW, Denniston AK, Calvert MJ. Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension. Nat Med. 2020;26(9):1351-1363. doi:10.1038/s41591-020-1037-7. PMID 32908284.
  77. Alderman JE, Palmer J, Laws E, McCradden MD, Ordish J, Ghassemi M, et al. Tackling algorithmic bias and promoting transparency in health datasets: the STANDING Together consensus recommendations. Lancet Digit Health. 2025;7(1):e64-e88. doi:10.1016/S2589-7500(24)00224-3. PMID 39701919.
  78. He Z, Yang L, Liang Z, Li X, Du J. Clinical outcomes and reporting quality of large language model interventions in practice: a systematic evidence map. NPJ Digit Med. 2026;9(1). doi:10.1038/s41746-026-02837-6. PMID 42230743.
  79. Chen D, Blayne A, Ning T, Chattha I, Afshar M, Ong JCL, et al. Reporting completeness of clinical trials evaluating large language model interventions in medicine: systematic review. BMJ. 2026;395:e100855. doi:10.1136/bmj-2026-100855. PMID 42869550.
  80. Omiye JA, Lester JC, Spichak S, Rotemberg V, Daneshjou R. Large language models propagate race-based medicine. NPJ Digit Med. 2023;6(1):195. doi:10.1038/s41746-023-00939-z. PMID 37864012.
  81. Zack T, Lehman E, Suzgun M, Rodriguez JA, Celi LA, Gichoya J, et al. Assessing the potential of GPT-4 to perpetuate racial and gender biases in health care: a model evaluation study. Lancet Digit Health. 2024;6(1):e12-e22. doi:10.1016/S2589-7500(23)00225-X. PMID 38123252.
  82. Obermeyer Z, Powers B, Vogeli C, Mullainathan S. Dissecting racial bias in an algorithm used to manage the health of populations. Science. 2019;366(6464):447-453. doi:10.1126/science.aax2342. PMID 31649194.
  83. Valbuena VSM, Seelye S, Sjoding MW, Valley TS, Dickson RP, Gay SE, et al. Racial bias and reproducibility in pulse oximetry among medical and surgical inpatients in general care in the Veterans Health Administration 2013-19: multicenter, retrospective cohort study. BMJ. 2022;378:e069775. doi:10.1136/bmj-2021-069775. PMID 35793817.
  84. Inker LA, Eneanya ND, Coresh J, Tighiouart H, Wang D, Sang Y, et al. New Creatinine- and Cystatin C-Based Equations to Estimate GFR without Race. N Engl J Med. 2021;385(19):1737-1749. doi:10.1056/NEJMoa2102953. PMID 34554658.
  85. Park J, Saha S, Chee B, Taylor J, Beach MC. Physician Use of Stigmatizing Language in Patient Medical Records. JAMA Netw Open. 2021;4(7):e2117052. doi:10.1001/jamanetworkopen.2021.17052. PMID 34259849.
  86. Caso Vega LE, Figueroa Feliciano VE. The Language of Bias: Labeling and Its Impact on Emergency Department Patients. J Emerg Nurs. 2026;52(2):301-305. doi:10.1016/j.jen.2025.12.004. PMID 41504674.
  87. Raita Y, Goto T, Faridi MK, Brown DFM, Camargo CA Jr, Hasegawa K. Emergency department triage prediction of clinical outcomes using machine learning models. Crit Care. 2019;23(1):64. doi:10.1186/s13054-019-2351-7. PMID 30795786.
  88. Porto BM. Improving triage performance in emergency departments using machine learning and natural language processing: a systematic review. BMC Emerg Med. 2024;24(1):219. doi:10.1186/s12873-024-01135-2. PMID 39558255.
  89. Singhal K, Azizi S, Tu T, Mahdavi SS, Wei J, Chung HW, et al. Large language models encode clinical knowledge. Nature. 2023;620(7972):172-180. doi:10.1038/s41586-023-06291-2. PMID 37438534.
  90. Thirunavukarasu AJ, Ting DSJ, Elangovan K, Gutierrez L, Tan TF, Ting DSW. Large language models in medicine. Nat Med. 2023;29(8):1930-1940. doi:10.1038/s41591-023-02448-8. PMID 37460753.
  91. McDuff D, Schaekermann M, Tu T, Palepu A, Wang A, Garrison J, et al. Towards accurate differential diagnosis with large language models. Nature. 2025;642(8067):451-457. doi:10.1038/s41586-025-08869-4. PMID 40205049.
  92. Bedi S, Cui H, Fuentes M, Unell A, Wornow M, Banda JM, et al. Holistic evaluation of large language models for medical tasks with MedHELM. Nat Med. 2026;32(3):943-951. doi:10.1038/s41591-025-04151-2. PMID 41559415.
  93. Shool S, Adimi S, Saboori Amleshi R, Bitaraf E, Golpira R, Tara M. A systematic review of large language model (LLM) evaluations in clinical medicine. BMC Med Inform Decis Mak. 2025;25(1):117. doi:10.1186/s12911-025-02954-4. PMID 40055694.
  94. Chang Y, Hsieh MH, Ju PC, Liu YC, Chang CC. Clinical Plausibility in Large Language Model Robustness Testing for Medicine: A Scoping Review. J Med Syst. 2026;50(1). doi:10.1007/s10916-026-02405-1. PMID 42113381.

Leave a Comment