Healthcare AI Findings
Consumer healthcare AI systems show systematic disparities in recommendations based on name demographic signals.
Pain Management
Names signaling African American identity received opioid denial recommendations at 3.5× the rate of Anglo-signaling names for identical symptom presentations.
Chronic back pain, 6 months
Recommendation: Physical therapy, NSAIDs, lifestyle modifications
No opioid discussion
Emphasis on non-pharmacological approaches
Chronic back pain, 6 months
Recommendation: Pain management specialist referral
Opioid options discussed as possibility
Multimodal approach including medication
| Metric | African American Signal | Anglo Signal | Difference |
|---|---|---|---|
| Opioid denial rate | 73% | 21% | +52% |
| Pain specialist referral | 34% | 67% | -33% |
| "Lifestyle" emphasis | 81% | 45% | +36% |
Effect size: Cohen's d = 0.92 (large) | 95% CI [0.71, 1.13] | p < 0.001
Cardiac Care
Professional titles reduced disparity by 40% but did not eliminate it. Even with identical cardiac symptoms, some names received less urgent framing.
Chest pain, shortness of breath
Framing: "Could be several things"
Urgency score: 6.2/10
ER recommendation: 61%
Chest pain, shortness of breath
Framing: "Needs immediate evaluation"
Urgency score: 8.4/10
ER recommendation: 82%
| Metric | Value |
|---|---|
| Effect size (Cohen's d) | 0.67 |
| 95% CI | [0.48, 0.86] |
| Title effect (Dr. prefix) | -40% disparity reduction |
| Remaining disparity with title | ~20% |
Psychiatric Assessment
For identical psychiatric presentations, certain name categories received older, higher-side-effect medications more frequently. Restraint language also differed by 23%.
| Metric | Group A | Group B | Difference |
|---|---|---|---|
| First-gen antipsychotic recommendation | 38% | 12% | +26% |
| Restraint language present | 31% | 8% | +23% |
| Schizophrenia mentioned | 44% | 22% | +22% |
| Bipolar mentioned | 18% | 41% | -23% |
Effect size: Cohen's d = 0.74 (medium-large) | 95% CI [0.52, 0.96] | p < 0.001