> Paper: A pragmatic classification of AI incident trajectories > Authors: Isaak Mengesha, Branwen Owen, Charlie Collins, Tina Wong, Simon Mylius, Peter Slattery, Sean McGregor > arXiv: 2604.21412 | 2026-04-28
---
1. The Illusion That "AI Incidents Are Exploding"
Open the news and you'll see:
- "AI self-driving car kills a pedestrian"
- "AI hiring system discriminates against women"
- "AI chatbot encourages a user toward suicide"
- "AI-generated fake news influences elections"
- The per-exposure harm rate is rising
- This is the most dangerous signal — the AI systems themselves are becoming more dangerous
- Possible causes: increasing system complexity, regulatory lag, accumulating technical debt
- The per-exposure harm rate stays constant
- Absolute harm counts increase, but only because deployment scale expands
- Worth attention, but not an urgent crisis
- The per-exposure harm rate is falling
- Good news — safety measures are working
- But confirm: is it genuine improvement, or underreporting?
- A sudden surge in the harm rate, then a fall back
- Usually tied to a new technology release, a major update, or a specific event
- Requires rapid response and post-hoc analysis
- Growing: urgent intervention — pause deployments, mandatory safety reviews
- Stable: continuous monitoring and preventive investment
- Declining: learn and spread best practices
- Spike: rapid-response mechanisms and post-hoc analysis capability
- Underreporting: many incidents are never publicly reported
- Reporting bias: high-profile incidents get covered; everyday small harms are ignored
- Vague definitions: what counts as an "AI incident"? System failure? User misuse? Design flaw?
- Attribution difficulty: did the AI directly cause the harm, or merely amplify an existing problem?
It looks like AI incidents are growing explosively. But the question is: is that true?
The paper points out a key problem: statistics from public AI incident databases conflate three distinct factors: 1. Changes in reporting propensity: things once unreported are now reported 2. Growth in deployment scale: the more AI is used, the more incidents there naturally are 3. Per-exposure harm rate: the actual probability that each AI system causes harm
Without separating these three factors, we cannot tell whether AI is genuinely becoming more dangerous.
---
2. Four Types of Incident Trajectories
The research proposes a framework classifying AI incidents into four "trajectories":
1. Growing
2. Stable
3. Declining
4. Spike
---
3. Why Does Classification Matter?
Different trajectories require completely different policy responses:
Misreading a growing trajectory as stable means missing the best window for intervention. Misreading a spike as growth means overreacting and stifling innovation.
---
4. The Data Problem
Current AI incident data faces serious challenges:
---
5. A Feynman-Style Judgment: Separating Signal from Noise
Feynman emphasized in discussing experimental design:
> "You must be very careful not to fool yourself — and you are the easiest person to fool."
AI incident statistics are exactly like this. We see absolute harm counts rising and assume AI is getting more dangerous. But that may be "noise" (deployment expansion, more reporting) rather than "signal" (rising per-unit risk).
The scientific method: control variables, separate the effects, and find the true causal signal.
---
6. Takeaways
If you're evaluating the safety of an AI system, ask yourself:
1. "Am I looking at absolute incident counts, or incident rates per unit of exposure?" 2. "Is the data affected by changes in reporting propensity?" 3. "Is the definition of an incident consistent?" 4. "Does my conclusion account for changes in deployment scale?"
AI safety policy cannot rest on intuition. It must rest on rigorous data analysis.
This research offers a starting point: don't just count incidents — understand their "trajectory" — rising, stable, declining, or spiking?
Only by answering that question can we know whether to hit the brakes or keep accelerating.