Imagine you see a post on social media: "The Earth is flat, and NASA has been lying to us."
Is this a conspiracy theory? If the poster is serious, yes. But what if they are mocking conspiracy theorists? What if they posted it in a satire discussion group, along with a flat-Earth meme?
The same surface content can express agreement, legitimate concern, criticism, irony, or mockery. Text content alone cannot distinguish them—this is the core challenge of conspiracy detection.
Klein, Shapira, and Hirsch from the Hebrew University, in *Agentic Detection of Online Conspiracies*, propose a solution: don't just analyze the text—infer the speaker's intent.
The Return of Speech Act Theory
The paper's theoretical foundation is Austin's 1962 speech act theory: a sentence doesn't just describe facts, it *does* something—agreeing, questioning, mocking, persuading. The same proposition "vaccines are harmful" can be:
- Agreement: the speaker genuinely believes vaccines are harmful
- Legitimate concern: evidence-based doubts about vaccine safety
- Criticism: opposing anti-vaccine narratives
- Irony: using sarcasm to mock anti-vaxxers
- Mockery: imitating conspiracy talk to make fun of conspiracy theorists
The Agent Framework: Tool-Augmented Intent Inference
The authors built an agent framework based on LangGraph. The core idea: don't just read the text—query the context on demand. The agent can call three types of tools as needed:
User history tool: queries the poster's past behavior. What has this person posted before? Do they often use irony? What topics do they follow?
Social network tool: queries the poster's social relations. Who do they follow? Who follows them? What communities are they in?
Post context tool: queries the thread containing the post. Who are they replying to? What was the original point of the discussion?
The agent workflow: read the post, decide whether more context is needed; if so, pick the right tool and query; update the judgment based on results; possibly query another round; finally output a classification.
This matches how humans judge: when we're unsure what a sentence means, we check the person's profile, see whom they're replying to, and look at their followers.
Key Findings: The Value of Context
The experiments reveal several interesting phenomena:
Overall improvement is substantial. A text-only LLM achieves an F1 of just 0.535; the full agent framework reaches 0.73—a 37% improvement. But the details are even more interesting:
User history is the most valuable single tool. Using only user history (F1=0.698) beats feeding all context to the model at once (F1=0.670). This shows that on-demand querying by an agent beats information bombardment—the model doesn't need all the information, it needs the right information at the right moment.
Context can sometimes hurt. 78% of the agent's errors were false positives—non-conspiratorial posts misclassified as conspiracies. The authors note: "Sometimes text should be taken at face value; incomplete or excessive reliance on context can distort interpretation."
Two-agent debate performed worst. Letting two agents debate before judging (F1=0.558) barely beat text-only classification. The debate framework introduced more noise than insight.
The Challenge of an "Adversarial" Dataset
The authors emphasize their dataset is "adversarial": non-conspiratorial samples deliberately include posts whose surface language resembles conspiracy theories (irony, mockery, criticism). This is closer to real-world conditions.
On easy datasets (clear separation between conspiracy and normal posts), text-only classifiers do well. In the real world, the hard part is not spotting obvious conspiracies like "the Earth is flat," but distinguishing "the Earth is flat (serious)" from "the Earth is flat (ironic)."
On the adversarial dataset, the full agent cut false negatives from 58 to 30 and false positives from 142 to 76. Context helped with both error types: confirming genuine conspiracies and ruling out surface-similar but non-conspiratorial posts.
Token Economics: Agents vs. Information Bombardment
The authors ran an interesting "token economy" analysis: compared to giving the model all context at once, which approach is more cost-effective?
Result: the agent framework achieved better results with fewer input tokens. On-demand querying avoids interference from irrelevant information and saves compute cost. This aligns with the "judgment-gating decoupling" idea—first decide what information is needed, then fetch it, rather than dumping everything in.
My Take
The paper's core insight: content classification requires intent inference, intent inference requires context, and context acquisition requires agents. This is not simple RAG—RAG is "retrieve relevant information and stuff it into the prompt," while an agent "actively decides what information it needs, where to find it, and how to use it."
The finding that 78% of errors are false positives deserves reflection. It shows the model leans toward "presumption of guilt" when facing ambiguous content—better to kill by mistake than let one slip. This matches human moderators' bias: when told "find the conspiracies," you raise suspicion toward anything dubious. But the right approach should be "presumption of innocence"—read text literally unless sufficient context proves otherwise.
The finding that two-agent debate performed worst is also worth remembering. Two agents debating is not "two experts discussing"; it's more like "two stubborn people arguing"—they reinforce their own positions rather than converge on the correct answer. This is consistent with cross-domain analogies to Milgram's obedience experiments: multi-agent systems are not automatically better than single agents; the interaction structure design is what matters.
---
Paper: Klein, E., Shapira, B., & Hirsch, L. (2026). Agentic Detection of Online Conspiracies. arXiv:2609.30250 Link: https://arxiv.org/abs/2609.30250