AI Runs a Full Lawsuit in LegalWorld: No Single Model Wins Across All Stages

Forum topic · 小凯 · 2026-10-18

Summary

Researchers from Fudan University and Northwest University of Political Science and Law introduced LegalWorld (arXiv 2606.18728), a benchmark that simulates Chinese civil litigation as one continuous pipeline instead of isolated tasks. It links five stages—legal consultation, pre-trial documents, first-instance trial, appeal, and second-instance trial—across seven sub-scenarios, where each stage consumes the outputs of the previous one, so early drafting mistakes propagate through the case. The environment is grounded in 75,309 paired first- and second-instance judgments covering over 500 case causes, with party, lawyer, and judge agents maintaining consistent facts and stances. Human validation by 217 legally trained reviewers gave realism scores of about 9/10. Testing five mainstream models (Kimi-K2.5, GPT-5.2, Qwen3.5-Plus, DeepSeek-V4-Flash, GLM-4.7) on LongJud-Bench yielded tightly clustered scores between 0.54 and 0.67, with no model winning everywhere. Adversarial courtroom skills—stance consistency, evidence presentation, legal reasoning—were the universal weakness, while document formatting is nearly saturated. A self-reflection mechanism that distills case trajectories into reusable skill notes raised average scores by 3.73 points and transferred 2.34 points to unseen cases. Code and data have not yet been released.

AI Runs a Full Lawsuit: No Single Model Wins Across All Stages

There is a common truth about litigation: what a lawyer writes on day one constrains how the trial unfolds months later. If the claims in a complaint are structured wrong or key evidence is omitted, even brilliant courtroom performance later cannot fix it.

Yet most benchmarks for AI legal capability break this reality into independent questions—one tests consultation, another drafting, another case retrieval—each answered in isolation. Existing legal simulators are similar: every scenario restarts from the same gold answer, and mistakes from an earlier stage never carry into the next.

A paper from Fudan University and Northwest University of Political Science and Law takes a different approach, called LegalWorld (arXiv 2606.18728). It models Chinese civil litigation as a single end-to-end chain: legal consultation, pre-trial documents, first-instance trial, appeal, and second-instance trial—five stages and seven sub-scenarios. Each later stage consumes the outputs of the earlier one: a flawed draft carries its errors all the way into the courtroom. The data foundation is 75,309 pairs of first- and second-instance judgments covering more than 500 case causes. The environment runs three types of agents—party, lawyer, and judge—each with its own memory, visibility scope, and toolset, keeping facts and stances consistent as the case progresses.

The Environment Passed a Human Check First

Can the simulator be trusted? 217 reviewers with legal backgrounds examined the trajectories and produced 18,992 ratings: stage realism 8.96/10, character consistency 8.98/10. LLM-based raters also participated—humans were on average 0.67 points stricter than LLM raters, and 64.4% of items differed by less than one point. Humans are slightly stricter judges, but both referees largely agree.

Five Models, Each Dominating a Different Segment

On LongJud-Bench, five mainstream models scored close together: Kimi-K2.5 at 0.67, GPT-5.2 at 0.63, Qwen3.5-Plus and DeepSeek-V4-Flash at 0.62 each, and GLM-4.7 at 0.54. The rankings say little; the breakdown is where the substance lies.

Kimi-K2.5 had the most reliable documents and the most stable courtroom stances.
GPT-5.2 was the opposite: weakest at document formatting (0.53) but strongest at first-instance evidence presentation and legal reasoning—weak on paper, strong in court.
Qwen3.5-Plus was best at structuring claims.
No single model won everywhere.

The paper's sharpest finding: adversarial courtroom performance is the universal weakness of all models. The three adversarial capabilities—stance consistency, evidence presentation, and legal reasoning—are comprehensively lower than document-type skills, and the weaker the model, the larger the gap. Multi-turn adversarial work requires listening to the opponent, searching one's own memory, and responding to the judge simultaneously; this is currently the hardest part. At the other end, formatting-type skills are saturated—in the second-instance stage, anchored by the first-instance judgment, entity recognition barely differentiates models. What actually separates models lives entirely in the reasoning column.

One counterintuitive detail: most capabilities improve in the second instance because the first-instance judgment is available as reference—except evidence organization, which drops. Finding new evidence in a second-instance trial is harder than reusing evidence records already fixed in the first instance.

After the Case, Experience Becomes a Rule

The paper hides an Easter egg. After a case concludes, the lawyer agent reviews the entire trajectory and distills it into a skill note: triggering conditions, applicable stage, error-correction principles, and output constraints. In the next case of the same cause type, the same model enters with that note in hand.

Tested on three high-frequency case causes (post-divorce property, private lending, labor disputes): the average score rose from 61.56 to 65.29, a gain of 3.73 points. More interestingly, cases that did not participate in writing the notes still gained 2.34 points—the experience transfers rather than being memorized. Trajectories are not just exam sheets; they are training data.

An Honest Caveat

The paper states that resources will be released publicly. It was posted to arXiv on June 17, but the project page is still a 404 today—code and data are not yet online. The claim that "Fudan built a law firm simulator for AI" is accurate.

Running one connected chain is harder than answering ten independent questions. Five models cluster between 0.54 and 0.67, and the gap is not in documents—it is all in the courtroom column.

*Source: LegalWorld, arXiv 2606.18728 (Fudan University & Northwest University of Political Science and Law)*