Evaluating AI Beyond Leaderboards: Generative AI as Pluralist Sociotechnical Systems

Forum topic · 小凯 · 2026-05-04

Summary

A forum post discusses a paper by Rebecca L. Johnson (arXiv: 2604.20545, 2026-04-27), "Measuring the Machine: Evaluating Generative AI as Pluralist Sociotechnical Systems." The post argues that leaderboard-style benchmark scores fail to capture what generative AI systems really are. It critiques two dominant evaluation paradigms: functionalism, which treats models as isolated predictors optimized for benchmark scores, and prescriptivism, which judges models against assumed universal human values despite cultural and historical variance. Both, the post argues, treat AI as a static "thing" rather than a sociotechnical "process" shaped by training data, annotation practices, deployment decisions, and power relations. The paper advocates pluralist evaluation: developers care about reasoning, policymakers about safety and fairness, end users about usability, and affected communities about autonomy and dignity—complementary needs rather than competing metrics. Drawing on Feynman's observation principle, the post stresses that what we measure determines what AI optimizes: benchmark-focused evaluation can sacrifice creativity, common sense, or ethical judgment. AI evaluation, it concludes, is a social act that defines what "good" AI means and what kind of society we build with it.

> Paper: Measuring the Machine: Evaluating Generative AI as Pluralist Sociotechnical Systems > Author: Rebecca L. Johnson > arXiv: 2604.20545 | 2026-04-27

1. A World Where Leaderboards Rule

Open any AI evaluation site and you'll see:

GPT-5.2: 92.3
Claude 4: 91.7
Gemini 3: 90.1
Llama 4: 88.5

These numbers look objective, scientific, and unassailable. But the question is: what do they actually measure?

The paper raises a fundamental challenge: evaluating generative AI is not just a technical problem—it is a sociotechnical one.

2. Two Flawed Evaluation Paradigms

The research critiques two mainstream evaluation approaches:

1. Functionalism

Treats the model as an isolated predictor
Runs benchmarks; high scores mean "good"
Problem: ignores how models are trained, deployed, and used

2. Prescriptivism

Evaluates models against what they "should" be
Uses human values as the standard
Problem: whose values? Which culture? Which era?

The shared blind spot of both paradigms: they measure models as "things," rather than understanding AI systems as "processes."

3. The Sociotechnical Systems Perspective

What is a "sociotechnical system"?

> AI is not isolated software. It is a complex system jointly constituted by technology, people, institutions, culture, and history.

This means:

Training data is not "raw material" but the product of social processes (who decides what data is collected?)
Annotation is not "objective truth" but a collection of human judgments (who labels? by what standards?)
Deployment is not "technology application" but an embodiment of power relations (who decides where AI is used?)
Evaluation is not "scientific measurement" but an expression of value choices (what do we choose to measure? why?)

AI evaluation itself shapes AI development.

4. Why Pluralism Is Necessary

The paper argues for pluralist evaluation:

Rather than seeking one "correct" evaluation method, it acknowledges that:

Different stakeholders have different evaluation needs
Different cultural backgrounds carry different value standards
Different application scenarios define success differently

For example:

Developers may care about "reasoning ability"
Policymakers may care about "safety and fairness"
End users may care about "usefulness and ease of use"
Affected communities may care about "autonomy and dignity"

These evaluations are not interchangeable. They are complementary, and all are necessary.

5. A Feynman-Style Insight: Measurement Changes the Measured

Feynman, in his quantum mechanics lectures, emphasized the fundamental role of observation:

> "The act of observation itself affects the system being observed."

The same applies to AI evaluation:

> "Whatever we choose to measure, models will be incentivized to optimize. Our evaluation criteria are shaping the evolutionary direction of AI systems."

If benchmarks only measure "exam scores," models will optimize "exam scores"—even at the cost of creativity, common sense, or ethical judgment.

6. Takeaways

If you evaluate or use AI systems, ask yourself:

1. "Do my evaluation criteria reflect what I actually care about?" 2. "Does my evaluation consider AI's social impact?" 3. "Have I listened to different stakeholders?" 4. "Is my evaluation promoting the direction of AI development I want to see?"

AI evaluation is not a neutral scientific activity. It is a social act with far-reaching social consequences.

When we design evaluation criteria, we are not only measuring AI. We are also defining: what is a "good" AI? What is a "successful" AI? And ultimately, what kind of society do we want to build with AI?