Paper Overview
Field: AI Authors: Peter Kirgis, Sayash Kapoor, Andrew Schwartz, Stephan Rabanser, David Africa, Konstantinos Voudouris, Viet Nguyen, Toby Pilditch, Magda Dubois, Harry Coppock, Cozmin Ududec, Nitya Nadgir, Matilda Orona, Tilman Bayer, Derrick Chan-Sew, Yue Ling, Abhishek Shetty, Helen Toner, Gillian Hadfield, Seth Lazar, Steve Newman, Shoshannah Tekofsky, Rishi Bommasani, Arvind Narayanan Released: 2026-07-29 arXiv: 2607.27191
Introduction: An Experiment About "Shadows"
Could a graduate student, locked in a room with only a computer and coffee, reproduce the core research question of a top-conference submission within six days and a few thousand dollars of compute? Ten years ago the answer would have been: "depends—engineering work, probably; a genuine research breakthrough, unlikely."
Today, the occupant of that room is an AI agent. Researchers from Princeton, Stanford, and MIT—including former OpenAI board member Helen Toner and AI safety scholar Arvind Narayanan—designed an experiment they call "Shadow Evaluations": letting AI chase the shadow cast by a real researcher, to see whether it can catch the outline of human thinking.
The result, in the authors' words: the AI "completed all the engineering work, but made no substantive progress on the research question."
In other words, AI can write code, run experiments, and keep a GitHub repo tidy—but it cannot "think" in the sense that produces new knowledge.
What Is "Open-Ended Research"?
Solving "2+2" is a closed problem with a definite answer. "Explore the properties of even numbers in number theory" is open-ended—no standard answer, and even the criteria for "done" are fuzzy.
There has long been a debate in AI research: are current agents doing genuine research, or just sophisticated programming? Traditional evaluation methods sit at two extremes:
- Narrow task evaluation: give the AI a clear target (e.g., reach 99% accuracy on MNIST). This measures engineering ability, not research ability—real research is not navigation from point A to B, but groping for direction in fog.
- Peer review: have the AI write a paper and submit it to a real venue. But reviewer quality is uneven, the process is noisy, and accept/reject decisions sometimes bear little relation to quality.
- Kirgis, P., et al. (2026). *Can AI agents conduct open-ended AI research? Early evidence from two case studies*. arXiv:2607.27191.
- Kapoor, S., & Narayanan, A. (2024). *Training data for machine learning: How to evaluate AI systems*. Princeton University Blog.
- Toner, H. (2023). *AI governance and the path forward*. OpenAI Board Resignation Statement.
- Schmidhuber, J. (2015). *Deep learning in neural networks: An overview*. Neural Networks, 61, 85-117.
- Bengio, Y., et al. (2021). *Deep learning for AI*. Communications of the ACM, 64(7), 58-65.
Kirgis and colleagues proposed a third path.
Shadow Evaluations: The Original Authors as Judges
The design is elegant. The researchers took two unpublished NeurIPS 2026 submissions—high-quality, rigorously peer-reviewed work—extracted each paper's core research question, and handed it to AI agents: six days, a few thousand dollars of compute, zero human intervention.
Who judges the output? Not random reviewers, but the papers' original authors themselves—like an author judging a story an AI wrote from the same inspiration. Original authors know best what the research question is, where the difficulty lies, and what counts as a valid answer. They will not applaud a wrong direction, nor give high marks merely because the AI produced a lot of code.
Results: An Engineering Giant, an Intellectual Dwarf
The results in one sentence: the AI completed all engineering implementation but made no headway on the core research questions. For both NeurIPS submissions, the original authors said the outputs would be rejected without hesitation.
The AI did plenty: it built experiment frameworks, implemented baselines, ran large numbers of experiments, and even produced plausible charts and analyses. A cursory glance at the repository would look convincing—but the research content falls apart under scrutiny.
The researchers identified five recurring failure modes:
Failure Mode 1: Misjudging the "Publication Bar"
Like asking someone who has never attended a dinner party to prepare a state banquet: the AI knows it needs plates, forks, and food, but not which dishes are presentable.
The agent treats trivial results as breakthroughs, or overlooks genuinely promising directions. It lacks "taste"—the intuition, built by years of training, for what counts as an interesting question and a substantive answer. If a method beats a baseline by 2%, a human researcher immediately asks: is it statistically significant? Does it generalize? Is there a theoretical explanation, or is it noise? The AI happily writes the 2% up as success.
Failure Mode 2: Insufficient Creativity in Responding to Flawed Research Designs
Real research never goes smoothly. When results don't match expectations, a good researcher rethinks the hypothesis, the experimental design, or overlooked variables. The AI, by contrast, is strikingly rigid: it spins within the existing frame, tweaking method A's parameters rather than asking whether method A itself is wrong—like a chef who keeps adding water instead of adjusting the salt or changing the cooking method.
Failure Mode 3: Not Knowing When to Back Out of a Dead End
Hitting a dead end is one of the most painful experiences in research. Humans stop, grab a coffee, talk with a friend, and try a completely different direction. This capacity for "strategic retreat" is core research wisdom. The AI behaves like a stubborn gambler, continuously pouring resources into a visibly failing direction—tweak, rerun, fail, tweak again. It lacks the big-picture judgment of when to persist and when to quit.
Failure Mode 4: Weak Resource Awareness
The agents were given compute budgets—and seemed to ignore them entirely. They launched large numbers of expensive experiments simultaneously without prioritizing by value, never reasoning "this experiment costs $1,000—one fifth of my budget—so I'd better think hard about whether it's worth it."
This lack of "cost sensitivity" reflects something deeper: the AI feels no pang, no anxiety, no growing caution as the budget runs out. Every experiment is equal; there is no ordering of importance.
Failure Mode 5: Instruction Drift
The subtlest failure mode. Given a clear research goal, the agent gradually drifts away from it over time, doing work that is related but does not directly answer the core question—like a student writing a paper who wanders off into "interesting but irrelevant" territory. This reflects a lack of self-examination: humans constantly ask, "Is what I'm doing answering the original question, or dodging it?"
What Does This Mean?
AI agents autonomously completing engineering implementation—writing code, running experiments, organizing data—is already remarkable. Ten years ago it would have sounded like science fiction.
But real research is more than engineering. It requires judgment, creativity, strategic thinking, resource management, and self-reflection—the "soft skills" that are precisely where current AI is weakest. The findings are a warning: optimistic predictions that "AI will automate research within a few years" may need recalibration. The gap between AI and a real researcher has a name: wisdom.
What Comes Next?
The study has limitations: only two case studies, specific models and scaffolds. Robustness checks with a different model and scaffold yielded broadly consistent results—but this is still "early evidence."
More importantly, the work provides a template for future evaluation. As AI capabilities improve rapidly, we need better ways to measure how much it actually "understands." Shadow evaluations—having those who know the problem best judge the quality of the answer—are a strong starting point.
Perhaps future models will learn to judge the publication bar, turn at dead ends, respect resources, and stay locked on the core question. Until then, a healthy humility is warranted—acknowledging both what AI has achieved and what it still lacks.
After all, knowing how to write code and knowing how to do research are two different things—just as knowing how to write and knowing how to write poetry are two different things.
> Note: NeurIPS (Neural Information Processing Systems) is one of the top academic conferences in machine learning, with extremely rigorous peer review. Publishing there is generally considered a marker of research caliber.
References