Paper Overview
- Field: Machine Learning
- Authors: Andrew Bo Liu, Samira Nedungadi, Bryce Cai, Alex Kleinman, Harmon Bhasin, Seth Donoughe
- Published: 2026-06-09
- arXiv: 2606.11150
- Writing code to operate liquid-handling robots
- Designing DNA fragments for in vitro assembly
- Evading DNA synthesis screening
- All tested LLM agents surpassed the human expert median baseline across the three task categories.
- Performance was notably weaker on tasks requiring novel bioinformatics reasoning.
- In wet-lab validation, scripts generated by OpenAI's o4-mini-high successfully assembled DNA matching the expected sequences on an OpenTrons robot.
Summary
ABC-Bench (Agentic Bio-Capabilities Benchmark) is a suite of tasks designed to measure biosecurity-relevant capabilities of LLM agents. It evaluates agents on both benign and dual-use biology tasks:
Key Findings
Abstract (from the paper)
Large language models (LLMs) are rapidly acquiring capabilities relevant to biological research, from literature synthesis to interpretation of experimental data. Increasingly, LLM agents can also perform in silico biology tasks that previously required experienced human biologists. These emerging AI capabilities offer new opportunities for scientific discovery and biomedical advances, but they also shift the landscape of biosecurity risks. To address this, we introduce the Agentic Bio-Capabilities Benchmark (ABC-Bench), a suite of tasks to measure agentic biosecurity-relevant capabilities. ABC-Bench evaluates LLM agents on both benign and dual-use biology tasks: writing code to operate liquid handling robots, designing DNA fragments for in vitro assembly, and evading DNA synthesis screeni...
---
*Auto-collected on 2026-06-11.*