When the Agent's Scaffolding Starts Evolving Itself: How DrugHarness Makes Drug Discovery Agents Smarter With Use
A Medicinal Chemist's Morning
Imagine you are a medicinal chemist designing an HPK1 inhibitor. You have four known active reference molecules, the best with an IC50 of 37.47 nM. Your goal is to design a better molecule — higher potency, same scaffold, same double-bond configuration, changing only the substituent on the terminal amide nitrogen.
In front of you are 80+ tools: BindingDB for activity data, RDKit for molecular structures, ADMET-AI for pharmacokinetic predictions, Boltz-2 for structure prediction. You must weave between them — retrieve evidence, select candidates, compute properties, modify structures, evaluate. Every decision depends on the previous step's result.
This is not a question of whether a model is strong enough. It is an orchestration problem: how do you organize 80 tools, multiple data sources, a Python environment, and your own past experience into a reliable workflow?
DrugHarness's answer: let the harness itself evolve.
The Core Problem: Tools Multiply, But Experience Doesn't Automatically Become Capability
Scientific agents over the past few years have gone through three stages:
1. Tool integration: ChemCrow, Biomni, ToolUniverse turned scientific tools into callable interfaces 2. Workflow orchestration: PharmAgents, TRACE organized multi-step workflows with domain skills and expert collaboration 3. Experience reuse: DrugSAGE, SciToolAgent-Evo extracted reusable lessons from historical trajectories
But there's a neglected crack: experience doesn't automatically become reliable behavioral guidance.
The paper identifies three "coupled challenges" — not three independent problems, but three mutually binding constraints:
Challenge 1: Recommendations must be grounded in the right evidence. An activity measurement, a predicted property, the agent's own interpretation, and the final task score answer different questions. An IC50 value supports "this molecule is active" but not necessarily "substitute at this position." Confusing sources and targets turns recommendations into noise.
Challenge 2: Applicability must persist across tasks. A strategy that works in task A may be wrong in task B. For example, if a set of molecules has met property targets but lacks diversity, what's needed is not more optimization of similar candidates but broader structural exploration. The same advice means the opposite in different states.
Challenge 3: Guidance must be executable and correctable. A plausible recommendation may rest on an unverified scientific premise or a tool whose behavior doesn't match its description. You need to actively check it, then revise it as feedback arrives.
These three challenges point to one conclusion: you need a learning process that binds scientific evidence, observable conditions, active review, and continued skill use together.
Architecture: Executor + Learner + Reviewer
DrugHarness is designed like a scientific laboratory with three layers of checks and balances:
Layer 1: A Composable Scientific Execution Environment
80+ scientific operations across 10 capability categories — evidence retrieval, target preparation, molecular design, computational evaluation. BindingDB, RDKit, ADMET-AI, Boltz-2 and a full Python environment work side by side.
Key design: a shared scientific workspace. When the agent calls tools, the associations between candidate molecules, computational conditions, data sources, and outputs are preserved. Different tools' evaluations of the same molecule never lose their linkage. Returned structures or datasets can feed directly into the next analysis step.
This sounds simple, but it's the foundation of everything that follows. If tool calls are isolated — every call starting from zero context — experience has nowhere to grow.
Layer 2: Induction of Conditional Skills
The learner extracts experience from execution trajectories. Each experience is formalized as:
where \(x_i\) is the task goal and constraints, \(\tau_i\) is a \(T_i\)-step sequence of (action, observation) pairs, \(y_i\) is the deliverable, and \(f_i\) is the allowed feedback.
The learner identifies recurring decision problems across a batch of experiences, compares related scientific objects, endpoints, methods and evidence states, and generates a conditional behavioral hypothesis:
- \(g\): goal
- \(c\): conditions observable to the executor
- \(a_0\): current action; \(a\): recommended action
- \(R_h\): sources supporting or limiting the hypothesis
- \(U_h\): unresolved information
- admit: the skill enters the skill library and is available for future tasks
- reject: conflicts with key facts or constraints; rejected
- defer: insufficient evidence; shelved but recorded
- supported: conditions backed by the current state; the recommendation can guide action selection
- contradicted: conditions conflict with the current state; the executor turns to other grounds
- unknown: key conditions unknown; the executor decides whether further information is needed
- Screening accuracy: 96.0% → 100%
- Affinity comparison accuracy: 62.2% → 81.1%
- Optimization success rate: 79.5% → 87.2%
- Valid JSON submissions: 87.2% → 100%
- Jurkat p-SLP76 IC50 = 14.19 nM
- The four reference molecules' IC50 range: 37.47–227.77 nM
- HPK1 inhibition at 1 nM concentration: 96.92%
- Comparable potency to molecules independently designed by two medicinal chemists with 5+ years of experience
- Skill evolution was only retrospectively evaluated on a single benchmark; large-scale generalization is unknown
- Systematic cost comparison with other agent frameworks is missing
- Experimental validation is limited in scale; developability properties are incompletely covered
- The four HPK1 inhibition endpoints were selected post hoc — DrugHarness has lower median values but higher hERG inhibition and lower QED
This is not a simple "when X, do Y" rule, but "under condition C, for goal G, change from A0 to A, based on R, with U still unclear."
Example: when a set of molecules has met property targets but lacks diversity, the hypothesis suggests "prioritize expanding structural sources rather than continuing to optimize similar candidates." The condition is "properties met + insufficient diversity," the goal is "improving overall quality," the basis is "diminishing marginal returns from optimizing similar candidates."
Layer 3: Independent Review and Tri-State Admission
Once a candidate skill is generated, an independent reviewer agent checks three things:
1. Does the evidence support the recommendation? The reviewer can retrieve original sources and inspect interfaces 2. Can the executor evaluate the conditions? If conditions depend on information the executor can't see, the skill is castles in the air 3. Is the instruction executable? If the recommended operation can't run in the environment, the skill is dead code
The reviewer has a special power: when a scientific or operational question requires computational verification, it can run targeted experiments in the same scientific workspace — running a small experiment to validate a hypothesis.
The review outcome is not binary but tri-state:
Rejected and deferred proposals don't disappear — their review records, experiment findings, and open questions feed into the next learning round. This matters: a "not yet viable" skill may become "viable now" after more evidence accumulates.
At Task Time: Tri-State Applicability Assessment
Once a skill is admitted, when the executor encounters relevant skills during a task, it evaluates their applicability conditions:
Tri-state rather than binary is a key design choice. In scientific reasoning, "I don't know whether the condition holds now" and "the condition doesn't hold" are completely different states — the former means go check, the latter means change course.
The Numbers: From Benchmarks to Experimental Validation
Benchmarks: Comprehensively Surpassing Existing Systems
On MolBench (190 molecular decision tasks):
| Metric | MolClaw-CC | CAi | DrugHarness | |---|---|---|---| | Affinity comparison accuracy | 81.1% | 54.1% | 91.9% | | Average Hits@3 | 0.80 | 0.76 | 1.24 | | Screening accuracy | 98.0% | 18.0% | 98.0% | | Molecular editing overall | 97.4% | 87.2% | 100% | | Optimization success rate | — | 48.7% | 94.9% |
On SMDD-Bench (40 lead-optimization tasks): DrugHarness achieves a 60% success rate vs. 50% for the original SMDD harness. More notable is interaction-point discovery (IP): DrugHarness 28.0%, original harness 0.0%.
Skill Evolution: Experience Actually Helps
In retrospective evaluation with frozen domain skills:
A concrete case: in affinity task ms2-029, both skill versions led the executor to retrieve the same 933 records. The early skill suggested text search and structural reasoning; the revised skill suggested RDKit structural matching and distinguishing \(K_i\) from IC50. The decision went from wrong to right.
Prospective Validation: A Real Molecule Was Synthesized
In the HPK1 design task, DrugHarness's designed molecule was actually synthesized and experimentally verified:
This isn't benchmark-chasing — an agent-designed molecule beat all reference molecules in a wet lab.
Engineering Insights: Why "Harness Evolution" Beats "Model Evolution"
1. The Carrier of Experience Is Skills, Not Parameters
Most agent systems reuse experience by stuffing trajectories into context, or fine-tuning model parameters. DrugHarness chose a third path: solidifying experience into structured, evidence-backed, reviewable skills.
A skill is not a chunk of natural-language prompt. It has structured goals, conditions, recommended actions, evidence sources, and open questions. This means skills can be independently reviewed, rejected, and precisely revised as new evidence arrives.
2. Reviewer Independence Is Central to the Design
The reviewer and executor share the same scientific workspace, but are independent. The reviewer can run experiments to validate hypotheses — not a "read-and-score" reviewer, but a "reviewer that runs experiments."
This solves a deep problem: agent-generated experience may hallucinate. If a skill rests on a false premise, the executor will fail using it — but the failure only surfaces in later tasks. The reviewer's targeted experiments move this problem forward to the skill admission stage.
3. Tri-State Admission Is More Honest Than Binary Judgment
The admit/reject/defer tri-state acknowledges a fact: sometimes you don't have enough information to decide. Defer is not a euphemism for rejection — it is an explicit state: "this question cannot be answered now, but keep the record and revisit when more evidence arrives."
This matches how human scientists decide. Peer review isn't a binary accept/reject — there's also "major revision and resubmit," which is essentially defer.
4. Rejection Is Also Learning
Rejected skill proposals don't vanish. Review records, experiment findings, and open questions all feed the next round of learning. This means rejection isn't waste, it's information gathering — you pay one review's cost but learn "this path doesn't work" and "why it doesn't work."
Implications for Agent Engineering
DrugHarness's core insight generalizes beyond drug discovery:
"Evolvable harness" is a new architectural paradigm. Traditional agent architecture: model + tools + fixed workflow. DrugHarness proposes: model + tools + an evolving workflow. The workflow itself learns from execution experience, generates conditional skills, and enters the skill library after independent review.
This resonates with the previously discussed Harness/Loop/Graph three-layer architecture: the Harness is not a static pipeline — it should itself be a learning system. The Loop isn't just "observe-think-act"; it should include the meta-loop of "induce-review-revise."
Conditional skills beat generic guidelines. Many agent frameworks treat "experience" as a pile of natural-language tips — "if you hit X, try Y." DrugHarness's skills are structured: under what conditions, for what goal, changing from what action to what action, on what evidence, with what still unresolved. This structure makes skills machine-evaluable, independently reviewable, and precisely revisable.
Independent review + targeted experiments are the safety net for agent self-evolution. Any self-evolving system risks hallucinated improvements. DrugHarness's reviewer isn't a scoring model but an independent agent that runs experiments — it validates hypotheses in the same environment rather than just judging textual plausibility.
Limitations and Open Questions
The paper honestly lists its limitations:
Conclusion
DrugHarness did something hard: it moved an agent system from "can execute tasks" to "can learn from execution — with a trustworthy learning process."
The key isn't 80 tools, not 91.9% accuracy, not even the 14.19 nM IC50. The key is that it demonstrates a possibility: the harness can evolve, evolution can be reviewed, review can run experiments, and experimental results can feed the next round of evolution.
That's a closed loop. And a closed loop is the ultimate goal of all self-improving systems.
---
Paper: arxiv.org/abs/2610.23791
Code: The paper states that implementation, tool interfaces, and environment configs will be released; not yet publicly available.