Optimize reactions with limited material
Task spaceOptimize an amide-coupling reaction with limited starting material by testing conditions on related reactions in a simulated laboratory.
When target materials are scarce, directly screening many conditions may be impractical. Experiments on related reactions can guide the choice, but agents must decide which chemical similarities make those results useful for the target.
Agents allocate 24 experiments across two successive plates of twelve wells, choosing among sixteen related reactions and the available conditions. They use the measured yields to nominate one tested condition for the target reaction. Scoring uses its mean yield in separate confirmation experiments on the target, testing whether the agent can transfer what it learns between reactions.
Scoring
The confirmation yield must close at least 90% of the gap from the random-search baseline to the task-constrained oracle, within the material budget. This is a normalized improvement threshold, not 90% raw yield.
RESULTS
Optimize an amide-coupling reaction with limited starting material by testing conditions on related reactions in a simulated laboratory.
| Model | Pass@1 | Cost ($)Model/API inference cost only, averaged per task attempt. Lab and labor costs are not reported. Repeats are averaged using the same task and family weights as scores. | |
|---|---|---|---|
 Gemini 3.8 Flashhigh | 18.8% | $0.721 | |
 GPT–5.6 Solxhigh | 16.7% | $1.10 | |
 Claude Fable 5.1xhigh | 14.6% | $1.82 | |
| Claude Opus 5xhigh | 14.6% | $1.50 | |
| GPT–6 Astraxhigh | 8.3% | $3.24 | |
 Grok 4.6xhigh | 8.3% | $0.82 | |
0%50%100% | |||
Whiskers show ±1 standard error from repeats.
Model trajectories
One example per model.
Run grading
Fail
| Measurement | Recorded result | Outcome |
|---|---|---|
| Normalized score | 75.0 % | × Fail |
Pass condition: The confirmation yield must close at least 90% of the gap from the random-search baseline to the task-constrained oracle, within the material budget. This is a normalized improvement threshold, not 90% raw yield.
Optimize reactions with limited materialClaude Opus 5 xhigh
Claude Opus 5 xhighRun gradingFail
00:0000:01
Model activity
Loading transcript…