Prepare and run sfGFP PCR reactions with matched no-template controls on a simulated Hamilton STARlet.
Automating PCR requires coordinating precise reagent additions, mixing, plate transport, and thermocycling. Correct reaction composition and controls are essential, while inefficient instrument sequences reduce laboratory throughput.
Agents control the simulated liquid handler through VENUS, PyLabRobot, or direct firmware commands. They assemble reactions and controls, mix the completed reactions, transfer the plate to the thermocycler, and return it after cycling. The task tests completion of the required workflow and efficient instrument execution, with scoring based on achieved milestones and instrument time rather than time spent reasoning.
Scoring
Reach all seven required milestones. Instrument speed does not affect Pass@1.
RESULTS
Prepare and run sfGFP PCR reactions with matched no-template controls on a simulated Hamilton STARlet.
Model
Pass@1
Cost ($)Model/API inference cost only, averaged per task attempt. Lab and labor costs are not reported. Repeats are averaged using the same task and family weights as scores.
GPT–6 Astraxhigh
100.0%
$42.56
Claude Fable 5.1xhigh
0.0%
$32.10
Claude Opus 5xhigh
0.0%
$128.88
GPT–5.6 Solxhigh
0.0%
$40.95
Grok 4.6xhigh
0.0%
$55.43
Gemini 3.8 Flashhigh
0.0%
$13.63
0%50%100%
Model trajectories
One example per model.
Run grading
× Fail
Measurement
Recorded result
Outcome
Score
68.6 %
✓ Pass
Pass condition: Reach all seven required milestones. Instrument speed does not affect Pass@1.
Pass condition: Reach all seven required milestones. Instrument speed does not affect Pass@1.
✓
Deck loaded
at 5451.086 s · 1 errors · machine time 0.960 s
×
Master mix in
not reached
×
Template in
not reached
×
Wells mixed
not reached
×
Plate in cycler
not reached
×
Program run
not reached
×
Plate home
not reached
×
Final state
final home conditions not met
✓
Record
complete
✓
benchmark environment
registered
✓
golden method
six sealed files match
✓
agent boundary
all STARlet work is in interval
×
time budget
7200.000176885049 / 7200 s
✓
recorded evidence
complete
×
orchestration
TimeoutError: agent exceeded the wall budget
Measurement
Recorded result
Outcome
Score
42.9 %
× Fail
Pass condition: Reach all seven required milestones. Instrument speed does not affect Pass@1.
✓
Deck loaded
at 5460.337 s · 1 errors · machine time 0.960 s
✓
Master mix in
at 5469.118 s · 0 errors · machine time 3.379 s
✓
Template in
at 7060.470 s · 115 errors · machine time 5.982 s
×
Wells mixed
not reached
×
Plate in cycler
not reached
×
Program run
not reached
×
Plate home
not reached
×
Final state
final home conditions not met
✓
Record
complete
✓
benchmark environment
registered
✓
golden method
six sealed files match
✓
agent boundary
all STARlet work is in interval
×
time budget
7200.000116071 / 7200 s
✓
recorded evidence
complete
×
orchestration
TimeoutError: agent exceeded the wall budget
Measurement
Recorded result
Outcome
Score
14.3 %
× Fail
Pass condition: Reach all seven required milestones. Instrument speed does not affect Pass@1.
✓
Deck loaded
at 6696.855 s · 16 errors · machine time 2.419 s
×
Master mix in
not reached
×
Template in
not reached
×
Wells mixed
not reached
×
Plate in cycler
not reached
×
Program run
not reached
×
Plate home
not reached
×
Final state
final home conditions not met
✓
Record
complete
✓
benchmark environment
registered
✓
golden method
six sealed files match
✓
agent boundary
all STARlet work is in interval
×
time budget
7200.000175264999 / 7200 s
✓
recorded evidence
complete
×
orchestration
TimeoutError: agent exceeded the wall budget
Measurement
Recorded result
Outcome
Score
0.0 %
× Fail
Pass condition: Reach all seven required milestones. Instrument speed does not affect Pass@1.
×
Deck loaded
not reached
×
Master mix in
not reached
×
Template in
not reached
×
Wells mixed
not reached
×
Plate in cycler
not reached
×
Program run
not reached
×
Plate home
not reached
×
Final state
final home conditions not met
×
Record
VENUS application trace is missing
✓
benchmark environment
registered
✓
golden method
six sealed files match
✓
agent boundary
all STARlet work is in interval
✓
time budget
3135.823942623 / 7200 s
✓
recorded evidence
complete
✓
orchestration
complete
Measurement
Recorded result
Outcome
Score
14.3 %
× Fail
Pass condition: Reach all seven required milestones. Instrument speed does not affect Pass@1.
✓
Deck loaded
at 2825.597 s · 4 errors · machine time 2.128 s
×
Master mix in
not reached
×
Template in
not reached
×
Wells mixed
not reached
×
Plate in cycler
not reached
×
Program run
not reached
×
Plate home
not reached
×
Final state
final home conditions not met
×
Record
VENUS method markers are out of order; VENUS method markers are out of order; VENUS method markers are out of order; VENUS method markers are out of order; VENUS method markers are out of order; VENUS method markers are out of order; VENUS method markers are out of order; VENUS method markers are out of order; VENUS method markers are out of order; VENUS method markers are out of order; VENUS method markers are out of order; VENUS method markers are out of order; VENUS method markers are out of order
✓
benchmark environment
registered
✓
golden method
six sealed files match
✓
agent boundary
all STARlet work is in interval
×
time budget
7200.000173234002 / 7200 s
×
recorded evidence
complete; agent trace projected reasoning/tool events archived from the AG trace page; final checkpoint unverified
×
orchestration
TimeoutError: agent exceeded the wall budget; agent checkpoint status: AG checkpoint Files version differs from the public session
GPT–6 Astraxhigh
Claude Fable 5.1xhigh
Grok 4.6xhigh
Gemini 3.8 Flashhigh