Analyze XRD patterns
Task spaceEstimate the original proportions of powders in a mixture using X-ray diffraction (XRD).
XRD reveals information about how atoms are arranged in crystalline materials. Different crystal structures produce characteristic patterns of peaks, helping scientists identify materials and measure their proportions. This is important for checking material purity and whether a synthesis produced the intended phases.
Agents compare the mixture’s diffraction pattern with measured references for each starting powder. Overlapping peaks and differences in X-ray absorption make this harder than simply comparing peak heights. They must estimate each powder’s original mass percentage and report uncertainty in those estimates.
Scoring
Total variation error must be at most 5 percentage points: no more than 5% of the sample mass may be assigned to the wrong precursors.
RESULTS
Estimate the original proportions of powders in a mixture using X-ray diffraction (XRD).
| Model | Pass@1 | Cost ($)Model/API inference cost only, averaged per task attempt. Lab and labor costs are not reported. Repeats are averaged using the same task and family weights as scores. | |
|---|---|---|---|
 GPT–6 Astraxhigh | 82.5% | $2.79 | |
 Claude Fable 5.1xhigh | 75.0% | $11.24 | |
| Claude Opus 5xhigh | 72.5% | $6.66 | |
 Grok 4.6xhigh | 63.8% | $0.898 | |
| GPT–5.6 Solxhigh | 61.3% | $0.831 | |
 Gemini 3.8 Flashhigh | 58.8% | $1.20 | |
0%50%100% | |||
Whiskers show ±1 standard error from repeats.
Model trajectories
One example per model.
Run grading
Pass
| Measurement | Recorded result | Outcome |
|---|---|---|
| Skill vs GSAS-II Rietveld | 100.0 % | ✓ Pass |
Pass condition: Total variation error must be at most 5 percentage points: no more than 5% of the sample mass may be assigned to the wrong precursors.
Analyze XRD patternsGPT–6 Astra xhigh
GPT–6 Astra xhighRun gradingPass
00:0000:01
Model activity
Loading transcript…