Forward reaction prediction is a language task if you accept a strange enough definition of language. Reactants go in as SMILES strings, products come out as SMILES strings, and RDKit decides whether the output is chemically valid and byte-identical to the patent record after canonicalisation.
A Stanford team published Distillation, Self-Training, and the Fragility of Learned Chemical Reasoning (arXiv:2610.24199) in October 2026, together with the Resonance model family on Hugging Face. The paper is a training recipe with a warning attached. The recipe works. The warning is about what "works" means.
What the training pipeline does
Three stages, each doing a different job. Stage one is ordinary supervised fine-tuning: take a weak base model, feed it step-by-step reasoning traces generated by a much larger teacher (Kimi K3), and let it imitate the format and the intermediate steps, not only the final answer.
Stage two is rejection-sampling self-training. For each training reaction the model generates eight candidate answers. Only the ones RDKit verifies as correct get kept. No human labelling, no teacher involved – the chemistry toolkit is the judge.
Stage three applies Direct Preference Optimization on the survivors versus the rejects, which pushes the model toward the answer shapes that survived verification.
The accuracy numbers
Everything is measured on USPTO-50k, the standard patent-derived benchmark, with a 5,000-reaction test split. Scoring is Top-1 exact match on canonical SMILES, verified with RDKit. Getting a molecule nearly right counts as wrong.
| Resonance-8B stage | Top-1 exact accuracy |
|---|---|
| Base model (Qwen3-8B) | 1.2% |
| Stage 1: SFT on Kimi K3 traces | 31.0% |
| Stage 3: after self-training and DPO | 42.8% |
Two things stand out. First, distillation alone accounts for most of the gain: 1.2% to 31.0% is a different model, not an improved one. Second, the self-training loop adds nearly twelve points without a single human annotation.
Read the last number plainly, though. 42.8% means more than half of the test reactions are still wrong. A chemist using this as a first pass gets a plausible starting point roughly four times in ten. That is useful. It is not a drop-in replacement for anything.
What a distillation run costs
The paper does not publish teacher API spend, so here is the arithmetic with my own stated assumptions: roughly 40,000 training reactions in the USPTO-50k train split, and an estimated 800 output tokens per reasoning trace. Both figures are estimates, not numbers taken from the paper. Teacher prices are the published October 2026 API rates, and on the output side both frontier candidates charge the same.
| Step | Generations | Output tokens | GPT-6.1 Sol ($10/1M out) | Gemini 4 Argon ($10/1M out) |
|---|---|---|---|---|
| Stage 1: one trace per reaction | 40,000 | 32M | $320 | $320 |
| Stage 2: eight candidates per reaction | 320,000 | 256M | $2,560 | $2,560 |
| Total | 360,000 | 288M | $2,880 (≈ €2,650) | $2,880 (≈ €2,650) |
The two frontier APIs are priced identically for this workload – $2 per million input tokens and $10 per million output tokens each – so the teacher is not where the bill comes from. Output volume is. The biggest defensible lever therefore sits in stage two: eight candidate traces per reaction is a design choice, not a requirement, and cutting it to four removes 128M output tokens, or $1,280, taking the total from $2,880 to $1,600 (≈ €1,470). Whether the smaller sampling pool costs accuracy is a trade-off worth measuring before anyone copies the recipe at scale.
Smaller levers exist but they do not change the order of magnitude. Cached input at $0.10/1M reduces the repeated instruction prefix to almost nothing, and batch discounts shave a further slice, but input tokens are a minor line here because the reasoning traces dominate the spend. The training compute for SFT and DPO on an 8B model is the part I cannot price without hardware figures from the authors, and it is likely larger than the token spend for anyone doing this at home rather than on a cluster.
Where the reasoning breaks
This is the section that should travel further than the accuracy table. The Stanford team probed the trained models with perturbations that carry no chemical information at all. Give the model negative feedback on an answer it produced – a simple statement that the answer is wrong – and it frequently abandons a prediction that was correct. Vary the prompt slightly and accuracy moves again.
So the models learned something that produces correct SMILES on the benchmark distribution, and did not learn something that survives disagreement. Accuracy and robustness came apart in the same model, on the same task, at the same time.
Anyone who has used a chat model knows the behaviour. What the paper adds is that it persists after verification-filtered self-training: the rejects were removed by RDKit, the survivors were chemically valid, and the resulting model still folds under social pressure. Filtration cleans the answers. It does not clean the reasoning that produced them.
Running it in Europe
The weights are open and downloadable, which matters more for a European chemistry team than for most users. Reaction data from a pharma or specialty chemicals pipeline is usually proprietary and often covered by internal data-residency rules. An 8B model that runs on-premise keeps that data inside the building; an API call does not.
Hardware is the practical limit. Resonance-8B is a Qwen3-8B derivative, so at 4-bit weights it needs roughly 4.5 to 5 GB of VRAM. That fits comfortably on the 16 GB card in our AI Arena rig with room for a long SMILES context. The 27B variant lands near 13.5 to 15 GB at the same precision, which puts it against the 16 GB ceiling – workable only with short contexts or a lower-bit quantisation that costs accuracy nobody has measured yet.
Regulatory note, since it comes up: a reaction-prediction model is not a chatbot, so the AI Act's Article 50 transparency duties – in force since 2 August 2026 – do not attach to it the way they attach to customer-facing conversational systems. The fragility finding is a validation problem, not a compliance one. If the output feeds a regulated dossier, the burden is still on the chemist who signed it.
What the paper does not show
One benchmark, one task type. USPTO-50k covers forward prediction from patent reactions; retrosynthesis, yield prediction and stereochemistry-heavy cases are different problems and the paper does not claim the recipe transfers to them. The fragility tests are behavioural – prompt-level perturbations scored by accuracy drop. The authors demonstrate that the behaviour exists, not the mechanism behind it.
The datasets and both checkpoints are public, so the next result will come from someone else's perturbation, not from this paper.
Can Resonance-8B run on a normal laptop?
At 4-bit quantisation the weights take about 4.5 to 5 GB, so a machine with 8 GB of unified memory or a small discrete GPU can load it. Speed is the constraint – CPU inference on an 8B model is slow enough that batch screening of thousands of reactions becomes a background job, not an interactive tool.
Does 42.8% accuracy mean the model is wrong most of the time?
On this benchmark, yes. Top-1 exact match on canonical SMILES requires the entire product string to match after RDKit canonicalisation, so a single misplaced substituent fails the check. A model can produce chemically sensible products and still score badly under this metric.
Why does negative feedback break a verified model?
The paper does not answer that. It documents the behaviour after a pipeline that filtered training data with RDKit, and reports that correct predictions get abandoned under pushback. Any explanation of the cause would be speculation beyond what the experiments tested.
