A 27.6-Point Drop Shows What Multimodal Training Does to Text Reasoning

What the authors actually measured

Qi Zhang and colleagues tested ten multimodal configurations taken from six model families, each matched against a text-only backbone and against cross-family controls. The work is published as arXiv:2611.14942.

The method matters more than the headline number. Probing a network tells you that some information is decodable from its activations. It does not tell you the model uses that information to produce an answer. The authors went the other way: they removed a specific projection from the hidden states and re-ran the benchmarks. Then they checked three things about the effect – how concentrated it is, how closely it aligns with an identified vision-language axis, and where in the network it is located.

They named the phenomenon Emergent Vision-Shaped Thinking. The label is catchier than the finding. What the experiments support is narrower: image-text pretraining changes how a model represents certain spatial, text-only tasks, and that change carries causal weight in the output.

A double dissociation of 33.2 points

The cleanest result comes from a single ablation in Chameleon-7B, Meta's early-fusion multimodal research model. Cut the vision-language axis projection and navigate loses 27.6 percentage points. Boolean expressions gain 5.6. Together that is a 33.2-point double dissociation.

The direction of the two movements is what makes it interesting. A generic degradation would push every benchmark down. Here one task collapses while a symbolic one improves, which points at a specific representational structure being used for one kind of problem and getting in the way of another.

The signal lives in the generation pathway

The authors also looked at dual-path architectures, where understanding and generation run through separate stacks. In BAGEL, from ByteDance, and Janus-Pro, from DeepSeek, the causal handle sits in the generation pathway. The understanding pathway still holds representations that a probe can decode, but removing them does not change behaviour.

In the dual-path models the effective handle is described as a compact one-dimensional carrier. That is a small target for something that shows up as a 27-point swing on a benchmark.

Rule-based tasks do not move

Symbolic, rule-following benchmarks were largely unaffected by the same interventions. The result is therefore not "multimodal training makes models reason better". It is direction-specific: some spatially loaded text tasks lean on structures that vision-language training built or reinforced, and others do not.

Scale changes the picture

The effect is spatially concentrated at the 7B–8B scale in the configurations tested. In the 32B configuration it extends to non-spatial visual attribute binding. That suggests the same mechanism widens its reach as models grow, though the paper's evidence for that is a single larger configuration rather than a scaling series.

What it means when you choose a base model

If the finding holds up, a text-only fine-tune of a multimodal base does not erase the geometry underneath. The weights move, the pathway stays. Equally, a text-only model trained on the same text corpus will not have acquired it in the first place.

For teams building anything that parses layout – floor plans described in words, diagram captions, written route instructions, table structure – the lineage of the base model becomes a selection criterion, not just a benchmark score. The honest caveat: this is BIG-Bench Hard, a research benchmark. Nobody has shown the same effect on a production workload.

Reproducing the check on local hardware

A 7B model in FP16 needs roughly 14 GB for weights alone before KV cache and activations, which is right at the edge of a 16 GB card. In practice that means 8-bit or 4-bit quantization, and the reported results do not cover quantized inference at all. Whether a 1D carrier survives 4-bit rounding is an open question, and it is the one that decides whether this is reproducible on a desktop or stays in a cluster. Our own setup for this kind of work is documented on the AI Arena page.

What hosted evaluation prompts cost in 2026

The ablation itself cannot be run through hosted APIs: they do not expose hidden states and do not allow weight or activation modification. What the table below prices is the narrower job of sending the BIG-Bench Hard navigate and boolean-expression evaluation prompts to hosted models. The causal intervention requires local weights and a GPU.

Below are list prices per million input and output tokens, plus the arithmetic for a symmetric one-million-in, one-million-out prompt-only job compiled from current vendor pricing:

Add EU VAT on top. A Czech company on the standard 21% rate pays about $3.39 for that Mistral prompt-only workload before any currency conversion, and roughly $58 for the same prompt-only job on Claude Fable 5.2. The spread between the cheapest and most expensive option here is a factor of 32, which is larger than most benchmark differences people argue about.

Availability in the EU

All six APIs are reachable from EU member states and bill in USD, so the EUR figure moves with the exchange rate on your invoice date. Data residency depends on the contract, not the model. On the regulatory side, providers of general-purpose models now operate under mandatory documentation and transparency obligations rather than the voluntary commitments of two years ago, and Article 50 requires clear labelling of AI interactions and synthetic media. This paper changes none of that – it is research on internals, not a compliance input.

The open question the authors leave is a practical one. They show a direction that can be removed and measured. They do not show what happens when a model is trained on images and text and then fine-tuned for a year on text alone at production scale. Anyone deploying a fine-tuned multimodal base in the EU right now is running that experiment without a published result to cite.