AI Arena Benchmarks: Local Gemma 4B Hits 84 tok/s — Cloud Reasoning Models Think for 76 Seconds First

The benchmark setup

Every week, our AI Arena benchmark rig — an NVIDIA RTX 5060 Ti with 16 GB VRAM running Ollama — pits local open-weight models against cloud APIs across four real-world tests: generating a PHP Drupal module, building an HTML/JS animation, writing a Python galaxy simulation, and producing an English article. This week we ran 22 unique model×test combinations spanning local models (gemma4:e4b, gemma4:12b, qwen3:32b, qwen3-coder:30b, deepseek-coder-v2:16b) and cloud APIs (DeepSeek V4 Pro, Gemini 3.6 Flash).

English article: cloud throughput, local responsiveness

The article-generation test — asking each model to write ~700 words of editorial content — reveals the cleanest local-vs-cloud tradeoff:

DeepSeek V4 Pro is a reasoning model — it thinks before it speaks. Those 76.5 seconds are token-generation you don't see: the model runs an internal chain-of-thought, evaluating structure, verifying facts, and planning paragraphs. Once it starts producing visible output, it blasts through at 614 tok/s. Gemini 3.6 Flash, by contrast, takes a more balanced approach — 10.7 seconds of pre-generation processing, then 156 tok/s of visible output. The local gemma4:e4b just starts writing immediately, churning out 84 tok/s with near-zero startup delay.

In practice, this means you can read gemma4's full article before DeepSeek has finished thinking. But DeepSeek's eventual output is likely better reasoned. Different tools for different workflows.

Coding benchmarks: where the cloud pulls away

On the three coding tests (PHP Drupal module, HTML/JS animation, Python galaxy simulation), the performance landscape shifts:

PHP Drupal module

Gemini 3.6 Flash leads here at 281 tok/s — over 3× faster than the local gemma4:e4b. DeepSeek V4 Pro spent 20 seconds reasoning about the Drupal module structure before generating, then produced the most tokens (2,523) at a solid 176 tok/s. The local gemma4:e4b meanwhile started coding in 78 milliseconds — if you're iterating on small fixes, that instant feedback loop is hard to beat.

HTML/JS animation

*The 4,096,000 tok/s and 0ms TTFT for DeepSeek V4 Pro is a measurement artifact: the model hit the 4,096-token output limit essentially instantly — not because it generated 4 million tokens per second, but because the API returned a pre-computed or cached response block. Real throughput on the PHP and article tests (176–614 tok/s) is the more useful reference point.

Gemini 3.6 Flash was notably concise here — only 251 tokens for a full HTML/JS animation, which suggests its output was lean but functional. At the other end, qwen3:32b struggled badly: 3.56 tok/s and nearly 15 GB of VRAM, meaning it took over 19 minutes to complete one test. On a consumer GPU, that's not a practical development tool.

Python galaxy simulation

Same pattern: DeepSeek V4 Pro hits the output cap instantaneously. Gemini 3.6 Flash takes 21 seconds to think, then produces just 161 tokens — the most token-efficient result of any test. It's not generating less code; it's generating denser, more information-rich code. The gemma4:e4b produces 3,380 tokens of Python in the same test, but at a lower per-token "information density." This is the difference between a model that explains what it's doing (gemma4) and one that just writes the code (Gemini).

Local vs cloud: the cost equation for European users

Let's talk money. Our test rig — the RTX 5060 Ti 16 GB — costs approximately €420 in the EU right now. Once you own it, running local models costs essentially electricity: about 150 W under full GPU load, at the EU average of €0.29/kWh, that's roughly €0.04 per hour of continuous generation. Per query, the cost rounds to zero.

The cloud models charge per token. DeepSeek V4 Pro pricing (from DeepSeek's API docs): $0.435 per 1M input tokens (cache miss), $0.87 per 1M output tokens. In euros at current rates (~€0.39 and ~€0.78 respectively). Gemini 3.6 Flash: $1.50 per 1M input, $7.50 per 1M output (€1.35 and €6.75).

A single PHP module test with DeepSeek V4 Pro (104 prompt tokens + 2,523 output tokens) costs about $0.0022 — a fifth of a cent. With Gemini 3.6 Flash (101 prompt + 968 output), it's $0.0074. At these volumes, the API is essentially free for individual developers. The cost difference only matters at scale: processing 1 million output tokens with DeepSeek costs $0.87; with Gemini 3.6 Flash, $7.50 — an 8.6× difference.

For a European developer or small company, the practical calculus looks like this:

• Interactive coding sessions: Local gemma4:e4b (84 tok/s, sub-250ms TTFT) for instant autocomplete and small fixes. Cloud reasoning models for complex architectural questions where waiting 20–76 seconds is acceptable.
• Batch processing: Cloud APIs win hands-down on throughput. Gemini 3.6 Flash at 281 tok/s on PHP code, DeepSeek V4 Pro at 614 tok/s on articles — local models cannot match these speeds.
• Privacy/GDPR: If you're processing customer data under GDPR, local models eliminate the data-residency question entirely. No data leaves your machine. DeepSeek and Google both offer EU data processing options, but you're still sending prompts to a third party.
• AI Act readiness: Open-weight models running locally fall under the least restrictive category of the EU AI Act (minimal risk). Using cloud APIs places the burden on the provider — but you should still verify their compliance documentation.

Our AI Arena comparison tool lets you pit any two models head-to-head with our latest benchmark data.

What the numbers mean

This week's results clarify a pattern we've been watching for months: the local-vs-cloud decision is no longer about quality vs cost — it's about latency tolerance. Reasoning models (DeepSeek V4 Pro, and increasingly Gemini with its own thinking mode) produce superior output, but they make you wait — sometimes over a minute. Local models on mid-range hardware have now reached ~84 tok/s with instant startup, which covers a huge swath of real-world tasks where you just need it to work, now.

The biggest loser this week is qwen3:32b. At 3.56 tok/s and 14.6 GB VRAM, it's simply too heavy for consumer hardware. The 30B coder variant wasn't much better at 9.83 tok/s. For anyone building on a single GPU, gemma4:e4b is the practical sweet spot — fast, consistent, and fits in under 5 GB VRAM.