The mandate as published
The Commission's contract, as described in the announcement carried by The Manila Times, names Capgemini as leader of a consortium with Wavestone and the French public institution LNE (Laboratoire National de Métrologie et d’Essais). The assignment sits under Lot 2 of procurement procedure EC-CNECT-2025-OP-0032, awarded by the European AI Office within DG CNECT under the EU AI Safety programme. The scope is cyber threat modelling and risk assessment for General-Purpose AI (GPAI) models under the AI Act. The contract value is not disclosed in the announcement.
Why a consortium rather than a research group? Testing a frontier model is not one benchmark run. Someone has to write probe sets, score outputs consistently, keep results reproducible when a provider silently updates the model behind an API, and do all of it across several model families at once. That is lab work with a delivery schedule attached.
Safety assessment and cyber risk are different jobs
Safety evaluation asks whether a model does something harmful when asked directly, or when pushed with a jailbreak. Cyber risk evaluation asks something narrower: can the model help someone find, exploit or chain vulnerabilities? The second question is easier to score and harder to publish, because the probe sets themselves are dual-use material.
Both depend on volume. A thousand prompts produce anecdotes. A campaign that separates a two-point difference in refusal rate from noise needs tens of thousands of samples per model, and each of those samples travels to somebody's inference endpoint with the prompt attached.
The AI Act timetable behind the contract
Enforcement is no longer theoretical. Article 50 transparency rules have applied since 2 August 2026, and the European AI Office works with national surveillance authorities on cross-border cases. General-purpose model providers have carried obligations since August 2025, with the GPAI codes of practice describing how compliance gets documented.
The high-risk deadlines moved. The Digital Omnibus on AI, Regulation (EU) 2026/1744, pushed Annex III compliance from 2 August 2026 to 2 December 2027, and Annex I embedded systems to 2 August 2028. Companies that had budgeted for last summer gained 16 months of runway.
That postponement does not cover everything. Transparency duties and market-surveillance powers are active now, and they need evidence of how a model behaves rather than paperwork about how it was built.
What a test campaign costs at API prices
Evaluation is billed by the token like any other workload. Take a modest campaign of 50,000 prompts, 500 input tokens and 800 output tokens each. That is 25 million input tokens and 40 million output tokens. To keep the comparison verifiable, the table below carries only a price that could be tied to a provider's published API list at the time of writing: GPT-6 Rosalind's standard $15/$75 per million tokens.
| Model | Input $/1M | Output $/1M | Cost of 25M in + 40M out |
|---|---|---|---|
| GPT-6 Rosalind (standard API list price) | 15.00 | 75.00 | $3,375.00 |
That single row still shows why evaluation budgets are not trivial: 25 million input tokens cost $375, and 40 million output tokens cost $3,000, for $3,375 per campaign. No credible cheapest-versus-most-expensive spread can be calculated from the verified data: other per-token figures were left out because they could not be tied to a provider's published price list as of publication.
Two practical caveats. This is a USD list price, so European buyers add VAT and whatever their card issuer charges on the conversion, and the euro total shifts with the exchange rate. A real campaign also repeats each prompt with variations, reruns after model updates and stores outputs for human review, which multiplies token counts several times over.
The awkward part is that the models most worth testing are often the expensive ones, and GPT-6 Rosalind's $15/$75 list price means volume access is a budget line rather than an afterthought. Whether an evaluator can get ordinary list-price access at that volume, or needs a separate agreement, is a practical constraint that sits outside the published price itself.
Open weights change who can do the work
Mistral Large 4's weights were released on 29 October 2026 (the model was first shown as a public preview on 6 October 2026), free for eligible self-hosting. That matters for evaluation in a way the price list does not show. With open weights you run the model on your own hardware, log whatever you like, and keep the probe set inside your own infrastructure. With a closed API you get generated text back and nothing else, and your red-team prompts leave the building.
For a European consortium that is a data-protection question as much as a technical one. Hostile prompts can contain personal data scraped from a target organisation, so routing them to an endpoint outside the EU turns an internal test into a transfer with a legal basis attached.
Compute is the other constraint. Our own AI Arena rig, a single RTX 5060 Ti with 16 GB of VRAM, cannot hold a frontier-class model at all. White-box evaluation at this scale needs racks, which is one reason the work goes to a systems integrator rather than a university department.
What European teams can take from it
If you build on these APIs, the practical consequence is documentation. Article 50 already requires you to disclose AI interaction and synthetic content. Buyers in regulated sectors now ask for evaluation evidence alongside model cards, and a Commission-backed assessment programme gives them a reference point even before any results exist.
For providers, the open question is access. An external evaluator cannot score a model it is not allowed to query at volume. Ordinary list pricing is a starting point for budgeting, but volume terms and access conditions still decide whether a campaign is feasible.
The contract's published scope is assessment of advanced AI models. What the consortium will publish, and when, is not part of the announcement.
Does the consortium's work replace a provider's own safety testing?
No. It sits alongside it. Providers still document their own compliance under the AI Act and the GPAI codes of practice. External assessment produces evidence someone outside the vendor can check, which is a different thing from a self-declared model card.
Can a small team run this kind of evaluation itself?
Partly. Open-weight models such as Mistral Large 4 can be tested on your own hardware, though frontier-scale models need far more VRAM than a desktop GPU offers. Closed models can only be probed through their APIs, at published list prices, and within the provider's terms of service.
Which AI Act deadline applies to high-risk systems now?
Annex III systems, which cover recruitment, credit scoring, biometrics and education, moved to 2 December 2027 under Regulation (EU) 2026/1744. Annex I systems embedded in regulated products moved to 2 August 2028. Transparency rules under Article 50 have applied since 2 August 2026.
