robots.txt (DisCo's analysis via Google News).
Why a "technical detail" affects every creator
Since 2 August 2026, the EU AI Office and national authorities have actively enforced obligations on general-purpose AI (GPAI) providers: mandatory risk assessments, technical documentation and transparency rules. One of those obligations comes from Article 4 of the EU Copyright Directive (CDSM): if a rights holder has expressly reserved their right to text and data mining, AI companies must respect it. Under the AI Act, GPAI providers must put in place a policy to comply with EU copyright law — including those opt-outs. In principle, that protects photographers, writers, musicians, illustrators and small news sites. In practice, there is one giant open question: what does a valid, enforceable opt-out look like in machine-readable form? The European Commission has to clarify this through the AI Act's implementation — and the technical choices it makes will decide whether your opt-out is a real shield or a paper tiger.Seven options on the table, none deemed fit
DisCo's analysis points to a telling figure: at least seven opt-out solutions have been explored in EU policy discussions, and the group argues none of them is fit for purpose as a mandated standard. Two names keep coming up.TDMRep is a protocol designed specifically for "text and data mining reservations" at the level of individual web pages or files. CAWG (from the creative community's Copyright in the Age of AI working group) proposes a different layer of signalling. Both aim to tell crawlers "this content may not be mined for AI training." Alongside them, there is talk of asset-level metadata — embedding rights signals directly into image, audio or video files.
DisCo warns that forcing these protocols onto Europe's web ecosystem would fragment the web, increase compliance costs for small AI developers, and undermine the very European AI innovation the EU wants to foster. The better path, the group argues, is to support international, consensus-driven processes such as the IETF, where protocols like robots.txt are already being adapted to the age of generative AI.
What European open-source developers already experience
This is not an abstract debate. In late June 2026, a study commissioned from Centrum Cyfrowe and Open Future by the COMMUNIA network examined how European open-source AI projects cope with Article 4 opt-outs. The researchers conducted eight in-depth interviews with technical leads from European AI initiatives, including OpenEuroLLM, Pleias, PLLUM and SOOFI. Their finding: developers spend significant effort parsing unstandardised, inconsistent opt-out signals — often human-readable sentences or half-implemented metadata — and still face legal uncertainty about whether they have complied. One practical consequence: a responsible open-source team that wants to respect rights holders can be slower and more cautious than a large foreign model provider with a legal army. That is not the level playing field Brussels promised.The creative sector is watching closely
The stakes on the other side are just as real. The EU creative sector generates roughly 6.9% of the Union's GDP — a figure European Parliament resolutions on AI copyright protection have repeatedly cited. In a notable show of political will, the Parliament adopted its recommendations on AI copyright transparency and protection with 460 votes in favour, 71 against and 88 abstentions. Publishers, broadcasters and collective management organisations want a system that is actually enforceable — not a polite request that crawlers can ignore. Their worry is the mirror image of the developers': if an opt-out only works when voluntarily obeyed, the law becomes a suggestion.Why the standards fight is a European problem
Here is the deeper tension. If Brussels picks a European-only protocol, global AI companies — many of which train on data gathered worldwide — may find it hard to honour distinct regional signals, or may simply design crawlers around the least common denominator. If, instead, Brussels waits for international consensus at the IETF, the process is slower and less predictable. But a global standard it helps shape will be honoured far more widely than a Brussels-only mandate. DisCo's position is clear: the Commission should refrain from unilaterally imposing technical protocols and instead engage in international standardisation efforts — particularly updatingrobots.txt, the 30-year-old convention that web crawlers have respected for decades. Supporting that process, rather than competing with it, may give European creators the broadest possible protection in practice.
None of this is academic. The Commission's choice will determine whether a graphic designer in Prague can effectively say "not my work" to hundreds of AI trainers at once — or whether that statement drowns in a tower of competing technical formats. For European creators and for Europe's own AI ambitions, the next few months of standardisation meetings might matter more than the next model release.
If I add "no AI training" to my website's terms of use, is that enough?
Under the EU Copyright Directive's Article 4, a rights reservation is valid if expressed "in an appropriate manner", such as machine-readable means. For AI training purposes, the AI Act expects providers to respect reservations expressed pursuant to that article. But the enforcement gap is exactly what is being debated: without an agreed technical standard, AI developers may not reliably detect or honour your statement. Simply writing it in plain text is unlikely to be enough once the rules are fully tested.
What is the difference between robots.txt, TDMRep and CAWG?
Robots.txt is the longstanding protocol websites use to tell search-engine and other crawlers which parts of the site they may access. TDMRep is a newer mechanism specifically designed to carry text-and-data-mining reservations through web requests, while CAWG proposes signals embedded with creative content. The dispute is whether Brussels should mandate one of the newer formats or let international standards bodies adapt robots.txt and related protocols for AI training.
Does this affect small European AI companies too?
Yes. The June 2026 study by Centrum Cyfrowe and Open Future interviewed technical leads at European open-source projects such as OpenEuroLLM, Pleias, PLLUM and SOOFI and found that unstandardised opt-outs create operational friction and legal uncertainty. If the rules become too complex to parse, smaller developers may struggle to comply — while larger providers with more legal resources adapt more easily.