Why safety training can make AI text easier to spot
A raw pre-trained model learns to predict the next token from the full diversity of human writing — including grammatical errors, digressions, stylistic quirks and contradictions. Then comes post-training: reinforcement learning from human feedback, safety alignment, preference optimization. The model is gently but firmly nudged toward "preferred" answers: polite, structured, balanced, hedged.
Pangram's CTO describes the result as a form of mode collapse — the model's output distribution narrows and drifts away from natural human text, leaving a statistical fingerprint. Detectors don't work because models are "dumb." They work because alignment can make some model outputs unnaturally uniform in particular comparisons.
That complicates the popular claim that "AI writing is indistinguishable from human writing." A model may have the capacity to write like a human, but post-training changes what deployed systems actually produce. Whether that difference is easy to detect is an empirical question, and it varies by model, language, domain and detector.
What benchmark results do and don't show
Pangram’s essay cites detector benchmarks on standard AI-generated passages. The figures reported there are tied to the specific study’s detector versions, model versions, genres, sample sizes and definitions of false negatives and false positives; they are not a universal accuracy rate for all AI text. A detector that performs well on a fixed sample of ordinary model output can behave much worse on edited, translated, fine-tuned or deliberately style-mimicking text.
The EU AI Act section: what actually falls on providers and deployers
Since 2 August 2026, Article 50 of the EU AI Act has applied. Its transparency duties are split, not a single detector rule. Providers of AI systems intended to interact directly with natural persons must design those systems so people are informed they are interacting with AI unless that is obvious. Providers of generative AI systems must ensure synthetic text outputs are marked in a machine-readable format and detectable as artificially generated or manipulated, as far as technically feasible, subject to exceptions such as assistive editing that does not substantially alter the input. Deployers have separate duties, including disclosure for deep-fake image, audio or video content.
The law does not mandate a detector for every output, and detection is not a legal verification layer. Machine-readable marking is a provenance and transparency tool; where a tag is absent, a detector score cannot establish provenance or compliance on its own.
Where detectors are weaker: style mimicry and domain shift
Detection becomes harder when someone does not simply ask for "text," but explicitly prompts the model to imitate a particular author’s style or a constrained genre. Scientific writing is a useful example: it is already stylistically constrained, with formulaic structure, fixed vocabulary and predictable hedging. When a model imitates that style, it can move closer to the human distribution, and detectors that rely on distributional differences lose some of their grip.
For European publishers using AI in research papers, grant proposals or regulatory filings, this is not an academic exercise. Detection alone cannot guarantee transparency. Where Article 50(2) applies, machine-readable marking is part of the compliance backbone, but it is not the same as a detector and it has exceptions.
False positives are the real cost
Detection’s worst failure is not only what slips through the net — it is who gets wrongly caught in it. Published benchmarks of human-authored text, including samples written before LLMs were widely available, have shown that some detectors flag human work as AI-generated while others do not. False-positive risk depends on the detector version, score threshold and input sample, so comparisons should name all three.
Apply that to an editor's inbox: real human writers can be accused of using AI. In high-stakes EU uses, a detector score should not be an automated gatekeeper without human review, technical documentation and a clear threshold policy.
What this means for European developers
From our production and AI Arena experience, we see that detectability varies by model version, fine-tuning, prompt and domain. Some strongly aligned cloud models produce more uniform outputs under test conditions; some locally run or fine-tuned models produce more varied text. But we have not benchmarked specific named open-weights releases for this article, so we are not claiming that any particular release is inherently less detectable.
If your EU startup runs open-weights models in a user-facing service, the AI Act’s transparency duties do not disappear because the model is open source. The exact obligation depends on whether you are the provider placing the system on the market or the deployer using it in your own service. Disclosing AI interaction and applying machine-readable marking where Article 50 applies are process questions. They cannot be replaced by hoping a detector will catch what your own model produced.
The useful question is no longer "can detectors really work?" but "under which conditions do particular detectors work?" The European response is not to trust any single detector. It is to pair provenance marking and transparency duties with detection as a secondary, human-reviewed signal. That is not a revolution — it is good engineering and cautious legal design.
Is AI text detection reliable enough for production use?
On specific published benchmarks, high-performing detectors can identify ordinary AI-generated text from particular models with low false-negative rates. But performance varies by detector version, model version, genre, sample size and how false negatives and false positives are defined. A single benchmark is a snapshot, not a general guarantee.
Can a human author be falsely accused of using AI?
Yes. Some detector comparisons have shown that certain tools flag human-authored texts as AI-generated while others do not. False-positive risk depends on detector version, threshold and input distribution. Detector choice and human review matter a lot in practice.
Does the EU AI Act require AI text detection?
No. Article 50 requires providers of certain AI systems to inform people they are interacting with AI and to mark synthetic outputs in a machine-readable format where applicable, with exceptions. It does not require organisations to run standalone AI-text detectors. Detection cannot establish provenance; it is only one possible verification signal.
