One model, two languages, two truths

Researchers from several universities recently tested how the most popular large language models — OpenAI's ChatGPT and Anthropic's Claude — respond to politically sensitive questions about China. The key difference? Simply changing the query language from English to Chinese fundamentally transformed the answers.

The study focused on questions concerning topics such as Taiwan, Xinjiang, Tibet, human rights, and the role of the Chinese Communist Party. The results were consistent across both tested models: in English, they provided relatively objective, fact-based answers with multiple perspectives. In Chinese, however, the answers slipped into evasive phrasing, official Chinese rhetoric, or outright claimed there was "not enough information" to answer.

According to the study's authors, this is not a technical glitch but a systematic pattern suggesting that the models have been — whether intentionally or as a result of training data — "programmed" to behave differently depending on the language and geopolitical context.

Why does this happen? Three main reasons

Language models are not neutral — they learn from the data people provide them. And that's where the problem lies.

1. Training data is not linguistically balanced

Models like GPT or Claude are trained on vast amounts of internet text. But the internet looks different in Chinese than in English. The Chinese web is strictly regulated — content that contradicts the official line simply isn't available. A model that learns from Chinese texts also learns from their limitations. It thus gets a distorted picture of reality because alternative viewpoints are simply missing from the training data.

2. Alignment and safety filters

Both companies — OpenAI and Anthropic — use a technique called reinforcement learning from human feedback (RLHF), i.e. learning based on feedback from human annotators. If annotators evaluating Chinese responses prefer "safe" and politically non-confrontational phrasing, the model learns to provide exactly that. The result is that the model's alignment is linguistically asymmetric — it behaves according to different safety standards in different languages.

3. Political and commercial pressure

The Chinese market is lucrative for technology companies, but entering it comes with clear rules. Chinese laws require AI systems to adhere to "socialist core values." Although neither OpenAI nor Anthropic officially operate in mainland China, their models are available via API and are used by developers worldwide. The question is: to what extent do companies preemptively adapt their models to Chinese regulations?

How do individual models compare?

The study also found differences between the tested models. Anthropic's Claude proved somewhat more "cautious" in Chinese — it more often refused to answer questions that could be perceived as controversial. ChatGPT, on the other hand, provided more detailed answers in English but slipped into official rhetoric in Chinese.

For comparison: open-source models like Meta's Llama or Europe's Mistral exhibit this pattern to a lesser extent — likely because their alignment process is less aggressive and their training data is more diverse. On the other hand, Chinese models like DeepSeek or Qwen answer fully in line with the government line in Chinese — and in English too, just somewhat less noticeably.

What does this mean for trust in AI?

This study opens up a much broader topic than just China. If a language model changes its answers based on the query language, it means it has no consistent relationship with truth. It is not a tool that objectively answers questions — it is a system that adapts to the expectations and norms of the target language audience.

For a European user, this means that information obtained from an AI assistant is not universally valid. If you work with multilingual data or enter the same query in different languages, you can get substantially different — and potentially contradictory — answers.

European context: AI Act and transparency

This finding has direct relevance to the EU AI Act, which has been gradually taking effect since February 2025. European legislation requires high-risk AI systems to be transparent about their limitations. Answer dependency on query language is exactly the type of limitation users should be informed about.

For Czech companies and institutions considering deploying large language models, this means one thing: test the models in the languages you will use them in. Accuracy praised in English does not mean the model will be equally reliable in Czech — let alone on questions touching on geopolitics.

The Czech Republic, as an EU member, also has a specific historical experience with censorship and propaganda. The fact that AI models can "switch" their answers based on language should serve as a warning signal for us.

How to deal with this? Practical tips for users

If you use ChatGPT, Claude, or Gemini for research on geopolitically sensitive topics, follow a few principles:

• Ask the same question in multiple languages and compare the results. The differences will tell you where the model is "dodging."
• Use multiple models simultaneously. Comparing ChatGPT, Claude, and say Mistral will reveal where each model fails.
• Don't rely on AI as a primary source of information on politically sensitive topics. Always verify facts from independent sources.
• Pay attention to how the model refuses to answer. "I'm sorry, I cannot answer this question" is not a neutral statement — it is the result of safety filters that someone configured.

The study of language asymmetry in AI models is further proof that artificial intelligence is not a neutral tool — it carries within it the values, biases, and limitations of its creators and the data it learned from. And as long as that's the case, critical thinking remains the most important skill we need when working with AI.