General assistants such as ChatGPT, Gemini, Copilot and Claude are good with language. A health question is not only a language problem. It involves a body that has to be examined, a history nobody typed in, and a decision that carries a professional licence behind it.
What follows is not an argument about whether AI belongs in medicine. It is about the ordinary case: a person in Europe types a symptom into a chat box and gets an answer in seconds.
What a chatbot does with your question
Describe a symptom and the model does not search a database of diseases. It produces text that fits the pattern of your question and of the material it was trained on. That makes it very good at sounding like a clinician and unreliable at knowing when it is wrong. OpenAI's own researchers argued in a 2025 paper that models hallucinate partly because the training and scoring setup rewards a confident guess over an admission of ignorance.
A second effect matters more in health. These models tend to agree with the person asking. In April 2025 OpenAI rolled back an update to GPT-4o after users reported that it flattered them and went along with them. Open with "I think this is just stress" and you have made "probably stress" a more likely answer.
There is also no record. A chatbot does not know your last blood test, your medication list, your allergies or what your GP said in March. It answers the sentence in front of it.
Where it helps
Used as a writing and explaining tool, a chatbot earns its place. It can turn a discharge letter into plain language, list the questions worth asking at an appointment, explain what a test measures, help you build a timeline of symptoms, and describe how a referral works in your country's health system.
There is evidence for part of this. In a 2023 study published in JAMA Internal Medicine, blinded reviewers rated ChatGPT's replies to patient questions higher than doctors' replies in 78.6% of cases, on quality and on empathy. The questions came from a public medical forum. Reviewers judged how the answers read, not whether they were right for that particular person. Read carefully, the study supports a narrow claim: with a clear prompt, a model can explain things well and kindly. It does not support letting it work out what is wrong with you.
Where it fails
Four things are missing from every consumer chat window: an examination, your records, accountability and triage. Someone has to look at the rash, press on the abdomen, listen to the chest. A model cannot do any of that, and it will rarely say so unless you ask.
Language adds another gap. Models tend to be weaker in languages with less training data, which in the EU include Czech, Slovak, Lithuanian, Latvian, Estonian and Maltese. Ask in your own language and then translate the reply, and you have added a second step where meaning can shift.
What never to ask
| Question | Why it is a problem | What to do instead |
|---|---|---|
| "Do I have cancer?" "Is this serious?" | The model cannot examine you, so it guesses while sounding certain. | Write the question down and take it to your doctor. |
| "How much of this medicine should I take?" "Can I stop my blood thinners?" | Dosing and withdrawal depend on weight, kidney function and interactions. A wrong number is not a wording error. | Ask a pharmacist or the prescriber. |
| Photos of a rash, a mole, a wound or an X-ray | No consumer chatbot in the EU is authorised to diagnose from an image. | Show it to a clinician, in person or through your health provider's own service. |
| Full medical reports with your name, national ID and insurance number | That is identifiable health data, the most protected category under GDPR. | Paste only the line you do not understand, with identifiers removed. |
| Chest pain, stroke signs, breathing trouble, heavy bleeding, thoughts of self-harm | These need a person with equipment within minutes, not a text reply. | Call 112, the EU emergency number. |
| A baby's or child's symptoms | Paediatric assessment depends on age and weight, and a small error matters more. | Call your paediatrician or an out-of-hours service. |
The European rules that apply
Two EU laws shape what a chatbot is allowed to be, and neither of them turns a general assistant into a doctor.
GDPR. Information about your physical health is special category data under Article 9 of the GDPR. When you type symptoms, medication names or a diagnosis into a chat window, you are handing over exactly that. The company processing it needs a legal basis, usually your consent under its terms, and it disappears when you close the account. You have the right under Article 15 to ask what personal data a provider holds about you, and the right to complain to your national data protection authority. In December 2024, Italy's Garante fined OpenAI €15 million over how ChatGPT processed personal data. OpenAI said it would appeal.
Medical device rules. Under the Medical Device Regulation (EU) 2017/745, software counts as a medical device only when the manufacturer intends it for a medical purpose: diagnosis, prevention, monitoring, prediction, prognosis, treatment or relief of disease. ChatGPT, Gemini, Claude and Copilot make no such claim. They carry no CE mark as medical devices, which means no notified body has checked their health answers and no medical device vigilance applies when one of them is wrong.
The AI Act. Since 2 August 2025, providers of general-purpose AI models have had duties under the AI Act: technical documentation, a copyright policy, a public summary of training data. Those duties concern the model, not the correctness of its health advice. AI that is itself a medical device falls under Annex I of the Act and is treated as high-risk, with those obligations applying from 2 August 2027. That is the regulated path. A chat box is not on it.
The World Health Organization's 2024 guidance on large multimodal models names bias, misinformation and privacy among the main risks in health use, and asks developers for transparency and clear accountability. Consumer chatbots publish neither in a form a patient can check.
Six rules that hold up
- Use it before the appointment, not instead of it. Ask for a list of questions to take to your GP.
- Do not lead with your own diagnosis. Describe symptoms in time order and leave the conclusion open.
- Strip identifiers. No name, national ID, insurance number, address or employer.
- Treat every dose as unverified. Dosing and stopping medication go to a pharmacist or the prescriber.
- Keep the conversation. If an answer worried you, show the chat to your doctor rather than summarising it from memory.
- Call 112 for emergencies. Chest pain, stroke signs, breathing trouble, heavy bleeding or thoughts of self-harm go to the emergency number.
Free tiers and paid plans
All four major assistants are available across the EU. Free tiers are enough for explaining, summarising and preparing questions. Paid consumer plans cost roughly €20 to €23 a month in most member states once VAT is added, and the exact figure moves with the national rate, so check the provider's local pricing page. Paying gets you a faster model and higher limits. It does not change what the tool is in regulatory terms: a text service with no CE mark and no duty to your doctor.
Common questions
Can I use AI to understand a diagnosis I have already been given?
Yes, for the words. Ask it to explain a term in plain language, to list what a test measures, or to draft questions for your next appointment. Check anything that changes a decision with the clinician who gave you the diagnosis. The model does not have your file and cannot see how you are doing.
Is it legal in the EU to paste my lab results into ChatGPT?
The GDPR does not forbid you from sharing your own health data. It puts conditions on the company that processes it. The practical risks are that the provider may use the conversation to improve its models, and that a full report carries your name and insurance number. Remove the identifiers and paste only the part you do not understand.
Where can I complain if a chatbot gave me dangerous advice?
General chatbots do not sit under medical device vigilance, so there is no medical regulator to report them to. Your routes are the provider's support and, for a data protection issue, your national supervisory authority. If the advice led to a purchase or a contract, a national consumer body is the next step.
