09 / 11
Using artificial intelligence without losing judgement
Artificial intelligence was one of the most practical and animated sessions of the day. Patients are already using tools such as ChatGPT, Gemini, Claude and NotebookLM to search for medical information, prepare for consultations, understand reports and explore questions they may not feel able to ask elsewhere.
The discussion did not attempt to tell patients simply to use or avoid AI. Instead, it focused on how these tools can be used more safely and critically.
A prompting framework built around the word CAUTION was introduced:
- C – Context: provide relevant background so that the answer is not generic.
- A – Alternatives: ask about different options and perspectives.
- U – Uncertainty: ask what is known, what is not known and how strong the evidence is.
- T – Trade-offs: request both potential benefits and disadvantages.
- I – Information missing: ask what additional details would be needed for a more useful answer.
- O – Overall picture: consider how one question fits into the person’s broader health and care.
- N – Next questions: ask what should be discussed with a healthcare professional and what follow-up questions may be helpful.
The quality of an AI-generated answer depends heavily on the prompt. Search-engine habits— entering only a few keywords—often produce answers that are too broad. Follow-up questions and requests for sources can improve the result.
However, several cautions were strongly emphasised. First, patients should protect their privacy. Exact names, dates of birth, addresses and other identifying details are rarely necessary. Approximate ages or ranges may provide enough context without making the information directly identifiable.
Second, generative AI can “hallucinate”: it may provide false information, invent sources or present uncertain statements with inappropriate confidence. Different systems may give contradictory answers to the same question, and even the same system may respond differently on different occasions.
Participants suggested asking the tool to show its sources, checking those sources directly, comparing answers and using systems grounded in selected documents—such as a guideline uploaded to a retrieval-based tool. But limiting a system to one guideline also has disadvantages: guidelines cannot contain every clinical situation or the accumulated experience of specialists. Human expert review remains indispensable.
Third, AI systems may be overly agreeable or reassuring. A pleasant answer is not necessarily an accurate one. Users should explicitly ask for counterarguments, weaknesses, uncertainties and reasons the initial conclusion might be wrong.
Most importantly, AI should support—not replace—the relationship between patients and healthcare professionals. It can help patients formulate questions and prepare for appointments, but it cannot perform a physical examination, understand every element of an individual medical history or take responsibility for a clinical decision. Urgent or worrying symptoms still require direct medical advice.
Participants saw a clear role for patient organisations in developing practical prompt toolkits, plain-language guidance and trusted collections of source material. Follow-up workshops were proposed so that patients could test prompts, compare answers and contribute to the further development of the CAUTION framework.