Are LLMs accurate in giving child health information?

What the evidence says

Researchers have started testing chatbots like ChatGPT and Gemini on children's health questions, and the results are mixed. On general questions, like common treatments for everyday childhood illnesses, one study found ChatGPT gave correct information about 82 percent of the time. But it made more mistakes when it came to specific things like medication names, surgery details, and exact disease definitions.

The picture gets worse for harder problems. When researchers gave ChatGPT complicated pediatric case studies, similar to medical mystery puzzles doctors use for training, it got the final diagnosis wrong most of the time, with error rates as high as 83 percent in one study. This suggests chatbots are much better at answering simple, common questions than at figuring out unusual or complex symptoms.

One important finding is what researchers call a trust-accuracy gap. In a study where medical experts and parents both reviewed the same chatbot answers about vitamin D, food allergies, and sleep problems, experts found real inaccuracies in more than a quarter of the responses. But most parents still rated the answers as clear, helpful, and trustworthy. This means parents may not always notice when a chatbot's answer is wrong.

There is currently no specific Canadian guideline from groups like the Canadian Paediatric Society on using AI chatbots for children's health questions. A Canadian health technology report noted that these tools are becoming more available, but solid clinical evidence about how well they actually work is still limited, and there is no formal approval process for them in Canadian health care.

The bottom line for parents: chatbots can be a helpful starting point for simple questions, like general information about a common condition, but they should never replace a real conversation with your child's doctor, especially for anything serious, unusual, or urgent.