Hallucinations

Ethical considerations

One school librarian identified their work with students about using AI. “I would primarily caution them about hallucinations, copyright, and bias.” This is because “AI engines are designed to give you the answer that you want. To do so, they may hallucinate and answer that may not have basis in reality - either in whole or in part. AI can make up articles, authors, quotes, and events in an effort to generate what the user is requesting. As such, AI products should be thoroughly fact checked - which builds amazing research skills.”

(Craddock and Wilson, 2024) 

Hallucinations in context 

LIS educator Amanda Harrison makes a connection with previous concerns about another information source silo:

AI-generated content can be rife with errors, often called “hallucinations,” although they might more clearly be considered glitches. Identifying the “hallucinations” in AI-generated content can spark curiosity and encourage users to engage with AI and critically evaluate the information it produces. Since AI-generated content is based on statistical likelihood rather than verified accuracy, several writers and leaders in educational technology have suggested that AI is the new Wikipedia: a helpful starting point for learning, but one that requires deeper, more rigorous research to follow.

(Harrison, 2025, page 4)

Hallucinations - some can be minor but others serious

Hallucinations in AI can be thought of as bluffing, which sounds innocuous, but isn't and is something that could be rectified in the technology (by saying: I don't know) though this doesn't fit with AI-developer business model (Zhou, 2025). Researchers at Cornell and Washington universities identified that when AI tools are used in everyday life, some hallucinations “may be minor – when a chatbot gives the wrong answer to a simple question, the user may end up ill-informed. But in other cases, the stakes are much higher. From courtrooms where AI software is used to make sentencing decisions to health insurance companies that use algorithms to determine a patient's eligibility for coverage, AI hallucinations can have life-altering consequences. They can even be life-threatening. Autonomous vehicles use AI to detect obstacles, other vehicles and pedestrians.” (Choi and Mei, 2025)

Hallucinations might make citations too perfect

Here is an interesting and helpful explanation about AI generated sources. 

Issues for librarians

No answer is better than a false answer

Reporting in a popular science magazine, rather than a library science publication, in a short but punchy piece with a very specific title Librarians can’t keep up with bad AI: From false sources to hallucinations, it’s become a major problem, the piece set the scene: 

Generative artificial intelligence continues to have a problem with hallucinations. Although many responses to user queries are largely accurate, programs like ChatGPT, Google Gemini, and Microsoft Copilot are still prone to offering made-up information and facts. As bad as that is on its own, the issue is further complicated by a tendency for these AI programs to produce seemingly reputable, yet wholly imaginary, sources. But as annoying as that is for millions of users, it’s becoming a major issue for the people trusted to provide reliable, real information: librarians.

"For our staff, it is much harder to prove that a unique record doesn’t exist," Sarah Falls, a research engagement librarian at the Library of Virginia, told Scientific American.

15% of reference inquiries at the particular university library resulted from AI generated false sources. The library strategy was to point students to go to the source - and prompted students to think about their critical thinking skills - and the report identified that there may be a future need to limit support to students given that it is a recurring issue. (Paul, 2025)

Inaccuracies and lack of trust

One study looked specifically at the ability of ChatGPT to provide reliable references for research proposals. It analysed 178 references and found that over a third of the references did not have a DOI (Digital Object Identifier) and 28 references could not be traced online. Other references were not research articles, but secondary sources. The study concluded that “the 28 articles that neither turned up on an extensive internet search nor had a functional DOI are undoubtedly the product of AI hallucination” and “AI hallucination usually occurs due to adversarial examples such as varied input data that confound the AI systems into misclassifying and misinterpreting them resulting in inappropriate and hallucinating output. AI hallucination is a problem because it hampers a user’s trust in the AI system”. (Athaluri et al, 2023)

Another study that looked purely at ChatGPT considered that, whilst there were positive uses of the AI tool, a balance should be maintained for accuracy and ethical issues and (again) there was a need for library/ian education to students in the areas of critical thinking, fact checking and verification skills. (Oladokun et al, 2025).

Verification skills are sophisticated

Older secondary school students find verifying information from AI sources because there are nuances and subtleties and skills and knowledge that comes from experience, plus lack of experience in independently evaluating sources and a perhaps latent trust in seemingly well-written sources (Reyes, 2026). The verifying process can be complex - a lot of advice is aimed at students in HE and new professionals, where there is an assumption of context and previous knowledge.