How Reliable Is AI? A Deep Dive into the ‘Hallucination’ Mechanism of Large Models

Have you ever encountered a situation where you ask an AI a question, and it provides you with a particularly detailed and logical-sounding answer? However, when we fact-check it, we find that the information is completely fabricated?

This is the famous ‘AI Hallucination’ phenomenon.

How Reliable Is AI? A Deep Dive into the 'Hallucination' Mechanism of Large Models

Scroll up and down to see more, image source: 河森堡 新浪微博

Why does AI hallucinate? Let’s unravel this mystery together today.

Why Does AI Hallucinate?

AI hallucination refers to the phenomenon where AI generates information that seems reasonable but is actually false, most commonly manifesting as the fabrication of non-existent facts or details.

Just like when we encounter questions we can’t answer in an exam, we try to infer the answer using our known knowledge. When AI faces missing or uncertain information, it attempts to fill in and infer based on its ‘experience’ (training data).

This is not because it wants to deceive us, but because it is trying to complete the task using the patterns it understands.

1

Prediction Based on Statistical Relationships

Because AI (especially language models like ChatGPT) learns the statistical relationships between words through a vast amount of training data. Its core goal is to predict the most likely next word based on context, rather than truly understanding the question or content. So AI essentially generates content by maximizing probabilities, not through logical reasoning.

In simple terms, AI is like a wise person who has read extensively, gaining knowledge from a massive amount of texts and materials. However, it does not truly understand this knowledge; it merely ‘predicts’ the next most suitable word by finding statistical relationships and patterns between words. That is, AI guesses the next most likely word based on the numerous examples it has learned.

However, sometimes the model can ‘guess wrong’. If there is a slight deviation in the beginning, the subsequent content can snowball into a much larger problem.This is why AI sometimes starts with a small mistake and ends up weaving a completely fabricated story.

2

Limitations of Training Data

Since AI does not have real-world experiences, all its ‘cognition’ comes from training data. However, training data cannot possibly encompass all information in the world and may sometimes even contain erroneous information. It is like a person answering questions based only on the books they have read; if the books contain errors or lack knowledge in certain areas, incorrect judgments can easily arise.For example, during earlier instances of AI hallucination, it might have learned the facts that ‘Beijing is the capital of China’ and ‘Paris has the Eiffel Tower’. When we ask it, ‘What famous buildings are in Beijing?’, it might incorrectly mix these facts and say, ‘Beijing has the Eiffel Tower.’

3

Overfitting Problem

Due to the vast number of training parameters in large models, they can suffer from an ‘overfitting’ problem. That is because they remember too many erroneous or irrelevant details, making AI overly sensitive to noise in the training data, ultimately leading to hallucinations.

4

Limited Context Window

Due to technical limitations, although the context window of large models is becoming larger (for example, handling 64k or 128k tokens), they still understand text within a limited range.This is like looking at a book through a small window; you can’t see the whole content of the book, which can lead to misunderstandings.

5

Design for Generating Fluent Answers

Many large models are now designed to provide fluent answers. When they are uncertain about a question, instead of saying ‘I don’t know’, they tend to fabricate what seems like a reasonable answer based on existing knowledge.These various factors combined have resulted in the serious AI hallucination problem we see today.How Reliable Is AI? A Deep Dive into the 'Hallucination' Mechanism of Large Models

Image copyright, reproduction may lead to copyright disputes

How Can We Reduce AI Hallucinations?

AI seems very convenient, but AI’s serious ‘nonsense’ can be quite a headache. The information often needs to be repeatedly verified, sometimes making it less practical than directly searching online. So, how can we deal with AI hallucinations? We summarize the following methods to help everyone.

1

Optimize Questions

To get accurate answers, the way you ask questions is crucial. Communicating with AI also requires clarity and specificity, avoiding vague or open-ended questions. The more specific and clear the question, the more accurate the AI’s answer will be. At the same time, when asking questions, providing enough context or background information can also reduce the likelihood of AI making random guesses.Summarized as prompt techniques, here are four ways to ask:

1. Set Boundaries: “Please strictly limit to research published in the journal Nature in 2022”;

Example: “Introduce the development history of ChatGPT” → “Please introduce the development history of ChatGPT based solely on OpenAI’s official documents from 2022-2023.”

2. Mark Uncertainty: “For vague information, please mark ‘This is speculative content’”;

Example: “Analyze Tesla’s market share in 2025” → “Analyze Tesla’s market share in 2025; for unofficial data or speculative content, please mark [speculative content].”

3. Break Down Steps: “First, list the confirmed facts; second, conduct a detailed analysis”;

Example: “Evaluate the impact of AI on employment” → “Please evaluate the impact of AI on employment in two steps:

1) First, list specific impact cases that have already occurred;

2) Based on these cases, conduct a future trend analysis.”

4. Specify Constraints: Clearly instruct AI to answer based on existing facts without making guesses..

Example: “Predict the real estate market trends in 2024” → “Please analyze based solely on the actual real estate data from 2023 and relevant policies that have been implemented, without including any speculative content.”

2

Batch Output

Since AI content is generated based on probabilities, the more content generated at once, the higher the probability of AI hallucinations. We can proactively limit the amount of output.For example, if I want to write a long article, I would say to AI: “Let’s write it section by section, starting with the introduction. Once I’m satisfied with this part, we’ll continue to the next section.” This not only makes the content more accurate but also makes it easier to control the quality of the generated content.

3

Cross-Validation

To improve the reliability of AI answers, a practical method is to use “multi-model cross-validation.” Using an AI aggregation platform allows multiple AI models to answer the same question simultaneously. When faced with questions requiring rigorous answers, this feature is activated, enabling different large models to participate in discussion, thereby gaining a more comprehensive understanding through comparison of their answers.How Reliable Is AI? A Deep Dive into the 'Hallucination' Mechanism of Large Models

Click to enlarge, image source: provided by the author

For instance, the “multi-model collaboration” feature of the Nano AI search platform allows different AI models to perform their respective roles, forming an efficient collaborative team.Let DeepSeek R1, which excels in reasoning, handle analysis and planning, then have Tongyi Qianwen correct and supplement it, and finally pass it to Doubao AI for organization and summarization. This ‘expert group’ style of collaboration not only enhances the credibility of the content but also provides more comprehensive and in-depth insights.How Reliable Is AI? A Deep Dive into the 'Hallucination' Mechanism of Large Models

Scroll up and down to see more, image source: provided by the author

4

RAG Technology

AI is a smart but forgetful entity. To make it perform more reliably, we can equip it with a super encyclopedia that it can refer to at any time to answer questions.This ‘encyclopedia’ is the core of RAG, allowing AI to find relevant information from reliable sources before generating answers. This way, AI is less likely to ‘talk nonsense’.Currently, RAG technology is commonly used in specialized fields such as healthcare, law, and finance, enhancing the accuracy of responses through knowledge base construction.Of course, in practical use, in high-risk fields like healthcare, law, and finance, the content generated by AI must still be reviewed by professionals.

5

Leveraging AI Hallucinations

Finally, let’s discuss a benefit of AI hallucinations.Many times, AI hallucinations can also spark creative ideas! Like a whimsical artist, unbound by conventional thinking, it can come up with surprising ideas.Just look at DeepSeek; it may indeed experience hallucinations more easily than ChatGPT and Claude, but this year, DeepSeek’s powerful creativity has contributed to its breakout success.Sometimes, rather than viewing AI hallucinations as flaws, we might consider them a source of creativity!In writing, artistic creation, or brainstorming, these ‘leaps of thought’ may actually help us open doors to new worlds.How Reliable Is AI? A Deep Dive into the 'Hallucination' Mechanism of Large ModelsImage copyright, reproduction may lead to copyright disputesThe essence of AI hallucination—AI, in the fog of knowledge, sometimes creates ‘shadows’ that seem real but are actually illusory. However, like any tool, the key lies in how we use it.When we learn to communicate with AI correctly, effectively utilize its creativity while maintaining independent thinking, AI can become a valuable assistant rather than a ‘smooth-talking liar.’After all, in this era of AI and human progress, it’s not about blaming AI for its imperfections, but about learning to collaborate better with it.

Leave a Comment