OpenAI has a confession to make: even the smartest chatbots are still making stuff up.
In a new research paper, the company asks the question many of us have screamed at our screens: why do large language models like GPT-5 still hallucinate?
And the short answer is: because guessing is baked into their DNA.
Hallucinations, OpenAI explains, are those totally confident but totally wrong answers AI loves to serve up.
To prove the point, researchers ran a little experiment on one of the most popular chatbots (guess which one).
When asked for the title of co-author Adam Tauman Kalai’s PhD dissertation, the bot spat out three different titles, all fake. Same deal with his birthday: three dates, zero accuracy.
So how can a machine that crunches billions of data points still bomb on basic facts?
OpenAI says it comes down to how these models are trained. During pretraining, LLMs aren’t told what’s “true” or “false,” they’re just rewarded for predicting the next word. (Via: TechCrunch)
That works great for consistent patterns, like spelling or punctuation, but when it comes to obscure trivia, the model is basically winging it.
The paper doesn’t propose redoing pretraining from scratch. Instead, it points the finger at how models are evaluated.
Right now, evals are like multiple-choice tests graded only on correct answers. If you guess, you might get lucky. If you skip, you’re guaranteed a zero.
So, surprise surprise, the models learn to bluff instead of admitting, “I don’t know.”
OpenAI’s fix? Change the scoring system. Think of it like the SAT: wrong answers should hurt more than leaving it blank, and uncertainty should earn partial credit.
If models are rewarded for honesty, they’ll stop confidently fabricating nonsense just to climb the leaderboard.
The takeaway: hallucinations aren’t going away entirely, but maybe we can train AI to BS a little less often.
Until then, treat your chatbot like a charming friend who will absolutely lie to your face, but with style.
Should AI companies prioritize making chatbots admit uncertainty over giving confident but wrong answers? Do you think changing evaluation systems to penalize hallucinations will actually make AI more trustworthy, or will users just get frustrated with more “I don’t know” responses? Tell us below in the comments, or reach us via our Twitter or Facebook.
