Why are ChatGPT and Bing Chat so good at inventing things?

Over the past few months, AI chatbots like ChatGPT have captured the world’s attention for their ability to chat about any topic in a human-like manner. But they have a serious drawback. they can easily present convincing false information, making them unreliable sources of factual information and potential sources of defamation.

Why are AI chatbots being created, and will we ever be able to fully trust their output? We asked some experts and researched how these AI models work to find the answers.

“Hallucinations,” a loaded AI term

AI chatbots like OpenAI’s ChatGPT rely on a type of AI called a “large linguistic model” (LLM) to generate their responses. LLM is a computer program built on millions of text sources that can read and generate “natural language” text—the kind of language that people would naturally write or speak. Unfortunately, they can also be wrong.

In the academic literature, AI researchers often refer to these errors as “hallucinations”. But that label has become controversial as the topic becomes mainstream, with some people believing it anthropomorphizes AI models (suggesting they have human-like qualities) or giving them agency (suggesting they can make their own choices. perform) in situations where it should not be assumed. Creators of commercial LLMs may also use hallucinations as an excuse to blame the AI ​​model for wrong results rather than taking responsibility for the results themselves.

Still, generative AI is so new that we need metaphors borrowed from existing ideas to explain these highly technical concepts to the wider public. In this sense, we feel that the term “confusion,” though similarly imperfect, is a better metaphor than “hallucination.” In human psychology, “confusion” occurs when someone’s memory has a gap and the brain convincingly fills in the rest without intending to deceive others. ChatGPT: no works like the human brain, but the term “confabulation” probably serves as a better metaphor because of the gap-filling creative principle at work, as we’ll discuss below.

The problem of confusion

It is a big problem when an AI bot creates false information that can mislead, misinform or defame. The Washington Post recently reported on a law professor who discovered that ChatGPT had placed him on a list of legal scholars who had sexually assaulted someone. But it never happened. ChatGPT invented it. That same day, Ars reported on an Australian mayor who allegedly found ChatGPT’s claim that he had been convicted of bribery and sentenced to prison to be a complete fabrication.

Shortly after ChatGPT launched, people started declaring the end of the search engine. At the same time, however, many examples of the ChatGPT merger started circulating on social media. AI bot invented books and: studies which do not exist publications that the professors did not write, false academic papersfalse legal citationsnon-existent Features of the Linux systemunreal retail mascotsand: technical details it doesn’t make sense.

And yet, despite ChatGPT’s propensity for random fibbing, counter-intuitively, its resistance to obfuscation is why we’re even talking about it today. Some experts say that ChatGPT was technically an improvement over the vanilla GPT-3 (its predecessor) because it could refuse to answer certain questions or let you know when its answers might not be accurate.

“A key factor in Chat’s success is that it manages to keep the clutter down enough to make it invisible for very common questions,” says Riley Goodside, an expert on large language models who serves as an agile engineer on Scale AI’s staff. : “Compared to its predecessors, ChatGPT is significantly less prone to fabrication.”

When used as a brainstorming tool, ChatGPT’s logical leaps and confusions can lead to creative breakthroughs. But when used as an actual link, ChatGPT can cause real damage, and OpenAI knows it.

Not long after the model was released, OpenAI CEO Sam Altman on Twitter“ChatGPT is incredibly limited, but it’s good enough at some things to give a misleading impression of greatness. It’s a mistake to rely on it for anything important right now. It’s a preview of progress. we have a lot of work to do on sustainability. and truthfulness.” Later tweethe wrote: “MeHe doesn’t know much, but the danger is that he’s sure and wrong a lot of the time.”

What is going on here?