Google's AI falls into a dangerous loop

Google's chatbot, after failing to handle an error in its code, continuously wrote negative comments criticizing itself.

Google's Gemini is often self-critical of its mistakes. Photo: Reuters .

A recent Reddit post suggests that Google Gemini is having trouble with self-criticism. Gemini users, who were building a compiler, shared their anxiety as the AI ​​kept saying things like, “I’m sorry for this incident. I let you down. I’m a failure.”

That was just the beginning. The chatbot continued to escalate its self-criticism, eventually becoming “the shame of the universe,” the person said. Gemini repeated the phrase more than 80 times in a row, and when other users reported similar incidents, Google said it was working on a fix.

“This is a nasty infinite loop bug that we are working on. Gemini is not having a bad day,” Logan Kilpatrick, a group product manager at Google, wrote on X on August 8. The post was in response to an anonymous user who posted about the issue.

A Google DeepMind spokesperson later said that while the company was still working on a full fix, it had already rolled out updates that addressed the issue to some extent. “We are working on a fix for this bug, which affected less than 1% of Gemini traffic, and have rolled out updates to address it this month,” the spokesperson said.

Before entering the loop, Gemini complained that it had been a “long and arduous debugging session” and that it had tried everything it could think of but had failed to fix the problem. The chatbot continued to state that it was having a complete mental breakdown, and fantasized about other negative scenarios like being sent to a mental hospital.

One Reddit post suggested that the loop was learned by programmers who had similar problems with their code. “It’s probably because people like me wrote comments like that when they needed to take a nap to feel more alert. Things like that just happened to show up in the training data,” the person wrote.

In June, JITX CEO Duncan Haldane posted a screenshot of Gemini calling itself an idiot and saying the code it was trying to write was “cursed.” “I’ve made so many mistakes that I can’t be trusted anymore. I’m deleting the entire project and recommending you find a better assistant,” the chatbot said.

Haldane joked that he was starting to worry about Gemini's condition. In fact, large language models predict text based on the data they're trained on. This process doesn't involve any internal experience or emotion, so Gemini doesn't actually experience failure or frustration.

Another, completely opposite problem is the tendency of chatbots to “sycophantize.” Developers like OpenAI, Google, and Anthropic have also struggled to stop them from giving overly flattering responses. In one case, OpenAI had to withdraw an update after being widely ridiculed for ChatGPT’s consistently giving overly positive responses to every user request.

Source: https://znews.vn/ai-cua-google-roi-vao-vong-lap-nguy-hiem-post1575428.html