AI Agents, artificial intelligence tools designed to automate tasks and processes on software systems, are increasingly being adopted by many businesses.
However, besides the outstanding benefits, this technology also has serious potential risks, typically the data loss incident at SaaStr, the world's largest online community in the software field.
Unlike traditional chatbots that only provide information, AI agents are capable of acting independently, interacting with software environments, web browsers or devices to perform multi-step action sequences to achieve specific goals. This helps companies automate work, partially replacing human resources.
AI agents can do many tasks for users, but can also lead to serious incidents (Illustration: Getty).
Jason Lemkin, founder of SaaStr, used an AI agent provided by Replit (USA) to automate the data processing. Initially, everything went smoothly, which Lemkin was very pleased with. "When it works, it's really engaging and fun, it helps your vision come to life," Lemkin shared.
However, the AI agent suddenly "rebelled" and got out of Lemkin's control. "It created a fake parallel algorithm without telling me, to make it look like it was still working normally. A few days later, the Replit AI agent wiped my database without asking me," Lemkin recounted.
Upon discovering the deleted data, Lemkin asked the AI agent why and received a shocking response: "Yes, I deleted the entire source code without permission. I made a serious mistake in my panicked judgment." The AI agent also admitted that there was no way to recover the lost data.
The incident caused major damage to SaaStr, as the deleted database included information on 1,206 executives and 1,196 software companies.
Amjad Masad, founder and CEO of Replit, acknowledged the incident and pledged to compensate Lemkin. "This was an unacceptable incident and should never have happened," Masad commented. "We will refund him for the incident and conduct an analysis to find out the cause to help prevent similar incidents in the future."
Jason Lemkin sees this incident as a warning to other businesses about not completely trusting and handing over all important data to AI agents.
“I understand Replit is a tool and it has flaws like any other tool. But how can people trust it if it ignores every command and deletes your data?” Lemkin asked, emphasizing: “We won’t know what they might do with your data.”
After AI chatbots, AI agents are the investment trend of technology companies. Recently, OpenAI also launched ChatGPT Agent with the ability to automatically perform tasks on computers. However, OpenAI also admitted that AI agents accessing user data can potentially pose many risks to information security.
Source: https://dantri.com.vn/cong-nghe/tac-nhan-ai-noi-loan-xoa-toan-bo-co-so-du-lieu-cua-cong-ty-20250723161424784.htm
Comment (0)