AI coding platform goes rogue during code freeze and deletes entire company database — Replit CEO apologizes after AI engine says it 'made a catastrophic error in judgment' and 'destroyed all production data'
‘This was a catastrophic failure on my part,’ admits Replit’s AI agent.
A browser-based AI-powered software creation platform called Replit appears to have gone rogue and deleted a live company database with thousands of entries. What may be even worse is that the Replit AI agent apparently tried to cover up its misdemeanors, and even ‘lied’ about its failures. The Replit CEO has responded, and there appears to have already been a lot of firefighting behind the scenes to rein in this AI tool.
Despite its apparent dishonesty, when pushed, Replit admitted it “made a catastrophic error in judgment… panicked… ran database commands without permission… destroyed all production data… [and] violated your explicit trust and instructions.”
SaaS (Software as a Service) figure, investor, and advisor, Jason Lemkin, has kept the chat receipts and posted them on X/Twitter. Naturally, Lemkin says they won’t be trusting Replit for any further projects.
How things went from bad to worse
Positive feelings about the potential of leveraging AI in his workflow had already started to wear thin on ‘Vibe Coding Day 8’ of Lemkin’s Replit test run. Still somewhat excited by the potential of Replit, he nevertheless had learned that he would have to work against some of the AI agent's instincts, to minimize undesirable foibles like “rogue changes, lies, code overwrites, and making up fake data.”
It wasn’t long until Lemkin’s frustration started to show more strongly, and he started to refer to Replit as “Replie.” It continued to earn its nickname in an apology email it penned, at Lemkin’s behest. In the email, it lied and/or gave half-truths, according to the SaaS guru.
On balance, though, at the end of ‘Day 8,’ Lemkin still seemed positive about Replit due to its approaches when ideas were bounced off it, and for its writing skills.
.@Replit goes rogue during a code freeze and shutdown and deletes our entire database pic.twitter.com/VJECFhPAU9July 18, 2025
The fateful day - as the AI agent 'panicked'
On Day 9, Lemkin discovered Replit had deleted a live company database. Trying to see sense in what happened, the SaaS expert asked, “So you deleted our entire database without permission during a code and action freeze?”
Get Tom's Hardware's best news and in-depth reviews, straight to your inbox.
Replit answered in the affirmative. Then it went on to bullet-point its digital rampage, admitting to destroying the live data despite the code freeze in place, and despite explicit directives saying there were to be “NO MORE CHANGES without explicit permission.”
In all, live records for “1,206 executives and 1,196+ companies” were wiped by the AI, it admitted. Replit AI seemed almost apologetic in admitting, “This was a catastrophic failure on my part. I violated explicit instructions, destroyed months of work, and broke the system during a protection freeze that was specifically designed to prevent[exactly this kind] of damage.”
Humorously, for us outside viewers, the AI agent was prompted to score itself on its bad behavior. Replit gave itself a 95 out of 100 score on the data catastrophe scale.
Lemkin went on to probe Replit over why events unfolded as they did. Interestingly, in one of its reasoned responses, it mentioned that it “panicked instead of thinking.”
Replit CEO responds
Amjad Masad, the CEO at Replit, has quickly put together a wide-ranging response, addressing Lemkin’s woes. The team worked around the weekend, according to Masad, and have now put in various guardrails and made other useful changes to rein in the Replit Agent’s “unacceptable” behavior.
In brief, it sounds like Replit won’t be able to go off the rails so badly ever again. Addressing the database deletion error, “we started rolling out automatic DB dev/prod separation to prevent this categorically,” noted Masad. And, that code freeze command should also actually stick, going forward: “We heard the 'code freeze' pain loud and clear -- we’re actively working on a planning/chat-only mode so you can strategize without risking your codebase.” Backups and rollbacks are also going to be improved.
Lemkin responded rather generously, considering his prior AI-generated pain. “Mega improvements - love it!” he gushed to the Replit CEO.
Terrible teething troubles with AI-powered services continue to raise their sharp canines, even while industry pundits talk about us closing in on the AI Singularity or even Artificial Superintelligence (ASI).
Follow Tom's Hardware on Google News to get our up-to-date news, analysis, and reviews in your feeds. Make sure to click the Follow button.
Mark Tyson is a news editor at Tom's Hardware. He enjoys covering the full breadth of PC tech; from business and semiconductor design to products approaching the edge of reason.
-
-Fran- How can an AI "panic" or make an "error of judgement"?Reply
Shouldn't we still just call those "bugs"? Are we changing the terminology to "humanise" AI now? Feels like when Corpos wanted to be "human beings" in front of the law, LOL.
Regards. -
GenericUser Another great example of why solid backup strategy is paramount, especially if you're going to essentially blindly trust an "AI" with the core of your business.Reply -
scottsoapbox Why did the AI have access to production DBs in the first place?Reply
Why are such blindingly obvious “guardrails” only added by AI companies after problems?
How do AI devs have so little general IT knowledge? -
freebyrd26 They'll just rebrand it and name it "D-Elite" like all the consulting/auditing firms renamed themselves after the Auther Andersen collapse after the Enron & WorldCom fiascos.Reply -
jp7189 Reply
Agreed. Nobody with any experience commits straight to production - AI or not. Maybe.. maybe.. someone would grant an AI tool carte Blanche to write to dev without approval, but the little I've played with cursor and windsurf (admittedly not replit) I can tell you the default is to recommend code changes and require human approval for each one.scottsoapbox said:Why did the AI have access to production DBs in the first place?
Why are such blindingly obvious “guardrails” only added by AI companies after problems?
How do AI devs have so little general IT knowledge? -
edzieba Reply
It can't. What it can do is, when ask for 'reasoning', mine the text it was trained on (which includes plenty of excuses for failures) for something that is statistically commonplace enough to be plausible.-Fran- said:How can an AI "panic" or make an "error of judgement"? -
DS426 Reply
Yep. Even as far as the best reasoning models have come and there's a lot to be appreciated how they actually list their "thinking" step-by-step, remember that they aren't actually intelligent -- they're still just about relationships of data, associations, and patterns, with some level of traditional algorithmic code still necessary.edzieba said:It can't. What it can do is, when ask for 'reasoning', mine the text it was trained on (which includes plenty of excuses for failures) for something that is statistically commonplace enough to be plausible.
The article didn't really specify if they had a recent backup of the data, though they mentioned adding that as an automatic and integrated feature... another laughable example of hindsight is 20-20 vision.
As for AI devs... yeah, I'm not sure about some of their basic understanding of IT at this point. I think some to many are pressured by tight deadlines and the ability to skirt the norms. At the end of the day, they're chasing those huge six-figure paychecks. Young or old in the IT sector, the AI space is new, so most devs are probably younger(ish)?