Cloudflare's CTO apologizes after error takes huge chunk of the internet offline — 'we failed our customers and the broader internet'
CTO blames bot mitigation bug triggered by routine config change.
Cloudflare has confirmed that a bug in one of its core services caused a major outage on Tuesday, taking large portions of the internet offline and affecting traffic to services including X, ChatGPT, and, ironically, Downdetector. The company’s CTO, Dane Knecht, posted a public apology shortly after services were restored, calling the incident "unacceptable" and attributing the disruption to a routine configuration change that triggered a crash in its bot mitigation layer.
The incident began at approximately 11:48 UTC on November 18, with Cloudflare's official status site acknowledging “internal service degradation”. As the issue spread, users across several regions reported failures to access not only Cloudflare-backed websites but also its Access and WARP services. The company later identified a specific dependency in its bot defense tooling as the source of the problem.
"We failed our customers and the broader internet," Knecht wrote. "A latent bug in a service underpinning our bot mitigation capability started to crash after a routine configuration change. That cascaded into a broad degradation to our network and other services. This was not an attack."
By 14:42 UTC, Cloudflare had deployed a fix and began restoring affected components. Dashboard functionality, including analytics and error logging, remained partially degraded into the afternoon as engineers monitored for residual faults. A temporary suspension of WARP access in London was also enacted as part of the mitigation process.
Cloudflare’s bot mitigation stack, which includes challenge flows such as Turnstile and JavaScript verification layers, sits inline with traffic to many high-profile websites and APIs. Because these systems are used not only to block malicious actors but also to gate access for legitimate users, faults in this layer can result in widespread service disruption even when core CDN or DNS infrastructure remains operational.
This is the third major outage to affect major sites in less than a month. In October, a large section of AWS’s US-East-1 region went offline for over two hours following what Amazon later attributed to a broken DNS configuration. Then, just days later, a huge Azure outage hit Microsoft.
These incidents raise broader questions about how widely used services and platforms handle internal service faults and dependency isolation at scale — roughly 19% of the Internet relies on Cloudflare, while Azure and AWS account for roughly 24% and 30% of the cloud computing market, respectively.
Get Tom's Hardware's best news and in-depth reviews, straight to your inbox.
Follow Tom's Hardware on Google News, or add us as a preferred source, to get our latest news, analysis, & reviews in your feeds.

Luke James is a freelance writer and journalist. Although his background is in legal, he has a personal interest in all things tech, especially hardware and microelectronics, and anything regulatory.
-
thisisaname They just demonstrated that relying on just them is a very bad idea, they just dos'ed their customers.Reply -
bit_user Didn't we just learn this lesson in the massive CrowdStrike outage that hit their customers, last year?Reply
https://www.tomshardware.com/software/windows/global-it-issue-strikes-windows-machines-cause-of-issue-allegedly-linked-to-crowdstrike-software-update
Always test your changes in a sandbox, before deploying on the production network. -
derekullo Reply
https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcR2CN0r--Ly3V0W40xRVkyZLje6irddck36zA&sbit_user said:Didn't we just learn this lesson in the massive CrowdStrike outage that hit their customers, last year?
https://www.tomshardware.com/software/windows/global-it-issue-strikes-windows-machines-cause-of-issue-allegedly-linked-to-crowdstrike-software-update
Always test your changes in a sandbox, before deploying on the production network. -
tamalero Reply
My bet is they are really trying to sweep code changes programmed by AI.. by using QA powered by AI..S58_is_the_goat said:Anybody thought about doing config changes on the weekend maybe? 😂
Hence why everything fails XD -
bill001g ThReply
Nope they learn nothing. When the worst that happens is the company AI bot says " i am so sorry" . This is not like a regular company that the end users could boycott.bit_user said:Didn't we just learn this lesson in the massive CrowdStrike outage that hit their customers, last year?
Even if they had to pay some fine to the government that would also make no difference because the employees who are not doing their job do not have to pay the fine it comes out of any profit that the shareholders have.
There also is not likely a way to find a single person who is responsible. They have their massive committees and they can all point fingers at each other say it wasn't me.
I bet pretty much everyone has gotten a letter saying your information was access and we will give you $.05 and a year of credit reporting. People seem numb to the incompetence in these large companies and the lack of any responsibility by employees to do what they are getting paid for. -
thisisaname Reply
Yes but the. will say "next time lessons will be learnt" except they will not be.bill001g said:Th
Nope they learn nothing. When the worst that happens is the company AI bot says " i am so sorry" . This is not like a regular company that the end users could boycott.
Even if they had to pay some fine to the government that would also make no difference because the employees who are not doing their job do not have to pay the fine it comes out of any profit that the shareholders have.
There also is not likely a way to find a single person who is responsible. They have their massive committees and they can all point fingers at each other say it wasn't me.
I bet pretty much everyone has gotten a letter saying your information was access and we will give you $.05 and a year of credit reporting. People seem numb to the incompetence in these large companies and the lack of any responsibility by employees to do what they are getting paid for.