Cloudflare vs. AI (with new stats)

Cloudflare Takes on AI

Across email, video, and communities, Cloudflare has been all over my feeds for the past week.

As we get into the next couple of sections, a little level set: Cloudflare provides CDN (content delivery network), DNS (domain name service) and security for roughly 20% of the Internet.

Also, Cloudflare has the best domain registration that I've used. I moved most of my domains over to Cloudflare a year or two ago, and it's the least expensive, easiest to use vendor out there (this isn't an affiliate deal, I just really like their domain services).

Cloudflare Shares Data on AI's Impact on Search

At the 2025 Cannes Lions, Cloudflare's CEO, Matthew Prince, shared some shocking data with how Search has changed over the past decade. In his talk, he goes over the original agreement between search engines and content creators.

The idea was that content creators would allow their information to be scraped, and the tradeoff was the search engines would drive traffic to the content creators.

But then the search engines started to change the rules.

As Cloudflare's data shows, the number of pages scraped used to lead to more site traffic.

• 10 years ago: Google scraped 2 pages, sent 1 visitor
• 6 months ago: Google scraped 6 pages, sent 1 visitor
• Today: Google scraped 18 pages, sent 1 visitor

These numbers are being skewed so heavily in the search engine's favor because of AI. Pure AI platforms, like Anthropic, are scraping 60,000 pages per visitor sent.

Search engines, which are basically all powered by AI, are taking our content and no longer providing value to content creators.

What's the Solution?

Tyler Stambaugh shared the news that Cloudflare is going to block AI bots from scraping content by default. This feature has been an option within Cloudflare for a while. The difference is now the default will be set to "don't scrape."

Joe and Robert from This Old Marketing dig further into this:

If I Opt Out, Doesn't That Hurt Me?

It does. If you and I are targeting the same prospect, and information about me and my services surfaces in AI results, that gives me a competitive advantage.

You have to fish where the fish are.

Search and AI have broken the contract we all agreed to 20-25 years ago. But this hasn't drastically changed user behavior.

Users are still searching for answers. The only real difference is Google is no longer the only game in town. Amazon, TikTok, Pinterest, YouTube, and more are working to show you the answers you're looking for, while selling you ads and products at the same time.

This is why you have to diversify and repurpose your content. It's all about building a brand so you're top of mind when someone is looking to solve a problem. A great way to do this is to appear in the areas where people are searching.