Cloudflare, responsible for routing about 20% of global web traffic, has announced that it will begin blocking artificial intelligence (AI) crawlers by default.
Changes in AI Crawler Access to Content
With changes effective Tuesday, AI companies will face restrictions in accessing content following publisher pressures. New Cloudflare clients will need to decide whether to allow AI bots to access their content or block them entirely.
New Tools for Access Control
Cloudflare has introduced tools to help publishers control AI access to their content. CEO Matthew Prince stated: "AI crawlers have been scraping content without limits. Our goal is to put the power back in the hands of creators while still helping AI companies innovate."
Publisher Support for Cloudflare
Major media companies, including TIME and The Associated Press, back Cloudflare's initiative. Publishers who have benefited from ad revenue and traffic from Google now confront AI platforms like ChatGPT that utilize their content without compensation.
Cloudflare will continue working with developers wishing to have their AI crawlers access the web, aiming to ensure fair use of digital content online.
