Cloudflare to block cynical search-and-scrape bots from ad-supported web pages

Summary

Some crawlers gather data for both search and AI training, so when publishers block them to protect content they risk disappearning from search results ...