startups: The web is now mostly bots. Cloudflare is rebuilding its
Cloudflare has launched Precursor, a new defense system, in response to bots now generating over 57% of all web traffic. Precursor monitors entire user sessions to distinguish humans from sophisticated bots, moving beyond traditional single-check methods. This initiative is part of a broader strategy to classify and manage AI agents, control content reuse, and rebuild the web's foundational infrastructure for a machine-dominated internet.
In a significant shift marking a new era for the internet, automated bots now account for over half of all web traffic, surpassing human users. This pivotal milestone, with bots generating approximately 57% of all web requests, has prompted cybersecurity giant Cloudflare to launch an entirely new defense mechanism: Cloudflare Precursor. Unveiled on Monday, July 13, 2026, Precursor represents a fundamental overhaul in how websites distinguish between human visitors and sophisticated automated agents, preparing the web for an increasingly AI-driven future.
Rethinking Bot Detection with Cloudflare Precursor
Traditional web security measures, like CAPTCHAs, typically rely on a single, momentary check to verify a user's humanity. However, modern bots have become adept at bypassing these superficial gatekeepers. Cloudflare Precursor moves beyond this approach, embedding itself within the browser to monitor an entire user session.
Instead of a one-time ID check, Precursor observes a visitor's comprehensive behavior, including subtle cues like mouse movements, scrolling patterns, typing cadence, clipboard usage, and how long a page remains in view. This holistic, continuous monitoring makes it significantly harder for malicious bots to mimic genuine human interaction, transforming what was once an easy task for sophisticated automation into a complex engineering challenge.
Dane Knecht, Cloudflare’s chief technology officer, emphasized this shift: "Traditional security checks look at a single moment in time, but modern bots have gotten smart enough to fake their way through the front door." He noted that the crucial period between a user logging in and making a purchase has historically been a "black box," a gap Precursor aims to close. The tool is designed with privacy at its core, logging only behavioral patterns such as typing rhythm and cadence, never the actual keystrokes or content. Activation is streamlined, requiring just one click with no code modifications needed for websites.
Navigating the New Landscape of AI Agents
Precursor is part of a broader strategic re-evaluation by Cloudflare to manage the diverse and growing array of automated traffic. Not all bots are malicious; many perform essential functions. To address this, Cloudflare is now categorizing AI traffic into three distinct types: Search bots, which index content for later queries; Agent bots, designed to act in real time on behalf of a person; and Training bots, which absorb web content to train AI models.
Beginning September 15, new websites utilizing Cloudflare's services will automatically block Training and Agent bots on pages that display advertisements, while still allowing Search bots to operate freely. This policy change is driven by economic considerations, as Search bots typically direct traffic back to sites, unlike Training and Agent bots, which often do not. This initiative builds on Cloudflare’s previous stance that AI crawlers should compensate publishers for content usage or face blocking.
To further empower publishers and maintain control over their content, Cloudflare is introducing new options for sites to specify how bots may interact with their data: whether to store nothing, index and link back, or summarize and reproduce content. A new comprehensive database, dubbed "BotBase," will catalog every known web crawler, enhancing transparency and accountability.
Building Trust and Reshaping Web Infrastructure
Crucially, Cloudflare is pushing for greater accountability among bot operators. The company advocates for these operators to openly declare their identity using existing web headers, allowing websites to grant or deny access to specific entities like "OpenAI," with this choice persisting across complex network layers. Cloudflare suggests that the risk of losing trusted status across the more than 20 percent of web domains it protects will serve as a powerful deterrent against misuse.
This move aligns with broader industry discussions, such as Estonia's proposal to assign unique ID numbers to AI agents, and efforts to enable publishers to opt out of AI training without disappearing from search results. The implications become increasingly significant as AI agents evolve to perform tasks like shopping and making payments on behalf of users.
The foundational infrastructure of the web itself is also undergoing quiet but profound changes. On the same day as Precursor's launch, the Internet Engineering Task Force (IETF) published a new HTTP method called QUERY. Co-authored by engineers from Cloudflare and Akamai, this standard introduces a dedicated, safe, and cacheable verb for complex search requests. This eliminates the need for such searches to masquerade as data-altering POST requests, streamlining web communication built for a machine-dominated environment. These intertwined developments underscore a fundamental transformation of the internet's core machinery, shifting its design from human clicks to the intricate interactions of an "agentic" web.
FAQ
Q: What is Cloudflare Precursor and how does it differ from existing bot detection methods?
A: Cloudflare Precursor is a new tool launched by Cloudflare that detects sophisticated bots by monitoring a visitor's entire session within the browser, rather than relying on single-point checks like CAPTCHAs. It analyzes behavioral patterns such as mouse movement, scrolling rhythm, and typing cadence, making it far more difficult for modern bots to mimic human activity.
Q: Why is Cloudflare changing its approach to bot detection now?
A: Cloudflare is adapting because bots now generate approximately 57% of all web traffic, surpassing human users. The existing web infrastructure and traditional security methods were primarily designed for human interactions and are no longer sufficient to secure against the advanced capabilities of modern AI agents and other automated traffic.
Q: How does Cloudflare plan to manage different types of AI bots?
A: Cloudflare is categorizing AI traffic into Search bots, Agent bots, and Training bots. Starting September 15, new sites using Cloudflare will block Training and Agent bots by default on ad-supported pages, while allowing Search bots. They are also introducing tools for publishers to control how bots can reuse their content and creating a "BotBase" database to identify known crawlers.
Related articles
JPMorgan Chase Taps Seattle for Critical AI Control Layer Development
Global financial giant JPMorgan Chase is making a significant strategic investment in Seattle, establishing a new AI software infrastructure team. This pivotal group will build an "AI control layer" to manage the bank's AI operations, aiming to control costs, protect intellectual property, and prevent vendor lock-in.
The Motorola Edge 70 Max is all about power: Android — Key Details
Motorola has launched its new flagship, the Edge 70 Max, designed for power users with a massive 7100mAh silicon-carbon battery and 25W Qi2 wireless charging. It’s the first Android phone since the Pixel 10 Pro XL to support full 25W Qi2, surpassing other Qi2-enabled Androids capped at 15W. The device also offers 90W wired charging and a Snapdragon 8 Gen 5 chip.
Alone Australia S4 Access Guide: Mostly Free, VPN Required Abroad
TechRadar's guide on watching Alone Australia S4 is a solid resource, detailing free access for Australians via SBS on Demand and recommending NordVPN for international viewers. While the show is free, a VPN subscription is needed for global access, making the 'free from anywhere' claim slightly nuanced. It offers clear instructions and regional alternatives.
DeepMind CEO calls for independent body to regulate frontier AI
DeepMind CEO Demis Hassabis has proposed an independent standards body, modeled after FINRA, to regulate frontier AI models. The body would test advanced AI systems and develop best practices for their release, initially on a voluntary basis before potentially becoming mandatory. This initiative aims to provide technically focused, adaptable oversight to the rapidly evolving field of AI.
OnePlus is reportedly bailing on the US: Oppo — Key Details
OnePlus, and parent company Oppo, are reportedly exiting the US and European markets, with an announcement due shortly. This follows months of rumors and signals a major shift in the Western smartphone landscape.
OpenClaw Machines: Scaling Enterprise AI Agents with Bare Metal
OpenClaw Machines offers an open-source, self-hosted platform for running AI agents with enterprise-grade security and cost efficiency. It utilizes Firecracker microVMs for hardware isolation on your own Linux servers, providing full data sovereignty and predictable costs, especially at scale. The platform includes a control plane for orchestration, a Cloudflare data plane for secure access, and integrated LLM proxying.