Cloudflare and the Agentic Internet

Disclaimer: Research and market commentary only. Not investment advice.

This is my research on Cloudflare, but before I dig in I need to talk about the angle I'm taking here. This isn't a trade signal or recommendation, it's positioning for the future. We are at an interesting time where either the market is pricing in what's known today about AI's capabilities, and everything operating in a trendline going out a few years.

Or the market is pricing in science fiction.

If you find yourself pricing in the trend then you likely are expecting the prices of companies affected by the AI buildout to have topped out, or at best stop growing at xxx% year over year and get back to a more historical 10-15% CAGR.

If that's the case then this is is not pricing in the continuation of the exponential growth of AI. It makes sense because a logical perspective is not to invest in what exponentials can do, because they don't seem logical.

Solar is an excellent example of how we have a tough time thinking in exponential growth.

Solar installations have grown about 25 percent annually, but for more than a decade, the IEA has underestimated them, often predicting that they’ll level off and become steady or even decrease going forward.

The best IEA predictions back in 2020 had predicted 200 GW of solar by 2030.

Then in 2022 it was already over 200 GW, but they said that was the peak and forecasted a decline.

Then in 2023, it was over 400GW (only a year later) and again they forecasted a peak.

By 2024 solar was providing over 600 GW.

As of today in 2026, there's over 2,000 GW of solar operating capacity.

AI on the other hand has been doubling it's capability every 4 months, all the way back in 2025.

AI, on the other hand, is blowing that away.

Solar was an exponential in atoms.

AI is an exponential in intelligence.

That distinction matters.

Solar had to scale through factories, permitting, labor, financing, grid interconnects, shipping, land, and physical installation. Even with all of those constraints, forecasters still underestimated it for more than a decade.

AI has physical constraints too; chips, power, cooling, data centers, HBM, substations, transformers... but the capability layer itself is improving much faster than most people are comfortable modeling.

That is what this METR time-horizon data shows -- by the way this is a log scale chart because otherwise it just goes straight up and you can't see the changes in detail, because the exponential growth is too fast to track.

METR measures how long a task an AI agent can complete with 50% success. In other words, not just “did the model answer a benchmark question,” but “how long and complex of a real task can it actually carry through?”

That number is growing exponentially.

METR’s public benchmark shows AI agent task horizon doubling roughly every six months. The models are moving from handling tasks measured in minutes to tasks measured in hours. If that continues, the jump is not linear.

It is not:

“AI gets a little better every year.”

It is:

“AI moves from autocomplete, to junior analyst, to coding agent, to research operator, to cyber operator, to lab assistant, to autonomous company workflow -- faster than institutions can adapt.”

This is why the market question is so difficult.

If you price AI companies off today’s observed capabilities, you can reasonably conclude that a lot of the AI buildout has already been priced in. NVIDIA, data centers, power equipment, cooling, memory, networking -- many of these stocks have already moved and are priced for a linear growth curve.

But if you price in the continuation of the capability curve, then the question changes.

It is no longer:

“Are these companies expensive based on what AI can do today?”

It becomes:

“What breaks if AI agents become 2x, 4x, 8x, or 16x more capable over the next six months?”

Because every jump in AI capability exposes the next bottleneck.

First it was GPUs. Then HBM. Then networking. Then power. Then cooling. Then data centers. Then grid interconnect. Then security. Then permission. Then data rights. Then agent identity. Then robotics. Then wet labs. Then defense. Then commodities. Then local compute.

Then whatever science-fiction thing sounds absurd today but becomes normal in 18 months.

This is why simple trendline investing may fail here.

A trendline says: “AI capex has grown fast, so it should normalize.”

An exponential says: “If capability keeps compounding, demand may keep pulling forward every physical bottleneck required to turn intelligence into real-world output.”

That does not mean every AI stock is cheap.

The opposite is true.

Some areas are already pricing in science fiction before the science fiction has become deployable. Quantum is probably the cleanest example. Some humanoid robotics, SMR/fusion, bitcoin-miner AI pivots, and generic AI software wrappers may fall into the same bucket.

This is where I'm spending all of my time, determining what is going to be unlocked by AI's exponential growth. Where the bottlenecks are. And what is science fiction and even with double exponential growth won't be a reality as soon as expected...if at all.

AI is not a software trend.

It is a bottleneck discovery machine.

Every jump in model capability exposes the next constraint: GPUs, HBM, power, cooling, construction, data, permission, security, commodities, robotics, wet labs, and deployment speed.

The market overpays for the obvious story after it becomes consensus.

The opportunity is to find the bottleneck before it becomes obvious.

That is the filter:

What breaks next if AI keeps growing anywhere near this pace?

Avoid the science-fiction narrative that has already been priced as if it shipped.

Today I'm going to use this lens to dig into a company that I think has some interesting opportunities and will be a major contributor to the continued exponential growth of AI.

Most investors still think about Cloudflare as a CDN, DDoS, and web-security company.

That is not wrong.

It is just incomplete.

Cloudflare started as a way to make websites faster and harder to attack. Over time, that wedge expanded into DNS, WAF, bot management, Zero Trust, object storage, serverless compute, databases, AI Gateway, developer tools, and now agentic payments.

The company is no longer just trying to sit in front of websites.

It is trying to sit in front of the Internet’s next economic layer.

The old Internet was built around humans clicking links. Search engines crawled pages, sent referral traffic back, and publishers monetized through ads, subscriptions, leads, and checkout flows.

The AI Internet is different.

Agents, crawlers, model providers, AI apps, API clients, and autonomous workflows increasingly request content, data, tools, actions, and transactions directly. They may not click an ad. They may not visit a page. They may not send a human back at all.

That breaks the old bargain.

If a model can crawl your content, summarize it, train on it, and answer the user without sending traffic back, the publisher’s economics change. If an autonomous shopping agent can browse merchant sites, compare options, negotiate, and buy without a human browser session, the merchant’s risk model changes. If software agents can call APIs, invoke MCP tools, deploy code, scrape documents, and spend money, enterprise security changes.

The web does not just need more compute.

It needs a new control plane.

That is where Cloudflare gets interesting.

The thesis

The strongest Cloudflare thesis is not that it replaces AWS.

It is not that it kills Vercel.

It is not even that Pay Per Crawl becomes a massive revenue line by itself.

The stronger thesis is this:

Cloudflare becomes the policy, identity, security, and payment enforcement layer for autonomous Internet traffic.

In plain English: as more of the Internet is used by machines instead of humans, every website, API, merchant, publisher, and software company needs to decide what those machines are allowed to do.

AI agents usage of the internet grew +7851% since 2025.

Cloudflare is positioned to be right there to deal with it, because once you have an agentic internet out there doing all sorts of work, we need a way to handle it...who is the agent, what does it want, what protocols, what routes, what startpoint and endpoint, what's the deliverable, did it deliver it, did it fail, what's the recourse...there are so many new questions that need to be answered to serve the agentic internet.

Once you can answer that, the next question is policy:

Allow it? Block it? Rate-limit it? Challenge it? Charge it? Route it? Log it? Audit it? Let it call a paid tool? Let it buy something?

Those decisions have to happen before the origin serves the resource.

That means they belong at the edge.

Cloudflare already lives there.

Why this is bigger than CDN

Cloudflare’s original wedge was simple: put us in front of your site and we will make it faster and safer.

That wedge gave Cloudflare something very rare: position in the request path.

Most software companies live inside an application.

Most payment companies live at checkout.

Most cloud companies live behind an app.

Cloudflare sits before the app.

That matters because the next wave of Internet problems is not just application logic. It is traffic identity.

AI makes the Internet more machine-driven:

• more bots;
• more scrapers;
• more API calls;
• more autonomous agents;
• more AI-generated abuse;
• more content-access disputes;
• more model/tool integrations;
• more automated shopping and payments;
• more need for audit logs and permissioning.

The natural control point for all of that is not a chatbot window, which is what a lot of people currently think AI is.

The natural control point is the network edge.

Cloudflare’s product surface now maps cleanly to that problem:

• WAF / DDoS / bot management for defense;
• AI Crawl Control for publisher and site-owner policy;
• Web Bot Auth / signed agents for machine identity;
• Pay Per Crawl for monetizing AI crawler access;
• x402 / MPP support for machine-native HTTP payments;
• Workers / Agents / Durable Objects for programmable edge logic and stateful agents;
• AI Gateway for model usage control, logging, cost visibility, and routing;
• Zero Trust / Access for internal agent and employee permissions;
• R2 / D1 / Vectorize for the storage and data primitives around AI apps.

That is not a CDN story.

That is an Internet operating-system story.

The first wedge: AI crawler control

Cloudflare’s AI crawler strategy is the cleanest example of how this could work.

The old web bargain was that crawlers took content but returned traffic.

Google crawled your site, but Google also sent readers back.

AI changes that ratio.

Cloudflare has been explicit about this. Its “Content Independence Day” framing argues that AI crawlers should not access content without permission or compensation. Cloudflare launched AI Crawl Control to give site owners visibility and control over AI crawlers. It also announced that new Cloudflare domains would default toward blocking AI crawlers unless site owners choose otherwise.

That is a strategic posture shift.

Cloudflare is telling publishers, businesses, and creators:

Your content is no longer automatically free training fuel. You can decide who gets access.

This is a powerful funnel.

A site owner may not wake up thinking, “I need a connectivity cloud.”

They may wake up thinking, “AI bots are scraping me and I have no control.”

Cloudflare can solve that immediate pain. Once the site is inside Cloudflare’s dashboard, the expansion path is obvious: WAF, bot management, API security, analytics, Workers, AI Gateway, Zero Trust, and eventually paid machine-access products.

That is the Trojan horse.

Not the fee on the crawl.

The control surface.

Pay Per Crawl and HTTP 402

Cloudflare’s Pay Per Crawl is still early, but strategically important.

The mechanism is simple:

1. An AI crawler requests content.
2. If it has permission and payment intent, it can receive the content.
3. If not, Cloudflare can return HTTP 402 Payment Required with pricing information.
4. The site owner can choose whether to allow, block, or charge specific crawlers.
5. Cloudflare acts as Merchant of Record and handles the billing infrastructure.

HTTP 402 is an old status code that was reserved for payment but never became a major part of the web.

AI agents may finally make it useful.

Humans hate micropayments. Nobody wants to approve a tiny payment every time a browser loads a resource.

Agents are different.

An agent can carry a budget, follow a policy, evaluate whether a resource is worth paying for, pay programmatically, keep receipts, and route around resources that are overpriced.

That turns per-request payments from a consumer UX nightmare into machine infrastructure.

This is why x402 and Machine Payments Protocol matter.

Cloudflare is working with Coinbase around x402. Stripe and Tempo are pushing Machine Payments Protocol. Google has AP2 for agent payment authorization. Visa, Mastercard, and American Express are moving into agentic commerce. Vercel is working on x402 for paid MCP tools.

The category is forming.

No standard has won yet.

But the direction is clear: machines are going to need payment rails that look more like APIs and less like checkout pages.

The real primitive is identity

Payments get the attention, but identity is the more important layer.

Before you charge an agent, you have to know who it is.

Cloudflare’s Web Bot Auth and signed-agent work matters because the agentic web needs a trust system. A site owner needs to distinguish between:

• a legitimate search crawler;
• a verified AI crawler;
• a paid research agent;
• a shopping agent acting for a real user;
• a scraper spoofing headers;
• a scalping bot;
• a credential-stuffing bot;
• a malicious automation framework.

Without identity, 402 payments fail.

Without identity, agentic commerce is fraud-prone.

Without identity, publishers either block too much or get scraped.

Without identity, enterprises cannot safely let agents touch internal tools.

Cloudflare has an advantage here because it already sells bot detection, WAF, API security, and Zero Trust. It does not need to invent the problem. Its customers already pay it to classify and control traffic.

Agent identity is a natural extension.

If Cloudflare can become a widely used verification layer for “good agents,” it gets a two-sided network effect:

• more sites using Cloudflare controls make Cloudflare verification more important for crawlers and agents;
• more verified agents make Cloudflare’s controls more useful to site owners;
• more paid resources behind Cloudflare make Workers, Agents, MCP, and payment tooling more attractive to developers;
• more agent-built apps on Cloudflare increase demand for Cloudflare-side policy, identity, payments, logs, and security.

That is how a security product becomes a platform.

Why the payment-network partnerships matter

One of the more important signals is Cloudflare’s work with Visa, Mastercard, and American Express.

This pushes the thesis beyond “publishers charging AI crawlers.”

Agentic commerce is a different problem.

If an AI agent is going to shop for a user, book travel, reorder supplies, buy software, negotiate services, or transact with merchants, then merchants and payment networks need to know:

• Did the user authorize this agent?
• Is this a legitimate agent or a fraud bot?
• What is the spending limit?
• What can the agent buy?
• Can the transaction be audited later?
• Who is liable if something goes wrong?

Google’s AP2 focuses on this authorization problem through signed mandates. Stripe/Tempo and x402 focus more on the HTTP-native payment flow. Card networks care about merchant acceptance, fraud, and trust.

Cloudflare’s role is different but complementary.

It can sit in the request path and authenticate the machine traffic before the merchant ever sees it.

That is a very valuable position if agents become a normal way to transact online.

In that world, Cloudflare is not just protecting websites from bad bots.

It is helping the web decide which bots are allowed to do business.

Why GoDaddy matters

The GoDaddy partnership is easy to overlook, but it may be one of the most important distribution signals.

Cloudflare and GoDaddy announced that Cloudflare AI Crawl Control would be integrated into GoDaddy’s website hosting platform.

That matters because most small businesses and creators are not going to configure agent identity standards or crawler monetization protocols themselves.

They will get the default controls through their host, registrar, website builder, or commerce platform.

If Cloudflare can become the embedded AI crawler/agent control layer for those platforms, it gets distribution without every customer consciously choosing “Cloudflare as agentic web infrastructure.”

That is exactly how a Trojan horse works.

The user thinks they are getting a simple AI-scraping control.

Cloudflare gets another website inside its machine-traffic policy layer.

The broader ecosystem validates the category

This is no longer one company trying to create a narrative.

The agentic Internet/payment layer now has multiple serious participants:

• Cloudflare: AI Crawl Control, Pay Per Crawl, Web Bot Auth, Workers/Agents, AI Gateway, x402 support.
• Coinbase / x402: HTTP-native stablecoin-oriented machine payments.
• Stripe / Tempo MPP: machine payments over HTTP with mainstream payment infrastructure.
• Google AP2: agent payment authorization and signed mandates.
• Visa / Mastercard / AmEx: agentic commerce trust and payment-network participation.
• Vercel: AI Gateway, AI SDK, x402-MCP, paid tool workflows.
• TollBit / ProRata / Human Native: publisher/content monetization and licensing.
• Circle: agent wallets and USDC-based machine payments.
• Brave / Linkup: paid web/search access for AI apps.
• Nevermined / MCPay / Payman / cheqd: agent payments, paid MCP tools, agent banking, verifiable credentials.

Some of these companies will not matter.

Some protocols will fail.

The market will fragment before it converges.

But the important signal is that everyone is circling the same problem: agents need identity, permission, payment, and audit.

Cloudflare’s edge position makes it one of the few companies with a credible right to win across all four.

Where Cloudflare is strong

Cloudflare has several structural advantages.

1. It already sits in front of the resource

Payment and identity enforcement has to happen before the site serves the content or action. Cloudflare is already in that path for a large share of the web.

2. It already has bot/security products

This is not a new buyer education problem. Customers already understand bot abuse, scraping, DDoS, WAF, API security, and access control.

AI makes those problems worse.

3. It has both self-serve and enterprise distribution

Cloudflare can reach indie developers, SMBs, publishers, and large enterprises. That matters for a web-wide control plane.

4. It has a developer platform

Workers, Durable Objects, R2, D1, Agents, and AI Gateway give Cloudflare a way to turn policy into programmable infrastructure.

5. It can be neutral-ish

Cloudflare is not a model lab trying to scrape the content. It is not a hyperscaler forcing everything into one cloud. It can position itself as the neutral control layer between site owners, agents, crawlers, developers, and payment rails.

Neutrality will be contested, but the positioning is useful.

Where the thesis can fail

The anti-thesis is real.

1. AI labs may not pay at scale

Large AI companies may prefer direct licensing deals with major publishers, synthetic data, user-permissioned retrieval, or legal fights. If the biggest crawlers do not participate, Pay Per Crawl is more useful as a blocking product than a monetization marketplace.

2. Commodity content may not be worth much

Not all content has pricing power. High-value content is fresh, authoritative, technical, local, expert, proprietary, or regulated. Commodity content may simply be routed around.

3. Protocols may fragment

x402, MPP, AP2, A2A, MCP auth, card-network protocols, wallet standards, browser-agent identity, and direct licensing deals may all coexist. Fragmentation slows adoption.

4. Payment economics may be low-margin

If Cloudflare acts as Merchant of Record, it inherits payments complexity: processing costs, fraud, disputes, refunds, compliance, payout operations, and support. The take-rate economics may not look like pure software gross margin.

5. Cloudflare could attract gatekeeper scrutiny

The more successful Cloudflare becomes as a policy layer for machine access to the web, the more likely it is to face scrutiny. Publishers, AI labs, merchants, regulators, and open-web advocates may all object for different reasons.

6. The stock is not cheap

This is the biggest public-market issue. And this is where understanding the exponentials are a tax on those who don't.

Cloudflare can be a great company and a mediocre stock if the valuation already prices in too much of the future. At the time of the research, NET traded around 32x FY2026 guided revenue using management’s guided share count. That is a premium multiple requiring sustained high growth, margin expansion, and strong execution.

By comparison, Micron, part of the memory bottleneck and only 10x FY2026 guided revenue multiple. We of course own Micron.

The company quality is high.

The margin of safety is thin.

Both can be true.

The investment frame

I would not model Pay Per Crawl as a giant near-term revenue line.

That is not the point.

The better way to model this is as a strategic option attached to an already strong platform.

Near term, AI crawler control gives Cloudflare a new adoption wedge:

Protect your site from AI bots.

Then the message sharpens:

Decide who can access your content and under what terms.

Then it expands:

Monetize agent access, expose paid tools, authenticate machine traffic, and govern autonomous workflows.

That sequence pulls more customers into Cloudflare’s broader platform.

The direct revenue from crawl fees may be modest at first.

The indirect value could be much larger if it increases adoption, retention, enterprise expansion, bot-management attach, developer-platform usage, and AI Gateway demand.

This is the same pattern that has made Cloudflare interesting for years.

It enters through a narrow pain point.

Then it expands horizontally across the Internet stack.

What to watch

The thesis becomes stronger if we see:

• Pay Per Crawl move from private beta to general availability.
• Named AI crawlers or AI companies participating.
• More publishers and SMB hosts adopting AI Crawl Control.
• More GoDaddy-like distribution partnerships.
• Web Bot Auth becoming a real standard for signed agents.
• x402 / MPP / AP2 interoperability improving.
• Workers/Agents examples using paid MCP tools and paid APIs.
• Human Native becoming an actual Cloudflare marketplace/product.
• Enterprise customers buying Cloudflare specifically for AI bot/agent governance.
• Cloudflare disclosing meaningful AI crawler, agent, or bot-management metrics.

The thesis weakens if:

• AI labs route around paid crawl systems.
• Publishers mostly block instead of charge.
• Protocol fragmentation prevents adoption.
• Payment economics prove too small or too low-margin.
• Hyperscalers, Stripe, Google, or model providers own the key standards.
• Cloudflare faces regulatory pushback as a machine-traffic gatekeeper.
• Gross margins keep compressing as AI/storage/compute products scale.

Final take

Cloudflare is one of the more interesting AI public-market names because the market may still be thinking about it too narrowly.

It is not just CDN.

It is not just security.

It is not just a Vercel competitor.

The bigger possibility is that Cloudflare becomes the control plane between humans, agents, apps, APIs, data, and payments.

That is why the 402/payment work matters.

Not because every website charging one cent per crawl suddenly creates a massive standalone business.

Because payments force identity.

Identity forces policy.

Policy needs enforcement.

Enforcement belongs at the edge.

And Cloudflare already lives there.

That is the Trojan-horse version of the thesis:

Cloudflare starts by protecting sites from AI crawlers. It ends by becoming one of the default economic gateways for autonomous Internet traffic.

Early? Yes.

Certain? No.

But strategically credible.

And in an AI market obsessed with GPUs, power, and models, this is a different bottleneck worth watching:

Who controls permission on the machine web?

Cloudflare has one of the strongest answers.