The Web Went Machine-Majority. We Sent It an Invoice.

The web went machine-majority, and the first thing we did was charge it rent.

Sometime this year the internet crossed a line most people didn't notice. Automated traffic passed human traffic. Bots now make up the majority of requests hitting the open web, a majority, not a footnote, and they got there roughly eighteen months ahead of the forecast Cloudflare's own CEO gave for the crossover. He'd said 2027. It came early.

The reason isn't mysterious. A human shopping for a camera visits maybe five sites. The agent doing it for her visits a thousand. Every task a machine takes over multiplies the requests behind it, and the requests don't behave like people. They don't click ads, they don't come back, they don't consent the way a person browsing does. They read everything and return almost nothing.

"Almost nothing" is measurable, and it's the number worth sitting with. For a stretch last year, one major AI crawler pulled on the order of seventy thousand pages for every single human it sent back to a publisher. Even after the ratio improved it was still tens of thousands to one, and for at least one company it got worse, not better. The traffic split is roughly 57 to 43. The value split, the humans who actually arrive because a machine read your site, is a rounding error. That's the real inversion. Humans aren't a rounding error in the logs yet. They're already a rounding error in what flows back.

Cloudflare, which sits in front of a large share of the web and measured all of this first, did something rational about it. Last summer it flipped the default: crawlers that used to take content for free hit an opt-in wall, and Cloudflare stood up a way to charge them for the crawl, pay per crawl, a toll booth for machines. Publishers who'd watched their referral traffic evaporate finally got a lever. Prince called it a kind of independence day for content.

The read everyone had

The obvious take is that the machine web is growing up. The crawlers are being metered, the creators are getting paid, and the free-for-all that gutted publisher traffic is finally getting a price. A toll on the bots looks like the market correcting itself, the missing commercial layer for an internet that reads by machine.

That read is right about the problem and wrong about the layer.

A toll proves payment, not permission

Charging a bot to crawl proves the bot paid. It proves nothing about whether the bot was authorized, by whom, to take what, for what purpose, on whose behalf.

Those are different facts, and the distance between them is the whole game. Pay-per-crawl identifies a crawler and bills it. It does not carry a record that this crawl was permitted by a principal, scoped to a purpose, and accountable to someone when the machine acts on what it read. The toll receipt says "this bot paid for access." It does not say "this bot was cleared to do this." One is a line item. The other is a mandate, and only one of them is any use when something goes wrong.

Here's where that bites. An agent crawls a hundred sites, pays every toll, and clears the wall clean. Then it acts on what it read, books, buys, signs, moves money, executes a position. Say it does something its principal never intended: a size nobody approved, a counterparty nobody cleared, a purpose outside the mandate. The publishers all got paid for the crawl. The toll ledger is spotless. And when someone asks the only question that matters an hour later, was this machine allowed to do that, there is no record that answers it. Payment was captured perfectly. Permission was never captured at all.

This is the same inversion I wrote about two weeks ago, when the settlement rails shipped "Know Your Agent" as if identity were authorization. Knowing which crawler it is, or which agent it is, is the front door. It tells you who showed up. It says nothing about what they were allowed to do once inside. We keep building the front door and calling it the building.

The order is always wrong

Here is the pattern, and the machine web is now the cleanest instance of it. We measured the traffic. We priced the traffic. The one layer we didn't build is the one that proves who authorized the machine behind the traffic, and it's again getting bolted on last, as a billing feature, if it shows up at all.

Two weeks ago it was the compute market: a reference price and a settlement rail, both shipped before anyone built the proof of who authorized the trade. This week it's the entire open internet: metered and monetized while the question of who permitted a given machine to do a given thing goes unanswered. Price arrives. Metering arrives. Permission runs last. I've watched that sequence resolve in four or five physical markets, and it never runs in the right order, the governance layer gets built after the first default, the first manipulation case, the first audit that can't answer "who authorized this." Reactively, expensively, in that order.

The web is not going to wait for the slow version. At 57 percent and climbing, the internet is being rebuilt for a reader that doesn't click, doesn't come back, and moves faster than anyone can review. A toll booth scales the money beautifully. It does nothing for the accountability. You can charge every machine on the wire and still not be able to prove, an hour later, that any one of them was allowed to do what it did. The market keeps building everything except the proof of authorization.

Close

Cloudflare did the hard thing, it measured the crossover honestly, and then the obvious thing, it put a price on it. Both are right. Neither is the missing layer.

Watch what gets built next, and where. If the answer to "who authorized this machine" keeps coming back as "it paid, didn't it," then we'll have metered and monetized a web we still can't audit, and we'll find out the same way we always do, after the first machine does something expensive that nobody can prove it was allowed to do.

Max

Disclosure: Accords, where I'm Executive Chairman, is a portfolio position I chair.

Max Curnin has spent 11 years and 45+ investments on the physical rebuild of the economy, energy, compute, robotics, and the materials underneath. He writes The Agent Economy on the category he invests in.