About
A reference price is only worth citing if you know who publishes it and how they are paid. So: this is a one-person project, and here is the person.
I love getting lost in data. This project is the intersection of the three things I find hardest to put down: analysis, spotting a pattern before anyone has named it, and watching what AI is actually doing rather than what it is said to be doing.
By trade I am a freelance consultant and analyst for financial institutions — banks, funds — with ten years of it behind me. That work is where the habit comes from: the first question about any number is who produced it, against what denominator, and whether anyone else could reproduce it. Almost nothing published about AI and the web survives that question. Block rates get quoted as shares of “the web” with no frame; prices get cited with no sample size.
Data and AI started as the hobby I disappeared into at the weekend. The Crawl Price Index is what happened when it stopped staying in the weekend. I built the crawler, the pipeline, the dashboard and the methodology, and I read every edition myself before it goes out, twice a week. I get carried away in the detail, which for this particular job turns out to be the right defect — and what is being measured here is going to change a great deal about how the web works.
I am happy to talk about any of it — the method, the numbers, where they are weak. Email me or connect on LinkedIn.
The index is funded by subscriptions to the Terminal — that is the whole of it. It takes no money from AI companies, no money from publishers, no advertising, no affiliate revenue, and no sponsorship. Nobody in the market it measures pays for its existence.
That constraint is the point. A reference price that takes money from a participant stops being a reference price, which is why the free tier — the headline figures, the aggregate dashboard and the weekly email — is free to cite with attribution by anyone, including the companies the data covers.
I hold no direct position in any AI company, publisher, CDN or payments provider named anywhere in this dataset. Like most people with retirement savings I hold broad, diversified index funds, and those almost certainly contain AI companies among hundreds of others; I do not pick them and cannot vary the weights. I do not sell consulting to the sites I measure, and the index does not recommend prices to publishers — it records what they declare.
One disclosure worth making plainly: the machine-payment registry the index tracks is a market I find genuinely interesting, and I may eventually list an endpoint of my own in it. If that happens it will be labelled as mine in the data, and excluded from any headline figure.
Every figure states its denominator and every method is written down so it can be checked. If you can reproduce a different answer, or your domain is recorded incorrectly, write to hello@crawlpriceindex.com. Corrections are made in the next edition and logged on status (method changes in the changelog) — the archive is never quietly rewritten.