Cloudflare and OpenAI reported that change signals helped a research pilot discover hundreds of millions of unknown pages, without a public API or downstream quality results.
Cloudflare and OpenAI report early results from a web-discovery pilot
2026-10-15Frontier watchWATCH
Cloudflare and OpenAI reported early results from a research pilot that uses freshness, change, redirect, duplicate and potentially harmful-page signals to improve web discovery. A cohort covering about 1% of Cloudflare’s network helped OpenAI identify hundreds of millions of previously unknown pages. OpenAI still decides relevance and ranking, and discovery does not grant crawl permission. There is no public API or generally available product.
What this means for you
For teams building web search, RAG or research agents, change signals could reduce repeated crawling and improve freshness. The scale figures are jointly self-reported, downstream quality is not yet measured, and the pilot cannot be adopted today, so the right action is to watch for a public interface and independent outcome data.
FRONTIER WATCH
Public artifacts and validated results are different.
The recorded fact is the public work and its artifacts. Author-reported outcomes do not establish independent reproduction. Next checks include usable code, task conditions and third-party validation.
Evidence & related material
Technical details & scores
| Stage | research pilot; not GA and no public API |
|---|---|
| Cohort | about 1% of Cloudflare network |
| Signals | freshness, content changes, bad URLs, redirects, duplicates and potentially harmful pages |
| Reported Result | jointly reported discovery of hundreds of millions of previously unknown pages |
| Data Boundary | web metadata for discovery; post says page content is not shared for model training and discovery does not grant crawl permission |
| Time Basis | official page metadata 2026-10-15T13:00:00Z |
Technical impact 3
Engineering value 3
Adoption signal 1
Maturity 1
Verification cost 4
Risk 3