Five Years of Watching the Internet
Cloudflare Radar launched in 2020 with a simple premise: use the aggregated traffic flowing through Cloudflare’s network to make the Internet’s patterns visible. The original trio of components — Internet Insights, Domain Insights, and IP Insights — remains the core, but the scope has broadened considerably. Radar now covers routing, protocol adoption, DNS, security disruptions, and AI bot activity, all exposed through a public API designed for easy programmatic access.
That expansion reflects a changing Internet. New protocols and use cases emerge alongside new threats; connectivity is disrupted not only by physical accidents but by deliberate government blocking. Radar’s role is to provide actionable visibility into these trends at local, network, and global levels.
Shining a Light on Security
Cloudflare’s Research team takes a practical approach, and several of its projects have produced datasets that have made their way onto Radar. The 2025 launch of the Certificate Transparency (CT) section on Radar is one example. This new section consolidates metrics for the CT ecosystem into a single dashboard, allowing users to explore certificates, Certificate Authorities (CAs), and the CT logs that record every issued certificate. The launch effectively retires the older Merkle Town CT dashboard, which had been in service since 2018.
Another Research collaboration added visibility into connection tampering. A 2024 paper, “Global, Passive Detection of Connection Tampering,” found that roughly 20% of all connections to Cloudflare close unexpectedly before any useful data exchange occurs — a pattern consistent with third-party interference. That work led to a new graph on Radar’s Network Layer Security page showing TCP resets and timeouts. The visualization provides a country-level perspective on potential tampering activity and can corroborate reports of content blocking.
Post-quantum encryption is a third area where Research and Radar have converged. Cloudflare has supported post-quantum work since 2017 and made it a free, default part of its delivery services in 2023. But client-side support is required for full adoption, so Radar’s Adoption & Usage page now tracks the post-quantum encrypted share of HTTPS request traffic. The numbers show clear momentum: starting 2024 at under 3%, the share has grown to just over 47% as major browsers and code libraries enabled post-quantum support by default.
Tracking the AI Crawler Economy
The rise of AI platforms has fundamentally changed the economics of web crawling. For decades, content creators allowed their sites to be crawled in exchange for referral traffic from search engines. That dynamic has broken down — AI platforms crawl aggressively to collect training data without compensating publishers, while search engines morph into answer engines that send fewer visits back to the source. Content owners have responded by demanding more visibility and more control.
Cloudflare offers customers tools to see how frequently AI crawlers are scraping their content. Radar extrapolates that into an aggregated view. The AI Insights page includes graphs for traffic trends by bot and by crawl purpose, both filterable by industry set. This allows customers to benchmark their own traffic patterns against the broader industry.
A key metric on that page is the crawl-to-refer ratio: the number of HTML pages a crawler consumes relative to the number of page visits it refers back to the site. Tracking this ratio by platform over time gives content creators a concrete sense of how unbalanced the reciprocal traffic has become — and how the transition from search engines to answer engines is accelerating the shift.
AI Insights also examines how websites are signaling their preferences. The robots.txt file has served for decades as the gatekeeper for crawler access, and well-behaved crawlers respect it. Radar analyzes the robots.txt files across its top 10,000 domains to show how many sites explicitly allow or disallow specific AI crawlers, and how complete that access or restriction is. Filtering by domain category gives site owners a view into how their peers are handling AI crawlers.
Routing visibility for network health
Routing selects the path that IP packets take across networks from origin to destination. When routing goes wrong, entire networks can be knocked offline, and the blast radius can extend to other providers and services depending on how interconnected the affected network is. Route leaks happen when a routing announcement propagates beyond its intended scope, reaching networks it shouldn’t. Origin hijacks occur when an attacker creates fake announcements for a targeted prefix, falsely claiming an AS under their control is the origin — which would redirect traffic for those IP addresses to the attacker’s network.
Cloudflare added route leak detection to Radar in 2022 and origin hijack detection in 2023. Both features help network operators and researchers identify which networks may be involved in such events — whether as leaker or hijacker, or as victim. Also in 2023, Radar launched notifications for route leaks and origin hijacks, automatically alerting subscribers via email or webhook when an event is detected so they can act quickly.
In 2025, two new capabilities arrived. Real-time BGP route visibility shows how a given network prefix connects to other networks — the path packets take from those IP addresses to major “tier 1” providers. Network administrators can use this during outages, new deployments, or route leak investigations.
An AS-SET is a grouping of related networks, historically used for purposes like listing the downstream customers of a provider. AS-SET monitoring, also newly announced, lets network operators track valid and invalid AS-SET memberships for their networks, which can prevent misuse and issues like route leaks.
Data access and programmatic use
Radar has expanded beyond visualizations to expose its underlying data directly. The Radar API, launched in 2022, requires only an access token and provides access to all data shown on Radar, plus more specific filters. Users can pull Radar data into their own tools, websites, and applications. The example below shows a simple API call returning the global distribution of human and bot traffic over the last seven days.
curl -X 'GET' \
'https://api.cloudflare.com/client/v4/radar/http/summary/bot_class?name=main&dateRange=1d' \
-H 'accept: application/json' \
-H 'Authorization: Bearer $TOKEN'{
"success": true,
"errors": [],
"result": {
"main": {
"human": "72.520636",
"bot": "27.479364"
},
"meta": {
"dateRange": [
{
"startTime": "2025-10-19T19:00:00Z",
"endTime": "2025-10-20T19:00:00Z"
}
],
"confidenceInfo": {
"level": null,
"annotations": []
},
"normalization": "PERCENTAGE",
"lastUpdated": "2025-10-20T19:45:00Z",
"units": [
{
"name": "*",
"value": "requests"
}
]
}
}
}The Model Context Protocol (MCP) is a standardized way to make information available to large language models. Similar to how an API works, MCP offers a documented method for programs to integrate external services. It allows AI programs to go beyond their training data and connect to new sources of information and external tools. The Radar MCP server gives MCP clients access to Radar data and tools, enabling natural language exploration.
Radar’s URL Scanner has become one of its most popular tools, scanning millions of sites since launching in 2023. It lets users safely check whether a site may contain malicious content and provides details on technologies used, headers, cookies, and links. The scanner is available on Radar, through the API, and via the MCP server.
The Radar user interface has also improved for usability over the years. New data sets and capabilities are continually added to the search bar, which now supports countries, ASNs, IP address prefixes, certificate authorities, bot names, and IP addresses. Default date ranges have grown, and users can select custom ranges of up to one year. In 2024, Radar launched in 14 languages and dialects, applying localization to downloaded and embedded content as well.
These updates over the last five years support Internet measurement, transparency, and resilience. Combined, the data sets and tools offer one of the most comprehensive views of the Internet available. Cloudflare plans to keep extending Radar with more tools, data sets, and visualizations to answer questions in areas like AI, performance, adoption, usage, and security.
