Skip to the yard
Waxsealai

Cloudflare Origin Range Requests Cuts R2 Latency and Cache Traffic

Shipped
Shipper
Matthew Gumport
Readers
316

Cloudflare’s chunk based range caching cut R2’s p99 time to first byte roughly in half and tiered cache traffic about 6%.

Cloudflare Origin Range Requests Cuts R2 Latency and Cache Traffic

Cloudflare’s new Origin Range Requests feature changes how its cache retrieves large files: instead of downloading a customer configured block for each partial request, the edge fetches bounded chunks aligned with its cache storage. For R2, Cloudflare reports that the change roughly halved p99 time to first byte across storage regions and, after a later improvement, reduced tiered cache traffic for R2 and similar workloads by about 6% week over week.

The feature is now available as a self service setting on all plans. Customers select traffic with a Cache Rule; Cloudflare manages the fetches and response assembly. The practical target is traffic that asks for pieces of large cacheable objects, such as video seeks, resumable downloads, package files, disk images, and model artifacts.

Why range caching needed a new design

An HTTP range request asks for only part of a resource. A media player might request bytes near a seek position, while a download manager might fetch a missing section after an interruption. The client sends a regular GET request with a Range header. If the server can satisfy it, the response uses status 206 Partial Content, with Content-Range identifying the bytes returned.

Cloudflare has supported range caching for years, but its earlier implementation mapped client requests onto fixed size ranges configured for each zone. That setting forced a tradeoff. Small ranges created many origin requests and held memory while Cloudflare combined data. Large ranges reduced request counts but could make the network transfer far more than a client needed. Changing the configured size also invalidated cache entries because the range size formed part of the cache key.

The costs could compound across Cloudflare’s distributed cache. Extra bytes might travel from an origin to one data center, across cache tiers, and then toward the client. A request for a small portion of a file could trigger a much larger transfer at each stage. Concurrent requests within the same configured range could also block one another, even when they needed different bytes.

The older nginx based cache had another limitation: it could not combine range caching with concurrent streaming, which lets multiple clients read a file while it is still being written to cache. Enabling both could truncate responses. Cloudflare therefore had to finish downloading each range before serving it, and popular files could require a separate origin fetch for every range. The right configured size varied by workload and cache layout, leaving customers to tune around internal cache behavior.

Cache storage supplies the chunk size

Cloudflare’s move from nginx to its Pingora based caching server provided a different foundation. Pingora stores cached files on disk in fixed size chunks, which already supports resuming interrupted downloads. Origin Range Requests uses those same chunks as the unit for fetching and storing partial content; customers no longer set a separate range size.

For each client request, Cloudflare checks which requested bytes are already cached. It serves available data and asks the origin for missing bytes using ranges aligned to chunk boundaries. The fetched data goes into the ordinary cache storage, while the client receives only the bytes it asked for. A request for bytes 150-250, for example, might require fetching the chunk that contains them, but the downstream response still covers only bytes 150 through 250.

That means the design still does extra work when a request is smaller than a chunk or does not align with chunk boundaries. The difference is that this overhead is bounded by the cache’s chunking scheme rather than by a customer selected range size that could lead to unpredictable transfers and operational pressure.

One response can draw on several data sources

Serving a range becomes a coordination task when its bytes are split between disk, a download still in progress, and a new origin response. The edge reads the asset headers and range metadata, retrieves cached or streaming data, fetches any missing coverage, stores the new bytes, and records the completed ranges. It then assembles the requested portion and sets the client facing Content-Range and Content-Length.

One response can therefore combine a cached prefix, bytes from an in flight download, and a newly fetched suffix. If another request needs bytes covered by an existing download, it can join that work instead of triggering another origin fetch. This sharing matters during bursts of requests for popular large files.

Correctness depends on every piece belonging to the same version of the file. Cloudflare checks that origin responses match the requested ranges, that the total object size stays consistent, and that the entity tag remains the same across subrequests. The client’s If-Range header can require a known validator; if the representation has changed, Cloudflare returns the full representation instead of joining bytes from different versions.

Compression can also make a requested range unsafe to compose. A range over compressed bytes may refer to different content than the same range over a decompressed representation. When Cloudflare is about to decompress an object, it serves the transformed full response rather than claiming to return a range it cannot honor accurately. Satisfiable ranges receive 206 Partial Content; unsatisfiable ones receive 416 Range Not Satisfiable; responses that cannot safely be ranged are served in full.

Production results and practical limits

Cloudflare moved users of its earlier range caching onto the new implementation, beginning with workloads that produced the greatest strain: concurrent requests for very large files with bursty traffic. Previous configured sizes ranged from tens of kilobytes to tens of gigabytes. The extremes could turn one client request into either a very large transfer across data centers or a long queue of tiny origin requests holding memory.

R2 was the largest internal user, with a configured range size of 64 MiB. A small read crossing a range boundary could therefore drive 128 MiB of transfer between data centers. After the migration, Cloudflare measured roughly half the p99 time to first byte across R2 storage regions on both cache hits and misses.

A subsequent coordinator change allowed Cloudflare to combine bytes already on disk with bytes fetched upstream in one response. Previously, a response came entirely from cache or entirely from upstream. With the mixed response path enabled for R2, tiered cache traffic for R2 and similar workloads fell about 6% week over week. Cloudflare describes the reduction as petabytes less data moving between its data centers each day.

Origin Range Requests does not change whether an object qualifies for caching: enabling the feature will not make an otherwise uncacheable response cacheable. Partial origin fetches also depend on the origin supporting byte ranges. Customers can enable the setting for matched large file traffic in the dashboard under Caching > Cache Rules, or through the API with origin_range_requests: { "mode": "on" }. The next step Cloudflare describes is making this the default for certain file types, with Cache Rules available to modify behavior.

Source: Matthew Gumport · blog.cloudflare.com

Same vesselRelated cargo