R2 Super Slurper: A New Migration Path for Object Storage
Moving large datasets between cloud storage providers has traditionally meant writing and maintaining custom migration scripts, dealing with rate limits, and carefully orchestrating multi-step processes. Cloudflare's R2 Super Slurper aims to replace that workflow with a managed import service that handles the heavy lifting through a user interface and API.
The first iteration of the Super Slurper targets S3-compatible buckets as the source, copying their objects into an existing R2 bucket in a single, one-time import operation. You point the service at your source bucket, grant read permissions to the objects you want to move, and an asynchronous job takes care of the rest. This design is intended to cover the most common migration scenarios without requiring any scripting on your part.
Reusable Source Configurations and Incremental Migration
For buckets that contain many different paths that need to be validated, the Super Slurper lets you save the definitions and credentials for a given source bucket. You can then run separate migration operations against different folders within that bucket without entering the same connection details repeatedly.
The planned roadmap extends beyond one-off imports. An automatic "sip by sip" mode is in development for incremental migrations, where objects are copied to R2 on demand as end-user requests arrive. This approach allows you to begin serving traffic from R2 immediately while gradually populating the bucket from the origin storage. For any object that has already been copied to R2, subsequent requests are served entirely from R2, meaning you stop paying egress fees on that object's traffic right away.
The request flow for this incremental mode is as follows:
- Check for Object — A request arrives at Cloudflare, and the R2 bucket is checked for the requested object. If the object exists, R2 serves it.
- Copy the Object — If the object does not exist in R2, the request is forwarded to the origin bucket. Once the object is returned, it is served to the requester and copied into R2.
- Serve the Object — All subsequent requests for that object are served directly from R2.
Migration Sources and Access
The beta launch of the Super Slurper focuses solely on S3-compatible buckets. Additional source types are expected to become available during 2023. The service is currently in private beta; to use it, you must be an R2 customer and join the waitlist from your Cloudflare dashboard. Cloudflare will work with early beta users to refine the service before opening up a broader public beta.
