Memory-Efficient Image Passing in the Document Scanner

Why UIImage Becomes a Bottleneck in a Scanning Pipeline

Dropbox’s iOS document scanner previously leaned on UIImage for all image handling. That container is convenient: it is immutable, reference-counted, carries orientation and display-scale metadata, and abstracts away pixel layout and color-space details. But those conveniences come with a hidden cost for processing pipelines. UIImage does not expose its raw pixels directly, and any attempt to read them — via CGDataProviderCopyData, for example — forces a full copy of the rasterized image. On modern iPhones capturing 12-megapixel photos, that means a 48 MB copy just to get at the data.

Pipeline Copies Add Up

A typical scan flow runs the captured image through several stages: resize, document detection, rectification, enhancement, and compression. When each processing step is written as a module that pulls pixels from the previous stage, wraps results back into a UIImage, and passes it along via an Objective-C block callback, every stage has to extract and copy the bitmap before doing any real work. The result is a chain of memcpy operations, with the image data being duplicated over and over for no functional benefit.

The practical symptom was visible in Instruments: using the scanner to capture a three-page document showed pronounced memory spikes while the camera was live and again at the moment the shutter was tapped. Those spikes sometimes escalated into out-of-memory process terminations — a serious problem, since iOS can kill the app when memory pressure builds.

A Custom Pixel Container

The fix was to abandon UIImage as the pipeline’s core currency and introduce a lightweight wrapper around a memory block, called DBPixelBuffer. It gives the scanner read access to raw pixels with no per-read copying. The trade-off is the loss of the immutability guarantees UIImage provides, which demands careful discipline when sharing buffers across processing stages.

The custom container brought side benefits. GPU acceleration requires images in a specific layout, typically 32-bit RGBA; having direct control over the buffer made it possible to minimize format conversions. DBPixelBuffer also kept an orientation flag, mirroring UIImage’s zero-cost rotation behavior, while still allowing the pipeline to normalize orientation before handing images to computer vision routines that expect upright input.

One conversion still required care: the iOS camera produces a UIImage backed by JPEG data. Decoding that with Apple’s SDK happens in an iOS-managed temporary buffer, so transitioning to a controlled DBPixelBuffer would require copying out of it — meaning two copies in memory at once. To avoid that, the scanner switched to libjpeg for decompression, writing decoded pixels directly into its own buffer. An added benefit was the ability to decode at reduced resolution when only a thumbnail was required.

Deferring Resolution Decisions

Even with direct pixel access, there are cases where an extra buffer is unavoidable — for instance when first converting a UIImage from the camera. The scanner defers that expense where possible. The pipeline was redesigned to accept a DBPixelBufferProvider, a protocol implemented by both UIImage and DBPixelBuffer, instead of requiring a concrete image type up front.

That deferral changes how thumbnails are produced. The old approach generated a large thumbnail immediately after capture, because the size of the eventual document crop was unknown and the crop needed to be at least screen resolution. With the lazy provider, no thumbnail is created early; the crop is rendered directly at the required output resolution when it is actually needed. The final figure above contrasts these two paths.

Laziness is not free. It can introduce latency at the moment the pixels are finally requested, and it raises questions about caching the resulting buffers. Even so, applying this pattern across all code paths reduced memory footprint uniformly and was the key to bringing down peak usage.

Measured Impact

The migration from UIImage to DBPixelBuffer cut memory spikes from 60 MB to 40 MB and reduced peak memory usage by more than 50 MB, according to the Instruments trace. The added complexity of owning a custom image container was justified by the significant drop in resource utilization and the improved stability of the scanning flow.