Using Hermes’s Quicksort to run Doom: A tale of JavaScript exploitation

A Quicksort Flaw That Went from Blind Read to Code Execution

In August 2020, Meta’s Bug Bounty program received a peculiar report about Hermes, the company’s open source JavaScript engine optimized for React Native and its Spark AR platform. The report detailed a crash in Hermes’s Quicksort implementation that resulted in blind out-of-bounds (OOB) memory reads. While similar submissions are typically awarded between $500 and $3,000, deeper investigation showed this flaw could be escalated to arbitrary code execution — ultimately earning the researcher a total bounty of $12,000.

To prove the impact, the research team turned the exploit into a working demo: running the classic 1993 video game Doom directly from inside Hermes.

Sorting the Unsorted

The initial report arrived as a tangled JavaScript file of roughly 180 lines, likely the output of a fuzzing engine. After cleaning it up, the team traced the root cause to Array.prototype.sort — specifically to a recent set of changes that made Hermes’s sort implementation stable.

In a stable sort, equal elements retain their original relative order. Hermes achieves this by maintaining a parallel index vector prefilled with original array positions. Whenever two array elements are swapped, their corresponding indices are swapped too. If two elements compare as equal, the algorithm falls back to comparing their stored indices to break the tie.

ExecutionStatus quickSort(SortModel *sm, uint32_t begin, uint32_t end) {
  uint32_t len = end - begin;
  // [1]
  std::vector<uint32_t> index(len); // Array of original indices of items
  for (uint32_t i = 0; i < len; ++i) {
    index[i] = i;
  }
  ...
  return doQuickSort(sm, index, /* ... */ );
}

...

ExecutionStatus
_swap(SortModel *sm, std::vector<uint32_t> &index, uint32_t i, uint32_t j) {
  if (sm->swap(i, j) == ExecutionStatus::EXCEPTION) {
    return ExecutionStatus::EXCEPTION;
  }
  // [2]
  std::swap(index[i], index[j]);
  ...
}

...

CallResult<bool>
_less(SortModel *sm, std::vector<uint32_t> &index, uint32_t i, uint32_t j) {
  auto res = sm->compare(i, j);
  if (res == ExecutionStatus::EXCEPTION) {
    return ExecutionStatus::EXCEPTION;
  }
  // [3]
  return (*res != 0) ? (*res < 0) : (index[i] < index[j]);
}

The Array.prototype.sort method accepts an optional callback that defines a custom comparison order:

const array = [3,1,4,2];
array.sort(function compareFn(first, second) { ... });

Here lies the twist: what happens when that callback mutates the array mid-sort? When sorting begins, the engine operates on the array’s initial state. But after the comparison function runs, the engine sees the mutated version. The index vector, however, is sized to the original array and stays fixed. That disconnect — where the array can grow beyond its original length while the index vector remains stale — allows sort logic to touch indices that are out of bounds for the index buffer.

A Quick Walk Through Quicksort

To understand which elements become accessible, recall how Quicksort works. It starts by picking a pivot, usually the median of three candidate elements. The algorithm then scans from the left (i) for an element the callback deems larger than the pivot, and from the right (j) for something smaller. Those two elements are swapped, and the search resumes until i and j cross. At that point, the pivot is swapped into its final spot.

Engineering an OOB Read

The exploitation strategy uses the callback to steer the sort beyond the original array’s bounds:

1. Extend the target array so it is longer than the index vector.
2. Survive the first three comparisons — the median-of-three pivot selection.
3. Return -1 continuously to advance i toward a cell past the original array’s length.
4. Return 0 once i reaches that target cell, indicating equality and triggering an OOB read on the index buffer.

var array = [1,2,3,4,5,6,7,8,9,10];
const initial_length = array.length
const extended_length = initial_length*2
var cnt = 0

array.sort(function compareFn(first, second) {

  cnt++;

  // [1] Extend the array
  if (cnt == 1){
    for (i=0; i<extended_length-initial_length; i++){
      array.push('X')
    }
  }

  // [2] Get past the median-of-three step
  if (cnt > 3){

    // i represents the left index moving to the right
    const i = cnt - 1;

    // target_cell can be anything between initial_length and extended_length
    const target_cell = 15;

    if (i != target_cell){
      // [3] If we didn't yet reach the target cell, keep advancing
      return -1;
    } else {
      // [4] If we did reach the target cell, trigger an OOB index lookup
      print("Look for the crash...")
      return 0;
    }
  }
});

When run against an ASAN build of Hermes, the sequence produced this crash:

Look for the crash...
=================================================================
==1519183==ERROR: AddressSanitizer: heap-buffer-overflow ...
READ of size 4 at 0x604000003fc8 thread T0
SCARINESS: 27 (4-byte-read-heap-buffer-overflow-far-from-bounds)
#0 0xcd011a in hermes::vm::_less(...) hermes/lib/VM/JSLib/Sorting.cpp:35
#1 0xccf3de in hermes::vm::quickSortPartition(...) hermes/lib/VM/JSLib/Sorting.cpp:175
#2 0xccf3de in hermes::vm::doQuickSort(...) hermes/lib/VM/JSLib/Sorting.cpp:262
#3 0xccef6e in hermes::vm::quickSort(...) hermes/lib/VM/JSLib/Sorting.cpp:332
#4 0x953f41 in hermes::vm::arrayPrototypeSort(...) hermes/lib/VM/JSLib/Array.cpp:1248
#5 0x8b3ee9 in hermes::vm::NativeFunction::_nativeCall(...) hermes/include/hermes/VM/Callable.h:539
#6 0xa27a2f in hermes::vm::Interpreter::handleCallSlowPath(...) hermes/lib/VM/Interpreter.cpp:318
...

From Array Sort to Arbitrary Code Execution

Once we confirmed the out-of-bounds read/write in Hermes’s Array.prototype.sort, the critical question became: could this go beyond a crash? The sort algorithm works by swapping elements, which also swaps the indices of those elements. In theory, that meant we might be able to swap values outside the bounds of the index vector itself.

Forcing the Crossing Condition

Quicksort performs two kinds of swaps: between elements at indices i and j, and between j and the pivot. For loosely controlled writes outside the index, the first type is the interesting one because the algorithm gives us control over both pointers. A swap only occurs if the pointers haven’t crossed (i.e., i < j), but to write beyond the index, they’d need to pass each other.

We bypassed that restriction by underflowing j (a 32-bit unsigned integer) to 0xffffffff, which on a 32-bit system points to the element before the index vector. The catch: reading beyond the array returns an “empty” element that the spec treats as greater than all others, causing an infinite loop in the comparator. We fixed that by installing a custom getter at index 0xffffffff, so the sort wouldn’t stall. With that in place, swapping index[0xffffffff] with index[i] gave us a usable primitive, especially since we could nudge i and j to control which out-of-bounds positions we touched.

Controlling the Contents

Since we had only limited control over what the swap wrote, we targeted the adjacent memory. The index is stored in a std::vector allocated by malloc. If we could shape the heap so that we controlled the blocks on either side of the index, we could dictate what got swapped.

malloc hands out large allocations via mmap, which encouraged a predictable layout: each mmap region is placed next to existing mapped memory. While Hermes objects usually go through its garbage collector, the ArrayBuffer backing stores are allocated directly through malloc. So we built an arrangement of four adjacent mmap buffers where we retained full control of the contents.

• Allocate two ArrayBuffers (C and B).
• Start the sort, which creates the index between them.
• Allocate a third ArrayBuffer (A) from inside the sort callback.

With the buffers laid out in memory, we could use our swap out-of-bounds write to exchange elements belonging to A with elements from B or C.

Each buffer lives inside its own allocation, with malloc metadata—including chunk size—stored before the region. The key upgrade came when we swapped a chunk’s size field with a larger value. If the forged size covered both B and C, then freeing B while still holding C produced an inconsistency: C’s contents stayed mapped while the system believed the whole region was released. Allocating a new ArrayBuffer sized to that region reclaimed it, giving us overlapping chunks—where writes through one buffer alias the contents of the other.

From Overlap to Arbitrary Access

Overlapping allocations on their own only allow editing whatever lands on top. To turn that into established read/write primitive, we needed Hermes’s GC to place JS object metadata inside the freed slab that we could still write through. That was possible because the GenGC’s 4 MiB segments are mmap allocations too. So we could request a fresh ArrayBuffer whose buffer overlaps an old one, keep editing it, free the old buffer, and let the GC map a new segment into exactly those pages.

That gave us simultaneous control over a memory aperture that visibly doubled as GC-allocated ArrayBuffer metadata. Each ArrayBuffer object holds a pointer to its data. To gain arbitrary access, we wrote through our controlled views to patch the pointer of a target ArrayBuffer so that it pointed anywhere we wanted in the process address space. Reading and writing through that buffer, which still behaved like an ordinary JS object on the application side, became generic read/modify/write access to memory.

A Useable Chain

• Stage adjacent mmapped chunks to control an overlapping allocation.
• Corrupt a chunk size to acquire two live handles to the same buffer.
• Surrender one of the reclaimed slots to the GC as heap segment, then overwrite internal object pointers, like ones pointing to ArrayBuffer contents.

With ordinary JS wrappers at arbitrary addresses, the standard exploitation steps follow: place a ROP chain somewhere reachable, reveal the address space layout to bypass ASLR, redirect a function pointer or similar dispatch through a stack-pivot gadget, and mark shellcode executable. We didn’t go through every step for this article because these are common techniques—the first half, converting the out-of-bounds swap into a core read/write capability, was the novel work. More practically, it was enough to escape Hermes’s sandbox and run Doom.

Mitigations and Bounty

Since the culprit was introduced only days before the report, this flaw never ended up in a Hermes public release. The researcher who filed the bug earned a $12,000 payout. Since then, engineering efforts have centered on expanding internal fuzzing that targets this class of code and hardening surfaces to shrink the blast radius if another native bug appears, especially on ArrayBuffer and sorting internals. The vulnerability was found through Meta’s dedicated bug bounty track for Hermes and SparkAR, and standalone reports with full proof-of-concept exploits are eligible for additional bonuses.