Testing a custom hook starts with the same question as testing anything else: how would you exercise this by hand? A hook written for a single component is usually covered by that component's tests. A reusable hook published to GitHub or npm is a different matter, since it needs to keep working as it changes.

Calling the hook directly in a test is the obvious first instinct — it is, after all, just a function. But it is not a pure function, and invoking it outside a component breaks the rules of hooks:

Error: Invalid hook call. Hooks can only be called inside of the body of a function component. This could happen for one of the following reasons:
  1. You might have mismatching versions of React and the renderer (such as React DOM)
  2. You might be breaking the Rules of Hooks
  3. You might have more than one copy of React in the same app
  See https://fb.me/react-invalid-hook-call for tips about how to debug and fix this problem.

Mocking useState and useEffect to make direct calls possible is a tempting workaround. It is also a bad one: it trades away most of the confidence the test was supposed to provide.

Testing by hand means rendering a component that uses the hook and interacting with it — in Storybook, for instance. The automated version of that process looks like this:

import * as React from 'react'
import useUndo from '../use-undo'

function UseUndoExample() {
	const { present, past, future, set, undo, redo, canUndo, canRedo } =
		useUndo('one')
	function handleSubmit(event) {
		event.preventDefault()
		const input = event.target.elements.newValue
		set(input.value)
		input.value = ''
	}

	return (
		<div>
			<div>
				<button onClick={undo} disabled={!canUndo}>
					undo
				</button>
				<button onClick={redo} disabled={!canRedo}>
					redo
				</button>
			</div>
			<form onSubmit={handleSubmit}>
				<label htmlFor="newValue">New value</label>
				<input type="text" id="newValue" />
				<div>
					<button type="submit">Submit</button>
				</div>
			</form>
			<div>Present: {present}</div>
			<div>Past: {past.join(', ')}</div>
			<div>Future: {future.join(', ')}</div>
		</div>
	)
}

export { UseUndoExample }

The test that automates the same steps:

import {render, screen} from '@testing-library/react'
import userEvent from '@testing-library/user-event'
import * as React from 'react'

import {UseUndoExample} from '../use-undo.example'

test('allows you to undo and redo', () => {
  render(<UseUndoExample />)
  const present = screen.getByText(/present/i)
  const past = screen.getByText(/past/i)
  const future = screen.getByText(/future/i)
  const input = screen.getByLabelText(/new value/i)
  const submit = screen.getByText(/submit/i)
  const undo = screen.getByText(/undo/i)
  const redo = screen.getByText(/redo/i)

  // assert initial state
  expect(undo).toBeDisabled()
  expect(redo).toBeDisabled()
  expect(past).toHaveTextContent(`Past:`)
  expect(present).toHaveTextContent(`Present: one`)
  expect(future).toHaveTextContent(`Future:`)

  // add second value
  input.value = 'two'
  await userEvent.click(submit)

  // assert new state
  expect(undo).not.toBeDisabled()
  expect(redo).toBeDisabled()
  expect(past).toHaveTextContent(`Past: one`)
  expect(present).toHaveTextContent(`Present: two`)
  expect(future).toHaveTextContent(`Future:`)

  // add third value
  input.value = 'three'
  await userEvent.click(submit)

  // assert new state
  expect(undo).not.toBeDisabled()
  expect(redo).toBeDisabled()
  expect(past).toHaveTextContent(`Past: one, two`)
  expect(present).toHaveTextContent(`Present: three`)
  expect(future).toHaveTextContent(`Future:`)

  // undo
  await userEvent.click(undo)

  // assert "undone" state
  expect(undo).not.toBeDisabled()
  expect(redo).not.toBeDisabled()
  expect(past).toHaveTextContent(`Past: one`)
  expect(present).toHaveTextContent(`Present: two`)
  expect(future).toHaveTextContent(`Future: three`)

  // undo again
  await userEvent.click(undo)

  // assert "double-undone" state
  expect(undo).toBeDisabled()
  expect(redo).not.toBeDisabled()
  expect(past).toHaveTextContent(`Past:`)
  expect(present).toHaveTextContent(`Present: one`)
  expect(future).toHaveTextContent(`Future: two, three`)

  // redo
  await userEvent.click(redo)

  // assert undo + undo + redo state
  expect(undo).not.toBeDisabled()
  expect(redo).not.toBeDisabled()
  expect(past).toHaveTextContent(`Past: one`)
  expect(present).toHaveTextContent(`Present: two`)
  expect(future).toHaveTextContent(`Future: three`)

  // add fourth value
  input.value = 'four'
  await userEvent.click(submit)

  // assert final state (note the lack of "third")
  expect(undo).not.toBeDisabled()
  expect(redo).toBeDisabled()
  expect(past).toHaveTextContent(`Past: one, two`)
  expect(present).toHaveTextContent(`Present: four`)
  expect(future).toHaveTextContent(`Future:`)
})

This is the recommended approach in most cases. The test reads clearly, and the thing under test is the hook as it is actually used.

Pulling the hook away from the UI

The real-world example breaks down when the supporting component grows complicated. Test failures then come from the example rather than the hook, and coverage of awkward cases forces multiple example components. Those components still have value — Storybook can use them — but a helper with no UI, which touches the hook's return value directly, can be more convenient:

import * as React from 'react'
import { render, act } from '@testing-library/react'
import useUndo from '../use-undo'

function setup(...args) {
	const returnVal = {}
	function TestComponent() {
		Object.assign(returnVal, useUndo(...args))
		return null
	}
	render(<TestComponent />)
	return returnVal
}

test('allows you to undo and redo', () => {
	const undoData = setup('one')

	// assert initial state
	expect(undoData.canUndo).toBe(false)
	expect(undoData.canRedo).toBe(false)
	expect(undoData.past).toEqual([])
	expect(undoData.present).toEqual('one')
	expect(undoData.future).toEqual([])

	// add second value
	act(() => {
		undoData.set('two')
	})

	// assert new state
	expect(undoData.canUndo).toBe(true)
	expect(undoData.canRedo).toBe(false)
	expect(undoData.past).toEqual(['one'])
	expect(undoData.present).toEqual('two')
	expect(undoData.future).toEqual([])

	// add third value
	act(() => {
		undoData.set('three')
	})

	// assert new state
	expect(undoData.canUndo).toBe(true)
	expect(undoData.canRedo).toBe(false)
	expect(undoData.past).toEqual(['one', 'two'])
	expect(undoData.present).toEqual('three')
	expect(undoData.future).toEqual([])

	// undo
	act(() => {
		undoData.undo()
	})

	// assert "undone" state
	expect(undoData.canUndo).toBe(true)
	expect(undoData.canRedo).toBe(true)
	expect(undoData.past).toEqual(['one'])
	expect(undoData.present).toEqual('two')
	expect(undoData.future).toEqual(['three'])

	// undo again
	act(() => {
		undoData.undo()
	})

	// assert "double-undone" state
	expect(undoData.canUndo).toBe(false)
	expect(undoData.canRedo).toBe(true)
	expect(undoData.past).toEqual([])
	expect(undoData.present).toEqual('one')
	expect(undoData.future).toEqual(['two', 'three'])

	// redo
	act(() => {
		undoData.redo()
	})

	// assert undo + undo + redo state
	expect(undoData.canUndo).toBe(true)
	expect(undoData.canRedo).toBe(true)
	expect(undoData.past).toEqual(['one'])
	expect(undoData.present).toEqual('two')
	expect(undoData.future).toEqual(['three'])

	// add fourth value
	act(() => {
		undoData.set('four')
	})

	// assert final state (note the lack of "third")
	expect(undoData.canUndo).toBe(true)
	expect(undoData.canRedo).toBe(false)
	expect(undoData.past).toEqual(['one', 'two'])
	expect(undoData.present).toEqual('four')
	expect(undoData.future).toEqual([])
})

Calling into the hook this way requires act, and in exchange it reaches cases that are hard to express as rendered components.

The helper gets unwieldy once the hook waits on mocked HTTP requests, or needs to rerender with different props. Each new requirement pushes more domain-specific logic into the setup function. That is precisely the gap renderHook from @testing-library/react fills:

import { renderHook, act } from '@testing-library/react'
import useUndo from '../use-undo'

test('allows you to undo and redo', () => {
	const { result } = renderHook(() => useUndo('one'))

	// assert initial state
	expect(result.current.canUndo).toBe(false)
	expect(result.current.canRedo).toBe(false)
	expect(result.current.past).toEqual([])
	expect(result.current.present).toEqual('one')
	expect(result.current.future).toEqual([])

	// add second value
	act(() => {
		result.current.set('two')
	})

	// assert new state
	expect(result.current.canUndo).toBe(true)
	expect(result.current.canRedo).toBe(false)
	expect(result.current.past).toEqual(['one'])
	expect(result.current.present).toEqual('two')
	expect(result.current.future).toEqual([])

	// add third value
	act(() => {
		result.current.set('three')
	})

	// assert new state
	expect(result.current.canUndo).toBe(true)
	expect(result.current.canRedo).toBe(false)
	expect(result.current.past).toEqual(['one', 'two'])
	expect(result.current.present).toEqual('three')
	expect(result.current.future).toEqual([])

	// undo
	act(() => {
		result.current.undo()
	})

	// assert "undone" state
	expect(result.current.canUndo).toBe(true)
	expect(result.current.canRedo).toBe(true)
	expect(result.current.past).toEqual(['one'])
	expect(result.current.present).toEqual('two')
	expect(result.current.future).toEqual(['three'])

	// undo again
	act(() => {
		result.current.undo()
	})

	// assert "double-undone" state
	expect(result.current.canUndo).toBe(false)
	expect(result.current.canRedo).toBe(true)
	expect(result.current.past).toEqual([])
	expect(result.current.present).toEqual('one')
	expect(result.current.future).toEqual(['two', 'three'])

	// redo
	act(() => {
		result.current.redo()
	})

	// assert undo + undo + redo state
	expect(result.current.canUndo).toBe(true)
	expect(result.current.canRedo).toBe(true)
	expect(result.current.past).toEqual(['one'])
	expect(result.current.present).toEqual('two')
	expect(result.current.future).toEqual(['three'])

	// add fourth value
	act(() => {
		result.current.set('four')
	})

	// assert final state (note the lack of "third")
	expect(result.current.canUndo).toBe(true)
	expect(result.current.canRedo).toBe(false)
	expect(result.current.past).toEqual(['one', 'two'])
	expect(result.current.present).toEqual('four')
	expect(result.current.future).toEqual([])
})

Under the hood it does much the same work as a hand-written setup function. On top of that it supplies a way to rerender the component rendering the hook (useful for effect dependency changes), a way to unmount it (useful for testing effect cleanup), and several async utilities for waits of unspecified length. Multiple hooks can be tested at once by calling each of them inside the callback passed to renderHook.

Without such a tool, the boilerplate needed for a test-only component is error-prone, and it can demand more time than the hook it supports.

Choosing

For a hook like useUndo, the real-world example usage is the better trade-off between understandability and coverage. More complicated hooks are where @testing-library/react earns its place.