Pretending to Order Lunch Is Incident Response Practice

Incident response training has a classic problem: how do you give people hands-on practice before they're needed in a real incident? Classroom sessions and prepared slide decks teach the mechanics of roles and processes, but they don't replicate the pressure of an actual response. At Slack, the answer turned out to be lunch.

What started as an exercise run by Blackrock 3 Partners during a 2018 training session became a regular part of Slack's incident management program. The core idea is simple: assemble a group, announce that the promised lunch has fallen through, and have them use the company's real incident response practices to solve the problem under time pressure.

Ingredients for a Working Exercise

Slack's Incident Lunch is intentionally lightweight. Participants need no preparation or subject matter expertise beyond being able to order food. The exercise assumes that everyone knows how lunch works, which makes it accessible across the company, including to non-technical teams such as Customer Experience, Customer Success, and Sales.

The framework requires minimal resources to run repeatedly:

  • An outline for setup and kickoff, stored as a markdown file in a GitHub repository and run as a GitHub Pages slide deck
  • A conference room or two
  • A workflow to announce and invite participants
  • One or two facilitators
  • A budget of roughly $300–500 per session

That cost is an order of magnitude less than bringing in a third-party firm for an hour of on-site training. And the facilitators get a free lunch out of the deal—an incentive Slack used to keep the program going weekly.

A crucial addition came from a former lead engineer on Slack's App Ops team: Chaos Cards. These cards introduce variability and unpredictability into the exercise. One card might randomly disable a participant's laptop for the rest of the session; another might force the team to draw a new card in two minutes. Cards are drawn on a timed cadence, typically starting at five-minute intervals, and simulate the kind of unexpected disruptions that surface during real incidents.

Why It Works for the Organizers

Elaborate incident games—role-playing scenarios, story-driven adventures, or tools similar to Keep Talking and Nobody Explodes—can be engaging for engineers, but they demand significant setup and maintenance effort. They also tend to limit participation to small groups. The Incident Lunch avoids that overhead entirely and can scale to any group that can fit in a conference room.

Slack set up a channel to coordinate facilitators, and the exercise can be run in remote offices where a willing facilitator exists. The low barrier to entry made it a practical and repeatable training option alongside more formal coursework.

What Happens During the Exercise

The session starts with a 15-minute refresher on Slack's incident process. Participants are walked through the basics of IMS, common incident roles, how an incident starts at the company, and the primary goal of incident response: restoring service as quickly as possible. Tips are offered on communication, developing multiple plans, time-boxing work, and focusing on roles rather than individuals.

Then the surprise is sprung: the lunch order fell through. The bike messenger delivering the food ended up on the other side of town. The group must mobilize as an incident response team and get lunch delivered before the two-hour session ends.

Ground rules keep the exercise realistic:

  • The order must be placed outside a designated lunch exclusion zone
  • Teams can choose pickup or delivery
  • A per-person budget limit of roughly $25 is set
  • Receipts must be kept for expense reporting
  • Dietary restrictions must be accommodated
  • Any tools at hand—laptops, phones, Slack, Zoom—are fair game

An Incident Commander is selected, ideally in advance. For groups new to the process, the facilitator often takes the role. Someone draws a Chaos Card every five minutes, with the facilitator able to adjust the pace up or down depending on how the group is doing. Once lunch arrives, the group runs a quick retrospective while eating, with the facilitator sharing observations and prompting the participants for their own insights.

What Repeated Runs Revealed

The Chaos Cards proved valuable for compounding time pressure with unpredictability. They give facilitators levers to make the exercise harder or easier, whether by speeding up the draw cadence or ensuring difficult cards surface early for returning participants. Reusing participants also uncovered people with a natural aptitude for incident facilitation that standard training never surfaced. That discovery became an internal recruiting tool for Slack's incident management program.

The exercise also exposed behavioral patterns relevant to real incidents. Teams that moved quickly placed their order within 15–20 minutes. Teams that tried to poll the whole room about where to eat often risked running out of time. That tendency to seek consensus became a teaching point about the difference between incident response and democracy: the goal is to make fast, efficient decisions, often with tradeoffs that wouldn't be acceptable without time pressure. When teams fell into extended polling, facilitators used it as a moment to introduce the "Are there any strong objections?" tactic for building fast consensus.

Choosing delivery over pickup tended to slow teams down by adding dependencies to the response, an analogy to the complexity introduced by external systems in real incidents. One session saw lunch arrive quite late after an online ordering system failed to transmit the order to the restaurant.

What Didn't Fully Work

The exercise works best in person, and Slack never found a satisfying virtual substitute during the pandemic. Remote participants could be included in a limited fashion, but the logistics of delivering lunch to them didn't make sense. Experimenting with one or two remote employees per session did add realism to the exercise, since it forced teams to coordinate across distances.

The sessions also didn't use Slack's real incident tooling. Setting up large groups of users in a staging environment carried too much overhead. The author suggested that a dry-run or demo mode for internal tooling would be a valuable future addition, as would keeping a log of retrospective insights for evolving the program over time.

Slack published an example public repository for others who want to run their own version of the exercise.