Building a Magic Mirror: AI retail experiences with Remix (2026) - Shopify

When the Storefront Looks Back

Physical retail is no longer competing on convenience alone. With online delivery nearly instant, the in-person visit has become a different kind of signal—one that brands are answering with interactive, personalized installations. At Shopify, we've been prototyping one such format: a "magic mirror" that uses a hidden camera and AI to respond to whoever steps in front of it.

The mirror looks like an ordinary full-length mirror, but behind the glass sits a display and a webcam. Computer vision and a language model let it interpret visual signals—clothing, movement, facial features—and respond with tailored messages, animations, and even integrations with other systems like discount code generation or receipt printing. Each interaction takes under a minute, making it viable for high-traffic stores, not just showroom demos.

Our first full implementation was a makeup shade-matching experience for Rare Beauty. A guest presses a button, gets their recommended blush shade, and can opt in to sync that data to their customer profile. But the architecture supports many "modes," including:

• Delivering compliments or roasts
• Recommending complementary products
• Analyzing style or recognizing gestures
• Scoring actions, like a "Simon Says" challenge
• Unlocking exclusive products or generating unique discount codes

These can combine—for instance, a mirror where customers "pay with a pose," earning a discount (generated via the Admin API) based on how well they execute a yoga position in front of the camera.

Core Components and Flow

The build requires a full-sized mirror, a webcam mounted behind it, a display, a mount for rotating the camera, a computer running a Remix server and browser, and a physical button or keyboard for triggering the experience.

The interaction sequence for the Rare Beauty mirror is:

1. A customer walks up and presses the button.
2. The screen counts down, then takes a photo.
3. A vision model processes the image with a curated prompt.
4. The mirror returns a personalized compliment and a blush shade recommendation.

The result stays on screen for 10 seconds (enough for a selfie), then a 5-second countdown cycles back to the idle state. The entire flow runs on standard HTTP—a POST from the browser to the server, and a rendered response.

The full implementation of a barebones version—a single Remix route with no persistence or authentication—is available as a reference gist.

Co-Located Server and Client Logic in Remix

We chose Remix, a full-stack React framework, because each route file can export a loader (server, GET), an action (server, POST), and a default browser component. This keeps the server logic and client UI in one place—no separate API layer needed for a single-purpose device.

The production mirror keeps the route file thin: it defines the config, sets up auth, and re-exports shared action and loader logic. Auth on a parent route automatically propagates to nested routes.

The server action has one job beyond calling the LLM—it broadcasts status over Server-Sent Events (SSE). The broadcast fires twice: once when processing starts, and once when it completes. This lets a separate "remote" device (for a brand associate) know what's happening without polling or watching the screen.

Prompt Assembly and the LLM Call

The processImageWithLLM function calls buildPrompt to assemble three parts:

1. A base prompt string defining persona, tone, and task.
2. A product recommendation table as plain text: Product ID | Product Name | When to Recommend | Inventory.
3. A JSON output schema instruction specifying the return shape.

Inventory appears in the table so the model can weight recommendations toward stock that's over-supplied. Sending products as a plain-text table rather than structured JSON is deliberate: LLMs parse natural-language tables well, and it keeps prompts human-readable for brand teams reviewing the recommendations.

The call uses the OpenAI SDK, but the client points at an internal proxy via baseURL, so the underlying model provider can change without touching this code.

Three implementation choices matter here:

• response_format: { type: "json_object" } guarantees a well-formed JSON envelope, making JSON.parse() reliable.
• The image is passed as a data URL in the same message as the text prompt—no upload or CDN step.
• max_tokens: 300 is generous for an output constrained to roughly two sentences, preventing runaway completions.

The model returns a recommended_product_id integer, resolved to a full product object via strict equality lookup. If find returns undefined, the response screen renders the compliment text without a product card—graceful degradation.

SSE for a Remote Control

The mirror has one display, but floor staff need a way to trigger a capture or reset the experience without opening the device. An SSE connection from a remote device to /api/mirror/events handles this without polling.

The server holds connections in an in-memory SSEClientManager, and broadcasts are a simple loop. Each message carries a channel field, allowing multiple mirror instances on one server to ignore traffic meant for other booths. It's optional, but in a retail environment, not having to open the display for resets is a real advantage.

Client State Machine

The React component tracks five states: IDLE, LOADING, RESPONSE, EXIT, and ERROR. During a normal session, pressing the button moves IDLE → LOADING. A useEffect watching useActionData triggers LOADING → RESPONSE, and a countdown timer inside the ResponseScreen component drives RESPONSE → EXIT.

Two timing details materially affect quality. First, the Rare Beauty version delays the screenshot by two seconds after the button press. The loading screen tells the customer to "Step back and smile," and photos taken at arm's length with the subject centered produce significantly better skin tone analysis than close-up, off-angle shots. Second, the response screen shows its result for 10 seconds, then displays a circular SVG countdown for five seconds before advancing to EXIT. Pressing the button at that point reloads the page to idle.

The circular countdown is a pure SVG component using stroke-dashoffset: as a countdownValue decrements each second, the arc erases clockwise, smoothed by a CSS transition: stroke-dashoffset 1s linear.

UI transitions between states fade out and back in. A ScreenTransition component detects the screen change, sets opacity: 0 with scale(0.98) and blur, waits half the transition duration, swaps the content at the fully invisible midpoint, then fades back in.

Prompt Engineering Lessons

The Rare Beauty base prompt runs about 100 words: role definition, a curated list of approved compliments, hard tone constraints (≤ 100 characters including spaces), and explicit exclusions—teeth, wrinkles, race, deformities, gender. The appended product list tells the model which visual features should trigger each shade recommendation.

Tuning revealed several reliable principles:

• Concrete limits beat adjectives. "No more than 100 characters" outperforms "be concise."
• Curated lists beat free generation. Giving the model 50+ pre-approved compliments yields more brand-appropriate output than letting it invent responses. It selects, rather than creates.
• Exclusion lists are non-negotiable. Explicitly listing what not to mention (race, religion, deformities) is more reliable than model defaults, especially in a public brand activation.
• Inventory deltas work as nudges. The prompt can tell the model to favor low-stock items, but only "if it respects the non-negotiable guidelines"—a soft push, not an override.

Configuration-Driven Deployments

Each mirror deployment is a thin route file exporting a MirrorConfig. That object holds the brand-specific prompt, product list, and auth secret. Reusing the mirror for another brand or event means swapping those values; the component, action, and SSE infrastructure stay unchanged.

A Minimal Starting Point

To make this approach accessible, Shopify has published a companion demo that strips the system down to its most basic viable form. The result is a single file with no authentication, no server-sent events, no screen transitions, and no custom branding. It's intended as a clean scaffold for anyone who wants to build a similar experience and needs a foundation to add production features on top of.

Beyond the Basics

Once the core mechanics are in place, the platform becomes a flexible canvas for in-store engagement. Because the underlying architecture is essentially a browser application running behind reflective glass, new "mirror modes" can be generated in just a few hours. The same system can be directed toward a helpful concierge flow, a transactional catalog browse, a playful game, or any combination of those functions.