A closer look at how Figma quantified Make’s time savings

Figma’s Data Science team set out to answer a question many teams are asking about AI tools: how much time are they actually saving? For Figma Make, the answer, based on a new randomized controlled trial, is that design work got 20% faster and 16% easier overall. Product managers saw the largest gains, with tasks completed 23% faster and rated 37% easier.

The team’s full study report details the methodology behind those numbers, which required moving beyond conventional measurement approaches.

Why standard methods fell short

Measuring time savings from AI usage is complicated by confounders—variables like job tenure, experience, and task complexity that affect speed regardless of AI. Common data science methods struggle to isolate AI’s contribution in this context.

A/B testing can randomize participants into treatment and control groups, but it doesn’t account for the fact that users are working on different, inherently unequal tasks. Causal inference methods on historical log data also have limitations. Propensity score matching (PSM) requires that all confounders be captured in the log data, which isn’t the case when anonymized user IDs reveal nothing about a person’s design experience. Instrumental variables (IV) analysis requires finding an "instrument" in the data that influences AI usage without affecting task speed through any other path; for Figma’s logs, no such valid instrument exists.

Designing a controlled study

Given these constraints, the team opted for a randomized controlled trial (RCT), the gold standard for causal impact measurement. The design controlled for confounders through three mechanisms: random assignment of participants to treatment or control groups, standardized tasks across both groups, and trained moderators following a script to reduce variability.

The study focused exclusively on Figma Make, chosen for its popularity and broad applicability. To determine sample size, the team used effect sizes from peer studies like the GitHub Copilot RCTs, running a power analysis that led to 100 participants: 50 product designers and 50 product managers. The inclusion of PMs was deliberate, as user research had suggested Make was opening design workflows to non-designers.

Participants in both groups worked through the same design tasks—editing social media posts—selected for wide familiarity. The tasks spanned three workflows: UI and appearance changes, creating new views in a base design, and adding interactability and responsiveness. Internal pilots with Figma employees led to three rounds of task redesign to ensure the scenarios were neither too trivial for designers nor too difficult for PMs.

Moderators, who had varying levels of Figma fluency, followed a standardized script and consulted a troubleshooting guide, which was updated as the study progressed.

Key findings across roles and tasks

All 100 participants completed the study. The results, analyzed with hypothesis testing and OLS regression modeling, showed statistically significant improvements unless noted otherwise:

  • Overall, participants saw a 20% reduction in cumulative time to completion, a 16% improvement in task ease, and a 15% improvement in perceived Figma usability.
  • Product managers showed consistently larger gains. With Make access, PMs were 23% faster cumulatively across tasks, making them nearly as efficient as product designers working without Make.
  • Task type matters. PMs saw significant time savings on the two simpler tasks but no significant gains on the most complex scenario. Product designers showed the reverse pattern: significant time improvements only on the most challenging task. Make appears to accelerate zero-to-one and complex interaction design for designers, while serving PMs as a baseline tool for quick design contributions.

Because of the controlled design, the team can assert that Make causes these improvements rather than merely correlating with them.

Figma plans to extend this research approach. A similar RCT is planned for the new Figma design agent, with attention to how parallel agent capabilities might amplify time savings. Future work will also explore AI’s impact on design quality and multiplayer collaboration.