Personalizing Reels Beyond Likes and Watch Time
Facebook's Reels recommendation system has typically relied on engagement signals—likes, shares, and watch time—to infer what users want to see. But these signals are noisy and often miss the deeper reasons why a particular video resonates. A video might get high engagement for reasons unrelated to genuine interest, or a user might watch something they don't actually enjoy. To better capture what people truly care about, Meta researchers have integrated direct user feedback into the ranking pipeline, using survey responses as a training signal for a new model layer called the User True Interest Survey (UTIS) model.
This approach has shifted the focus toward "interest matching," which considers not just topical alignment but also factors like audio, production style, and the motivation behind a user's viewing session. The goal: recommend videos that users actually want to see, which in turn drives retention and long-term engagement.
Measuring Perceived Relevance Directly
To move beyond heuristic guesses, Meta deployed large-scale, randomized surveys within video feeds on Facebook Reels and adjacent surfaces. Users were asked a single question after viewing content: "How well does this video match your interests?" with a 1–5 scale. Thousands of in-context responses were collected daily.
The results validated the need for a new approach. Previous interest heuristics achieved only 48.3% precision in identifying true user interests. To correct for sampling and nonresponse bias, the team weighted the survey responses, building a dataset that more accurately reflects real user preferences than implicit engagement signals alone.
The UTIS Model Layer
The UTIS approach is implemented as a lightweight alignment model trained on the survey data. Rather than replacing the main ranking model—a large multi-task, multi-label system—UTIS takes the main model's existing predictions as input features. Survey responses were binarized to simplify modeling and reduce variance in user ratings.
New features were engineered to capture user behavior, content attributes, and interest signals, with the objective of optimizing prediction of how well a video matches a user's stated interests. The model outputs the probability that a user will be satisfied with a given video, and its design prioritizes interpretability so that the factors contributing to a good interest match can be understood.
Deploying UTIS in the Ranking Funnel
Meta has integrated the UTIS model into multiple stages of the ranking process, each producing measurable improvements in tier-0 user retention metrics:
- Late Stage Ranking: UTIS runs alongside the late-stage ranking (LSR) model, contributing an additional feature to the final value calculation. This lets the final ranking stage balance true interest signals against other concerns like engagement and integrity.
- Early Stage Ranking (Retrieval): UTIS predictions are aggregated to reconstruct user interest profiles, estimating affinity for any user-video pair. This re-ranks retrieval candidates, surfacing more content aligned with users' true interests. Additionally, large user-to-item retrieval sequences are aligned using knowledge-distillation objectives, with UTIS predictions from the LSR stage serving as labels.
In production, the UTIS score is now one of the inputs to the ranking system. Videos with high predicted interest receive a modest boost; low-scoring content is demoted. This has resulted in more niche, high-quality content being delivered, fewer generic popularity-based recommendations, and improvements in likes, shares, follows, and overall engagement.
Measured Gains in Accuracy and Engagement
Both offline and online tests confirmed the model's effectiveness. Offline, UTIS improved accuracy from 59.5% to 71.5% over the heuristic baseline, with precision rising from 48.3% to 63.2% and recall from 45.4% to 66.1%.
Online A/B testing with more than 10 million users showed consistent wins in real-world conditions: a +5.4% increase in high survey ratings, a -6.84% reduction in low ratings, a +5.2% boost in total user engagement, and a -0.34% decrease in integrity violations.
Remaining Challenges and Next Steps
While the survey-driven approach has improved recommendation quality, Meta researchers note several open problems: better serving users with sparse engagement histories, reducing bias in how surveys are sampled and delivered, improving recommendation diversity, and personalizing more effectively for varied user cohorts. To tackle these, they are exploring large language models and more granular user representations for ranking tasks.
Details of the methodology and results appear in the paper "Improve the Personalization of Large-Scale Ranking Systems by Integrating User Survey Feedback," presented at the ACM Recommender Systems conference.
