An Early Look at WhatsApp’s Scam Alert

WhatsApp is adding an optional, on-device layer of scam protection that preserves the core privacy guarantee of end-to-end encryption. Scam Alert runs a small machine learning model locally that classifies incoming messages from non-contacts and flags messages that match patterns seen in previously reported scam campaigns. No content or model output leaves the device unless the user explicitly chooses to report it.

Scope of the System

The feature is designed to fit within a strict set of constraints that keep inference and control wholly with the user. The most important design decisions are functional limits, not just compliance checks: the model lives on-device, operates on messages in place, and has no path to automatically transmit data to WhatsApp, Meta, or third parties.

  • No server-side inference: The model and the message data it processes stay on the device.
  • No automatic reporting: A detected scam, and the message content triggering it, is known only on-device until the user acts. The only unsolicited data path to WhatsApp occurs when the user files an explicit report, matching existing user reporting flows.
  • Controllable at any time: The feature is an optional addition to the chat surface; users can toggle it and clear flags per chat at will.

Classification and User Feedback Loop

Once enabled, the feature runs inference on messages from unknown senders. The model is trained on conversation patterns users have already sent to WhatsApp as reports, and it scores incoming text based on conversational structure and linguistic signals with probabilistic output. If that score crosses the warning threshold, a notice appears in the chat — visible only to the recipient.

The user keeps three options at that point: block the sender, report the conversation, or continue chatting. Users who think the detector is wrong can mark the conversation as trusted, immediately suppressing further warnings for that chat. As an optional improvement step, the user can also contribute a snippet of context — the last five messages already received — to WhatsApp. This secondary consent flow feeds the training set; it is never mandatory, and it is separate from the detection process.

Architectural Safeguards and Verifiability

Scam Alert couples the on-device rule engine with built-in verifiability mechanisms. The system holds three structural safeguards (detailed below) that are meant to be reproducible, not merely claimed. The engineering rests on available open review channels plus a minimal analytics path designed to avoid raw message data.

  1. On-device inference with privacy-preserving telemetry: Content stays local. The only data placed on the wire is an anonymous count of warnings shown and actions taken, sent as an aggregate — not as a stream of raw events. That aggregate passes through a confidential computing pipeline over confidential virtual machines (CVMs). The TEE boundary makes the analytics operation externally testable rather than trusting the client.
  2. No targeted model delivery: It is not possible for Meta to deploy a particular model version at a granular or individual level. Each model revision and experimental build is posted to a public ledger prior to distribution, so the release process is auditable end-to-end.
  3. Verifiable model behavior: The published weights allow reviewers to check that detection is confined to scam‑specific features. This is an explicit guard against function creep, giving independent research teams a basis to audit the actual classifier, not just documentation about it.

The remainder of the preview expands on the technical implementation and the threat model for each safeguard.

Privacy-first measurement for Scam Alert

Scam Alert runs inference entirely on-device, which means message content never leaves the phone. But Meta still needs to know whether the feature actually catches scams and whether the model needs updating as scams evolve. The measurement system that supports this is built around a core set of data minimization principles: message content never leaves the device, and logging is restricted to only the signals needed to determine whether the feature is working.

Those signals are processed inside a confidential computing environment built on trusted execution environments (TEEs). No one — including Meta and WhatsApp — can access that processing environment. The design draws on Meta's experience building Private Processing systems. What Meta and WhatsApp ultimately receive are anonymous, differentially private aggregates: noise is added so that adding or removing any single person's data has a mathematically negligible effect on the aggregate numbers. The aggregates reveal how the feature performs across the population, but nothing about individuals.

For Scam Alert, that data is limited to two approximate, aggregate counts:

  • Warning counts – how often the on-device model surfaced a scam warning. This measures whether the model triggers at the right rate, which is needed to gauge precision and catch regressions between model versions.
  • User action counts – when a user sees a warning, they can either trust the sender or block and report. The action category is logged as an aggregate count to assess whether users find the warnings accurate, which informs false positive rate.

Confidential federated analytics

To anonymize those counts, Meta built a confidential federated analytics pipeline with the following guarantees. Each is architecturally enforced and externally verifiable.

  • On-device data minimization: For Scam Alert, raw signals don't leave the device. The client aggregates them locally into counts and sends only those totals. Metrics are transmitted at randomized times, contain no device identifiers, and timestamps are limited to coarse intervals. Neither the transmission itself nor the content can identify a user.
  • Confidential processing: Metrics are processed inside TEEs — secure hardware environments built on CPU-based confidential virtualization with attestation rooted in hardware. Before any data is transmitted, the client checks attestations against a third-party log of acceptable binaries. Data is encrypted between the client and the TEE, so no intermediary — including Meta, WhatsApp, or any relay — can access it.
  • Secure aggregation: Individual device metrics are unreadable by Meta, WhatsApp, or anyone outside the TEE. They're merged into running aggregates, and only statistics above a minimum cohort size, with differential privacy noise applied, are released. Scam Alert devices send pre-aggregated counts directly to the TEE for secure aggregation.
  • Enforceable guarantees: Before transmission, the client verifies that the code in the TEE matches what was published on the third-party ledger, and that privacy parameters (like differential privacy ε and δ, and k-anonymity thresholds) satisfy locally enforced guardrails. If verification fails or parameters fall short, the client refuses to send. Attempts to modify the processing guarantees cause the system to fail closed or become publicly discoverable.
  • Encrypted recovery checkpoints: Because aggregation runs over long periods, the system periodically saves encrypted checkpoints of in-progress aggregates so a crash doesn't force a restart. Checkpoints hold only partial aggregate counts, are encrypted with keys that never leave the TEEs, and can be decrypted only by confidential federated analytics TEEs running the same attested binary. They're retained only for the bounded period needed for recovery.
  • Non-targetability: An attacker can't target a specific user without attempting to compromise the whole system. All metrics route through an OHTTP relay that strips the requester's IP address, and clients authenticate with anonymous credentials. The system can verify that metrics come from a legitimate WhatsApp client without knowing which one, so small-scale attacks can't be used to isolate a specific user's data.
  • Verifiable transparency: In-app capabilities will let users review what data was shared with the pipeline, which privacy parameters were applied, and how each secure session was established. Meta will publish the CVM image binary and the source code of privacy-relevant components so researchers can independently verify that published code is exactly what runs in the TEE. The Bug Bounty program will expand to cover the pipeline, and a detailed engineering white paper will follow.

The pipeline's foundations come from Meta's peer-reviewed federated analytics work, described in "PAPAYA Federated Analytics Stack: Engineering Privacy, Scalability and Practicality" (USENIX NSDI 2025). Scam Alert applies and extends that base.

How the pipeline runs

  1. On-device data collection: Data is collected and stored in a dedicated local store, isolated from other application data. The confidential federated analytics system only accesses data the application explicitly exposes to it. Hardcoded guardrails enforce data lifetime, scope, and access. For Scam Alert, this is only counts of warning events and user actions.
  2. Job selection: At randomized intervals — when the device is idle and within a self-enforced daily resource limit — the client connects to the application server via OHTTP. The client authenticates with anonymous credentials that prove it's a legitimate WhatsApp client without revealing which one, and fetches the list of active jobs. For each job, the client checks privacy parameters against local guardrails, whether the device has new metrics, and whether participating would exceed daily limits. Any job that fails these checks is rejected.
  3. Local transformation: For accepted jobs, the client retrieves relevant data and converts raw signals into anonymous counts over a specified time period (e.g., daily warning counts, daily user action counts). Only aggregated counts are transmitted. Raw signals stay on-device and are automatically deleted after a retention period.
  4. Attestation, authentication, and session establishment: The client establishes a Remote Attestation + Transport Layer Security (RA-TLS) session with the orchestrator TEE. The attestation quote contains orchestrator measurements that the client cross-checks against a third-party transparency ledger to confirm it's connecting only to published code. Authentication uses anonymous credentials, and the connection routes through an OHTTP relay that strips the IP address — the relay can't read the metrics because they're encrypted end-to-end between device and TEE.
  5. Orchestrator TEE — ephemeral processing: Encrypted metrics arrive at a stateless orchestrator hosted on a TEE. It validates that the client's privacy configuration matches the job configuration, batches metrics from multiple devices, and forwards them to the appropriate aggregator TEE.
  6. Aggregator TEE — secure aggregation: The aggregator merges incoming metrics into running histograms; individual metrics are discarded after aggregation. Periodically the aggregator enforces k-anonymity thresholds to suppress results with too few contributors and applies differential privacy noise. Release count and timing are limited so the overall privacy budget (ε, δ) isn't exceeded. Only noisy, thresholded aggregates reach WhatsApp.
  7. Datastore: Only differentially private, anonymous aggregates — total warning counts and action rates across all users — leave the TEE boundary. These contain no message content, no per-user data, and no conversation-level signals. They exist solely to measure whether the feature works.

By the time any totals reach WhatsApp, counts have been aggregated across many users plus noise — so Meta only sees approximate counts of warnings shown and actions taken. It never sees message content, who sent or received what, or which conversation triggered a warning.

Threat model and layered defenses

The pipeline operates in a highly adversarial environment. The threat model covers three attacker categories: third-party or supply chain vendors with component access, malicious or compromised insiders with infrastructure access, and external actors probing the pipeline's attack surface.

External actors attempting to intercept or extract unaggregated data in transit or during processing. Data in transit is encrypted between device and TEE and routed through a third-party OHTTP relay. The relay can only strip the client's IP address; it can't decrypt, inspect, or modify data. Even a compromised relay can't access data or associate datasets with a specific user. During processing, TEE code isolation protects data, with entry points limited to a small set of reviewed components.

Insiders with infrastructure access attempting to access unaggregated data within the TEE. The TEE prohibits remote shell access, including from the host machine. Neither Meta engineers nor networked systems can reach the CVM shell at runtime. Software is built exclusively from checked-in source and artifacts — any change requires multiple engineers to modify build artifacts or the build pipeline. All code changes are auditable for both internal audits and external researchers. Unaggregated data is never readable outside the TEE; when stored, it's encrypted under keys released only to a TEE running the same attested binary, and retained for a bounded period before merging into aggregates and deletion.

Attackers with physical or remote access attempting to interfere with the TEE to bypass confidential processing. TEE guarantees aren't absolute, so defense-in-depth applies: encrypted DRAM, CVM hardening, enhanced host monitoring, and OHTTP relay routing prevent steering a specific user's data to a specific machine. A targeted attack would require compromising the entire system in a way that's publicly discoverable through verifiable transparency.

No Targeted Model Delivery

Scam Alert downloads its detection model from a CDN rather than bundling it in the app, allowing updates to reach users without forced app upgrades. To prevent abuse of that flexibility, the system is designed so no one can deliver a distinct model to a specific user.

Before a model version is served to anyone, its SHA-256 hash is published to a third-party append-only transparency ledger. This public log is tamper-evident: entries can be added but never modified or deleted, giving researchers a verifiable history of every shipped model.

The download system is built around three guarantees:

  1. Auditability: Every model version is publicly recorded on the transparency ledger before it is served. A targeted delivery attempt would require publishing to a ledger anyone can inspect, making it publicly discoverable. Users can verify entries by downloading the in-app transparency log and checking the namespace and epoch, using the format: https://akd-auditor.cloudflare.com/namespaces/<namespace>/audits/<epoch>.
  2. Anonymous Download Requests: The download endpoint cannot determine which user is requesting a model. Requests are authenticated with anonymous credentials and routed through an OHTTP relay that strips the requester’s IP address. The payload contains no identity selectors, and the CDN serves only publicly published files with no role in choosing which model a device uses.
  3. Non-targetability: No one can steer a specific model to a specific user. The client randomizes download timing, and experiment group assignment happens entirely on-device using locally generated randomness.

Model Download and Verification

  1. Model Publication: When a model is ready, the server computes SHA-256 hashes of weights, tokenizers, and other assets, then builds a manifest JSON containing those hashes, the model version, and a timestamp. The manifest digest is sent to a third-party signer (Cloudflare), which signs it with Ed25519 keys. Meta never holds the signing key. The signed digest goes to the transparency ledger, and the assets are uploaded to the CDN.
  2. Anonymous Download Request: The client connects via OHTTP with anonymous credentials. The relay strips the client’s IP — the relay sees the IP but cannot decrypt the request; the server sees the request but only the relay’s IP. The server responds with the manifest, the digital signature, and CDN URLs.
  3. Client-Side Verification: The client verifies the manifest digest against hardcoded Cloudflare Ed25519 public keys, cross-references the digest with the transparency ledger, enforces freshness checks to block replay attacks, and checks that each asset’s SHA-256 hash matches the manifest. Any failed step prevents model loading.
  4. Model Loaded on Device: The model is installed and used only after all verification steps pass.

Private Experimentation

Before global rollout, model variants are tested with subsets of users. The download flow prevents this experimentation from creating a targeting path:

  • Client-Side Group Assignment: The download response lists available model configurations, fetched at randomized intervals via the same anonymous OHTTP and ACS flow. Using a local random seed, the client assigns itself to a group and selects its model. The server cannot influence which variant a specific user receives.
  • Experiment Configuration Tamper Checks: The client enforces integrity on experiment configurations. Group properties cannot be modified after publication, experiment sizes can only expand (never shrink), and groups must meet a minimum size threshold — preventing an attacker from narrowing groups to isolate individuals.
  • Experiment Models on the Ledger: Every experimental model variant is published to the transparency ledger with the same signing and verification as production models. No model reaches a device without being publicly recorded.
  • Anonymized Experiment Metrics: Performance is measured through the confidential federated analytics pipeline. If an experiment group is too small to meet k-anonymity thresholds, the TEE suppresses the data entirely.

Because verification and experimentation run entirely on the client, security researchers can audit the app binary to confirm the checks are performed.

Verifiable Model Behavior

The system ensures no targeted model delivery and preserves privacy in analytics, but a deeper question remains: how can anyone verify that the model is built only to identify potential scam messages? The model verification system provides two guarantees:

  1. User Visibility: Users have on-device access to what the model did — which messages were scanned, the outcome, and which model version was used.
  2. Independent Verifiability: External researchers can obtain the exact model that runs on user devices, verify its integrity against the transparency ledger, and analyze its behavior.

How Model Transparency Works

  • Published Model Artifacts: Every model version’s hashes are recorded in the signed manifest on the same transparency ledger used for delivery verification. Anyone can inspect the ledger to confirm every shipped version, including experimental variants, is publicly recorded.
  • Client-Side Transparency Logs: Users can enable transparency logs showing which messages were flagged by the on-device machine learning model. These appear under Account > Request Info > Scam Alert Activity and include the outcome of each analysis — whether the model flagged the message and whether a warning was shown — plus the model version used.
  • Bug Bounty Program: Ahead of Beta rollout, external security researchers were engaged through the Bug Bounty program to stress-test the system:
    • Privacy Architecture Review: Researchers received an early access APK to confirm no message content leaves the device and there is no auto-reporting.
    • Model Integrity Review: AI/ML researchers received the model weights to confirm the model is purpose-built for scam detection and nothing else.

The Bug Bounty scope will expand to include the models, allowing researchers to test them against their own inputs, analyze behavior across scenarios, and report deviations from the declared purpose or capabilities that can be systematically evaded.

Next Steps

The confidential federated analytics pipeline ensures that measuring model performance protects user privacy. The transparency ledger and third-party signing ensure every shipped model is publicly recorded and tamper-evident. Published artifacts, client-side transparency logs, and the Bug Bounty program ensure model behavior can be independently verified.

Scam Alert is rolling out in a limited Beta capacity. Development will continue through the Beta phase before production, with improvements based on feedback from users, security researchers, and the broader security community.