Why review quality matters

Product reviews have a measurable effect on sales. Retailers that show reviews can see a 270% increase in conversions, and 82% of online shoppers actively look for negative reviews before making a purchase decision. Negative feedback builds credibility, but getting customers to write useful, specific criticism is harder than it sounds.

Generative AI running entirely in the browser can help. This article walks through a product review suggestion feature that uses the Gemma 2B model via the MediaPipe LLM Inference API to guide shoppers toward more informative reviews.

Why run AI on the client

The demo implementation chooses client-side inference for two practical reasons:

  • Latency: Suggestions appear as soon as the user pauses typing, with no server round-trip delay.
  • Cost: Experimenting with a zero-server-cost implementation lets you validate the feature before committing to production infrastructure.

The choice of Gemma 2B through the MediaPipe GenAI package comes down to two factors. First, the model provides a solid balance of size and accuracy; with proper prompting, it produces satisfying results for this use case. Second, MediaPipe runs in all browsers that support WebGPU, giving the feature solid cross-browser coverage.

Designing the user experience

Even a small LLM like Gemma 2B is a sizeable download. The demo applies standard performance best practices, including running inference in a web worker to keep the main thread responsive.

The feature is also strictly optional. Users can post a review even if the model has not finished loading. The AI suggestions enhance the workflow; they never block it.

Inference takes longer than the 100-millisecond threshold users perceive as instantaneous. The UI communicates that the model is "thinking" with clear states and animations, so users understand the application is working as intended even during the wait.

Production considerations

A production deployment should account for more than the happy path:

  • Feedback mechanism: Add thumbs-up/down controls or heuristics (such as whether users disable the feature) to gauge whether suggestions are actually helpful.
  • Opt-out: Some users will prefer to write in their own voice. Make the feature easy to turn off and back on.
  • Explain the purpose: A one-line rationale, such as "Better feedback helps fellow shoppers decide what to buy, and helps us create the products you want," encourages participation and feedback usage.
  • AI disclosure: Client-side inference keeps user content private since nothing is sent to a server. If you add a server-side fallback, update your privacy policy and terms of service accordingly.

Implementation highlights

Worker-based MediaPipe setup

The core MediaPipe LLM inference code is only a few lines: create a file resolver, instantiate the LLM inference object with a model URL, then call it to generate a response. The demo is more involved because the code runs in a web worker, communicating with the main script through custom message codes.

// Trigger model preparation *before* the first message arrives
self.postMessage({ code: MESSAGE_CODE.PREPARING_MODEL });
try {
  // Create a FilesetResolver instance for GenAI tasks
  const genai = await FilesetResolver.forGenAiTasks(MEDIAPIPE_WASM);
  // Create an LLM Inference instance from the specified model path
  llmInference = await LlmInference.createFromModelPath(genai, MODEL_URL);
  self.postMessage({ code: MESSAGE_CODE.MODEL_READY });
} catch (error) {
  self.postMessage({ code: MESSAGE_CODE.MODEL_ERROR });
}

// Trigger inference upon receiving a message from the main script
self.onmessage = async function (message) {
  // Run inference = Generate an LLM response
  let response = null;
  try {
    response = await llmInference.generateResponse(
      // Create a prompt based on message.data, which is the actual review
      // draft the user has written. generatePrompt is a local utility function.
      generatePrompt(message.data),
    );
  } catch (error) {
    self.postMessage({ code: MESSAGE_CODE.INFERENCE_ERROR });
    return;
  }
  // Parse and process the output using a local utility function
  const reviewHelperOutput = generateReviewHelperOutput(response);
  // Post a message to the main thread
  self.postMessage({
    code: MESSAGE_CODE.RESPONSE_READY,
    payload: reviewHelperOutput,
  });
};

export const MESSAGE_CODE ={
  PREPARING_MODEL: 'preparing-model',
  MODEL_READY: 'model-ready',
  GENERATING_RESPONSE: 'generating-response',
  RESPONSE_READY: 'response-ready',
  MODEL_ERROR: 'model-error',
  INFERENCE_ERROR: 'inference-error',
};

Prompting and output parsing

The full prompt relies on few-shot prompting and incorporates the user's draft review, the product type, and example reviews. At runtime, a generatePrompt utility function builds the prompt from the user's current input.

Client-side models lack some conveniences available on the server. For instance, JSON mode is typically unavailable, so the output format must be embedded in the prompt itself rather than supplied as a schema. Smaller models are also more prone to structural mishaps.

In practice, Gemma 2B produces cleaner structured text than JSON or JavaScript output. The demo therefore asks for a text-based response, then parses it into a JavaScript object for the application to process.

Improving prompt quality with LLMs

The demo uses a surprising trick: LLMs to improve the prompt itself.

  • Few-shot examples: Gemini Chat generates the high-quality example reviews used in the prompt.
  • Prompt refinement: Once the structure was set, Gemini Chat polished the wording, which improved overall output quality.

Context improves output

Including the product type in the prompt enables significantly more relevant suggestions. The demo uses a static product type (socks), but a real application can populate this dynamically based on the page the user is browsing.

Common Gemma 2B pitfalls and workarounds

Gemma 2B needs more careful prompting than a larger server-side model. The demo hits three recurring problems:

  • Excessive politeness: The model hesitates to mark a review as unhelpful. More neutral labels ("specific" vs. "unspecific") and added examples did not help. Repetition and insistence in the prompt did. A chain-of-thought approach would likely improve results further.
  • Instruction ambiguity: The model would sometimes continue the example list instead of evaluating the review. Adding a clear transition line in the prompt separates the few-shot examples from the actual input and fixes the behavior.
  • Wrong referent: The model occasionally suggests product changes rather than text improvements. Splitting the prompt into distinct sections clarifies the task and keeps the model focused on the review itself.

The resulting feature helps shoppers write reviews that actually aid purchase decisions, without sacrificing privacy or adding server load.