Benefits and limits of large language models

Learn about the benefits and drawbacks of on-device and client-side large language models.

W

web.devweb.dev

·

Benefits and limits of large language models

Why LLMs matter for web apps

Large language models (LLMs) are shifting from research curiosities to practical building blocks in software development. Because they excel at generating and processing natural language, they unlock features that would otherwise require rigid, hand-coded logic: extracting structured data, summarizing content, or powering conversational interfaces over user data.

This article—the first of three on LLMs in practice—lays out the technical trade-offs of running models locally versus in the cloud, using a to-do list application as the working example. Subsequent parts cover building a chatbot with the WebLLM runtime and with Chrome's experimental Prompt API.

What an LLM can add to a simple app

Application before chatbot is added. — Our to-do list app.

The starting point is a classic to-do list: users add items, mark them complete, and delete them. Rather than leaving the data inert, a chatbot layer lets users interrogate and manipulate that same information through natural language. Concrete capabilities include:

Asking how many tasks remain open.
Flagging duplicate or near-identical entries.
Grouping to-dos into logical categories.
Suggesting new tasks based on completed ones.
Translating list items into other languages.
Exporting the entire list as XML.

All of these are natural fits for an LLM, and each is demonstrated in the finished demos and source code that accompany the series.

Inside the model

LLMs are artificial neural networks trained to process and generate text. Most contemporary models, including Google's Gemini and Gemma, OpenAI's GPT series, and open models like Meta's LLaMa and Mistral, are built on the Transformer architecture that originated at Google. Training on enormous corpora gives them multilingual comprehension, broad factual recall, translation ability, and even code generation. The depth of these capabilities scales with model size, a topic covered separately in Understand LLM Sizes.

Architecturally, LLMs change the software contract. Instead of calling a function with a typed signature, the developer—or end user—expresses intent in a prompt. Natural language becomes the API.

Known failure modes

That flexibility comes with three well-documented caveats:

Non-determinism: the same prompt can yield different, even contradictory, answers because outputs are sampled from a probability distribution rather than computed from fixed rules.
Hallucination: models generate text that follows learned patterns but can be factually wrong or nonsensical.
Prompt injection: crafted inputs can steer the model away from its intended role or trigger undesired actions.

None of these are hypothetical. Results should be treated as unverified suggestions until a human—or another deterministic system—checks them. For user-facing features, that means designing guardrails into the surrounding code.

Local versus cloud: the real trade-offs

The default instinct is to call a hosted API from a vendor like OpenAI or Google. Cloud models are high quality, and many are exclusive to their provider. Inference is fast and pricing is typically per token, which can be cost-efficient for spiky or low-volume traffic.

But running the model on the user's own device flips the calculus. Response times become consistent, because there's no network hop or shared backend to contend with. The app keeps working offline. There are no per-token fees to pay. And because data never leaves the device, the privacy argument is straightforward: no personally identifiable information (PII) is transmitted to an external provider or across borders.

The catch is practical. On-device models reach file sizes of several gigabytes and must be fully downloaded before the first use. Smaller quantized versions are faster to load but produce lower-quality responses than their larger cloud-hosted counterparts. The right answer depends on whether predictability, privacy, and cost—or raw model capability—matter more for the feature at hand.