Beyond The Chat Window: Rethinking AI Interfaces
Every computing paradigm shift has arrived through a new abstraction layer. The command line hid the binary switches of early machines; the graphical user interface (GUI) hid the command line behind visual metaphors. Now generative AI promises another leap, but many in the industry are fixating on a single interface form: conversation.
Text-based large language models (LLMs) power most of the recent innovation, and their natural language capabilities have led many to declare the chatbot the natural successor to the GUI. Bill Gates calls generative AI “the most important advance in technology since the graphical user interface,” arguing that “our main way of controlling a computer will no longer be pointing and clicking.” Robb Wilson of UX Magazine calls conversation “the infinitely scalable interface.” Jesse Lyu of Rabbit says natural language will be “so intuitive that you don’t even need to learn how to use it.”
But the enthusiasm ignores a critical reality: GUIs never fully replaced command lines, and conversation is not a panacea. For certain interactions, conversational interfaces can decrease usability, increase cost, and introduce security risk relative to graphical alternatives. The question facing designers is not which paradigm wins, but which jobs each interface serves best.
The Historical Precedent For Interface Shifts
Conversational interfaces are nothing new. Joseph Weizenbaum built ELIZA, the first chatbot, at MIT in 1966. Decades of successors — from Alexa to phone tree menus — followed, yet most consumers used them for little beyond basic tasks like setting timers. The recent resurgence began with OpenAI’s 2018 invention of the Generative Pre-trained Transformer (GPT), a type of LLM built on the transformer architecture introduced in 2017, which itself relies on the attention mechanism from 2014. These innovations enabled parallel processing and selective weighing of input parts, producing responses with human-like intelligence.
GUIs followed a similar trajectory. Ivan Sutherland’s Sketchpad, demonstrated at MIT in 1963, was the inaugural GUI but lacked support for concepts like overlapping windows. The breakthrough came later with bitmaps, invented by Alan Kay’s group at Xerox Parc. Bitmaps enabled complex pixel patterns that vector displays could not handle, giving rise to the WIMP (windows, icons, menus, and pointers) paradigm that made computers accessible to non-technical users.
The parallel is instructive: both paradigms emerged from 1960s prototypes, but bitmaps and GPTs provided the missing data-processing ingredient that made each viable for mainstream adoption. Computers are no longer just calculating machines or exploratory tools — they can now act as life-like entities.
Where Conversation Fails
The argument for conversational interfaces rests on the assumption that talking is more intuitive than pointing and clicking. It is true that the WIMP paradigm has documented usability problems: Nielsen Norman Group reports that cultural differences make universal icon recognition rare, and menus become unusable as complexity grows over time. The appeal of a chatbot is that you can simply tell the system when you are confused.
But replacing menus with input fields trades one set of usability problems for another. Natural language input is ambiguous, and users frequently lack the vocabulary to articulate what they want. Text-based interaction also suffers from high cognitive load, requiring users to formulate and type out requests rather than recognizing options from a visible list. There is ample research showing that for certain interactions, conversation is slower, more error-prone, and more frustrating than a well-designed graphical interface.
Lessons From Early Accessibility Tools
Early computing accessibility tools offer a useful cautionary tale. Before graphical interfaces matured, assistive technologies helped visually impaired users navigate command-line systems by converting text to speech. These screen readers worked by parsing the text output of the command line and speaking it aloud. As GUIs proliferated, screen reader developers faced the challenge of interpreting pixel-based displays, which lack the textual structure of a command line. Some proposed encoding accessibility information directly into the bitmap data itself, creating a kind of visual overlay that assistive tools could parse.
The idea was technically clever but practically flawed: it assumed a universal standard for embedding metadata in pixels, which never materialized. The fragmentation of GUI frameworks made this approach unworkable. This pattern echoes the current rush toward conversational interfaces. Designers see a technology that seems to solve an input problem — the difficulty of expressing intent — and assume it must be the universal answer. But like the pixel-encoding proposal, it ignores the reality that different users and different tasks require different interaction modalities.
Toward A Multi-Modal Future
The most productive frame for AI interface design is not conversational versus graphical, but a spectrum of interaction patterns. Command lines remain the tool of choice for technical users who need flexibility and efficiency, even as no-code graphical tools target a broader audience. Similarly, conversational AI will excel at certain tasks — open-ended exploration, hands-free operation, personalized recommendations — while GUIs remain superior for precise, repeatable, and parallel operations.
Designers should consider replacing chat-based input fields with more structured graphical interfaces in many AI applications. Task-specific forms, steppers, and wizards can reduce ambiguity and errors compared to free-form text. Visualization of model outputs, transparent options for steering parameters, and the ability to review and edit results directly remain essential AI-application design patterns.
Artificial intelligence will permeate the interface layer of digital products, but that does not mandate a singular interface modality. Just as the command line and GUI now coexist, conversational and graphical interfaces will serve distinct purposes. The goal is not to move beyond interfaces entirely, but to select the right abstraction for the task at hand.
- Task determinism: For well-defined operations (e.g., data entry, configuration), designed graphical elements are often clearer than natural language prompts.
- Discoverability: GUIs expose options; conversations require knowledge of what to ask for — the same recognition versus recall problem.
- Parallelism: Visual interfaces allow many simultaneous, glanceable elements; a conversation is serial, requiring memory of prior turns.
- Cost and latency: A pixel rendered on screen costs little; a generative AI response costs compute and time. Graphical presentation remains dramatically cheaper for data-dense output.
- Hallucination and error handling: Communicated interface state helps users verify system behavior against expectations, a crucial safeguard when relying on probabilistic models.
The future of AI interfaces will be neither purely visual nor purely conversational. It will be a layered system that uses text generation where we lack neat graphical fields, dialogue where user intent is genuinely underspecified, and graphical interactive components where — even with transformer-based models — their cost and error-prone edge remain problematic.
The Hidden Costs of a Simple Chat Box
Conversational interfaces dominate AI product design for a straightforward reason: they are easy to build. As AI research engineer Linus Lee notes, a chat window is essentially the “raw interface” of a language model. The underlying technology is optimized for text in, text out, so wrapping it in a conversational UI requires little additional invention. This is also why OpenAI shipped ChatGPT as a “low-key research preview,” according to The Atlantic. Expectations were so modest that employees predicted 100,000 first-week users, a number that missed the actual figure by a factor of ten.
The problem is that low upfront effort often translates to downstream UX debt. A chat box is a convenient starting point, but it forces all interaction into a single pattern, whether or not that pattern fits the task.
When a Blank Box Is the Worst Interface
Jakob Nielsen’s usability heuristics define discoverability as the ability to “encounter new content or functionality that you were not aware of.” Many generative AI interfaces fail this test. They present a user with an empty input field and expect them to know what to type. As Casey Newton observed on the Hard Fork podcast, AI has a “blank box” problem: users are left to guess what the system can do for them.
This is a case where industry veterans are unlearning principles they helped establish. Don Norman’s The Design of Everyday Things has long argued that good design provides explicit signifiers and affordances. Text boxes offer neither. Visible buttons, sliders, and menus all tell the user what actions are possible. A conversational interface hides that information inside the model’s documentation.
Even companies building these tools seem to recognize the friction. Midjourney, an image generator that originally accepted text-only prompts through Discord, is building a GUI to reduce the learning curve.
Conversation Is a Poor Tool for Orientation
The first of Nielsen’s ten usability heuristics is visibility of system status: users should always know what the system is doing and why. This is as important for chat-based systems as any other, because conversational turns can change the model's underlying state without any visible signal.
The lack of persistent context matters. Users tell an LLM to act like a historian, and the model dutifully switches its style. But that instruction disappears into the message history, and any shift in the conversation may silently reset it. The user must either remember the roleplay instruction or keep restating it. This is functional for experts, who can mentally track the session, but it is a significant cognitive load for novices who rely on recognition rather than recall — a concept Nielsen built into his heuristics.
LLMs process prompts by navigating their neural network to different regions of "knowledge." Role-playing is a technique to guide this navigation: telling the model to act as a historian biases it toward factual detail, while assigning a storyteller role produces narrative prose. These roles are effectively system-level settings that live on a state reset. If those settings were presented in a persistent form — a visible chip, toggle, or dropdown — they would not need to be juggled in memory. They would also be easier for a chat backend to re-apply reliably to every outgoing prompt.
A Menu Is Better Than Asking "What Do You Want?"
Character.ai is an educational example of persistent context in practice. Instead of asking the user to define context each session, it packages historical and cultural figures as anchoring points. The choice of converse with "Al Pacino" vs. "Socrates" itself dictates what the model is likely to discuss. The "character" acts as a heuristic for both user expectations and system configuration, similar to a restaurant menu: it presents options so visitors don't have to invent a meal from scratch.
Words Are Weak for Some Tasks
For tasks that are complex to describe but easy to point at, a graphical component often wins. Adobe's Generative Fill is a direct integration of AI with traditional skeuomorphic controls. Users need to explain which part of an image to fill, and clicking with the lasso is drastically more efficient than spelling out the coordinates in a prompt.
The same holds for adjustable quantities. Words like "larger" are loose and subjective, while a slider that maps directly to a numeric parameter gives unambiguous control. Controls that match a model's accepted field parameters eliminate the need for textual guess-and-check loops. Some specific values are far simpler to select from a palette than to articulate, such as colors and aspect ratios.
These heuristics may be decades old, but the underlying psychology has not changed since Nielsen formalized it. His original rules still apply; he himself says, “When something has remained true for 26 years, it will likely apply to future generations of user interfaces as well.” New entrants like Krea are already experimenting with new GUI mechanics, and mixing these with conversational inputs is part of an evolving shift. But as that happens, designers are wise to remember that the interface should make the underlying system legible — whether the medium is speech, text, or a drag handle.
Good Inputs Are Hard to Design For
Conversational interfaces inherit a fundamental usability problem from the command line: they expect users to know the right thing to type. The practice of crafting inputs that produce good output — “prompt engineering” — has become a specialized skill, one prized enough to command salaries around $200k. That speaks to its difficulty, and to how much expert knowledge it demands of end users.
Natural language as a programming interface is a remarkable technical achievement, but it is a strange requirement for consumer software. Users may share a vocabulary with a system and still have no idea what to say or how to phrase it. Every new technology has a learning curve; this one is steep enough to threaten adoption and retention.
Marketing has not helped. Describing AI features as “magic” frames the underlying mechanics as a dark art, which may intrigue early adopters but sets a poor expectation for everyone else. Steve Krug’s long-standing advice in Don’t Make Me Think still applies: most users will not study prompting techniques — they will muddle through. But trial and error is uniquely unproductive with generative AI because there are no error states. An LLM asked to do arithmetic will confidently return incorrect answers, making it impossible to learn from mistakes when you cannot tell a response is a hallucination. As OpenAI’s Andrej Karpathy has noted, hallucinations are not necessarily a bug; LLMs are “dream machines.” Whether that is a problem depends on how the interface manages expectations.
“But as with people, finding the most meaningful answer from AI involves asking the right questions. AI is neither psychic nor telepathic.”
— Stephen J. Bigelow in 5 Skills Needed to Become a Prompt Engineer
Language that hints at omniscience leads users to assume capabilities the system does not have. It will not be obvious to them that the model’s knowledge stops at its training data, nor that answers could be improved with Retrieval Augmented Generation or techniques such as Few-Shot and Chain of Thought prompting. Once the novelty fades, the burden of making these choices will land squarely on product designers:
- Empty states could make the boundaries of an AI’s knowledge explicit and let users supply missing context.
- Onboarding flows could infer user goals and steer them to models tuned for the right kind of reasoning.
- A fuzzy-search-like mechanism could annotate user inputs, educating them on tweaks that improve results.
There is already a hint of this pattern in OpenAI’s image generator DALL-E 3, which silently rewrites a user’s prompt to optimize the image output.
The Price of Every Prompt
Conversational interfaces carry a monetary cost that graphical interfaces do not. Each utterance requires the model to reason through a reply, and that computation is far more expensive than handling a button click. At current compute prices, the added intelligence is not always worth it. As the Wall Street Journal put it, using an LLM for email summarization is “like getting a Lamborghini to deliver a pizza.”
The problem is structural: AI systems cannot benefit from economies of scale the way traditional software does. Costs rise with usage because every interaction demands fresh computation, undermining the subscription model that depends on near-zero marginal costs for reproduction. Consumers may ultimately prefer paying less for a GUI that harnesses AI behind the scenes. History offers some reassurance — early GUIs were similarly held back by the cost of the processing power and memory required for bitmaps, until RAM prices fell. The same could happen here.
Security is a further cost. Natural-language interfaces invite manipulation: prompt injections turn input fields into attack vectors, giving malicious actors a route to private data and integrations. Positioning AI as a member of the team is risky given that human employees are already considered the weakest link in cyber security. Design mitigations help: sensible defaults can hide AI where it is not needed, using cost-effective elements instead of conversational ones. Sensitive data should be partitioned so it is reachable only by secure systems. And since LLMs are unreliable at arithmetic, they are best reserved for creative collaboration, not billing details.
Interfaces That Learn From Users
In a previous article, I introduced algorithm-friendly interfaces, which treat every interaction as a chance to improve both user understanding and system predictions. Such interfaces feed performance data back to the system and system feedback back to the user, maximizing data collection touchpoints to get better at forecasting. Accuracy gains in prediction generally improve retention, so good data compounds in value.
Generative AI is also predictive, and it could learn from individual user behavior. Just as Spotify learns musical taste, a system like Midjourney could set image generation parameters based on past usage, and ChatGPT could invoke the right roles at the right times — visibility permitting.
How algorithm-friendly conversational interfaces truly are remains unclear. The same low discoverability that hurts their usability also muddies the engagement signals they collect, weakening personalization. A “like” button sends a clean, unambiguous message to the backend. A user typing the word “like” may be expressing approval, using a simile, or just filling conversational space. Given that weak data undermines the network effects that make personalization valuable, this regression is a real risk.
One possible fix is to run user inputs through a second LLM, turning unstructured conversation into clean engagement signals. Until that approach proves itself, designers should weigh whether a conversational interface’s benefits justify the likely loss in personalization quality.
The Next Interface Abstraction: Integrated, Contextual, And Multimodal
As AI-first computing takes hold, conversational interfaces will certainly remain a core modality, especially as voice input lets us offload tasks without sacrificing our posture to screens. But dialogue alone will never cover every use case: we also need interfaces that respond to intent expressed through gestures, diagrams, or even silence. Great experiences should be integrated, contextual, and multimodal — capable of interpreting partial or visual cues when users are too rushed to type and able to see what we see when a full description would be wasteful.
Learning From The Touchscreen Trap
The current fixation on chat recalls an earlier moment of tunnel vision. Shortly after the iPhone launched, touchscreens dominated every futurist’s render of what was to come. Bret Victor, then Human-Interface Inventor at Apple, pushed back in his well-known essay pointing out that those screens were ironically anti-touch: they leaned almost entirely on sight, ignoring the hand’s capacity for haptic feedback. The lesson for AI design is the same—be wary of a single modality that starves the rest of our capabilities.
“A tool addresses human needs by amplifying human capabilities.”
— Bret Victor in “A Brief Rant on the Future of Interaction Design”
A tidy formula for when to use chat over another surface would be convenient—something like D∝1/G, positing that usability of conversation falls as desirability of graphical input rises. That observable relationship may be roughly true, but it is not actionable. Rather than predict the one true abstraction, it is more practical to invert the problem, borrowing a trick from Charlie Munger: avoid bad outcomes first, and let a good outcome follow.
Designing Backwards
Designing forward from the current technology tends to yield yet another chatbot, and we know that by itself is incomplete. Designing backward asks what failure modes are unacceptable. An intelligent interface should never force a conversation when a user has no time for one. When chat is appropriate, it should not reintroduce the same class of usability frustrations it replaces. Everyday tasks should not require extravagant solutions, and non-expert users must not be expected to master prompt engineering. Most importantly, systems that mimic human interaction should not mimic our gullibility, or we will have built perfectly social pathways to our most sensitive data.
A more intelligent interface simply does not make those mistakes.
