A voice-driven setup for writing code

When a repetitive-strain injury made it nearly impossible to use a mouse or keyboard, the author of this piece had to rethink how to code. After exploring physiotherapy, ergonomics, braces, and various other fixes, the solution turned out to be a combination of a microphone and an eye-tracker. The result is a workflow that allows near-total hands-free software development.

Here's what that setup looks like in practice, and how it handles the demands of real programming.

The software: Talon Voice

Typical dictation software is built for transcribing natural speech, not for writing syntax-heavy, convention-laden code. Talon Voice is different: it's designed specifically for developers who can't use their hands, and it runs in two flavors. The free public version is available to anyone, but the more capable features live in a paid private beta, which you can access via the creator's Patreon.

Spelling and special characters

For individual letters, Talon uses its own phonetic alphabet—mostly single-syllable words like "air" for a, "bat" for b, "cap" for c, and "drum" for d. This avoids the ambiguity inherent in spoken English. Speaking "drum" writes the letter d just as if you'd pressed the key. To capitalize, prefix with "ship": "ship drum" produces D. Numbers are straightforward, spoken digit by digit, so saying "one zero two four" outputs 1024.

Special characters and hotkeys have intuitive mappings, like command cap for copy, or control command space to open the MacOS emoji drawer. Some keys get nicknames—backspace becomes "junk," delete becomes "dell"—and every mapping is user-editable.

Navigation uses the word "go": "go left" moves the cursor left. Repeating commands relies on ordinals rather than plain numbers, since numbers are reserved for literal output. Saying "go left ninth" moves left nine spaces. To write the number 1000, you'd say "one zero third" to repeat the 0 three times. Ordinals work with every command in the system.

Formatters for code style

Programming languages are full of naming conventions, and Talon handles that with formatters, which transform spoken text into a specified format. Saying "camel hello world" yields helloWorld, while "snake hello world" gives hello_world. If you want plain text, the command is say: "say hello world" writes hello world.

Formatters can be stacked:

const DARK_COLORS = {
  primary: 'hsl(230deg, 100%, 50%)',
  // ...and so on
};

Combining allcaps and snake turns "dark colors" into DARK_COLORS, perfect for JavaScript constants.

Extending Talon with your own commands

Talon runs in a command mode by default, treating spoken phrases as function calls. Saying "focus chrome" is like invoking a focus command with the argument chrome, switching to that application. But focus isn't a built-in black box; it's part of a community-maintained package. The source is accessible and written in Python:

class Actions:
  def switcher_focus(name: str):
    """Focus a new application by name"""
    for app in ui.apps():
      if name in app.name and not app.background:
        app.focus()
        break

The real power lies in authoring new commands. Simple mappings can be defined with a YAML-like syntax:

react: insert("import React from 'react';")

Speaking "react" outputs import React from 'react';. For more complex logic, you can write full Python functions. A custom command can recognize a spoken HTML element and component name, generate corresponding styled-component code, and even position the cursor:

const FancyButton = styled.button`
  | <-- Cursor placed here
`;

The underlying implementation uses a Python function and a Talon mapping to connect the spoken words to that function:

@ctx.capture(rule='styled <user.html_elements> <user.text>')
def create_styled_component(m):
  component_name = actions.user.formatted_text(
    m.text,
    'PUBLIC_CAMEL_CASE'
  )

  return f'const {component_name} = styled.{m.html_elements}``'
<user.create_styled_component>:
  insert(create_styled_component)
  key('left enter enter up tab')

Resolving homophones

Speech recognition has inherent ambiguity—two words can sound identical. If you say "check out my site," the software might write sight or cite instead. To fix this, Talon includes the phones command. Selecting the misrecognized word and saying "phones" pops up a numbered list of alternatives like cite, sight, and site, after which you can say "pick 3" to choose the correct spelling.

The hardware: eye tracking for mouse control

Most of the mouse work is handled by an eye-tracker, specifically the Tobii 5, a hardware bar with infrared sensors that mounts under the monitor. Although marketed for gaming on Windows, Talon includes custom drivers that let the device control the cursor on a Mac.

Clicking is a two-phase process:

  1. Look at the target and make a popping noise with your mouth. This zooms into the area for precision.
  2. Pop again to perform a left-click.

There are also voice commands for double-clicking, right-clicking, and drag-and-drop. Practice improves accuracy considerably, enough for precise UI interactions. The Tobii 5 sells for $229 USD; an older Tobii 4C model is said to work even more smoothly with Talon, though it's harder to find.

Current status and growing pains

The author reports working at roughly half their previous speed—partly due to slower dictation, but more from a need to prioritize ruthlessly. Voice strain is the biggest ongoing challenge, since talking for hours a day isn't natural at first. The early weeks were also difficult because customizing Talon own commands was necessary, but configuring it entirely by voice became a milestone once endurance built up. Learning Vim might improve efficiency further, though it's not been tried yet. There are promising alternatives emerging in the space, including Serenade and Neuralink, but Talon continues to get better through proprietary machine learning improvements. For now, the discovery that hands are no longer an absolute requirement for writing software has been a significant relief.

Why accessibility is everyone's problem

It's easy to treat accessibility as a concern for a hypothetical "other" group—until you're the one who needs it. I'm an edge case: most people won't develop Cubital Tunnel Syndrome, and when they do, it usually resolves on its own or with conservative treatment. But nearly all of us will eventually experience some form of impairment, whether temporary or permanent. The best case scenario is that we live long enough to deal with vision loss and declining motor skills.

I've known accessibility matters for years, but it felt abstract. I'd never watched someone struggle with something I built because I hadn't tested it without a mouse or keyboard. That changed when I started relying on an eye-tracker. Before I got comfortable with it, navigating an internet not built for alternative input mechanisms was genuinely tricky—and certain tasks were vastly harder than they should have been.

The internet is critical infrastructure, a necessary part of modern life. It needs to be accessible, and as front-end developers, advocating for that is part of the job. If you want to dig deeper, a11y.coffee is a good starting point.

Don't wait for the perfect moment

This experience also reshaped my priorities. One of my first web apps, built about a decade ago with PHP, MySQL, and jQuery, was an education platform. I abandoned it when I discovered Khan Academy—essentially the same idea, done far better. I later worked there as a software engineer, some of the most fulfilling work of my career.

I'd long imagined starting something of my own in education, but kept postponing it. The realization that my time isn't unlimited was the nudge I needed: if there's something important, do it now, because later isn't guaranteed.

A few weeks ago, I left my post as Senior Staff Software Engineer at Gatsby Inc to pursue that goal. My first project is an interactive online course teaching advanced CSS to JavaScript developers, aimed at building real confidence with layouts and next-level interfaces. More details are on the CSS for JavaScript Developers site.

Credits

Thanks to former coworkers Amberley and Madalyn, who suggested dictation—it never would have crossed my mind on my own.

I also drew inspiration from two conference talks on coding by voice: