Small models, big production footprint
Open source has repeatedly reshaped how software gets built and deployed, and artificial intelligence is following the same trajectory. Models such as the DeepSeek family, Meta's Llama releases and Mistral AI's offerings can be downloaded and run on infrastructure the user controls, with licenses that often permit modification and study. Many of these are trained on narrower data sets than their massive proprietary counterparts, earning them the label of small language models (SLMs) — and they are now competitive with larger systems in specific scenarios.
GitHub's survey of 2,000 enterprise developers across the US, Germany, India and Brazil found that nearly all respondents had experimented with open source AI models at some point. The survey predates the Open Source Initiative's formal definition of open source AI and did not isolate generative AI or LLMs specifically, so the numbers may include other machine learning categories. Still, the signal is clear: open source models are already a production reality, not a research curiosity.
Why teams pick narrow models
Three practical advantages drive adoption of smaller open models, according to Idan Gazit, head of GitHub Next: lower operating costs, the ability to run on end-user devices and — most critically — easier customization. General-purpose LLMs remain strong for broad conversational chatbots, but organizations with niche requirements often find smaller models more tractable.
Artificial intelligence consultant Hamel Husain, a former GitHub employee, argues that the narrower the task, the more open models shine. "Open models aren't always better, but the more narrow your task, the more open models will shine because you can fine tune that model and really differentiate them," he says.
Husain points to a project for an observability platform that needed to translate natural language into the company's custom query language. Because that query language is esoteric — unlike Python, the model had not seen many examples of it during training — fine-tuning was especially valuable. The client also wanted full control over the data flowing through the model, avoiding any third-party API dependency. Husain built the solution on the latest Mistral AI open models, relying on their existing fine-tune ecosystem. "I typically use popular models because they've generally been fine-tuned already and there's usually a paved path towards implementing them," he explains.
Structured output goes open source
Adoption of open models is visible in the tooling springing up around them. Outlines, a library for building custom LLM applications, helps developers constrain model output to defined structures such as JSON. It grew out of a frustration co-creator Rémi Louf experienced at a previous job when extracting structured information from document collections using general-purpose LLMs: ChatGPT and similar models produced malformed JSON outputs too often.
Louf and colleague Brandon Willard brought compiler backgrounds to the problem, recognizing that constraining an LLM's token output resembles building a parser. They built Outlines to solve their own pain point, posted it to Hacker News and found an immediate audience. "It turns out that a lot of other people were frustrated with not being able to use LLMs to output to a particular structure reliably," Louf says. The project now has over 100 contributors, led to a funded startup called dottxt, and helped inspire OpenAI's structured outputs feature. Louf confirms that unnamed "very large companies" run Outlines in production.
Tradeoffs and the road ahead
Building on open models has costs. Teams must invest in prompt engineering and, depending on the use case, stand up and manage their own inference infrastructure rather than simply paying per API call. That demands more engineering effort than calling a hosted service.
Yet some organizations want exactly that operational control. "Sometimes organizations want more control over their infrastructure," Husain says. "They want predictable costs and latency and are willing to make decisions about those tradeoffs themselves."
As small models improve, new deployment patterns emerge: running models on local hardware, embedding custom LLMs inside existing applications and outperforming larger models on tightly scoped tasks. Gazit anticipates developers will stitch together multiple specialized small models, routing each prompt to the model best suited for it — a database-optimization question to one, a code-completion request to another. "The strengths of many Davids might be mightier than one Goliath," he says.
Louf expects the open source wave to grow, driven not by incumbents but by constrained innovators. "Innovation often comes from people who are resource constrained," he notes. Developers who want to experiment can start with the free GitHub Models playground, where both open and proprietary models are available for side-by-side testing. Pick a model, click >_ Playground and begin.
