LangChainJS Comes to Cloudflare Workers
LangChain has announced support for multiple JavaScript environments, including Cloudflare Workers. The move lets developers use the popular LLM framework in serverless environments, opening up new possibilities for AI-powered applications at the edge.
"Our goal for LangChain is to empower developers around the world to build with AI. We want LangChain to work wherever developers are building, and to spark their creativity to build new and innovative applications." — Harrison Chase, Co-Founder and CEO, LangChain
LangChain provides a unified interface for working with different large language models, making it possible to swap between providers and chain prompts together into more complex workflows. Combined with Cloudflare's worker runtime, developers can build and deploy LLM applications without managing traditional server infrastructure.
Project Setup
Before you start, you'll need three things:
- An OpenAI account (free tier is available)
- A Cloudflare Workers account upgraded to the $5/month plan
- Node.js and npm installed locally
Create a new folder called langchain-workers, navigate into it, and run the Cloudflare Workers scaffolding tool:
When prompted, choose these options:
- Application name:
langchain-worker - Application type: "Hello World" script
- TypeScript: No
- Deploy now: No
Next, store your OpenAI API key as an encrypted environment variable using wrangler, then install the LangChain.js package via npm. With the Worker scaffolded and the package installed, verify everything runs with wrangler dev — pressing b in the terminal should open your browser to a "Hello World" page.
Loading Source Material
Language models only know what they were trained on. To ask questions about new or specific text, you must feed that text to the model. In LangChain, this is done with documents — objects that contain text plus optional metadata describing the content.
Rather than constructing documents manually, you typically load them from existing sources. LangChain provides a set of document loaders for different formats (CSV, PDF, HTML, plain text) that can pull content locally or from the web. This example uses the Cheerio-based loader for web pages, which requires installing the cheerio package:
npm install langchainWith Cheerio installed, import CheerioWebBaseLoader at the top of src/index.js and instantiate it with the URL of a Wikipedia article of your choice. Calling its load() method retrieves the page content and wraps it in a single document object:
import { CheerioWebBaseLoader } from "langchain/document_loaders/web/cheerio";Check the Worker logs after triggering a request — you'll see the full article content in a document object. You can point the loader at any URL, which makes testing with different sources trivial.
Splitting Text Into Manageable Chunks
LLMs often have limits on how much text you can pass in a single request, and some APIs charge per token. Sending an entire Wikipedia article for every query is wasteful. LangChain's text splitters divide content into smaller chunks, each stored in its own document object.
Swap the load() call for loadAndSplit() to apply a default splitter:
const docs = await loader.loadAndSplit();Trigger the Worker again and inspect the logs. The loader now returns an array of many document objects instead of a single large one — each slice splitting the article into more appropriately sized pieces for model requests.
Embeddings and Vector Storage
With the article chunked up, you still need a way to determine which documents are relevant to a given question. That's where embeddings and vector stores come in.
Embeddings translate text into vectors of floating-point numbers, placing semantically similar content closer together in a multidimensional space. The OpenAI Embeddings API generates these representations:
Once captured, the embeddings and their source text need a home. Vector stores are databases specialized for indexing and querying text by embedding similarity. This app uses MemoryVectorStore, an ephemeral in-memory store; larger projects may prefer persistent options like Chroma or Pinecone, both of which LangChain supports.
Add the appropriate imports and replace the previous logging code with the embedding and storage setup:
const store = await MemoryVectorStore.fromDocuments(docs, new OpenAIEmbeddings({ openAIApiKey: env.OPENAI_API_KEY}));Models and Chains for Q&A
The final pieces are models and chains. LangChain's model interface abstracts away provider-specific details — here, you plug in OpenAI by passing your API key. Chains combine a model with other data sources or APIs.
This app uses a RetrievalQAChain, which queries the vector store for documents related to the incoming question and feeds those to the model to generate an answer. Import the necessary classes and instantiate the chain after the vector store is ready:
const model = new OpenAI({ openAIApiKey: env.OPENAI_API_KEY});
const chain = RetrievalQAChain.fromLLM(model, store.asRetriever());
const question = "What is this article about? Can you give me 3 facts about it?";
const res = await chain.call({
query: question,
});
return new Response(res.text); The chain call returns a text response you can send back directly from the Worker. At this point, the question is hard-coded. To let users submit arbitrary queries, parse a question parameter from the URL query string using the native URLSearchParams API, falling back to the default question when no parameter is present.
Run wrangler dev again and visit the local Worker URL with a query string like ?question=When%20was%20Brooklyn%20founded? — the Worker should return an answer based on the article content you loaded.
Deploying to Production
When everything works locally, run npx wrangler publish to deploy. Wrangler returns a Workers URL where your application is live.
The complete example application is available on GitHub. Questions and community discussion can be directed to the Cloudflare Discord server, Twitter handle, or community forum.
