A Co-Pilot for Strategy Work
Boba is an experimental AI co-pilot built to augment product strategy and generative ideation. The goal is to test how LLM-powered applications can move beyond simple chat interfaces and into structured tools that support creative work. Boba is designed around a set of patterns that help users interact more effectively with an LLM, navigate complex conversational flows, and incorporate knowledge the model doesn't have access to.
The application targets the early stages of strategy work—divergent thinking, concept generation, and rapid idea formulation. It uses OpenAI's GPT 3.5 to assist with several distinct tasks:
- Research signals and trends: Searches the web for articles and news to answer qualitative research questions (e.g., "How is the hotel industry using generative AI today?").
- Creative Matrix: A concepting method that generates ideas at the intersections of different dimensions. Given a strategic prompt like "How might we use generative AI to transform wealth management?", Boba combines dimensions (e.g., value chain stages and personas) to produce combinations of ideas.
- Scenario building: Generates future-oriented narratives from signals of change in business, culture, and technology. Users can specify time horizons and levels of optimism or realism.
- Strategy ideation: Uses the Playing to Win framework to brainstorm "where to play" and "how to win" choices for a given prompt and set of future scenarios.
- Concept generation: Produces product or feature concepts—including value proposition pitches and testable hypotheses—from a "how might we" prompt.
- Storyboarding: Creates visual storyboards from a prompt or narrative describing customer journeys, with customizable scene styles.
The web interface presents a task panel on the left, and clicking a task changes the main panel to a tailored UI for that workflow. For instance, in the scenario design view, users enter a prompt and use additional controls to set a time horizon and prediction tone. Boba then enriches the user's simple request with task-specific knowledge before forwarding it to the LLM, returning a structured response that is rendered as individual UI elements for each scenario. From there, a user can select a scenario and open a focused conversation about it—without needing to re-state the context for each follow-up question.
Beyond a Plain Chat Box
A simple front-end that talks directly to an LLM forces the user to become a prompt engineer, learning by trial how to phrase requests to get useful answers. Boba instead mediates the interaction with UI elements that structure the conversation, so naive user input is enriched automatically before it reaches the model.
Because the LLM's training data has a cutoff date, it cannot handle current information on its own. Boba addresses this with a feature that combines the LLM with regular web search: the user gives a research question, Boba sends an enriched version of that query to a search engine, retrieves articles, and then asks the LLM to summarize each one. This also guarantees that any source links presented to the user are real—they come from the search results, not from the LLM's ability to fabricate plausible citations.
The patterns emerging from this design work are generally applicable to LLM-powered applications: templating prompts to enrich a user's raw request with task context, structuring and validating the model's output for clean rendering, and maintaining conversational context across different exploration branches so the user can move between tasks fluidly. Boba's emphasis on orchestrating between the user and the model—not merely exposing the model's raw chat interface—is what makes it a co-pilot rather than a chatbot.
Patterns for Building a Generative Co-Pilot
Building Boba taught us several practical lessons about mediating conversations between users and LLMs like OpenAI's GPT-3.5/4. The patterns below reflect what we learned while designing for interactivity, context, and iterate-ability — and they're far from exhaustive.
Templated Prompts and Chaining
The simplest pattern is text templating: enriching a prompt with context and structure before sending it to the model. We used Langchain, which provides a standard interface for composing "chains" of prompts. If you've used a Javascript templating engine like Nunjucks or Handlebars, Langchain feels familiar — but it's built specifically for prompt engineering workflows, with support for input variables, few-shot templates, validation, and composable multi-step chains.
For instance, Boba's strategic brainstorming feature accepts a prompt like "Show me the future of payments" or even just a company name. The underlying prompt template looks like this:
You are a visionary futurist. Given a strategic prompt, you will create
{num_scenarios} futuristic, hypothetical scenarios that happen
{time_horizon} from now. Each scenario must be a {optimism} version of the
future. Each scenario must be {realism}.
Strategic prompt: {strategic_prompt}
Response quality is inherently bounded by prompt quality. We leaned heavily on techniques like Adopt a Persona — telling the model it is a visionary futurist — to steer output toward useful, relevant completions. We iterated on prompts directly in ChatGPT first, which gave us the fastest feedback loop. That said, we spent roughly 80% of our time on the UI and prompt engineering, and only 20% on the AI plumbing itself.
We also deliberately kept prompt templates free of conditional logic. When a user action required a fundamentally different output — like clicking "Add details (signals, threats, opportunities)" — we switched to an entirely separate template rather than complicating a single one.
Structured Responses
Most real applications need to parse LLM output into structured data to operate on it further. Boba relies heavily on JSON, and we were surprised by how consistently GPT returned well-formed JSON when instructed. Here's an example of the response instructions for scenario generation:
You will respond with only a valid JSON array of scenario objects.
Each scenario object will have the following schema:
"title": <string>, //Must be a complete sentence written in the past tense
"summary": <string>, //Scenario description
"plausibility": <string>, //Plausibility of scenario
"horizon": <string>
Even fairly complex nested JSON schemas worked well, provided we described them — sometimes in pseudo-code. Here's how we described the nested response for strategy generation:
You will respond in JSON format containing two keys, "questions" and "strategies", with the respective schemas below:
"questions": [<list of question objects, with each containing the following keys:>]
"question": <string>,
"answer": <string>
"strategies": [<list of strategy objects, with each containing the following keys:>]
"title": <string>,
"summary": <string>,
"problem_diagnosis": <string>,
"winning_aspiration": <string>,
"where_to_play": <string>,
"how_to_win": <string>,
"assumptions": <string>
Describing the schema also gave us a lever to nudge response quality. In the Creative Matrix feature, we wanted the model to consider each idea within the context of its row and column intersection. A few-shot example with a specific output schema helped the model "think" in the right frame:
You will respond with a valid JSON array, by row by column by idea. For example:
If Rows = "row 0, row 1" and Columns = "column 0, column 1" then you will respond
with the following:
[
{{
"row": "row 0",
"columns": [
{{
"column": "column 0",
"ideas": [
{{
"title": "Idea 0 title for prompt and row 0 and column 0",
"description": "idea 0 for prompt and row 0 and column 0"
}}
]
}},
{{
"column": "column 1",
"ideas": [
{{
"title": "Idea 0 title for prompt and row 0 and column 1",
"description": "idea 0 for prompt and row 0 and column 1"
}}
]
}},
]
}},
{{
"row": "row 1",
"columns": [
{{
"column": "column 0",
"ideas": [
{{
"title": "Idea 0 title for prompt and row 1 and column 0",
"description": "idea 0 for prompt and row 1 and column 0"
}}
]
}},
{{
"column": "column 1",
"ideas": [
{{
"title": "Idea 0 title for prompt and row 1 and column 1",
"description": "idea 0 for prompt and row 1 and column 1"
}}
]
}}
]
}}
]
Because LLMs "think" in tokens, having the model output the row and column values before generating ideas effectively re-anchored its context, resulting in better matches.
Since we built this, OpenAI released Function Calling, which formalizes structured output by letting developers declare callable functions and JSON schemas, after which the model returns a function call with conforming arguments. It's especially useful for triggering external tools — like a web search or API call — in response to a prompt. Langchain already has similar functionality, and we'd expect native integration between its tools API and OpenAI's function calling soon.
Real-Time Progress Streaming
One of the first things you learn when putting a GUI on an LLM is that waiting for the full completion is too slow. ChatGPT hides this by streaming. Users won't wait long on a spinner, so we decided users should see partial responses within a few seconds.
We recommend streaming responses across the full stack for any prompt that takes longer than a few seconds. Both the Langchain and OpenAI APIs support doing exactly that:
const chat = new ChatOpenAI({
temperature: 1,
modelName: 'gpt-3.5-turbo',
streaming: true,
callbackManager: onTokenStream ?
CallbackManager.fromHandlers({
async handleLLMNewToken(token) {
onTokenStream(token)
},
}) : undefined
});
Streaming enabled real-time progress, plus the ability for users to stop a generation mid-completion if ideas were going off track. The trade-off is added complexity on the view and controller layers: best-effort JSON parsing and maintaining temporal state during the LLM call. New tools are emerging to help — the Vercel AI SDK, for example, targets edge-ready streaming chat UIs.
Select and Carry Context
A chat window locks users into a single-threaded context, which is limiting. We recommend thinking hard about UX affordances that perform actions within a selection — like pointing at something while describing it.
Select and Carry Context lets users narrow or broaden scope for a task by selecting UI elements, then acting on them. In Boba, that means selecting an idea (via checkbox or click) and then generating variations or starting a focused "Explore" conversation about it. With "Brainstorm strategies and questions for this scenario," the selected scenarios carry over as subprompts.
Implementation difficulty depends on context size and nature. When context fits within a single LLM context window, prompt engineering suffices. For the "Explore" chat, we assembled a multi-message conversation in the backend:
const chatPrompt = ChatPromptTemplate.fromPromptMessages([
HumanMessagePromptTemplate.fromTemplate(contextPrompt),
HumanMessagePromptTemplate.fromTemplate("{input}"),
]);
const formattedPrompt = await chatPrompt.formatPromptValue({
input: input
})
Another approach embeds the context into the prompt within tag delimiters:
Your questions and strategies must be specific to realizing the following
potential future scenarios (if any)
<scenarios>
{scenarios_subprompt}
</scenarios>
When context exceeds the LM's context window — or you need richer interaction history — you'll steer toward external short-term memory, typically a vector store, as shown in the next section. For deeper thinking on this pattern, we recommend Linus Lee's (Notion) talk, "Generative Experiences Beyond Chat."
Contextual Conversation
A special case of carrying context is giving users a direct conversational channel with the model. Even with rich UI affordances, sometimes a plain natural-language exchange is the clearest. Offering a contextual chat covers interactions you haven't designed for.
We suggest pairing this with example messages so users know what kind of questions work. The response to such a chat can render as formatted Markdown, and the entire conversation context can be supplied via a system message.
Out-Loud Thinking
Andrej Karpathy's phrase — "LLMs 'think' in tokens" — deserves a literal reading: models make more reasoning errors when forced to answer immediately than when given more tokens to work through the problem. In Boba, Chain of Thought (CoT) prompting, or asking for reasoning steps before final answers, moved quality up noticeably.
For strategy and concept generation, we ask the model to first propose a set of questions expanding the user's input, then generate the ideas. Show those intermediate questions to the user; they improve transparency and suggest other directions to explore, refining the next action. A variant keeps the monologue internal, in a part of the response your parser discards — a pattern documented in OpenAI's GPT Best Practices Guide.
Iterative Response
Even strong prompts lead to misinterpretations and off-target results. A co-pilot's real value lies in supporting fluid, back-and-forth refinement:
- Corrections to original input parameters
- Edits to a specific part of a response
- Scalar or written feedback nudging future results
We see this clearly in Boba's Storyboarding feature. We generate scenes with Stable Diffusion, but LLM-generated diffusion prompts leave image quality up to chance. So we let users click any image, edit the Stable Diffusion prompt in place, and regenerate just that scene context — while preserving the rest of the board.
We're also exploring richer feedback patterns — combining select-and-carry with iterative response, accepting thumbs up/down or conversation feedback to improve the next batch of recommendations. GitHub Copilot, which demotes ignored code suggestions, is a working example. This pattern is among the most valuable, but demands short- or long-term memory to fold feedback into later responses.
Embedded External Knowledge
Sometimes persistent context and current data are required. Boba's Research feature answers qualitative questions from live web content, like "How is the hotel industry using generative AI today?" This meant equipping the model with a Google web search tool and a way to read articles longer than the context window, and to chat about them later.
Here's the implementation sequence:
- Use a Google SERP API to return the top 10 results.
- Read full article content using the Extract API.
- Save article content in short-term memory — specifically an in-memory vector store, with embeddings generated by the OpenAI API from chunks per article.
- Embed the user's search query.
- Query the vector store with the query embedding.
- Prompt the LLM with a prefix of vector-store results as context.
Many steps, but Langchain's VectorDBQAChain reduced the heavy lifting to a few lines:
const researchArticle = async (article, prompt) => {
const model = new OpenAI({});
const text = article.text;
const textSplitter = new RecursiveCharacterTextSplitter({ chunkSize: 1000 });
const docs = await textSplitter.createDocuments([text]);
const vectorStore = await HNSWLib.fromDocuments(docs, new OpenAIEmbeddings());
const chain = VectorDBQAChain.fromLLM(model, vectorStore);
const res = await chain.call({
input_documents: docs,
query: prompt + ". Be detailed in your response.",
});
return { research_answer: res.text };
};
We used HNSWLib, an in-memory HNSW graph index — among the best for vector similarity. For larger production workloads, an external vector database like Pinecone or Weaviate is the right call.
We chose manual Google search via Langchain's toolkit rather than its full external tools API because it gave us control and avoided mixed, slow outputs. OpenAI's Function Calling offers another viable path for hooking in external actions.
In sum, Boba's research feature combines two distinct techniques: use of external search tools and a short-term in-memory vector store.
What’s next for Boba
The prototype described here is only a starting point. Boba is an early exploration of what a generative co-pilot for product strategy and ideation might look like, and the full scope of such a system remains largely uncharted. We expect many of the core principles and engineering patterns for LLM-powered co-pilots to emerge over time as the field matures.
For now, we’re focused on learning from real usage and iterating on the architecture. The design space is wide open, and we’re eager to share more as we refine the approach and uncover new practices for building reliable, useful generative applications.
