Scaling Agile: Lessons from Banff

In February 2003, practitioners and researchers gathered in Banff, Alberta for a workshop dedicated to a question that was just beginning to surface in the Agile community: how do XP and other Agile methods scale beyond teams of 10–12 people? The event drew on keynote presentations from Ken Schwaber and Martin Fowler, along with experience reports from early adopters working on larger projects.

Participants were asked to consider three specific questions: how scalable the Agile methods really are, what experiences early adopters have had, and what experiments could be conducted to expand adoption. The definition of "large" varied widely among attendees, ranging anywhere from 20 to more than 200 developers.

Scrum on Large Projects: Ken Schwaber's Experience

Ken Schwaber, co-founder of the Scrum methodology along with Jeff Sutherland, opened the conference by describing his work on large projects. He emphasized the importance of proving out the system architecture early. On one project, early sprints contained stories deliberately focused on validating the architecture, interleaved with real business cases. After several iterations of continuous refinement and testing, the team had strong confidence in the architecture's ability to meet the business's demands.

Scaling was then achieved by seeding new teams with members of the original architecture team. With the bulk of architectural issues resolved, the new teams could concentrate on building business logic atop a stable foundation.

Schwaber stressed the importance of self-organizing teams. In his experience, teams were far more effective when they decided for themselves how to complete assigned work, and he saw previously average performers rise to excellence when placed in a supportive, creative team environment. This philosophy often clashed with managers accustomed to a command-and-control style of leadership, and many found the shift intuitive.

He also raised a cautionary comparison to the Capability Maturity Model (CMM). Despite having strong advocates backing it, CMM was never embraced to the degree its authors intended, with roughly two-thirds of implementations reported as broken. A cynical view took hold in industry when certification levels seemed to depend on the firm hired to conduct the evaluation. Schwaber wondered whether Agile methods might meet a similar fate, noting that early adopters are often highly skilled and passionate teams—people who might succeed regardless of methodology.

Field Reports from Large Team Practitioners

The middle portion of the workshop was given over to practitioners sharing their experiences with large teams.

Philippe Kruchten of Rational described how teams of 200+ developers using RUP had delivered large projects successfully. In the early stages, a senior team establishes the high-level architecture and development practices. As the project moved into construction, additional teams were added, seeded with founders of the original elaboration team. When broken up appropriately, each team could work autonomously in an iterative cycle, with sync points and milestones coordinating dependencies between teams. This approach gave teams the feeling of being decoupled enough to enjoy the benefits of small Agile teams.

Gerard Meszaros of ClearStream Consulting shared his experience building digital switching systems for a large telecommunications company. The project made use of frequent releases and significant architectural change, but the system only reached this point once it had achieved critical mass. He also noted a gap in the industry literature: most writing on XP and Agile targets developers who already know how to build software. As he put it, "Today we have the cook books for people who know how to cook. What we do not have today are cook books for people who do not know how to cook."

A Canadian energy company reported success using a hybrid of Scrum and XP to manage their projects. With solid management backing, they had dedicated a team to helping other internal groups adopt Agile. Practitioners in their organization appreciated the ability to adapt practices to the needs of individual teams, which they preferred to the "one size fits all" project management approach they had previously used.

The authors of this report—both XP practitioners—used the opportunity to share their views on which XP practices scale. Those related to the mechanics of writing code—test writing, refactoring, and pair programming—seem likely to scale to large teams; these are commonly accepted engineering practices that predate XP and Agile and work on projects of any size. Communication-centric practices pose a greater challenge. Daily standup meetings become awkward beyond 20 people, and iteration planning involving all developers and analysts at once incurs significant organizational overhead. Ultimately, scaling XP runs into the same challenge facing all large projects: effectively coordinating and communicating among large numbers of people.

The authors also questioned the premise behind the question, preferring instead to examine ways of breaking large projects down into smaller ones where XP is known to work, rather than figuring out how to stretch XP into configurations where it was never intended.

Martin Fowler: Why Scaling Is the Last Thing You Want

Martin Fowler's closing keynote, titled "Why Scaling Agile is the Last Thing You Want To Do," reshaped the conference conversation. What had largely been framed as solving the problem of how to scale XP was turned on its head: is this something we really want to do?

Fowler's talk was split into two parts. First came an experience report from a Thoughtworks project involving a large team using XP on a complex leasing application. He reminded the audience why business applications are so difficult to build: business logic is an oxymoron. There is little logical about most aspects of business, particularly leasing. Capturing business rules, hidden assumptions, and special cases and abstracting them into a solid domain model is among the most difficult parts of design.

Requirements gathering on large projects is similarly difficult. Fowler drew an analogy to book writing: writing a book requires enormous time and energy, and even with the input of professional editors and peer reviewers an author frequently fails to communicate exactly what was intended. If authors struggle to convey a message with the resources available to book publishing, consider the challenge embedded in a complicated business application being built with comparable requirements-capturing resources.

One practice that worked remarkably well was rotating developers across different parts of the application. This frustrated project analysts initially, since developers would start understanding a domain just as they were shifted elsewhere. Over time the benefits became apparent: developers gained experience enough across the codebase to fill numerous roles. The team became less dependent on a small set of specialists, developers strengthened their technical skills and business understanding simultaneously, and those with a global view of the application had the context needed to maintain the overall architecture while adding new features.

Fowler don't contest that some projects genuinely require very large teams—telecommunications and major military programs will typically demand what amounts to thousands of lines of work that small teams cannot produce quickly enough. For those projects, scaling Agile is a genuinely pressing question.

His experience, however, suggested that many teams are needlessly large, and they should search first for ways to scale down rather than scaling up their methodologies. He asked attendees to raise their hands if they had ever been on a project that could have dropped its weakest half with roughly equal productivity; the majority raised theirs. That visual captured a shared sentiment: before looking at ways to make XP work for a 200-person team, it makes sense to look at why the team is 200 people in the first place.

Evidence and the Scaling Question

A recurring point of uncertainty at the workshop was how to gather empirical proof that Agile methods hold up on large projects. The research community sees this as a central task, and is working to design experiments that demonstrate whether methods like XP can scale beyond small, co-located teams.

The goal is reasonable, but the path is not straightforward. Measuring software quality and productivity objectively remains a fundamental problem—without quantifiable metrics, controlled experiments around engineering practices are difficult to construct. Another view raised at the workshop is that the real insight lies in sociology. Since communication is so central to Agile success, understanding how humans interact—and why certain dynamics work—might provide more useful answers than traditional metrics-driven studies.

Outlook

The workshop was productive, and attendees valued the opportunity to discuss the hard problems the research community is tackling. Agile methods are increasingly gaining serious attention in academic circles, which is a meaningful shift. Still, as with any new approach, wide adoption will bring both correct and incorrect applications. The coming years will likely show that the practice of scaling XP is as much about judgment as it is about process.