We’re big fans of using Agile for agencies, as you might guess by our name.  But we’re not starry-eyed practitioners by any stretch – doing Agile well in an agency setting is not easy, and in fact some techniques in the “Agile toolkit” mostly don’t work.  The Agile project estimation process is one of them.  There are ways to address this shortfall, but they are more-involved and require that you re-think some traditional relationships between priorities, the backlog, risk management and how to maintain scope.

In this article, we’ll explain why we think Agile estimation techniques come up short for agencies, and point toward a few approaches that we’ve found to be more useful and implementable.

Overview of (Traditional) Agile Project Estimation

In common Agile terminology, estimation and planning are generally treated as a duo – when you do one, you usually do the other in some form.  And because of the way that Agile is conceptually rooted (capacity is the constraint, rather than scope), in order to do estimating, you generally have to do some planning from which to build an estimate.

There are multiple times when an Agile project is subject to estimation, including the Project/Product backlog formation and grooming, cycle/sprint planning, cycle management (burndown tracking, for example), and retrospective velocity/drag-factor analysis, to name a few.  We’re most concerned with the initial estimates for a project – the point where the agency and Team knows the least about what will finally be built, and in agencies, the point at which much contracting is done around scope, i.e., the SOW or statement of work.

For the rest of this paper, what we’re calling “Agile project estimation” also includes product backlog/roadmap development and sizing.  It is the process by which “possible” scope is elaborated (usually in a roadmap on the wall with cards), prioritized (using value/risk/cost factors), and then “priced” in terms of level of effort (LOE).  Often this whole exercise is done on a wall, and the LOE is typically done in imprecise terms (tee-shirt sizing (XS, S, M, L, XL) or Fibonacci points (1, 2, 3, 5, 8, etc.) that can be mapped back to hours or sprint/cycle capacity.

Classic Agile is a capacity-based model: one determines a team’s per-cycle capacity and uses each cycle as a “bucket” for the prioritized stories in the backlog.  The typical practice has the team assess how many points they can put in a cycle and then work with the Coach and Product Owner to allocate potential stories to cycles.  The outcome looks like: “We can probably do the following stories during the course of XX cycles.”

As vague as this sounds, it actually works well in Agile software development environments.  One of the mistakes that traditional (not Agile) project estimating makes is “assumed precision”: that someone would actually give you a precise estimate (say, “36 hours”) for a  feature (“sign-up page”) doesn’t mean that it is right.  Because of the way that the brain manages uncertainty, “36 hours” often is less effective as an estimate (in understanding resource needs and potential risk/volatility) than saying it is an “L” (large), or breaking it into four stories of sizes 3, 5, 1, and 8.  “36 hours” implies that much is known, where “L” implies that there is still a range of uncertainty.  The marrying of precisions (or lack thereof) between story definition and the estimation values makes for better estimates.  Remember, the goal of estimating is to understand probable project duration and resource needs.  Probability is a cloud, not a number.

And while this works well in software development environments, for some other reasons, it is impractical in most agency settings.  They’re very different worlds in many ways.

Why Agile Estimating Doesn’t Work for Agencies

We use a 70-plus item diagnostic tool (a scorecard of sorts) to assess our agency clients as part of our AgileBlueprint process.  We then tailor the transformation to Agile across numerous themes in our Flow-Noise-Waste framework.  It takes a couple of weeks to do this usually.  Why so much work?  Because not only do agencies vary greatly from one to another, but also because they operate in a very different project context than do software development organizations.  In fact, we’ve spoken to multiple agencies that have product development teams (they build solutions) in addition to their agency delivery, project teams.  They’ve always told us that Agile works well in product development, but has been hard to do with client project delivery.  It’s not the company, it’s the context.

Here are some factors from our scorecard that play a big role in determining the suitability of Agile estimation techniques, and also help to inform what you might do otherwise.  We’ll describe them in terms of the differences.

  • Persistence of team: Agencies typically use an ad hoc team model when generating estimates – the people that do the estimate may not be the same people doing the project.  In addition, larger agencies may have teams that have never worked together, such is the turnover and also odds of getting the same CD or UX person even when the rest of the team is the same or similar.  In software environments, the teams typically work together for long periods of time… their projects are longer.

This means that an agency project team is typically lacking in terms of understanding how they will work together – they may never have done it before – and certainly their precision in this regard is much lower.

  • Persistence and experience of client or product owner:  Likewise, in most software organizations, the client/product owner (who is typically either an internal “product manager” or a real client) will tend to be more of a constant.  As such, they and the Team will be familiar with each other through prior releases or other projects.  Agencies are pretty much the opposite because of the one-off nature of the projects, again, with the exception of product development and long-term AOR relationships.

The work that is most challenging to an agency is usually from a new or inexperienced customer, often unfamiliar with their role (working with an agency) and most-often very unfamiliar with Agile.  They have never worked with his team before, and as stated above, the team doing the estimating may be a different team from the delivery team.

  • Familiarity of platform and application:  A new client can mean a new platform.  Typical digital projects include interfaces with the client’s systems or web services, and with the proliferation of new technologies in social and mobile (to name two), even similar projects may not be so similar.  Contrast this with software development environments where the platform may be a constant for years and the team familiar with the application from prior work.

Consider how much variability there is in a dialog confirmation box in a packaged or enterprise software project, versus what “confirmation” might look like in a modern cross-platform web environment.  The agency team’s ability to encapsulate features and draw analogies is much more limited than a traditional software team.

  • Experience of team:  So we’re really saying that the agency team may never have worked together, may not have worked with the client, may not have worked with the platform, and these days, they may never have done a project using Agile.  This combination is probably a roadblock for many trying to get started with Agile – the first steps of a project, estimation and planning, as a result are incredibly difficult to do, much less do well.  In addition, this team will likely be led by a PM who has (only) attended the Certified Scrum Master (CSM) training class.  It’s a good start, and will enable that PM to “ride along” with someone who knows how to do Agile, but the provides no training or guidance on how to deal with a situation like this.

Remember that a key part of the Agile estimation process involves “sizing” the cycles, deciding how many points each cycle can deliver.  This agency team has no experience in that.  You are almost asking the impossible.  Like being asked “How many cubits to the nearest Starbucks?”  What’s a cubit?  Even if you knew that it is a “unit of length measure approximately equal to the length of a grown man’s forearm”, your estimate is not going to be very precise, nor is your confidence of it.  You could guess in feet, yards or miles and be a lot closer…because that’s what you’re used to.  The newly-minted CSM, by the way, doesn’t know what a cubit is either.

This is only a partial list.  And you can see pretty quickly why we don’t really advocate using “classic” Agile estimation techniques for agency projects.  You can do it over time (all of the above challenges are cured quite well by repetition and constancy), but you can’t start there.

The Biggest Problem of All: Scope, Schedule, and LOE are Agency Contract Components

Software development is far simpler in terms of contracting – often the Product Manager has an annual or release budget and they will want to build the most important features as soon as possible, knowing that other features will get built eventually, or if not, then they weren’t that important anyway.  This is very different than the typical agency project that we see: fixed-price, under-funded, short on schedule, “optimistically estimated” (more on this in a sec), and likely staffed by shared resources (assigned to multiple projects).

We believe Agile estimating is more usable if applied early in the sales cycle.  That is, the real challenge in this situation is that scope is not discovered well in most agency sales processes.  This is a fundamental design flaw of most agency intake processes – usually resulting from overtaxed delivery resources who either don’t have the time to do good estimating, are not the resources that will actually do the work, and/or a chaotic sales process.  This is something that can be fixed with Agile estimation: the Agile roadmap exercise is a powerful and fast way to explore scope before or at the start of the project.  You may not be able to use a XS-S-M-L-XL estimate to price things (so don’t), but having the added detail and process understanding can improve your (and your client’s) actual understanding of scope and priority.

Including Agile roadmapping in your selling process can also be a differentiator.  SapientNitro, for example, has long used a transparent detailed scoping process as a competitive lever: clients love that the SapientNitro team helps them (the client) understand their scope better; the client’s perception of the agency’s “Ability to Deliver” is a major factor in contracting decisions, and Agile-style estimation and roadmapping can help a lot.  We actually teach an “AgilePitch™” process that is based on a combination of traditional “discovery” activities combined with Agile estimation techniques.

Changing your intake process is not trivial, but we believe it is a worthy pursuit – virtually every agency we speak with has problems between as-booked scope/LOE and as-delivered.  It’s not just a delivery or production problem.

Some Ways Around the Challenges Posed by Agile Estimation at Agencies

But we still need an estimate, right?  Estimates are valuable, more so in the process of team learning, but also in the very real need to forecast resources and better understand risk.  Here are a couple of techniques that we employ in our work:

  • Use a Traditional (Waterfall) Estimate:  If you have a good estimate in the first place (say, you have already booked the business), then just use that.  It’s just an estimate, and you can spend a lot of time trying to do Agile estimating and not really get any better information about LOE.  But why is it that we can use a waterfall estimate?  Because running your project with Agile (assuming that you really know how to do it) will always beat the waterfall approach.  Just the reduction in Noise and the increase in Team Flow will give you 20-30% gain.  We point out to many clients: if you booked the deal already (and most of these deals are fixed price), then let’s just get it done faster – you’re stuck with the scope anyway.

Of course, the scoping is often a problem – Sales may have booked the deal with “fictitious” hours or feature definitions.  Ten templates may really be fifteen or more…and nobody finds out until the project is underway, sometimes well-underway.  You probably want to know if this is the case; hence our recommendation on better estimating in the Sales process.  But you might want to do a quick roadmapping, just to make sure that everyone understands what the scope actually is. This is something we almost always do when we take over a project for a client.  You would be surprised at how often “well-defined scope” ends up being “Oh, yeah, I meant to mention that….” You can save yourself some misery – better to find out early rather than late.

  • Use Process-based Estimation:  At the core of most flawed sales estimation processes is the use of “feature-based” estimates.  Features are generalizations, not specifics: “a home page plus 10 pages.”  Again, when dealing with generalizations, most people estimate optimistically, not realistically.  They either can’t recall a similar feature to estimate from, or they generalize to a simpler idea than is really there (we look for phrases like “…can probably get it done in XXX hours….” being applied to a general feature).  It can also happen when the “A” team does the estimate, whether it is a separate team (supporting sales) or just a team of discipline leads that tend to be optimistic about how fast things should happen.

A quick example: How long will it take for you to go get some coffee from Starbucks?  This is a feature-based question, it describes the outcome, and you will probably estimate a different question, like: “How quickly could you get coffee from Starbucks?”  You’ll be planning optimistically, thinking about the optimal path that you, the perennial optimist could, on your best day, achieve.  To counter this normal human tendency toward optimism, we sometimes use process-based estimation to add more realism in the estimate.

Process-based estimation would change the feature/story a bit: What are the steps than you will need to do to get coffee from Starbucks and how long will they take?  All of a sudden people start thinking about how long it takes to get to their car in the parking garage, or how likely there will be a line at the store.  They’ll think through the feature a lot better. This also creates cross-discipline communication during the process, which can be very illuminating to the team (and client, if they’re there).  In general, this results in a higher LOE – often this is more correct than the optimistic LOE.  We’ve also used these numbers to construct a “Likely” LOE, somewhere between the two.

  • Use Hours, Not Points Nor T-Shirt Sizes:  You teams have probably estimated before…using hours.  As you can tell from everything we’ve covered, they don’t need another variable to deal with.  It will break their Flow and also add Noise to the important discussion.

In Summary

Agile estimating does work for agencies today, but mostly in environments that have more constancy like with product development teams and AOR-style teams that have built trust and rapport with the client.  Adapting Agile estimation to less-constant work like new clients and new projects can be challenging, but worthwhile with a some adjustments in both expectations and practice.  Ultimately, you’ll want to push the whole process upstream – this will be a bit painful at first, but better-booked projects, better resource estimates and happier teams follow pretty quickly.