I get the opportunity to coach several teams that are trying to adopt agile and understand how the worlds of business analysis and agile intersect. Inevitably, one of the questions that I always get asked is something along the lines of “how do we know whether we are successfully adopting agile?”
This is the wrong question.
Boy, that seems like a bold statement, doesn’t it? Where do I come off telling my clients that they are asking the wrong questions? That is pretty bold on my part, isn’t it?
Sort of, but not really.
You see, many organizations adopting agile these days belong to the late majority or laggard category on the adoption curve where agile is concerned. They often have, lets say, different reasons for adopting agile. Frequently it’s because their competitors are doing it and they reason they need to also in order to keep up, or they have fallen for some of the claims by some later comer consultants promising fantastic gains and the solution to world peace if the organization would only adopt agile methods.
These organizations also fall into the trap of believing that the whole purpose of adopting agile is… to adopt agile. Because they view the effort from this rather tautological perspective they seek measures of success that help them to measure how agile they are. This quest has led to the Nokia Test, multiple Agile Maturity Models (including one that is more fool than fact), stories committed compared to stories completed or agile earned value management (which should really be called expended cost management).
Trouble is, those are all measuring the wrong thing.
Let me let you in on a little secret. You shouldn’t be adopting agile for the sake of doing agile. You should be adopting agile so that you can help satisfy your customers/stakeholder’s needs better. If you are measuring how well you are adopting agile, you may be measuring some form of success, but it’s not the kind of success that really matters.
This is not new. For years teams have used Key Process Indicators (KPIs) such as adherence to agreed upon scope, sticking to the identified schedule, and keeping within budget. These KPIs do a good job of measuring how well the project adhered to project management “best practices”, but do not necessarily tell anyone if the project was successful. Well, I guess it could depending on what your definition of success is. If you look at it from the perspective of delighted customers, those KPIs aren’t too helpful.
What should we use as KPIs?
The KPIs you use should be things that tell you, at the end of the day, whether the work your project team is doing right now will satisfy the stakeholder’s needs that you set out to satisfy. The measure may be financial in nature such as a project can reduce costs, increase revenue, or protect revenue. The measure may be non financial, such as improving customer satisfaction (using a measure such as Net Promoter Score), reducing the number of inventory turns you have, or some other measurable thing that is not directly tied to financials, but still very important. This is a true measure of success because it represents some objective that the organization is trying to reach. If the project that does that ends up going over budget but not so much that the cost out weighs the benefit, the project could still be gauged a success. What’s more, the things you thought at the beginning you’d need to do to satisfy your stakeholder’s needs (assuming you determined it using that perspective) may end up being different than what you really need to do to satisfy the stakeholder’s needs. Measuring agreed upon Scope can give a false reading on whether a project is successful, especially if the scope was determined incorrectly, which is likely given how little was known about the problem and solution when the project started and scope was set.
What does that mean for agile projects?
The KPIs that make sense for a project are not necessarily dependent on the approach you use. You can use the same KPIs for a project being run using a very prescriptive, task- based plan approach.