Organizations who are embracing scaled agile and digital agility need to recognize something very important (if they haven’t already): in order to be successful, a paradigm shift must occur in every aspect of corporate culture. In fact, the concept of business agility encompasses:

  • Portfolio/Program/Product Management
  • Development/Deployment
  • Budgeting
  • Talent Management
  • Project Management

These areas all have to be re-engineered, re-modeled, or re-purposed. Establishing a long-term agile culture cannot be realized by maintaining and thinking of metrics “the way you have always done them”. Your metrics must also be re-engineered, re-modeled, and re-purposed. If not, how you define and measure success won’t achieve full business agility.

Metrics of the Past Have Taught Us A Lesson

Many traditional KPIs can instigate and proliferate bad behaviors in software development. In the past, programmers have measured their progress by how many lines of code they’ve written, quality was measured by the number of bugs fixed, and user needs were indicated as having been met by checking off a list of requirements.

The software development environment, which is legendary and infamously known as the Waterfall method, included several common KPI ingredients: performance, process, and delivery.

  • Performance goals » fast delivery
  • Process goals » lean and at the lowest cost
  • Product or service delivery goals » all or nothing requirements, error free and perfect

On the surface these all seem relevant and worthy of measuring. What could possibly go wrong?

Goal that Drives Bad Behaviors

Unintentional Outcome

Deliver things fastDeliver the wrong things fast;
Sacrifice quality
Focus on low costCut corners;
Amass technical debt;
Sacrifice quality
Make processes leanCut necessary and unnecessary resources and steps without differentiating them
Evaluating that the process and/or the product is “error” freeExpect perfect process and punish errors; Expect perfect products and discredit anything that is less than perfect (people will hide the errors)
Only accept all or nothingFocus on checking the box and not prioritizing what was of highest value

Metrics in themselves can result in people figuring out ways to play the “number game”. They will do anything to make the “numbers look good”. Unfortunately, bad metrics have found their way into the agile environment. It’s important to recognize and repackage them into something productive. Metrics today need to create the type of behaviors that cultivate the “being agile” world many of us are striving towards.

Hard vs. Soft Metrics

The metrics of the past focused on what is known as hard metrics. The metrics that support the agile definition of success require soft metrics.

Hard metrics rely on verifiable data points. They use quantitative data points to evaluate a typical work routine and how well the employees carry out their assigned tasks.

Unlike hard metrics, soft metrics use subjective data and interactive responses to determine effectiveness. Soft metrics stress the impact that human capital has on business outcomes.

Hard metrics aren’t all bad, but soft metrics create a necessary balance. Here are three hard and soft metrics that balance each other and can set a very different tone for the working environment.

Hard Metrics

Soft Metrics

Performance Goals – Fast DeliveryHealthy Teams
Process Goals – Lower CostMinimum, High Value Products
Product/Service Goals – Error Free, PerfectionLearn Fast and Adjust

Another way to think about it is: hard metrics indicate if you are doing agile, soft metrics measure if you are being agile.

Agile Redefines Success

The Agile approach fundamentally redefined success factors for software delivery. People no longer work in silos and their success can no longer be measured in an individual capacity. If the product fails, the team fails, and everyone on that team fails. With the measurement of success based on a team’s effort, the ability to finger point is significantly reduced.

Let’s do a quick exercise to determine how you are measuring success. Make a check beside each criteria that you are using. Do you have more agile or legacy checks?

Agile Definition of Success

Legacy Definition of Success

Measure how well value was deliveredMeasure how well the plan was followed
Measure product successMeasure project success
Measure if an outcome was achieved Measure if the documented requirements were followed
Measure if quick feedback was received and adjustments were made quicklyMeasure if changes were avoided and the solution was sold AS IS
Measure collaborationMeasure individual contributors to the team and target bottlenecks
Measure shared learning achievementsMeasure individualized learning achievements
Measure and reward team cohesivenessMeasure and award individual contributors
Measure the Return on InvestmentMeasure cutting cost

Agile project metrics are different because they are people-centric; they recognize that people drive the outcomes. It’s more than delivering a product in the agile mindset. It becomes what the team did (or didn’t do) in their pursuit of creating a product as a team.

Consider the Values Defined in the Agile Manifesto

Individuals and Interactions Over Processes and Tools

By just counting lines of code, developers don’t need to collaborate with their colleagues or even talk to others in the development life cycle as long as they are producing code – no matter if it’s even good, bad, or usable.

The intent of an agile approach is for developers to collaborate with their colleagues, talk through designs, and transfer knowledge as they are producing code. These actions will likely slow them down at first, but it is ok. I’ve seen people go from counting lines of code to now finding ways to make their story points good. In the long run, this team collaboration will only enable faster (more complete) work.

Working Software Over Comprehensive Documentation

Using bug fixes as measurement misses the point of why there are so many mistakes in the first place. In the Agile mindset, instead of counting how many defects you close and how fast you close them, count the days of being “error free”.

I’ve seen signs in factories that display the number of days without an accident. We should create similar radiators to measurement software quality. If a team can go 90, 60, or even 30 days with no reported urgent bugs, that’s a metric worth celebrating! If 30 days is not attainable, get to the root of what’s causing the urgent bugs in the first place.

Customer Collaboration Over Contract Negotiation

Don’t think that just because you have a lot of meetings scheduled that there is true collaboration taking place. In many environments people walk out of a meeting in agreement but once the work starts and something doesn’t go as planned (or something unplanned gets identified), the finger pointing starts. More time is spent trying to recall what was or was not decided in the previous meeting rather than trying to move forward to find a solution.

I’ve also seen co-located teams sitting side-by-side, but all wearing head phones. They can go the whole day without talking. Co-location does not necessarily equal collaboration.

A good way to measure collaboration is to consider how much time is spent paired with a team mate, how much time is spent working on the same work item, voluntary participation in group problem solving, and transparency of what you are working on: “working out loud”.

Responding to Change Over Following a Plan

Unfortunately, I’ve seen some teams have success measured by the product owner getting their wish list fulfilled. Usually, the teams fighting this kind of structure can’t express their opinions and never get the chance to say ‘no’. In these cases, the product owner is (of course) 100% happy because they are getting their way without ever being questioned. The plan they laid out was followed… project complete, right?

Responding to a change does not mean the team has no input or power to push back, especially when it’s justifiable. You want to use measurements that show the team is responding to change but that it’s not haphazard or reckless. Capacity, risk, and impact on the team all need to be considered.

What’s Next

At this point, if you are at least questioning or rethinking whether the metrics you are using are undermining the mental shift that is required in an agile environment, then you’ve accomplished the first step: admitting you might need to make some changes.

Design a framework to architect and measure success for your agile organization in our Agile Metrics for Evaluating Success.

The benefit of Agile is being able to learn quickly from new information and make the necessary adjustments as soon as possible. So, how do you measure success in an agile environment? Read my Start Measuring Success With These 5 Agile KPIs blog post!

Thank you,

Jacqueline

More on Agile and Agile Metrics

Agile Metrics for Evaluating Success Course

Kanban vs Scrum Infographic

Agile Transformation Maturity and Optimization Roadmap

From a Dysfunctional to Cross-Functional Agile Team

Just Enough Doesn’t Mean None! The Agile Business Analyst Role


Also published on Medium.

Pin It on Pinterest

Share This