September 2015 Archives

Two Guidelines for Metrics

During the recent Advanced ScrumMaster course, two guidelines for performance metrics emerged.


Continuous Improvement over Performance Evaluation


"Measuring and Managing Performance in Organizations" by Robert D. Austin is a great source for the topic of metrics. He made the distinction in terms of the purpose of using the metrics, either as motivational or as informational.


Motivational purpose ties to Performance Evaluation, while informational purpose ties to Continuous Improvement. Metrics could be the same, but used with different purposes. Take the example of unit test coverage. Management uses this metric as one KPI for team, the higher, the better. This is motivational. Team itself may find that coverage provides meaningful insights to guide their improvement in unit testing, thus, they decide to measure it. This is informational.


Austin claims that all metrics with motivational purpose inevitably lead to measurement dysfunction to some extent. This is what we see in reality that some teams don't add any check/assert in their unit test purely for the sake of achieving measured coverage target.


This leads to the first guidance: Continuous Improvement over Performance Evaluation


Collaboration over Accountability


When you cannot yet avoid motivational use of metrics, the technique of measuring up helps. Traditionally, we measure based on the span of control. By measuring up, we measure based on the span of influence.


Take a couple of examples. Traditionally, we measure developers based on the development output, e.g. lines of code; we measure testers based on the testing output, e.g. number of bugs found. By measuring up, we measure the collective output from the cross-functional team. This applies at broader scale too. Suppose that multiple teams work on different features and those features together form a complete customer scenario. Traditionally, we measure single team's output, e.g. delivered feature. By measuring up, we measure the collective output by multiple teams, e.g. delivered customer scenario.


Through measuring up, we promote the bigger common goal, thus more collaboration. Would this sacrifice clear accountability? Probably yes, but does it matter?


Yves Morieux's Ted Talk "How too many rules at work keep you from getting things done" makes a great point on this question.


In relay race, who's to blame (sorry, who's accountable:) when the baton drops? It's not clear, so, in order to have more clear accountability, you may introduce a third person, whose sole responsibility is to take the baton from one and pass that to the other, thus, he would be accountable when it fails. This is a system designed for failure, failure with clear accountability. However, would that help win the race?


Moreover, you may try measuring one-level up, thus, still take accountability into consideration to some extent.


This leads to the second guidance: Collaboration over Accountability


Conclusion:


So often, people ask what metrics we shall use in Agile context. In my humble opinion, these two guidances are more critical for your success in effectively using metrics.


Note: special thanks to my co-trainer Sun Ni for great insights.


When work and skill mismatch

You have more work in one area than what the people in that area can do. You also have available people in the organization, but their skill does not match the work.


Will this happen? Believe me, it will always happen at some point, either at team level or at individual level, or both. This is simply due to dynamic work vs. "static" skill (really? maybe as snapshot, and hopefully not in the life span). When that happens, there are a few strategies to deal with it.


  • Push it

If you are working in matrix and responsible for one project, the obvious strategy is to grab more people (or resource, are people resources?) with the right skill and allocate them to the area having more work. What if other projects suffer even more? It's none of your business.


If you fail to get more people, another strategy is that you anyway push this through the developers working in your project. This inevitably leads to the application of developer's secret toolbox, which sacrifices quality and gradually leads to more and more legacy code, and eventually slows down the development, a lot.


  • Accept it

If you take a bigger and longer-term view, you see that "push it" strategy creates more problems than it solves. Another strategy is simply accept the fact, pull the work based on the capacity, with the acknowledgement that the output is constrained by the bottleneck. You either extend the schedule or reduce the scope, or a bit of both.


This is wise in the sense that it respects the system, thus the system would not backfire. However, the speed or the value of delivery suffers and you may not survive.


  • Grow it

This strategy is quite logical, as bottleneck area constrains the output, you grow the capacity in that area. It does solve the problem of delivery. However, as bottleneck area moves, it leads to staffing based on the worst scenario.


This is not lean. As it is practically hard to shrink in size, some teams and individuals will inevitably work on low-value stuff and create wastes. And, the more important waste is the  underutilization of the people potential. People differ greatly from resource because people can learn. By limiting people only in their familiar areas does not show the respect for people, which is one of the two pillars in lean thinking.


  • Learn it

Then it comes to the most sensible strategy. You treat mismatch as an opportunity to learn and expand skills. This does not mean that specialization is bad. You still utilize the specialization for efficiency, meanwhile take advantage of mismatch for people to learn new areas. This is lean. Long-lived feature team adopts this strategy, and LeSS explicitly promotes feature team structure.


Reference


  1. feature team primer
  2. more on feature team chapter in "Scaling Lean & Agile Development: Thinking and Organizational Tools for Large-Scale Scrum" book

About this Archive

This page is an archive of entries from September 2015 listed from newest to oldest.

August 2015 is the previous archive.

October 2015 is the next archive.

Find recent content on the main index or look in the archives to find all content.