Reasoning before measuring

I often got this question - what do you measure while adopting Agile?

The conversation usually went like this:

[me] What is about adopting Agile?
[him] ... doing Scrum.
[me] What effect do you expect after adopting Agile?
[him] higher efficiency.
[me] How would you define higher efficiency?
[him] ... finishing more tasks.
[me] Why would adopting Agile lead to higher efficiency?
[him] Agile means higher efficiency, doesn't it?!

The thinking is fuzzy, to say the least. I suggest to focus on reasoning before measuring.

Let's take Specification by Example (SbE) as one example of agile practices. What do you measure while adopting SbE?

Practice and Purpose

Firstly, we learn about SbE by asking 1) what is SbE; and 2) why SbE.

A good source of learning is of course the book "Specification by Example".

1) What is SbE?

"Specification by Example is a set of process patterns that facilitate change in software products to ensure the right product is delivered effectively."

Blog - reasoning before measuring.png

Suppose that among those process patterns, the below two are our main focus of initial adoption. We dive deep on them.

  • Specifying collaboratively

"Instead of relying on a single person to get the specifications right in isolation, successful delivery teams collaborate with the business users to specify the solution."

  • Illustrating using examples

"The team works with the business users to identify key examples that describe the expected functionality. During this process, developers and testers often suggest additional examples that illustrate edge cases or address areas of the system that are particularly problematic."

2) Why SbE?

There are various benefits from the whole set of SbE process patterns. Let's focus on the benefits from "Specifying collaboratively" and "Illustrating using examples". In other words, let's understand the purposes of these two specific practices.

"Specification by Example helps to improve the quality of software products, significantly reduces rework, and enables teams to better align analysis, development, and testing activities."

There are three areas of benefits.

  • Higher product quality

"Specification by Example improves collaboration between delivery team members, facilitates better engagement with business users, and provides clear objective targets for delivery - leading to big improvement in product quality."

  • Less rework

"Specification by Example helps teams establish a collaborative specification process that lowers problems in the middle of an iteration. Most teams have significantly reduced or completely eliminated rework that occurred as a result of misunderstood requirements or neglected customer expectations."

  • Better work alignment

"Specification by Example enables tams to clearly define a target that's universally understood and objectively measured. As a result, many teams find that their analysis, development, and testing activities became better aligned."

The above reasoning could be summarized as below.

Blog - reasoning before measuring - 1.jpg

Only then do we go to the measuring part.

"Practice has purpose, and we measure both the practice and the purpose." I learned this in one private workshop by David Hussman for Odd-e in 2013 - one of the most memorable experiences with "the dude".

We may come up with the following measures.

  1. For the practice
  • %stories done with SbE (measuring the extent of doing SbE)
  • %stories with cross-functional conversations (measuring the extent of specifying collaboratively)
  • %stories specified with examples (measuring the extent of illustrating using examples)
  1. For the purpose
  • #defects due to misunderstood/neglected requirements after done (measuring product quality)
  • #defects due to misunderstood/neglected requirements during the sprint (measuring rework)
  • %stories reaching completion at the end of the sprint (measuring work alignment)

Causal Diagram

In the "The Book of Why", it introduces causal diagram to move from the correlation to the causation. Causal diagram is similar to causal loop diagram in systems thinking, as they both are based on the causation.

After understanding the practice and the purpose, we reason further for finer-grained causation by expanding with the intermediates. Why doing SbE leading to higher product quality, less rework and better work alignment?

The below causal diagram illustrates one reasoning.

Blog - reasoning before measuring - 2.jpg

Then, we measure the intermediates, so as to learn the whole chain of causation via validation or invalidation. Note that some variables are harder to measure than others. For example, "amount of rework" is harder to measure than "internal defects". We don't have to measure all the variables. With a few intermediate ones, we build our confidence in this causation.

We may end up with measuring:

  • %stories of doing SbE (i.e. specifying collaboratively and illustrating using examples)
  • #internal defects during the sprint
  • #external defects after done
  • %story completion at the end of the sprint


We reason about the cause and effect, before getting to the measure. On the other hand, measuring also improves our reasoning in two senses - 1) thinking about how to measure increases the rigor in the reasoning; 2) the measuring result validates or invalidates the reasoning.

Notice that measuring here is for learning and improving. See my previous blog "Two Guidelines for Metrics" for further reference.

Read more

Scrum master ทำแค่เนี๊ยะ

Scrum master ทำแค่เนี๊ยะ

เวลามีคนถามว่า Scrum master ทำอะไร แล้วผมตอบว่าทำให้ Scrum เวิร์คสำหรับทั้งองค์กร ซึ่ง โฟกัสหลัก ๆ 4 อย่างก็จะอยู่ที่ Product owner, ทีม, engineering practices และ องค์กร บางครั้งที่ผมจะได้ยินเสียงตอบกลับมาเบา ๆ ว่า “แค่เนี๊ยะ?” ในฐานะ

By Chokchai
how to สร้าง Knowledge Management

how to สร้าง Knowledge Management

ตอนเรียน Large Scale Scrum กับ Jurgen de Smet สิ่งหนึ่งที่ผมได้เรียนรู้ คือ ปัจจัยสำคัญหนึ่งที่ทำให้องค์กรหนึ่ง ๆ จะเร็วขึ้นได้ คือ จะต้องเรียนรู้ไปพร้อม ๆ กันได้ ซึ่งถ้าอยากทำแบบนั้นได้ก็จะต้อง share ownership

By Chokchai
โลกการเขียนโค้ด ตอน ซามูไรกับสปาตั้น

โลกการเขียนโค้ด ตอน ซามูไรกับสปาตั้น

ซามูไรที่ได้รับความไว้วางใจให้แก้ core logic จะมีสัญชาตญาณซามูไร คือแก้ตรงนี้ จับยามสามตาแล้วรู้เลยว่าจะไประเบิดตรงโน้น แล้ววิ่งไปสกัดบั๊กไว้ก่อนความเสียหายจะเกิด (ถ้าเป็นในหนัง ตอนนี้เป็นบทที่บั๊กร้องว่า “มืงรู้ได้ไง?!” :D) หลังจากที

By Chokchai
ประสบการณ์ TDD

ประสบการณ์ TDD

มันมีบางชั่วขณะ ที่ผมอินกับ Test-Driven Development (TDD) มาก จนอยากจะแนะนำทักษะนี้ให้คนเขียนโค้ดทั่วโลกที่สนใจเลย ผมคิดว่า ทักษะนี้มีผลเยอะมาก ๆ กับความรู้ความชำนาญในการเขียนโค้ดของผมทุกวันนี้ แต่ที่ผมไม่เคยอธิบายเป็นคำพูดออกมาได้คือ ทำไมนะ? เมื่อเช้าตอนกำลังอ่านเกี

By Chokchai