Product learning in Sprint review

Sprint review is the Scrum element for which I evolved my understanding the most during the past 10+ years.

There still exists two prevalent but mistaken views.

• Sprint review is about showcase. The demo could be an effective vehicle for product learning, but showing "good" case, and hiding "bad" case completely miss the point of sprint review.
• Sprint review is about acceptance. The deep mindset involved is still contract and blame, rather than collaboration and learning.

In my view, the proper focus of Sprint review is product learning.

Levels of product learning

We learn about product from what and why. For what, there are various levels. We learn about story details on acceptance criteria; we learn about release progress on release turndown; we learn about meaningful product slices on story map. For why, we learn about product goal on impact map. Let me explain.

• what/story, on acceptance criteria

We learn if the team has delivered the right stories. The main learning here is whether we had the shared understanding for those stories. If PO and team collaborate well during the sprint, chances are, PO has already checked this with team. Then, it becomes mainly checking with stakeholders to see if they have the same shared understanding. The results lead to the acceptance or decline, and possibly some improvement ideas on story level, which may become new or updated stories.

• what/release, on release burndown

We learn if the release is on the right track. This is more about project view than product review. Traditional release burndown helps us understand if the release deviates from the plan, and then we adapt by varying scope, schedule or cost. In an environment where right stories are more known, it makes sense that the learning about delivery progress becomes the focus. Otherwise, we shall go deep into further product learning.

• what/product, on story map

The learning on story level may be limited, particularly when we split stories for delivery purpose but break the natural size of uses. During initial backlog creation and backlog refinement, you may have done some story mapping and mapped separate stories into user scenarios. Sprint review is the time to revisit your story map. The granularity of discussion is around user scenarios. This reflects better about the progress, as those are more meaningful things for users, rather than separate stories. This supports better in discussing about what's still missing and what's next. This connects better with users as they are more engaging with meaning product slices, which ofter leads to more insights and product ideas.

• why/product, on impact map

The next level of learning is how much progress we made in achieving why. During initial backlog creation and backlog refinement, you may have done some impact mapping and mapped stories to the goal and measures. Sprint review is the time to revisit your impact map and assumptions. We collect data for the released stories in the past sprints. We evaluate the real impact and come up with next stories to learn or build.

Ways of product learning

Product learning happens with the right people. Besides PO and team, stakeholders such as users and marketing are invited.

I learned the below 4 quadrants of product learning based on what users say vs. do and qualitative vs. quantitative from a former Alibaba product manager Su Jie.

He puts one example method in each quadrant.

• Interview, qualitative learning, based on what users say
• Survey, quantitative learning, based on what users say
• Usability testing, qualitative learning, based on what users do
• Data analysis, quantitative learning, based on what users do

Inspired by this, we could organize product learning in sprint review with various ways.

• demo stories and user scenarios
• interview with prepared questions
• survey with prepared questions
• observe uses of stories and scenarios by users
• data analysis to understand impact
• ...

Conclusion

The number of emergent requirements from Sprint review is a reflection of the amount of product learning in Sprint review. My colleague Terry came up with Agility Index that beautifully illustrates this point.

Turn your Sprint review into a product learning session and let your valuable requirements emerge.