Contract game is the traditional way that Business and R&D work together. At the beginning of the project, Business asks R&D to commit a big batch of work and holds R&D accountable for execution. The mindset behind is "plan then execute".
Clearly this is not Agile way. One of Agile values is "Customer collaboration over Contract negotiation", which exactly addresses the dysfunction of contract game. Agile is about adaptive planning, and Scrum is about empirical process. The mindset behind is "guided by goal, inspect & adapt".
- Is Release planning predicative or adaptive?
Release planning can be a beneficial practice, as it creates the initial understanding about the work. However, in practice, it is often mistaken as upfront and predictive planning. It is even used to provide the input for contract game. Once you see that work is split into every sprint in the release with great details, the focus is primarily on contract negotiation. With the mindset of contract game, you may see that business people only join Release planning, but not the inspection and adaptation on sprint basis.
So, in Scrum, Release planning creates the initial product backlog, and you may regard that as the first backlog refinement, which is from none to the initial. Then, every backlog refinement updates the release plan and new requirements emerge, thus, Release planning is continuous.
Backlog Refinement = Continuous Release Planning
The key to break contract game in this level is that Business people and PO are working with R&D on sprint basis, and making decisions regarding ROI every sprint.
- How about Sprint planning?
If more people have realized that the release should be managed empirically, the sprint is still largely managed predictively - we plan in Sprint planning, then, we execute the sprint. This is a misconception, because if it is a predictive process, then, what is the point of having Daily Scrum, whose purpose is to manage it empirically with daily inspection and adaption? With the mindset of contract game, you may see that the sprint commitment is regarded as certainty, and PO disappears after Sprint planning and only sees team again in the Sprint review.
So, in Scrum, Sprint planning creates the initial sprint plan (aka. sprint backlog), and we don't sign up for all tasks in Sprint planning, only those you start doing next day. Then, every Daily Scrum updates the sprint plan and new tasks emerge, thus, Sprint planning is continuous too.
Daily Scrum = Continuous Sprint Planning
The key to break contract game in this level is that PO collaborates with team in the sprint through just-in-time review, defining sprint goal, re-negotiating the scope when necessary, occasionally joining Daily Scrum, and just talking to each other.
While PO manages release and team manages sprint to large extent, contract game is harmful at both levels. Are you in the mode of contract game?
Specialization vs. Responsibility
Specialization is about being good at something, while responsibility is about having a duty to deal with something. While they are related, it may not be good idea to couple them together.
Component specialization and Feature responsibility
Conway's law states, "organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations"
Once you setup team with certain component responsibility, they specialize on the component naturally. Then, this becomes the source of inflexibility, when specialization is used to justify tying those teams with those components. Evolving architecture becomes harder when it is coupled with organizational structure.
The solution is feature teams with collective code ownership. The responsibility is on feature, while component responsibility is shared, do we still have component specialization? Most likely yes, at least at individual level. Some people are more knowledgable thus better at doing certain component work. Generalizing specialists starts with some speciality, then expands to other specialities. For example, you are the main developer on component A, and you expand to work on component B and develop specialty there over time. This may apply to team level too, since some features may touch certain part of system and the related components more often than other features. Specialization just happens. Since the responsibility is on feature, thus, the component specialization does not limit us from picking the most valuable feature items.
In short, have feature responsibility, and let feature guide component specialization.
Feature specialization and Product responsibility
The same dynamic happens with feature specialization. When the team has feature responsibility, e.g. it is a "Payment" team, the team specializes on the relevant domain. Then, this becomes the source of inflexibility, when specialization is used to justify tying those teams with those features. Evolving product becomes harder when it is coupled with organizational structure.
The solution is feature teams with common product backlog. Those teams have product responsibility, and they don't have special feature area responsibility such as "Payment". Do they still have feature specialization? Most likely yes. Given the team developed features in this area, when another high-value feature comes to same area, the team likely picks this one. Over time, they specialize on this feature area. This is even a good thing since it helps develop the deep knowledge and skills, but we don't want to label them as "payment" team and define their responsibility around this feature area. When that happens, we start to select features because we have suitable team available, rather than customer value driven. Therefore, every team is product team without having name tied to feature area. Instead of having "Payment" team, we have "Gryffindor" team. For specialization, we may even want to track and take advantage of it when possible.
In short, have product responsibility, and let product guide feature specialization.
If you work on large-scale development and adopt LeSS, you should have heard of Requirement area. Do we want to introduce areas with clear responsibility on product domains, or areas simply as groups of teams and let specialization on product domains happen and evolve? There is no simple answer. The bottom line is, Requirement area is dynamic. When you give Requirement area clear responsibility and associate it with meaningful area name (e.g. Security), it may help build the identity thus accelerate the specialization. On the other hand, this may also lead to Requirement areas standing still forever.
Specialization and Responsibility are two different things, and we shall not confuse them. Specialization happens, and you may want to track and take advantage of it, while narrow responsibility, defined around specialization and often labelled in name, decreases flexibility and leads to local optimization.
Back to fixed scope
I'd like to revisit the rationale behind moving fixed scope to fixed time in Agile development. By understanding what is essential, we may get back to the thinking of fixed scope.
Fixed scope in traditional development
In traditional development, we often start by fixing the scope (of the release), then work on how much time and how many people we need. The number of people is the main cost driver in software product development.
Fixed time in Agile development
In Agile development, we often start with fixed time and fixed cost, then work on how much scope we can deliver within those constraints. Fixed time is implemented as iteration and is also called timebox. When the team is stable, we have pretty much fixed cost. Release consists of multiple iterations, and the number of iterations may or may not be fixed.
The rationale behind moving from fixed scope to fixed time:
- Scope often has the most flexibility, particularly when you look into details. For complex product development, we learn the right scope over time, while fixed scope reduces flexibility and makes it difficult to respond to change.
- Increasing the number of people, although increasing cost, may not increase the speed. This is best illustrated by Brooks's law - adding manpower to a late software project makes it later.
- Time has less flexibility due to the growing need for short time to market and even occasions when the time delay is impossible (e.g. Christmas). Timebox helps prioritize and focus, as well as build development discipline.
Back to fixed scope
If you look at the rationale, it assumes that the fixed scope is big. When it is small and minimum, the problems with fixed scope disappear. Therefore, the key problem is big fixed scope. Timebox is one approach to reduce fixed scope. Another approach is to limit WIP directly as done in Kanban. Limiting WIP helps prioritize and focus too. The remaining advantage from timebox may be the support for building development discipline.
With further scope optimization, our focus moves towards identifying the meaningful minimum. It is MMF (Minimum Marketable Feature). In terms of story mapping, it is the minimum slice rather than a single story. The time to deliver MMF is not fixed, but usually short due to the minimum scope. Once we identify MMF, we develop it and release it, with the discipline of continuous delivery. We are back to the thinking of fixed scope, but small fixed scope.
The thinking of fixed scope is not the problem. The problem is that fixed scope is too big. We solve it by reducing it. We may apply timebox, which is the way behind moving fixed scope to fixed time. We may also limit WIP directly, which is the Kanban way. Eventually, if we identify one MMF each time and make continuous delivery, we are back to fixed scope, but very small.
Tighten or loosen roles?
In my recent CSPO course, I got a question about when we should focus on the whole team (team, PO and SM), and when we should highlight different roles. Some people are somewhat confused because on one hand we define different roles and responsibilities, on the other hand we talk about the whole team. How strictly should those roles be defined and responsibilities be respected? Should we tighten or loosen roles?
MOI and Agile value
That question made me think and reminded me of MOI (Motivation, Organization, Information) mode from Jerry Weinberg. In MOI, neither too little nor too much leads to effectiveness. The most effectiveness is achieved when you strike a balance. This applies onto Motivation, Organization and Information, and organization is the dimension related to the question.
It implies that it depends on your context. You observe what makes your situation less effective, is it due to too loose or too tight organization? Roles belong to organization. Your effectiveness is low. When it is caused by lack of organization, you increase it by for example highlighting roles. When it is caused by too much organization, you decrease it by for example focusing on the whole team.
The first Agile value is "Individuals and interactions over processes and tools", roles are part of processes. If what you do depends on the situation, does it conflict with that we value more on Individuals and interactions, which implies loosening roles? No, it does not. In general, we lean towards loosening roles and favor individuals and interactions, while in your specific situation, it is still possible that you lean too far away from organization, thus you actually benefit from having a bit more processes by for example tightening roles a bit.
How do we tighten or loosen roles? CDE (Container, Difference, Exchange) from Glenda Eoyang provides useful insights, in particular, we influence the organization by changing container. Expanding container leads to more loosen roles and more room for self-organization. This includes practices such as collective code ownership, PO and team collaborating on requirement clarification, creating the whole product team, etc.
LeSS is natural
I realize that LeSS is a natural way to scale. And let me illustrate.
LeSS is still Scrum
LeSS provides two different large-scale frameworks: LeSS for 2-8 teams and LeSS huge for 8+ teams. We focus on LeSS here. LeSS is trying to reach the same purpose as one-team Scrum while staying within the constraints of the standard Scrum rules. So, let's examine what's essential for one-team Scrum.
One summary could be "deliver value on sprint basis". This is achieved by one Product Backlog (PB) by one Product Owner (PO), representing value; and one Team self-organizing to deliver at the end of each Sprint, getting Done and leading to one Potentially Shippable Product Increment (PSPI).
That basically has:
- One PO and one PB
- One Done and one PSPI
- One Sprint
- One Team
What's the challenge when scaling? Team size is too big. We need to split them into multiple teams. Think about 20 people, we may split them into 3 teams. But meanwhile keep the rest as intact as possible.
That leads to:
- One PO and one PB
- One Done and one PSPI
- One Sprint
- Multiple Teams
The design goal for team structure is to enable any Team (after splitting) take items from one PB, get to common Done and become part of common PSPI (integrated with other items done by other Teams). Feature team (the majority of the teams) is thus a natural choice.
This is LeSS way, and it is natural to scale this way. More reference here.
Scale Scrum when we grow
In a recent scaling workshop, an interesting comment was raised. "We pretty much did LeSS even without knowing it." I dug deeper on how it evolved. It turned out that they started with one-team Scrum, but grew to the size that is too big for one-team any more. They tried to introduce minimal change by splitting the big team into several small parallel teams. By "parallel", i mean that they could work on any item in the common backlog and didn't create specialization area to constraint themselves. This worked well for them.
This reminded me of my previous experience. Back to 2005-2006, in my first Scrum project, we started from one team Scrum. After 6 months, it grew into 3 teams. We kept one PO and one Product Backlog. We shared the Sprint rhythm, defined common Done, had joint sprint planning and sprint review to keep the whole product focus. This is in line with LeSS. Again in 2008-2009, we had one team in my department, growing into 3 teams. It used similar approach leading to similar LeSS structure.
If you start with one team Scrum, it is rather natural to scale to LeSS. Moreover, regardless of how big the product eventually becomes, it is almost always wise to start from one team. Therefore, scale to LeSS, not start from LeSS.
How Scrum team benefits from Kanban practices
While discussing some struggles of Scrum team with my friend He Mian during his Kanban course recently, i realize that Scrum team can benefit from many Kanban practices.
There are 6 core practices defined in Kanban method. They are:
- Limit Work-in-progress
- Manage Flow
- Make Policies Explicit
- Implement Feedback Loops
- Improve Collaboratively, Evolve Experimentally
Let's look through them to see how Scrum team or any team doing iterations can benefit from those Kanban practices.
Scrum team usually adopts some form of task boards to help coordinate their work in sprint. The key is to effectively inspect where we are so as to adapt accordingly. Visualization in Kanban goes deeper than usual task board. When the cycle time of your stories is still long (e.g. a week or more), the additional details expose problems earlier and help us adapt faster. Kanban uses more elements when visualizing, such as area, color, shape, number, etc. For example, the impediment could be visualized as attached note with different color on the story. This brings everybody's attention immediately.
Scrum limits WIP indirectly by iteration. It's recommended to work on one story at a time, however, it may not be viable with small size story (e.g. 2-3 days, 2-3 people) and big size team (e.g. 7-9 people). It is good practice to limit WIP further inside sprint, and visualize that by limiting the number of lanes for example. One argument against doing that may be the risks involved in not starting the work, considering that the goal is to complete them all by the end of the sprint. We shall take risk factors into account when prioritizing stories. By limiting WIP, we actually improve the chance of completing them all by the end of sprint.
When managing the progress, reminder of sprint goal is a good step forward, while focusing on flow provides more clear guidance. Anything that prevents flow becomes impediment.
Bottleneck is one common reason that prevents flow. In Kanban, a few ways are suggested to address those both in the short term and in the long term. For example, when testing becomes bottleneck, you may first consider removing any non-bottleneck work from those people who are testing; then, you may consider improving quality of the work flowing there by doing more developer tests; then, you may consider having developers help testing. Scrum team benefits from ideas of flow management.
By measuring cycle time - the time you start working on a story till the time you get the story done, you get insights from control chart and distribution chart on how to improve the flow in the long term.
The focus on managing flow is also reflected in the way that daily standup is done in Kanban. We walk through the board from right to left, story by story. This helps making more effective inspection and adaptation.
The Definition of Done (DoD) is one of the most important policies in Scrum. The Definition of Ready (DoR) gets popular in Scrum community, which is another policy. BTW, in practice those may lead to the wrong focus on handover from one group to another, but those could be and are achieved collaboratively in many contexts.
Many Scrum teams create and evolve their working agreement, which forms a dynamic set of processes and policies. Kanban points out more opportunities to make policies explicit, and they become areas for improvement. This fits well with the inspection and adaptation on process. Only when you understand how you do things now, can you improve further. The improvement involves the update of policies, which becomes the baseline of next improvement. Scrum teams benefit from treating working agreement as the carrier of continuous improvement. And the working agreement is visualized in the board.
There are 3 practices defined as feedback loops - daily standup, improvement kata and operational review. We have talked about daily standup in managing flow, and let's look at the other two.
Improvement kata is the daily improvement activity. I have been promoting the paring of manager and ScrumMaster on process improvement. Usually, they work together on removing impediments. The impediments may come from daily scrum or retrospective. Kanban measures flow, and provides the feedback with data. The data is systematic, and supplements well for the impediments. Operational review acts as feedback loop in large scale. The key again lies at the analysis of data.
In Scrum, two levels of feedback are built-in, daily and sprint. Daily feedback is provided through status sharing in daily scrum and sprint burndown. Sprint feedback is provided through conversation in sprint review and release burndown, as well as sprint retrospective. Comparing to metrics defined in Kanban, such as cycle time control chart and distribution chart, CFD, etc., those feedback in Scrum is more subjective. Scrum team benefits from integrating more data in the feedback loop. In particular, we shall consider gathering objective data in sprint retrospective to achieve more balanced view and better insights.
- Improve Collaboratively, Evolve Experimentally
The improvement practice in Kanban emphasizes using models and the scientific method. The requires more rigidity in our improvement activities. While it is one direction to improve retrospective through more effective facilitation so as to increase team engagement and promote ownership, it is another direction to improve retrospective through more rigid analysis and followup. I have seen teams applying PDCA in their retrospective and reaping solid benefits.
Even though flow and iteration are different concepts and practices, when we dive deep, we find that Scrum teams benefit from doing some Kanban practices.
- use visualization to help inspection
- limit WIP for flow
- use flow to guide adaptation
- improve on DoD, DoR and working agreement
- create feedback from data
- retrospective with data
Examples, Acceptance Criteria and Acceptance Tests
Acceptance criteria vs. Acceptance tests
Recently, i had an interesting discussion with my good friend Xu Yi. We discovered an interesting difference in our way of thinking about acceptance criteria and acceptance tests.
In his thinking, acceptance criteria and acceptance tests have one-to-many relationship; while in my thinking, acceptance criteria and acceptance tests have many-to-many relationship.
Let me illustrate with a story about "cancel reservation".
Acceptance criteria could be:
- cancel 1-day before travel begins
- charge 10% for normal user, while no charge for vip user
- email notice about success or failure
His list of Acceptance tests is something like this:
- fail when user tries to cancel on the same day as travel begins
- succeed when user tries to cancel 2-day before travel begins
- normal user is charged with 10% for successful cancellation
- vip user is not charged for successful cancellation
- email with success notice is sent for successful cancellation
- email with failure notice is sent for failed cancellation
My list of Acceptance tests is something like this:
- a normal user cancels a reservation 2-day before travel begins, succeeds with 10% charge, and success notice is sent
- a vip user cancels a reservation 2-day before travel begins, succeeds with no charge, and success notice is sent
- any user cancels a reservation on the same day as travel begins, fails and failure notice is sent
[note] the above tests are still not concrete enough, but this is not particularly relevant for this discussion.
He likes his flavor because that makes each test very focused, if it fails, it fails exactly one thing. I like my flavor because that makes each test a user task, it creates more understandable specification.
Then, the discussion went to what examples are, are they acceptance tests, acceptance criteria or something else? Examples are concrete, while acceptance criteria are more abstract rules. We tend to think that examples are not the same as acceptance criteria, while both examples and acceptance tests are concrete, thus they are more similar. Are they the same? If we ask users to give an example, they would usually not go to the granularity in Xu Yi's list, while possibly similar to the granularity in my list. From here, i realize that while we talk about examples, there are two different roles they play.
Examples as discovery vehicle
Examples are powerful vehicle for the discovery. In big discovery to creating stories, you tell customer journeys, while in small discovery to eliciting acceptance criteria, you tell stories. They both are examples. Starting from some examples, we apply heuristics or play "what about" to get variations and alternatives, then rules emerge, then more examples. We switch between abstract thing and concrete thing, and iterate for discovery. Examples in this context are not specification.
Examples as Acceptance tests
Eventually, examples are refined into specification, and they are also called Acceptance tests. There is some tradeoff during refinement. It is more understandable from business and user perspective once the granularity matches their natural size. It is easier to debug from development and testing perspective with smaller granularity. It is also possible to supplement very small thus very focused examples, as illustrated in Xu Yi's list, with more integrated big-picture examples.
In short, here's my current way of thinking about examples, acceptance criteria and acceptance tests. We start discovery from rough or un-refined concrete examples, derive abstract acceptance criteria from those, then, refine examples into acceptance tests, which are still concrete but refined ones.
The future of Project managers
As to this topic, I am only talking about project managers in software industry. Same thoughts may not apply in other industries.
Traditional project managers
There is traditionally contract game between business (or product or customer) and R&D (or IT or technology or vendor). In the beginning of the project, business and R&D negotiate a contract, then it is handed over to R&D, and R&D is held accountable for the delivery.
Project manager is usually located in R&D side, and he/she is responsible for the successful delivery of the contract. The success is usually defined as on time, scope and budget. The focus is on the output - the delivered features, rather than the outcome - the delivered value. Even today if you look at chaos report, it keeps the same assumption. Project is still considered successful as long as it is well delivered with those measures, even when nobody uses it after it is delivered. Therefore, traditional project managers are delivery focused.
Change from Agile
In Agile, business and R&D work together to optimize value. There is (or should be) no such thing called delivery success, but business success. Agile, Scrum in particular, holds business side accountable for the project success, which is measured by delivered value and ROI. Self-organizing team is focused on delivery and business collaborates with them to maximize value on sprint basis. With this setting, it makes no sense and is actually harmful to keep delivery-focused project managers.
The future of Project managers
While discussing with a few thoughtful PMO heads about the future of their team, two possibilities emerged.
Some project managers developed strong domain knowledge and network with business side after working in the industry for 10-20 years. If they can abandon traditional project management thinking (e.g. fixed scope, push team on short-term delivery, manage tasks for team) and adopt Agile thinking (e.g. value driven, support team to build high performance, enable team self-organization), they become good candidates for PO. From what i have experienced, the lack of good POs is one big constraint to adopt Scrum effectively. It is certainly valuable if some project managers could transform to good POs.
Traditional project manager has little overlapping responsibility as ScrumMaster, and their qualification is also quite different. It is actually misleading to call ScrumMaster as Agile project manager. However, some project managers are indeed very good at working with group. If they are open to practice Agile and gain experience, they become good candidates for ScrumMaster and even Agile coach. Remember that great ScrumMasters work beyond team coaching, they coach Product Owners and organizations too. Project managers often developed broad view, which helps them on coaching beyond teams.
I foresee that traditional delivery-focused project managers will become less and less relevant in the Agile world. On the other hand, i believe that great people can adapt well and sustain their value in a different way.
Empirical process control in Scrum
Scrum framework is based on empirical process control, i.e. you inspect and adapt to achieve the goal. In my CSM course, we do an exercise to relate empirical process control with events defined in Scrum framework. The below table is one typical outcome.
Information for Inspection
Example about Adaptation
- Why have this sprint?
- Story/task status
- Sprint burndown
- Add/remove/update tasks
- Change daily plan to work on different tasks or solve impediments
- Renegotiate on stories
- Sprint abnormal termination
Product goal (1)
- Release goal
- Product vision and roadmap
- Product increment
- Product backlog
- Release burndown
- Add/remove/update story in backlog
- Change priority
- Next sprint goal
- Update release plan
- Cancel the project
Process goal (2)
- What happened in the last sprint?
- Any data (sprint backlog, sprint burrndown, bugs, etc.)?
- What worked well?
- What can be improved?
- Improvement actions
- Update working agreement
- Expand DoD
- Try pair programming
- Setup CI server
- Product goal. If your release after multiple sprints, for each sprint, release goal guides your inspection and adaptation in each sprint. However, in some domains where you release in every sprint, or even daily, product vision and roadmap guides your sprint-by-sprint inspection and adaptation.
- Process goal. You may define your improvement vision for the next period (multiple sprints) by team envisioning workshop, assessment such as health check.
Explore PO team
PO (Product Owner) is defined as single person in Scrum. However, for various reasons, in practice a team may be formed to fulfill the role of PO. I am using the term of team rather freely, strictly speaking, some of them are only working groups. I'd like to explore different types of PO teams, and hopefully generate insights for you to use while designing your PO team.
Types of PO teams
Here's the main types of PO teams I have encountered.
This is probably not considered as a team, since it is normal that PO as a single person needs to collaborate with others such as domain experts, business stakeholders, and the development team. There is certainly team work here, however, to large extent, it is close to the surgical team described in "The Mythical Man-Month" book by Fred Brooks. I am now talking about the PO work - define the right product, not the whole product development.
In large-scale Scrum context, PO together with APOs (see details from LeSS introduction) form a PO team. This is similar to most management team. Each APO is accountable for its own area, meanwhile serving as team member in PO team. The PO is the leader of PO team.
In dual-track Scrum, product discovery is done by a team collaboratively - "the product manager, designer and lead engineer are working together, side-by-side, to create and validate backlog items." It somewhat implies that product manager (or PO in Scrum) still leads the team, but the members work more interdependently as peers.
Value team, introduced by Ahmed Sidky in his Agile 2014 session, in many ways is similar to product discovery team. The composition of the team has a bit different focus. Value team seems accommodating more traditional roles, while i prefer the product focus in product discovery team. However, I view the emphasis of facilitator role in value team as an important addition. It highlights 3 important aspects regarding value team roles - ownership of vision, facilitation of team, accountability for results. I see that two of them (ownership of vision and accountability for results) belong to PO, while SM should facilitate value team (Ahmed has a different view on this). In this understanding, PO seems still taking somewhat leader role in value team.
New type of PO team
If we view discovery and delivery as two sides of product development, and now look at product discovery team and product delivery team, we may wonder why we define a leader for product discovery team, while we don't for product delivery team. If it is possible to have no leader in self-organizing delivery team (aka development team in Scrum), why don't we abandon the idea of defining one leader (i.e. PO) in discovery team? Instead, we create a self-organizing PO team without one defined leader. SM has the responsibility to coach PO in Scrum, while in this setup, SM facilitates and coaches the PO team (aka product discovery team, or value team), in addition to the development team.
Would the new type of PO team work well? This could certainly be controversial. I am listing a few points to trigger the debate.
- having one PO helps more efficient decision making, but efficient decision making is possible with self-organizing development team.
- product discovery is more divergent than product delivery, but there could be much divergence in finding architecture solutions too.
- there is often one leader behind successful products, but successful products where a group of people stand behind exist too.
- the cases of having group behind successful products are rarer and more accidental, but is it because there has been lack of facilitation and coaching in making the group work effectively?
- we need one person to interface with team, but why is it possible for PO to interface with development team (without leader), while not for development team to work with PO team (without PO as leader)?
If you experiment this type of PO team, I appreciate that you share back the experience.