Prabhakar Karve's weblog

Next 10 years of agile

In last 11 months there is a flood of material about “Next 10 years of agile”. Do we need one more? I think I do, because a blog is like a private diary which is publicly shared. I have been closely following views expressed by the authors of the agile manifesto as well as other experts in the field. A layman like me gets overwhelmed by the variety and richness of the thoughts, and needs to digest it all to make sense of it for himself before he can use it in his work. So here goes what it all means to me.

Let me start with the unanimous agreement during the retrospective on 10 years of Agile software development at Snowbird Utah last Feb, about four key success factors for the next 10 years.

1. Demand Technical Excellence
2. Promote Individual Change and Lead Organizational Change
3. Organize Knowledge and Promote Education
4. Maximize Value Creation across the Entire Process

Let me share some exciting possibilities to work on, as I see them.

Let us start with “Maximize Value Creation across the Entire Process” and “Promote Individual Change and Lead Organizational Change”. As Jeff Sutherland said, “individuals adapting to change is not enough, organizations must be structured for agile response”. At Impetus, when we tried to extend Scrum beyond software to other areas of the organization, the immediate roadblock was the specific terms used in Scrum for different roles practices and events. For many non-engineering teams, there is no product that they directly work on and the terms like product owner or product backlog are confusing to them. However, being part of the organization they need to be agile and Scrum can be quite useful to them. So we need a more generic terminology which does not lose the original intent and at the same time makes sense to a wider audience across the organization. This provides a great opportunity to clearly & unambiguously understand the underlying concepts, which will help not only the wider audience but also help us in software development itself.

Scrum is currently used for software product development which means it is used for taking the product from its current state to a new and more desired state. In short it is about effectively managing this change. Every business organization is involved in creating & supplying products and / or services. To keep pace with market & competition, every business organization needs to keep changing its product / service offering. Similarly, every part of the organization is also playing this role for the rest of the organization. So there is no reason why Scrum can’t be useful to manage these types of changes in short increments and frequent feedback. If we use more general terms like “Change owner” and “Change backlog”, they would fit all such situations.

Scrum is useful for software product development because currently there is no way it can be completely automated and therefore people are essential. Unlike physical systems, human systems are complex and their behavior is not completely predictable. Any attempt to control such systems can lead to disappointments. Scrum relies on self-organizing capabilities of complex systems to manage this unpredictability. But there is a thin line between self-organization and anarchy. If it is not managed properly, the outcome can be worse than the days of close monitoring & control. Leaders at the team level need to work hand-in-hand to patiently bring about this transformation.

Scrum encourages being realistic while planning as well while reviewing the daily status. It encourages transparency and making the latest update visible to all concerned. This would become extremely important when Scrum is applied to the whole value chain. In the absence of latest and reliable information, the proactivity and quality of work suffers.

In the current context, technical excellence relates to large enough backlog being available in a ready state with definition of DONE clearly stated along with good practices in place to ensure proper testing within the same sprint. The emphasis on quality & excellence needs to be brought to all the practices the team adopts.

Last but not the least; right organizational structures for agile response are very important. An organization has structures of different types, like hierarchical, legal and cultural. Structures are difficult to form and even more difficult to change even after they become outdated. The concept of structured freedom from systems thinking can be effectively used for this purpose.

To summarize, there are great opportunities to take Scrum to the next level by using it not merely for software development but applying it to different parts of the organization. But to do this effectively,

• We need to understand the underlying concepts and use appropriate terms, which while not losing the original intent of the Scrum terms, makes them comfortable to use.
• We can use self-organization far more widely and effectively. But for this to happen, the leadership must understand what is involved and put right structures & mechanisms in place to make sure that self-organization does not lead to anarchy but frees the leadership from need to spend their time on close monitoring and controlling. When this happens the team leaders can spend their time helping & empowering the teams to fully tap into the immense human potential.
• We need to understand importance of being realistic and making the latest status widely visible across the whole value chain, so that different parts don’t merely act as cogs in the wheel but proactively drive communication actions & decisions both upstream & downstream.
• Both technical excellence and practice excellence are important.
• Structured freedom is a useful concept to manage structures within an organization.