November - 11th agile principle

2019-11-13

The best architectures, requirements, and designs emerge from self-organising teams.

The eleventh principle of the agile manifesto is a statement about organisation. The wording is slightly problematic, at least in my book, because on one hand it expresses a truth, on the other hand it fails to mention an important detail, namely that self-organising doesn’t imply self-managing. Let me explain. What is a self-organising team? It’s a group of people expected to achieve defined goals in a self-determined manner. To that end, it is given a high degree of autonomy. The self-organising team does not entirely play by its own rules, however, many important decisions are left to it, especially decisions about how to structure and organise work, how to collaborate, how to communicate, which tools to use, etc. In short, team members are not told “what to do”, but “what is expected” of them. Nowadays, this is commonplace of course, as authoritarian management style seems to be a fading memory of the 20th century. But does this hold true for all cultures around the globe? Can we expect any group of people to self-organise? What is the role of the manager in such a team? How does team accountability versus individual accountability play out?

workplace

These questions have been mulled over extensively in the management literature of the past decades. Today, the idea of agile organisations is discussed by the likes of McKinsey. Before I suggest possible answers, let me state that I believe self-organisation is a perfect fit for agile teams and software development in general. I just don’t believe that successful teams happen by chance. More to the point, I don’t believe self-organising teams are able realise their potential without careful intervention. To this effect, favourable conditions are required to achieve the best possible results. But what do careful intervention and favourable conditions exactly mean? Let me begin with the basics. A favourable condition would be colocation, for example, simply because working together is easier when people are in one place. An unfavourable condition would be people working remotely in different time zones. Another unfavourable condition would be mixing people from different cultures and backgrounds. A favourable condition, on the other hand, would be having experienced engineers and seniors on the team. Team members with good communication skills and experience in agile methods are also a big advantage. The biggest asset is probably motivation, since a highly motivated team can really move mountains.

Yet in reality, things are not always perfect. We may have to begin with less than ideal conditions. Let’s say we can’t find sufficiently skilled talent locally to staff an entire team. Should we go with hiring juniors only, or should we work with remotely located senior engineers? It will probably be a compromise, but since a team without any senior staff is likely to be more problematic than a team with some remote members, the second option seems preferable. Or let’s say we have “inherited” an offshore team six hours east of us from a corporate merger. It is definitely going to be a challenge. The time offset is one thing to deal with, but it is not our biggest problem. The biggest problem is getting to know our team, acquiring an insight into their existing work procedures, organisational culture and national culture. The latter is often completely underestimated. Having worked more than 20 years in Asia myself, I can say, that there are large differences in national cultures which affect how people solve problems and communicate. These differences need to be well understood by “culturally sensitive” managers in order to make cross-cultural teams work and communicate efficiently.

But even if we encounter really good conditions, say a well-established colocated team with senior engineers, it doesn’t follow that successful self-organisation automatically happens. There could be many reasons for that. For example, if the team comes from a traditional top-down organisation where employees had to report directly to their superiors, they will simply be overburdened by the mandate of self-organisation. If the team members are introverted, prefer to work alone, dislike meetings, and have diverging interests, self-organisation won’t come natural either. If a team has two or more dominant members competing with each other while trying to get the less dominant members on their side, things may get out of control. If the team has one dominant senior member, but that member is not aligned with the company’s goals, the outcome is likewise questionable. Group dynamics, organisational structure, corporate culture, individual interests, and member characteristics all have to be taken into account. This opens up a new role of leadership, which unlike traditional leadership does not rest on formal authority. Agile leaders are coaches, not bosses. Agile leaders use careful intervention to enable the team to work to collaborate more efficiently.

Careful intervention -in a nutshell- means providing guidance and support, removing obstacles and distractions, helping the team to reach consensus and to stay focused. If this sounds awfully close to the description of the role of a Scrum Master, it’s no coincidence. The Scrum Master is the textbook leadership role in Scrum. So, strictly speaking, there is no place for traditional managers in an agile team. Many activities covered by traditional project management, such as planning, initiation, and controlling have shifted to the team. That’s why it is called self-organising after all. There are some exceptions, of course, which fall into the area of responsibility of the Product Owner. These mainly revolve around the backlog, which is a written record of tasks and requirements that comprises a road map for product development. In a manner of speaking, the Product Owner is the person who tells the team what is expected of them. This is also the main reason why the Product Owner should never be the same person as the Scrum Master. If we merge these two roles into one, we compromise the idea of a self-organising team and we inevitably move back to the traditional project management role, where one person tells the team what is expected and also takes over planning, scheduling, monitoring and controlling functions.

I am not saying that this is necessarily a bad thing. In some situations, where there is little team coherence, for example, it could actually be beneficial. But lacking team coherence should always be considered a temporary condition. In a well-oiled team, concentrating all these functions in one person is most likely detrimental. After all, there is a sound reason for self-organisation. The team just knows better. It knows how to get its work done and how to accomplish its goals most efficiently. If this should appear contradictory to what I said about careful intervention, the apparent contradiction can be resolved easily. Classical theories of economics understand people as motivated by self-interest. They must therefore be guided and rewarded to cooperate successfully. However, this idea has been shown to be too narrow. The current understanding is that people are motivated by many other things in addition to economic gain. Self-determination is motivating, for example. Team members giving each other credit and trust is likewise motivating.

But there is another reason why Product Owner and Scrum Master should not be the same person. The Scrum Master represents the interests of the team. The Scrum Master attends to the ideas and concerns of the individual members. The Product Owner, on the other hand, represents the interests of business and is tasked with creating maximum value from product development. These interests cannot always be reconciled easily. Sometimes, they even contradict each other. For instance, the Sprint scope is a common subject of discussion in Scrum teams. Obviously, business wants as many features as possible to be released at the end of each iteration. Developers, on the other hand, are interested in maintaining a sustainable pace and a sound architecture. The QA team is interested in maintaining testability and high code quality. These different goals have to be balanced. Since the Product Owner doesn’t contribute to the code, tests, or technical planning, he might lack knowledge of the technical details of the project. Therefore, decisions about scope and work packages need to be made together with the team. This is generally easier if the team is represented by a Scrum Master.

A self-organising software development team is entrusted with certain decisions, which typically include the following things:

  • The number of tasks that are being tackled in each iteration.
  • The technical implementation plan for each task.
  • The technical design of each feature as well as the overall architecture.
  • The languages, libraries and frameworks to be used.
  • The complexity estimation for each task and/or feature.
  • The technical documentation.
  • The work procedures, for example task state transitions, tests, deployment automation, etc.
  • The ground rules for coding style, programming conventions, code reviews, testing methods, etc.

The Product Owner typically makes decisions about these things:

  • What to put into the backlog, the sprint planning and refinement sessions
  • The priority of tasks
  • The functional and non-functional specifications
  • How user stories are elaborated

The developers and the Product Owner often (but not always) make joint decisions about these things:

  • The UI and UX design
  • The release and deployment plan
  • Customer support and hotfixes
  • Operation guidelines
  • User documentation

Lastly, some decisions are made jointly by individual developers and the team:

  • Who is working on which task (developers often pick tasks)
  • The software tools that are being used
  • How to work on tasks (pair programming, mob programming, etc.)

Why does this recipe work? Developers use their practical knowledge to plan and design the architecture that they then proceed to build. There is no external architect that provides blue-prints but is not involved in the coding. The success of this method rests on the experience of the team members and their ability to communicate. It works because of short feedback loops. If there is a problem anywhere in the technical planning, it can be corrected quickly without consulting an external authority. If a member gets stuck in the implementation process, other team members help out. If a member deviates from agreed practices and procedures, the review process ensures that this is also corrected. Agile values, provided that team members have internalised them, allow for creativity, communication, and growth. To make this happen, it is important to understand that self-organised doesn’t mean self-managed. Agile teams still need leadership.