Adopting Agile sounds as simple as taking the SCRUM master certification tests and pushing them down the framework to a team of developers. Usually, we find that the challenge of becoming agile is way larger than most companies like to admit.
I personally lived this experience myself when working for a large enterprise that decided to move away from its old “waterfall” practices to adopt new and “more agile” processes. While the intention at the executive level was clear, they themselves did not really understand why the change was important. It was more a requirement to use the fancy word agile, than a real transformation program. Therefore, we went to sleep on a Friday and next Monday we were now an “Agile” team.
What the middle management teams did to adopt such change was not an executive’s concern. The task had been set, and we were expected to become an Agile organization in no time. So we reverted to making some changes in the process, the development teams adopted SCRUM-like sprints, and the reporting cycle changed… other than that, not much really changed. The results were less than impressive, to say the least.
Not As Easy As 1, 2 , 3 … SCRUM
Years later, as a consultant at Modus Create, I started noticing how frequently companies of all sizes were facing similar challenges, even in 2020 when agile is supposed to be the norm. We see organizations that make it a goal to adopt Agile practices, but either oversimplify what it means and do not manage the transformation at all levels or get caught in a web of processes, tools and miscommunications that end up slowing things up. If the intention of the executives was to push forward changes to make things move faster and increase throughput, they end up getting just the opposite: a prevailing perception in their product teams and business stakeholders that the development teams are stuck and that priorities are not managed. Suddenly, the backlogs start to grow, the development team loses focus on what is important and the releases get pushed out, solidifying the perception that the development team is not making good progress.
Situations like this are not uncommon. Our Agile and Atlassian experts have frequently found similar situations that make us already expect some common potential causes. Most organizations that struggle with adopting agile are very conscious of their own challenges but struggle to identify the underlying causes of the problem. If you are now part of a development team and expected to deliver software in an agile manner, these are 3 of the most common symptoms of bad agile practices:
- Teams do not clearly understand their roles, the boundaries of their products and the priorities for the business. There is misalignment and no clear prioritization between distributed teams; priorities are unclear or change often and responsibilities dilute, while technical and non-technical teams struggle to prioritize, coordinate and define common goals. For example, Product Owners may be expecting all of their requests to be priority 1, while the actual development team is mostly busy fixing the infrastructure in which these and other products run.
- Methodologies and processes that are defined as Agile do not really follow an industry best practice, mainly as a result of a misunderstanding of what Agile means, how to operate it and put it into action. The result is the teams interpret the Agile methods as the culprit, disregard process discipline and reduce the actual visibility of their daily progress to senior management. We frequently see cases where companies schedule all ceremonies, but the actual meetings do not adhere to the goals and time allocations recommended, after a few of these the value attenuates until eventually crucial team members stop caring or attendending.
- Very frequently the timely delivery of quality software misses deadlines or the quality of releases impacts service performance. Generally, the teams are so blocked by process gridlock that their ability to implement technical automation, process changes and other technical best practices affects the team’s throughput by forcing too many manual processes on the shoulders of the developers. The power of habit in an entangled team forces them to keep trying ineffective manual processes, never coming out for air to apply QA automation, CI and automated code deployment, or even checking their own velocity, performance and quality with some healthy retrospectives.
The underlying causes of these issues manifest in many other smaller symptoms. Teams tend to struggle with configuring the agile project management tools they use and to report on progress in an effective way. Finger-pointing between product owners and developers indicates misaligned expectations, and many other smaller issues can indicate that the strings that unify a team are in tension, at great danger of snapping and hurting those that are pulling them in opposite directions.
Start With Cognitive Load and Software Boundaries
How to solve this is a million dollar question. Each team, each company and each project needs to be assessed carefully to uncover the main issues affecting performance. Still, most teams can improve quickly by adopting some of the principles of the agile manifesto that have become mainstream best practices and contribute to improving the working conditions and the outcomes obtained by many organizations. Agile requires simple adoption of concepts like small, stable and self-organizing teams who collectively own the software they work on. This, with time, can result in cohesive units of workers and a collective progress that becomes many times more effective than the sum of individual efforts.
However, the separation between the technical architecture of the software and organizational architecture of the teams who develop it, own it and manage it, remains a largely unexplored discipline in most organizations.
The tendency to hastily assemble a team of developers to resolve a throughput problem unfairly assumes that the team will have the empowerment and leadership support to make all the changes necessary to actually resolve it. Many times those changes require reconsidering the software architecture, its boundaries and ownerships, and the organizational structure and roles before just assuming that agile development will “come to the rescue”. As recognized by Skelton and Pais in their signature book “Team Topologies”, it is important to think of the software design and its boundaries to align them to the available resources and the team’s cognitive load to ensure the throughput flows without blockers.
Cognitive load is a psychological theory that has now permeated the world of organizational design. It refers to the used amount of working memory resources and has to do with how we process information. When a team’s collective cognitive load is overloaded, the throughput to manage changes, process requirements and actually perform coding tasks is hurt, resulting in teams feeling stuck, too slow to execute the work they want to do and having to delay their releases often.
This is especially common when dealing with internally-focused development teams, working on internal tools, usually home-grown. These teams have to deal with too many knowledge domains at once, having to constantly shift between projects, tools, languages, infrastructure designs and task types. The result is that the intrinsic cognitive load (things you need to know to deliver on the task, eg: “what is the structure of a React component?” or “How were we defining variables in this Javascript app?”) and the extraneous cognitive load (things related to the environment, eg: “how do I deploy this?”, “who can answer details on this requirement?”) get in the way of developing a high performing sequence of tasks and a daily working habit that would allow development to flow. Internal teams tend to be asked to do maintenance, development and infrastructure management of a series of internally designed tools that have no clear audience, no good product backlogs or roadmaps and no funding to prioritize features. So they struggle with cognitive overload and task shifting, because the software they work on has no clear boundaries defined and the stakeholders are not clearly aligned to roles and specific tasks.
So researching and understanding early on, the complexity of the different software domains the teams need to deal with, what are the boundaries of such domains and how the architecture of the software enables clearer team responsibilities and cross-domain interactions becomes critical for the long term success of a team. Domains (or areas of the software that are distinct and specific to a functionality or service) should be classified according to their complexity, their size and team’s abilities matched with the subject area of the domains. According to the research from Skelton and Pais, teams should be able to handle two to three simple domains (where each task generally has a clear path to action). If the domains are complicated (each task requires multiple iterations and interactions until its resolved), a strong team should focus on handling only it and no other complicated domains. And last, if the domain is complex, (where the tasks require time and focus to do experimentation, discovery, research and testing) single teams dedicated to solving complex problems need to be allowed to dedicate time to focus on the problems. If these teams are given any extraneous tasks, they will tend to tackle the simpler issues and disrupt the flow needed to tackle the tougher challenges. Unless the team’s cognitive load is protected (reducing intrinsic and extraneous types of load) the team will struggle to focus and the interactions and priorities across stakeholders will pull the team in too many directions, until one or several of the strings snaps or gets dropped.
And Then What?
Once there is a clearer understanding of the domain complexity, the software architecture and its boundaries, aligning them to the team’s cognitive load reduces bottlenecks and accelerates the throughput. Of course, many other areas of action need to be addressed. The basics of good project management need to be applied with discipline and decisiveness, but with an understanding of the teams and the software they need to work on, it’s possible to start fixing other areas that are more visible to all. To mention three examples of areas of action to focus next, think of:
- Alignment: the project goals and objectives have to be well aligned across stakeholders, with proper roadmaps, backlogs, user stories and properly documented requirements and acceptance criteria. This must be a two way road: allow the technical team to understand the business use and the users while allowing all stakeholders to see the complexity and load that the teams have to deal with in order to execute.
- Environment: the tools utilized to manage the project and keep the agile throughput need to be optimized to ensure all teams work together from a prioritized backlog in an effective way and to start adopting CI and automation solutions. Unloading the team’s cognitive capacity by simplifying the backlogs, the QA tasks and the deployment tasks as much as possible.
- Teamwork: the team size, roles, interactions and collaboration should be enhanced via well-managed agile ceremonies that build trust and allow a mindset that prioritizes team outcomes vs individual goals. Teams that need to interact with each other should find familiar and standardized ways of exchanging information. When multiple teams interact they should find it easy to navigate each other’s repositories and understand the code, the versioning, common practices and each other’s backlog and priorities.
At Modus Create we have developed a series of consulting engagements that address particular areas of a software project to assess and understand the challenges our clients face when adopting agile or starting a new product. We have come to realize that becoming agile means acting in all these different areas, and short-term assessments help focus our clients efforts on one specific topic to address it effectively, before trying to “boil the ocean” .
Some companies prefer a tools-first approach, and use our Atlassian assessments to improve how the team documents and manages a project. Others need a review of their product and its goals which we address via our Modus Kickstart engagement and others need specific technical help to review the architecture or the code quality of their software (via short Code and Architecture assessments). Now we are delivering more and more team Agility Assessments, which evaluate some of the dimensions discussed in this article to uncover team challenges. The Agile Assessment combines tools, process, team and architecture reviews to understand the levels of agility of our client’s teams, and provide actionable recommendations to address their own shortcomings and start adopting agile best practices faster.
If your team is struggling while adopting agile, the first action is to consider how heavy is the complexity of the software domain on your team’s cognitive load , then improve it as much as possible allowing the team to focus their skills on the business goals. Just after that, look into the processes, tools and agile best practices. Or better yet, at Modus we’re ready to help you achieve your agility goals and grow to your next level.
