Project development using agile methods is nothing new. However, it is very common to associate the idea that in agile projects there is no need to design the architecture of a system. The question that remains is: How the system's architecture design would fit into Agile development?

Many will argue that in an agile development scenario, it's not useful stopping to think about the architecture so that it will inevitably change over time. We agree with the second part of this statement, the architecture will change during the project and the longer the project time, the greater the probability of changes in the initially defined architecture.

Much of the perception against architectural planning and the understanding of this activity as something bureaucratic and often even a bottleneck in project progress has its origins in experiences with software development using predictive methods of software development.

The section below provides a comparison between predictive and adaptive methods.

Characteristics and Distinctions Between Predictive/Adaptive Methods

Predictive Methods

Every time we talk about traditional software development, we are usually talking about the predictive software development process. The most common of these methods is the famous waterfall.

In this type of process, there are clearly defined phases to:

Requirement - Understanding all system requirements. The functions it should perform, volumetric (static and dynamic), integrations, and any other functional or non-functional needs.

Project - In this stage, the system design takes place. It is here that it is defined how the system will interact with the external world, what will be its data model, technologies that will be used, its classes/programs, associations, and interactions. It is at this stage that the System Architecture is defined.

Implementation - Here the materialization of the system defined in the design phase takes place. Basically, the project is coded and unit tested.

Verification - The system is tested in an integrated manner and approved by the user.

Maintenance - After being tested, approved, and installed in production, the system enters the maintenance phase.

As we can see, in predictive methods the system solution is given right at the beginning of the process. Every system project is made based on the information of the needs perceived at its beginning.

Furthermore, the architecture is designed to meet all system requirements upfront, often without hands-on testing to validate the architectural decisions made during this phase. It is not uncommon for problems related to the design phase to arise during the implementation phase, such as incompatibilities between the selected technologies and/or failure to meet the requirements.

When this happens, it is necessary to go back to the previous phase and change the project. As the project was designed for the system as a whole, changing it is usually a high-effort task (changing all documentation to correct the design error, for example) causing an impact in terms of time and cost in the development of the system.

Another problem with this type of method is that the system requirements are not reviewed during its development. The longer the development cycle, the greater is the chance that when the system finally makes it to production, the business needs that gave rise to its construction are significantly changed.

The Figure below exemplifies this. The time aims a target (perceived needs) early on in the project. But these needs change over time and the system ends up missing the target when it is finally released to production:

Adaptive Methods

"Product development teams are facing a quiet revolution in which both engineers and managers are struggling to adjust. In industry after industry—pharmaceuticals, software, automobiles, integrated circuits—customer demands for continuous innovation and the plunging cost of experimentation are signaling a massive switch from anticipatory to adaptive styles of development."

The excerpt above is by  Jim Highsmith, published in 2009 in his book Agile Project Management. For those who don't know him, Jim Highsmith is one of the signatories of the Agile Manifesto and it was he who proposed the ASD - Adaptive Software Development in 1999. He bet on the collaboration and self-organization of teams for the development of complex systems.

The term adaptive here is used to describe that the system will be adapted to changes in its surroundings. That is: "Respond to change rather than follow a plan" as set out in the agile manifesto.

There are several methods of agile development. These methods break the development of a system/product into small increments/iterations. This break allows you to reduce the amount of planning and design required before implementation. It's like we had several waterfall cycles over time.

At the end of each cycle, a functional product is presented to the stakeholders. This allows evaluating what is being built and verifying the need to change/adapt the system's objectives/requirements in each cycle.

Each iteration involves a cross-functional team that will address planning, analysis, design, coding, unit and acceptance testing. The goal is to have a product that can be released to production at the end of each iteration. With the product being tested in production as soon as possible, flaws in the design or business premises for its construction are discovered as soon as possible. This gives us the chance to adapt the system/product and correct its course/direction.

The figure below exemplifies exactly that. At each release/iteration (grey circle), the product released to production, the perception of the needs that the system must meet is changed, the planning for the system is remade, correcting its course to reach the new target (the need perceived after the release of the previous increment).

Agile Architecture

Taking the definition of Agile Architecture present in SAFe:

"Agile Architecture is essentially a set of values, practices, and collaborations that actively support the design and evolutionary architecture of a system. This approach adopts the DevOps mindset, allowing a system's architecture to continuously evolve over time, while at the same time It supports the needs of today's users. It avoids the overhead and delays associated with the start-stop-start nature and large-scale redesign inherent in phase-gate and Big Up Front Design (BUFD) processes. 

The agile architecture supports Agile development practices through collaboration, emergent design, intentional architecture, and design simplicity. Along with Agile development practices, the Agile architecture also enables design for testability, implementation, and release. domain and decentralized innovation."

From this definition, the two very important terms emerge including, Emerging Design and Intentional Architecture.

Emergent Design is the process of analyzing and extending the architecture just enough to implement and validate the next increment in the development cycle.

Intentional Architecture is about seeing the big picture. Large corporations need to simultaneously respond to new business challenges with large-scale architectural initiatives. On large scale we can understand that to meet the business objective, multiple teams, products, and systems will be involved. In this case, Emergent Design is not enough as it is circumscribed in a single team. Without Intentional Architecture, we can have several problems such as difficulty integrating, validate and maintaining the fulfillment of non-functional system requirements, low reuse, redundancy of solutions, etc.

The intentional architecture will give the teams a common objective/destination to be reached, allowing the alignment of efforts and the parallelization of the work of independent teams. In other words, it will be the guiding track, the glue between the teams' work.

Agile Architecture is about balancing these two forces. If too much emphasis is placed on Emerging Design, we will end up with components that are difficult to integrate, poor software quality, and all the other points covered above. If you follow the opposite path, that is, if we start to exaggerate the level of detail of the future architecture components, it can become a bottleneck and drastically slow down your team development speedy.

Going deep, Agile Architecture is about designing/selecting the structural components and standards of a system in a way that will allow it to respond to changes in the perceived needs (function and non-functional requirements) on the way of its construction. 

To achieve this, avoid a big upfront design but state a clear view of your end state. The desired end state will change over time but it is ok. Secondly, try your system as soon as possible. Ensure that each release is delivering a production-ready system. It will allow you to test and check if your system is really delivering value and you will be able to identify any fail (in requirements or design) as soon as possible and make the needed changes.

Conclusion

Agile Architecture is the way to design architecture in adaptive software development methods. It allows the architecture to evolve with each system increment, avoiding BUDF - Bid Up Front Design.

References

Highsmith Jim Robert (2009-07-09T22:58:59). Agile Project Management (Agile Software Development Series). Pearson Education. Kindle Edition.