12.1 Introduction to Agile

1980s and 1990s were the years software development teams started to have serious problems with developing and finishing their software projects which were getting more complex with more computer power and digitalization of businesses and organizations^[1]. Project managers and developers faced more and more problems when they were using the traditional waterfall approach (Figure 12.2 in the section 12.1.2). The development of software necessitated more collaboration among the project manager, developers, testers and users, and constant feedback from the users. Besides, new requirements were emerging while the software was being developed, and the client and end users were not always happy with the final product when they had the chance to see it after some time passed. Therefore, new methods started to emerge among software developers who lead this type of projects. Lean methodology created by Toyota for their production system, and utilized by many manufacturers was an inspiration for many of these software developers. In addition to the lean manufacturing process, just-in-time process and total quality management movement were other influencers of agile project management^[2]. Furthermore, Kanban which is a process improvement method in manufacturing was adapted to the software development life cycle.

As indicated above, new methods began to emerge in the 1980s and mostly in the 1990s as a response to some of the challenges that developers faced when using a waterfall approach.

One of the first to become popular was Rapid Application Development (RAD). James Martin, an IT consultant, developed an approach through which he divided the process into four distinct phases, namely, (1) requirements planning phase, (2) user design phase, (3) construction phase, and (4) cutover phase^[3]. It involved creating prototypes and using them to elicit requirements, validate designs and evolve toward usable solutions (Figure 12.1). RAD, itself, led to the development of alternative approaches as well.

Figure 12.1: Rapid Application Development (RAD) Phases^[4]

Before Agile Alliance had their first meeting in 1999, there had been a diversification of agile methods which were called as lightweight methods as compared to the heavyweight waterfall models. The most popular methods were Crystal methods (1992), Dynamic Systems Development Model (DSDM) (1995), Scrum (1995), Feature Driven Development (FDD) (1997), Extreme Programming (XP) (1999), and Adaptive Software Development (ASD) (2000). Through the 1990s, a number of alternative ways to organize and structure the development of software products emerged as mentioned above, and they eventually evolved into the “agile” in 2001 with “Agile Manifesto”.

12.1.1    Agile Manifesto

At their initial meeting in 1999, the pioneers of agile methods discussed the similarity of XP with other methods and decided to meet again, with a broader range of people, to explore common ground. In 2001, a group of 17 people met at Snowbird Ski Resort in Utah to talk, ski, relax and try to find common ground between their thoughts on software development^[5]. They represented the leading thinkers from XP, Scrum, DSDM, ASD, Crystal, FDD and Pragmatic Programming, along with others who wanted a viable alternative to traditional heavyweight and documentation-driven development processes. They published the “Agile Manifesto” that embodied the values they all believed in and a set of guiding principles (Figure 12.2). Thus, the Agile Alliance was born.

Twelve Agile Principles behind the Agile Manifesto are listed below^[6]:

1. Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
2. Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.
3. Deliver working software frequently, from a couple of weeks to a couple of months, with a preference for the shorter timescale.
4. Business people and developers must work together daily throughout the project.
5. Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
6. The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
7. Working software is the primary measure of progress.
8. Agile processes promote sustainable development. The sponsors, developers and users should be able to maintain a constant pace indefinitely.
9. Continuous attention to technical excellence and good design enhances agility.
10. Simplicity – the art of maximizing the amount of work not done – is essential.
11. The best architectures, requirements and designs, emerge from self-organizing teams.
12. At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

12.1.2    Differences between Waterfall and Agile Methods

As can be seen in Figure 12.3, in the waterfall (predictive, traditional) project management approach, the process is linear and sequential. Project teams should finish one stage to proceed with the following stage. As seen in Figure 12.3, the agile (adaptive) project management approach compresses the sequential phases in small timeboxes. Therefore, agile teams work on a limited number of requirements (i.e., user stories) at a time, and they can receive frequent stakeholder feedback at the end of each timebox when they create a partial working product.

The frequent interaction with the client and stakeholders offers an environment where the frustration level of stakeholders is minimized. Nevertheless, agile methods may not be appropriate for all projects. Waterfall project approach may work better when:

• Requirements are very well understood and can be fixed before the implementation begins.
• Client’s and customers’ expectations from the product is stable and substantial changes can be avoided.
• Technology and the tools utilized are understood very well by the stakeholders, and changes that may affect the project’s progress are not expected.
• The duration of the project is relatively short.
• The uncertainty is at a low level.
• Resources are available, and easy to acquire.

Therefore, we can implement a waterfall project when we expect low uncertainty and define the project and product scope, and project constraints easily with clear and unambiguous procedures. The production domain and processes involved are usually well understood and there are typically low levels of execution uncertainty and risk^[7]. However, if we cannot define the product scope, and uncertainty level is high, an agile approach would be better to move gradually by clarifying the requirements and uncertainties with the active involvement of clients and stakeholders.

In agile projects, client and stakeholder involvement can be provided constantly since product owners are a part of the team, and the increments are reviewed by the product owner and the client frequently. In waterfall projects, the interaction with the client is deferred to the end of a phase, for instance, when the software coding and testing are complete. However, in real life projects, teams often implement a hybrid approach which is a blend of waterfall approach and some principles of agile such as client involvement and feedback.

12.1.3    An Example: Grocery LLC Decides on the Project Approach for the “Self-Checkout Stations Project”

As discussed in section 1.2.3 titled “Case Study 1.1: Characteristics of a Project Undertaken by Grocery LLC”, the grocery chain, Grocery LLC, considers establishing self-checkout station areas in all fifty branches to find a solution to long lines in front of the current checkout stations where the cashiers work. The project selection committee discussed the solution and decided to implement this project. As part of requirements elicitation, business analysis team surveyed managers and employees, and interviewed customers. Besides, the team visited the competitors’ markets and made market research to have a better understanding of possible solutions in the market. The team concluded that self-checkout has been increasingly deployed in many grocery store and pharmacy chains (e.g., Walmart, Target, CVS) across the United States, and the manufacturers and service providers are experience. Therefore, the sponsor, project manager, and relevant departments had a meeting to discuss the project management approach. They decided to implement a waterfall approach supported by continuous stakeholder interaction as is done in agile methods.

• Managers, employees, and customers conveyed their requirements. The analysis revealed that requirements are consistent with the current implementations and the research consultancy company’s report. Therefore, it was concluded that requirements are very well understood and can be fixed to a large extent before the implementation begins. However, in order to keep up with the technological advancements and to be prepared for the risks such as new waves of pandemic, it was decided to establish a continuous interaction and feedback system with the stakeholders.
• There are experienced companies which provide competitive prices to acquire the systems. They implemented numerous projects across the United States. They have comprehensive knowledge of the current and upcoming technologies.
• The duration of the project is 8 months.

12.1.4    An Example: Grocery LLC Decides on the Project Approach for the “M-Commerce Project”.

As discussed in section 1.2.4 titled “Case Study 1.2: Characteristics of Another Project Undertaken by Grocery LLC (M-Commerce Project)”, the grocery chain, Grocery LLC, considers having a better online presence by creating a mobile app and optimizing the website for mobile devices in particular due to the negative effects of pandemic on the number of in-store customers, hence significantly decreased revenue and profit. In the initiation stage, project manager prepared a report justifying the need for conducting an agile methodology, i.e., Scrum, during this project in order to accelerate the pace and achieve the outcome in a relatively shorter time, receive constant and timely feedback from the employees and customers, keep a healthy interaction with the stakeholders, and adjust the specifications to the fast and unprecedented changing technologies and socioeconomic conditions. Since it was not easy for the project manager and the team to oversee the near future in a turbulent environment (e.g., Covid cases and hospitalizations rocketing to the peak in a very short time), the sponsor and the project steering committee supported the project manager’s decision to implement an agile method. The Scrum cycles (sprints) were set as two weeks so that a working app and mobile website can be developed gradually with some of the requirements and functions at the end of each sprint, which can allow frequent end-user feedback and interaction.