Opening Remarks: This article is the synthesis of our experience as Agile coaches and data gathered from Agile practitioners in our network. We do not have the presumption to cover every aspect of Agile complexity. We are gladfully welcoming feedback to start a conversation on any aspects that might spark your interest. 

Introduction on hands-on guide to avoid pitfalls in executing an Agile project
Pitfall #1: Losing faith in the team
Pitfall #2: Confusing agile and disorganization
Pitfall #3: Hiding yourself in the basement
Pitfall #4: Underestimating planning value
Pitfall #5: Not getting out of your Agile bubble
​Pitfall #6: Being overwhelmed by technical debt

Pitfall #7: Keeping old ways to test


Fundamentally, Agile is about going quicker. Quicker interactions between Business and IT, quicker increments of your app in production, quicker feedback loops from customers. To achieve such velocity, every element of the product development lifecycle (Planning, Coding, Reviewing, Testing, Deploying and Operation) should be shortened. Most Agile projects can reduce the cycle time for the first 3 steps naturally as a consequence of new management practices: 

  • Planning – by engaging the team in good sprint planning sessions 

  • Developing – by breaking down and estimating customer needs in granular user stories 

  • Reviewing – by adapting the pace of code reviews to meet the sprint goal 


Exhibit 1: Illustration of Agile maturity levels and associated practices 

But when it comes to testing, we regularly hear feedbacks along the line of “The PO wants to test every User Stories to make sure it meets his needs. It usually takes up to 4 weeks…”; “Of course someone does regression testing manually every sprint! How else would you ensure nothing breaks up in the application?” or even “Before every demo, someone in the dev team spends an entire day making sure everything runs fine.” 

There are three main differences in the way testing should be performed in an Agile set up to see significant impact

A- Testing responsibility should be embedded within the dev team 

B- Code should be written in a way that allows automated testing, and test should be automated 

C- Types and number of tests performed should be redefined 

A- Shift responsibilities 

The pre-Agile world of Software development was inspired by other industrial best practices. Taylorism, and segregation of tasks for example were very influential when it came to dividing work between each step of the product development lifecycle. While it made some sense then – with little to no change between the moment the software was designed to the moment it was put in production – it is not relevant in a fast-paced environment. To deliver frequently, all those tasks that were previously split among different actors need to be concentrated on a few individuals, with tools that allow them to be more efficient. 

We went through two of those tools concerning testing: adapting test types and test automation. With those best practices in place, it is not necessary anymore for a third party to go through lengthy manual testing to ensure the absence of regression with new versions of the software. Neither it is necessary for a Product Owner to exhaustively check the behavior of newly developed features. 

The key to convince every member of the project to adopt this new testing methodology is to demonstrate results quickly. Here are some tricks to support this transition in testing responsibilities: 

  • Do not give up on manual testing right away. This will allow comparison between both methods to identify opportunities for improvement, and let people see the impact of automation 

  • Keep track of regression and bug metrics after implementing automated tests. Usually those two will improve in a positive way very quickly 

  • Demonstrate during sprint demos a part of the test suite to business representatives. They can be easily convinced that automatic testing allows for more test scenarios to be run in a much quicker way than manual testing 

During one of our coaching projects that had a strong technical focus (i.e. refactoring, test automation) we estimated a significant velocity increase (+66%) thanks to reduced time spent on manual testing and dealing with technical stories. The forecasted ROI of the coaching project was expected to be achieved quickly: in approximately 6 months, the added velocity would compensate time spent during the 3-month coaching period. 

B- Testable code and test automation 

The first step to fully automate your tests is to produce testable code. To do so, you should be able to both: 

  • Select a small piece of code that is testable in isolation 

  • Write test cases that follows both “good” and “bad” scenarios 

  • Writing these tests should not take longer than writing the tested code 

Therefore, you should not spend more than a few seconds understanding the internal logic of the functions mentioned previously. Its outputs should be very specific and for a precise purpose. This is the Single Responsibility Principle (SRP) (1): every module, class or function in a computer program should have responsibility over a single part of that program's functionality, which it should encapsulate. Put differently, each logical piece of the code has one reason to change. To make sure your code follows this principle, you should consider these rules of thumb: 

  • When a class has more than 3 fields, it should be split  

  • When a method has more than two conditional levels (loops included) it should be broken down 

  • When a method has more than 50 lines, we should break it down 

Otherwise, you should consider breaking down your code into smaller elements that will simplify the tests. Once this refactoring has been completed, you will be able to move to the second step of the automation process:  actually writing tests. Reaching this point, you might wonder: “I’ve cleaned my code: my functions are now trivial, they cannot break. Why would I bother writing tests? I don’t need tests.” There are three main reasons for testing anyway: 

  • Those granular functions you have just broke down are going to be highly interconnected between each other. Integration tests are a way to make sure that any refactoring on another “trivial” function has no impact on other parts of the code 

  • Tests serve as documentation of the expected behavior of a function, not only testing. Properly written tests make it easier to read your code 

  • Tests act as a trust certificate. Any outside stakeholder will have high confidence in your codebase if it has a high test coverage 

The last step to test automation is to trigger those tests on a regular basis, ideally every time any change is introduced in the codebase. We will not go into more details regarding this last step, as the ambition of this article is not to detail CI/CD automation. 

C- Types and amount of tests 

Let us do a quick comparison between the way you would perform test on the same application being developed in waterfall vs. Agile to understand why it is important to change the way you perform tests

In the first case, testing will occur you will have both (1) all your features completed and (2) pre-identified all key subsets of the features produced and (3) never seen the increments of the code while being produced. Consequently, you will need to proceed to several weeks of “acceptance testing”, composed of an extensive set of end-to-end tests on your 15 features, some integration tests on key subsets and very few unit tests. 

In the second case, testing will occur (1) during development, (2) with dependencies between several part of the application not yet identified and (3) with a close inspection of increments during sprint demos. Consequently, you will need to test small pieces of your code along the way with simple and fast unit tests to make sure you meet the acceptance criteria of your user stories, some integration tests to make sure those small pieces of code work together, and a few end-to-end tests to ensure the global logic. 


Exhibit 2: Comparison of test repartition by types between Waterfall and Agile 

“Shifting the testing pyramid” is the only way to adapt to the Agile way of delivery. To perform this shift, you can use these principles: 

  • One feature roughly translates into one end-to-end test, 2-3 integration tests 

  • One acceptance criterion should roughly translate in one unit test 

  • Unit and integration test coverage of your codebase should not decline through time 

  • End to end tests should be also used as performance tests (e.g. Load, traffic simulation), and not only as general functionality tests 

    (1) Martin, Robert C. (2003). Agile Software Development, Principles, Patterns, and Practices

    This article is the last one of a series of articles on Agile pitfalls and tips to avoid them. You will find the complete series here addressing People, Processes and Technology. 

    Are you interested to work on Agile projects and support our clients in their digital transformation? We are hiring: check our job offer!

Parutions récentes

  • Image de l'article
    Abécédaire d'Orphoz
    Oct. 26, 2021

    F comme Flexibilité


    Lire la suite
  • Image de l'article
    Who we are
    Sept. 21, 2021

    Que sont-ils devenus?

    En déjà dix ans d'existence, Orphoz a vu grandir bon nombre de talents. Nous avons décidé d'aller à leur rencontre et de les interroger sur leur nouveau métier.
    Merci à Claire, ancienne consultante chez Orphoz, qui dévoile les coulisses de son rôle de PMO et consultante interne en industrie.

    Lire la suite