The Association for Software Testing is crowd-sourcing a book, Navigating the World as a Context-Driven Tester. The book is edited by Lee Hawkins, who posts questions on Mastodon, LinkedIn, BlueSky, Twitter, Slack, and the AST mailing list. You don’t have to be a member to give your answer to the most recent prompt, “How can I possibly test »all the stuff« every iteration?”. Mine is below.
Depends on what you mean by “all the stuff”.
You might mean the full regression testing of entire application, including all the features, negative paths, edge cases etc. And you are right - you can’t possibly do all these things every iteration, maybe except for some very small products.
But the thing is: you shouldn’t do this kind of comprehensive testing in every iteration, you shouldn’t feel the need to do it, and you shouldn’t be asked to do it.
The iteration is the concept from the agile software development world - in the model where software is developed for extended period of time and delivered once, at the end, the word “iteration” has no meaning. The whole point of iterating over software during the development is to introduce small, but valuable change and deliver it to the users. Since the change is small, most of the product is the same as it was and therefore does not need to undergo comprehensive verification again. It is assumed that it was already tested in the past.
But maybe changes have broader scope than it looks like, or they introduce unintended consequences in other parts of the program. These are fair concerns. However, most changes introduced by developers don’t impact the entire system and you should not assume by default that they might. It definitely pays off to identify code with potentially large impact and make informed decision about scope of testing needed during the iteration.
It’s worth adding that if the team routinely finds itself in the situation that changes ripple through the entire system and comprehensive testing is necessary, then that’s a sign of deeper issues to tackle. It might mean that the architecture of the system is wrong and the team should seriously consider paying off the technical debt. Or maybe the software is more of a prototype, proof of concept, than the solution. In that case, perhaps comprehensive testing is not needed until requirements are better understood by the team and stakeholders can green-light one approach.
Many testers struggle with working in this kind of environment. They see their role as a last line of defense between a possibly buggy software and its users. The idea that users might get a software that was not comprehensively tested and verified make them feel uneasy.
If you are one of them, you need to understand that this kind of risk is calculated and intrinsic to software development model that values short iteration and frequent delivery. These ideas reinforce each other. The product is not comprehensively tested, which means there might be bugs, mistakes and problems that won’t be caught before users encounter them. But because the product is not comprehensively tested, it can be released faster. When the team learns about problems in the software, they can be fixed and a new release may be made.
If this argument does not convince you, I would urge to just give it a try. Acknowledge that you personally are not sold to this approach, but don’t block it if this is what the team wants to do. Keep your reservations to yourself for few releases and see what happens. You might learn that the product is not, in fact, significantly lower quality than before. And if it turns out the quality has lowered, then you will have hard empirical data to show to the team. Most people will agree these results are undesirable and some kind of change is needed.
Finally, you should not be asked to do comprehensive testing during each iteration. Personally, I don’t have much experience in that regard - I have never been in a team that would attempt that. While I can’t say what things might work in such situation, I guess the first thing to do is understanding the position such request is coming from. Perhaps stakeholders need to be taught the tradeoffs of iterations; or perhaps the team needs to adopt to more cautious software development models at the cost of speed.
I have not mentioned automation yet, which contributes to all of the points so far. Your work as a tester should be supported by automation that runs at least once a day and delivers up-to-date and reliable information about the state of the product. While automation has limits and can’t check everything, it can check some things. You should understand what automation can and can’t do in your project, so you can make an informed decisions about which features and paths you don’t need to spend much time on. You should feel confident in automation results. And when someone asks you to repeatedly perform specific tests, you might use that as opportunity to discuss spending some time on improving automation instead.
At the beginning of this article I said that the answer depends on the meaning of “all the stuff”. It might mean comprehensive testing, which I focused on so far. But it might also mean all the things that have changed during the iteration.
It does happen there’s so much work done by developers that testers in the team are swamped. I was in that situation not too long ago, and what did work for us was pushing for whole team ownership. Most of development tasks don’t need to be tested or verified by a dedicated tester - they only need to be tested by someone who did not write the code. In our case, all we had to do was signalling the problem during standup meeting, and rest of the team quickly came up with a solution - that some of developers should stop taking new work and instead focus on testing things that are already done. Having clearly spelled out definition of done and acceptance criteria for each ticket definitely helps.
However, the best way to deal with that problem is to actively avoid it. In general, as a tester, try to test changes as soon as possible. Don’t wait until the end of the iteration. If the team practices PR-based workflows, you can try testing the development build created before the code is merged. It helps to have the CI system that builds artifacts for each PR, but alternatively you can set up the build environment on your own machine. For exceptionally large features, give feedback to developer before code is written, and encourage early sharing of code and ideas. Daily standup, if there is one, is a good opportunity to show desire of this kind of conversations. You can also consider reaching out to developer and discuss things in private.
It might happen that despite using all the suggestions above, testers are still routinely swamped with work at the end of the sprint. In my experience, this is more likely to happen in teams that try to strictly follow rigid scrum structure, with iterations taking a week or two. Since work is assigned at the beginning of the sprint, there is a period of time when developers are focused on their work, and testers don’t have many things to do. That dynamic changes towards the end of the sprint. In that case, I would highly encourage to bring this up during the retrospective meeting. Most of the people will agree that this is a systematic problem that is not sustainable in the long term. As usual with self-organizing teams, it’s up to the team to come up with a solution. Just remember to monitor the situation and raise this topic again if there is no visible improvement.
Comments