You have a quality strategy…it just might not be very sustainable

Why is it that some projects – even though they have no dedicated tester and no automation or specific quality practices to speak of – manage to release new versions that are apparently good enough, without horrific errors in production?

As a professional tester, this confused me for years. After all, my knowledge and experience tell me that having a trained tester on a team and having specific, identifiable and planned quality activities in place is necessary for good quality software. (As an aside: the tester isn’t solely responsible for all quality activities, but is a mix of hands-on tester, test-analyst and quality coach/enabler – but that’s another story).

And yet, I often encounter projects where this isn’t the case – and they are doing fine! Of course, anyone who isn’t thrilled about testing loves to use these examples to say we don’t need testing or that it’s pointless talk about a quality strategy.


An implicit strategy

A few months ago, the reason hit me. Projects that were doing fine with their quality despite not having quality practices in place, had most of these things in common:

  • A well-known, well-documented and stable tech-stack. One that is stable in the project, the company and the industry
  • A stable team of experienced developers who have been working on this tech stack in this project for at least 2 years
  • Within this team, everyone has a good overview of the architecture and can assess impact of new development or changes better
  • Also, a large amount of domain knowledge has already been transmitted to the developers in the team, meaning that misunderstandings are rarer
  • Good standing with the customer due to good quality in the past
  • One person on the team who checks the work of the others – usually someone with yet more experience. (These people obviously still make mistakes, but in this context, perhaps much less frequently).
  • The opportunity to fix problems reasonably quickly without annoying the customer
  • Little direct contact to support problems due to 1st and 2nd level support being handled by others

I realised that these points are basically the quality strategy of such teams. In short:

  • Be in a tech stack, architecture and domain that you know well
  • With an experienced team that has been together for a while
  • And a customer that is easy going

For me, that explains it. In a situation like that, it’s somewhat less likely that errors will be made. And if they are, then the impact isn’t maybe as bad.


The dangers

But wow, is this quality strategy dangerous!

All that would have to happen in this case would be one of the following:

  • Some experienced people leave
  • A few juniors join
  • One large error happens so that the customer isn’t as calm any more
  • The most senior developer who has been reviewing and keeping a general eye on everything leaves the team
  • A new tech stack or domain area is introduced

And suddenly, the quality could come crashing down. Nothing about the strategy outlined above is sustainable past the current context.

Even more critically, it lulls people into a false sense of security. A developer on this project might try to transfer this strategy to a new project where things aren’t the same. “We managed fine without automation / reviews / 3 amigos…in previous projects…”


Make your strategy explicit

There are generally accepted practices that belong to a good quality strategy for modern development. None of these are best practices, since context will always be different. However, teams should think about:


o   Build and test pipelines

o   Test data provisioning

o   Test automation at various levels (not just one!)

  • Unit
  • Integration
  • API
  • UI
  • Visual Testing


o   Exploratory Testing

o   Observability


Development practices for quality

o   Test Driven Development

o   Contract Testing

o   Pairing

o   Mob Testing and Mob Programming

o   3 or 4 Amigos

o   Skill matrices (for avoidance of bus factors)

o   Expectation matrices (for avoidance of gaps)

o   Regular refactoring

o   Comb-shaping for all roles

o   Cross-functional teams

Agile practices

o   Quick feedback cycles

o   Small, valuable stories

o   Retrospectives

Preventative measures

o   Tester involvement from the outset

o   Design for testability, controllability and observability

o   Team training in whole team quality


Obviously, it is nice if the project context itself means that errors are less likely – just as described above. However, relying on an implicit “easy mode” isn’t the kind of strategy I see working in the long term or on future projects.

Regardless of what aspects from the list above you use for your quality strategy, you need to be explicit about what it is – so that you can concretely implement it and also question it when things change. That’s how to make it sustainable.