ISO29119: Why we think you can’t standardise testing

In a climate where testers are trying to break away from the document-heavy days of the ISEB/ISTQB certification and become a more integrated part of the development process, the ISO have rolled out a standard for Software Testing.

Several of the big names in the Testing world have already commented on 29119, explaining why it’s a bad thing much better and in more detail than I could. James Bach, James Christie and Michael Bolton to name a few. And now eBay have formally rejected the standard with the most concise argument that I have seen so far.

You might think standards are a good thing, and in a lot of cases they are. As Ben Kelly points out in his eBay statement, they are relevant for things like hardware or data interactions where it’s important that all parties can communicate in the same manner. But I fail to see how you can apply a standardised approach to any part of an Agile process. In fact, the word ‘agile’ doesn’t appear at all in the overview of the Process section. It is finally shoehorned into Annex C for information purposes! The rest of the standard assumes that you are working in a completely Waterfall setup. There is no scope to expand into new tools or better practices, no mention of any influence on the actual requirements and no mention of collaboration with other teams.

Standardisation and certification run hand in hand. But the days of certification seem to count for very little in most disciplines now. People seem to be realising that the skills required to pass exams are very different to the skills required to excel in the real world!

Developers are often expected to complete coding challenges as part of an interview. Designers are judged on a portfolio of their previous work. So why should determining someone’s ability to test software be any different? We’re looking for a person’s ability to assess and complete a task. Not how rigidly they can follow a checklist of pre-written approaches.

Recently, when I’ve been part of interviews for new testers, we’ve given them a short basic spec for something like a Registration form and asked how they would approach it. I’m not asking for a full test plan. Quite the opposite. I’m looking for someone to query every bit of ambiguity (and suitability) that they can spot. And then show some domain knowledge by suggesting things like field validation, data storage, performance or usability.

“So why not just ignore it then?” I hear you say… Well the biggest threat from ISO29119 is that big companies who like standards (like banks, healthcare, local councils) will insist that their software is compliant with this archaic collection of methods and principles. So any software provider trying to win business with them will have to jump through hoops and pay for the documentation and assessments to comply.

Ultimately, I’m not too worried. It seems the majority of people in the software industry that I have spoken to are against 29119. Coincidentally these people seem to be part of the ‘new crowd’ and the fast growing startups and tech success stories of the past 10 years. Whereas the monolithic global outsourcing companies that have ‘always done it this way’ are sticking to their paperwork.

And as far as standards go - XKCD gets the last word.