It’s Not Testing

Written by Lisa Crispin Wednesday, 14 September 2011 00:00

Here is the third essay from our short-list of entries to our essay competition.  All week we will be releasing an essay with the winners finally announced at the end of the week (16 September 2011).  So here is Lisa Crispins essay "It’s Not Testing, It’s Not Coding, It’s Software Development".

It’s Not Testing, It’s Not Coding, It’s Software Development: A Rant About Why The Whole-Team Approach Rules

Like so many people I know in software development, I started my software career by accident. By virtue of acing a “programmer aptitude test” and having an MBA, I was offered a programmer trainee position with the University of Texas Data Processing Division. The managers there had a brilliant concept: hire subject matter experts with little or no programming experience, and train them all to write code exactly the same way. The truth is, it’s hard to teach untrained people accounting, finance, or library science. But it’s easy to teach people with those skills to write code. We could and did easily work on each other’s code - collective code ownership!

Since none of us had any real programming experience, we had no idea about official ways to develop software. We didn’t have many roles. We were basically all combination programmers and business analysts. Our department also had a DBA, a staff of machine room operators, and some system administrators. We sat down with our customers, learned what they wanted, and started coding. I don’t recall any requirements documents. We weren’t the worlds’ best programmers, but we used frameworks that let us produce software quickly, so we could just throw it away and start over if it wasn’t quite right or needed changing.

We always had time to learn and try new technology. And there was plenty of new technology around, since companies such as Xerox and Apple showered us with their latest hardware and software. We experimented with the Xerox Star, the Apple Lisa, and moved big-time into Macs when they came on the market.

It never occurred to us to have independent testers, but we could fix problems quickly. We developed large systems such as student registration, library circulation, accounting, payroll and an online library catalog. Looking back, we had a kind of whole-team mentality. Nobody felt they “owned” any part of the system, and everyone could jump in wherever they were needed.

This was a “whole-team approach” by default – we didn’t know any other way to develop software. Unfortunately, I moved on to teams that strayed far from this approach.

Learning About Testing

I joined my first test team at a software company whose developers were based in Germany, Canada and other countries, but whose customer support and testing were based in Denver, Colorado. While we tested new releases, the developers were already working on the next release. When we reported bugs, they had to stop and scratch their heads and try to remember what that old code was doing so they could fix it.

We automated regression tests, but it was so long after the fact of coding that it probably didn’t contribute much to code quality. We built relationships with the development teams, but that isn’t the same thing as being ON the same team. We did catch bugs and get them fixed either before release or in a patch release. But as they say, you can’t test quality in.

The programming teams weren’t on a death march, in fact it was quite a pleasant working environment, but they weren’t committed to delivering high quality software, either. We testers understood the needs of our customers well, but we couldn’t meet those needs on the back end of development.

Focus on Quality

At a software company where I worked in the mid-90s, our VP of Engineering possessed a laser-like focus on quality. Not only did she require the programmers to automate all of their unit tests – they actually had to write “unit test plans”! That was a bit of overkill, but it got them thinking about their code design up front. We had continuous integration on a daily basis with excellent unit test coverage. Our QA team was large, we had a big budget, and lots of time for learning. I remember spending weeks learning SQA Robot (as it was called at the time) so that we could automate a decent percentage of GUI-level regression tests as well.

This was back in the days of year-long release cycles. Our customers weren’t looking for new features every month. We had plenty of time for manual exploratory testing. We testers were involved in projects from early requirements gathering up to the release. The company culture revolved around quality, rather than speed. The last quarter I worked there, we had zero critical bugs in production – that’s a real accomplishment.

Was this a true whole-team approach? While it was the most functional Waterfall environment I ever enjoyed, we all stuck to our roles most of the time. Testers coped with testing issues, programmers wrote code (and unit tests), source code control managers managed the source code control, business analysts wrote requirements documents. We communicated and coordinated extremely well, but we each were representing our own agendas.

A True Whole-Team Approach to Quality

My first conscious introduction to the whole-team approach to quality came with my first Extreme Programming (XP) teams. What a difference to work with a team where every developer was test-infected! My programmer teammates struggled to master test-driven development. In fact, every agile team I’ve worked on took about eight months to get traction on TDD. But we had adopted XP specifically so we could build killer-quality software, and everyone on the team had a true commitment to doing so.

This was in the early 2000s, when the selection of test frameworks and drivers wasn’t anywhere close to what it is today. Of course we had JUnit, but knew we needed higher-level tests. At that time, we were mainly limited to vendor tools which used a proprietary scripting language for GUI testing. Open source tools such as HTTPUnit were starting to appear, but to this tester used to sophisticated commercial GUI test tools, they looked kind of lame.

I worked with wonderful programmers, and they all bought into the whole-team approach to quality. I was the only tester on teams of eight or more programmers, they knew I couldn’t possibly do all testing activities by myself. They suffered through having to learn a proprietary, C-like scripting language to help automate GUI tests. They pitched in on manual exploratory testing. The reward for all of us was delighted customers.

Diversity is Crucial

Back at the U.T. Data Processing Division, we had a delightful mix of backgrounds. Music majors, accountants, math majors, finance majors, liberal arts majors, librarians, a wide range of subject-matter experts who turned into wicket coders. We had a huge diversity of skills, background and viewpoints. I’m sure that is a big reason our customers were always happy.

Clearly, there are areas of expertise that require plenty of practice and study. We do need expert exploratory testers, expert security testers, expert performance testers. But they can’t work in isolation. Most projects require all this testing expertise, plus skills in other aspects of software development. While we value expertise in one area, we value experts working together towards a common goal more.

In my experience, when we adopt a whole-team approach to quality, we get people with diverse backgrounds and experience focused on delivering the best quality software we can possibly produce. When programmers commit to producing great software, they start thinking about ways to facilitate test automation. When business analysts are driven to delight customers, they look for opportunities to coordinate with testers and programmers to turn requirements into tests that drive development. Technical writers with passion for the user experience notice ways to enhance features and share those ideas with the team. DBAs wanting happy users find ways to tune performance.

We need lots of roles to help develop software, but the key is, we can’t divide these roles into silos. Maybe we don’t need to join hands and sing Shambhala, but we do need diversity. And we need that in a context of one team, where everyone who helps deliver software – programmers, testers, analysts, DBAs, system administrators, technical writers, UX designers – is an equally-valued member of that development team.

Stop the Divisiveness

Sure, we always need to find better techniques for activities such as exploratory testing, performance testing, security testing, usability testing. But we need to lose this “us against them” mentality. I’m so tired of reading tweets such as “turned software over to QA, I expect it to come back bruised and broken”. The QA team isn’t going to break the software, any more than they can “test quality in” to it. We need to get over is this idea of “turning software over” to some other team. We aren’t different teams. We’re all committed to producing the best code we possibly can. So why don’t we put more energy into learning to communicate better, and less energy into being divisive?

Back in 2001 I joined a company doing a pretty good job of Extreme Programming (XP). They were practicing test-driven development, continuous integration, pairing, refactoring, lots of XP practices. But they had no professional testers on their teams. They delivered stable code to their customers – it just wasn’t exactly the functionality those customers desired. So even though the server never fell over with a null pointer exception or an index out of range, the customers weren’t happy. You can have the best programmers in the world, but that’s no guarantee their wonderful code will do what the customers want. You also need people on your team who know how to elicit examples of desired and undesired behavior from customers, and turn those into tests that guide development.

Conversely, back in my waterfall days, I worked on some awesome QA teams that diligently identified every problem with the application. Unfortunately, we spent most of our time finding unit-level bugs that would easily have been caught with automated unit tests. You can have the best testers in the world, but if your programmers aren’t committed to producing high-quality code, your product is still going to suck.

That’s Why It’s About the Whole Team

Software quality has so many aspects. Just look at the Agile Testing Quadrants