Software Testing

The role of testing is heuristic

Hang on a sec, didn’t I just get done saying testing is an activity and not a role? I did say that, didn’t I? Did I mean it? Well, it’s true in the same sense that Darth Vader killing Anakin Skywalker is true. As Obi Wan said – ‘from a certain point of view’ – namely how we as testers pitch our role to non-testers.

How we position software testing to non-software testers is important. I have a strong sense that currently we explain the role of software testing to non-testers very much in terms of what testers do and others don’t (or can’t, or won’t). As testers we bring skills and experience that are different to those of a programmer, or UX or product management and so on and I think it’s important that the value of these skills be recognised. I think though, to say that only skilled testers can/should be responsible for exercising these skills is a bridge too far. I want both testers and programmers to think more fluidly in terms of what their role and responsibilities are. The short version is – mostly because I think the ‘that’s not my job’ mindset is super unhelpful to all involved in software development.

In the comments of my last post James Bach said ‘I think the role of testing is a very useful heuristic’. I agree. It is. I didn’t state that explicitly in my last post and really I should have. It’s a realisation that I have only come to recently and the realisation shocked me. I identified so strongly with the role of a tester that relaxing my grip on ‘tester’ as an identity was incredibly confronting.

It’s not that the role of testing as a concept is not useful, but like any other heuristic, it is fallible. If one is careless in describing the responsibilities and characteristics of testing in terms of what testing is and what other roles are not, it can help to reinforce stereotypes that are not useful. By way of example, here are a few beliefs that I’ve heard from testers about why programmers can’t test, that I think are unhelpful.

‘Programmers shouldn’t test their own code’

I think programmers should not be the only ones to test their own code if quality is at stake. What we think we’ve written is often not what we’ve actually written. Talk to a programmer about reading code they wrote more than a month ago and they’ll often say ‘I wonder wtf I was thinking’. If you talk to a screenwriter, or any other kind of writer really, they’ll often say the same thing. At the time of writing, we often lack the perspective to be effectively critical of what we’ve written. With all that said, if any programmer is writing anything that matters, they absolutely should be testing their own code.

‘Programmers and testers think too differently for either one to be good at each other’s job’

While I believe it’s true that the focus of a tester and that of a programmer are very different, that doesn’t mean we cannot have a good fundamental understanding of each other’s work. I would go so far as to say that if testers and programmers don’t have good understanding of the fundamentals of each other’s craft, then they are almost certainly going to be less effective than someone that does have that knowledge. As with a tester knowing how to code, knowing the basics of the technology stack the programmers are working with, understanding of the patterns they’re using and their advantages and disadvantages is helpful in spotting possible problems, so too should coders have an understanding of testing fundamentals, not just whatever other automated testing they’re doing. You should be able to talk to them about oracles, test heuristics, the various ‘ilities’ and risk without them wondering what the hell you’re on about.

‘Programmers are too tightly focused on what they’re building to see the bigger picture’
Which seems to be saying ‘programmers don’t know how to defocus and wouldn’t see the value of doing so if they did’. Like other testing skills, focusing and defocusing are learned skills and can be honed with practice. Full stack developers have practice doing this because they need to understand the different technologies they’re working with and how they interact, their various gotchas and pitfalls. It is a skill that can be learned and there is benefit for programmers to know how to do it.

There are lots of reasons out there for why programmers are bad at testing. Testers reinforce that mindset every time they trot these little truisms out. It doesn’t have to be that way. Rather than looking at the tester role as something that is altogether separate from a programmer role, consider how the two roles can interact.

The advent of test driven development in its various flavours has helped blur the lines between the roles. TDD is generally used as a way to drive design and thereafter support programmers as they maintain and change code. Programmers write failing tests and then use the support of their IDE to fill in the code to make that test work. They build small pieces one at a time, each supported by tests that exercise what was just written. If a test is difficult to write, it points to a possible problem in the intended implementation. The initial focus of the tests is to help the programmer implement code that is elegant and maintainable. The fact that it may also cover things we’re interested in from a higher level is a bonus. It’s not exactly testing in the way a tester might consider testing, but there is definitely a relationship there.

Automated acceptance testing seems to sit more squarely between the roles. Where unit testing is code-supporting, or tech facing (if you want to go to Brian Marick’s Agile Quadrants model), acceptance tests can potentially have aspects of both code supporting and product supporting tests (tech facing & business facing).

Good programmers write tests before they write code. Great programmers critically question the requirements they’re given before they start building and keep the big picture in mind as they code. In an agile context, well written user stories will help them to do that as the story itself describes the big picture, or is part of an epic that does. Great programmers who pair will often spot and correct issues in the code they write and they’ll use the conversations they have while working to highlight possible remaining problems. If necessary, they’ll ask for specialist help (ie a tester).

In my current team, there is a strong sense of shared ownership of what we build. The programmers I work with are highly motivated to get testing right, because if we put out a substandard product, we are all responsible. We succeed or fail as a unit based on our ability to deliver value to our stakeholders. We’re a pretty new unit, relatively untried. We have a couple of wins on the board, but the quality of the work we put out reflects on us as individuals, as a team and on the department we’re a part of (not to mention the company as a whole). That’s a fair amount of responsibility. When things don’t go to plan, as will inevitably occur, we don’t waste time and energy in finger pointing. By the same token, if someone screws up, they’re the first to put their hand up for it. We fix what we need to fix, work out what we can improve and crack on. We succeed or fail as a unit. We own it. That’s just the way it is and it’s pretty awesome, I have to say.

Is it perfect? Hell no. There’s lots I want to improve, but at the basic level is that shared belief of joint responsibility and that is something that I believe is lacking from most tester/programmer relationships. That’s a damn shame and I want that to change.

Why aren’t more teams out there like this? My hypothesis is twofold.

1. There are a lot of people out there that call themselves testers who are really, really crap at software testing. Unfortunately, most programmers have only encountered this type of ‘tester’.

2. There are several different flavours of the sentiment that ‘programmers can’t test because…reasons’. Programming and Testing are different skills. How you focus your thinking for each of these skills is different, but to say that a programmer can’t test is a fucking cop out and lets them off the hook for work they should be doing.

I think it is a reasonable expectation to hold that developers take some interest in improving at testing if their current abilities are close to nil. Having attained some level of competence in testing fundamentals, I also think it reasonable that they are able to improve further should they so choose.

I also think that programmers are unlikely to spend enough time practicing or improving testing if we take that expectation away by saying things like ‘developers are crap at testing because they’re developers’. I’m not expecting that they’re as proficient as I am but I do expect a significantly higher standard than ‘I wrote a few unit tests and the code does what it should’. I want to be able to chat freely with programmers about what oracles they used to test against and how they approached testing the code they’ve written and what they think still needs attention. That’s not an unreasonable expectation to have from a programmer who values their craft and shares responsibility with you the tester for delivering value.

Is that lazy? Am I expecting someone else to be doing my work for me? No. Not at all. A programmer who has a solid understanding of testing fundamentals will deliver higher quality code so that when I do get ahold of it, I have a challenge on my hands. The obvious holes have been thought of and plugged already. As a tester, I get to do what I do best – exercise my tester skills to find those issues that are both difficult to spot and a significant risk to delivering value.

The roles of programmer and tester contain significant overlap in terms of thinking, skills and activities. It makes sense to me that the duties of each likewise overlap. Knowledge of one does not and should not preclude understanding of the other. The better we understand how each other works, the better we can help each other do better work. It takes effort. You’ll have to do stuff that makes you uncomfortable or feel dumb. The programmers you work with may resist taking on the responsibilities of testing. You might have to have difficult conversations, maybe repeatedly. What works well in one team may not work well in another.

By sharing the work we do, by working closely with our non-testing peers, helping them understand the work we do and educating ourselves about their work, I believe we will better demonstrate the value of the tester’s skill set and better set expectations of what testing is, whether it be a skill set embodied in a specialist role, a set of activities that a team undertakes, or some combination of both.


2 thoughts on “The role of testing is heuristic

Leave a Reply

Your email address will not be published. Required fields are marked *