Chapter 1 - A Case for Testing

Why bother testing?

Few people disagree that software should be tested. Where they disagree is how it should be tested. This guide is intended to provide a practical approach to software testing that aims to solve the most significant challenges associated with effective software testing.

The first issue to overcome is what may seem (and arguably should be) a moot question, namely whether we really need testing or not. Well, in short, we do and the reasons for this are both underpinned by some basic assumptions and overshadowed by some fallacies.

Before we launch into a detailed view of how to test we’ll first investigate why we test. Let’s look at what the world looks like without any formalized testing.

Humans make mistakes

There WILL be bugs. Humans will make mistakes which lead to bugs, errors or simple misunderstanding of requirements. This cannot reasonably be avoided so we have to assume it will happen and create systems to accommodate. The only uncertainty is surrounding the impact of the bugs.

Key point:

Bugs will happen, you can bank on it, only the severity of their impact will vary.

More code means more bugs

Given the above assumption, it follows that the more humans there are or the more time spent the more bugs there will be. Therefore the more lines of code there are the more bugs there will be as this simply represents more “output” from the engineer.

But it gets worse. In most cases new features added will not only introduce the potential for new bugs in the new code, but also raise the possibility of exposing bugs in old code as either minor changes are made to accommodate the new code, or the old code is exercised in a new way leading to previously unseen problems.

Key point:

Bug incidents can increase dramatically as new features are added

More code means more elusive bugs

Often the engineer who is charged with fixing a bug is not the engineer who created it. When this is the case any assumptions about the prior knowledge of the engineer being useful in tracking down and fixing a problem become much less relevant. The implication here is that the time it takes to find and fix a bug also increases as features are added.

Key point:

Bug fix times can increase dramatically as new features are added

The role of the test

The obvious assumption is that software testing, automated or otherwise, is intended to reduce the faults experienced by end users. That’s true, but there’s a subtle yet vital nuance to this assumption:

Testing is NOT designed to make sure code works when it’s written

The common belief is that an engineer writes a test to ensure the code they wrote operates as intended, but this is a dangerously flawed assumption.

In an effort to reduce development time the engineer may be tempted to make a judgement as to whether a test is required at all based on the complexity or importance of the code written. Code that is high in importance may be considered a good candidate for tests whereas simpler code may be skipped in preference to building the next feature.

This is dangerous because it assumes that any bug that may arise in the simpler code affects that code. In reality the effect of a bug could arise in any part of the application and it is somewhat pointless to try to predict where a bug may or may not occur. Thus,

Key point:

Software testing is NOT designed to make sure code works when it’s written but rather to ensure future changes do not break existing code.

Tests change the way you write code

One of the most significant, yet most subtle products of implementing a deeply integrated testing methodology is the effect it has on the nature of the code written.

The overriding impetus to have all code subject to automated tests changes the way an engineer approaches a problem. They are no longer motivated by solely writing code to satisfy the requirement but now need to write code that is testable.

Testable code is often very different in appearance from that which would have been created were there no requirement for testing in the mind of the engineer. Concepts like dependency injection become more like an obvious conclusion and in many cases a necessity and less like a choice or personal preference. Code typically becomes more readable because the engineer is considering how the code is used rather than how it works, and units of work become smaller simply because they are easier to test.

Finally code becomes more reusable, with fewer repetitions. The simple requirement of having to write tests means that engineers are far less inclined to simply copy and paste sections of code because it means they’ll have to write more tests. In an effort to avoid writing unnecessary tests the engineer will put greater effort towards creating reusable code.

The hundreds of tiny decisions that an engineer makes on a daily basis are tinged with an overlay of thinking, “how will I test this?” which alters the thinking and results in output which literally “can be tested”. This is often misunderstood in the argument that testing adds significantly to development time.

Key point:

A well run, tightly managed testing regime leads to more stable, reusable code.