Software testing is a topic which has undergone much debate over the course of software development’s recent history.
In the most broad terms people fall somewhere along a spectrum of those who believe it is, for the most part,
a “nice to have”, and those who cannot live without the confidence and satisfaction a robust testing regime can offer
despite its perceived overhead.
The following guide will attempt to outline some of the key reasons testing should be considered a critical
component of any mature software development process as well as some basic guidelines for the creation of a testing
approach which maximizes release confidence while minimizing the impact on development times.
Language Use
The majority of code examples in this guide use Java as the default programming language however the concepts
presented are generally language non-specific. Java has been chosen because although the syntax is more verbose
than some other languages, its familiarity means the examples can be understood by a wide audience.
Definitions
Note
Not all types of tests are covered in this list, merely the ones referred to in this guide.
For the purposes of avoiding ambiguity or confusion we are going to start with defining some terms. Some definitions
may not correspond exactly to what you’re used to, but these are how we use the terms in this guide.
Unit Test
Although the term “unit test” is in the common vernacular for software engineers it is actually a very imprecise and
subjective term. Wikipedia suggests that a unit is “often an entire interface, such as a class, but could be an individual method” .
This ambiguity can lead to misunderstanding when talking about testing so for the purposes of this discussion we will define a unit test as:
“A unit test is a one which tests a single public method or function of a class or module”
The two key words here are “single” and “public”. A public method which internally calls private or protected
methods in the same class could be considered a single unit, whereas public methods that call other
public methods would not.
Black Box Test
This is a test in which the mechanisms involved in processing data are unknown to the test and the test simply
provides pre-conditions and asserts post-conditions. That is, the test provides data and expects certain results
without knowing how those results are derived.
Integration Test
For the purposes of this discussion we will define an Integration Test as one in which many parts of the system
are tested at once. Importantly an integration test will test a complete end-to-end lifecycle of an application
or a portion thereof as opposed to testing selected components independently. Note that an integration test may
be done with mocked components and the mere term Integration Test does not necessarily mean real systems are used,
although often this is the case. During this discussion when the term Integration Test is used it will typically
refer to tests that are performed with real systems as opposed to mocks.
Manual Test
A manual test is simply one that is conducted by a human via using the software directly without any intermediate
testing tools. This is often done by Quality Assurance staff who would typically follow a prescribed script of
actions and expected results.
Load/Scale Test
Load tests are tests done specifically to impose stress on the application by simulating heavy load conditions in
addition to highly concurrent load conditions (several simultaneous actions). This can be done to identify failures
but is often done to simply gauge scalability. That is, “how is performance affected with increases in load?” as
opposed to simply “does the application function under load?”.
Infrastructure Test
Infrastructure tests are those which test or verify the state of the infrastructure underlying the software.
This typically includes servers and networks associated with providing application uptime.
Automated tests that verify database operations are working, for example, may not signal a failure if/when a
database server ceases to be visible to the application due to a failure in a network device. Infrastructure Tests
would also encompass monitoring and alerting systems designed to provide “early warning” of imminent or recent failures.
QA Test
QA (Quality Assurance) tests, for the purposes of this guide, are equivalent to Manual Tests. The term Manual
Test is used to avoid any confusion between QA Testing and UAT (User Acceptance Testing)
UAT (User Acceptance Testing)
User Acceptance Testing is the process by which (human) testers will assess the implementation of the software to
determine if it accurately reflects the expectations and/or specifications originally provided to the engineers.
Although this is an activity which is distinct from Quality Assurance, which is typically intended to assure the
“quality” of the software (i.e. look for bugs), UA testing is often performed by the same staff who are responsible
for QA. This distinction is nevertheless an important one and is discussed in further detail later in the guide.