Traditionally, in a software development process, developers and quality assurance testers have a clear divide. Developers implement functionality and QA tests to make sure the functionality they’ve developed meets the requirements. This type of testing, done by QA, is known as black-box testing. The way black-box testing works is to test functionality without knowing the internal workings of the system. In comparison, white-box testing is done by someone who has a clear understanding of the internals. In this blog we will go over the advantages and disadvantages of white-box testing compared to traditional black-box testing.
White Box Testing
During a white-box test case, a section of code is run with predetermined input values to validate predetermined output values. If these values do not match, then a bug is found. This is repeated many times throughout the application. The goal of white-box testing is to verify:
- Expected outputs
- Validate sections of code
- Ensure complete coverage of testing
- Incomplete paths, conditional statements
The advantages of white box testing include thoroughness, automation, time, optimization, and introspections.
The main tenant of white-box testing is complete code coverage. Basically, the idea is to test as much of the code as possible, which is much more thorough than traditional black-box testing. The thorough nature of white-box testing also gives a clear structure to testing. Testing must be clear, engineering-based, and have clearly defined rules.
Ability to Automate
Knowledge of the internals of the application allow for unit tests. As the name suggests, unit tests test small pieces of code, or units, to see if they run as expected. Because these tests are simple, developers can programmatically run these tests to quickly see if something has broken. Unit tests are a good way of testing if something, which was previously working, has recently broken.
There are always deadlines to meet in software development which makes time a priority in the development process. White-box testing can speed up the testing process significantly. Often times, a developer can see a bug and immediately have a general idea of what the issue is and how to fix it. Furthermore, white-box testing eliminates the communication cost between developers and QA, as developers find and fix issues themselves without needing to wait for QA.
Going through code section by section allows developers to remove superfluous sections of code or condense existing code. As well, code can be optimized by removing hidden errors which may not come up during normal testing.
White-box testing allows developers to carefully reason about implementation. Developers are forced to consider individual sections of code and how they link to other sections. Maybe the current implementation is fine but will not scale well in the future or has unnecessary parts that can be cut out. White box testing gives developers a chance to reevaluate designs and how they could be improved.
The disadvantages to white-box testing include its cost, rapidly changing code, and missed cases.
Because white-box testing is more thorough it becomes very expensive in time and cost to conduct. Although unit tests alleviate this somewhat, there is an initial investment that must be done to write the unit tests. Also, this type of testing can scale badly with large applications. It becomes virtually impossible to test every branch of code.
Compared to black-box testing, white-box testing requires skilled testers with programming knowledge. This increases the cost and could mean that developers are pulled off of developing new features. These costs all must be considered when conducting white-box testing.
Rapidly Changing Code Base
Automated test cases become a waste if the code base is rapidly changing. Often times, redesigns or reworks will cause most written test cases to be useless and in need of a rewrite.
White-box testing only validates and tests features that are currently there. If a feature is only partially implemented or something is missing, white-box testing will not pick up on this. This is where requirements driven black-box testing is superior.
White-box testing has several clear advantages and disadvantages. Whether the cost is worth the advantages must be carefully considered, especially since mileage may vary from project to project.