Home of a Creative Engineer

Tools for Test Driven Development – Test Frameworks

In my article Write better code with Test Driven Development I showed you the benefits of Test Driven Development (TDD). Here I show you the tools that you need to perform TDD.

 

Test Frameworks

The first tool that you require is a test framework. This framework executes your tests and checks the results. When the tests are run, it shows you the result of each test, all failed tests and the error messages from these tests.

How a Test Framework runs the tests

To write a new test, you implement a test method. The test framework executes all test methods. How the framework knows which method is a test depends on the programming language and the test framework. For example, the C++ test framework GoogleTest uses macros:

The Java framework JUnit uses annotations. These are special properties that you can assign to methods. The Java Runtime Environment will read them and know that these methods contain tests.

And Python recognizes methods as test methods if their names start with test:

Test Fixtures

Tests that belong together – like all tests for a class – can be grouped into a test fixture. The test fixture is a class that has all these test methods as members.

With a test fixture, you can write code that is run before and after each test is executed. These methods are often called setUp and tearDown. Look at this test class in Python:

When you run it, it produces the following output that shows that setUp and tearDown are called before and after each test method:

You can put code that is needed for every test into the setUp method, like complex object creation or acquisition of a database lock.

In the tearDown method you can write code that leaves the test environment in a clean state. It is very important that each test can be run independently. This means that one test must not rely on the state that another test left after it was executed. You can ensure this with a proper teardown method. It might reset a test database that is used by all tests, or it might cleanup objects that cannot be destroyed by the garbage collector.

How the test framework checks the results

The test framework not only runs the test, it also checks for success or failure of the tests. You have to tell the test framework how to perform these checks. The framework provides methods that your test calls to check for certain conditions. If these conditions are not met, than the test framework will mark the test case as failed and print out the kind of error. These methods are called asserts, and they come in different flavors. The simplest version is assertTrue. It checks that a boolean condition is true. Moreover, there are specific versions like assertGreaterEqual, which checks if one expression is equal to or greater than another one. The more specific versions make the code easier to read, and they also make the error output of the test more informative.

Consider this example. Both of the following tests perform the same check:

But if we make them fail by changing the expected value to 7, then the second version produces the more comprehensible output:

Therefore, it is always good to use the most specific version of every check, because it makes it much easier to determine the reason for a failed test.

You can make the error message even more specific by providing a custom error message.

This produces the following output:

Assertions and Expectations

Some test frameworks support not only assertions, but also expectations. Both can be used in almost the same way. The difference between them is that an assertion aborts the execution of the test as soon as it fails, and the expectation lets the test continue.

In this kind of test frameworks, you can write tests that check a necessary condition with an assert, e.g. that a reference is valid. If it is not valid, it does not make sense to do any checks on it so the test is aborted. And if it is valid, several different properties of the reference can be checked with expectations. Thus the test result can show more than one failure on the properties at once.

Outlook

A test framework is a necessary tool for doing test driven development. Many different frameworks are available for almost all programming languages. Many programming languages come with a test framework in their standard libraries, like Python. For other programming languages there are de-facto standards, like JUnit for Java or GoogleTest for C++.

To start test driven development, make yourself familiar with the default test framework for your programming language. Once you have mastered the basics, you will easily understand other test frameworks as well.


I hope you enjoyed this article. If you have any questions or comments, please contact me.