Unit Testing

import matplotlib
if not hasattr(matplotlib.RcParams, "_get"):
    matplotlib.RcParams._get = dict.get

2.4. Unit Testing#

In the previous chapter, you have read about Assert statements. They are quick and easy to implement, but need to be run manually and are limited in what error reports they can raise.

Unit testing, on the other hand, is a more structured approach to testing code. They often contain assert statements, but also provides the framework to run these statements. The main advantage of setting up a unit test is that you can run tests automatically and report the results. They also give you more control in terms of checking for specific exceptions, return values etc.

A rule of thumb is that assert statements will help you here and now, while unit testing is something you take the time to implement in order to help others later.

Check the cells below to see the difference between a lone assert statement, and an assert statement wrapped inside a unit test:

import numpy as np

x_array = np.array([1, 2, 3])
x = np.sum(x_array)

assert x == 4, "Check failed!"
assert x_array[0] == 6, "Check failed!"

print(x)

The assert statement only works inline and the code stops running if it fails (the print statement does not work). If you want to run several assert statements, you will also need to run and check them all individually. Notice how the second assert statement does not run if the first one fails.

import unittest
import numpy as np

x_array = np.array([1, 2, 3])
x = np.sum(x_array)

class TestSum(unittest.TestCase):
    def test_sum(self):
        self.assertEqual(x, 4, 'wrong sum!')
    
    def test_first_element(self):
        self.assertEqual(x_array[0], 0, 'first element is wrong!')

# This line is necessary to run unittest in a Jupyter notebook
unittest.main(argv=['first-arg-is-ignored'], exit=False)

Here, you can ‘bundle’ together several assert statements that run automatically. The code can also still run even if the test fails (the print statement still works) and all tests can be executed. Running a unit test also gives a report of which tests passed or failed. Try fixing the checks so that the assert statements pass. You will see that the error report prints out ‘OK’.

There are two main frameworks that are commonly used to write tests, pyTest and unittest (as seen above).

The pytest framework#

MUDE exam information

The contents of the section on pytest is partially adapted from this book.

By Geet George from Delft University of Technology, built with TeachBooks and Jupyter Book, CC BY 4.0.

pytest is testing framework that is very common. If pytest is not already installed in your environment, you can install it by running pip install pytest in the command line. You can run your tests simply by running pytest <your-filename>.py in the terminal.

The syntax is different from that of unittest (more on this below). You can simply write a function as normal, with an assert statement inside. pytest will automatically find all the tests in your file, by searching for functions that start with test_.

import numpy as np

def test_sum():
    assert x == 4, "wrong sum!"

def test_first_element():
    assert x_array[0] == 0, "first element is wrong!"

pytest will not automatically run in a Jupyter notebook cell. and needs to run via terminal to report an error message. (There are workarounds, but these are clunky) If you run the test above you will get the following output:

============================= test session starts ==============================
platform linux -- Python 3.12.3, pytest-9.0.1, pluggy-1.6.0
rootdir: /home/user
collected 2 items

test_example.py FF                                                [100%]
================================== FAILURES ==================================
__________________________________ test_sum __________________________________

    def test_sum():
>       assert x == 4, "wrong sum!"
E       AssertionError: wrong sum!
E       assert 6 == 4

test_current_cell.py:7: AssertionError
______________________________ test_first_element _____________________________

    def test_first_element():
>       assert x_array[0] == 0, "first element is wrong!"
E       AssertionError: first element is wrong!
E       assert 1 == 0

test_current_cell.py:10: AssertionError

=================================== short test summary info ===================================
FAILED test_current_cell.py::test_sum - AssertionError: wrong sum!
FAILED test_current_cell.py::test_first_element - AssertionError: first element is wrong!
=============================== 2 failed in 0.04s ===============================

The output tells us that both tests failed. It also tells us which test failed and why.

In the beginning of the output, you’ll see some information about your platform and Python details, and the root directory you ran the tests in. This is unimportant for now. Then, you’ll see the following:

  • collected 2 items: This tells us that pytest found 2 tests in our file.

  • test_example.py FF This tells us that both test failed. A . would indicate a passing test.

The unittest framework#

Here, we will focus on unittest. It is a package native to Python. We will also focus on implementing assert statements in these tests. If you are interested to learn more you can check out the Python documentation for it here.

It is important to note that running unittest in a Jupyter notebook requires a special wrapper, as seen above. However, it is far more common and useful to run these tests from your CLI. We will therefore focus on the latter in the sections below.

Outside of Jupyter notebook, you can run your tests simply by running pytest <your-filename>.py in the terminal.

Assert statements vs unittest methods#

In unittest you can use assert functions as normal. You may also opt to use the built-in methods. Generally, it is best practice to use the methods as they are more robust and can give you more detailed error reports. However, using assert statements will also work so it is up to you what you prefer. As you already know how to write assert statements, the sections below will focus on the unittest methods.

unittest methods#

A test case is considered a single unit of testing, and it’s represented by the TestCase class. The TestCase class has a variety of methods that are unittest’s equivalent of assert statements.

Instead of the syntax assert <boolean statement here>, you will need to call a specific method that corresponds to different boolean checks. So, instead of assert x == 4, we would use assertEqual(x, 4).

Here’s a list of the most commonly used assert methods in the TestCase class:

Method

Checks that

assertEqual(a, b)

a == b

assertNotEqual(a, b)

a != b

assertTrue(x)

bool(x) is True

assertFalse(x)

bool(x) is False

assertIs(a, b)

a is b

assertIsNot(a, b)

a is not b

assertIsNone(x)

x is None

assertIsNotNone(x)

x is not None

assertIn(a, b)

a in b

assertNotIn(a, b)

a not in b

assertIsInstance(a, b)

isinstance(a, b)

assertNotIsInstance(a, b)

not isinstance(a, b)

Syntax and implementation#

Let us go through the syntax of the test shown above step by step.

import unittest
import numpy as np

x_array = np.array([1, 2, 3])
x = np.sum(x_array)

#class TestSum(unittest.TestCase):
#    def test_sum(self):
#        self.assertEqual(x, 4, 'wrong sum!')
#    
#    def test_first_element(self):
#        self.assertEqual(x_array[0], 0, 'first element is wrong!')

# This line is necessary to run unittest in a Jupyter notebook
#unittest.main(argv=['first-arg-is-ignored'], exit=False)

In the first part of the code, we simply import the needed packages and define our variables.

import unittest
import numpy as np

x_array = np.array([1, 2, 3])
x = np.sum(x_array)

class TestSum(unittest.TestCase):
#    def test_sum(self):
#        self.assertEqual(x, 4, 'wrong sum!')
#    
#    def test_first_element(self):
#        self.assertEqual(x_array[0], 0, 'first element is wrong!')

# This line is necessary to run unittest in a Jupyter notebook
#unittest.main(argv=['first-arg-is-ignored'], exit=False)

Here we define a test case. We create a class called TestSum that inherits the properties from unittest.TestCase. This just means that we can use the properties of unittest.TestCase but customize it with what we would like to check.

import unittest
import numpy as np

x_array = np.array([1, 2, 3])
x = np.sum(x_array)

class TestSum(unittest.TestCase):
    def test_sum(self):
#        self.assertEqual(x, 4, 'wrong sum!')
    
    def test_first_element(self):
#        self.assertEqual(x_array[0], 0, 'first element is wrong!')

# This line is necessary to run unittest in a Jupyter notebook
#unittest.main(argv=['first-arg-is-ignored'], exit=False)

Next, we define some test methods. The test_ prefix is required for unittest to recognise it as a test. Note how we need to use self qas an argument so that Python recognises it as a method and not an ordinary function. If you were to omit the self argument, the code will not be able to run and you will run into a TypeError!

import unittest
import numpy as np

x_array = np.array([1, 2, 3])
x = np.sum(x_array)

class TestSum(unittest.TestCase):
    def test_sum(self):
        self.assertEqual(x, 4, 'wrong sum!')
    
    def test_first_element(self):
        self.assertEqual(x_array[0], 0, 'first element is wrong!')

# This line is necessary to run unittest in a Jupyter notebook
#unittest.main(argv=['first-arg-is-ignored'], exit=False)

Next, we write out the actual things we would like to test. In the first part of the code, we check if x is equal to 4. As the equality fails, the test is marked as failed and the optional message 'wrong sum!' will be included in the failure output Note how we use the method assertEqual on self instead of directly using an assert statement.

# This line is necessary to run unittest in a Jupyter notebook
unittest.main(argv=['first-arg-is-ignored'], exit=False)

The last line in this cell is specific to running unittest in a Jupyter notebook. Usually you would not do this, but we have implemented it here for the book. unittest.main runs the test itself, we pass the arguments rgv=['first-arg-is-ignored'], exit=False to make unittest ignore any command line arguments called outside the notebook and to prevent the Jupyter Kernel from stopping.

Syntax using unittest and assert statements#

If you prefer to use an assert statement, you can use the syntax shown below. Here, the wrapper around your assert statement stays the same as before. However, instead of calling a specific assert method on self, we simply run a standard assert statement.

class TestSum(unittest.TestCase):
    def test_sum(self):
        assert x == 4, "wrong sum!"
    
    def test_first_element(self):
        assert x_array[0] == 0, "first element is wrong!"

Attribution

This chapter is written by Jialei Ding and Geet George. Find out more here.

Contents