Running distributed tests

To test the behavior of the distributed parts of LiberTEM, there are now a few automated test cases available. Because of the distributed nature of the tests, they need to be run across multiple network nodes (computers, VMs, or containers). We ship a docker-compose configuration to spin up a scheduler and two workers as separate docker containers.

Requirements

To run the distributed tests, you need to install (a current version of) docker and docker-compose. See the official docker documentation to get started. To start the test environment, change to the packaging/docker/ directory and run the following:

$ bash dist-build.sh
$ docker-compose down && docker-compose run tests ; docker-compose down

Note that you need to run the above command as a user that can access the docker daemon, for example by being in the docker group. The containers are running as long as the docker-compose command is running, so you can stop them using Ctrl-C.

You can then run the distributed tests using (from the same directory):

$ docker-compose run --rm tests

As we are running the tests in a docker container, the environment of the tests container will automatically match the environment of the workers.

After changing LiberTEM code, you need to rebuild and restart the containers. Just cancel the first docker-compose up run using CTRL-C and start from the top.

The rebuild should be faster than the initial build, which is accomplished by careful use of the layer caching feature of docker. This also means that you may need to update the packaging/docker/requirements.txt file by running the provided script update_reqs.sh when the dependencies of LiberTEM change, or when new versions of dependencies are released. In CI, this is done automatically.