Containers and clusters

New in version 0.9.0: LiberTEM repository on Docker Hub with images for public use.

Changed in version 0.12.0: Container images are now available from the GitHub container registry instead.

Note

The LiberTEM server will only bind to localhost by default, unless token-based authentication is enabled or the --insecure flag is provided. It should run behind a reverse proxy that supplies the token and adds encryption, authentication and authorization when access from an untrusted network is desired. Jupyter integration can be used for that.

Third-party commands like dask-scheduler may bind to public interfaces by default, which can expose a machine to remote code execution. For that reason the interface should always be specified and limited to trusted networks.

Furthermore, the default command of the LiberTEM Docker container starts a server that binds to all interfaces to facilitate integration. Docker runs containers in an isolated environment and requires the user to expose the port explicitly. Singularity, however, will run the container like a regular user program. That means it exposes an insecure LiberTEM server to all interfaces when running the default container command. For that reason the command to run should always be specified explicitly when using Singularity.

Containers

Docker images can be found in the LiberTEM repository on the GitHub contrainer registry. LiberTEM is installed in a virtual environment in /venv/ in the Docker image. The executables libertem-server, dask-scheduler and libertem-worker can be found in /venv/bin/, consequently. The container runs /venv/bin/libertem-server --host 0.0.0.0 --insecure --port 9000 by default.

When using Docker, you can run and expose the LiberTEM server to localhost while accessing local data like this:

$ docker run -p 127.0.0.1:9000:9000 \
  --mount type=bind,source=/path/to/your/data/,dst=/data/,ro ghcr.io/libertem/libertem

To use the Docker image and Singularity to start libertem-server:

$ singularity exec docker://ghcr.io/libertem/libertem /venv/bin/libertem-server

Available versions

The tag “latest” (default) points to the stable release with the highest version number. Version tags for all stable releases are available as well. See the LiberTEM contrainers on the GitHub container registry for details.

Note

Older versions are also available on Docker Hub.

Updating

You can update to the latest release like this:

$ docker pull ghcr.io/libertem/libertem

or

$ singularity pull docker://ghcr.io/libertem/libertem

Starting a custom cluster

LiberTEM can connect to a running Dask cluster. To start a cluster on localhost, first run a scheduler:

(libertem) $ dask-scheduler --host localhost

GPU support in LiberTEM requires specific resource tags and environment settings on the dask workers. The easiest way to start workers with the appropriate settings is

(libertem) $ libertem-worker tcp://localhost:8786

There are a few command line options available:

Usage: libertem-worker [OPTIONS] [SCHEDULER]

Options:
  -k, --kind TEXT             Worker kind. Currently only "dask" is
                              implemented.
  -d, --local-directory TEXT  local directory to manage temporary files
  -c, --n-cpus INTEGER        Number of CPUs to use, defaults to number of CPU
                              cores without hyperthreading.
  -u, --cudas TEXT            List of CUDA device IDs to use, defaults to all
                              detected CUDA devices. Use "" to deactivate
                              CUDA.
  -p, --has-cupy BOOLEAN      Activate CuPy integration, defaults to detection
                              of installed CuPy module.
  -n, --name TEXT             Name of the cluster node, defaults to host name
  -l, --log-level TEXT        set logging level. Default is 'info'. Allowed
                              values are 'critical', 'error', 'warning',
                              'info', 'debug'.
  --preload TEXT              Module, file or code to preload on workers, for
                              example HDF5 plugins. Can be specified multiple
                              times. See also
                              https://docs.dask.org/en/stable/how-
                              to/customize-initialization.html#preload-scripts
                              for the behavior with Dask workers (current
                              default)and https://libertem.github.io/LiberTEM/
                              reference/dataset.html#hdf5 for information on
                              loading HDF5 files that depend on custom
                              filters.
  --help                      Show this message and exit.

New in version 0.6.0.

New in version 0.9.0: --preload was added.

For a cluster setup, you can run the scheduler on the appropriate network interface and run workers on all cluster nodes to connect to the scheduler.

You can then connect to the cluster’s scheduler URL in the LiberTEM web GUI.

For easier deployment of in container-based environments, you can also use the Docker image.

Example: Start a scheduler and workers in an isolated environment with Docker.

$ docker run --mount type=bind,source=/path/to/your/data/,dst=/data/,ro \
  ghcr.io/libertem/libertem /venv/bin/dask-scheduler
$ docker run --mount type=bind,source=/path/to/your/data/,dst=/data/,ro \
  ghcr.io/libertem/libertem /venv/bin/libertem-worker tcp://<scheduler-addr>:8786

Example: Start a scheduler and workers in the context of the local user with Singularity.

$ singularity exec docker://ghcr.io/libertem/libertem /venv/bin/dask-scheduler --host localhost
$ singularity exec docker://ghcr.io/libertem/libertem /venv/bin/libertem-worker tcp://localhost:8786