Containers and clusters
New in version 0.9.0: LiberTEM repository on Docker Hub with images for public use.
The LiberTEM server will only bind to localhost by default, unless
token-based authentication is enabled or the
--insecure flag is
provided. It should run behind a reverse proxy that supplies the token and
adds encryption, authentication and authorization when access from an
untrusted network is desired. Jupyter integration can be used for
Third-party commands like
dask-scheduler may bind to public
interfaces by default, which can expose a machine to remote code execution.
For that reason the interface should always be specified and limited to
Furthermore, the default command of the LiberTEM Docker container starts a server that binds to all interfaces to facilitate integration. Docker runs containers in an isolated environment and requires the user to expose the port explicitly. Singularity, however, will run the container like a regular user program. That means it exposes an insecure LiberTEM server to all interfaces when running the default container command. For that reason the command to run should always be specified explicitly when using Singularity.
Docker images can be found in the LiberTEM repository on Docker Hub. LiberTEM is
installed in a virtual environment in
/venv/ in the Docker image. The
libertem-worker can be found in
/venv/bin/, consequently. The
/venv/bin/libertem-server --host 0.0.0.0 --insecure --port 9000
When using Docker, you can run and expose the LiberTEM server to
localhost while accessing local data like this:
$ docker run -p 127.0.0.1:9000:9000 \ --mount type=bind,source=/path/to/your/data/,dst=/data/,ro libertem/libertem
To use the Docker image and Singularity to start
$ singularity exec docker://libertem/libertem /venv/bin/libertem-server
The tag “latest” (default) points to the stable release with the highest version number. Version tags for all stable releases are available as well. See the LiberTEM repository on Docker Hub for details.
You can update to the latest release like this:
$ docker pull libertem/libertem
$ singularity pull docker://libertem/libertem
Starting a custom cluster
LiberTEM can connect to a running Dask cluster. To start a cluster on
localhost, first run a scheduler:
(libertem) $ dask-scheduler --host localhost
GPU support in LiberTEM requires specific resource tags and environment settings on the dask workers. The easiest way to start workers with the appropriate settings is
(libertem) $ libertem-worker tcp://localhost:8786
There are a few command line options available:
Usage: libertem-worker [OPTIONS] [SCHEDULER] Options: -k, --kind TEXT Worker kind. Currently only "dask" is implemented. -d, --local-directory TEXT local directory to manage temporary files -c, --n-cpus INTEGER Number of CPUs to use, defaults to number of CPU cores without hyperthreading. -u, --cudas TEXT List of CUDA device IDs to use, defaults to all detected CUDA devices. Use "" to deactivate CUDA. -p, --has-cupy BOOLEAN Activate CuPy integration, defaults to detection of installed CuPy module. -n, --name TEXT Name of the cluster node, defaults to host name -l, --log-level TEXT set logging level. Default is 'info'. Allowed values are 'critical', 'error', 'warning', 'info', 'debug'. --preload TEXT Module, file or code to preload on workers, for example HDF5 plugins. Can be specified multiple times. See also https://docs.dask.org/en/stable/how- to/customize-initialization.html#preload-scripts for the behavior with Dask workers (current default)and https://libertem.github.io/LiberTEM/ reference/dataset.html#hdf5 for information on loading HDF5 files that depend on custom filters. --help Show this message and exit.
New in version 0.6.0.
New in version 0.9.0:
--preload was added.
For a cluster setup, you can run the scheduler on the appropriate network interface and run workers on all cluster nodes to connect to the scheduler.
You can then connect to the cluster’s scheduler URL in the LiberTEM web GUI.
For easier deployment of in container-based environments, you can also use the Docker image.
Example: Start a scheduler and workers in an isolated environment with Docker.
$ docker run --mount type=bind,source=/path/to/your/data/,dst=/data/,ro \ libertem/libertem /venv/bin/dask-scheduler $ docker run --mount type=bind,source=/path/to/your/data/,dst=/data/,ro \ libertem/libertem /venv/bin/libertem-worker tcp://<scheduler-addr>:8786
Example: Start a scheduler and workers in the context of the local user with Singularity.
$ singularity exec docker://libertem/libertem /venv/bin/dask-scheduler --host localhost $ singularity exec docker://libertem/libertem /venv/bin/libertem-worker tcp://localhost:8786