LiberTEM - Open Pixelated STEM platform

gitter travis appveyor zenodo

LiberTEM is an open source platform for high-throughput distributed processing of pixelated scanning transmission electron microscopy (STEM) data [MNR+16].

It is designed for high throughput and scalability on PCs, single server nodes, clusters and cloud services. On clusters it can use the Hadoop file system with fast distributed local storage on high-performance SSDs. That way it achieves very high collective IO performance on a compact and cost-efficient system built from stock components. With cached file system reads it can reach a throughput of up to 14 GB/s per processing node with a quad-core CPU.

LiberTEM is supported on Linux, Mac OS X and Windows. Other platforms that allow installation of Python 3 and the required packages will likely work as well. The GUI is running in a web browser.

LiberTEM currently opens most file formats used for pixelated STEM:

  • Raw binary files, for example for the Thermo Fisher EMPAD detector [TPC+16]

  • Quantum Detectors MIB format (currently alpha, more testing and sample files highly appreciated)

  • Nanomegas .blo block files

  • Gatan K2 IS raw format (currently beta)

  • FRMS6 from PNDetector pnCCD cameras [SRB+15] (currently alpha, gain correction still needs UI changes)

  • FEI SER files (via openNCEM)

  • HDF5-based formats such as Hyperspy files, NeXus and EMD

  • Please contact us if you are interested in support for an additional format!

The 0.1 release implements anything that can be done by applying masks to each detector frame, for example the numerous virtual detector methods (virtual bright field, virtual HAADF, …) or center of mass.

The GUI of the 0.1 release allows interactive analysis and data exploration with basic virtual detectors, center of mass [KMM+16] and display of individual detector frames.

The Python API can be used for more complex operations with arbitrary masks and other features like data export. There are example Jupyter notebooks available in the examples directory. If you are having trouble running the examples, please let us know, either by filing an issue or by joining our Gitter chat.

LiberTEM is suitable as a high-performance processing backend for other applications, including live data streams. Contact us if you are interested!

Deployment as a single-node system for a local user is thoroughly tested and can be considered stable in the 0.1 release. Deployment on a cluster is experimental and still requires some additional work, see Issue #105.

LiberTEM is evolving rapidly and prioritizes features following user demand and contributions. In the future we’d like to implement live acquisition, and more analysis methods for all applications of pixelated STEM. If you like to influence the direction this project is taking, or if you’d like to contribute, please join our gitter chat, our development mailing list, and our general mailing list.

Installation

The short version:

$ virtualenv -p python3.6 ~/libertem-venv/
$ source ~/libertem-venv/bin/activate
(libertem) $ pip install libertem[torch]

Please see our documentation for details!

License

LiberTEM is licensed under GPLv3. The I/O parts are also available under the MIT license, please see LICENSE files in the subdirectories for details.

Documentation

Indices and tables

Citations

KMM+16

Matus Krajnak, Damien McGrouther, Dzmitry Maneuski, Val O\textquotesingle Shea, and Stephen McVitie. Pixelated detectors and improved efficiency for magnetic imaging in STEM differential phase contrast. Ultramicroscopy, 165:42–50, jun 2016. doi:10.1016/j.ultramic.2016.03.006.

MNR+16

Ian MacLaren, Magnus Nord, Andrew Ross, Matus Krajnak, Martin Hart, Alastair Doye, Damien McGrouther, Rantej Bali, Archan Banerjee, and Robert Hadfield. Pixelated STEM detectors: opportunities and challenges, pages 663–664. American Cancer Society, 2016. URL: https://onlinelibrary.wiley.com/doi/abs/10.1002/9783527808465.EMC2016.6284, doi:10.1002/9783527808465.EMC2016.6284.

SRB+15

M. Simson, H. Ryll, H. Banba, R. Hartmann, M. Huth, S. Ihle, L. Jones, Y. Kondo, K. Muller, P.D. Nellist, R. Sagawa, J. Schmidt, H. Soltau, L. Striider, and H. Yang. 4d-STEM imaging with the pnCCD (s)TEM-camera. Microscopy and Microanalysis, 21(S3):2211–2212, aug 2015. doi:10.1017/s1431927615011836.

TPC+16

Mark W. Tate, Prafull Purohit, Darol Chamberlain, Kayla X. Nguyen, Robert Hovden, Celesta S. Chang, Pratiti Deb, Emrah Turgut, John T. Heron, Darrell G. Schlom, Daniel C. Ralph, Gregory D. Fuchs, Katherine S. Shanks, Hugh T. Philipp, David A. Muller, and Sol M. Gruner. High dynamic range pixel array detector for scanning transmission electron microscopy. Microscopy and Microanalysis, 22(01):237–249, jan 2016. doi:10.1017/s1431927615015664.