LiberTEM Context API¶

See Python API for introduction and complete examples.

Context¶

class libertem.api.Context(executor: Optional[JobExecutor] = None, plot_class=None)[source]¶

Context is the main entry point of the LiberTEM API. It contains methods for loading datasets, creating analyses on them and running them. In the background, instantiating a Context creates a suitable executor and spins up a local Dask cluster unless the executor is passed to the constructor.

Changed in version 0.7.0: Removed deprecated methods create_mask_job, create_pick_job

Parameters

executor (JobExecutor or None) – If None, create a local dask.distributed cluster and client using make_local() with optimal configuration for LiberTEM. It uses all cores and compatible GPUs on the local system, but is not set as default Dask scheduler to not interfere with other uses of Dask.
plot_class (libertem.viz.base.Live2DPlot) –
Default plot class for live plotting. Defaults to libertem.viz.mpl.MPLLive2DPlot.

New in version 0.7.0.

plot_class¶

Default plot class for live plotting. Defaults to libertem.viz.mpl.MPLLive2DPlot.

New in version 0.7.0.

Type: libertem.viz.base.Live2DPlot

Examples

>>> ctx = libertem.api.Context()  

>>> # Create a Context using an inline executor for debugging
>>> # See also Context.make_with() for a more convenient interface!
>>> from libertem.executor.inline import InlineJobExecutor
>>> debug_ctx = libertem.api.Context(executor=InlineJobExecutor())

create_com_analysis(dataset: DataSet, cx: int = None, cy: int = None, mask_radius: float = None, flip_y: bool = False, mask_radius_inner: float = None, scan_rotation: float = 0.0) → COMAnalysis[source]¶

Create a center-of-mass (first moment) analysis, possibly masked.

Parameters

dataset – the dataset to work on
cx – reference center x value
cy – reference center y value
mask_radius – mask out intensity outside of mask_radius from (cy, cx)
mask_radius_inner –
mask out intensity except for the ring between mask_radius_inner and mask_radius, centered around (cy, cx)

New in version 0.8.0.
flip_y (bool) –
Flip the Y coordinate. Some detectors, namely Quantum Detectors Merlin, may have pixel (0, 0) at the lower left corner. This has to be corrected to get the sign of the y shift as well as curl and divergence right.

New in version 0.6.0.
scan_rotation (float) –
Scan rotation in degrees. The optics of an electron microscope can rotate the image. Furthermore, scan generators may allow scanning in arbitrary directions. This means that the x and y coordinates of the detector image are usually not parallel to the x and y scan coordinates. For interpretation of center of mass shifts, however, the shift vector in detector coordinates has to be put in relation to the position on the sample. The scan_rotation parameter can be used to rotate the detector coordinates to match the scan coordinate system. A positive value rotates the displacement vector clock-wise. That means if the detector seems rotated to the right relative to the scan, this value should be negative to counteract this rotation.

New in version 0.6.0.

Returns

COMAnalysis – When run by the Context, this Analysis generates a libertem.analysis.com.COMResultSet.

Return type

libertem.analysis.base.Analysis

create_disk_analysis(dataset: DataSet, cx: int = None, cy: int = None, r: int = None) → DiskMaskAnalysis[source]¶

Create an Analysis that integrates over a disk (i.e. filled circle).

Parameters

dataset – the dataset to work on
cx – center x value
cy – center y value
r – radius of the disk

Returns

DiskMaskAnalysis – When run by the Context, this Analysis generates a libertem.analysis.masks.SingleMaskResultSet.

Return type

libertem.analysis.base.Analysis

create_mask_analysis(factories: Union[Callable[[], Union[ndarray, coo_matrix, dok_matrix]], Iterable[Callable[[], Union[ndarray, coo_matrix, dok_matrix]]]], dataset: DataSet, use_sparse: bool = None, mask_count: int = None, mask_dtype: dtype = None, dtype: dtype = None) → MasksAnalysis[source]¶

Create a mask application analysis. Each factory function should, when called, return a numpy array with the same shape as frames in the dataset (so dataset.shape.sig).

This is a more high-level interface than ApplyMasksUDF and differs in the way the result is returned. With ApplyMasksUDF, it is a single numpy array, here we split it up for each mask we apply, make some default visualization available etc.

Parameters

factories (Union[Callable[[], array_like], Iterable[Callable[[], array_like]]]) – Function or list of functions that take no arguments and create masks. The returned masks can be numpy arrays, scipy.sparse or sparse https://sparse.pydata.org/ matrices. The mask factories should not reference large objects because they can create significant overheads when they are pickled and unpickled. If a single function is specified, the first dimension is interpreted as the mask index.
dataset (libertem.io.dataset.base.DataSet) – dataset to work on
use_sparse (bool or None) –
- None (default): Use sparse matrix multiplication if all factory functions return a sparse mask, otherwise convert all masks to dense matrices and use dense matrix multiplication
- True: Convert all masks to sparse matrices.
- False: Convert all masks to dense matrices.
mask_count (int, optional) – Specify the number of masks if a single factory function is used so that the number of masks can be determined without calling the factory function.
mask_dtype (numpy.dtype, optional) – Specify the dtype of the masks so that mask dtype can be determined without calling the mask factory functions. This can be used to override the mask dtype in the result dtype determination. As an example, setting this to np.float32 means that returning masks of type float64 will not switch the calculation and result dtype to float64 or complex128.
dtype (numpy.dtype, optional) – Specify the dtype to do the calculation in. Integer dtypes are possible if the numpy casting rules allow this for source and mask data.

Returns

MasksAnalysis – When run by the Context, this Analysis generates a libertem.analysis.masks.MasksResultSet.

Return type

libertem.analysis.base.Analysis

Examples

>>> # Use intermediate variables instead of referencing
>>> # large complex objects like a dataset within the
>>> # factory function
>>> shape = dataset.shape.sig
>>> analysis = ctx.create_mask_analysis(
...     factories=[lambda: np.ones(shape)],
...     dataset=dataset
... )
>>> result = ctx.run(analysis)
>>> result.mask_0.raw_data.shape
(16, 16)

create_pick_analysis(dataset: DataSet, x: int, y: int = None, z: int = None) → PickFrameAnalysis[source]¶

Create an Analysis that picks a single frame / signal element from (z, y, x). The number of parameters must match number of navigation dimensions in the dataset, for example if you have a 4D dataset with two signal dimensions and two navigation dimensions, you need to specify x and y.

Parameters

dataset – The dataset to work on
x – x coordinate
y – y coordinate
z – z coordinate

Returns

PickFrameAnalysis – When run by the Context, this Analysis generates a libertem.analysis.raw.PickResultSet.

Return type

libertem.analysis.base.Analysis

Examples

>>> dataset = ctx.load(
...     filetype="memory",
...     data=np.zeros([16, 16, 16, 16, 16], dtype=np.float32),
...     sig_dims=2
... )
>>> analysis = ctx.create_pick_analysis(dataset=dataset, x=9, y=8, z=7)
>>> result = ctx.run(analysis)
>>> assert result.intensity.raw_data.shape == tuple(dataset.shape.sig)

create_point_analysis(dataset: DataSet, x: int = None, y: int = None) → PointMaskAnalysis[source]¶

Create an Analysis that selects the pixel with coords (y, x) from each frame

Returns: PointMaskAnalysis – When run by the Context, this Analysis generates a libertem.analysis.masks.SingleMaskResultSet.
Return type: libertem.analysis.base.Analysis

create_radial_fourier_analysis(dataset: DataSet, cx: float = None, cy: float = None, ri: float = None, ro: float = None, n_bins: int = None, max_order: int = None, use_sparse: bool = None) → RadialFourierAnalysis[source]¶

Create an Analysis that calculates the Fourier transform of rings around the center.

See Radial Fourier Series for details on the method!

Parameters

dataset – the dataset to work on
cx – center x value
cy – center y value
ri – inner radius
ro – outer radius
n_bins – number of bins
max_order – maximum order of calculated Fourier component

Returns

RadialFourierAnalysis – When run by the Context, this Analysis generates a libertem.analysis.radialfourier.RadialFourierResultSet.

Return type

libertem.analysis.base.Analysis

create_ring_analysis(dataset: DataSet, cx: int = None, cy: int = None, ri: int = None, ro: int = None) → RingMaskAnalysis[source]¶

Create an Analysis that integrates over a ring.

Parameters

dataset – the dataset to work on
cx – center x value
cy – center y value
ri – inner radius
ro – outer radius

Returns

RingMaskAnalysis – When run by the Context, this Analysis generates a libertem.analysis.masks.SingleMaskResultSet.

Return type

libertem.analysis.base.Analysis

create_sum_analysis(dataset) → SumAnalysis[source]¶

Create an Analysis that sums all signal elements along the navigation dimension, preserving the signal dimension.

Parameters: dataset – the dataset to work on
Returns: SumAnalysis – When run by the Context, this Analysis generates a libertem.analysis.sum.SumResultSet.
Return type: libertem.analysis.base.Analysis

display(dataset: DataSet, udf: UDF, roi: Optional[Union[ndarray, SparseArray, spmatrix, Tuple[int, ...], Iterable[Tuple[Tuple[int, ...], bool]]]] = None)[source]¶: Show information about the UDF in combination with the given DataSet.

export_dataset(dataset: DataSet, *, path: PathLike, progress: bool = False, overwrite: bool = False)[source]¶

Export the dataset to another format on disk

At this time can only convert to numpy .npy format, but future extensions are possible.

The written data will have any reshaping / sync_offset properties of the DataSet effectively baked into the new file.

Note

All workers used by the Context must have access to the same file path, over a network file system if necessary, in order to correctly save data to the file.

New in version 0.12.0.

Parameters

dataset (lt.Dataset) – The dataset to save to disk
path (os.PathLike) – The file path to export the data to, will raise ValueError if the suffix is unrecognized (currently supports only .npy)
progress (bool, optional) – Whether to display a progress bar for the export, by default False
overwrite (bool , optional) – If the save path already exists, raise FileExistsError unless overwrite is True, by default False

Raises

ValueError – If the path suffix is not supported
FileExistsError – If overwrite is True and the save path exists

load(filetype: str, *args, io_backend=None, **kwargs) → DataSet[source]¶

Load a DataSet. As it doesn’t load the whole data into RAM at once, you can load and process datasets that are bigger than your available RAM. Using fast storage (i.e. SSD) is advisable.

Changed in version 0.5.0: Added support for filetype=”auto”

Changed in version 0.6.0: Added support for specifying the I/O backend

Parameters

filetype (str) – one of: hdf5, raw, raw_csr, mib, blo, k2is, ser, frms6, empad, dm, seq, mrc, tvips, npy, dask, memory; or use “auto” to automatically determine filetype and parameters
io_backend (IOBackend or None) – Use a different I/O backend for this data set
args – passed on to the DataSet implementation
kwargs – passed on to the DataSet implementation

Returns

DataSet – The loaded dataset

Return type

libertem.io.dataset.base.DataSet

Note

Additional parameters are passed to the concrete DataSet implementation.

Note

See Data Set API for format-specific documentation.

Examples

Load a data set from a given path, automatically determinig the type:

>>> ds = ctx.load("auto", path="...")  

To configure an alternative I/O backend, in this case configuring the mmap backend to enable readahead hints:

>>> from libertem.io.dataset.base import MMapBackend
>>> io_backend = MMapBackend(enable_readahead_hints=True)
>>> ds = ctx.load("auto", path="...", io_backend=io_backend)  

classmethod make_with(executor_spec: Literal['synchronous', 'inline', 'threads', 'dask', 'dask-integration', 'dask-make-default', 'delayed', 'pipelined'] = 'dask', *, cpus: Optional[Union[int, Iterable[int]]] = None, gpus: Optional[Union[int, Iterable[int]]] = None, plot_class: Optional[Live2DPlot] = None) → Context[source]¶

Create a Context with a specific kind of executor.

New in version 0.9.0.

This simplifies creating a Context for a number of common executor choices and allows specification of the resources used to create the executor. For more fine-grained control of resource allocation create the executor manually and pass it to the Context.

See Executors for general information on executors.

Changed in version 0.12.0: Prior to version 0.12.0, this function accepted *args, **kwargs and passed them to the initializer of Context. Given that the Context accepts only the plot_class keyword-argument this was hard-coded into this function for backwards-compatibility, enabling the addition of the cpus and gpus parameters.

Parameters

executor_spec (ExecutorSpecType, optional, by default "dask") –
A string identifier for executor variants:

”synchronous”, “inline”:
Use a single-process, single-threaded InlineJobExecutor

”threads”:
Use a multi-threaded ConcurrentJobExecutor

”dask”:
Create a standard DaskJobExecutor without considering any pre-existing Dask schedulers available on the system, similar to the default behaviour of Context() called with no arguments.

”dask-integration”:
Use a JobExecutor that is compatible with the currently active Dask scheduler. See get_dask_integration_executor() for more information.

”dask-make-default”:
Create a local dask.distributed cluster and client using make_local(). The Client will be set as the default Dask scheduler and will persist after the LiberTEM Context closes, which is suitable for downstream computation using dask.distributed.

”delayed”:
Create a DelayedJobExecutor which performs computation using dask.delayed. This functionality is highly experimental at this time, see Create Dask arrays with UDFs for more information.

”pipelined”:
Create a PipelinedExecutor, which is suitable for multi-process streaming live processing using LiberTEM-live
cpus (int | Iterable[int], optional) – The number of CPU workers to create, where possible. The meaning of CPU worker depends on the type of executor created - threaded executors will interpret cpus to choose the number of threads, while process-based executors will interpret the value to choose the number of processes to spawn. The iterable form of of the argument is intended contain CPU-id values to enable cpu-pinning where the exector and the system support it, but most use cases will only require an integer argument. Executors where this parameter does not make sense will raise an error if provided. When unspecified the default executor behaviour is retained.
gpus (int | Iterable[int], optional) – Similar to cpus, specifies the number of GPU workers to create where the executor chosen supports it, else raise an error. The integer form of the argument specifies the total number of workers to create, assigned according to the implementation of each executor, while the iterable form allows assigment of multiple workers to specific GPUs by repeating the id of any given GPU in the argument. When unspecified the default executor behaviour is retained.
plot_class (libertem.viz.base.Live2DPlot, optional) – Plot class for live plotting, passed to Context.

Raises

libertem.exceptions.ExecutorSpecException : – for invalid executor choice or unsupported worker specifications

Return type

Instance of Context using a new instance of the specified executor.

map(dataset: DataSet, f, roi: Optional[Union[ndarray, SparseArray, spmatrix, Tuple[int, ...], Iterable[Tuple[Tuple[int, ...], bool]]]] = None, progress: Union[bool, ProgressReporter] = False, corrections: CorrectionSet = None, backends=None) → BufferWrapper[source]¶

Create an AutoUDF with function f() and run it on dataset

Changed in version 0.5.0: Added the progress parameter

Changed in version 0.6.0: Added the corrections and backends parameter

Parameters

dataset – The dataset to work on
f – Function that accepts a frame as the only parameter. It should return a strongly reduced output compared to the size of a frame.
roi (numpy.ndarray, sparse array or coordinate tuple(s), optional) – Region of interest as bool mask over the navigation axes of the dataset. See Regions of interest.
progress (bool | ProgressReporter) – Show progress bar. Toggle with boolean flag or supply instance of libertem.common.progress.ProgressReporter for custom handling of progress display.
corrections – Corrections to apply while running the function. If none are given, the corrections that are part of the DataSet are used, if there are any. See also Corrections.
backends (None or iterable containing 'numpy', 'cupy' and/or 'cuda') – Restrict the back-end to a subset of the capabilities of the UDF. This can be useful for testing hybrid UDFs.

Returns

BufferWrapper – The result of the UDF. Access the underlying numpy array using the data property. Shape and dtype is inferred automatically from f.

Return type

libertem.common.buffers.BufferWrapper

run(job: Analysis, roi: Optional[Union[ndarray, SparseArray, spmatrix, Tuple[int, ...], Iterable[Tuple[Tuple[int, ...], bool]]]] = None, progress: Union[bool, ProgressReporter] = False, corrections: Optional[CorrectionSet] = None) → Union[ndarray, AnalysisResultSet][source]¶

Run the given Analysis and return the result data.

Changed in version 0.5.0: Added the progress parameter

Changed in version 0.6.0: Added the corrections parameter

Changed in version 0.7.0: Removed deprecated Job support, now only UDF-based analyses are supported

Parameters

job – the analysis to run
roi (numpy.ndarray, sparse array or coordinate tuple(s), optional) – Boolean mask of the navigation dimension. See Regions of interest.
progress (bool | ProgressReporter) – Show progress bar. Toggle with boolean flag or supply instance of libertem.common.progress.ProgressReporter for custom handling of progress display.
corrections – Corrections to apply, i.e. dark frame substraction, applying a gain map, …

Returns

result – Running an Analysis returns a libertem.analysis.base.AnalysisResultSet. See the matching create_*_analysis function for documentation of the specific AnalysisResultSet subclass or numpy.ndarray that is being returned.

Return type

libertem.analysis.base.AnalysisResultSet

run_udf(dataset: DataSet, udf: Union[UDF, Iterable[UDF]], roi: Optional[Union[ndarray, SparseArray, spmatrix, Tuple[int, ...], Iterable[Tuple[Tuple[int, ...], bool]]]] = None, corrections: Optional[CorrectionSet] = None, progress: Union[bool, ProgressReporter] = False, backends=None, plots=None, sync=True) → Union[Mapping[str, BufferWrapper], List[Mapping[str, BufferWrapper]], Coroutine[None, None, Mapping[str, BufferWrapper]], Coroutine[None, None, List[Mapping[str, BufferWrapper]]]][source]¶

Run udf on dataset, restricted to the region of interest roi.

Changed in version 0.5.0: Added the progress parameter

Changed in version 0.6.0: Added the corrections and backends parameter

Changed in version 0.7.0: Added the plots and sync parameters, and the ability to run multiple UDFs on the same data in a single pass.

Raises

UDFRunCancelled – Either the run was cancelled using AsyncJobExecutor.cancel(), or the underlying data source was interrupted.

Parameters

dataset – The dataset to work on
udf – UDF instance you want to run, or a list of UDF instances
roi (numpy.ndarray, sparse array or coordinate tuple(s), optional) – Region of interest as bool mask over the navigation axes of the dataset. See Regions of interest.
progress (bool | ProgressReporter) – Show progress bar. Toggle with boolean flag or supply instance of libertem.common.progress.ProgressReporter for custom handling of progress display.
corrections – Corrections to apply while running the UDF. If none are given, the corrections that are part of the DataSet are used, if there are any. See also Corrections.
backends (None or iterable containing 'numpy', 'cupy' and/or 'cuda') – Restrict the back-end to a subset of the capabilities of the UDF. This can be useful for testing hybrid UDFs.
plots (None or True or List[List[Union[str, Tuple[str, Callable]]]] or List[LivePlot]) –
- None: don’t plot anything (default)
- True: plot all 2D UDF result buffers
- List[List[...]]: plot the named UDF buffers. Pass a list of names or (name, callable) tuples for each UDF you want to plot. If the callable is specified, it is applied to the UDF buffer before plotting.
- List[LivePlot]: LivePlot instance for each channel you want to plot
New in version 0.7.0.
sync (bool) –
By default, run_udf is a synchronous method. If sync is set to False, it is awaitable instead.

New in version 0.7.0.

Returns

Return value of the UDF containing the result buffers of type libertem.common.buffers.BufferWrapper. Note that a BufferWrapper can be used like a numpy.ndarray in many cases because it implements __array__(). You can access the underlying numpy array using the data property.

If a list of UDFs was passed in, the returned type is a Tuple[dict[str,BufferWrapper], …].

Return type

dict or Tuple[dict, …]

Examples

Run the SumUDF on a data set:

>>> from libertem.udf.sum import SumUDF
>>> result = ctx.run_udf(dataset=dataset, udf=SumUDF())
>>> np.array(result["intensity"]).shape
(32, 32)
>>> # intensity is the name of the result buffer, defined in the SumUDF

Running a UDF on a subset of data:

>>> from libertem.udf.sumsigudf import SumSigUDF
>>> roi = np.zeros(dataset.shape.nav, dtype=bool)
>>> roi[0, 0] = True
>>> result = ctx.run_udf(dataset=dataset, udf=SumSigUDF(), roi=roi)
>>> # to get the full navigation-shaped results, with NaNs where the `roi` was False:
>>> np.array(result["intensity"]).shape
(16, 16)
>>> # to only get the selected results as a flat array:
>>> result["intensity"].raw_data.shape
(1,)

run_udf_iter(dataset: DataSet, udf: Union[UDF, Iterable[UDF]], roi: Optional[Union[ndarray, SparseArray, spmatrix, Tuple[int, ...], Iterable[Tuple[Tuple[int, ...], bool]]]] = None, corrections: CorrectionSet = None, progress: Union[bool, ProgressReporter] = False, backends=None, plots=None, sync=True) → Union[ResultGenerator, ResultAsyncGenerator][source]¶

Run udf on dataset, restricted to the region of interest roi. Yields partial results after each merge operation.

New in version 0.7.0.

Parameters

dataset – The dataset to work on
udf – UDF instance you want to run, or a list of UDF instances
roi (numpy.ndarray, sparse array or coordinate tuple(s), optional) – Region of interest as bool mask over the navigation axes of the dataset. See Regions of interest.
progress (bool | ProgressReporter) – Show progress bar. Toggle with boolean flag or supply instance of libertem.common.progress.ProgressReporter for custom handling of progress display.
corrections – Corrections to apply while running the UDF. If none are given, the corrections that are part of the DataSet are used, if there are any. See also Corrections.
backends (None or iterable containing 'numpy', 'cupy' and/or 'cuda') – Restrict the back-end to a subset of the capabilities of the UDF. This can be useful for testing hybrid UDFs.
plots (None or True or List[List[Union[str, Tuple[str, Callable]]]] or List[LivePlot]) –
- None: don’t plot anything (default)
- True: plot all 2D UDF result buffers
- List[List[...]]: plot the named UDF buffers. Pass a list of names or (name, callable) tuples for each UDF you want to plot. If the callable is specified, it is applied to the UDF buffer before plotting.
- List[LivePlot]: LivePlot instance for each channel you want to plot
sync (bool) – By default, run_udf_iter is a synchronous method. If sync is set to False, an async generator will be returned instead.

Returns

Generator of UDFResults container objects. Their attribute buffers is the list of result buffer dictionaries for the UDFs. Attribute damage is a BufferWrapper of kind='nav', dtype=bool indicating the positions in nav space that have been processed already.

Return type

Generator[UDFResults]

Examples

Run the SumUDF on a data set:

>>> from libertem.udf.sum import SumUDF
>>> for result in ctx.run_udf_iter(dataset=dataset, udf=SumUDF()):
...     assert np.array(result.buffers[0]["intensity"]).shape == (32, 32)
>>> np.array(result.buffers[0]["intensity"]).shape
(32, 32)
>>> # intensity is the name of the result buffer, defined in the SumUDF

class libertem.api.ResultAsyncGenerator(result_generator: ResultGenerator)[source]¶

async wrapper of ResultGenerator.

async update_parameters_experimental(parameters: List[Dict[str, Any]])[source]¶

Update parameters while the UDFs are running.

parameters should be a list of dicts, with one dict for each UDF you are running.

The dicts should only contain items for those parameters you want to update.

class libertem.api.ResultGenerator(task_results: Generator[UDFResults, None, None], runner: UDFRunner, result_iter)[source]¶

Yields partial results from one or more UDFs currently running, with methods to control some aspects of the UDF run.

update_parameters_experimental(parameters: List[Dict[str, Any]])[source]¶

Update parameters while the UDFs are running.

parameters should be a list of dicts, with one dict for each UDF you are running.

The dicts should only contain items for those parameters you want to update.

libertem.api.RunUDFAGenType¶: alias of ResultAsyncGenerator

libertem.api.RunUDFAGenTypeL¶: alias of ResultAsyncGenerator

libertem.api.RunUDFGenType¶: alias of ResultGenerator

libertem.api.RunUDFGenTypeL¶: alias of ResultGenerator

Analysis API¶

class libertem.analysis.base.Analysis(dataset: DataSet, parameters: Dict)[source]¶

Abstract base class for Analysis classes.

An Analysis is the interface between a UDF and the Web API, and handles visualization of partial and full results.

Passing an instance of an Analysis sub-class to libertem.api.Context.run() will generate an AnalysisResultSet. The content of this result set is governed by the specific implementation of the Analysis sub-class.

New in version 0.3.0.

Changed in version 0.7.0: Removed deprecated methods get_results and get_job

get_parameters(parameters: Dict) → Dict[source]¶: Get analysis parameters. Override to set defaults

get_roi() → Optional[ndarray][source]¶

Get the region of interest the UDF should be run on. For example, the parameters could describe some geometry, which this method should convert to a boolean array. See also: libertem.analysis.getroi.get_roi()

Returns: region of interest for which we want to run our analysis
Return type: numpy.ndarray or None

get_udf() → UDF[source]¶: Set TYPE=’UDF’ on the class and implement this method to run a UDF from this analysis

get_udf_results(udf_results: Mapping[str, BufferWrapper], roi: Optional[ndarray], damage: nt.ArrayLike) → AnalysisResultSet[source]¶

Convert UDF results to a AnalysisResultSet, including visualizations.

Parameters

udf_results – raw results from the UDF
roi (numpy.ndarray or None) – Boolean array of the navigation dimension

Returns

one or more annotated results

Return type

list of AnalysisResult

need_rerun(old_params: Dict, new_params: Dict) → bool[source]¶

Determine if the analysis needs to be re-run on the data. If not, we can just call get_udf_results again, for example if the parameters only change the visualization.

Parameters

old_params (Dict) –
new_params (Dict) –

Returns

True iff the parameter change needs to cause a re-run on the data

Return type

bool

class libertem.analysis.base.AnalysisResult(raw_data: ndarray, visualized: ndarray, title: str, desc: str, key: str, include_in_download: bool = True)[source]¶

This class represents a single 2D image result from an Analysis.

Instances of this class are contained in an AnalysisResultSet.

raw_data¶

The raw numerical data of this result

Type: numpy.ndarray

visualized¶

Visualized result as numpy.ndarray with RGB or RGBA values

Type: numpy.ndarray

title¶

Title for the GUI

Type: str

desc¶

Short description in the GUI

Type: str

key¶

Key to identify the result in an AnalysisResultSet

Type: str

class libertem.analysis.base.AnalysisResultSet(results: Union[List[AnalysisResult], Callable[[], List[AnalysisResult]]], raw_results=None)[source]¶

Base class for Analysis result sets. libertem.api.Context.run() returns an instance of this class or a subclass. Many of the subclasses are just introduced to document the Analysis-specific results (keys) of the result set and don’t introduce new functionality.

The contents of an AnalysisResultSet can be addressed in four different ways:

As a container class with the AnalysisResult.key properties as attributes of type AnalysisResult.
As a list of AnalysisResult objects.
As an iterator of AnalysisResult objects (since 0.3).
As a dictionary of AnalysisResult objects with the AnalysisResult.key properties as keys.

This allows to implement generic result handling code for an Analysis, for example GUI display, as well as specific code for particular Analysis subclasses.

raw_results¶: Raw results from the underlying numerical processing, if available, otherwise None.

Examples

>>> mask_shape = tuple(dataset.shape.sig)
>>> def m0():
...    return np.ones(shape=mask_shape)
>>> def m1():
...     result = np.zeros(shape=mask_shape)
...     result[0,0] = 1
...     return result
>>> analysis = ctx.create_mask_analysis(
...     dataset=dataset, factories=[m0, m1]
... )
>>> result = ctx.run(analysis)
>>> # As an object with attributes
>>> print(result.mask_0.title)
mask 0
>>> print(result.mask_1.title)
mask 1
>>> # As a list
>>> print(result[0].title)
mask 0
>>> # As an iterator
>>> # New since 0.3.0
>>> for m in result:
...     print(m.title)
mask 0
mask 1
>>> # As a dictionary
>>> print(result['mask_1'].title)
mask 1

class libertem.analysis.masks.MasksResultSet(results: Union[List[AnalysisResult], Callable[[], List[AnalysisResult]]], raw_results=None)[source]¶

Running a MasksAnalysis via libertem.api.Context.run() on a dataset returns an instance of this class.

If any of the masks or the dataset contain complex numbers, the regular mask results attributes carry the absolute value of the results, and additional attributes with real part, imaginary part, phase and full complex result are available.

New in version 0.3.0.

mask_0, mask_1, ..., mask_<n>

For dataset and masks containing only real numbers: Results of the element-wise multiplication and sum of each individual mask with each detector frame. Each mask result has the shape of the navigation dimension. These keys contain the absolute value of the result if dataset or masks contain complex numbers.

Type: libertem.analysis.base.AnalysisResult

mask_0, mask_0_real, mask_0_imag, mask_0_angle, mask_0_complex, mask_1, mask_1_real, ..., mask_<n>, ..., mask_<n>_complex

If masks or dataset contain complex numbers: Absolute, real part, imaginary part, phase angle, complex result of the element-wise multiplication and sum of each individual mask with each detector frame. Each mask result has the shape of the navigation dimension.

Type: libertem.analysis.base.AnalysisResult

class libertem.analysis.masks.SingleMaskResultSet(results: Union[List[AnalysisResult], Callable[[], List[AnalysisResult]]], raw_results=None)[source]¶

A number of Analyses that are based on applying a single mask create an instance of this class as a result when executed via libertem.api.Context.run().

If the dataset contains complex numbers, the regular result attribute carries the absolute value of the result, and additional attributes with real part, imaginary part, phase and full complex result are available.

New in version 0.3.0.

intensity¶

Sum of the selected region for each detector frame, with shape of the navigation dimension. Absolute of the result if the dataset or mask contains complex numbers.

Type: libertem.analysis.base.AnalysisResult

intensity_log¶

Log-scaled sum of the selected region for each detector frame, with shape of the navigation dimension. Absolute of the result if the dataset or mask contains complex numbers.

New in version 0.6.0.

Type: libertem.analysis.base.AnalysisResult

intensity_real¶

Real part of the sum of the selected region. This is only available if the dataset or mask contains complex numbers.

Type: libertem.analysis.base.AnalysisResult

intensity_imag¶

Imaginary part of the sum of the selected region. This is only available if the dataset contains complex numbers.

Type: libertem.analysis.base.AnalysisResult

intensity_angle¶

Phase angle of the sum of the selected region. This is only available if the dataset or mask contains complex numbers.

Type: libertem.analysis.base.AnalysisResult

intensity_complex¶

Complex result of the sum of the selected region. This is only available if the dataset or mask contains complex numbers.

Type: libertem.analysis.base.AnalysisResult

class libertem.analysis.com.COMResultSet(results: Union[List[AnalysisResult], Callable[[], List[AnalysisResult]]], raw_results=None)[source]¶

Running a COMAnalysis via libertem.api.Context.run() on a dataset returns an instance of this class.

This analysis is usually applied to datasets with real values. If the dataset contains complex numbers, this result contains the keys x_real, y_real, x_imag, y_imag instead of the vector field.

By default, the shift is given in pixel coordinates, i.e. positive x shift goes to the right and positive y shift goes to the bottom. See also Concepts.

Changed in version 0.6.0: The COM analysis now supports flipping the y axis and rotating the vectors.

New in version 0.3.0.

field¶

Center of mass shift relative to the center given to the analysis within the given radius as a vector field with components (x, y). The visualized result uses a cubehelix color wheel.

Type: libertem.analysis.base.AnalysisResult

magnitude¶

Magnitude of the center of mass shift.

Type: libertem.analysis.base.AnalysisResult

divergence¶

Divergence of the center of mass vector field at a given point

Type: libertem.analysis.base.AnalysisResult

curl¶

Curl of the center of mass 2D vector field at a given point.

New in version 0.6.0.

Type: libertem.analysis.base.AnalysisResult

x¶

X component of the center of mass shift

Type: libertem.analysis.base.AnalysisResult

y¶

Y component of the center of mass shift

Type: libertem.analysis.base.AnalysisResult

x_real¶

Real part of the x component of the center of mass shift (complex dataset only)

Type: libertem.analysis.base.AnalysisResult

y_real¶

Real part of y component of the center of mass shift (complex dataset only)

Type: libertem.analysis.base.AnalysisResult

x_imag¶

Imaginary part of the x component of the center of mass shift (complex dataset only)

Type: libertem.analysis.base.AnalysisResult

y_imag¶

Imaginary part of y component of the center of mass shift (complex dataset only)

Type: libertem.analysis.base.AnalysisResult

class libertem.analysis.sum.SumResultSet(results: Union[List[AnalysisResult], Callable[[], List[AnalysisResult]]], raw_results=None)[source]¶

Running a SumAnalysis via libertem.api.Context.run() returns an instance of this class.

If the dataset contains complex numbers, the regular result attribute carries the absolute value of the result and additional attributes with real part, imaginary part, phase and full complex result are available.

New in version 0.3.0.

intensity¶

Sum of all detector frames along the navigation dimension, preserving the signal dimension. Absolute value of the sum if the dataset contains complex numbers. Log-scaled visualization.

Type: libertem.analysis.base.AnalysisResult

intensity_lin¶

Sum of all detector frames along the navigation dimension, preserving the signal dimension. Absolute value of the sum if the dataset contains complex numbers. Lin-scaled visualization.

New in version 0.6.0.

Type: libertem.analysis.base.AnalysisResult

intensity_real¶

Real part of the sum of all detector frames along the navigation dimension, preserving the signal dimension. This is only available if the dataset contains complex numbers.

Type: libertem.analysis.base.AnalysisResult

intensity_imag¶

Imaginary part of the sum of all detector frames along the navigation dimension, preserving the signal dimension. This is only available if the dataset contains complex numbers.

Type: libertem.analysis.base.AnalysisResult

intensity_angle¶

Phase angle of the sum of all detector frames along the navigation dimension, preserving the signal dimension. This is only available if the dataset contains complex numbers.

Type: libertem.analysis.base.AnalysisResult

intensity_complex¶

Complex result of the sum of all detector frames along the navigation dimension, preserving the signal dimension. This is only available if the dataset contains complex numbers.

Type: libertem.analysis.base.AnalysisResult

class libertem.analysis.raw.PickResultSet(results: Union[List[AnalysisResult], Callable[[], List[AnalysisResult]]], raw_results=None)[source]¶

Running a PickFrameAnalysis via libertem.api.Context.run() returns an instance of this class.

If the dataset contains complex numbers, the regular result attribute carries the absolute value of the result and additional attributes with real part, imaginary part, phase and full complex result are available.

New in version 0.3.0.

Changed in version 0.4.0: Picking now returns data in the native dtype of the dataset with the new UDF back-end.

intensity¶

The specified detector frame. Absolute value if the dataset contains complex numbers. Log-scaled visualization.

Type: libertem.analysis.base.AnalysisResult

intensity_lin¶

The specified detector frame. Absolute value if the dataset contains complex numbers. Linear visualization.

New in version 0.6.0.

Type: libertem.analysis.base.AnalysisResult

intensity_real¶

Real part of the specified detector frame. This is only available if the dataset contains complex numbers.

Type: libertem.analysis.base.AnalysisResult

intensity_imag¶

Imaginary part of the specified detector frame. This is only available if the dataset contains complex numbers.

Type: libertem.analysis.base.AnalysisResult

intensity_angle¶

Phase angle of the specified detector frame. This is only available if the dataset contains complex numbers.

Type: libertem.analysis.base.AnalysisResult

intensity_complex¶

Complex result of the specified detector frame. This is only available if the dataset contains complex numbers.

Type: libertem.analysis.base.AnalysisResult