Dask API
LiberTEM can generate an efficient Dask array from data sets. See Load datasets as Dask arrays for details!
- libertem.contrib.daskadapter.make_dask_array(dataset: DataSet, dtype='float32', roi=None, array_backend: Literal['numpy', 'numpy.matrix', 'cuda', 'cupy', 'sparse.COO', 'sparse.GCXS', 'sparse.DOK', 'scipy.sparse.coo_matrix', 'scipy.sparse.csr_matrix', 'scipy.sparse.csc_matrixcupyx.scipy.sparse.coo_matrix', 'cupyx.scipy.sparse.csr_matrix', 'cupyx.scipy.sparse.csc_matrix'] | None = None)[source]
Create a Dask array using the DataSet’s partitions as blocks.
Dask result buffer
- class libertem.executor.utils.dask_buffer.DaskResultBufferWrapper(data, *args, **kwargs)[source]
A DaskBufferWrapper that is meant to be used for results.
In the LiberTEM internals, we need to make sure everything is still lazy/delayed.
Once we leave LiberTEM internals, the external interface is used, where both
data()andraw_data()eagerly return numpy array-like objects.- property data
Eager version of
delayed_data()
- property delayed_data
- property delayed_raw_data
- classmethod from_buffer_wrapper(buffer: BufferWrapper)[source]
- property raw_data
Eager version of
delayed_raw_data()