Get a frame stream from a merlin detector

This notebook shows how you can access the stream of frames from a Merlin Medipix detector, without using the high-level features of LiberTEM, like user-defined functions (UDFs). You get access to the data in a chunked fashion, that is, you’ll always get a small frame stack to work on.

Usage with the simulator

If you want to use this with the simulated data source, run something like this in the background:

libertem-live-mib-sim ~/Data/default.hdr --cached=MEM --wait-trigger

The --wait-trigger option is important for this notebook to function correctly since that allows to drain the data socket before an acquisition like it is necessary for a real-world Merlin detector.

A suitable MIB dataset can be downloaded at https://zenodo.org/record/5113449.

On Linux, MEMFD is also supported as a cache. Use NONE to deactivate the cache.

# set this to the host/port where the merlin data server is listening:
MERLIN_DATA_SOCKET = ('127.0.0.1', 6342)
MERLIN_CONTROL_SOCKET = ('127.0.0.1', 6341)

%matplotlib nbagg

import time
import logging
import numpy as np
import matplotlib.pyplot as plt

from libertem_live.detectors.merlin import MerlinDataSource
from libertem_live.detectors.merlin import MerlinControl

data_source = MerlinDataSource(host=MERLIN_DATA_SOCKET[0], port=MERLIN_DATA_SOCKET[1], pool_size=3)
control = MerlinControl(host=MERLIN_CONTROL_SOCKET[0], port=MERLIN_CONTROL_SOCKET[1])

result = np.zeros((256, 256), dtype=np.float32)
result_nav = np.zeros(128 * 128, dtype=np.float32)

# connect the control and data sockets, and close after the block automatically:
with control, data_source:
    # Drain data from previous acquisitions out of the data socket,
    # which could for example happen if a connection to the data socket was cancelled.
    # Draining might not be necessary with current Merlin software anymore.
    data_source.socket.drain()

    # Tell the detector to send us some data:
    # (in a real setup, you probably want to properly configure triggering,
    # or even start the microscope scan here)
    control.cmd('STARTACQUISITION')
    control.cmd('SOFTTRIGGER')

    index = 0

    for frames in data_source.stream(
        # How many frames do we expect in total?
        # If you put a too large number here, the loop will possibly block;
        # a too small number means you won't get the data for the whole scan
        # (should match the `result_nav` shape above)
        num_frames=result_nav.size,

        # Get frames in chunks of N, which can have an effect on performance
        # (too small or too large chunks are slower than the optimum, which depends
        # on the CPU you are using):
        chunk_size=16,
    ):
        result += frames.buf.sum(axis=0)
        real_size = frames.buf.shape[0]
        result_nav[index:index+real_size] = frames.buf.sum(axis=(1, 2))
        index += real_size

fig, axes = plt.subplots(1, 2)
axes[0].imshow(np.log1p(result))
axes[1].imshow(np.log1p(result_nav.reshape((128, 128))))

<matplotlib.image.AxesImage at 0x7f0682aecd30>