Fig. 1

Streaming pipeline: Overview of the real-time streaming framework of Nanosurveyor. The modular server–client infrastructure is divided into a back-end (running the data processing unit) and a front-end (running the visualization and control unit). The data flow is depicted as a red arrow, while communication channels for controlling experiment and back-end are shown in gray. Once an experiment has started (trigger signal), the data collection unit continuously receives new data packets from a detector and sends raw data frames to the data processing unit. Depending on the specific needs of the experiment, different modules (from dark calibration to data writing) can be plugged into the pipeline. At all times, there is an active connection (asynchronous socket communication) between all components (including the visualization interface) allowing the scientist to monitor progress while data are still being acquired and processed