In a significant step towards operational readiness, the ESS Data Management and Software Centre (DMSC) recently showcased the full capabilities of the ESS data processing pipeline in a live online demonstration. The demonstration highlighted the journey from proposal submission to fully analysed and curated results, showcasing ESS's ability to handle and process large datasets effectively for future users.
The demonstration was presented to the DMSC Science and Technology Advisory Panel (STAP) and aimed to evaluate and validate the readiness of ESS data systems for the facility’s upcoming neutron science operations. Data pipelines for the instruments BIFROST, DREAM, ESTIA, LOKI, and NMX were featured, using simulated data or, where possible, real data from other neutron sources.
Key Highlights of the Demonstration
Proposal Submission and Integration
Users submitted proposals through the ESS user portal, initiating an instance on the VISA remote desktop solution and creating an entry in the SciCat data catalogue.
Data Access and Analysis
After logging into VISA, users could retrieve, reduce, analyse, and visualise data using instrument-specific software, demonstrating a seamless workflow.
End-to-End Data Accessibility
Both proposals and the corresponding generated data were automatically accessible through the metadata catalogue SciCat, emphasising ESS's commitment to open science by making data easily findable and accessible.
To further validate the pipeline’s functionality on actual hardware, the team conducted a light tomography demonstration on YMIR—the test stand located in the B02 workshop building at ESS in Lund, operated and managed by the Experiment Control and Data Curation Group (ECDC). In this demonstration, YMIR served as a stand-in for ODIN, showcasing the pipeline’s compatibility with neutron instrument hardware.
A Milestone in ESS Data Infrastructure Development
This demonstration underscores the growing maturity of ESS’s data management and processing systems, reflecting the collaborative efforts between the ECDC group and DMSC teams. It offers a glimpse into the future, where seamless data handling will enable groundbreaking neutron science.
Thomas H. Rod, Head of the DMSC, shared his thoughts on the achievement:
This milestone not only reflects ESS’s commitment to operational excellence but also ensures that the infrastructure is primed to meet the demands of a world-class neutron facility.