Thomas Jefferson National Accelerator Facility (Jefferson Lab) and the Energy Sciences Network (ESnet) achieved a groundbreaking milestone by streaming nuclear physics data in real time across the United States. Using ESnet6’s high-speed backbone, raw data from a nuclear physics experiment at Jefferson Lab in Virginia was transmitted 3,000 miles to the National Energy Research Scientific Computing Center (NERSC) at Lawrence Berkeley National Laboratory in California. The data, which flowed at a rate of 100 Gbps, was processed by more than 10,000 cores on NERSC’s Perlmutter supercomputer and sent back to Jefferson Lab for analysis—all without buffering or data loss. This proof-of-concept test showcases a major leap in data streaming for large-scale scientific research, which has traditionally relied on time-consuming storage and retrieval methods.
The successful test, which was conducted earlier this year, was a result of nearly three years of collaboration between Jefferson Lab and ESnet, and it marks a significant breakthrough in connecting scientific instruments to remote high-performance computing (HPC) facilities. The core of the solution is a novel hardware prototype, the ESnet-JLab FPGA Accelerated Transport (EJFAT) device, which uses traffic-shaping and load-balancing technology to handle real-time data processing. This device allows scientific instruments across different locations to operate as a single, integrated system, paving the way for future advancements in research infrastructures like the Department of Energy’s Integrated Research Infrastructure (IRI) initiative.
Jefferson Lab, known for its Continuous Electron Beam Accelerator Facility (CEBAF), plays a crucial role in nuclear physics research. The lab’s accelerators generate large volumes of data that need to be processed quickly and efficiently, especially for the upcoming Electron-Ion Collider. By integrating technologies like EJFAT, Jefferson Lab aims to enhance its ability to process and analyze data streams in real time, which is essential for complex scientific experiments requiring low-latency responses. This collaboration with ESnet and other labs could revolutionize how scientific data is managed and processed in future research.
Key points:
• Achievement: Real-time streaming of physics data from Jefferson Lab to NERSC at 100 Gbps
• Technology: ESnet6 backbone and EJFAT device for real-time data processing without buffering
• Application: Advanced data processing for nuclear physics experiments and future projects like the Electron-Ion Collider
• Innovations: Remote high-performance computing integration, load balancing, and traffic shaping across the DOE’s research infrastructure
• Future impact: Potential to transform large-scale scientific workflows and data management across multiple facilities
“This proof-of-concept test shows that real-time data streaming and processing on a continental scale is not only possible but also scalable,” said Graham Heyes, Technical Director of the High Performance Data Facility (HPDF) at Jefferson Lab. “This breakthrough will be a game-changer for how we handle and process scientific data.”
