Workshop on Reproducible and Sustainable HPC Benchmarking
Details
Description
The Workshop on Reproducible and Sustainable HPC Benchmarking brings together experts from HPC centers and vendors around the world to present and discuss similarities and differences among reproducibility and sustainabiity of test frameworks for HPC systems. There is a increasingly diverse set of supercomputers currently in production and the landscape of supercomputer architectures is rapidly evolving due to the demand for more capable and more powerful high performance computing (HPC) systems. The diversity of architectures coupled with the distinct user workloads supported at each HPC center hinders reproducibility and sustainability of testing frameworks, resulting in each individual center developing their own procedures, tests, benchmarks, and tools to conduct testing. However, there is a similarity among the various solutions, and the tests utilized could be leveraged by multiple centers and benefit the community at large.
The workshop will include a series of paper presentations from peer-reviewed accepted submissions followed by a hands on component.
Call for Papers
There is a diverse set of supercomputers currently in production and the landscape of supercomputer architectures is rapidly evolving due to the demand for more capable and more powerful high performance computing (HPC) systems. The diversity of architectures coupled with the distinct user workloads supported at each HPC center hinders reproducibility and sustainability of testing frameworks, resulting in each individual center developing their own procedures, tests, benchmarks, and tools to conduct testing. However, there is a similarity among the various solutions, and the tests utilized could be leveraged by multiple centers and benefit the community at large.
This combined presentation and hands on format workshop will bring together experts from HPC centers and vendors around the world to present and discuss similarities and differences among reproducibility and sustainabiity of test frameworks for HPC systems. The workshop will encourage submissions highlighting current benchmarks, reproducibility tests, and procedures utilized in today’s HPC systems. This workshop provides an opportunity to showcase newly developed tools and methodologies, as well as tools and methodologies actively being designed to allow authors to gather input and feedback from the community to help improve their software. The event will invite and welcome international participation from HPC centers, academic institutions, and representatives from vendors in the supercomputing space. In addition to discussing procedures and tools utilized, submissions can describe challenges, lessons learned, and best practices used for reproducibility testing and sustainability evaluation.
Topics of interest include, but are not limited to:
Testing methodologies and procedures for future reproducibility
Experiment portability for code, performance, and related metrics
Reproducibility-aware testing infrastructure
Automation of testing and continuous regression testing and performance monitoring
Efforts to improve reproducibility and sustainability of tests that can be leveraged by the community.
Hardware and component focused testing (compute, memory, network, storage) at all scales from a single server (CPU/GPU) to clusters and cloud environments
System software, programming languages, and library testing
Best practices and lessons
Paper Submissions
The workshop will publish its proceedings with the ICS 2026 conference. Authors must
follow the same formatting guidelines as main conference papers (ACM template,
\documentclass[sigconf]{acmart} in LaTeX). The formatting guidelines for ICS 2026
Papers are available here.
Submissions can be in the form of a short or full paper (at most 4 pages for a short
paper, at least 5 pages for a full one).
More details to come about the submission system.
Workshop Deadlines
Paper Submission Deadline: April 5, 2026
Author Notification: April 22, 2026
Camera-Ready Upload: May 6, 2026
Organizing Committee
Workshop 2026 Organizers
Olga Pearce, Lawrence Livermore National Laboratory, USA
Robert Bird, Google, USA
Sunita Chandrasekaran, University of Delaware, USA
Tuomas Koskela, University College London, UK
Workshop 2026 Program Committee
Robert Bird, Google, USA
Sunita Chandrasekaran, University of Delaware, USA
Jens Domke, RIKEN Center for Computational Science, Japan
Andreas Herten, Juelich Supercomputing Centre, Germany
Doug Jacobsen, Google, USA
Tuomas Koskela, University College London, UK
Olga Pearce, Lawrence Livermore National Laboratory, USA
Keita Teranishi, Oak Ridge National Laboratory, USA