mappgene is a SARS-CoV-2 genomic sequence analysis pipeline designed for parallel HPC. Since the initial release in 2021, updates have included improved memory handling, default support for iVar plus additional parameters, and more.
Learn more:
The Livermore Big Artificial Neural Network toolkit (LBANN) is an open-source, HPC-centric, deep learning training framework that is optimized to compose multiple levels of parallelism. This release includes new training algorithms, new network structures, new layers, updates to the C++ API, and more.
Learn more:
LANL led with LLNL contributors, Charliecloud provides user-defined software stacks for HPC centers. It uses Linux user namespaces to run containers with no privileged operations or daemons and minimal configuration changes on center resources. Version 0.32 includes updates to ch-image, ch-test, ch-run, and other executables.
Please note the team’s request for feedback: We are considering removing ch-ssh, a utility program to facilitate SSH connections from one Charliecloud container into an equivalent container on another host. Please respond to and/or comment on our poll, especially if you use this tool, in discussion #1600.
Learn more:
MFEM is a lightweight, general, scalable C++ library for finite element methods. It enables high-performance scalable finite element discretization research and application development on a wide variety of platforms, from laptops to supercomputers. The v4.5.2 release includes:
Learn more:
Conduit provides an intuitive model for describing hierarchical scientific data in C++, C, Fortran, and Python. It is used for data coupling between packages in-core, serialization, and I/O tasks. The latest release includes DataType support for the Fortran API, updates to DataAccessor and blueprints, and more.
Learn more:
hypre is a library of high-performance preconditioners and solvers featuring multigrid methods for the solution of large, sparse linear systems of equations on massively parallel computers. Version 2.28.0 includes multiple new functions (e.g., matrix scaling, hypre_IntArray, vector resizing), updated docs, and more.
Learn more:
With DJINN (Deep Jointly-Informed Neural Networks), researchers can model complex multiphase equation of state data and respect physics. This code accompanies the Physics of Plasmas paper “Neural network surrogate models for equations of state.” The abstract follows:
Equation of state (EOS) data provide necessary information for accurate multiphysics modeling, which is necessary for fields such as inertial confinement fusion. Here, we suggest a neural network surrogate model of energy and entropy and use thermodynamic relationships to derive other necessary thermodynamic EOS quantities. We incorporate phase information into the model by training a phase classifier and using phase-specific regression models, which improves the modal prediction accuracy. Our model predicts energy values to 1% relative error and entropy to 3.5% relative error in a log-transformed space. Although sound speed predictions require further improvement, the derived pressure values are accurate within 10% relative error. Our results suggest that neural network models can effectively model EOS for inertial confinement fusion simulation applications.
Amazon Web Services (AWS) recently announced the availability of Spack configs for AWS ParallelCluster. Users can use these configurations to install optimized HPC applications quickly and easily on their AWS-powered HPC clusters. Spack configs for AWS ParallelCluster represent validated best practices developed by the AWS HPC Performance Engineering Team. They contain fixes and general optimizations that can increase the performance of any compiled application on a wide range of architectures. Read more in an AWS blog post.
Aluminum provides a generic interface to high-performance communication libraries with a focus on allreduce algorithms. Blocking and non-blocking algorithms and GPU-aware algorithms are supported. Aluminum also contains custom implementations of select algorithms to optimize for certain situations. The latest release adds in-place SendRecv support.
Learn more:
LaSDI (Latent Space Dynamics Identification) enables a fast and accurate solution process on various partial differential equations (i.e., Burgers’ equations, radial advection problem, nonlinear heat conduction problem), achieving more speed-ups and less relative error with respect to the corresponding full order models. The repo includes four examples: 1D Burgers, 2D Burgers, a radial advection example as from MFEM, and a time-dependent diffusion example.
The accompanying repo gLaSDI (“greedy” LaSDI, or Parametric Physics-informed Greedy Latent Space Dynamics Identification) provides a framework for accurate, efficient, and robust data-driven reduced-order modeling of high-dimensional nonlinear dynamical systems. gLaSDI’s autoencoder discovers intrinsic nonlinear latent representations of high-dimensional data, while dynamics identification models capture local latent-space dynamics.
The Elasticsearch Excellence Awards recognize the efforts of visionary teams. LLNL was recognized as a public sector organization leading the way in innovative, sustainable, and critical use cases. The winning entry for the Public Sector Award supports the Lab’s HPC center and enables extreme-scale work across multiple research domains and scientific applications. With a single tool for system security and monitoring, the organization can search large datasets with improved visualizations providing the insight needed to protect its environments.
Coming up on April 21, LLNL computer scientist Vanessa Sochat and Google software engineer Michał Woźniak will present “Enabling HPC and ML Workloads with the Latest Kubernetes Job Features” at KubeCon. Registration is required. The talk abstract follows:
In this talk, we present the new features in Kubernetes Job API and how they can be used to stand up to challenges of running distributed Batch/AI/HPC workloads at scale, based on real-world experiences from DeepMind and the Flux Operator from Lawrence Livermore National Laboratory. We showcase the Indexed Jobs feature by presenting its production use. First, we demonstrate how it simplifies running parallel workloads which require pod-to-pod communication, including distributed machine learning examples based on its use by DeepMind. Next, we demonstrate the orchestration of HPC workloads using the Flux Operator. Here, we create a “Mini Cluster” within Kubernetes built on top of an indexed job, providing a rich ecosystem for orchestration of batch workloads, related user interfaces, and APIs. We also discuss the challenge of handling pod failures for long-running workloads. We show how Pod Failure Policy can be used to continue job execution despite numerous pod disruptions (caused by events such as node maintenance or preemption), yet reduce costs by avoiding unnecessary pod retries when there are software bugs.
Flux is a flexible framework for resource management consisting of a suite of projects, tools, and libraries which may be used to build site-custom resource managers for HPC centers. This version includes support for RFC 36 submission directives, flux-core configuration in ascii-only mode, flux-mini subcommands available as top-level flux commands, and more.
Learn more:
SAMRAI (Structured Adaptive Mesh Refinement Application Infrastructure) is an object-oriented C++ software library that enables exploration of numerical, algorithmic, parallel computing, and software issues associated with applying structured adaptive mesh refinement (SAMR) technology in large-scale parallel application development. The latest release includes RAJA-based kernel fusion features and an option to change how small patches are treated during load balancing.
Learn more:
LLNL security operations team lead Ian Lee recently gave a webinar describing how the Lab uses Elasticsearch for HPC. The webinar is available to watch on demand (19:27); the abstract follows:
High Performance Computing (HPC) systems generate massive amounts of data and logs. In addition, the retention requirements are only increasing to ensure data remains available for incident response, audits, and other business needs. Ingesting and making sense of all the data takes a correspondingly large amount of computing power and storage. With El Capitan, a 2 Exaflop computer arriving and being deployed at LLNL in 2023, we’ll have even larger processing needs in the future. Therefore over the past year, Livermore Computing at LLNL has been migrating our current logging infrastructure to Elasticsearch and Kibana in an effort to handle the increasing amount of data even faster than before. This talk will focus on the changes we’ve made, why we decided to go with Elastic, and address some of the bumps we’ve hit along the way.
The Spack development team has launched their annual user survey and requests responses through March 17. Previous years’ survey results and analysis are available from the Spack repository.
Amazon Web Services (AWS) recently introduced Palace (PArallel, LArge-scale Computational Electromagnetics), a parallel finite element code for full-wave electromagnetics simulations. Palace is used at the AWS Center for Quantum Computing to perform large-scale 3D simulations of complex electromagnetics models and enable the design of quantum computing hardware. This open source, parallel finite element code for full-wave 3D electromagnetic simulations in the frequency or time domain uses the MFEM finite element discretization library and can be installed via Spack. Learn more about Palace in AWS’s blog post.
DeltaUQ is a simple strategy that can be used to train a single deep neural network (DNN) model to produce meaningful uncertainties. The code accompanies the paper “Single Model Uncertainty Estimation via Stochastic Data Centering” from the 2022 Conference and Workshop on Neural Information Processing Systems (NeurIPS). The abstract follows:
We are interested in estimating the uncertainties of deep neural networks, which play an important role in many scientific and engineering problems. In this paper, we present a striking new finding that an ensemble of neural networks with the same weight initialization, trained on datasets that are shifted by a constant bias gives rise to slightly inconsistent trained models, where the differences in predictions are a strong indicator of epistemic uncertainties. Using the neural tangent kernel (NTK), we demonstrate that this phenomena occurs in part because the NTK is not shift-invariant. Since this is achieved via a trivial input transformation, we show that this behavior can therefore be approximated by training a single neural network – using a technique that we call ∆−UQ – that estimates uncertainty around prediction by marginalizing out the effect of the biases during inference. We show that ∆−UQ ’s uncertainty estimates are superior to many of the current methods on a variety of benchmarks– outlier rejection, calibration under distribution shift, and sequential design optimization of black box functions.
HiOp is an optimization solver for solving certain mathematical optimization problems expressed as nonlinear programming problems. This lightweight HPC solver leverages application’s existing data parallelism to parallelize the optimization iterations by using specialized linear algebra kernels. The latest version hosts a series of comprehensive internal developments and software re-engineering to improve the portability and performance on accelerators/GPU platforms, including a new execution space abstraction to allow multiple hardware backends to run concurrently.
Learn more:
With LUAR (Learning Universal Authorship Representations), researchers conduct the first large-scale study of cross-domain transfer for authorship verification considering zero-shot transfers involving three disparate domains: Amazon Reviews, fanfiction short stories, and Reddit comments. This code accompanies the paper “Learning Universal Authorship Representations” from the Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP). The abstract follows:
Determining whether two documents were composed by the same author, also known as authorship verification, has traditionally been tackled using statistical methods. Recently, authorship representations learned using neural networks have been found to outperform alternatives, particularly in large-scale settings involving hundreds of thousands of authors. But do such representations learned in a particular domain transfer to other domains? Or are these representations inherently entangled with domain-specific features? To study these questions, we conduct the first large-scale study of cross-domain transfer for authorship verification considering zero-shot transfers involving three disparate domains: Amazon reviews, fanfiction short stories, and Reddit comments. We find that although a surprising degree of transfer is possible between certain domains, it is not so successful between others. We examine properties of these domains that influence generalization and propose simple but effective methods to improve transfer.
dbComm is a module for integrating a Python codebase with MongoDB using the PyMongo library. The methods contained are used to connect to a Mongo database and push/pull data.
With software development underpinning much of LLNL’s mission-driven science, the Lab’s developer community has a long history of figuring out solutions to a huge number of complex challenges. But how do we know what’s working and what isn’t? What tools or other support do developers need to do their best work? The RADIUSS project team conducted a survey of LLNL developers in late 2022 to assess their happiness and needs, both for awareness and targeted action that might be pursued in the next year. Some of this insight has been published publicly for the purposes of knowledge sharing and community building. Additionally, since RADIUSS products are open source, external users can benefit from improvements LLNL makes internally and pushed out to publicly available projects. Read the survey report.
VisIt is an open source, interactive, scalable, visualization, animation, and analysis tool. Version 3.3.2 includes:
Learn more:
MuyGPyS is a Gaussian process estimation method that affords fast hyperparameter optimization by way of performing leave-one-out cross-validation. The latest release introduces a new interface for handling backend implementations of the math functions, a new PyTorch backend, and a bespoke PyTorch MuyGPs layer for use in deep kernel models built in MuyGPs.
Learn more:
FOSDEM is a free event for software developers to meet, share ideas, and collaborate. The Spack team had three presentations at this year’s event in February. If you missed them, you can watch the videos here:
Aluminum provides a generic interface to high-performance communication libraries with a focus on allreduce algorithms. Blocking and non-blocking algorithms and GPU-aware algorithms are supported. Aluminum also contains custom implementations of select algorithms to optimize for certain situations. The v1.1.0 release includes improvements in benchmarking/testing infrastructure, better progress engine binding on HIP/ROCm systems, a project logo, and more. v1.2.0 adds better support for low-precision data.
Learn more:
MSR-SAFE allows safer access to model-specific registers. v1.7.0 includes a major overhaul of the docs, adds the first AMD allowlist, and addresses cleanup throughout the code.
Learn more:
LANL led with LLNL contributors, Charliecloud provides user-defined software stacks for HPC centers. It uses Linux user namespaces to run containers with no privileged operations or daemons and minimal configuration changes on center resources. Version 0.31 includes an overhaul of the tutorial, updates to ch-image, and updates to ch-run.
Learn more:
The U.S. modern critical infrastructure—from the electrical grid that sends power to homes to the pipelines that deliver water and natural gas and the railways and roadways we travel—is full of digitized components. In a power grid, this includes distributed energy resources such as smart meters, solar inverters, power-quality sensors, and protection devices that are geographically spread out, programmable and network connected. These networks, as currently designed, typically rely on a single control center for analysis and decision making.
To defend against cyber-attacks and harden the system, LLNL mathematicians, systems analysts, power engineers, cybersecurity experts and computational scientists have turned to collaborative autonomy—a new class of computational techniques that teach networked devices how to self-organize into a collective whole. And recently, an LLNL research team has developed Skywing—an open-source, high-reliability, real-time, decentralized software platform for domain scientists, mathematicians, and computer scientists exploring collaborative-autonomy applications for critical infrastructure. Skywing provides approaches and solutions for real-world applications that solve problems and allow for confidence in the results. It also helps lower the barrier to entry for those who may lack fluency in decentralized software development.
Learn more:
Discrete resource event modeling and multi-cluster scheduling simulator (DR_EVT) aims to provide a computational environment for simulating job scheduling and resource management using a set of heterogeneous clusters. Currently, the repo provides means to load and process job trace files, and the team is working on a workload model based on the trace data. DR_EVT is designed for Linux-based systems and contains information specific to Livermore Computing (LC) platforms.
The RAJA performance suite (RAJAPerf) is designed to explore performance of loop-based computational kernels found in HPC applications. It is used to assess, monitor, and compare runtime performance of kernels implemented using RAJA and variants implemented using standard or vendor-supported parallel programming models directly. This release has a new naming convention as per coordinated releases of RAJA Portability Suite projects. New features include CONVECTION3DPA finite element kernel, basic memory operation kernels MEMSET and MEMCPY, improved CI testing, and more.
Learn more:
UnifyFS is a user-level burst buffer file system under active development. The repo supports scalable and efficient aggregation of I/O bandwidth from burst buffers while having the same lifecycle as a batch-submitted job. This release includes changes to support Node Local extents, improvements to pkg-config support, additional unit tests, and more.
Learn more:
Quandary provides optimal control for open quantum systems via an optimization solver. The underlying quantum dynamics model open quantum systems, using the Lindblad master equation to evolve a density matrix in time. With this latest release, a Python interface allows the user to use an (arbitrary) user-defined system and control Hamiltonian operators during the simulation and optimization process.
Learn more:
Developed as part of the Center for Efficient Exascale Discretizations (CEED) within the U.S. Department of Energy, libCEED is a high-order API library that provides a common algebraic low-level operator description, allowing a wide variety of applications to take advantage of the efficient operator evaluation algorithms in the different CEED packages. This release includes improved support for H(div) bases, a new compressible Navier-Stokes mini-app, improvements to the interface, and more.
Learn more:
Conduit provides an intuitive model for describing hierarchical scientific data in C++, C, Fortran, and Python. It is used for data coupling between packages in-core, serialization, and I/O tasks. The latest release adds new methods, support for Wedges and Pyramids, optional support for Caliper performance annotations, and more.
Learn more:
Johannes Doerfert was selected as one of six 2023 Better Scientific Software fellows recognizing his leadership and advocacy of high-quality scientific software. The Better Scientific Software (BSSw) community is an international group of researchers, practitioners, and stakeholders from national laboratories, academic institutions, and industry who are dedicated to curating, creating, and disseminating information that leads to improved software for the advancement of computational science and engineering (CSE) and related technical computing areas, with a particular interest in CSE on high-performance (parallel) computers. The BSSw Fellowship Program gives recognition and funding to leaders and advocates of high-quality scientific software. Each 2023 Fellow receives up to $25,000 for an activity that promotes better scientific software. Doerfert will focus his fellowship funding on improving developer productivity by demystifying the compiler black box. His plans include making short introductory videos on compiler technology with a focus on improved interaction and available tooling. The new fellows will be recognized at the 2023 Exascale Computing Project Annual Meeting in January.
Adiak is a library for collecting metadata from HPC application runs, and distributing that metadata to subscriber tools. Adiak has a tool interface, which allows tools to subscribe to this metadata. Among the changes in version 0.2.2, launchdate now works on OSX.
Learn more:
hypre is a library of high-performance preconditioners and solvers featuring multigrid methods for the solution of large, sparse linear systems of equations on massively parallel computers. Version 2.27.0 includes support for multi-component vectors, HIP support added to the MGR solver, optimizations for IJ solvers, CMake build system updates, and more.
Learn more:
Large-scale parallel computing systems generate massive amounts of data every second, and file systems are crucial for handling data transfer at LLNL’s HPC center. Livermore Computing’s (LC’s) Scalable Storage group helps manage the hardware and software necessary to keep file systems—and therefore supercomputers—running smoothly and efficiently. One key tool in the group’s portfolio is the ZFS (Zettabyte File System) project, which controls I/O operations and optimizes storage volume capacity. In 2013, the original ZFS project and ZFS on Linux evolved into what is now known as OpenZFS, which is maintained by a global developer community that includes LC staff. The Scalable Storage group has adapted OpenZFS for the Lab’s needs, especially as new generations of Lustre-based HPC systems—including the upcoming El Capitan exascale supercomputer—are designed and installed. Read more about OpenZFS on the LLNL Computing website.
Merlin is a tool for running machine learning based workflows. The goal of Merlin is to make it easy to build, run, and process the kinds of large scale HPC workflows needed for cognitive simulation. Version 1.9.0 includes support for Python 3.11, JSON schema validation for Merlin spec files, updated documentation, and more.
Learn more:
LLNL’s Vanessa Sochat and collaborators from the Wellcome Sanger Institute, the Pawsey Supercomputing Research Institute, and the University of Texas at Dallas have written a paper about the Singularity Registry HPC (“shpc”). Sochat breaks down the team’s methods in a Twitter thread. You can also download the preprint PDF; abstract follows:
Linux container technologies such as Docker and Singularity offer encapsulated environments for easy execution of software. In high performance computing, this is especially important for evolving and complex software stacks with conflicting dependencies that must co-exist. Singularity Registry HPC (“shpc”) was created as an effort to install containers in this environment as modules, seamlessly allowing for typically hidden executables inside containers to be presented to the user as commands, and as such significantly simplifying the user experience. A remaining challenge, however, is deriving the list of important executables in the container. In this work, we present new automation and methods that allow for not only discovering new containers in large community sets, but also deriving container entries with important executables. With this work we have added over 8,000 containers from the BioContainers community that can be maintained and updated by the software automation over time. All software is publicly available on the GitHub platform, and can be beneficial to container registries and infrastructure providers for automatically generating container modules to lower the usage entry barrier and improve user experience.
YGM is a general-purpose, pseudo-asynchronous communication library built on top of MPI in C++. The repo’s utility is provided through its mailbox abstractions, which are used for point-to-point and broadcast communications. Release v0.5 includes a parser for Parquet files, improved broadcasts, and new containers.
Learn more:
The IEEE international eScience conference, which emphasizes compute- and data-intensive research methods, bestowed the 2022 Best Paper Award on a multidisciplinary team that includes LLNL staff and external collaborators. The paper, “Scalable Composition and Analysis Techniques for Massive Scientific Workflows,” details the optimization of a drug screening workflow for the American Heart Association (AHA). The AHA Molecule Screening (MoleS) workflow combines specialized software tools to manage HPC hardware heterogeneity. The MoleS end-to-end workflow relies on both general-purpose and domain-specific software tools, some of which are open source and/or developed at LLNL: Maestro for workflow execution, Flux for workload management and scheduling, RabbitMQ for message brokering (orchestrated by Kubernetes), ConveyorLC for docking automation tasks, Fusion machine learning algorithms for binding affinity predictions, and GMD (Generative Molecular Design) for the small-molecule discovery loop. Read more on the LLNL Computing website.
Carol Woodward joined the Lab’s Center for Applied Scientific Computing (CASC) in 1996, first as a postdoc and then as a staff researcher. CASC has developed a reputation over the years, she notes, as an organization that can solve tough problems, so she and her colleagues are asked to consult on a diverse array of projects. “It’s nice because it means I can work at the same place and not just do one thing for a long time—I get to keep changing what I work on,” she says. She is the principal investigator for SUNDIALS, a package of time integrators and nonlinear solvers that garners more than 100,000 downloads annually and is used in myriad simulation-dependent applications. Her group’s technical contributions to the software have modernized it and upgraded its functionality for more than two decades, enabling it to scale to DOE’s highest-end computing systems. With its innovative solvers and flexibility for a variety of computing systems, SUNDIALS was awarded the prestigious 2023 SIAM/ACM Prize in Computational Science and Engineering. In 2022 Woodward was promoted to Distinguished Member of Technical Staff, LLNL’s highest technical job classification level. Read more about her work.
Caliper is a program instrumentation and performance measurement framework. It is designed as a performance analysis toolbox in a library, allowing one to bake performance analysis capabilities directly into applications and activate them at runtime. The latest release includes improved sample profiling support and event tracing support, options for filtering regions by name, and more.
Learn more:
LLNL’s Center for Applied Scientific Computing (CASC) has published a new issue of its external newsletter. Included in the issue articles featuring open source software projects:
LANL led with LLNL contributors, Charliecloud provides user-defined software stacks for HPC centers. It uses Linux user namespaces to run containers with no privileged operations or daemons and minimal configuration changes on center resources. Version 0.30 includes build cache performance improvements and many changes to ch-image.
Learn more:
The high-performance computing publication HPCwire announced LLNL as the winner of its Editor’s Choice award for Best Use of HPC in Energy for applying cognitive simulation (CogSim) methods to inertial confinement fusion (ICF) research. The award recognizes the team for progress in their machine learning-based approach to modeling ICF experiments performed at the National Ignition Facility (NIF) and elsewhere, which has led to the creation of faster and more accurate models of ICF implosions. The CogSim work addresses the need for better models that can fully utilize available datasets, can accurately estimate uncertainty, and can improve with additional data. Much of the CogSim work has been done on HPC machines including Sierra, Lassen, and Corona, using the open source projects Merlin, a custom deep-learning workflow tool, and the Livermore Big Artificial Neural Network toolkit (LBANN), a deep-learning training framework optimized for HPC.
The MFEM team hosted the second annual MFEM Community Workshop on October 25, 2025. The goal of the workshop was to foster collaboration among all MFEM users and developers, share the latest MFEM features with the broader community, deepen application engagements, and solicit feedback to guide future development directions for the project. If you missed the workshop, check out these resources: 2022 agenda (with speakers’ slides linked as PDFs) and news article. Videos of the talks are in production and will be posted soon.
The @Livermore_Comp account hosted a Twitter Space during the 34th annual Supercomputing Conference (SC22) on November 15. LLNL’s Meg Epperly moderated a panel featuring Elsa Gonsiorowski (SCR and other projects), Greg Becker (Spack), and David Gardner (SUNDIALS). The panel discussed answered questions about why open source software is important for HPC centers, who benefits from using and contributing to it, and why LLNL actively develops and nurtures open source software. Listen to the recording.
Spack is a flexible, configurable, Python-based, and open-source HPC package manager. Spack automates the installation and fine-tuning of simulations and libraries, operating on a wide variety of HPC platforms and enabling users to build many code configurations. Version 0.19.0 includes:
Learn more:
The latest issue of LLNL’s Science & Technology Review magazine showcases the R&D 100 award–winning Flux software framework. Honored with a 2021 R&D 100 Award, Flux is a scalable, flexible next-generation workload management framework that meets this need—maximizing resource utilization while allowing scientific applications and workflows to run faster and more efficiently. Developed in collaboration with university partners, Flux also enables new resource types, schedulers, and services to be deployed at data centers as they continue to evolve.
Todd Gamblin, an LLNL Distinguished Member of Technical Staff, gave a presentation on November 9 for the Dell Technologies HPC Community. His talk, “Sustaining the HPC Software Ecosystem,” described how HPC software can be managed more easily for all customers and users with Spack and included an overview of recent developments in the Spack community such as a partnership with AWS to provide infrastructure for a worldwide binary cache, a recent machine learning special interest group within Spack, and work to handle the complexities of installing software for GPUs. Slides can be downloaded, and a recording is available with free registration.
The Society for Industrial and Applied Mathematics (SIAM) and Association for Computing Machinery (ACM) announced that they have awarded the 2023 SIAM/ACM Prize in Computational Science and Engineering to the team behind the LLNL-developed SUNDIALS software suite. The prestigious award is handed out every two years and recognizes outstanding contributions to the development and use of mathematical and computational tools and methods for the solution of science and engineering problems. It is one of SIAM’s most significant awards and will be presented to the team at the 2023 SIAM Conference on Computational Science and Engineering in Amsterdam next February. Because of its ease, flexibility and extensive documentation, SUNDIALS has become internationally recognized as one of the most effective and efficient time integration libraries. It is widely used by government laboratories and in academic institutions and industry, leading to advances in a variety of applications including fusion device modeling, watershed modeling, and reacting flow simulations.
Team members are Distinguished Member of the Technical Staff Carol Woodward, computational scientists Cody Balos and David Gardner, senior scientist Peter Brown, LLNL guest scholar and retiree Alan Hindmarsh, former LLNL postdoc Daniel Reynolds, and former LLNL scientist Radu Serban. Read more about the award at LLNL News and the SUNDIALS: SUite of Nonlinear and DIfferential/ALgebraic Equation Solvers at LLNL Computing.
Flux is a flexible framework for resource management consisting of a suite of projects, tools, and libraries which may be used to build site-custom resource managers for HPC centers. This version includes new functionality for job-list, broker, and other functions, along with updates to the test suite and CI.
Learn more:
RAJA is a software abstraction framework that systematically encapsulates platform-specific code to enable applications to be portable across diverse hardware architectures without major source code disruption. This release includes:
RAJA::forall and reduction interfaceRAJA::forall kernelsatomicInc and atomicDec functionsLearn more:
Umpire is a resource management library that allows the discovery, provision, and management of memory on next-generation architectures. This release adds new HIP Advise operations, methods, event tracking, and UMAP allocation resource.
Learn more:
MFEM is a lightweight, general, scalable C++ library for finite element methods. It enables high-performance scalable finite element discretization research and application development on a wide variety of platforms, from laptops to supercomputers. The v4.5 release includes:
Learn more:
SUNDIALS is a SUite of Nonlinear and DIfferential/ALgebraic equation Solvers. This minor release adds:
Learn more:
As the creator of Spack, Todd Gamblin states, “Open source tools developed at LLNL enable people around the world to use HPC resources more effectively and to do better science.” In turn, these tools provide a framework for an entire community to maintain software needed by LLNL and its programs—for instance, LLNL could never maintain Spack’s thousands of software packages alone, and it benefits from the work of Spack’s enthusiastic contributors. Ultimately, he notes, “Sustaining open source communities is about finding leverage. It’s worthwhile for LLNL to put in the effort to build and maintain something like Spack if it incites a community of thousands to work together for the benefit of all.” In 2022 Gamblin was promoted to Distinguished Member of Technical Staff, LLNL’s highest technical job classification level. Read more about his recent work.
The Exascale Computing Project (ECP) celebrates a new era of scientific discovery with Exascale Day on October 18, or “10^18” to represent the exascale threshold of floating-point operations per second. The event runs all week and provides multimedia and articles that educate explain the impact areas of exascale computing from the Department of Energy national laboratories (including LLNL), HPC manufacturers, and universities and industrial organizations. Much of the ECP’s software stack is open source.
The MSU Disentanglement Analysis Software (MDAS) is used to disentangle the forced and unforced components of tropospheric temperature change over the satellite era (after 1979) using maps of surface temperature change as a predictor. Input data is available via Zenodo (doi: 10.5281/zenodo.7199961). An accompanying publication is forthcoming from the Proceedings of the National Academy of Sciences.
GIDIplus is a suite of C++ libraries for accessing nuclear data from the Generalized Nuclear Database Structure (GNDS). The latest version includes full GNDS-2.0 support, a new RISI module for reading .ris files, and more.
Learn more:
MuyGPyS is a Gaussian process estimation method that affords fast hyperparameter optimization by way of performing leave-one-out cross-validation. The latest release introduces support for distributed memory processing using MPI and now supports three distinction implementations of all of the math functions.
Learn more:
Under a recently funded project, researchers at LLNL and the Massachusetts Institute of Technology (MIT) will address the challenge of efficiently differentiating large-scale applications for the Department of Energy by building on advances in LLNL’s MFEM finite element library and MIT’s Enzyme AD tool. The team’s project will address the challenge of efficiently differentiating large-scale DOE applications—predicting how adjustments in design parameters will impact the output of a code. Knowledge of optimal outputs is increasingly needed for complex simulation codes to be used for design optimization, machine learning, uncertainty quantification and sensitivity analysis, among other applications. While automatic differentiation (AD) has made the differentiation process easier, traditional AD tools require significant changes that are not feasible for many existing large-scale DOE applications. Read more about the project at LLNL News.
The Rollerball Notepad++ Plugin repo contains code for a benignware (pseudo malware) plugin for the Notepad++ editor based on this plugin template. The plugin purports to be an AutoSave feature, but in fact it exfiltrates file content based on keywords to a webserver that you specify. This code was designed as a way to test detection capabilities.
Flux is a flexible framework for resource management consisting of a suite of projects, tools, and libraries which may be used to build site-custom resource managers for HPC centers. This version includes enhancements to inject functionality, the queue state, flux-jobs, and more.
Learn more:
BLT (Building, Linking, and Testing) is a streamlined CMake build system foundation for developing HPC software. BLT makes it easy to get up and running on a wide range of HPC compilers, operating systems, and technologies. This release includes a new macro and variable.
Learn more:
The 34th annual Supercomputing Conference (SC22) will be held in Dallas throughout November 13–18.
Twitter accounts to follow: @Livermore_Comp, @LLNL_OpenSource, @NatLabsHPC, @Livermore_Lab.
Check out LLNL’s full lineup of events. All initial broadcast times are listed in Central Standard Time.
HiOp is an optimization solver for solving certain mathematical optimization problems expressed as nonlinear programming problems. This lightweight HPC solver leverages application’s existing data parallelism to parallelize the optimization iterations by using specialized linear algebra kernels. This version includes a Fortran interface and examples, implementation of CUDA CSR matrices, and improved robustness and performance of the mixed dense-sparse solver.
Learn more:
VisIt is an open source, interactive, scalable, visualization, animation, and analysis tool. Version 3.3.1 includes:
Learn more:
ExaCA is a cellular automata (CA) code for grain growth under additive manufacturing conditions by ExaAM within the Exascale Computing Project. This release adds options to include remelting and to control the density of grains.
Learn more:
The Assured Timing Detector (ATD) software provides an implementation in C++ of an adaptable, model-based system for monitoring a timing signal for anomalies versus a reference time source ensemble. This system is designed to be model-based, adaptive, and customizable. A companion status display software package, ATD-SD, is also available for use with this software.
Variorum is a platform-agnostic library exposing monitor and control interfaces for several features in hardware architectures. Its general interfaces provide privileged functionality for monitoring and controlling various hardware-level features of multiple hardware architectures. The latest release includes support for new architectures, new Python wrappers, API updates, and more.
Learn more:
Skywing is a high-reliability, real-time, decentralized platform for collaborative autonomy–focused applications. The repo includes installation instructions, examples, and a tutorial; some dependencies are managed as git submodules.
After 10 years and 33 hackathons, nothing can stop this beloved tradition. “Hackathons are wildly popular not just because they allow employees to try new things and develop new skills, but also because they are so much fun!” says Computing’s associate director Bruce Hendrickson. Read an article about this milestone, and check out the LLNL Flickr album of hackathon photos throughout the years.
Flux is a flexible framework for resource management consisting of a suite of projects, tools, and libraries which may be used to build site-custom resource managers for HPC centers. This version includes new options to flux-jobs(1) and flux-mini(1), show detailed job state in flux-jobs(1) output, automatic KVS garbage collection, and more.
Learn more:
CAMP collects a variety of macros and metaprogramming facilities for C++ projects. This release catches CAMP up to the version nomenclature for the RAJA Portability Suite, which has changed format to indicate year, month, and patch number. New features include support for CTAD and structured bindings as well as fixes for exported target issues.
Learn more:
Held in a hybrid format for the first time, LLNL’s Developer Day 2022 convened more than 70 people for an agenda of lightning talks, a town hall discussion, and guest speakers. Dev Day provides different ways for the audience to learn about and engage in topics of interest to the developer community—such as lightning talks about small projects, summaries of literature, deep dives into project planning and implementation, discussions of career opportunities and challenges, and networking and brainstorming sessions—and the format varies every year. This year’s event spotlighted guest speakers who explained the impact of unique software projects on their organizations. For instance, Miranda Mundt, a research software engineer at Sandia National Laboratories, described her team’s tiered approach to software quality practices, while Dr. Arun Viswanathan from NASA’s Jet Propulsion Laboratory presented work that keeps space missions resilient from cyber threats.
A Sandia software development team has launched a new website based on one of LLNL’s open-source projects. The new software.sandia.gov website (repository: github.com/sandialabs/sandialabs.github.io) serves as a portal into Sandia’s GitHub repositories, providing more than a dozen browsable categories alongside data visualizations of GitHub data—repository relationships, commit and pull request activity, common licenses, and more. Forked from this very website (repository: github.com/LLNL/llnl.github.io), Sandia’s site is also built on a Jekyll template and served via GitHub pages. Read more about the implementation process and the ways the two websites vary.
The Advanced Technology Development and Mitigation (ATDM) program within the Exascale Computing Project (ECP) shows that the best way to support the mission is through open collaboration and a sustainable software infrastructure. Although ATDM primarily supports NNSA’s traditionally closed mission of national security, LLNL’s ATDM Software Technology (ST) project contributes key open source components of a full-featured, integrated, and maintainable software stack for exascale systems that will impact both the ECP and the broader HPC community. A new article on the ECP website describes the project’s goals and technical focus areas in an interview with Livermore Computing division leader Becky Springmeyer and ATDM ST deputy lead Todd Gamblin.
This summer, LLNL’s RADIUSS team conducted a series of tutorials in collaboration with Amazon Web Services (AWS), demonstrating how to use several GPU-ready projects in the cloud and on premises. The tutorials were open to everyone and held every week throughout August. Participants followed along on their own AWS EC2 instance (provided). No previous was experience necessary. Materials are accessible on the RADIUSS website.
UnifyFS is a user-level burst buffer file system under active development. The repo supports scalable and efficient aggregation of I/O bandwidth from burst buffers while having the same lifecycle as a batch-submitted job. This major release includes improved file staging, updated example programs, new unit tests and CI tests, a client API for I/O middleware libraries, and more.
Learn more:
The Exascale Computing Project, a joint effort between the DOE Office of Science and NNSA, brings together several national laboratories to address many hardware, software, and application challenges inherent in the organizations’ scientific and national security missions. The ECP’s annual meeting was held this year on May 2–6. Each day’s sessions are available in a dedicated YouTube playlist. Individual sessions highlighted below feature LLNL staff and open source projects.
Conduit provides an intuitive model for describing hierarchical scientific data in C++, C, Fortran, and Python. It is used for data coupling between packages in-core, serialization, and I/O tasks. The latest release adds several methods, updates to mesh blueprint functionality, and more.
Learn more:
Hatchet is a Python-based library that allows Pandas dataframes to be indexed by structured tree and graph data. This release resolves issues with package install, modifies graphframe copy/deepcopy, and more.
Learn more:
H5Z-ZFP(https://github.com/LLNL/H5Z-ZFP) is a highly flexible floating point and integer compression plugin for the HDF5 library using zfp compression. This release includes updates to the decompression compatibility check, testing against previous zfp versions, CMake build, Windows build added to CI, and much more.
Learn more:
As part of the Exascale Computing Project (ECP), the Center for Efficient Exascale Discretizations (CEED) is a research partnership between two U.S. Department of Energy laboratories and five universities. LLNL leads the Center. All of CEED’s software is open source.
CEED held its sixth annual meeting (CEED6AM) on August 9-11 in a hybrid format: in-person at the Siebel Center for Computer Science on the University of Illinois Urbana-Champaign campus in Urbana and virtually using ECP Zoom for videoconferencing and Slack for side discussions. Learn more about the agenda on the CEED6AM event website.
SUNDIALS is a SUite of Nonlinear and DIfferential/ALgebraic equation Solvers. This minor release:
GetUserData functions in each packageMRIStepReset to call the corresponding MRIStepInnerResetFn appropriatelyLearn more:
Spack has already been removing the ugly work from building HPC codes, but with the announcement of the Spack Binary Cache—which is hosted on AWS—build and deploy times for these complicated applications will drop by 95% or more in most cases. Spack core developer Greg Becker spoke with HPC Tech Shorts to explain how it works and discuss what’s behind it. The video “Get your HPC codes installed and running in minutes using Spack’s Binary Cache” runs 15:11.
LANL led with LLNL contributors, Charliecloud provides user-defined software stacks for HPC centers. It uses Linux user namespaces to run containers with no privileged operations or daemons and minimal configuration changes on center resources. Version 0.29 includes fully unprivileged end-to-end workflows and updates to ch-image.
Learn more:
Kosh allows codes to store, query, and share data via an easy-to-use Python API. This software aims to make data access and sharing as simple as possible. The latest release includes added support for Windows systems and improvements to importing sina json files.
Learn more:
As the number of smart meters and the demand for energy is expected to increase by 50% by 2050, so will the amount of data those smart meters produce. While energy standards have enabled large-scale data collection and storage, maximizing this data to mitigate costs and consumer demand has been an ongoing focus of energy research. GridDS is an open source data science toolkit for power and data engineers that will provide an integrated energy data storage and augmentation infrastructure, as well as a flexible and comprehensive set of state-of-the-art machine learning models. By providing an integrative software platform to train and validate machine learning models, GridDS will help improve the efficiency of distributed energy resources, such as smart meters, batteries and solar photovoltaic units. GridDS also is designed to leverage advanced metering infrastructure, outage management systems data, supervisory control data acquisition, and geographic information systems to forecast energy demands and detect incipient grid failures. Read more about GridDS at LLNL News.
GEOSX and its predecessor, GEOS, were developed from the ground up with the help of experts from across LLNL, combining a range of disciplines including engineering, seismology, hydrology, computational geoscience, and oil- and gas-industry expertise to build a tool that can take advantage of advanced computing platforms. A Research Highlight article in the latest issue of Science & Technology Review describes how GEOSX will improve the management and security of geological repositories and support planning for the widespread implementation of CO2 storage at an industrial scale by simulating how fluids flow and rocks break deep underground. Read “GEOSX Simulates Carbon Dioxide Storage” on the S&TR website.
zfp is a library for compressed numerical arrays that support high-throughput read and write random access. zfp also supports streaming compression of integer and floating-point data, e.g., for applications that read and write large data sets to and from disk. zfp is primarily written in C and C++ but also includes Python and Fortran bindings. This major release includes numerous changes to function signatures and data structures, a more complete API for pointers and iterators, rounding modes for reducing bias in compression errors, support for pointers and iterators into array views, refactored compressed-array C++ implementation, and much more.
Learn more:
PDBspheres is a structure-based method for finding and evaluating structural similarities in protein regions relevant to ligand binding. PDBspheres comprises an exhaustive library of protein structure regions (“spheres”) adjacent to complexed ligands derived from the Protein Data Bank (PDB), along with methods to find and evaluate structural matches between a protein of interest and spheres in the library.
ExaCA is a cellular automata (CA) code for grain growth under additive manufacturing conditions by ExaAM within the Exascale Computing Project. ExaCA runs with the default enabled Kokkos backend and has been tested with Serial, OpenMP, Pthreads, CUDA, and HIP backends. The first major release includes simulations of directional solidification, grain structure output and analysis routines, and more. Installation instructions are in the ExaCA README.
Learn more:
Wintap is an extensible host-based agent for Windows. Wintap provides a singular and extensible service-based runtime environment, a unified data model, API abstraction, and data discovery for an integrated, locally hosted web-based analytic “workbench” from where real-time event streams can be queried and explored.
DFTT (Detection Framework Testbed and Toolkit) is a database and associated Java programs intended to facilitate the development and testing of algorithms for operating suites of correlation and subspace detectors. This framework is a generalization of the system described in Harris and Dodge (2011). It allows retrospective processing of sequences of data using various system configurations. Results are saved in a database, so it is easy to compare the results obtained using different configurations of the system.
The Coda Calibration Tool (CCT) calculates reliable moment magnitudes for small- to moderate-sized seismic events. This release includes updates to zooming functionality, user adjustable start times, new REST endpoint, updates to uncertainty quantification metrics, and more.
Learn more:
After a few years off, the North American High Order Methods Conference (NAHOMCon) returned on July 18-19 in San Diego. NAHOMCon provides a North American forum for computational scientists, mathematicians, scientists, and engineers to share ideas and techniques on, and further the state of the art of, high order methods for the solution of partial differential equations with applications to a broad range of scientific and engineering applications. The DOE co-design Center for Efficient Exascale Discretizations (CEED) participates in this conference. CEED is a partnership between two U.S. DOE laboratories (Livermore & Argonne) and five universities in support of the Exascale Computing Project.
The NAHOMCon22 program featured several LLNL speakers, whose GitHub profiles and abstracts are linked here:
LLNL’s Data Science Institute launched the Open Data Initiative (ODI) a few years ago. The ODI shares LLNL’s rich, challenging, and unique datasets with the larger data science community. The goal is for these datasets to help support curriculum development, raise awareness around LLNL’s data science efforts, foster new collaborations, and be leveraged across other learning opportunities. The ODI currently has 13 publicly available datasets in its collection, the newest of which builds off the Cardioid open source code, which simulates the electrophysiology of the human heart. A research team has conducted a computational study to generate a dataset of cardiac simulations at high spatiotemporal resolutions. This dataset was built using real cardiac bi-ventricular geometries and clinically inspired endocardial activation patterns under different physiological and pathophysiological conditions. It consists of pairs of computationally simulated intracardiac transmembrane voltage recordings and electrocardiogram signals. Read more and download the data from the ODI web page.
spdlayers provides symmetric positive definite (SPD) enforcement layers for PyTorch. Examples, an API, and documentation are included in the repo. This release adds the option for the Eigen layer to support semi-definite prediction.
Learn more:
EchemFEM provides finite element solvers for electrochemical transport. Both continuous Galerkin (CG) and discontinuous Galerkin (DG) schemes are provided. The following transport mechanisms are available: diffusion, advection, electromigration. EchemFEM supports both non-porous and porous cases. The ionic potential can either be described using an electroneutrality constraint or a Poisson equation. In the porous case, the electronic potential can be described by a Poisson equation.
SAC2000 (Seismic Analysis Code for the third millennium) is a general purpose interactive program designed for the study of sequential signals, especially time-series data. Emphasis has been placed on analysis tools used by research seismologists in the detailed study of seismic events. Analysis capabilities include general arithmetic operations, Fourier transforms, three spectral estimation techniques, IIR and FIR filtering, signal stacking, decimation, interpolation, correlation, and seismic phase picking.
Caliper is a program instrumentation and performance measurement framework. It is designed as a performance analysis toolbox in a library, allowing one to bake performance analysis capabilities directly into applications and activate them at runtime. The latest release includes AMD ROCm/HIP activity profiling and tracing, MPI message size and count metrics, and more.
Learn more:
VisIt is an open source, interactive, scalable, visualization, animation, and analysis tool. Version 3.3.0 includes
Learn more:
XScope finds inputs that trigger floating-point exceptions, such as NaN (not a number) and infinity, in CUDA functions using Bayesian optimization (BO). XScope assumes that the CUDA functions are a black box, i.e., the source code is not available. It searches the input space using several methods to guide BO into extreme cases. When an input is found to trigger an exception in the target CUDA function, the input is shown to the user. A forthcoming paper related to this repo has been accepted to SC22.
graphite is a repository for implementing graph network models based on atomic structures. It is meant to be a general toolbox of graph model codes (e.g., helper functions, custom graph convolutions, template graph models) for atomic structures. The repository contains examples and links to implemented and replicated works.
LANL led with LLNL contributors, Charliecloud provides user-defined software stacks for HPC centers. It uses Linux user namespaces to run containers with no privileged operations or daemons and minimal configuration changes on center resources. Version 0.28 includes changes to ch-image and ch-run.
Learn more:
spdlayers provides symmetric positive definite (SPD) enforcement layers for PyTorch. Examples, an API, and documentation are included in the repo. This release adds a positivity function and sets its default.
Learn more:
The RADIUSS project has a new activity portal (source code for this GitHub action) that displays contributors’ activities, such as pull requests, issue comments, and releases. Visitors to this page can browse the entire list or filter by activity type to learn more about who contributes to RADIUSS and how they do so. RADIUSS aims to develop and deploy a common base of foundational scientific software with opt-in adoption from LLNL applications in order to reduce long-term software costs and increase agility.
Variorum is a platform-agnostic library exposing monitor and control interfaces for several features in hardware architectures. Its general interfaces provide privileged functionality for monitoring and controlling various hardware-level features of multiple hardware architectures. The latest release includes support for new architectures (i.e., AMD CPUs, Intel Ice Lake), API improvements, and more.
Learn more:
MetallData is an HPC platform for interactive data science applications at HPC-scales. It provides an ecosystem for persistent distributed data structures, including algorithms, interactivity, and storage.
Our @LLNL_OpenSource Twitter account is now officially verified with a blue checkmark. This badge affirms the account’s authenticity and notability to users and confirms that our account is active. Read more about verified accounts on Twitter’s help website.
Flux is a flexible framework for resource management consisting of a suite of projects, tools, and libraries which may be used to build site-custom resource managers for HPC centers. Version 0.40.0 includes upgrades to verify database integrity during module load and support new sdexec job launch plugin, as well as fixes to reconnect, job attributes, Python docstrings, and more.
Learn more:
The Coda Calibration Tool (CCT) calculates reliable moment magnitudes for small- to moderate-sized seismic events. This release includes updates to many naming conventions, updates to the Data filter functionality, new functions to export spectra value JSON directly from the spectra plots, and much more.
Learn more:
Spack is a flexible, configurable, Python-based, and open-source HPC package manager. Spack automates the installation and fine-tuning of simulations and libraries, operating on a wide variety of HPC platforms and enabling users to build many code configurations. Version 0.18.0 includes:
Learn more:
LLNL and Amazon Web Services (AWS) have signed a memorandum of understanding (MOU) to define the role of leadership-class HPC in a future where cloud HPC is ubiquitous. Under the MOU, LLNL and AWS will explore software and hardware solutions spanning cloud and on-premises HPC environments, with the goal of establishing a common stack of open source software components that can run equally well at both large HPC centers and on cloud resources. LLNL and AWS have an existing open source collaboration involving Spack; building off that collaboration, the organizations will look to better understand how HPC centers can best utilize cloud resources to support HPC and explore models for cloud-bursting, data staging, and data migration for deploying both on-site and in the cloud. Read more about the MOU at LLNL News.
GLVis is a lightweight OpenGL tool for accurate and flexible finite element visualization. Version 4.2 includes:
Learn more:
Magpie contains a number of scripts for running big data software in HPC environments. This release includes support for Python 3 as well as new versions of Spark and Hadoop. See the release notes for information on backwards compatibility.
Learn more:
ATS (Automated Testing System) is a Python-based tool for automating the running of an application’s tests across a broad range of high-performance computers. ATS features include generating and comparing plot files, testing code correctness and performance with different software libraries and compiler configurations, and archiving metrics related to each test run. The tool is flexible enough for experts and non-experts alike. ATS Documentation is available, and more information about how LLNL uses ATS can be found on the Computing website.
ISC High Performance Conference (ISC22) returns on May 29 through June 2, with in-person events held in Hamburg, Germany. The event brings together the HPC community—from research centers, commercial companies, academia, national laboratories, government agencies, exhibitors, and more—to share the latest technology of interest to HPC developers and users. View LLNL’s lineup of tutorials, BOFs, and workshops.
Held since 2012, LLNL’s hackathons are 24-hour opportunities to brainstorm, foster creativity, prototype, and explore. Participants work in groups or individually and often strive to learn new skills, programming languages, and tools in service to LLNL’s missions. This year’s spring event (May 26-27) will be held in person at the Livermore Valley Open Campus. Sponsors are two Computing divisions: Enterprise Applications Services and National Ignition Facility Computing.
Flux is a flexible framework for resource management consisting of a suite of projects, tools, and libraries which may be used to build site-custom resource managers for HPC centers. Version 0.39.0 includes new features to get all job attributes, track protocol changes, and add color and highlight, plus testsuite updates and more.
Learn more:
The RADIUSS project has a new look including an About page and an interactive catalog of open-source products. These new web pages are managed in a dedicated repo under the LLNL GitHub organization. RADIUSS aims to develop and deploy a common base of foundational scientific software with opt-in adoption from LLNL applications in order to reduce long-term software costs and increase agility.
The RAJA performance suite (RAJAPerf) is designed to explore performance of loop-based computational kernels found in HPC applications. It is used to assess, monitor, and compare runtime performance of kernels implemented using RAJA and variants implemented using standard or vendor-supported parallel programming models directly. The v0.12.0 release adds new command line options, basic MPI support, and new kernels, as well as updated versions of RAJA and BLT submodules.
Learn more:
Hatchet is a Python-based library that allows Pandas dataframes to be indexed by structured tree and graph data. This release catches Hatchet up to the new version nomenclature for the RAJA Portability Suite, which has changed format to indicate year, month, and patch number. New features include query language extensions, interactive visualization enhancements, and new color maps for terminal tree visualization. Additionally, the release has changes to existing APIs as well as updated tutorials and documentation.
Learn more:
The Exascale Computing Project (ECP) 2022 Community Birds-of-a-Feather (BOF) Days will take place May 10–12 with multiple sessions per day. The BOF Days provide an opportunity for the HPC community to engage with ECP teams to discuss latest development efforts. Each BOF will be a 60- to 90-minute session on a given topic, with a brief overview followed by Q&A. All sessions will be conducted via Zoom. View the schedule; each session has its own registration link.
NumFOCUS is a nonprofit organization that promotes and supports open source scientific computing projects, such as Jupyter, Julia, and NumPy. NumFOCUS affiliate projects are “scientifically oriented, open, and kind,” which means they contribute to the advancement of scientific research, are published under a standard open source license, seek engagement with the public, and openly state their values and standards for the project community. MFEM, a scalable finite element methods library led by an LLNL team, was recently selected to join the list of NumFOCUS affiliate projects, broadening the project’s exposure and potential user and contributor community.
SUNDIALS is a SUite of Nonlinear and DIfferential/ALgebraic equation Solvers. This minor release includes
SUNLogger API for logging of errors, warnings, informational output, and debugging output (many functions have been deprecated in favor of this API)Learn more:
AMM, which stands for Adaptive Multilinear Meshes, is a new framework to represent piecewise multilinear volumetric data using mixed-precision adaptive meshes. AMM is designed to reduce in-memory and on-disk data footprint using a spatial hierarchy with “rectangular cuboidal” cells. AMM also supports mixed-precision representation of function values in byte-sized increments. The current version provides several examples of data streams that can be selected through command line. For details on AMM’s data structure and representation, see the accompanying publication, which won the Best Paper Award at the 2022 PacificVis Symposium. The abstract follows:
Adaptive representations are increasingly indispensable for reducing the in-memory and on-disk footprints of large-scale data. Usual solutions are designed broadly along two themes: reducing data precision, e.g., through compression, or adapting data resolution, e.g., using spatial hierarchies. Recent research suggests that combining the two approaches, i.e., adapting both resolution and precision simultaneously, can offer significant gains over using them individually. However, there currently exist no practical solutions to creating and evaluating such representations at scale. In this work, we present a new resolution-precision-adaptive representation to support hybrid data reduction schemes and offer an interface to existing tools and algorithms. Through novelties in spatial hierarchy, our representation, Adaptive Multilinear Meshes (AMM), provides considerable reduction in the mesh size. AMM creates a piecewise multilinear representation of uniformly sampled scalar data and can selectively relax or enforce constraints on conformity, continuity, and coverage, delivering a flexible adaptive representation. AMM also supports representing the function using mixed-precision values to further the achievable gains in data reduction. We describe a practical approach to creating AMM incrementally using arbitrary orderings of data and demonstrate AMM on six types of resolution and precision datastreams. By interfacing with state-of-the-art rendering tools through VTK, we demonstrate the practical and computational advantages of our representation for visualization techniques. With an open-source release of our tool to create AMM, we make such evaluation of data reduction accessible to the community, which we hope will foster new opportunities and future data reduction schemes.
Conduit provides an intuitive model for describing hierarchical scientific data in C++, C, Fortran, and Python. It is used for data coupling between packages in-core, serialization, and I/O tasks. The latest release adds new blueprint mesh examples and C/C++ version macros. In addition, the pip install logic has been improved.
Learn more:
LOPE (Lattice Optimization for Porous Electrodes) performs an optimization over the structure of a porous electrode formed from a lattice of unit cells. This code accompanies the paper “Computational Design of Microarchitected Porous Electrodes for Redox Flow Batteries.” The abstract follows:
Porous electrodes are used as the core reactive component across electrochemical technologies. In flowing systems, controlling the fluid distribution, species transport, and reactive environment is critical to attaining high performance. However, conventional electrode materials like felts and papers provide few opportunities for precise engineering of the electrode and its microstructure. To address these limitations, architected electrodes composed of unit cells with spatially varying geometry determined via computational optimization are proposed. Resolved simulation is employed to develop a homogenized description of the constituent unit cells. These effective properties serve as inputs to a continuum model for the electrode when used in the negative half-cell of a vanadium redox flow battery. Porosity distributions minimizing power loss are then determined via computational design optimization to generate architected porosity electrodes. The architected electrodes are compared to bulk, uniform porosity electrodes and found to lead to increased power efficiency across operating flow rates and currents. The design methodology is further used to generate a scaled-up electrode with comparable power efficiency to the bench-scale systems. The variable porosity architecture and computational design methodology presented here thus offers a novel pathway for automatically generating spatially engineered electrode structures with improved power performance.
BinFPE detects floating-point exceptions (NaN, infinity, and subnormal quantities) in NVIDIA GPU applications using binary instrumentation. It requires no re-compilation of the application and can analyze libraries. The tool extends NVBit, which is provided by NVIDIA Labs to analyze binaries.
The Lab’s software developers and engineers have accumulated a wealth of expertise by keeping LLNL operational while carrying out its national security mission. Launched in 2022, the Software Development Resource Center (SDRC) connects developers across LLNL through best practices in software tools, development methodologies, DevOps, security compliance, and more. Funded as an Institutional Scientific Capability Portfolio (ISCP) project, the SDRC serves as a focal point for software leadership, including coordinating working groups and providing technical advice to project teams. For instance, the SDRC committee plans to schedule seminars and invited talks as well as conduct workshops during this summer’s Developer Day. The SDRC also arises from Computing’s 10-year Strategic Plan, which prioritizes support for mission-driven programs through robust software engineering and maintenance. Read more about the SDRC.
MTNN, which stands for Multilevel Neural Networks, is a PyTorch-based library for the application of multilevel algorithms to the training of neural networks. The algorithms behind MTNN mathematically work whenever the neural network can be decomposed into a set of operational subsets such that each subset consists of a set of neurons, channels, or other similar operational units. In this case, pairs of neurons (or channels or other operational units) can be matched up and restricted into a coarse network. Requires Python 3.6 or newer.
TargetID is a drug target and chemotype identification pipeline that enables rapid identification and characterization of binding sites in SARS-CoV-2 proteins as well as the core chemical components with which these sites interact. The repo contains three Jupyter Notebooks and input data files.
Pond B documents the code used to analyze the Pond B (Savannah River Site, South Carolina) microbial community for a manuscript titled “Microbial Dynamics Impact Plutonium and Iron Biogeochemical Cycles in a Seasonally Stratified Pond.” The repo includes figures and tables from the paper, which is being prepared for publication.
XNAS, which stands for eXplainable Neural Architecture Search, provides code for the paper “Learning Interpretable Models Through Multi-Objective Neural Architecture Search.” Requires Python 3.6 or newer. The abstract follows:
Monumental advances in deep learning have led to unprecedented achievements across a multitude of domains. While the performance of deep neural networks is indubitable, the architectural design and interpretability of such models are nontrivial. Research has been introduced to automate the design of neural network architectures through neural architecture search (NAS). Recent progress has made these methods more pragmatic by exploiting distributed computation and novel optimization algorithms. However, there is little work in optimizing architectures for interpretability. To this end, we propose a multi-objective distributed NAS framework that optimizes for both task performance and introspection. We leverage the non-dominated sorting genetic algorithm (NSGA-II) and explainable AI (XAI) techniques to reward architectures that can be better comprehended by humans. The framework is evaluated on several image classification datasets. We demonstrate that jointly optimizing for introspection ability and task error leads to more disentangled architectures that perform within tolerable error.
op is a lightweight general optimization solver interface. Its primary purpose is to simplify the process of integrating different optimization solvers (serial or parallel) with scalable parallel physics engines. From a user perspective, optimization problems can be described abstractly, and op guarantees portability for different optimization engines and optimization problem configurations.
HiOp is an optimization solver for solving certain mathematical optimization problems expressed as nonlinear programming problems. This lightweight HPC solver leverages application’s existing data parallelism to parallelize the optimization iterations by using specialized linear algebra kernels. This version includes the primal decomposition (PriDec) solver for structured two-stage problems as well as improved support for NVIDIA GPUs for solving sparse optimization problems.
Learn more:
LANL led with LLNL contributors, Charliecloud provides user-defined software stacks for HPC centers. It uses Linux user namespaces to run containers with no privileged operations or daemons and minimal configuration changes on center resources. Version 0.27 includes new tests, examples, and FAQ; as well as additional functionality for ch-convert, ch-image, and ch-run.
Learn more:
CHAI is a C++ library providing an array object that can be used transparently in multiple memory spaces. CHAI can be used standalone, but is best when paired with the RAJA library, which has built-in CHAI integration that takes care of everything. This release catches CHAI up to the new version nomenclature for the RAJA Portability Suite, which has changed format to indicate year, month, and patch number.
Learn more:
ddd, which stands for Delaunay Density Diagnostic, implements algorithms described in a numerical analysis paper. The abstract follows:
Accurate approximation of a real-valued function depends on two aspects of the available data: the density of inputs within the domain of interest and the variation of the outputs over that domain. There are few methods for assessing whether the density of inputs is \textit{sufficient} to identify the relevant variations in outputs – i.e., the “geometric scale” of the function – despite the fact that sampling density is closely tied to the success or failure of an approximation method. In this paper, we introduce a general purpose, computational approach to detecting the geometric scale of real-valued functions over a fixed domain using a deterministic interpolation technique from computational geometry. Our algorithm is based on the observation that a sequence of piecewise linear interpolants will converge to a continuous function at a quadratic rate (in L2 norm) if and only if the data are sampled densely enough to distinguish the feature from noise. We present numerical experiments demonstrating how our method can identify feature scale, estimate uncertainty in feature scale, and assess the sampling density for fixed (i.e. static) datasets of input-output pairs. In addition, we include analytical results in support of our numerical findings and will release lightweight code that can be adapted for use in a variety of data science settings.
MFEM is a lightweight, general, scalable C++ library for finite element methods. It enables high-performance scalable finite element discretization research and application development on a wide variety of platforms, from laptops to supercomputers. The v4.4 release includes:
Learn more:
Developed as part of the Center for Efficient Exascale Discretizations (CEED) within the U.S. Department of Energy, libCEED is a high-order API library that provides a common algebraic low-level operator description, allowing a wide variety of applications to take advantage of the efficient operator evaluation algorithms in the different CEED packages. This release includes single precision support, a capability to assemble operators on GPUs, performance enhancements, interface and error checking improvements, and mini-app improvements.
Learn more:
krowkee is a toolkit for scalably and efficiently summarizing many data streams in distributed memory. It is intended for applications where one needs to summarize huge loosely structured data, such as matrices or graphs, and where individual components such as rows/columns or vertex adjacency information are impractical to store and directly inspect.
The Coda Calibration Tool (CCT) calculates reliable moment magnitudes for small- to moderate-sized seismic events. This release includes updates to the multi-waveform display panel, term and relative site term plots, Site and Measurement results, and more.
Learn more:
TOPE, which stands for Topology Optimization for Porous Electrodes, produced the results of a paper published in Applied Physics. The abstract follows:
Porous electrodes are an integral part of many electrochemical devices since they have high porosity to maximize electrochemical transport and high surface area to maximize activity. Traditional porous electrode materials are typically homogeneous, stochastic collections of small scale particles and offer few opportunities to engineer higher performance. Fortunately, recent breakthroughs in advanced and additive manufacturing are yielding new methods to structure and pattern porous electrodes across length scales. These architected electrodes are emerging as a promising new technology to continue to drive improvement; however, it is still unclear which structures to employ and few tools are available to guide their design. In this work we address this gap by applying topology optimization to the design of porous electrodes. We demonstrate our framework on two applications: a porous electrode driving a steady Faradaic reaction and a transiently operated electrode in a supercapacitor. We present computationally designed electrodes that minimize energy losses in a half-cell. For low conductivity materials, the optimization algorithm creates electrode designs with a hierarchy of length scales. Further, the designed electrodes are found to outperform undesigned, homogeneous electrodes. Finally, we present three-dimensional porous electrode designs. We thus establish a topology optimization framework for designing porous electrodes.
RAJA is a software abstraction that systematically encapsulates platform-specific code to enable applications to be portable across diverse hardware architectures without major source code disruption.
Note that the release version nomenclature has changed format to indicate year, month, and patch number. This format applies to the coordinated release of RAJA Portability Suite components. The v2022.03.0 release includes:
As of this release, RAJA requires CMAKE version 3.14.5 or newer and a C++14-compliant compiler to build.
Learn more:
Kosh allows codes to store, query, and share data via an easy-to-use Python API. This software aims to make data access and sharing as simple as possible. The latest release includes updates to transformers, operators, dataset cloning, dataset objects, and more.
Learn more:
Computational mathematician Julian Andrej began using LLNL-developed, open source software while in Germany. Now at Livermore, he lends his expertise to the Center for Applied Scientific Computing, developing code for next-generation computing hardware. “I know that every part of my work is contributing to a mission, and I can see a clear traceable path of its impact,” he says. Andrej contributes to MFEM and SUNDIALS projects. Read more about his work.
BLT (Building, Linking, and Testing) is a streamlined CMake build system foundation for developing HPC software. BLT makes it easy to get up and running on a wide range of HPC compilers, operating systems, and technologies. The v0.5.0 release includes added support for IntelLLVM compilers and hip targets, along with changes to the CMAKE_HIP_ARCHITECTURES variable and other modifications.
Learn more:
MuyGPyS is a Gaussian process estimation method that affords fast hyperparameter optimization by way of performing leave-one-out cross-validation. The latest release introduces just-in-time compilation and GPU support using JAX. This change allows for the acceleration of workflows on CPU (1-2.5x) and NVidia GPU (30-60x).
Learn more:
The LLNL-developed Flux project addresses challenges posed by complex scientific research supercomputing workflows, and the team has played a major role in the Exascale Computing Project’s ExaWorks effort. A recent ECP article describes how Flux works, its impact, and what’s next for the project.
CAMP collects a variety of macros and metaprogramming facilities for C++ projects. This version incorporates BLT updates to better support rocm builds, and uses the cmake setup to include a binary component that lets CAMP avoid pulling in expensive standard library headers at include time.
Learn more:
The Spack team presented a long-term roadmap at the Networking and Information Technology Research and Development (NITRD) Program’s recent meeting on software sustainability. NITRD coordinates federal research and development to identify, develop, and transition into use the Unites States’ IT, HPC, networking, and software capabilities. With more than 6,000 software packages and nearly 1,000 contributors, Spack’s long-term strategy is based around broad adoption and collaboration. Current R&D efforts include a multi-year strategic initiative aimed at reducing the human maintenance burden, as well as improvements to Spack’s continuous integration workflow and sustainability plans after the conclusion of the Exascale Computing Project.
The Scalable Checkpoint/Restart (SCR) library enables MPI applications to utilize distributed storage on Linux clusters to attain high file I/O bandwidth for checkpointing, restarting, and writing large datasets. This release includes:
Learn more:
SYNDATA includes a suite of statistical/machine learning models to generate discrete/categorical synthetic data. To train each model, the user must provide the input data from which the model parameters will be inferred. Once the models are trained, they can be used to generate entirely synthetic data. In addition to the actual models, SYNDATA includes code to process data, evaluate results (based on cross validation), and create a PDF report.
The Merlin team has published a paper in Future Generation Computer Systems that describes Merlin’s machine learning–integrated workflow system and the considerations driving its design. The authors detail Merlin’s performance results on LLNL’s Pascal and Sierra supercomputers. For example, to demonstrate Merlin’s scalability, researchers created an unprecedentedly large fusion simulation dataset consisting of the multivariate results of approximately 100 million individual simulations on Sierra. Other case studies show Merlin’s flexibility in cascading and iterative scientific workflows.
PyMFEM provides Python binding for MFEM, a high-performance parallel finite element method (FEM) library. Installer (setup.py) builds both MFEM and binding together. By default, pip install mfem downloads and builds the serial version of MFEM and PyMFEM. Additionally, the installer supports building MFEM with specific options together with other external libraries, including MPI version.
Hatchet is a Python-based library that allows Pandas dataframes to be indexed by structured tree and graph data. Version 1.3.0 includes a new interactive tree visualization in Jupyter, new APIs, new graph output formats, an updated tutorial, and more.
Learn more:
VisIt is an open source, interactive, scalable, visualization, animation, and analysis tool. Version 3.2.2 includes
build_visit featuresLearn more:
Quandary provides optimal control for open quantum systems via an optimization solver. The underlying quantum dynamics model open quantum systems, using the Lindblad master equation to evolve a density matrix in time. With this latest release, Quandary can solve either Lindblad’s master equation for the density matrix (open quantum systems), or Schrödinger’s equation for the state vector if no collapse operators are present (closed quantum systems). The latter drastically reduces the computational complexity; however, system interactions with the environment are not considered in this model.
Learn more:
The Department of Energy’s (DOE’s) Office of Science interviewed LLNL computer scientist Peter Lindstrom about his work since receiving the 2011 Early Career Award. Lindstrom leads the zfp project, which provides a compressed format for representing multidimensional floating-point and integer arrays. zfp is currently used in the DOE’s Exascale Computing Project.
libyogrt (Your One Get Remaining Time library) provides functions to query a resource manager for the time remaining in a job. It supports LCRM, LSF, MOAB, SLURM, and AIX with SLURM. The project has been around for many years and just recently added support for the Flux resource management framework.
Learn more:
LANL led with LLNL contributors, Charliecloud provides user-defined software stacks for HPC centers. It uses Linux user namespaces to run containers with no privileged operations or daemons and minimal configuration changes on center resources. Version 0.26 includes added functionality for several ch- scripts and an expanded test suite.
Learn more:
Did you know that the LLNL GitHub catalog contains several supplemental repos? They are “companions” to primary repos and provide additional data or functionality, such as examples or test cases. For instance, and this is not an exhaustive list:
MuyGPyS is a Gaussian process estimation method that affords fast hyperparameter optimization by way of performing leave-one-out cross-validation. The latest release streamlines the codebase by moving all of the computation inside of pure functions.
Learn more:
Magpie contains a number of scripts for running big data software in HPC environments. This release includes support for multiple versions of Spark and Hadoop.
Learn more:
Kosh allows codes to store, query, and share data via an easy-to-use Python API. This software aims to make data access and sharing as simple as possible. The latest release aligns Kosh with Sina and makes it the only backend. Other new features include support for ensembles, ability to clone datasets, and additional command line options.
Learn more:
UMT, which stands for Unstructured Mesh Transport, is an LLNL ASC proxy application (mini-app) that solves a thermal radiative transport equation using discrete ordinates (Sn). It utilizes an upstream corner balance method to compute the solution to the Boltzmann transport equation on unstructured spatial grids. For additional information, the repo’s README includes references to relevant radiation transport publications.
The MFEM team has announced a new FEM@LLNL seminar series focusing on finite element research and applications talks of interest to the MFEM community. Visit the MFEM website to see the full lineup of speakers. Seminars will be hosted and recorded via WebEx; videos of the recordings will be available from the MFEM website.
Abmarl is a package for developing Agent-Based Simulations and training them with MultiAgent Reinforcement Learning (MARL). It provides an intuitive CLI for engaging with the full workflow of MARL experimentation: training, visualizing, and analyzing agent behavior. Abmarl leverages RLlib’s framework for reinforcement learning and extends it to more easily support custom simulations, algorithms, and policies. Abmarl documentation is also available.
The Coda Calibration Tool (CCT) calculates reliable moment magnitudes for small- to moderate-sized seismic events. This release includes updated displays for site and measurement tabs, additional minimum and maximum coda parameter timelines, updates to the way polygon selection works, and more.
Learn more:
ExaCA is a cellular automata (CA) code for grain growth under additive manufacturing conditions by ExaAM within the Exascale Computing Project. ExaCA runs with the default enabled Kokkos backend and has been tested with Serial, OpenMP, Pthreads, CUDA, and HIP backends. Supplementary temperature data files are available in the ExaCA-Data repo.
Conduit provides an intuitive model for describing hierarchical scientific data in C++, C, Fortran, and Python. It is used for data coupling between packages in-core, serialization, and I/O tasks. The latest release includes a new partition capability for Blueprint Meshes and several new functions and methods.
Learn more:
YGM is a general-purpose, pseudo-asynchronous communication library built on top of MPI in C++. The repo’s utility is provided through its mailbox abstractions, which are used for point-to-point and broadcast communications. Releases v0.3 and v0.4 include new ygm::container functionality, API updates, improvements to subdependencies in CMake, and various performance improvements.
Learn more:
The Spack development team has launched the second annual user survey and requests responses through January 7. Last year’s survey analysis is available on the Spack website.
SUNDIALS is a SUite of Nonlinear and DIfferential/ALgebraic equation Solvers. This major release includes
SUNContext object on which all other SUNDIALS objects dependSUNMemoryHelper functionsLearn more:
croquis (“KROW-kee”) is a distributed multi-stream data sketching toolkit for scalably and efficiently summarizing many data streams in distributed memory. croquis is intended for applications involving where one needs to summarize huge loosely structured data, such as matrices or graphs, where individual components such as rows/columns or vertex adjacency information are impractical to store and directly inspect.
Thomson Scattering Cross-Section Calculator, or TSC, contains MATLAB code for calculate cross-sections associated with Thomson scattering from thermally excited plasma waves. The scripts in this repo require a MATLAB license to run.
MuyGPyS is a Gaussian process estimation method that affords fast hyperparameter optimization by way of performing leave-one-out cross-validation. The latest release overhauls the handling of the sigma_sq parameter. Additionally, the MuyGPyS tutorial has moved out of the README and into Jupyter Notebooks.
Learn more:
GIDIplus is a suite of C++ libraries for accessing nuclear data from the Generalized Nuclear Database Structure (GNDS). The latest version simplifies the build system and adds support for the draft GNDS-2.0 specification.
Learn more:
The 33rd annual Supercomputing Conference (SC21) was held in a hybrid (i.e., virtual and in-person in St. Louis) format during November 14–19.
CHAI and RAJA provide a foundation on which to build portable code. CARE expands that functionality, adding new features such as loop fusion capability, a portable interface for many numerical algorithms, and additional data structures. It provides all the basics for anyone wanting to write portable code.
Learn more:
spdlayers provides symmetric positive definite (SPD) enforcement layers for PyTorch. Examples, an API, and documentation are included in the repo.
The MFEM team held the first annual MFEM Community Workshop on October 20, 2021. MFEM, which stands for Modular Finite Element Methods, is an open source C++ software library that provides high-order mathematical algorithms for large-scale scientific simulations. The project’s discretization methods enable HPC systems to run these simulations more efficiently. More than 150 researchers from dozens of organizations and countries attended the one-day virtual workshop organized by Aaron Fisher, Tzanio Kolev, Will Pazner, and Mark Stowell. According to the registration survey, more than half of the participants were new users. An article about the workshop is available on LLNL’s Computing website, and links to videos of the presenters can be found at the MFEM website.
libROM is lightweight, scalable C++ library for data-driven physical simulation methods from the intrusive projection-based reduced order models to non-intrusive black-box approaches. The first official release adds the capability to create a sample mesh and relevant DOF maps for samples in an arbitrary number of finite element spaces.
Learn more:
Pytaridx is a Python interface for reading and appending tar files, while keeping a fast index for finding and reading files in the archive. This interface has been successfully used to store and manage over one billion files in one of the Pilot-2 simulation campaigns.
The Coda Calibration Tool (CCT) calculates reliable moment magnitudes for small- to moderate-sized seismic events. This release refactors the spectral plot displays for site and measurement tabs, adds group velocity lines to coda waveform plots, and more.
Learn more:
Spack is a flexible, configurable, Python-based, and open-source HPC package manager. Spack automates the installation and fine-tuning of simulations and libraries, operating on a wide variety of HPC platforms and enabling users to build many code configurations. Version 0.17.0 includes:
Learn more:
Caliper is a program instrumentation and performance measurement framework. It is designed as a performance analysis toolbox in a library, allowing one to bake performance analysis capabilities directly into applications and activate them at runtime. The latest release includes native Windows support, initial support for Variorum, updated report configs, and more.
Learn more:
DataRaceBench is a benchmark suite designed to systematically and quantitatively evaluate the effectiveness of data race detection tools. It includes a set of microbenchmarks with and without data races. Parallelism is represented by OpenMP directives. This version adds data race ground truth encoding into all microbenchmarks, new C/C++ microbenchmarks and Fortran microbenchmarks, new high-precision evaluation scripts, and more.
Learn more:
The renowned worldwide competition announced the winners of the 2021 R&D 100 Awards, among them LLNL’s Flux workload management framework in the Software/Services category. Watch the Flux video to learn how it enables modern supercomputing workflows.
Exascale computing will transform the ability to tackle some of the world’s most important challenge. The Exascale Computing Project (ECP) celebrates this new era of scientific discovery with the now-annual Exascale Day on October 18, or “10^18” to represent the exascale threshold of floating-point operations per second. This virtual event runs all week and provides multimedia and articles that educate explain the impact areas of exascale computing from the Department of Energy national laboratories (including LLNL), HPC manufacturers, and universities and industrial organizations. Much of the ECP’s software stack is open source.
Variorum is a platform-agnostic software library exposing monitor and control interfaces for several features in hardware architectures from IBM, ARM, and NVIDIA. In a two-part lecture series, the Variorum team demonstrates everything necessary to start using Variorum to write portable power management code. These videos were recorded as part of the Exascale Computing Project’s lecture series.
The Scalable Checkpoint/Restart (SCR) library enables MPI applications to utilize distributed storage on Linux clusters to attain high file I/O bandwidth for checkpointing, restarting, and writing large datasets. The second 3.0 release candidate includes:
SCR_Config behaviorLearn more:
Reduced order models (ROMs) combine data and underlying first principles to accelerate physical simulations, reducing computational complexity without losing accuracy. The C++ software library called libROM provides data-driven physical simulation methods from intrusive projection-based ROMs to non-intrusive black-box approaches. The project has a new website that contains documentation and examples. Additionally, computational scientist Youngsoo Choi has recorded three user tutorials, with plans to record more in the future:
YGM is a general-purpose, pseudo-asynchronous communication library built on top of MPI in C++. The repo’s utility is provided through its mailbox abstractions, which are used for point-to-point and broadcast communications. When using YGM, individual cores queue messages into a mailbox instead of directly sending them.
SUNDIALS is a SUite of Nonlinear and DIfferential/ALgebraic equation Solvers. This release includes
Learn more:
HiOp is an optimization solver for solving certain mathematical optimization problems expressed as nonlinear programming problems. This lightweight HPC solver leverages application’s existing data parallelism to parallelize the optimization iterations by using specialized linear algebra kernels. This version includes
Learn more:
Spack is a flexible, configurable, Python-based, and open-source HPC package manager. Spack automates the installation and fine-tuning of simulations and libraries, operating on a wide variety of HPC platforms and enabling users to build many code configurations. Version 0.16.3 includes
Learn more:
LANL led with LLNL contributors, Charliecloud provides user-defined software stacks for HPC centers. It uses Linux user namespaces to run containers with no privileged operations or daemons and minimal configuration changes on center resources. Version 0.25 bundles the Python module Lark (which no longer needs to be installed externally), updates how SSH data is handled in the build, and includes several other fixes and improvements.
Learn more:
MSR-SAFE allows safer access to model-specific registers. v1.6.0 changes the sysconfig override, stack size warning, and other updates.
Learn more:
The Coda Calibration Tool (CCT) calculates reliable moment magnitudes for small- to moderate-sized seismic events. This release adds two significant features including energy calculations (BETA) and a new data table filter feature. The current beta code integrates theoretical spectra to determine empirically the necessary integration constant to give the correct energy value. In the final version of the code, this will be tied to the analytical form of the seismic energy.
Learn more:
The VisIt visualization and analysis tool has supported scalable, high-quality evaluation of simulation results for over 20 years. A Research Highlight article in the latest issue of Science & Technology Review describes the software’s history and importance. Eric Brugger, who has worked on VisIt since its inception, states, “Our software is used broadly throughout the Department of Energy complex, at universities, and at other supercomputing centers. It’s running on some of the world’s most powerful machines. VisIt strikes a balance between application support and capability development that appeals to the larger HPC community.” Read “Visualization Software Stands the Test of Time” on the S&TR website.
As part of the Exascale Computing Project’s CEED software suite, FMS provides a high-order Field and Mesh Specification interface that allows a wide variety of applications and visualization tools to represent unstructured high-order meshes with general high-order finite element fields defined on them. FMS is intended as a lightweight format and API that can represent general finite elements within a common, easy to use framework. This includes high-order solutions and meshes as well as non-standard finite elements, such as Nedelec and Raviart-Thomas elements. Version 0.2 includes:
Learn more:
Quandary provides optimal control for open quantum systems via an optimization solver. The underlying quantum dynamics model open quantum systems, using the Lindblad master equation to evolve a density matrix in time. The control problem aims to find control pulses that drive the system to a desired target state. The README contains installation information as documentation is under development.
Devil Ray is a visualization and analysis library for high-order element meshes targeting modern HPC architectures. It runs on both GPUs and many-core CPUs and leverages MPI for distributed-memory computations. The repo supports meshes consisting of hexs, quads, tets, and triangles of arbitrary polynomial order, with fast paths for constant, linear, quadratic, and cubic elements. Originally architected as a ray tracer, Devil Ray is capable of rendering volumes and surfaces and has been demonstrated running concurrently on over 4,000 GPUs.
The Python-based library Hatchet will be part of a new Supercomputing 2021 (SC21) tutorial on performance tools. Hatchet allows Pandas dataframes to be indexed by structured tree and graph data and is intended for analyzing hierarchical performance data. The development team has released a video titled “User-Centric Automated Performance Analysis of Hybrid Parallel Programs” to preview the tutorial and give attendees an idea of what to expect. SC21 will take place in a hybrid format on November 14–19.
LLNL computer scientist and open source advocate Vanessa Sochat delivered a keynote presentation titled “The Stories We Tell Ourselves” at the 5th Conference of Research Software Engineers on September 6. Sochat’s work includes developing container technologies, supporting tools, and fostering open source communities. She founded and hosts the Research Software Engineer Stories podcast and is an active member of the U.S. Research Software Engineer Association.
The RAJA performance suite (RAJAPerf) is designed to explore performance of loop-based computational kernels found in HPC applications. Specifically, it is used to assess, monitor, and compare runtime performance of kernels implemented using RAJA and variants implemented using standard or vendor-supported parallel programming models directly. The v0.11.0 release adds new kernels, features, and runtime options.
Learn more:
mappgene is a SARS-CoV-2 genomic sequence analysis pipeline designed for parallel HPC. It wraps V-pipe and iVar with a collection of useful scripts for deployment in almost any Linux environment. Requires Python 3.7+ and Singularity.
GLVis is a lightweight OpenGL tool for accurate and flexible finite element visualization. Version 4.1 includes:
Learn more:
At the AWS/Arm Cloud Hackathon, LLNL’s Todd Gamblin and Greg Becker discussed (video, 31:42) the essential skills and concepts needed to understand how to create and deploy Spack recipes to build scientific codes. The Hackathon was held July 12-16 and aimed to assemble the HPC community around a common goal of beginning the porting, testing, and tuning processes for dozens of codes to use Arm-based processors.
The annual R&D 100 Awards finalists have been announced. The Flux workload management software framework is a finalist in the Software/Services category. Winners will be announced later this fall. Watch the Flux video to learn how it enables modern supercomputing workflows.
Each new season brings another hackathon, and LLNL’s summer event took place on August 12–13. The event was sponsored by the Center for Applied Scientific Computing (CASC) and Livermore Computing (LC) divisions and organized by Stephanie Brink (CASC), Tammy Dahlgren (LC), and Stephen Herbein (LC). Additionally, Computing’s summer interns were encouraged to participate in the event. Ian Lee, open source advocate and Computing’s Alternate Organizational Information System Security Officer, kicked off the hackathon with a presentation titled “When a Hackathon Project Ends…Does It Make a Sound?” He gave participants a larger picture beyond the event’s concentrated 24 hours, detailing how he has shepherded hackathon projects into real-world applications with benefits to a wide group of users and developers across the Lab.
In a new twist on documentation, the Spack package manager team created the Spack Encyclopedia, or Spackpedia, to help users understand frequently used (and sometimes funny) terms. Click on a term to see a quick definition with an option to read more. Click “read more” for additional details about the term, an example of its usage, a link to Spack documentation, and related terms. Spackpedia is a separate repo within the Spack organization on GitHub, and contributions are welcome.
Umpire is a resource management library that allows the discovery, provision, and management of memory on next-generation architectures.
As with other RAJA Portability Suite components, the release version nomenclature has changed format to indicate year, month, and patch number. The v2022.03.0 release includes new interfaces for several functions, CI updates, and more.
Learn more:
RAJA is a software abstraction that systematically encapsulates platform-specific code to enable applications to be portable across diverse hardware architectures without major source code disruption. The v0.14.0 release includes
Learn more:
CHAI is a C++ library providing an array object that can be used transparently in multiple memory spaces. CHAI can be used standalone, but is best when paired with the RAJA library, which has built-in CHAI integration that takes care of everything. This release bumps RAJA to v0.14.0, Umpire to v6.0.0, and BLT to v0.4.1.
Learn more:
The MFEM team is happy to announce the first annual MFEM Community Workshop, which will take place online on October 20, 2021.
ISO-DART, which stands for Independent System Operator Data Automated Request Tool, allows the user to create multiple automated requests of ISO data from California, Midcontinent, and New York electric power transmission system operators. The repo also includes a weather module and solar data.
As part of the Exascale Computing Project (ECP), the Center for Efficient Exascale Discretizations (CEED) is a research partnership between two U.S. Department of Energy laboratories and five universities. LLNL leads the Center. All of CEED’s software is open source.
CEED held its fifth annual meeting (CEED5AM) virtually on August 3-4. The goals of the meeting were to report on recent progress; deepen existing and establish new connections with ECP hardware vendors, ECP software technologies projects, and other collaborators; plan project activities; and work as a group to make technical progress. Presentations covered activities related to GPU support and GPU-enabled solvers, high-order methods and finite elements, software products including the AmgX linear solver library and libCEED algebraic library, benchmarking and optimization, various types of simulations enabled by CEED development, and much more.
Attendance included 97 researchers from 36 organizations:
Traumatic brain injuries (TBIs) affect millions of people each year, whether from car accidents, sports injuries, or on the battlefield. LLNL is part of the Transforming Research and Clinical Knowledge in Traumatic Brain Injury (TRACK-TBI) project, which aims to uncover fundamental new insights into how to diagnose and treat TBI as well as help deliver precision medicine to patients using advanced supercomputing and machine learning technologies. To this end, researchers have developed the open source MaPPeRTrac: the Massively Parallel, Portable, and Reproducible Tractography brain tractography workflow. MaPPeRTrac incorporates novel technologies to simplify and accelerate neuroimaging research. Read more about LLNL’s efforts to address TBI.
MFEM is a lightweight, general, scalable C++ library for finite element methods. It enables high-performance scalable finite element discretization research and application development on a wide variety of platforms, from laptops to supercomputers. The v4.3 release includes many improvements and new features, such as:
Learn more:
MuyGPyS is a Gaussian process estimation method that affords fast hyperparameter optimization by way of performing leave-one-out cross-validation. MuyGPys achieves best-in-class speed and scalability by limiting inference to the information contained in k nearest neighborhoods for prediction locations for both hyperparameter optimization and tuning. Documentation and other resources are available at the MuyGPyS Read the Docs site.
The Exascale Computing Project, a joint effort between the DOE Office of Science and NNSA, brings together several national laboratories to address many hardware, software, and application challenges inherent in the organizations’ scientific and national security missions. The ECP’s annual meeting was held virtually this year on April 12-16. Several sessions are available in a dedicated YouTube playlist. LLNL’s highlights feature open source projects that are crucial to the ECP’s collaborative goals:
Held virtually on July 15, our fifth annual Developer Day was a success. The morning session included lightning talks, a security-focused technical deep dive, and “quick takes” on remote-development resources. The afternoon session provided presentations about career paths and the Lab’s diversity and inclusion goals, capped by a career development panel discussion co-sponsored by the Data Science Institute.
The Spack team presented a tutorial at the 2021 Practice & Experience in Advanced Research Computing (PEARC’21) conference on July 19. You can view go through the tutorial yourself.
The Center for Efficient Exascale Discretizations (CEED) within the US Department of Energy’s ECP is helping applications leverage future architectures by developing state-of-the-art discretization algorithms that better exploit the hardware and deliver a significant performance gain over conventional methods. libCEED is a high-order API library that provides a common algebraic low-level operator description, allowing a wide variety of applications to take advantage of the efficient operator evaluation algorithms in the different CEED packages. This release includes portability improvements, GPU support for helper utility functions, mini-app improvements, and more.
Learn more:
Merlin is a tool for running machine learning based workflows. The goal of Merlin is to make it easy to build, run, and process the kinds of large scale HPC workflows needed for cognitive simulation. Version 1.8.0 includes new examples and fields, a reorganized tests module, GitHub Actions for handling CI, and more.
Learn more:
LANL led with LLNL contributors, Charliecloud provides user-defined software stacks for HPC centers. It uses Linux user namespaces to run containers with no privileged operations or daemons and minimal configuration changes on center resources. In version 0.24, ch-image is architecture aware, and the ch-run --bind destination path defaults to host. Many other commands are updated or improved, and the mailing list is set to announce-only.
Learn more:
Axom provides robust, flexible software infrastructure for the development of multi-physics applications and computational tools. Supported by the Advanced Simulation and Computing program at LLNL and part of the Lab’s RADIUSS project, Axom is a library of robust “building block” software components that provide foundational capabilities needed for production simulation codes. The Axom team focuses on developing software infrastructure components that can be shared by HPC applications running on diverse computing platforms. Read more about Axom on the LLNL Computing website.
The Coda Calibration Tool (CCT) calculates reliable moment magnitudes for small- to moderate-sized seismic events. This release replaces all plots with new HTML5 based plots that utilize PlotlyJS and D3.js. In addition, CCT now has updated navigation buttons and menu items, additional data exposed in the UI, upgrades to waveform and spectral plots, and much more.
Learn more:
A growing number of scientific computing applications must deliver high performance on central and graphics processing unit (CPU and GPU) hardware platforms. Compute node architectures are becoming more complex and diverse than earlier generation platforms as hardware vendors strive to deliver performance gains while adhering to physical constraints, such as power usage. Moreover, DOE laboratories are procuring machines with GPUs from three different hardware vendors. This environment makes developing applications that can run efficiently on multiple platforms increasingly time consuming and difficult.
The RAJA Portability Suite is a collection of open source software libraries that enable developers to build portable high performance applications. RAJA provides software abstractions that enable portable parallel numerical kernel execution and memory management for computing platforms with heterogeneous memory systems, specifically the Umpire and CHAI libraries. Development is supported by LLNL’s ASC Program, the RADIUSS project, and the DOE Exascale Computing Project. Read more about RAJA’s capabilities on the LLNL Computing website.
MaPPeRTrac (Massively Parallel, Portable, and Reproducible Tractography) is a brain tractography workflow for high performance computing. It incorporates novel technologies to simplify and accelerate neuroimaging research. MaPPeRTrac’s data and dependencies are de-coupled from code to enable rapid prototyping and modification. The container enables high-performance, parallel, parametrized, and portable generation of the brain’s neural connectomes that is fast, efficient, well-tested, robust, and easy-to-use.
Caliper is a program instrumentation and performance measurement framework. It is designed as a performance analysis toolbox in a library, allowing one to bake performance analysis capabilities directly into applications and activate them at runtime. The latest release includes new cuda config recipes, improvements to table and tree formatters, support for OpenMP profiling, a new API, and more.
Learn more:
The CppCast podcast recently hosted Spack creator Todd Gamblin and core developer Greg Becker on an episode (59:13) to discuss a documentation tool, a blog post about floating point numbers, and ABI changes. The podcast is created by and for C++ developers.
INGRID (Interactive Grid) is a Python-based tokamak edge plasma grid generator capable of automatic generation of grids for magnetic topologies with up to two x-points anywhere in the domain. INGRID can be utilized in a GUI mode and noninteractively via Python scripts; both modes of operation support the use of the YAML-formatted parameter file. The repo provides installation instructions and a guide to getting started. INGRID v2.0.0 is now available.
The Livermore Big Artificial Neural Network toolkit (LBANN) is an open-source, HPC-centric, deep learning training framework that is optimized to compose multiple levels of parallelism. This release includes support for new training algorithms, network structures, and layers as well as a Python front-end, performance optimizations, and much more.
Learn more:
Flux is an open-source software framework that manages and schedules computing workflows to maximize available resources to run applications faster and more efficiently. Flux’s fully hierarchical resource management and graph-based scheduling features improve the performance, portability, flexibility, and manageability of both traditional and complex scientific workflows on many types of computing systems—in the cloud, at remote locations, on a laptop, or on next-generation architectures. Watch this video to learn more about Flux (runtime 7:14).
IREP is a tool that enables mixed-language simulation codes to use a common, Lua-based format for their input decks. The repo contains link to build documentation and examples. v1.0.0 is the first official GitHub release.
Spack is a flexible, configurable, Python-based, and open-source HPC package manager. Spack automates the installation and fine-tuning of simulations and libraries, operating on a wide variety of HPC platforms and enabling users to build many code configurations. Version 0.16.2 includes
Learn more:
Conduit provides an intuitive model for describing hierarchical scientific data in C++, C, Fortran, and Python. It is used for data coupling between packages in-core, serialization, and I/O tasks. This release includes several new methods, a new example, Relay HDF5 support, and much more.
Learn more:
Vanessa Sochat has built her software engineering and computer science career in an unconventional way. After earning an undergraduate degree in Psychology, her first research assistant job involved using command line software and writing scripts. “I had no idea what I was doing, nor did anyone teach me, but I thrived in this environment,” she says. Vanessa recently joined LLNL to work on the BUILD project, Spack package manager, and other open-source initiatives. She was one of the original developers of the Singularity container technology, and she created and continues to produce the RSE Stories podcast. Read Vanessa’s profile at LLNL Computing.
Under a new memorandum of understanding, researchers at LLNL, IBM, and Red Hat will aim to enable next-generation workloads by integrating LLNL’s open source Flux scheduling framework with Red Hat OpenShift to allow more traditional HPC jobs to take advantage of cloud and container technologies. “Cloud systems are increasingly setting the directions of the broader computing ecosystem, and economics are a primary driver,” said Bronis de Supinski, CTO of Livermore Computing at LLNL. “With the growing prevalence of cloud-based systems, we must align our HPC strategy with cloud technologies, particularly in terms of their software environments, to ensure the long-term sustainability and affordability of our mission-critical HPC systems.” Read more about the agreement at LLNL News.
LLNL Computing relies on engineers like Stephanie Brink to keep the legacy codes running smoothly. “You’re only as fast as your slowest processor or your slowest function,” says Stephanie, who works in the Center for Applied Scientific Computing. By analyzing a legacy code’s performance, Stephanie and her team can reduce the amount of time it takes to run and allow for more critical science to be accomplished. Stephanie is a frequent contributor to open source software, including Hatchet and Variorum. Read the full profile at LLNL Computing.
SAMRAI (Structured Adaptive Mesh Refinement Application Infrastructure) is an object-oriented C++ software library that enables exploration of numerical, algorithmic, parallel computing, and software issues associated with applying structured adaptive mesh refinement (SAMR) technology in large-scale parallel application development. The current release features a new alias tbox::ResourceAllocator to clean up the API for usage of Umpire allocators in pdat classes. This provides a valid type name that can be used and passed through the pdat classes’ APIs regardless of the status of the configuration.
Learn more:
GIDIplus is a suite of C++ libraries for accessing nuclear data from the Generalized Nuclear Database Structure (GNDS). The latest version includes
Learn more:
The latest issue of LLNL’s Science & Technology Review magazine showcases Computing in the cover story (see abstract below) and Commentary. Open source software plays a prominent role in the initiatives described in the story. The cover art shows an advection simulation powered by open source repos MFEM and GLVis.
As a leader in high-performance computing, Lawrence Livermore wields a large portion of the Department of Energy’s HPC resources to advance national security and foundational science. The Sierra supercomputer supports the National Nuclear Security Administration’s Stockpile Stewardship Program by enabling more accurate, more predictive simulations. This generation of computers is known as heterogeneous, or hybrid, because their architectures combine graphics processing units and central processing units to achieve peak performance well above 100 petaflops. (A petaflop is 10^15 floating-point operations per second.) The next generation’s processing capability—at least an exaflop (10^18 flops)—will be many times greater. HPC software must adjust to these new hardware standards. As the exascale era begins, two major initiatives leverage and expand Livermore’s HPC capabilities, with a spotlight here on software. The Exascale Computing Project, a joint effort between the DOE Office of Science and NNSA, brings together several national laboratories to address many hardware, software, and application challenges inherent in the organizations’ scientific and national security missions. Within the Laboratory, the RADIUSS project aims to benefit scientific applications through a robust software infrastructure.
Held since 2012, LLNL’s hackathons are 24-hour opportunities to brainstorm, foster creativity, prototype, and explore. Participants work in groups or individually and often strive to learn new skills, programming languages, and tools in service to LLNL’s missions. Like the hackathons of the past year, the spring event (April 29-30) will be held virtually using WebEx and Mattermost for collaboration. LLNL sponsors are two Computing divisions: Enterprise Applications Services and National Ignition Facility Computing.
GLVis is a lightweight OpenGL tool for accurate and flexible finite element visualization. It’s been recently upgraded for wider usability:
pip install glvis to use the full power of GLVis in an interactive Python notebook.The Scalable Checkpoint/Restart (SCR) library enables MPI applications to utilize distributed storage on Linux clusters to attain high file I/O bandwidth for checkpointing, restarting, and writing large datasets. The 3.0 release candidate includes:
Learn more:
MTtime (Time Domain Moment Tensor Inversion in Python) is a python package developed for time domain inversion of complete seismic waveform data to obtain the seismic moment tensor. It supports deviatoric and full moment tensor inversions, and 1-D and 3-D basis Green’s functions. MTtime documentation and a working example are provided.
LANL led with LLNL contributors, Charliecloud provides user-defined software stacks for HPC centers. It uses Linux user namespaces to run containers with no privileged operations or daemons and minimal configuration changes on center resources. Version 0.23 includes new functionality for ch-image, ch-image build, and ch-image push.
Learn more:
The HPC world is full of complexity—from applications to the software components they rely on and the hardware they need to run. With the first three exascale machines, including Livermore’s El Capitan, slated to come online in the next few years, addressing complexity challenges will be a heavier, more urgent lift. Like our current Sierra system, these exascale systems will derive most of their computational power from secondary accelerator processors called GPUs. Traditionally, HPC systems have used only CPUs. With these machines, developers will need to accommodate not just NVIDIA accelerators but also new offerings from AMD and Intel. Harnessing the power of these devices entails using rapidly evolving programming environments, which require new compilers, runtime libraries, and software packages whose relationships are not always well understood. Without automated approaches to integration, developers will fight these software stacks by hand—but manual integration and maintenance are unsustainable.
A new effort kicking off in fiscal year 2021 aims to develop a machine-verifiable model of package compatibility that will enable automated integration, reducing human labor and errors. The Binary Understanding and Integration Logic for Dependencies (BUILD) project will run for three years with computer scientist Todd Gamblin at the helm. He states, “This project will develop techniques that enable rapid integration of HPC software systems, especially for upcoming exascale machines.” The project will build on Spack—the widely adopted package manager with a repository of more than 5,000 packages. Created by Gamblin in 2013 and today supported by a core development team, Spack already incorporates package configuration capabilities with dependency solving techniques.
The Society for Industrial and Applied Mathematics (SIAM) has announced its 2021 Class of Fellows, including LLNL computational mathematician Rob Falgout. Falgout is best known for his development of multigrid methods and for hypre, one of the world’s most popular parallel multigrid codes. LLNL News has the fully story about this honor.
BLT (Building, Linking, and Testing) is a streamlined CMake build system foundation for developing HPC software. BLT makes it easy to get up and running on a wide range of HPC compilers, operating systems, and technologies. The repo includes unit testing and benchmarking. The v0.4.0 release includes added variables, support for clang-tidy static analysis check, user option for enforcing specific versions of autoformatters, new macros, and much more.
Learn more:
PairScore is a preliminary code to predict binding affinity from the pairwise distances between protein and ligand atoms. The repo contains an example.
The Coda Calibration Tool (CCT) calculates reliable moment magnitudes for small- to moderate-sized seismic events. This release includes updates to the measurement Mw fitting algorithm, autopicking, constraints (e.g., maximum stress changed from 10 to 100 MPa), and the spectra truncation feature.
Learn more:
The Center for Efficient Exascale Discretizations (CEED) within the U.S. Department of Energy’s ECP is helping applications leverage future architectures by developing state-of-the-art discretization algorithms that better exploit the hardware and deliver a significant performance gain over conventional methods. libCEED is a high-order API library that provides a common algebraic low-level operator description, allowing a wide variety of applications to take advantage of the efficient operator evaluation algorithms in the different CEED packages. This major release includes support for matrix assembly (mainly intended for low order and coarse grids), new HIP and MAGMA backends with kernel fusion, Julia and Rust interfaces, and more.
Learn more:
HiOp is an optimization solver for solving certain mathematical optimization problems expressed as nonlinear programming problems. This lightweight HPC solver leverages application’s existing data parallelism to parallelize the optimization iterations by using specialized linear algebra kernels. This version includes centers on sparse optimization solver and enhanced support for device computations, including:
Learn more:
UnifyFS is a user-level burst buffer file system under active development. The repo supports scalable and efficient aggregation of I/O bandwidth from burst buffers while having the same life cycle as a batch-submitted job. Version 0.9.2 includes updates for newer versions of dependencies, support for setting cores-per-server via environ, config option changes, new unit and CI tests, and more.
Learn more:
From HPC Tech Shorts, this video (25:09) shows Amazon Web Services team members discussing the NoTearsHPC cluster solution for 1-click launches. Evan Bollig and Sean Smith talk about how the cluster works, what it provides, and how to do complicated tasks quickly. They used Spack for installation.
Kosh allows codes to store, query, share data via an easy-to-use Python API. Kosh lies on top of Sina and, as a result, can use any database backend supported by Sina. This software aims to make data access and sharing as simple as possible. In this backwards-compatible release:
Learn more:
STAT, the Stack Trace Analysis Tool, is a highly scalable, lightweight tool that gathers and merges stack traces from all of the processes of a parallel application to form call graph prefix trees. STAT generates two prefix trees termed 2D-trace-space and 3D-trace-space-time. STAT’s source code also includes STATBench, a tool to emulate STAT. STATBench enables the benchmarking of STAT on arbitrary machine architectures and applications by fully utilizing parallel resources and generating artificial stack traces. Version 4.1.0 adds DynInst 10.2 support and initial rocgdb support.
Learn more:
LLNL’s computing website recently underwent a major overhaul to its design and information architecture. The site now features a taxonomy of Focus Areas that connect related content. These topics are tagged on News, People Highlights, and Projects. One of the topics is open source software. The site’s Livermore Computing page also directs users to this website for more information about open source projects.
LLNL computer scientists have developed a new framework and an accompanying visualization tool that leverages deep reinforcement learning for symbolic regression problems, outperforming baseline methods on benchmark problems. The paper was recently accepted as an oral presentation at the International Conference on Learning Representations (ICLR 2021), one of the top machine learning conferences in the world. The conference takes place virtually May 3-7, and the team’s deep symbolic regression code is available in the deep-symbolic-regression GitHub repo.
Aluminum provides a generic interface to high-performance communication libraries with a focus on allreduce algorithms. Blocking and non-blocking algorithms and GPU-aware algorithms are supported. Aluminum also contains custom implementations of select algorithms to optimize for certain situations. This, the v1.0.0 release, is stable and includes refactored communicators, a barrier operation, and support for vector collectives.
Learn more:
Variorum is a platform-agnostic library exposing monitor and control interfaces for several features in hardware architectures. Its general interfaces provide privileged functionality for monitoring and controlling various hardware-level features of multiple hardware architectures. The latest release includes:
Learn more:
Juqbox.jl is a package for solving quantum optimal control problems in closed quantum systems, where the evolution of the state vector is governed by Schrodinger’s equation. See the project’s README for installation instructions and workflow details. A few examples are also provided.
LLNL computer engineer Ian Lee presented at Wild West Hackin’ Fest (WWHF) 2020, and both of his talks are now available on YouTube:
WWHF offers high-quality information security education to beginners and seasoned professionals alike. A stated goal is to lower the barrier to entry for those seeking to enter into the world of information security.
PolyClipper is a C++ reimplementation of the geometric clipping operations in the R3D library originally written by Devon Powell, as documented in the paper Powell & Abell (2015). This release transforms PolyClipper to a header-only C++ library.
Learn more:
Saloon is a Vim plugin that simplifies Python code linter/fixer configuration and usage. Saloon’s menu lets developers toggle which static analysis tools to use and delegates those changes to ALE’s API. Since prospector already handles multiple tools, and is integrated with ALE, most of the actual linting will initially be handled via prospector calls. See the project’s README for information about how to get started using Saloon.
The MFEM team has created a helpful page of tips and tricks that explain how to make the most of GPUs when running finite element algorithms. This support documentation includes information about optimizing porting and performance. Learn more about these features and processes:
MFEM_FORALL macro to enable performance portabilityQuestions or suggestions can be posted to MFEM’s issue tracker.
Held since 2012, LLNL’s hackathons are 24-hour opportunities to brainstorm, foster creativity, prototype, and explore. Participants work in groups or individually and often strive to learn new skills, programming languages, and tools in service to LLNL’s missions. Like the spring and summer hackathons of 2020, this year’s winter event (February 11-12) was held virtually using WebEx and Mattermost for collaboration. With LLNL’s Data Science Institute (DSI) sponsoring the hackathon, the agenda included guest speakers (below) discussing data science topics relevant to the Lab’s missions as well as a deep learning tutorial. (Participants were not required to attend the talks.) Read the recap on the DSI website.
Conduit provides an intuitive model for describing hierarchical scientific data in C++, C, Fortran, and Python. It is used for data coupling between packages in-core, serialization, and I/O tasks.
Learn more:
LANL led with LLNL contributors, Charliecloud provides user-defined software stacks for HPC centers. It uses Linux user namespaces to run containers with no privileged operations or daemons and minimal configuration changes on center resources. Version 0.22 includes enhancements to ch-image pull and ch-image build and more.
Learn more:
mpibind is a memory-driven algorithm to map parallel hybrid applications to the underlying heterogeneous hardware resources transparently, efficiently, and portably. Unlike other mappings, its primary design point is the memory system, including the cache hierarchy. Compute elements are selected based on a memory mapping and not vice versa. Several new incremental versions have been released simultaneously:
Aluminum provides a generic interface to high-performance communication libraries with a focus on allreduce algorithms. Blocking and non-blocking algorithms and GPU-aware algorithms are supported. Aluminum also contains custom implementations of select algorithms to optimize for certain situations. In this release, the testing and benchmarking infrastructure has been rewritten to be significantly more comprehensive and cleaner. The repo also now includes scripts for nicely plotting benchmark results.
Learn more:
SUNDIALS is a SUite of Nonlinear and DIfferential/ALgebraic equation Solvers. This release includes a new NVECTOR implementation based on the SYCL abstraction layer and a new SUNMatrix and SUNLinearSolver implementation.
Learn more:
MacPatch – used at LLNL to manage 3,000+ computers – simplifies the act of patching and installing software on macOS-based systems. The client relies on using the built-in software update application for patching the macOS system updates and its own scan and patch engine for custom patches. v3.5.0.5 is a major release with a agent installation tracking, dynamic patch list, updated MPAgentUploader, and more.
Learn more:
The 2021 IEEE Winter Conference on Applications of Computer Vision (WACV 2021) announced that a paper co-authored by Rushil Anirudh received the conference’s Best Paper Honorable Mention award based on its potential impact to the field. The paper, titled “Generative Patch Priors for Practical Compressive Image Recovery,” introduces a new kind of prior—a characterization of the space of natural images—for compressive image recovery that is trained on patches of images instead of full-sized images. Unlike existing generative methods that are applicable only to images similar to the training dataset—i.e., similar kinds of objects, image sizes or aspect ratios—the generative patch prior (GPP) can recover a wide variety of natural images and compares favorably to other existing methods, researchers said. Anirudh presented the paper on behalf of the group during an awards session hosted by the virtual conference, the premier event of its kind in the world. The conference received about 1,100 paper submissions—only 5 were honored with awards. The code used in the paper is available on the open source repository GPP on GitHub.
SPOT is a web-based visualization tool for performance data. One use case is to link the Caliper performance library into an application, and every run of the application will produce a .cali performance data file. SPOT will then visualize the collective performance of an application across many runs. This could involve tracking performance changes over time, comparing the performance achieved by different users, running scaling studies across time, and so on.
We now have three repos that provide different functionality around SPOT:
To get started, fork spot2_container and let it check out the other two repos as submodules.
Lawrence Livermore National Laboratory
7000 East Avenue • Livermore, CA 94550
Operated by Lawrence Livermore National Security, LLC, for the
Department of Energy's National Nuclear Security Administration.
