Compare Experiment Builds

lib/scripts/diffExperimentBuilds.py

Compare two experiment builds, generated by Spack as part of ramble workspace setup command. This script builds the benchmarks that are available for the specified programmming-model on the target cluster and compares the differences (leveraging diffSpecs.py) for two different versions of benchpark.

Stages that occur during this script:

  • benchpark experiment init

  • benchpark system init

  • benchpark setup

  • ramble workspace setup

  • ramble on (optional)

Note

This script must be ran on the target system and cluster to work correctly.

Example: amg2023~mpi

For example, we can compare the builds for different benchpark branches of amg2023 on the ruby cluster. myBranch is a branch of benchpark where we have configured ~mpi.

$ benchpark-python diffExperimentBuilds.py -s llnl-cluster -c ruby -p openmp -b amg2023 --commit-hash2 myBranch

# Develop build
==> Streaming details to log:
==>   /usr/WS1/mckinsey/benchpark/lib/scripts/benchpark-develop-openmp/wkp/benchpark-develop-openmp/amg2023/benchpark-develop-openmp/ruby/workspace/logs/setup.2025-03-27_13.19.33.out
==>   Setting up 1 out of 1 experiments:
==> Experiment #1 (1/1):
==>     name: amg2023.problem1.amg2023_problem1_single_node_openmp_caliper_none_2_2_2_80_80_80_8_1
==>     root experiment_index: 1
==>     log file: /usr/WS1/mckinsey/benchpark/lib/scripts/benchpark-develop-openmp/wkp/benchpark-develop-openmp/amg2023/benchpark-develop-openmp/ruby/workspace/logs/setup.2025-03-27_13.19.33/amg2023.problem1.amg2023_problem1_single_node_openmp_caliper_none_2_2_2_80_80_80_8_1.out
==>   Returning to log file: /usr/WS1/mckinsey/benchpark/lib/scripts/benchpark-develop-openmp/wkp/benchpark-develop-openmp/amg2023/benchpark-develop-openmp/ruby/workspace/logs/setup.2025-03-27_13.19.33.out

# myBranch build
==> Streaming details to log:
==>   /usr/WS1/mckinsey/benchpark/lib/scripts/benchpark-myBranch-openmp/wkp/benchpark-myBranch-openmp/amg2023/benchpark-myBranch-openmp/ruby/workspace/logs/setup.2025-03-27_13.28.29.out
==>   Setting up 1 out of 1 experiments:
==> Experiment #1 (1/1):
==>     name: amg2023.problem1.amg2023_problem1_single_node_openmp_caliper_none_2_2_2_80_80_80_8_1
==>     root experiment_index: 1
==>     log file: /usr/WS1/mckinsey/benchpark/lib/scripts/benchpark-myBranch-openmp/wkp/benchpark-myBranch-openmp/amg2023/benchpark-myBranch-openmp/ruby/workspace/logs/setup.2025-03-27_13.28.29/amg2023.problem1.amg2023_problem1_single_node_openmp_caliper_none_2_2_2_80_80_80_8_1.out
==>   Returning to log file: /usr/WS1/mckinsey/benchpark/lib/scripts/benchpark-myBranch-openmp/wkp/benchpark-myBranch-openmp/amg2023/benchpark-myBranch-openmp/ruby/workspace/logs/setup.2025-03-27_13.28.29.out

The specs for amg2023 are different.
   amg2023+mpi [develop]
   -> [openmpi]
      hypre+mpi
         openblas
         openmpi
   amg2023~mpi [myBranch]
      hypre~mpi
         openblas
   DifferentSpecs=True

Optionally the experiments can be run using the --run-experiment argument.