Benchpark Lists
The easiest way to get started with Benchpark is to run already specified experiments on already specified systems, or to modify one that is similar. You can search through the existing experiments and benchmarks with the below commands.
Search for available system and experiment specifications in Benchpark.
Command |
Description |
Listing in the docs |
---|---|---|
benchpark list |
Lists all benchmarks, systems, and experiments specified in Benchpark |
|
benchpark list systems |
Lists all system specified in Benchpark |
|
benchmark list benchmarks |
Lists all benchmarks specified in Benchpark |
|
benchmark list experiments |
Lists all experiments specified in Benchpark |
|
benchmark list modifiers |
Lists all modifiers specified in Benchpark |
|
benchpark tags workspace |
Lists all tags specified in Benchpark |
|
benchpark tags -a application workspace |
Lists all tags specified for a given application in Benchpark |
|
benchpark tags -t tag workspace |
Lists all experiments in Benchpark with a given tag |
|
benchpark info system <system> |
Lists all information about a given system |
|
benchpark info experiment <experiment> |
Lists all information about a given experiment |
|
benchpark bootstrap |
Manually trigger bootstrapping or update the bootstrap |
Benchpark also has a help menu:
$ benchpark --help
usage: main.py [-h] [-V] {tags,system,experiment,setup,unit-test,audit,mirror,info,show-build,list,bootstrap,analyze} ...
Benchpark
options:
-h, --help show this help message and exit
-V, --version show version number and exit
Subcommands:
{tags,system,experiment,setup,unit-test,audit,mirror,info,show-build,list,bootstrap,analyze}
tags Tags in Benchpark experiments
system Initialize a system config
experiment Interact with experiments
setup Set up an experiment and prepare it to build/run
unit-test Run benchpark unit tests
audit Look for problems in System/Experiment repos
mirror Copy a benchpark workspace
info Get information about Systems and Experiments
show-build Show how spack built a benchmark
list List experiments, systems, benchmarks, and modifiers
bootstrap Bootstrap benchpark or update an existing bootstrap
analyze Perform pre-defined analysis on the performance data (caliper files) after 'ramble on'
The benchpark list
command is used to search the available benchmarks, systems, modifiers, and experiments in Benchpark.
Seaborn not found, so skipping imports of plotting in thicket.stats
To enable this plotting, install seaborn or thicket[plotting]
usage: main.py list [-h] {benchmarks,experiments,systems,modifiers} ...
positional arguments:
{benchmarks,experiments,systems,modifiers}
List available experiments, systems, and modifiers
options:
-h, --help show this help message and exit
Seaborn not found, so skipping imports of plotting in thicket.stats
To enable this plotting, install seaborn or thicket[plotting]
Benchmarks:
ad
amg2023
babelstream
genesis
gpcnet
gromacs
hpcg
...
Seaborn not found, so skipping imports of plotting in thicket.stats
To enable this plotting, install seaborn or thicket[plotting]
Systems - SYSTEM_DEFINITION CLUSTER/INSTANCE
aws-pcluster instance_type=[c6g.xlarge|c4.xlarge|hpc7a.48xlarge|hpc6a.48xlarge]
csc-lumi
cscs-daint
cscs-eiger
generic-x86
jsc-juwels
lanl-venado cluster=[grace-hopper|grace-grace]
...
Seaborn not found, so skipping imports of plotting in thicket.stats
To enable this plotting, install seaborn or thicket[plotting]
Modifiers:
affinity
(all benchmarks)
allocation
(all benchmarks)
caliper
amg2023
babelstream
hpcg
hpl
kripke
laghos
quicksilver
qws
raja-perf
remhos
saxpy
stream
hwloc
(all benchmarks)
Seaborn not found, so skipping imports of plotting in thicket.stats
To enable this plotting, install seaborn or thicket[plotting]
Experiments - BENCHMARK+PROGRAMMING_MODEL+SCALING
ad+[mpi]
amg2023+[openmp|cuda|rocm|mpi]+[strong|weak|throughput]
babelstream+[openmp|cuda|rocm]
genesis+[openmp|mpi]
gpcnet+[mpi]
gromacs+[openmp|cuda|rocm|mpi]
hpcg+[openmp|mpi]+[strong|weak]
...
Additionally, this command can be used to search for experiments with one or more programming models:
$ benchpark list experiments --experiment openmp rocm
Seaborn not found, so skipping imports of plotting in thicket.stats
To enable this plotting, install seaborn or thicket[plotting]
Experiments - BENCHMARK+PROGRAMMING_MODEL+SCALING
amg2023+[openmp|cuda|rocm|mpi]+[strong|weak|throughput]
babelstream+[openmp|cuda|rocm]
genesis+[openmp|mpi]
gromacs+[openmp|cuda|rocm|mpi]
hpcg+[openmp|mpi]+[strong|weak]
hpl+[openmp|mpi]+[strong|weak]
kripke+[openmp|cuda|rocm|mpi]+[strong|weak|throughput]
laghos+[cuda|rocm|mpi]+[strong]
lammps+[openmp|cuda|rocm|mpi]+[strong]
osu-micro-benchmarks+[cuda|rocm|mpi]
quicksilver+[openmp|mpi]+[strong|weak]
qws+[openmp|mpi]
raja-perf+[openmp|cuda|rocm|mpi]+[strong|weak|throughput]
remhos+[cuda|rocm|mpi]+[strong]
salmon-tddft+[openmp|mpi]
saxpy+[openmp|cuda|rocm|mpi]
Or search which experiments have the Caliper modifier (see Benchpark Modifiers) available:
$ benchpark list modifiers
Seaborn not found, so skipping imports of plotting in thicket.stats
To enable this plotting, install seaborn or thicket[plotting]
Modifiers:
affinity
(all benchmarks)
allocation
(all benchmarks)
caliper
amg2023
babelstream
hpcg
hpl
kripke
laghos
quicksilver
qws
raja-perf
remhos
saxpy
stream
hwloc
(all benchmarks)
Now that you know the existing benchmarks and systems, you can determine your necessary workflow in Benchpark Workflow.