Analyze a Scaling Study

Glossary terms: Benchmark, Experiment, System, Modifier, Scaling Study

Prerequisite: Build and Run an Experiment

Install the analysis dependencies once from the Benchpark repository (this has already been installed in the container for the hands-on tutorials):

$ pip install .[analyze]

Analyze the completed HPL scaling study from Build and Run an Experiment:

Strong Scaling

The default stacked area chart shows the exclusive average time per rank for each instrumented region. The x-axis records the node and MPI-rank counts varied in the strong-scaling study.

~/benchpark]$ benchpark analyze --workspace-dir wkp/fluxtainer-aws/hpl/workspace

../_images/hpl_mpi_strong_raw_exc.png — Strong-scaling area chart.

Percentage

The percentage chart expresses each region’s metric relative to the sum for all regions. This makes each region’s share of total time easier to compare.

~/benchpark]$ benchpark analyze --workspace-dir wkp/fluxtainer-aws/hpl/workspace --chart-type percentage

../_images/hpl_mpi_strong_percentage_exc.png — Strong-scaling data shown as a percentage of total time.

Inclusive Metrics

The inclusive chart uses Avg time/rank instead of Avg time/rank (exc).

~/benchpark]$ benchpark analyze --workspace-dir wkp/fluxtainer-aws/hpl/workspace --yaxis-metric "Avg time/rank"

../_images/hpl_mpi_strong_raw.png — Strong-scaling data using an inclusive metric.

Region Filtering

This figure combines regions that have the same name, then retains the regions with the highest metric values. The filtered view emphasizes the regions that contribute most to the measured time.

~/benchpark]$ benchpark analyze --workspace-dir wkp/fluxtainer-aws/hpl/workspace --group-regions-name --top-n-regions 10

../_images/hpl_mpi_strong_raw_exc_region.png — Strong-scaling data after filtering regions.

For additional chart types and filtering options, see Analyze a Scaling Study.

Choose Your Next Step

I have generated preconfigured performance figures for an experiment on my system. I want to begin another workflow by searching what is available in Benchpark.