Building Strawman

Overview

Strawman uses CMake for its build system. Building Strawman creates two separate libraries:

  • libstrawman : a version for execution on a single node
  • libstrawman_par : a version for distributed memory parallel

The CMake variable( ENABLE_MPI ON | OFF ) controls the building the parallel version of Strawman and included proxy-apps.

The build dependencies vary according to which pipelines and proxy-applications are desired. For a minimal build with no parallel components, the following are required:

  • Conduit
  • VTK-m
    • Boost
  • C++ and Fortran compilers

Build Dependencies

Strawman

  • Conduit
  • IceT
  • One or more pipelines

Conduit

  • MPI
  • Python + NumPy (Optional)
  • HDF5 (Optional)
  • Fortran compiler (Optional)

IceT

IceT is only needed for the parallel version of Strawman.

  • MPI

Pipelines

  • EAVL:

    • OSMesa (7.5.2) is used for rendering data set annotations and is required
    • OpenMP 3.5+
    • CUDA 6.5 + (Optional)
  • VTK-m:

    • Boost
    • TBB (Optional) Intel’s Threaded Building Blocks
    • CUDA 6.5+ (Optional)

Note

When building Stawman with VTK-m 1.0 and CUDA, nvcc becomes confused and emits warnings about calling host functions from device functions. When combined with template meta-programming, this can emit quite a large amount of text during compilation. These warning messages have been suppressed in later versions of VTK-m.

  • HDF5

    • Conduit with conduit_relay HDF5 support.

Note

Strawman uses VTK-m 1.0 which must be configured with rendering on, among other options. For a full list of options that need to be set, consult /uberenv_libs/spack/var/spack/repos/builtin/packages/vtkm/package.py. If you plan to use CUDA, a patch must be applied to VTK-m to prevent a compile error. Using the build script will apply this patch automatically, but if compiling manually, the patch must be applied. The patch can be found in the source repo at /uberenv_libs/spack/var/spack/repos/builtin/packages/vtkm/vtkm_patch.patch

Getting Started

Clone the Straman repo:

  • From Github
git clone https://github.com/llnl/strawman.git
  • From LLNL’s CZ Stash Instance (LLNL Users)
git clone https://{USER_NAME}@lc.llnl.gov/stash/scm/vis/strawman.git

Configure a build:

config-build.sh is a simple wrapper for the cmake call to configure strawman. This creates a new out-of-source build directory build-debug and a directory for the install install-debug. It optionally includes a host-config.cmake file with detailed configuration options.

cd strawman
./config-build.sh

Build, test, and install Strawman:

cd build-debug
make -j 8
make test
make install

Build Options

Straman’s build system supports the following CMake options:

  • BUILD_SHARED_LIBS - Controls if shared (ON) or static (OFF) libraries are built. (default = ON)
  • ENABLE_TESTS - Controls if unit tests are built. (default = ON)
  • ENABLE_DOCS - Controls if the Strawman documentation is built (when sphinx and doxygen are found ). (default = ON)
  • ENABLE_FORTRAN - Controls if Fortran components of Strawman are built. This includes the Fortran language bindings and Cloverleaf3D . (default = ON)
  • ENABLE_PYTHON - Controls if the strawman python module and related tests are built. (default = OFF)
The Strawman python module will build for both Python 2 and Python 3. To select a specific Python, set the CMake variable PYTHON_EXECUTABLE to path of the desired python binary. The strawman python module requires the Conduit python module.
  • ENABLE_OPENMP - Controls if EAVL and proxy-apps are configured with OpenMP. (default = OFF)
  • ENABLE_CUDA - Controls if VTK-m and EAVL are configured with GPU support. (default = OFF)
  • ENABLE_MPI - Controls if parallel versions of proxy-apps and Strawman are built. (default = ON)

We are using CMake’s standard FindMPI logic. To select a specific MPI set the CMake variables MPI_C_COMPILER and MPI_CXX_COMPILER, or the other FindMPI options for MPI include paths and MPI libraries.

To run the mpi unit tests on LLNL’s LC platforms, you may also need change the CMake variables MPIEXEC and MPIEXEC_NUMPROC_FLAG, so you can use srun and select a partition. (for an example see: src/host-configs/chaos_5_x86_64.cmake)

  • CONDUIT_DIR - Path to an Conduit install (required for parallel version).
  • ICET_DIR - Path to an ICET install (required for parallel version).
  • EAVL_DIR - Path to an EAVL install (optional).
  • VTKM_DIR - Path to an VTK-m install (optional).
  • OSMESA_DIR - Path to an VTK-m install (required for EAVL).
  • HDF5_DIR - Path to a HDF5 install (optional).

Host Config Files

To handle build options, third party library paths, etc we rely on CMake’s initial-cache file mechanism.

cmake -C config_file.cmake

We call these initial-cache files host-config files, since we typically create a file for each platform or specific hosts if necessary.

The config-build.sh script uses your machine’s hostname, the SYS_TYPE environment variable, and your platform name (via uname) to look for an existing host config file in the host-configs directory at the root of the strawman repo. If found, it passes the host config file to CMake via the -C command line option.

cmake {other options} -C host-configs/{config_file}.cmake ../

You can find example files in the host-configs directory.

These files use standard CMake commands. CMake set commands need to specify the root cache path as follows:

set(CMAKE_VARIABLE_NAME {VALUE} CACHE PATH "")

It is possible to create your own configure file, and an boilerplate example is provided in /host-configs/boilerplate.cmake

Warning

If compiling all of the dependencies yourself, it is important that you use the same compilers for all dependencies. For example, different MPI and Fortran compilers (e.g., Intel and GCC) are not compatible with one another.

Bootstrapping Third Party Dependencies

You can use bootstrap-env.sh (located at the root of the strawman repo) to help setup your development environment on OSX and Linux. This script uses scripts/uberenv/uberenv.py, which leverages Spack (http://software.llnl.gov/spack) to build the external third party libraries and tools used by Strawman. Fortran support in is optional, dependencies should build without fortran. After building these libraries and tools, it writes an initial host-config file and adds the Spack built CMake binary to your PATH, so can immediately call the config-build.sh helper script to configure a strawman build.

#build third party libs using spack
source bootstrap-env.sh

#copy the generated host-config file into the standard location
cp uberenv_libs/`hostname`*.cmake to host-configs/

# run the configure helper script
./config-build.sh

# or you can run the configure helper script and give it the
# path to a host-config file
./config-build.sh uberenv_libs/`hostname`*.cmake

Compiler Settings for Third Party Dependencies

You can edit scripts/uberenv/compilers.yaml to change the compiler settings passed to Spack. See the Spack Compiler Configuration documentation for details.

For OSX, the defaults in compilers.yaml are clang from X-Code and gfortran from https://gcc.gnu.org/wiki/GFortranBinaries#MacOS.

Note

The bootstrapping process ignores ~/.spack/compilers.yaml to avoid conflicts and surprises from a user’s specific Spack settings on HPC platforms.

Building with Spack

Note

Strawman developers use scripts/uberenv/uberenv.py to setup third party libraries for Strawman development. Due to this, the process builds more libraries than necessary for most use cases. For example, we build independent installs of Python 2 and Python 3 to make it easy to check Python C-API compatibility during development. In the near future, we plan to provide a Spack package to simplify deployment.

Using Strawman in Another Project

Under src/examples there are examples demonstrating how to use Strawman in a CMake-based build system (using-with-cmake) and via a Makefile (using-with-make). Under src/examples/proxies you can find example integrations using strawman in the Lulesh, Kripke, and Cloverleaf3D proxy-applications.

Building Strawman in a Docker Container

Under src/examples/docker/master/ubuntu there is an example Dockerfile which can be used to create an ubuntu-based docker image with a build of the Strawman github master branch. There is also a script that demonstrates how to build a Docker image from the Dockerfile (example_build.sh) and a script that runs this image in a Docker container (example_run.sh). The Strawman repo is cloned into the image’s file system at /strawman, the build directory is /strawman/build-debug, and the install directory is /strawman/install-debug.