RADIUSS Policies and Guidelines

Legend

Designates a mandatory point (policy).

Designates a recommended point (guideline).

Licensing

New projects must use an OSI-approved, permissive open-source license (e.g., Apache, Apache-2.0 with LLVM-exception, MIT, or BSD 3-Clause).

Details

Similarly, any required dependencies must use an OSI-approved license that is considered compatible with the open-source permissive license for distribution purposes. Non-critical optional dependencies may use any OSI-approved license.

Rationale

A permissive open-source license is friendlier to use by commercial entities. Note that strong copyleft licenses (e.g., GPL) are not considered compatible with permissive licenses. Weaker copyleft licenses (e.g., LGPL or GPL v2 with runtime exception) can be considered compatible for distribution purposes.

Ref

xSDK M7

Provide a list of dependencies and their associated licenses in SPDX format.

Details

The SPDX License List is a list of commonly found licenses and exceptions used in open-source software that enables easy and efficient identification of such licenses. At the minimum, this should be done for all internally bundled source code, and optionally for external dependencies as well. It should preferably be placed in the NOTICE (or LICENSE) file in the root of the repository or in the bottom of a README.md. Examples:

See the Spack COPYRIGHT file.

See the Cardioid README file.

Rationale

This provides important information in a concise format to users and projects who are making decisions about whether to adopt RADIUSS products without forcing them to track down this information for each dependency.

Ref

See this link for LLNL Software Licensing Guidelines.

Existing projects should consider moving to an OSI-approved, permissive open-source license for existing projects (e.g., Apache, MIT, or BSD 3-Clause).

Details

Unlike M.lic.1, we recognize that projects that have existed for a long time may have adopted non-permissive licenses at their outset and it may be infeasible to convert. But the long-term benefits may make this worth considering.

Rationale

See M.lic.1

Build

Package your installation and provide automated builds of dependencies using Spack.

Rationale

Automating the build process prevents duplication of effort and allows others to build a copy of the code and to incorporate needed fixes in dependency tools and libraries. Spack is a package management tool that allows easy configuration with multiple dependency versions, configurations, platforms, and compilers.

Ref

Enforce WSC 1.1 and WSC 1.2

Tag

Tools

Compilable projects (e.g., those written in C++ and/or Fortran) must be buildable with CMake.

Details

We suggest considering BLT (Build, Link, and Test) which handles known CMake limitations and provides a set of templates to easily stand up a robust CMake-based build system.

Rationale

Consistent use of CMake and BLT will enable individual users, computing centers, and package managers (such as Spack) to install the package in a way that is compatible with other RADIUSS products on the same system.

Ref

WSC 3.1

Tag

Tools

Provide build options for production (optimized) and debug deployments.

Rationale

Production versions should not be compiled with debug symbols and should include platform-specific optimizations so long as they pass all tests. Debug versions should be available for users who can self-assess bugs and thus provide more specific bug reports or fixes.

Ref

Smart Libraries practice 19

Packaging

Version number and build characteristics must be easily accessible

Details

Each product API must include the functionality to return the current version number of the software and indicate which configure/CMake and compiler options were used to build the package. For development versions of the software, a build or install should allow to trace back to the commit ID of the source used.

Rationale

This allows users to make an inventory of what they have, which is critical for reproducibility, and which can aid debugging and configuration management.

Ref

Smart Libraries Practice 10 / xSDK M8

Use a limited, unique, and well-defined symbol, macro, library, and include file namespace.

Details

i.e., “Don’t pollute the global namespace”. For example, there should be no publicly visible include files with generic names such as utils.h, a package named libutil.a, or macros named YES or TRUE. Namespacing of include files can be handled either by prepending installed include files with a package name (e.g.,<XXXutils.h>) or by placing and referencing all of them in a subdirectory with the package name (e.g.,<XXX/utils.h>).

Rationale

Library developers must acknowledge that their software will be built as part of much larger packages. In that context, the use of generic names in the global namespace increases the chance that users of their library will run into build- and link-time errors that must be worked around.

Ref

Smart Libraries Practice 18 / Less restrictive than xSDK M9

Use MPI in a way that is compatible with other products.

Details

Each RADIUSS product that utilizes MPI must restrict its MPI operations to MPI communicators that are provided to it and not use directly MPI_COMM_WORLD.

Products must use configure tests or version tests to detect MPI 2 or MPI 3 features that may not be available; it should not be assumed that a full MPI 2 or MPI 3 implementation is available.

Products can change the MPI error-handling mode by default but should have an option to prevent them from changing the MPI error handling (which may have been set by another package or the application). The product should also behave appropriately regardless of the MPI error handling being used.

This rule also implies that you should not mock interfaces in a way that overrides the public API of other packages. For example, including a header file from your package should not cause other code in the same compilation unit to use a mocked MPI interface instead of the expected MPI.

Ref

xSDK M3

Version control

The source code must be version controlled using Git.

Rationale

Version control is essential for collaborative development and change tracking. Git is the most popular version control system by a wide margin, many sophisticated web-based collaboration tools are available for it, and new developers will be immediately productive in your Git repository.

Ref

WSC 2.1

Tag

Tools

Provide a transparent, online contribution process based on pull requests and issues collection. The preferred platform is GitHub.

Rationale

GitHub is one of the most-used repositories of open-source software that includes collaboration tools such as issue tracking, pull requests, and wikis. Pull requests allow project maintainers to review changes, comment, and iterate with developers before new features are committed to a code base. Pull requests and branching simplify management of large numbers of contributions (particularly contributions across teams), and they enable many features to be developed concurrently. The product maintainer can review code and associated test results before merging it into a public branch. Pull requests can also be linked to GitHub issues, which can be created by anyone.

Ref

Enforce WSC 2.3, 2.4, part of xSDK M10 and R1.

Tag

Tools

Documentation

Develop documentation from a readable plain-text format that can be publish documentation in a web-based format or as a PDF

Rationale

Web-based documentation is accessible, searchable, and linkable. It allows users to easily point each other to relevant parts of the documentation in emails, the code, and other documents. It also makes it easier to manage under revision control (see also M.doc.3). All of these tools allow developers to create web-based manuals and API documentation automatically, using a familiar, non-HTML plain-text format. Examples:

Sphinx

LaTeX

Doxygen

Ref

WSC 4.1

WSC 4.4

Provide a concise description of the software functionality wherever your project is summarized

Details

The description should typically be answering “What is it for ?” and “Should I use it ?”, and not simply an extension of an acronym or otherwise fully generic description that does not differentiate the software from other products. This description (1-3 sentences) should be consistently used in:

The GitHub host web interface (modified with the ‘Edit’ button)

The top of the root README.md

The corresponding Spack package

Rationale

Developers who come across your project or repository should be able to determine almost immediately if your project is applicable to their needs. The GitHub repo description is also used as the basis to automatically describe each project at https://software.llnl.gov/.

Version control your documentation consistent with and alongside your source code.

Rationale

When documentation is version controlled consistent with the corresponding source code, developers and users can easily retrieve the documentation associated with any revision of the code. Versioning documentation along with source code (in the same repository) also allows and encourages any developer to easily contribute to the documentation as part of their pull request.

Ref

WSC 4.3

Provide a documented, reliable way to contact the development team.

Details

Contact can be through email address, a web-based submission form, or via GitHub (issues, pull requests). In the case of email, a generic name is preferred that can be reflected to alternate or multiple team members as needed. Joining a mailing list should not be required.

Rationale

Users should be able to easily contact the development team in order to report bugs or request basic assistance. RADIUSS can provide support for those libraries included in the RADIUSS project for basic fixes and support.

Ref

xSDK M5

Provide and maintain example source code along with documentation.

Details

Examples should be compiled and tested as part of a release to make sure they are kept up to date with API changes. New examples should be added when major new features are released, or existing examples should be extended to include those new features. Existing proxy apps are a potential good starting place.

Rationale

This recommendation serves two purposes. The first is for documentation of the concrete usage of the product, training, or new feature demonstration. The second is for maintenance purpose as these examples provides a primary set of tests for non-regression or installation verification (see Testing and Continuous Integration).

Provide a documented policy for handling pull requests from external contributors

Details

Successful open source projects will attract bug fixes and feature development from the external community, preferably via pull requests on GitHub associated with a particular issue in the tracker. Teams should have a policy documenting expectations for when a pull request may be accepted, rejected, or returned for modifications.

Rationale

Developers providing fixes will be discouraged from future contributions if they feel they are being ignored. Similarly, they are probably using a branched version of your software to workaround the fix they provided, which leads to additional inefficiencies.

Tests and Continuous Integration

Use regression tests in the development process.

Rationale

Regression tests prevent new features from affecting prior functionality, and they allow developers to have more confidence in their changes when modifying code.

Ref

WSC 5.1

Provide a test suite for verifying correctness of build and installation.

Details

A series of tests (including unit tests) should be runnable by anyone after the package is built to verify a correct build. The installation should also provide tests – which could derive from or be the same as example codes – must be run and installed alongside the library for future verification.

Rationale

An executable test suite gives users confidence that the product gives correct results. It is also recommended that at least a significant subset of the tests be can be run in batch-only environments.

Ref

xSDK M2

Provide a development test suite along with the source code for external contributors.

Details

The Git repository must contain test problems an easy way to run them and verify correctness for developers who are working on remote systems. External contributors must also have access to any results from continuous integration pipelines run against their changes before they make a pull request.

Rationale

External contributors should be able to test there code to the same standard as the development team. This ensure easier integration of new contribution and transparency in the process.

Use continuous integration (CI).

Details

Example CI systems in use by RADIUSS products:

RAJA with Travis CI on GitHub. See .travis.yml

MFEM with AppVeyor on GitHub. See .appveyor.yml

Ascent with Azure Pipelines on GitHub. See azure-pipelines.yml

Rationale

CI helps to prevent integration problems by identifying issues as they arise instead of allowing them to compound. This process helps reduce rework, cost, and time. Examples of CI tools include Travis CI for projects hosted on GitHub, or GitLab CI for projects hosted on a GitLab instance such as LC-GitLab. Some products may use Azure Pipelines in the future.

Continuously check compatibility with different compilers and versions.

Details

During the development process, tests should include compilation across a matrix of compilers and compiler versions, including both open source and vendor-proprietary when available

Rationale

This helps to guard against platform-specific or compiler-specific errors.

Ensure code quality/health in an automated manner.

Details

Example tools:

Asan (AddressSanitizer)

clang-tidy

cppcheck

cpplint

Rationale

Code quality checkers help to prevent code constructs that may result in errors in future environments where the code may run. They also reduce noise in build output that may obfuscate real errors. Code style checkers can also help enforce consistent style across a project.

Ref

WSC 5.4

Portability

Give best effort at portability to common HPC platforms and compiling toolchains.

Details

Projects should also share information about the preferred and supported platforms and toolchains.

Rationale

Better portability ensures a broad base of users. This includes, but is not limited to, standard Linux distributions, GNU, Clang, and vendors’ compilers.

Ref

xSDK M4

Support Apple MacOS (Xcode) and Microsoft Windows (Visual Studio).

Rationale

This allows code development on common desktop and laptop machines.

Ref

xSDK M4