RADIUSS Policies and Guidelines


Home

RADIUSS Policies and Guidelines

« Back to Policies and Guidelines home


Legend

generic badge Designates a mandatory point (policy).

generic badge Designates a recommended point (guideline).


Licensing

M.lic.1 New projects must use an OSI-approved, permissive open-source license (e.g., Apache, Apache-2.0 with LLVM-exception, MIT, or BSD 3-Clause).

Details
Similarly, any required dependencies must use an OSI-approved license that is considered compatible with the open-source permissive license for distribution purposes. Non-critical optional dependencies may use any OSI-approved license.
Rationale
A permissive open-source license is friendlier to use by commercial entities. Note that strong copyleft licenses (e.g., GPL) are not considered compatible with permissive licenses. Weaker copyleft licenses (e.g., LGPL or GPL v2 with runtime exception) can be considered compatible for distribution purposes.
Ref
xSDK M7

M.lic.2 Provide a list of dependencies and their associated licenses in SPDX format.

Details
The SPDX License List is a list of commonly found licenses and exceptions used in open-source software that enables easy and efficient identification of such licenses. At the minimum, this should be done for all internally bundled source code, and optionally for external dependencies as well. It should preferably be placed in the NOTICE (or LICENSE) file in the root of the repository or in the bottom of a README.md. Examples:
Rationale
This provides important information in a concise format to users and projects who are making decisions about whether to adopt RADIUSS products without forcing them to track down this information for each dependency.
Ref
See this link for LLNL Software Licensing Guidelines.

R.lic.3 Existing projects should consider moving to an OSI-approved, permissive open-source license for existing projects (e.g., Apache, MIT, or BSD 3-Clause).

Details
Unlike M.lic.1, we recognize that projects that have existed for a long time may have adopted non-permissive licenses at their outset and it may be infeasible to convert. But the long-term benefits may make this worth considering.
Rationale
See M.lic.1

Build

Generic badge Package your installation and provide automated builds of dependencies using Spack.

Rationale
Automating the build process prevents duplication of effort and allows others to build a copy of the code and to incorporate needed fixes in dependency tools and libraries. Spack is a package management tool that allows easy configuration with multiple dependency versions, configurations, platforms, and compilers.
Ref
Enforce WSC 1.1 and WSC 1.2
Tag
Tools

Generic badge Compilable projects (e.g., those written in C++ and/or Fortran) must be buildable with CMake.

Details
We suggest considering BLT (Build, Link, and Test) which handles known CMake limitations and provides a set of templates to easily stand up a robust CMake-based build system.
Rationale
Consistent use of CMake and BLT will enable individual users, computing centers, and package managers (such as Spack) to install the package in a way that is compatible with other RADIUSS products on the same system.
Ref
WSC 3.1
Tag
Tools

Generic badge Provide build options for production (optimized) and debug deployments.

Rationale
Production versions should not be compiled with debug symbols and should include platform-specific optimizations so long as they pass all tests. Debug versions should be available for users who can self-assess bugs and thus provide more specific bug reports or fixes.
Ref
Smart Libraries practice 19

Packaging

Generic badge Version number and build characteristics must be easily accessible

Details
Each product API must include the functionality to return the current version number of the software and indicate which configure/CMake and compiler options were used to build the package. For development versions of the software, a build or install should allow to trace back to the commit ID of the source used.
Rationale
This allows users to make an inventory of what they have, which is critical for reproducibility, and which can aid debugging and configuration management.
Ref
Smart Libraries Practice 10 / xSDK M8

Generic badge Use a limited, unique, and well-defined symbol, macro, library, and include file namespace.

Details
i.e., “Don’t pollute the global namespace”. For example, there should be no publicly visible include files with generic names such as utils.h, a package named libutil.a, or macros named YES or TRUE. Namespacing of include files can be handled either by prepending installed include files with a package name (e.g.,<XXXutils.h>) or by placing and referencing all of them in a subdirectory with the package name (e.g.,<XXX/utils.h>).
Rationale
Library developers must acknowledge that their software will be built as part of much larger packages. In that context, the use of generic names in the global namespace increases the chance that users of their library will run into build- and link-time errors that must be worked around.
Ref
Smart Libraries Practice 18 / Less restrictive than xSDK M9

Generic badge Use MPI in a way that is compatible with other products.

Details

  • Each RADIUSS product that utilizes MPI must restrict its MPI operations to MPI communicators that are provided to it and not use directly MPI_COMM_WORLD.
  • Products must use configure tests or version tests to detect MPI 2 or MPI 3 features that may not be available; it should not be assumed that a full MPI 2 or MPI 3 implementation is available.
  • Products can change the MPI error-handling mode by default but should have an option to prevent them from changing the MPI error handling (which may have been set by another package or the application). The product should also behave appropriately regardless of the MPI error handling being used.
  • This rule also implies that you should not mock interfaces in a way that overrides the public API of other packages. For example, including a header file from your package should not cause other code in the same compilation unit to use a mocked MPI interface instead of the expected MPI.
Ref
xSDK M3

Version control

Generic badge The source code must be version controlled using Git.

Rationale
Version control is essential for collaborative development and change tracking. Git is the most popular version control system by a wide margin, many sophisticated web-based collaboration tools are available for it, and new developers will be immediately productive in your Git repository.
Ref
WSC 2.1
Tag
Tools

Generic badge Provide a transparent, online contribution process based on pull requests and issues collection. The preferred platform is GitHub.

Rationale
GitHub is one of the most-used repositories of open-source software that includes collaboration tools such as issue tracking, pull requests, and wikis. Pull requests allow project maintainers to review changes, comment, and iterate with developers before new features are committed to a code base. Pull requests and branching simplify management of large numbers of contributions (particularly contributions across teams), and they enable many features to be developed concurrently. The product maintainer can review code and associated test results before merging it into a public branch. Pull requests can also be linked to GitHub issues, which can be created by anyone.
Ref
Enforce WSC 2.3, 2.4, part of xSDK M10 and R1.
Tag
Tools

Documentation

Generic badge Develop documentation from a readable plain-text format that can be publish documentation in a web-based format or as a PDF

Rationale
Web-based documentation is accessible, searchable, and linkable. It allows users to easily point each other to relevant parts of the documentation in emails, the code, and other documents. It also makes it easier to manage under revision control (see also M.doc.3). All of these tools allow developers to create web-based manuals and API documentation automatically, using a familiar, non-HTML plain-text format. Examples:
Ref
WSC 4.1
WSC 4.4

Generic badge Provide a concise description of the software functionality wherever your project is summarized

Details
The description should typically be answering “What is it for ?” and “Should I use it ?”, and not simply an extension of an acronym or otherwise fully generic description that does not differentiate the software from other products. This description (1-3 sentences) should be consistently used in:
  • The GitHub host web interface (modified with the ‘Edit’ button)
  • The top of the root README.md
  • The corresponding Spack package
Rationale
Developers who come across your project or repository should be able to determine almost immediately if your project is applicable to their needs. The GitHub repo description is also used as the basis to automatically describe each project at https://software.llnl.gov/.

Generic badge Version control your documentation consistent with and alongside your source code.

Rationale
When documentation is version controlled consistent with the corresponding source code, developers and users can easily retrieve the documentation associated with any revision of the code. Versioning documentation along with source code (in the same repository) also allows and encourages any developer to easily contribute to the documentation as part of their pull request.
Ref
WSC 4.3

Generic badge Provide a documented, reliable way to contact the development team.

Details
Contact can be through email address, a web-based submission form, or via GitHub (issues, pull requests). In the case of email, a generic name is preferred that can be reflected to alternate or multiple team members as needed. Joining a mailing list should not be required.
Rationale
Users should be able to easily contact the development team in order to report bugs or request basic assistance. RADIUSS can provide support for those libraries included in the RADIUSS project for basic fixes and support.
Ref
xSDK M5

Generic badge Provide and maintain example source code along with documentation.

Details
Examples should be compiled and tested as part of a release to make sure they are kept up to date with API changes. New examples should be added when major new features are released, or existing examples should be extended to include those new features. Existing proxy apps are a potential good starting place.
Rationale
This recommendation serves two purposes. The first is for documentation of the concrete usage of the product, training, or new feature demonstration. The second is for maintenance purpose as these examples provides a primary set of tests for non-regression or installation verification (see Testing and Continuous Integration).

Generic badge Provide a documented policy for handling pull requests from external contributors

Details
Successful open source projects will attract bug fixes and feature development from the external community, preferably via pull requests on GitHub associated with a particular issue in the tracker. Teams should have a policy documenting expectations for when a pull request may be accepted, rejected, or returned for modifications.
Rationale
Developers providing fixes will be discouraged from future contributions if they feel they are being ignored. Similarly, they are probably using a branched version of your software to workaround the fix they provided, which leads to additional inefficiencies.

Tests and Continuous Integration

Generic badge Use regression tests in the development process.

Rationale
Regression tests prevent new features from affecting prior functionality, and they allow developers to have more confidence in their changes when modifying code.
Ref
WSC 5.1

Generic badge Provide a test suite for verifying correctness of build and installation.

Details
A series of tests (including unit tests) should be runnable by anyone after the package is built to verify a correct build. The installation should also provide tests – which could derive from or be the same as example codes – must be run and installed alongside the library for future verification.
Rationale
An executable test suite gives users confidence that the product gives correct results. It is also recommended that at least a significant subset of the tests be can be run in batch-only environments.
Ref
xSDK M2

Generic badge Provide a development test suite along with the source code for external contributors.

Details
The Git repository must contain test problems an easy way to run them and verify correctness for developers who are working on remote systems. External contributors must also have access to any results from continuous integration pipelines run against their changes before they make a pull request.
Rationale
External contributors should be able to test there code to the same standard as the development team. This ensure easier integration of new contribution and transparency in the process.

Generic badge Use continuous integration (CI).

Details
Example CI systems in use by RADIUSS products:
Rationale
CI helps to prevent integration problems by identifying issues as they arise instead of allowing them to compound. This process helps reduce rework, cost, and time. Examples of CI tools include Travis CI for projects hosted on GitHub, or GitLab CI for projects hosted on a GitLab instance such as LC-GitLab. Some products may use Azure Pipelines in the future.

Generic badge Continuously check compatibility with different compilers and versions.

Details
During the development process, tests should include compilation across a matrix of compilers and compiler versions, including both open source and vendor-proprietary when available
Rationale
This helps to guard against platform-specific or compiler-specific errors.

Generic badge Ensure code quality/health in an automated manner.

Details
Example tools:
  • Asan (AddressSanitizer)
  • clang-tidy
  • cppcheck
  • cpplint
Rationale
Code quality checkers help to prevent code constructs that may result in errors in future environments where the code may run. They also reduce noise in build output that may obfuscate real errors. Code style checkers can also help enforce consistent style across a project.
Ref
WSC 5.4

Portability

Generic badge Give best effort at portability to common HPC platforms and compiling toolchains.

Details
Projects should also share information about the preferred and supported platforms and toolchains.
Rationale
Better portability ensures a broad base of users. This includes, but is not limited to, standard Linux distributions, GNU, Clang, and vendors’ compilers.
Ref
xSDK M4

Generic badge Support Apple MacOS (Xcode) and Microsoft Windows (Visual Studio).

Rationale
This allows code development on common desktop and laptop machines.
Ref
xSDK M4