Fork me on GitHub

Testing and Continuous Integration

Back to Policies and Guidelines home


Legend

generic badge Designates a mandatory point (policy).

generic badge Designates a recommended point (guideline).

 


Generic badge Use regression tests in the development process.

Rationale
Regression tests prevent new features from affecting prior functionality, and they allow developers to have more confidence in their changes when modifying code.
Ref
WSC 5.1

Generic badge Provide a test suite for verifying correctness of build and installation.

Details
A series of tests (including unit tests) should be runnable by anyone after the package is built to verify a correct build. The installation should also provide tests – which could derive from or be the same as example codes – must be run and installed alongside the library for future verification.
Rationale
An executable test suite gives users confidence that the product gives correct results. It is also recommended that at least a significant subset of the tests be can be run in batch-only environments.
Ref
xSDK M2

Generic badge Provide a development test suite along with the source code for external contributors.

Details
The Git repository must contain test problems an easy way to run them and verify correctness for developers who are working on remote systems. External contributors must also have access to any results from continuous integration pipelines run against their changes before they make a pull request.
Rationale
External contributors should be able to test there code to the same standard as the development team. This ensure easier integration of new contribution and transparency in the process.

Generic badge Use continuous integration (CI).

Details
Example CI systems in use by RADIUSS products:
Rationale
CI helps to prevent integration problems by identifying issues as they arise instead of allowing them to compound. This process helps reduce rework, cost, and time. Examples of CI tools include Travis CI for projects hosted on GitHub, or GitLab CI for projects hosted on a GitLab instance such as LC-GitLab. Some products may use Azure Pipelines in the future.

Generic badge Continuously check compatibility with different compilers and versions.

Details
During the development process, tests should include compilation across a matrix of compilers and compiler versions, including both open source and vendor-proprietary when available
Rationale
This helps to guard against platform-specific or compiler-specific errors.

Generic badge Ensure code quality/health in an automated manner.

Details
Example tools:
  • Asan (AddressSanitizer)
  • clang-tidy
  • cppcheck
  • cpplint
Rationale
Code quality checkers help to prevent code constructs that may result in errors in future environments where the code may run. They also reduce noise in build output that may obfuscate real errors. Code style checkers can also help enforce consistent style across a project.
Ref
WSC 5.4