Python support

Caliper provides Python bindings based on pybind11 for the annotation and ConfigManager APIs. To build Caliper with Python support, enable the WITH_PYTHON_BINDINGS option in the CMake configuration:

$ cmake -DWITH_PYTHON_BINDINGS=On ..

Using the Python module

The Python module requires pybind11 and an installation of Python that both supports pybind11 and provides development headers (e.g., Python.h) and libraries (e.g., libpython3.8.so).

The Caliper Python module is installed in either lib/pythonX.Y/site-packages/ and/or lib64/pythonX.Y/site-packages in the Caliper installation directory. In these paths, X.Y corresponds to the major and minor version numbers of the Python installation used. Additionally, lib/ and lib64/ will be used in accordance with the configuration of the Python installed. To better understand the rules for where Python modules are installed, see this thread from the Python Software Foundation Discuss.

To use the Caliper Python module, simply add the directories above to PYTHONPATH or sys.path. Note that the module will be automatically added to PYTHONPATH when loading the Caliper package with Spack if the python variant is enabled. The module can then be imported with import pycaliper.

Caliper Python API

The Caliper Python API supports a significant subset of the C and C++ annotation APIs. The simplest options are the pycaliper.begin_region() and pycaliper.end_region() functions. Caliper’s Python API also provides the pycaliper.annotate_function decorator as a higher-level way of annotating functions.

The Python API also supports the Caliper ConfigManager API (ConfigManager API reference). The example is examples/apps/py-example.py demonstrates the annotation and ConfigManager APIs for Python:

# Copyright (c) 2024, Lawrence Livermore National Security, LLC.
# See top-level LICENSE file for details.

from pycaliper.high_level import annotate_function
from pycaliper.config_manager import ConfigManager
from pycaliper.instrumentation import (
    set_global_byname,
    begin_region,
    end_region,
)
from pycaliper.loop import Loop

import argparse
import sys
import time


def get_available_specs_doc(mgr: ConfigManager):
    doc = ""
    for cfg in mgr.available_config_specs():
        doc += mgr.get_documentation_for_spec(cfg)
        doc += "\n"
    return doc


@annotate_function()
def foo(i: int) -> float:
    nsecs = max(i * 500, 100000)
    secs = nsecs / 10**9
    time.sleep(secs)
    return 0.5 * i


def main():
    mgr = ConfigManager()

    parser = argparse.ArgumentParser()
    parser.add_argument("--caliper_config", "-P", type=str, default="",
                        help="Configuration for Caliper\n{}".format(get_available_specs_doc(mgr)))
    parser.add_argument("iterations", type=int, nargs="?", default=4,
                        help="Number of iterations")
    args = parser.parse_args()
    
    mgr.add(args.caliper_config)

    if mgr.error():
        print("Caliper config error:", mgr, file=sys.stderr)

    mgr.start()

    set_global_byname("iterations", args.iterations)
    set_global_byname("caliper.config", args.caliper_config)
    
    begin_region("main")
    
    begin_region("init")
    t = 0
    end_region("init")
    
    loop_ann = Loop("mainloop")
    
    for i in range(args.iterations):
        loop_ann.start_iteration(i)
        t *= foo(i)
        loop_ann.end_iteration()

    loop_ann.end()

    end_region("main")
    
    mgr.flush()
    

if __name__ == "__main__":
    main()