Python reader API

The caliper-reader is a Python API for reading Caliper’s .cali file format. You can install it via pip:

$ pip install caliper-reader

or add the python/caliper-reader directory in the cloned Caliper repository to PYTHONPATH.

Usage

The caliperreader.CaliperReader class reads a Caliper file and then provides its contents in the records and globals class members, where records is a Python list-of-dicts containing the recorded performance data and globals is a Python dict with program metadata about the run. The dicts represent Caliper attribute:value records: the key is the Caliper attribute name; the value is a string or list of strings. The example below prints the avg#inclusive#sum#time.duration metric for every region path in the provided example profile data file:

import caliperreader as cr

r = cr.CaliperReader()
r.read('example-profile.cali')

metric = 'avg#inclusive#sum#time.duration'

for rec in r.records:
    path = rec['path'] if 'path' in rec else 'UNKNOWN'
    time = rec[metric] if metric in rec else '0'

    if (isinstance(path, list)):
        path = "/".join(path)

    print("{0}: {1}".format(path, time))

The CaliperReader attributes() function returns the list of Caliper attributes. The attribute() function returns an Attribute object to query metadata for a given Caliper attribute name:

a = r.attribute('avg#inclusive#sum#time.duration')
a.get('attribute.unit')
'sec'

You can use the caliperreader.read_caliper_contents() function as a shortcut to read Caliper data without creating a CaliperReader object:

(records,globals) = cr.read_caliper_contents('example-profile.cali')

API reference

caliperreader.read_caliper_contents(filename_or_stream)

Reads a Caliper .cali file and returns its contents.

Arguments:

filename_or_stream:

Stream object or the filename of the .cali file.

Returns:

(records, globals):

A tuple with the Caliper snapshot records and globals. The first element contains the snapshot record as a list of dicts. The second element contains the global key:value attributes as a dict.

caliperreader.read_caliper_globals(filename_or_stream)

Reads a Caliper .cali file and returns its global attributes.

Arguments:

filename_or_stream:

Stream object or the filename of the .cali file.

Returns:

dict:

The global key:value attributes as a dict.

class caliperreader.CaliperReader

Reads a Caliper .cali file and exports its contents.

Use the read() method to read a Caliper .cali file. Contents are then available in the “records” and “globals” attributes. Each CaliperReader object should read a single file.

Attributes:

recordslist-of-dicts

The snapshot records read from the .cali file.

globalsdict

The global key:value attributes defined in the .cali file

attribute(attribute_name)

Return a Caliper Attribute object for the given attribute name.

attributes()

Return the list of attribute names.

A file must have been read with read(). Only returns attribute keys not marked as hidden.

Returns:

The attribute keys (list of strings).

read(filename_or_stream)

Read a .cali file or stream.

After reading, the file’s contents are available in the CaliperReader object’s “records” and “globals” attributes.

Arguments:

filename_or_streamstr or stream object

The filename or stream to read.

class caliperreader.CaliperStreamReader

Reads a Caliper .cali data stream

Use the read() method to read a Caliper .cali file and process records using callback methods.

Attributes:

globalsdict

The global key:value attributes defined in the .cali file

num_recordsdict

The number of different kinds of records read

attribute(attribute_name)

Return a Caliper Attribute object for the given attribute name.

attributes()

Return the list of attribute names.

A file must have been read with read().

Returns:

The attribute keys (list of strings).

read(filename_or_stream, process_record_fn=None)

Read a .cali file or stream.

Snapshot records being read are fed to the user-provided process_record_fn callback function. After reading, global (metadata) values are stored in the globals attribute.

Arguments:

filename_or_streamstr or stream object

The filename or stream to read.

process_record_fnFunction accepting a dict

A callback function to process performance data records

class caliperreader.metadatadb.Attribute(node)

A Caliper attribute key.

Provides information about Caliper attribute keys.

attribute_type()

Return the name of the attribute’s Caliper data type.

get(attribute)

Return metadata value for this attribute.

Arguments:

attributestring or integer

Metadata attribute name or ID

Returns:

(object):

The metadata value.

id()

Return the node ID of the Caliper attribute.

is_aggregatable()

Is this an aggregatable attribute (i.e., a metric)?

is_global()

Is this a global attribute (run metadata)?

Attributes with the “global” property describe run metadata like the execution environment or program configuration.

is_hidden()

Is this a hidden attribute?

is_nested()

Does this attribute have the Caliper “nested” property?

Attributes with the “nested” property define the hierarchy of Caliper begin/end annotations.

is_value()

Does this attribute have the “as_value” property?

Attributes with the “as_value” property are stored directly in snapshot records.

metadata()

Return a dict with all metadata entries for this attribute

name()

Return the name of the Caliper attribute.

scope()

Returns the attribute’s scope (“process”, “thread”, or “task”)