Output formatting¶
Caliper provides a variety of formatters to create machine or human-readable output. These can be used in all I/O services and tools that process CalQL, e.g. cali-query, the report and mpireport services, as well as the report generation API.
Formatter configuration¶
To select a formatter, use the FORMAT statement in CalQL. Some formatters, e.g. tree, take additional arguments in a “function call” syntax, e.g.
FORMAT tree(source.function#callpath.address)
The SELECT CalQL statement determines the attributes that will be written by the formatter (all other attributes will be ignored). If applicable, attributes will be printed in the order they are listed in the SELECT statement.
Some formatters (e.g., table) support the ORDER BY CalQL statement to print records in a given order, but not all of them do.
Cali¶
The cali formatter writes Caliper raw data that can be read by
cali-query, the Python reader API, and the caliper_native_reader
importer in Hatchet.
Expand¶
The expand formatter writes records as comma-separated key-value
lists, with each record in a single line. Hierachical values are
written with slash / separators.
Example:
SELECT event.end#function,count(),sum(time.inclusive.duration) GROUP BY event.end#function FORMAT expand
Output:
event.end#function=TimeIncrement,count=100,time.inclusive.duration=1280
event.end#function=IntegrateStressForElems,count=100,time.inclusive.duration=358512
event.end#function=CalcFBHourglassForceForElems,count=100,time.inclusive.duration=700886
event.end#function=CalcHourglassControlForElems,count=100,time.inclusive.duration=1073630
event.end#function=CalcVolumeForceForElems,count=100,time.inclusive.duration=1456613
event.end#function=CalcForceForNodes,count=100,time.inclusive.duration=1466051
event.end#function=LagrangeNodal,count=100,time.inclusive.duration=1508828
Table¶
The table formatter prints a human-readable text table with automatically determined column widths. It supports ORDER BY.
Example:
SELECT event.end#function,count(),sum(time.inclusive.duration) GROUP BY event.end#function FORMAT table ORDER BY time.inclusive.duration DESC
Output:
event.end#function count time.inclusive.duration
main 1 3395643
LagrangeLeapFrog 100 3379753
LagrangeElements 100 1755059
LagrangeNodal 100 1508828
CalcForceForNodes 100 1466051
CalcVolumeForceForElems 100 1456613
ApplyMaterialPropertiesForElems 100 1163596
EvalEOSForElems 1100 1113417
CalcHourglassControlForElems 100 1073630
CalcEnergyForElems 3500 805605
CalcFBHourglassForceForElems 100 700886
IntegrateStressForElems 100 358512
CalcPressureForElems 10500 324182
CalcQForElems 100 309658
CalcLagrangeElements 100 266492
CalcKinematicsForElems 100 241132
CalcMonotonicQGradientsForElems 100 149492
CalcTimeConstraintsForElems 100 110236
CalcMonotonicQRegionForElems 1100 98944
CalcCourantConstraintForElems 1100 49281
CalcHydroConstraintForElems 1100 33125
CalcSoundSpeedForElems 1100 27975
UpdateVolumesForElems 100 8147
TimeIncrement 100 1280
Tree¶
The tree formatter prints records in hierarchical form based on the input data, such as nested functions and regions.
The attribute used to build the hierarchy can be given as an argument
to the formatter (e.g. FORMAT tree(function) to use function). By
default, the hierarchy is built from all attributes flagged as NESTED,
which includes the default function, loop, and region
annotations.
Example:
SELECT count(),sum(time.inclusive.duration) WHERE event.end#function GROUP BY function FORMAT tree
Output:
Path count time.inclusive.duration
main 1 3395643
LagrangeLeapFrog 100 3379753
CalcTimeConstraintsForElems 100 110236
CalcHydroConstraintForElems 1100 33125
CalcCourantConstraintForElems 1100 49281
LagrangeElements 100 1755059
UpdateVolumesForElems 100 8147
ApplyMaterialPropertiesForElems 100 1163596
EvalEOSForElems 1100 1113417
CalcSoundSpeedForElems 1100 27975
CalcEnergyForElems 3500 805605
CalcPressureForElems 10500 324182
CalcQForElems 100 309658
CalcMonotonicQRegionForElems 1100 98944
CalcMonotonicQGradientsForElems 100 149492
CalcLagrangeElements 100 266492
CalcKinematicsForElems 100 241132
LagrangeNodal 100 1508828
CalcForceForNodes 100 1466051
CalcVolumeForceForElems 100 1456613
CalcHourglassControlForElems 100 1073630
CalcFBHourglassForceForElems 100 700886
IntegrateStressForElems 100 358512
TimeIncrement 100 1280
Json¶
The json formatter writes a list-of-dicts style json file, where
each Caliper record is a JSON object with attribute:value
entries. Hierarchical values are written as a single string with a
slash (/) as separator. Note that the json formatter usually
produces larger files than json-split.
Optional arguments:
- pretty
More human-readable output (inserts extra whitespace and line breaks).
- split
Just write the records, skip the outer “[ ]” array delimiters.
- quote-all
Write all values as strings using
"quotes, even numerical ones.
Example:
SELECT function,loop,count(),sum(time.inclusive.duration) WHERE event.end#function GROUP BY function,loop FORMAT json
Output:
[
{"function":"main","count":1,"time.inclusive.duration":3395643},
{"loop":"lulesh.cycle","function":"main/TimeIncrement","count":100,"time.inclusive.duration":1280},
{"loop":"lulesh.cycle","function":"main/LagrangeLeapFrog","count":100,"time.inclusive.duration":3379753},
{"loop":"lulesh.cycle","function":"main/LagrangeLeapFrog/LagrangeNodal","count":100,"time.inclusive.duration":1508828},
{"loop":"lulesh.cycle","function":"main/LagrangeLeapFrog/LagrangeNodal/CalcForceForNodes","count":100,"time.inclusive.duration":1466051},
{"loop":"lulesh.cycle","function":"main/LagrangeLeapFrog/LagrangeNodal/CalcForceForNodes/CalcVolumeForceForElems","count":100,"time.inclusive.duration":1456613},
Json-split¶
The json-split formatter writes a JSON file with separate fields for Caliper records and metadata. It is generally a more efficient format than the list-of-dicts produced by the json formatter, and explicitly retains hierarchical information. Attributes flagged as NESTED (e.g., default annotations such as annotation, function, and loop) will be merged into the path column to retain hierarchical information between them.
The top-level object contains the following fields:
- data
The Caliper records as a 2D array (array of arrays). Each row is a record, each column is an attribute. The attribute/column labels are stored in the columns array. Missing values in a record are written as null entries.
A column is either a value or reference entry. Value entries are stored directly in each record, whereas reference entries contain the index of the corresponding generalized context tree node in the nodes array. The is_value entry in the column_metadata array determines which columns are value and which are reference entries.
- columns
The labels (i.e., attribute names) of the data columns as array of strings.
- column_metadata
Any metadata for the data columns as array of objects. The objects contain the is_value entry, which determines if the corresponding data column stores a value or a reference entry.
- nodes
An array of metadata node objects. A reference entry in a data record refers to the node at the corresponding index in this array.
The metadata nodes generally form a tree (or forest), retaining hierarchical information. Nodes have a label entry with the nodes’ value, and an optional parent entry, which is the index of the node’s hierarchical parent node in the nodes array. It is guaranteed that a parent node is placed before all of its children in the array.
Example:
SELECT function,loop,count(),sum(time.inclusive.duration) GROUP BY function,loop FORMAT json-split
Output:
{
"data": [
[ 1, 3395643, 0 ],
[ 100, 1280, 2 ],
[ 100, 3379753, 3 ],
[ 100, 1508828, 4 ],
[ 100, 1466051, 5 ],
[ 100, 1456613, 6 ],
[ 100, 358512, 7 ],
[ 100, 1073630, 8 ],
[ 100, 700886, 9 ],
[ 100, 1755059, 10 ],
[ 100, 266492, 11 ],
[ 100, 241132, 12 ],
[ 100, 309658, 13 ],
[ 100, 149492, 14 ],
[ 1100, 98944, 15 ],
[ 100, 1163596, 16 ],
[ 1100, 1113417, 17 ],
[ 3500, 805605, 18 ],
[ 10500, 324182, 19 ],
[ 1100, 27975, 20 ],
[ 100, 8147, 21 ],
[ 100, 110236, 22 ],
[ 1100, 49281, 23 ],
[ 1100, 33125, 24 ]
],
"columns": [ "count", "time.inclusive.duration", "path" ],
"column_metadata": [ { "is_value": true }, { "is_value": true }, { "is_value": false } ],
"nodes": [ { "label": "main" }, { "label": "lulesh.cycle", "parent": 0 }, { "label": "TimeIncrement", "parent": 1 }, { "label": "LagrangeLeapFrog", "parent": 1 }, { "label": "LagrangeNodal", "parent": 3 }, { "label": "CalcForceForNodes", "parent": 4 }, { "label": "CalcVolumeForceForElems", "parent": 5 }, { "label": "IntegrateStressForElems", "parent": 6 }, { "label": "CalcHourglassControlForElems", "parent": 6 }, { "label": "CalcFBHourglassForceForElems", "parent": 8 }, { "label": "LagrangeElements", "parent": 3 }, { "label": "CalcLagrangeElements", "parent": 10 }, { "label": "CalcKinematicsForElems", "parent": 11 }, { "label": "CalcQForElems", "parent": 10 }, { "label": "CalcMonotonicQGradientsForElems", "parent": 13 }, { "label": "CalcMonotonicQRegionForElems", "parent": 13 }, { "label": "ApplyMaterialPropertiesForElems", "parent": 10 }, { "label": "EvalEOSForElems", "parent": 16 }, { "label": "CalcEnergyForElems", "parent": 17 }, { "label": "CalcPressureForElems", "parent": 18 }, { "label": "CalcSoundSpeedForElems", "parent": 17 }, { "label": "UpdateVolumesForElems", "parent": 10 }, { "label": "CalcTimeConstraintsForElems", "parent": 3 }, { "label": "CalcCourantConstraintForElems", "parent": 22 }, { "label": "CalcHydroConstraintForElems", "parent": 22 } ]
}
