Class AMSMonitor

Class Documentation

class AMSMonitor
AMSMonitor can be used to decorate class methods and will
record automatically the duration of the tasks in a hashmap
with timestamp. The decorator will also automatically
record the values of all attributes of the class.

    class ExampleTask1(Task):
        def __init__(self):
            self.total_bytes = 0
            self.total_bytes2 = 0

        # @AMSMonitor() would record all attributes
        # (total_bytes and total_bytes2) and the duration
        # of the block under the name amsmonitor_duration_ns.
        # Each time the same block (function in class or
        # predefined tag) is being monitored, AMSMonitor
        # create a new record with a timestamp (see below).
        #
        # @AMSMonitor(accumulate=True) records also all
        # attributes but does not create a new record each
        # time that block is being monitored, the first
        # timestamp is always being used and only
        # amsmonitor_duration_ns is being accumulated.
        # The user-managed attributes (like total_bytes
        # and total_bytes2 ) are not being accumulated.
        # By default, accumulate=False.

        # Example: we do not want to record total_bytes
        # but just total_bytes2
        @AMSMonitor(record=["total_bytes2"])
        def __call__(self):
            i = 0
            # Here we have to manually provide the current object being monitored
            with AMSMonitor(obj=self, tag="while_loop"):
                while (i<=3):
                    self.total_bytes += 10
                    self.total_bytes2 = 1
                    i += 1

        # Example: We can also collect data at a finer grain
        @AMSMonitor(array=["myarray"])
        def f(self):
            i = 0
            while (i<2): myarray.append({"i":i})

Each time `ExampleTask1()` is being called, AMSMonitor will
populate `_stats` as follows (showed with two calls here):
    {
        "ExampleTask1": {
            "while_loop": {
                "02/29/2024-19:27:53": {
                    "total_bytes2": 30,
                    "amsmonitor_duration_ns": 4.004607439041138
                }
            },
            "__call__": {
                "02/29/2024-19:29:24": {
                    "total_bytes2": 30,
                    "amsmonitor_duration_ns": 4.10461138
                }
            },
            "f": {
                "myarray": [
                    {
                        "i": 0,
                    },
                    {
                        "i": 1,
                    }
            }
        }
    }

Attributes:
    record: attributes to record, if None, all attributes
        will be recorded, except objects (e.g., multiprocessing.Queue)
        which can cause problem. if empty ([]), no attributes will
        be recorded, only amsmonitor_duration_ns will be recorded.
    array: User can give a variable in which data can be accumulated over
        function calls. For example, `@AMSMonitor(array=["msg"])`
        give the possibilty to use the list `msg` within the decorated
        function to accumalate data.
    accumulate: If True, AMSMonitor will accumulate recorded
        data instead of recording a new timestamp for
        any subsequent call of AMSMonitor on the same method.
        We accumulate only records managed by AMSMonitor, like
        amsmonitor_duration_ns. We do not accumulate records
        from the monitored class/function.
    obj: Mandatory if using `with` statement, `object` is
        the main object should be provided (i.e., self).
    tag: Mandatory if using `with` statement, `tag` is the
        name that will appear in the record for that
        context manager statement.

Public Functions

__init__(self, record=None, array=[], accumulate=False, obj=None, tag=None, logging.Logger logger=None, **kwargs)
__str__(self)
__repr__(self)
__enter__(self)
__exit__(self, exc_type, exc_val, exc_tb)
info(cls)
stats(cls)
format_ts(cls)
convert_ts(cls, str ts)
json(cls, str json_output)
Write the collected metrics to a JSON file.
reset(cls)
start_monitor(self, *args, **kwargs)
stop_monitor(self)
__call__(self, Callable func)
The main decorator.

Public Members

accumulate
kwargs
record
object
start_time
internal_ts
tag
logger
array_names
variables_list
array_context
use_arrays
format_ts

Public Static Functions

lock()
unlock()

Protected Functions

_update_db(self, dict new_data, str class_name, str func_name, str ts)
This function update the hashmap containing all the records.
_remove_reserved_keys(self, Union[dict, List] d)
Remove all the reserved keys from the dict given as input.
_acc(self, dict original, dict new_data)
Sum up element-wise two hashmaps (ignore fields that are not common)
_filter_out_object(self, dict data)
Filter out a hashmap to remove objects which can cause errors
_filter(self, dict data, List[str] keys)
Filter out a hashmap to contains only keys listed by list of keys
_get_ts(self, str class_name, str func)
Return initial timestamp for a given monitored function.

Protected Static Attributes

_manager = multiprocessing.Manager()
_stats = _manager.dict()
_ts_format = "%m/%d/%Y-%H:%M:%S"
_reserved_keys = ["amsmonitor_duration_ns"]
_lock = threading.Lock()
_count = 0