Class AMSFluxExecutorFuture

Inheritance Relationships

Base Type

  • public Future

Class Documentation

ams_flux.AMSFluxExecutorFuture : public Future
A ``concurrent.futures.Future`` subclass that represents a AMSTrain job.

The class provides a future abstraction for training jobs. In 'AMS' training jobs
are actually a sequence of 2 jobs. First we do the sub-selection and right after
we schedule a ml model training job. The future provides accessor to the description
of the current AMS training job and the phase in which the job is part off.

In addition to all of the ``flux.job.FluxExecutorFuture`` functionality,
``AMSFluxExecutorFuture`` instances offer:

* The ``uri`` and ``add_uri_callback`` methods for retrieving the
  Flux uri of the instance executing this job. This is convenient for
  nested jobs

Valid events are contained in the ``EVENTS`` class attribute.

Public Functions

__init__(self, int owning_thread_id, flux_executor, bool track_uri, domain_descr, *args, **kwargs)
get_domain_descr(self)
add_done_callback(self, *args, **kwargs)
Attaches a callable that will be called when the future finishes.

:param fn: A callable that will be called with this future as its only
    argument when the future completes or is cancelled. The callable
    will always be called by a thread in the same process in which
    it was added. If the future has already completed or been
    cancelled then the callable will be called immediately. These
    callables are called in the order that they were added.
:return: ``self``
uri(self, timeout=None)
Return the uri of the Flux Instance that the future represents.

:param timeout: The number of seconds to wait for the jobid.
    If None, then there is no limit on the wait time.

:return: a flux uri.

:raises concurrent.futures.TimeoutError: If the jobid is not available
    before the given timeout.
:raises concurrent.futures.CancelledError: If the future was cancelled.
:raises RuntimeError: If the job could not be submitted (e.g. if
    the jobspec was invalid).
jobid(self, timeout=None)
Return the jobid of the Flux job that the future represents.

:param timeout: The number of seconds to wait for the jobid.
    If None, then there is no limit on the wait time.

:return: a positive integer jobid.

:raises concurrent.futures.TimeoutError: If the jobid is not available
    before the given timeout.
:raises concurrent.futures.CancelledError: If the future was cancelled.
:raises RuntimeError: If the job could not be submitted (e.g. if
    the jobspec was invalid).
flux_executor(self)
add_uri_callback(self, callback)
Attaches a callable that will be called when the uri is ready.

Added callables are called in the order that they were added and may be called
in another thread.  If the callable raises an ``Exception`` subclass, it will
be logged and ignored.  If the callable raises a ``BaseException`` subclass,
the behavior is undefined.

:param callback: a callable taking the future as its only argument.
:return: ``self``
add_jobid_callback(self, callback)
Attaches a callable that will be called when the jobid is ready.

Added callables are called in the order that they were added and may be called
in another thread.  If the callable raises an ``Exception`` subclass, it will
be logged and ignored.  If the callable raises a ``BaseException`` subclass,
the behavior is undefined.

:param callback: a callable taking the future as its only argument.
:return: ``self``
exception(self, *args, **kwargs)
If this method is invoked from a jobid/event callback by an executor thread,
it will result in deadlock, since the current thread will wait
for work that the same thread is meant to do.

Head off this possibility by checking the current thread.
result(self, *args, **kwargs)
If this method is invoked from a jobid/event callback by an executor thread,
it will result in deadlock, since the current thread will wait
for work that the same thread is meant to do.

Head off this possibility by checking the current thread.
set_exception(self, exception)
When setting an exception on the future, set the jobid if it hasn't
been set already. The jobid will already have been set unless the exception
was generated before the job could be successfully submitted.
cancel(self, *args, **kwargs)
If a thread is waiting for the future's jobid, and another
thread cancels the future, the waiting thread would never wake up
because the jobid would never be set.

When cancelling, set the jobid to something invalid.
add_event_callback(self, event, callback)
Add a callback to be invoked when an event occurs.

The callback will be invoked, with the future as the first argument and the
``flux.job.EventLogEvent`` as the second, whenever the event occurs. If the
event occurs multiple times, the callback will be invoked with each different
`EventLogEvent` instance. If the event never occurs, the callback
will never be invoked.

Added callables are called in the order that they were added and may be called
in another thread.  If the callable raises an ``Exception`` subclass, it will
be logged and ignored.  If the callable raises a ``BaseException`` subclass,
the behavior is undefined.

If the event has already occurred, the callback will be called immediately.

:param event: the name of the event to add the callback to.
:param callback: a callable taking the future and the event as arguments.
:return: ``self``
__repr__(self)

Public Static Attributes

EVENTS = frozenset(("memo", *list(MAIN_EVENTS)))

Protected Functions

_set_uri(self, uri, exc=None)
Sets the Flux uri associated with the future.

If `exc` is not None, raise `exc` instead of returning the jobid
in calls to `Future.jobid()`. Useful if the job ID cannot be
retrieved.

Should only be used by Executor implementations and unit tests.
_set_jobid(self, jobid, exc=None)
Sets the Flux jobid associated with the future.

If `exc` is not None, raise `exc` instead of returning the jobid
in calls to `Future.jobid()`. Useful if the job ID cannot be
retrieved.

Should only be used by Executor implementations and unit tests.
_get_uri(self)
Get the jobid, checking for cancellation and invalid job ids.
_get_jobid(self)
Get the jobid, checking for cancellation and invalid job ids.
_invoke_flux_callback(self, callback, *args)
_set_event(self, log_entry)
Set an event on the future.

For use by Executor implementations and unit tests.

:param log_entry: an ``EventLogEvent``.