encore.concurrent.futures Package

ThreadPool Executors

enhanced_thread_pool_executor Module

Implements EnhancedThreadPoolExecutor.

This builds off of concurrent.futures.thread and implements the following changes:

• Each worker can be initialized and uninitialised with specified functions.

• ‘map’ works without iterating (bugs.python.org/issue11777).

• Workers do not unnecessarily retain references to work items (bugs.python.org/issue16284).

• Workers do not use polling: http://bugs.python.org/issue11635

• Optional name argument to prefix executor threads’ names.

• Optional wait_on_exit argument to let worker threads die abruptly during jobs on interpreter exit instead of waiting to finish.

The implementation is largely copied to avoid reliance on undocumented, private parts of the code. For example, ‘_threads_queues’ is needed to properly manage threads in the ThreadPoolExecutor, but this is not guaranteed to exist in future implementations of concurrent.futures.thread.

class encore.concurrent.futures.enhanced_thread_pool_executor.EnhancedThreadPoolExecutor(max_workers, initializer=None, uninitializer=None, name=None, wait_at_exit=True)

Bases: Executor

map(fn, *iterables, **kwargs)

Returns a iterator equivalent to map(fn, iter).

Parameters:

• fn – A callable that will take take as many arguments as there are passed iterables.

• timeout – The maximum number of seconds to wait. If None, then there is no limit on the wait time.

Returns:

map(func, *iterables) but the calls may be evaluated out-of-order.

Return type:

An iterator equivalent to

Raises:

• TimeoutError – If the entire result iterator could not be generated before the given timeout.

• Exception – If fn(*args) raises for any values.

shutdown(wait=True)

Clean-up the resources associated with the Executor.

It is safe to call this method several times. Otherwise, no other methods can be called after this one.

Parameters:

• wait – If True then shutdown will not return until all running futures have finished executing and the resources used by the executor have been reclaimed.

• cancel_futures – If True then shutdown will cancel all pending futures. Futures that are completed or running will not be cancelled.

submit(fn, *args, **kwargs)

Submits a callable to be executed with the given arguments.

Schedules the callable to be executed as fn(*args, **kwargs) and returns a Future instance representing the execution of the callable.

Returns:

A Future representing the given call.

synchronous Module

class encore.concurrent.futures.synchronous.SynchronousExecutor

Bases: Executor

Simple Executor subclass that executes everything directly synchronously in the current thread. The submit method of this executor blocks until the call is complete. No cancellation of submitted tasks is possible.

shutdown(wait=True)

Clean-up the resources associated with the Executor.

It is safe to call this method several times. Otherwise, no other methods can be called after this one.

Parameters:

• wait – If True then shutdown will not return until all running futures have finished executing and the resources used by the executor have been reclaimed.

• cancel_futures – If True then shutdown will cancel all pending futures. Futures that are completed or running will not be cancelled.

submit(fn, *args, **kwargs)

Submits a callable to be executed with the given arguments.

Schedules the callable to be executed as fn(*args, **kwargs) and returns a Future instance representing the execution of the callable.

Returns:

A Future representing the given call.

future Module

class encore.concurrent.futures.future.Future

Bases: Future

set_exception(exception)

Sets the result of the future as being the given exception.

Should only be used by Executor implementations and unit tests.

traceback()

Return the formatted traceback of the error that occured in the Executor worker, or None if no error occurred.

abc_work_scheduler Module

class encore.concurrent.futures.abc_work_scheduler.ABCWorkScheduler(executor, name=None, callback=None)

Bases: object

An abstract class to implement various job scheduling and execution models using executors.

Warning

This is an experimental API and is subject to change.

shutdown()

Clean up and wait for pending operations.

The call is synchronous: when the call returns, all pending operations have completed.

submit(operation, *args, **kwargs)

Schedule an operation.

wait()

Wait for all current and pending operations to complete.

asynchronizer Module

class encore.concurrent.futures.asynchronizer.Asynchronizer(executor, name=None, callback=None)

Bases: ABCWorkScheduler

A ‘forgetful’ scheduling of operations.

The Asynchronizer executes at most a single operation at a time. Requests to submit a new operation while an operation is executed are stored for future execution, with each new submission overwriting the prior. When a running operation completes, the most recent submission (if one exists) is then executed. Therefore, operations submitted between the previous and current execution are discarded. The last operation submitted is guaranteed to eventually be executed.

Warning

This is an experimental API and is subject to change.

serializer Module

class encore.concurrent.futures.serializer.Serializer(executor, name=None, callback=None)

Bases: ABCWorkScheduler

Execute all submitted jobs in order.

All submitted operations are stored in a deque and are scheduled in sequence.

Warning

This is an experimental API and is subject to change.

serializing_asynchronizer Module

class encore.concurrent.futures.serializing_asynchronizer.SerializingAsynchronizer(executor, name=None, callback=None)

Bases: ABCWorkScheduler

Provides Asynchronizer functionality for multiple operations.

The SerializingAsynchronizer provides the same guarantees as the Asynchronizer for multiple different operations. For any submitted callable, requests to submit a new operation while an operation of the same callable is underway, the new operation is stored overwriting the prior. Different submitted callables are executed serially in the order in which they were originally submitted.

For example if long-running callable C is submitted, then callable A, then callable B, then callable A again (all while C is running), then the order of execution will be C, A, B.

Warning

This is an experimental API and is subject to change.