ProducerMixin

simple_producer_mixin module

The simplest mixin for creating and submiting tasks to task pool.

class aio_parallel_tools.aio_task_pool.core.mixins.producer_mixin.simple_producer_mixin.SimpleProducerMixin

Bases: object

Simple Producer Mixin.

Requirement:

queue (Property): message queue.

loop (Property): event loop.

size (Property): worker pool’s size.

waiting_tasks_number (Property): Waiting task size in queue.

make_message (Method): make task to message

close_workers (Asynchronous Method): Send worker pool size’s close signal to the queue.

close_workers_nowait_soft (Method): Send worker pool size’s close signal to the queue with no wait.

close_workers_hard (Method): Cancel worker hardlly.

Support:

paused (Property): Check if user can submit tasks.

closed (Property): Check if the pool is closed.

start_accept (Method): Start Accept tasks.

pause (Method): Pause the task pool.

submit (Asynchronous Method): Submit task to the task pool.

submit_nowait (Method): Submit task to the task pool with no wait.

close_pool (Asynchronous Method): Send close signal to all worker.

close_pool_nowait (Method): Send close signal to all workers with no waiting.

async close_pool(close_worker_timeout: Union[int, float, None] = None, close_pool_timeout: int = 3, safe: bool = True) → None

Close all workers and paused the task pool.

Parameters

• close_worker_timeout (Union[int, float, None], optional) – Timeout for closing all workers. Defaults to None.

• close_pool_timeout (int, optional) – Timeout for join left tasks. Defaults to 3.

• safe (bool, optional) – when getting exceptions, raise it or warning it. Defaults to True.

Raises

• te – close workers timeout.

• e – unknown error when closing workers.

• te – waiting for left tasks done timeout

• e – unknown error when waiting for left tasks done

close_pool_nowait(soft: bool = True) → None

Close all workers and paused the task pool without waiting.

Parameters

soft (bool, optional) – if True, this interface will send Signal to task pool to close workers; else all workers will be cancel. Defaults to True.

property closed

Check if the pool is closed.

pause() → bool

Pause the task pool.

Returns

Check if The task pool is paused

Return type

bool

property paused

Check if user can submit tasks.

If the task pool can accept new tasks,the result is False; else it’s True.

Returns

can submit or not.

Return type

bool

start_accept()

Start Accept tasks.

async submit(task_func: Callable[[Any], Any], *, func_args: List[Any] = [], func_kwargs: Dict[str, Any] = {}, blocking: bool = True, **kwargs) → Union[_asyncio.Future, Any]

Submit task to the task pool.

Parameters

• task_func (Callable[[Any], Any]) – The task function which will be called by the workers.

• func_args (List[Any], optional) – The positional parameters for the task function. Defaults to [].

• func_kwargs (Dict[str, Any], optional) – The keyword parameters for the task function. Defaults to {}.

• blocking (bool, optional) – Set if waiting for the task’s result. Defaults to True.

Raises

NotAvailable – The task pool is paused

Returns

if blocking is True, submit will return the result of the task; else it will return a future which you can await it to get the result.

Return type

Union[asyncio.Future, Any]

submit_nowait(task_func: Callable[[Any], Any], *, func_args: List[Any] = [], func_kwargs: Dict[str, Any] = {}, **kwargs) → _asyncio.Future

Submit task to the task pool with no wait.

Parameters

• task_func (Callable[[Any], Any]) – The task function which will be called by the workers.

• func_args (List[Any], optional) – The positional parameters for the task function. Defaults to [].

• func_kwargs (Dict[str, Any], optional) – The keyword parameters for the task function. Defaults to {}.

Raises

• NotAvailable – The task pool is paused or

• e – other exception

• NotAvailable – task pool is full, can not put task any more

Returns

a future which you can await it to get the result.

Return type

asyncio.Future