Interface¶
task_pool_abc module¶
Asynchronous Task Pool Base Class.
-
class
aio_parallel_tools.aio_task_pool.core.task_pool_abc.AioTaskPoolABC[source]¶ Bases:
abc.ABCAsynchronous Task Pool Abstract Base Class.
this pool is used when you need to limit the max number of parallel tasks at one time. It’s a derivative of Producer Consumer model. The pool instance will manage a number of consumer as worker. You can scale the worker’s number as you wish with the scale interface. And you, as the Producer, can send your task with the submit interface. If you want to close submit interface, you can use pause interface.
-
abstract async
close(close_worker_timeout: Union[int, float, None] = None, close_pool_timeout: int = 3, safe=True) → None[source]¶ Close all workers and paused the task pool.
- Parameters
close_worker_timeout (Union[int, float, None], optional) – Timeout for closing all workers. Defaults to None.
close_pool_timeout (int, optional) – Timeout for join left tasks. Defaults to 3.
safe (bool, optional) – when getting exceptions, raise it or warning it. Defaults to True.
- Raises
te – close workers timeout.
e – unknown error when closing workers.
te – waiting for left tasks done timeout
e – unknown error when waiting for left tasks done
-
abstract property
closed¶ Check if the pool is closed.
-
abstract property
max_tasks_number¶ Maximum number of the waiting tasks.
- Returns
The maximum number of the waiting tasks.
- Return type
int
-
abstract
pause() → bool[source]¶ Pause the task pool.
- Returns
Check if The task pool is paused
- Return type
bool
-
abstract property
paused¶ Check if user can submit tasks.
If the task pool can accept new tasks,the result is False; else it’s True.
- Returns
can submit or not.
- Return type
bool
-
abstract async
scale(num: int) → int[source]¶ Scale the number of the task pool’s worker.
- Parameters
num (int) – num to scale.positive will increase the worker,negative will decrease the worker.
- Returns
the number will scale to.
- Return type
int
-
abstract
scale_nowait(num: int, soft=True) → int[source]¶ Scale the number of the task pool’s worker without waiting.
- Parameters
num (int) – num to scale.positive will increase the worker,negative will decrease the worker.
soft (bool, optional) – if True, this interface will send Signal to task pool to close workers; else number of random workers will be cancel. Defaults to True.
- Returns
the number will scale to.
- Return type
int
-
abstract property
size¶ Pool’s size.
- Returns
Pool’s size
- Return type
int
-
abstract async
submit(task_func: Callable[[Any], Any], *, args: List[Any] = [], kwargs: Dict[str, Any] = {}, blocking: bool = True) → Union[_asyncio.Future, Any][source]¶ Submit task to the task pool.
- Parameters
task_func (Callable[[Any], Any]) – The task function which will be called by the workers.
args (List[Any], optional) – The positional parameters for the task function. Defaults to [].
kwargs (Dict[str, Any], optional) – The keyword parameters for the task function. Defaults to {}.
blocking (bool, optional) – set if waiting for the task’s result. Defaults to True.
- Raises
NotAvailable – The task pool is paused
- Returns
if blocking is True, submit will return the result of the task; else it will return a future which you can await it to get the result.
- Return type
Union[asyncio.Future, Any]
-
abstract
submit_nowait(task_func: Callable[[Any], Any], *, args: List[Any] = [], kwargs: Dict[str, Any] = {}) → _asyncio.Future[source]¶ Submit task to the task pool with no wait.
- Parameters
task_func (Callable[[Any], Any]) – The task function which will be called by the workers.
args (List[Any], optional) – The positional parameters for the task function. Defaults to [].
kwargs (Dict[str, Any], optional) – The keyword parameters for the task function. Defaults to {}.
- Raises
NotAvailable – The task pool is paused or
e – other exception
NotAvailable – task pool is full, can not put task any more
- Returns
a future which you can await it to get the result.
- Return type
asyncio.Future
-
abstract property
waiting_tasks_number¶ Now number of the waiting tasks.
- Returns
The number of the waiting tasks.
- Return type
int
-
abstract async
