Python Thread Pool: Efficient Multithreading with ThreadPoolExecutor

Learn how to efficiently manage multiple tasks using a thread pool in Python. The ThreadPoolExecutor class from the concurrent.futures module helps control thread creation and asynchronous task execution, making it ideal for handling multithreading in Python.

What is a Thread Pool?

A thread pool automatically manages a pool of worker threads. Each thread in the pool is called a worker thread and can be reused once its task is completed. A thread pool controls thread creation and task execution, making it efficient for managing multiple tasks.

In Python, the ThreadPoolExecutor class in the concurrent.futures module allows asynchronous execution of functions by multiple threads.

Key Classes in concurrent.futures Module:

• Future Class
• ThreadPoolExecutor Class
• ProcessPoolExecutor Class

The Future Class

The concurrent.futures.Future class handles asynchronous execution of any callable, such as a function. To get a Future object, call the submit() method on an Executor object.

Important Methods in Future Class:

• result(timeout=None): Returns the value from the callable, or raises a TimeoutError if not completed in the specified time.
• cancel(): Attempts to cancel the callable. Returns True if successful, False otherwise.
• cancelled(): Returns True if the callable was successfully cancelled.
• running(): Returns True if the callable is currently running and cannot be cancelled.
• done(): Returns True if the callable was successfully cancelled or finished running.

The ThreadPoolExecutor Class

The ThreadPoolExecutor class represents a pool of worker threads for executing calls asynchronously.

Syntax:

concurrent.futures.ThreadPoolExecutor(max_threads)

Example:

Below is an example of using the ThreadPoolExecutor class:

from concurrent.futures import ThreadPoolExecutor
from time import sleep

def square(numbers):
    for val in numbers:
        ret = val * val
        sleep(1)
        print("Number:{} Square:{}".format(val, ret))

def cube(numbers):
    for val in numbers:
        ret = val * val * val
        sleep(1)
        print("Number:{} Cube:{}".format(val, ret))

if __name__ == '__main__':
    numbers = [1, 2, 3, 4, 5]
    executor = ThreadPoolExecutor(4)
    thread1 = executor.submit(square, (numbers))
    thread2 = executor.submit(cube, (numbers))
    print("Thread 1 executed ? :", thread1.done())
    print("Thread 2 executed ? :", thread2.done())
    sleep(2)
    print("Thread 1 executed ? :", thread1.done())
    print("Thread 2 executed ? :", thread2.done())

Thread 1 executed ? : False
Thread 2 executed ? : False
Number:1 Square:1
Number:1 Cube:1
Number:2 Square:4
Number:2 Cube:8
Thread 1 executed ? : False
Thread 2 executed ? : False
Number:3 Square:9
Number:3 Cube:27
Number:4 Square:16
Number:4 Cube:64
Number:5 Square:25
Number:5 Cube:125
Thread 1 executed ? : True
Thread 2 executed ? : True