Python Multithreading: Enhancing Performance with Concurrent Execution

Discover how Python's multithreading enables concurrent execution by splitting tasks into sub-tasks. Learn about the benefits of threads, such as efficient memory usage and easy data sharing, and understand how threads can be interrupted or paused to optimize program performance.

Python - Multithreading

By default, a program executes sequentially. Multithreading splits the main task into sub-tasks and executes them simultaneously, speeding up the process.

Threads are light-weight sub-processes within a program. They share memory space and can communicate easily. They have an instruction pointer to track their execution.

  • Threads can be interrupted (pre-empted).
  • Threads can be paused (sleeping) to let others run (yielding).

Multithreading within a process allows easy data sharing. It requires less memory and is cheaper than processes.

Starting a New Thread

To create a new thread, use the following method from the thread module:

Syntax
thread.start_new_thread(function, args[, kwargs])

This method starts a new thread that calls function with args. The thread ends when function returns.

Example

import _thread
import time

# Define a function for the thread
def print_time(threadName, delay):
    count = 0
    while count < 5:
        time.sleep(delay)
        count += 1
        print(f"{threadName}: {time.ctime(time.time())}")

# Create two threads
try:
    _thread.start_new_thread(print_time, ("Thread-1", 2))
    _thread.start_new_thread(print_time, ("Thread-2", 4))
except:
    print("Error: unable to start thread")

while 1:
    pass
Output

Thread-1: Thu Mar 21 09:10:03 2013
Thread-1: Thu Mar 21 09:10:05 2013
Thread-2: Thu Mar 21 09:10:05 2013
Thread-1: Thu Mar 21 09:10:07 2013
Thread-2: Thu Mar 21 09:10:09 2013

The Threading Module

The threading module provides higher-level thread management:

  • threading.activeCount() - Returns the number of active threads.
  • threading.currentThread() - Returns the current thread object.
  • threading.enumerate() - Returns a list of all active threads.

The Thread class in threading module:

  • run() - Entry point for a thread.
  • start() - Starts a thread.
  • join([time]) - Waits for thread to terminate.
  • isAlive() - Checks if thread is still executing.
  • getName() - Gets the thread name.
  • setName() - Sets the thread name.

Creating Thread Using Threading Module

To create a thread using threading module:

  1. Define a subclass of the Thread class.
  2. Override __init__(self, [args]) to add arguments.
  3. Override run(self, [args]) to define the thread's behavior.
Example

import threading
import time

exitFlag = 0

class myThread(threading.Thread):
    def __init__(self, threadID, name, counter):
        threading.Thread.__init__(self)
        self.threadID = threadID
        self.name = name
        self.counter = counter
    def run(self):
        print(f"Starting {self.name}")
        print_time(self.name, self.counter, 5)
        print(f"Exiting {self.name}")

def print_time(threadName, delay, counter):
    while counter:
        if exitFlag:
            threadName.exit()
        time.sleep(delay)
        print(f"{threadName}: {time.ctime(time.time())}")
        counter -= 1

# Create new threads
thread1 = myThread(1, "Thread-1", 1)
thread2 = myThread(2, "Thread-2", 2)

# Start new Threads
thread1.start()
thread2.start()

print("Exiting Main Thread")
Output

Starting Thread-1
Starting Thread-2
Exiting Main Thread
Thread-1: Thu Mar 21 09:10:03 2013
Thread-1: Thu Mar 21 09:10:05 2013
Thread-2: Thu Mar 21 09:10:05 2013
Thread-1: Thu Mar 21 09:10:07 2013
Thread-2: Thu Mar 21 09:10:09 2013

Synchronizing Threads

The threading module has a simple locking mechanism for synchronizing threads:

  • acquire(blocking) - Forces threads to run synchronously.
  • release() - Releases the lock.
Example

import threading
import time

class myThread(threading.Thread):
    def __init__(self, threadID, name, counter):
        threading.Thread.__init__(self)
        self.threadID = threadID
        self.name = name
        self.counter = counter
    def run(self):
        print(f"Starting {self.name}")
        threadLock.acquire()
        print_time(self.name, self.counter, 3)
        threadLock.release()

def print_time(threadName, delay, counter):
    while counter:
        time.sleep(delay)
        print(f"{threadName}: {time.ctime(time.time())}")
        counter -= 1

threadLock = threading.Lock()
threads = []

# Create new threads
thread1 = myThread(1, "Thread-1", 1)
thread2 = myThread(2, "Thread-2", 2)

# Start new Threads
thread1.start()
thread2.start()

threads.append(thread1)
threads.append(thread2)

# Wait for all threads to complete
for t in threads:
    t.join()

print("Exiting Main Thread")
Output

Starting Thread-1
Starting Thread-2
Thread-1: Thu Mar 21 09:11:28 2013
Thread-1: Thu Mar 21 09:11:30 2013
Thread-2: Thu Mar 21 09:11:32 2013
Thread-2: Thu Mar 21 09:11:34 2013
Exiting Main Thread

Multithreaded Priority Queue

The Queue module lets you create a queue to manage tasks. It provides methods like get(), put(), qsize(), empty(), and full().

Example

import queue
import threading
import time

exitFlag = 0

class myThread(threading.Thread):
    def __init__(self, threadID, name, q):
        threading.Thread.__init__(self)
        self.threadID = threadID
        self.name = name
        self.q = q
    def run(self):
        print(f"Starting {self.name}")
        process_data(self.name, self.q)
        print(f"Exiting {self.name}")

def process_data(threadName, q):
    while not exitFlag:
        queueLock.acquire()
        if not workQueue.empty():
            data = q.get()
            queueLock.release()
            print(f"{threadName} processing {data}")
        else:
            queueLock.release()
        time.sleep(1)

threadList = ["Thread-1", "Thread-2", "Thread-3"]
nameList = ["One", "Two", "Three", "Four", "Five"]
queueLock = threading.Lock()
workQueue = queue.Queue(10)
threads = []
threadID = 1

# Create new threads
for tName in threadList:
    thread = myThread(threadID, tName, workQueue)
    thread.start()
    threads.append(thread)
    threadID += 1

# Fill the queue
queueLock.acquire()
for word in nameList:
    workQueue.put(word)
queueLock.release()

# Wait for queue to empty
while not workQueue.empty():
    pass

# Notify threads it's time to exit
exitFlag = 1

# Wait for all threads to complete
for t in threads:
    t.join()

print("Exiting Main Thread")
Output

Starting Thread-1
Starting Thread-2
Starting Thread-3
Thread-1 processing One
Thread-2 processing Two
Thread-3 processing Three
Thread-1 processing Four
Thread-2 processing Five
Exiting Thread-1
Exiting Thread-2
Exiting Thread-3
Exiting Main Thread