Python Thread Priority: Managing Queues with PriorityQueue and Queue Module

Discover how to manage thread priority in Python using the PriorityQueue class from the queue module. Learn how to safely exchange information between threads and control task priority with methods like get(), put(), qsize(), empty(), and full().



Python - Thread Priority

The queue module in Python's standard library is useful in threaded programming when information must be exchanged safely between multiple threads. The PriorityQueue class in this module implements all the required locking semantics.

With a priority queue, the entries are kept sorted (using the heapq module) and the lowest valued entry is retrieved first.

The Queue objects have the following methods to control the queue:

  • get() - Removes and returns an item from the queue.
  • put() - Adds an item to the queue.
  • qsize() - Returns the number of items that are currently in the queue.
  • empty() - Returns True if the queue is empty; otherwise, False.
  • full() - Returns True if the queue is full; otherwise, False.

Priority Queue Constructor

queue.PriorityQueue(maxsize=0)

This is the constructor for a priority queue. maxsize is an integer that sets the upper limit on the number of items that can be placed in the queue. If maxsize is less than or equal to zero, the queue size is infinite.

The lowest valued entries are retrieved first (the lowest valued entry is the one that would be returned by min(entries)). A typical pattern for entries is a tuple in the form:

(priority_number, data)

Example

This example demonstrates the usage of a priority queue:

Code

  from time import sleep
  from random import random, randint
  from threading import Thread
  from queue import PriorityQueue
  
  queue = PriorityQueue()
  
  def producer(queue):
      print('Producer: Running')
      for i in range(5):
          value = random()
          priority = randint(0, 5)
          item = (priority, value)
          queue.put(item)
      queue.join()
      queue.put(None)
      print('Producer: Done')
  
  def consumer(queue):
      print('Consumer: Running')
      while True:
          item = queue.get()
          if item is None:
              break
          sleep(item[1])
          print(item)
          queue.task_done()
      print('Consumer: Done')
  
  producer = Thread(target=producer, args=(queue,))
  producer.start()
  
  consumer = Thread(target=consumer, args=(queue,))
  consumer.start()
  
  producer.join()
  consumer.join()
              
Output

  Producer: Running
  Consumer: Running
  (0, 0.15332707626852804)
  (2, 0.4730737391435892)
  (2, 0.8679231358257962)
  (3, 0.051924220435665025)
  (4, 0.23945882716108446)
  Producer: Done
  Consumer: Done