Matplotlib - Multiprocessing

Multiprocessing is a technique used to execute multiple processes concurrently, taking advantage of multi-core processors. In Python, the multiprocessing module provides a convenient way to create and manage parallel processes. This is useful for tasks that can be parallelized, such as generating plots, running simulations, or performing computations on large datasets.

Multiprocessing in Matplotlib

Matplotlib is traditionally used in a single-threaded manner, combining it with the multiprocessing library allows for the creation of plots in parallel. This can be useful when dealing with a large number of plots or computationally intensive tasks.

Creating Multiple Matplotlib Plots

Creating multiple plots sequentially can lead to a slower execution, especially when dealing with a large number of plots. In such cases, using the multiprocessing technique can significantly improve performance by allowing the creation of multiple plots concurrently.

Example

Let's see a basic example that demonstrates how to create multiple Matplotlib plots in parallel using multiprocessing.

import matplotlib.pyplot as plt
import numpy as np
import multiprocessing

def plot(datax, datay, name):
   x = datax
   y = datay**2
   plt.scatter(x, y, label=name)
   plt.legend()
   plt.show()

def multiP():
   for i in range(4):
      p = multiprocessing.Process(target=plot, args=(i, i, i))
      p.start()

if __name__ == "__main__": 
   input('Press Enter to start parallel plotting...') 
   multiP()

Output

On executing the above program, 4 matplotlib plots are created in parallel see the video below for reference −

Saving Multiple Matplotlib Figures

Saving multiple Matplotlib figures concurrently is another scenario where multiprocessing can be advantageous.

Example 1

Here is an example that uses multiprocessing to save multiple Matplotlib figures concurrently.

import matplotlib.pyplot as plt
import numpy.random as random
from multiprocessing import Pool

def do_plot(number):
   fig = plt.figure(number)

   a = random.sample(1000)
   b = random.sample(1000)

   # generate random data
   plt.scatter(a, b)

   plt.savefig("%03d.jpg" % (number,))
   plt.close()

   print(f"Image {number} saved successfully...")

if __name__ == '__main__':
   pool = Pool()
   pool.map(do_plot, range(1, 5))

Output

On executing the above code we will get the following output −

Image 1 saved successfully...
Image 2 saved successfully...
Image 3 saved successfully...
Image 4 saved successfully...

If you navigate to the directory where the plots were saved, you will be able to observe the ved 001.jpg, 002.jpg, 003.jpg, and 004.jpg images as shown below −

Example 2

Here is another example that demonstrates how to use multiprocessing to generate data in one process and plot it in another using Matplotlib.

import multiprocessing as mp
import time
import matplotlib.pyplot as plt
import numpy as np

# Fixing random state for reproducibility
np.random.seed(19680801)

class ProcessPlotter:
   def __init__(self):
      self.x = []
      self.y = []

   def terminate(self):
      plt.close('all')

   def call_back(self):
      while self.pipe.poll():
         command = self.pipe.recv()
         if command is None:
            self.terminate()
            return False
         else:
            self.x.append(command[0])
            self.y.append(command[1])
            self.ax.plot(self.x, self.y, 'ro')
      self.fig.canvas.draw()
      return True

   def __call__(self, pipe):
      print('Starting plotter...')
      self.pipe = pipe
      self.fig, self.ax = plt.subplots()
      timer = self.fig.canvas.new_timer(interval=1000)
      timer.add_callback(self.call_back)
      timer.start()
      print('...done')
      plt.show()

class NBPlot:
   def __init__(self):
      self.plot_pipe, plotter_pipe = mp.Pipe()
      self.plotter = ProcessPlotter()
      self.plot_process = mp.Process(
         target=self.plotter, args=(plotter_pipe,), daemon=True)
      self.plot_process.start()

   def plot(self, finished=False):
      send = self.plot_pipe.send
      if finished:
         send(None)
      else:
         data = np.random.random(2)
         send(data)

# Main function for the integrated code
def main_with_multiprocessing():
   pl = NBPlot()
   for _ in range(10):
      pl.plot()
      time.sleep(0.5)
   pl.plot(finished=True)

if __name__ == '__main__':
   if plt.get_backend() == "MacOSX":
      mp.set_start_method("forkserver")
   input('Press Enter to start integrated example...') 
   main_with_multiprocessing()

Output

On executing the above program, will generate the matplotlib plots with random data see the video below for reference −