forked process in multiprocessing does not honour atexit

@pitrou

BPO	39675
Nosy	@pitrou, @applio, @pablogsal, @gaborbernat

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/pablogsal'
closed_at = None
created_at = <Date 2020-02-18.11:35:55.954>
labels = ['type-bug', 'library', '3.9']
title = 'forked process in multiprocessing does not honour atexit'
updated_at = <Date 2020-02-18.12:27:31.963>
user = 'https://github.com/gaborbernat'

bugs.python.org fields:

activity = <Date 2020-02-18.12:27:31.963>
actor = 'pablogsal'
assignee = 'pablogsal'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2020-02-18.11:35:55.954>
creator = 'gaborjbernat'
dependencies = []
files = []
hgrepos = []
issue_num = 39675
keywords = []
message_count = 1.0
messages = ['362197']
nosy_count = 4.0
nosy_names = ['pitrou', 'davin', 'pablogsal', 'gaborjbernat']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue39675'
versions = ['Python 3.9']

Linked PRs

gh-83856: Honor atexit for all multiprocessing start methods #114279

I've talked with Pablo about this in person, and as advised opening the issue here now.

I've discovered that forked processes do not honour atexit registrations. See the following example code:

from multiprocessing import Process, set_start_method
import time
import os
import atexit


def cleanup():
    print(f"cleanup {os.getpid()}")


atexit.register(cleanup)


def run():
    time.sleep(0.1)
    print(f"process done {os.getpid()}")
    # atexit._run_exitfuncs()


if __name__ == "__main__":
    set_start_method("fork")
    process = Process(target=run)
    process.start()
    process.join()
    print("app finished")

In case of a forked process childs the atexit is never executed (note it works if I ran them manually at the end of the child process; so they're registered correctly). Switching to spawn method makes it work as expected. The behaviour is the same even if you call register within the child process (as opposed to being inherited during forking). Also found this StackOverflow question that mentions this https://stackoverflow.com/a/26476585. At the very least the documentation should explain this; though I'd expect atexit to be called before finalization of the fork processes (assuming the child process exits with 0 exit code). d

@gpshead

CC: @gpshead @pitrou @ambv

See the PR and this comment for more context

gh-83856: Honor atexit for all multiprocessing start methods (GH-114279)

The discussion on the PR was useful. This is merged in time for 3.13beta1. It is a bit of a behavior change and a corner case that has existed for a long time and most code does not hit so I do not believe we should be backporting it as a bugfix despite it also being reasonably to call a bug. Thanks for the discussion and PR!

pythongh-83856: Honor atexit for all multiprocessing start methods (p…

Navigation Menu

Repository navigation

forked process in multiprocessing does not honour atexit #83856

Linked PRs

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

forked process in multiprocessing does not honour atexit #83856

Description

Linked PRs

Activity

gaborbernat commented on Feb 18, 2020

pablogsal commented on Jan 19, 2024

gpshead commented on May 3, 2024

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions