str.maketrans on a mutating dict is not thread-safe

[eendebakpt]

Crash report

What happened?

A reproducer (in the format of Lib/tests/test_free_threading/xxx.py):

import unittest
from test.support import threading_helper

threading_helper.requires_working_threading(module=True)


class ItertoolsThreading(unittest.TestCase):

    @threading_helper.reap_threads
    def test_maketrans(self):
        number_of_iterations = 10
        number_of_attempts=100
        for _ in range(number_of_iterations):
            print(f'iteration {_}')
            d = {2000: 'a'}
            def work_iterator(dct):
                for idx in range(number_of_attempts):
                    str.maketrans(dct)
                    idx = 2000 + idx # outside range of small ints
                    dct[idx] = chr(idx%16)
                    dct.pop(idx, None)
            threading_helper.run_concurrently(work_iterator, nthreads=10, args=[d])

if __name__ == "__main__":
    unittest.main()
    print('done')

At first glance it seems that the iteration using PyDict_Next is not safe (but maybe there are more unsafe parts):

cpython/Objects/unicodeobject.c

Line 13158 in 24cc998

while (PyDict_Next(x, &i, &key, &value)) {

(found when reviewing #145129)

CPython versions tested on:

CPython main branch

Operating systems tested on:

Windows

Output from running 'python -VV' on the command line:

No response

Linked PRs

• gh-145142: Make str.maketrans safe under free-threading #145157
• [3.14] gh-145142: Make str.maketrans safe under free-threading (gh-145157) #145320

[colesbury]

Yes, PyDict_Next isn't thread-safe if there are concurrent modifications of the dict. We should wrap the loop with a critical section on x here.

Do you want to make the change?

[eendebakpt]

Do you want to make the change?

I am adding the easy label for newcomers. If not picked up, I will make the change.

[VanshAgarwal24036]

I am working on this issue.