Skip to content

Deleting unicodedata from sys.modules can cause pointer use-after-free #149449

Closed
Closed
Deleting unicodedata from sys.modules can cause pointer use-after-free#149449
Labels
interpreter-core(Objects, Python, Grammar, and Parser dirs)topic-unicodetype-crashA hard crash of the interpreter, possibly with a core dump

Description

@stestagg
stestagg
opened on May 6, 2026

Crash report

What happened?

import gc, sys

print('😀'.encode('ascii', errors='namereplace'))
del sys.modules["unicodedata"]
gc.collect()
print('😀'.encode('ascii', errors='namereplace'))
b'\\N{GRINNING FACE}'
Job 1, 'python test3.py' terminated by signal SIGBUS (Misaligned address error)

This one was a beast to track down! It can be trigered by the parser too:

import gc, sys

compile(r"x = '\N{LATIN CAPITAL LETTER A}'", "<x>", "exec")
del sys.modules["unicodedata"]
gc.collect()
compile(r"x = '\N{LATIN CAPITAL LETTER B}'", "<x>", "exec")

I don't know if this is related to the sub interpreters lifetime stuff or not, but the culprit seems to be in here:

_PyUnicode_Name_CAPI *
_PyUnicode_GetNameCAPI(void)
{
    PyInterpreterState *interp = _PyInterpreterState_GET();
    _PyUnicode_Name_CAPI *ucnhash_capi;

    ucnhash_capi = _Py_atomic_load_ptr(&interp->unicode.ucnhash_capi);
    if (ucnhash_capi == NULL) {
        ucnhash_capi = (_PyUnicode_Name_CAPI *)PyCapsule_Import(
                PyUnicodeData_CAPSULE_NAME, 1);

        // It's fine if we overwrite the value here. It's always the same value.
        _Py_atomic_store_ptr(&interp->unicode.ucnhash_capi, ucnhash_capi);
    }
    return ucnhash_capi;
}

The ucnhash_capi is stored here, but seemingly without a refcount addition? so if you can get the unicodedata module to be unloaded/cleaned up, then ucnhash_capi becomes a stale pointer.

Stack trace 
(lldb) Process 396 stopped
* thread #1, name = 'python3', stop reason = signal SIGBUS: illegal alignment
    frame #0: 0xffffdddddddddddd
error: memory read failed for 0xffffdddddddddc00
bt
* thread #1, name = 'python3', stop reason = signal SIGBUS: illegal alignment
  * frame #0: 0xffffdddddddddddd
    frame #1: 0x0000aaaaabbcb5ec python3`_PyUnicode_DecodeUnicodeEscapeInternal2(s=<unavailable>, size=<unavailable>, errors=<unavailable>, consumed=<unavailable>, first_invalid_escape_char=<unavailable>, first_invalid_escape_ptr=<unavailable>) at unicodeobject.c:6654:25
    frame #2: 0x0000aaaaab492338 python3`decode_unicode_with_escapes(parser=<unavailable>, s="\\N{LATIN CAPITAL LETTER B}\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd\xcd", len=<unavailable>, t=<unavailable>) at string_parser.c:201:9 [inlined]
    frame #3: 0x0000aaaaab492290 python3`_PyPegen_decode_string(p=<unavailable>, raw=0, s=<unavailable>, len=<unavailable>, t=<unavailable>) at string_parser.c:247:12
    frame #4: 0x0000aaaaab1af660 python3`_PyPegen_parse_string(p=0x0000fbfff5dda9b0, t=0x0000fbfff5c4d5d0) at string_parser.c:338:12 [inlined]
    frame #5: 0x0000aaaaab1af4a4 python3`_PyPegen_constant_from_string(p=0x0000fbfff5dda9b0, tok=0x0000fbfff5c4d5d0) at action_helpers.c:1483:19
    frame #6: 0x0000aaaaab2c5a24 python3`string_rule(p=0x0000fbfff5dda9b0) at parser.c:17430:20
    frame #7: 0x0000aaaaab2c58d8 python3`_tmp_156_rule(p=0x0000fbfff5dda9b0) at parser.c:38042:27
    frame #8: 0x0000aaaaab2bca8c python3`_loop1_82_rule(p=0x0000fbfff5dda9b0) at parser.c:33602:29
    frame #9: 0x0000aaaaab253cd4 python3`strings_rule(p=0x0000fbfff5dda9b0) at parser.c:17501:34
    frame #10: 0x0000aaaaab24d888 python3`atom_rule(p=0x0000fbfff5dda9b0) at parser.c:15442:28
    frame #11: 0x0000aaaaab2a6e30 python3`t_primary_raw(p=0x0000fbfff5dda9b0) at parser.c:20251:18
    frame #12: 0x0000aaaaab2a50b4 python3`t_primary_rule(p=0x0000fbfff5dda9b0) at parser.c:20041:22
    frame #13: 0x0000aaaaab2a25ec python3`target_with_star_atom_rule(p=0x0000fbfff5dda9b0) at parser.c:19541:18
    frame #14: 0x0000aaaaab29e770 python3`star_target_rule(p=0x0000fbfff5dda9b0) at parser.c:19484:42
    frame #15: 0x0000aaaaab295398 python3`star_targets_rule(p=0x0000fbfff5dda9b0) at parser.c:19226:18
    frame #16: 0x0000aaaaab3a7518 python3`_tmp_159_rule(p=0x0000fbfff5dda9b0) at parser.c:38309:18
    frame #17: 0x0000aaaaab341008 python3`_loop1_12_rule(p=0x0000fbfff5dda9b0) at parser.c:29160:29
    frame #18: 0x0000aaaaab1eeb1c python3`assignment_rule(p=0x0000fbfff5dda9b0) at parser.c:2231:34
    frame #19: 0x0000aaaaab1ee740 python3`simple_stmt_rule(p=<unavailable>) at parser.c:1614:31
    frame #20: 0x0000aaaaab1cbdfc python3`simple_stmts_rule(p=0x0000fbfff5dda9b0) at parser.c:1509:18
    frame #21: 0x0000aaaaab1c02e0 python3`statement_rule(p=0x0000fbfff5dda9b0) at parser.c:1291:34
    frame #22: 0x0000aaaaab1bfbdc python3`_loop1_2_rule(p=0x0000fbfff5dda9b0) at parser.c:28534:30
    frame #23: 0x0000aaaaab1bfab8 python3`statements_rule(p=0x0000fbfff5dda9b0) at parser.c:1224:18
    frame #24: 0x0000aaaaab1b37a0 python3`file_rule(p=0x0000fbfff5dda9b0) at parser.c:1026:18
    frame #25: 0x0000aaaaab1b35ac python3`_PyPegen_parse(p=0x0000fbfff5dda9b0) at parser.c:39422:18
    frame #26: 0x0000aaaaab18f43c python3`_PyPegen_run_parser(p=0x0000fbfff5dda9b0) at pegen.c:940:17
    frame #27: 0x0000aaaaab1939a4 python3`_PyPegen_run_parser_from_string(str=<unavailable>, start_rule=<unavailable>, filename_ob=<unavailable>, flags=0x0000fbfff6980d40, arena=<unavailable>, module=<unavailable>) at pegen.c:1074:14
    frame #28: 0x0000aaaaac2b7154 python3`_PyParser_ASTFromString(str="x = '\\N{LATIN CAPITAL LETTER B}'", filename=0x0000fbfff5ce2ed0, mode=257, flags=0x0000fbfff6980d40, arena=0x0000fbfff5df3880, module=0x0000000000000000) at peg_api.c:14:21 [inlined]
    frame #29: 0x0000aaaaac2b7114 python3`_Py_CompileStringObjectWithModule(str="x = '\\N{LATIN CAPITAL LETTER B}'", filename=0x0000fbfff5ce2ed0, start=257, flags=0x0000fbfff6980d40, optimize=-1, module=0x0000000000000000) at pythonrun.c:1538:11
    frame #30: 0x0000aaaaabdd5b24 python3`builtin_compile_impl(module=0x0000fbfff5da4200, source=0x0000fbfff5d97220, filename=<unavailable>, mode="exec", flags=<unavailable>, dont_inherit=<unavailable>, optimize=<unavailable>, modname=0x0000000000000000, feature_version=<unavailable>) at bltinmodule.c:953:14 [inlined]
    frame #31: 0x0000aaaaabdd5a7c python3`builtin_compile(module=<unavailable>, args=<unavailable>, nargs=<unavailable>, kwnames=0x0000000000000000) at bltinmodule.c.h:470:20
    frame #32: 0x0000aaaaab86f210 python3`cfunction_vectorcall_FASTCALL_KEYWORDS(func=0x0000fbfff5d9a4b0, args=0x0000fbfff6980c28, nargsf=9223372036854775811, kwnames=<unavailable>) at methodobject.c:465:24
    frame #33: 0x0000aaaaab597f0c python3`_PyObject_VectorcallTstate(tstate=0x0000aaaaacacd6f8, callable=0x0000fbfff5d9a4b0, args=0x0000fbfff6980c28, nargsf=9223372036854775811, kwnames=0x0000000000000000) at pycore_call.h:144:11
    frame #34: 0x0000aaaaabecb5e4 python3`PyObject_Vectorcall(callable=0x0000fbfff5d9a4b0, args=<unavailable>, nargsf=<unavailable>, kwnames=0x0000000000000000) at call.c:327:12 [inlined]
    frame #35: 0x0000aaaaabecb5e0 python3`_Py_VectorCallInstrumentation_StackRefSteal(callable=<unavailable>, arguments=<unavailable>, total_args=3, kwnames=<unavailable>, call_instrumentation=<unavailable>, frame=<unavailable>, this_instr=<unavailable>, tstate=<unavailable>) at ceval.c:775:11
    frame #36: 0x0000aaaaabe442f0 python3`_PyEval_EvalFrameDefault(tstate=0x0000aaaaacacd6f8, frame=0x0000fbfff664d020, throwflag=<unavailable>) at generated_cases.c.h:1846:35
    frame #37: 0x0000aaaaabef8218 python3`_PyEval_EvalFrame(tstate=0x0000aaaaacacd6f8, frame=0x0000fbfff664d020, throwflag=0) at pycore_ceval.h:122:16 [inlined]
    frame #38: 0x0000aaaaabef8198 python3`_PyEval_Vector(tstate=<unavailable>, func=<unavailable>, locals=<unavailable>, args=<unavailable>, argcount=<unavailable>, kwnames=<unavailable>) at ceval.c:2156:12
    frame #39: 0x0000aaaaabef7388 python3`PyEval_EvalCode(co=<unavailable>, globals=<unavailable>, locals=0x0000fbfff5c63b30) at ceval.c:686:21
    frame #40: 0x0000aaaaac2a3b80 python3`run_eval_code_obj(tstate=0x0000aaaaacacd6f8, co=0x0000fbfff5d81180, globals=0x0000fbfff5c63b30, locals=0x0000fbfff5c63b30) at pythonrun.c:1369:12
    frame #41: 0x0000aaaaac2a5e54 python3`run_mod(mod=<unavailable>, filename=<unavailable>, globals=<unavailable>, locals=<unavailable>, flags=<unavailable>, arena=<unavailable>, interactive_src=<unavailable>, generate_new_source=<unavailable>) at pythonrun.c:1472:19
    frame #42: 0x0000aaaaac2a3e0c python3`pyrun_file(fp=0x0000fd9ff7b72680, filename=0x0000fbfff5cec6a0, start=257, globals=0x0000fbfff5c63b30, locals=0x0000fbfff5c63b30, closeit=1, flags=0x0000fbfff6947fe0) at pythonrun.c:1296:15
    frame #43: 0x0000aaaaac2a1678 python3`_PyRun_SimpleFileObject(fp=<unavailable>, filename=<unavailable>, closeit=<unavailable>, flags=<unavailable>) at pythonrun.c:518:13
    frame #44: 0x0000aaaaac29d6a0 python3`_PyRun_AnyFileObject(fp=0x0000fd9ff7b72680, filename=0x0000fbfff5cec6a0, closeit=1, flags=0x0000fbfff6947fe0) at pythonrun.c:81:15
    frame #45: 0x0000aaaaac3c0570 python3`pymain_run_file_obj(program_name=0x0000fbfff5cec7c0, filename=0x0000fbfff5cec6a0, skip_source_first_line=<unavailable>) at main.c:411:15 [inlined]
    frame #46: 0x0000aaaaac3c0398 python3`pymain_run_file(config=0x0000aaaaaca986b0) at main.c:430:15
    frame #47: 0x0000aaaaac3bb944 python3`pymain_run_python(exitcode=0x0000fbfff6a2f300) at main.c:715:21 [inlined]
    frame #48: 0x0000aaaaac3ba1e0 python3`Py_RunMain at main.c:796:5
    frame #49: 0x0000aaaaac3c5d64 python3`pymain_main(args=<unavailable>) at main.c:826:12
    frame #50: 0x0000aaaaac3c63c8 python3`Py_BytesMain(argc=2, argv=0x0000fffffffff998) at main.c:850:12
    frame #51: 0x0000fffff7f5cdf8 ld-musl-aarch64.so.1`libc_start_main_stage2(main=(python3`main at python.c:14), argc=2, argv=0x0000fffffffff998) at __libc_start_main.c:95:2
(lldb)

Reproduces on main build:
Python 3.15.0a8+ (heads/main-dirty:17975f9, May 6 2026, 07:29:10) [Clang 21.1.2 ]
and macos:
Python 3.14.3 (main, Feb 19 2026, 15:40:46) [Clang 17.0.0 (clang-1700.6.3.2)]

Doesn't seem to reproduce on 3.9.6
Python 3.9.6 (default, Dec 2 2025, 07:27:58)
[Clang 17.0.0 (clang-1700.6.3.2)]

I'll run a bisect

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.14.3 (main, Feb 19 2026, 15:40:46) [Clang 17.0.0 (clang-1700.6.3.2)]

Linked PRs

Activity

  1. stestagg
    added
    type-crashA hard crash of the interpreter, possibly with a core dump
    on May 6, 2026
  2. stestagg
    changed the title [-]Deleting unicodedata from sys.modules can cause use-after-free becuase _PyUnicode_GetNameCAPI doesn't retain a reference to ucnhash_capi.[/-] [+]Deleting unicodedata from sys.modules can cause pointer use-after-free[/+] on May 6, 2026
  3. sobolevn
    added
    topic-unicode
    on May 6, 2026
  4. StanFromIreland
    added
    extension-modulesC modules in the Modules dir
    on May 6, 2026

  5. stestagg commented on May 6, 2026

    @stestagg
    stestagg
    on May 6, 2026
    ContributorAuthor

    Ok, This is the commit: 61d2639

    commit 61d2639
    Author: Erlend Egeberg Aasland (@erlend-aasland)
    Date: Wed Jan 20 12:03:53 2021 +0100

    bpo-41798: Allocate unicodedata CAPI on the heap (GH-24128)

    #24128

  6. StanFromIreland
    added
    interpreter-core(Objects, Python, Grammar, and Parser dirs)
    and removed
    extension-modulesC modules in the Modules dir
    on May 6, 2026
  7. eendebakpt
    added a commit that references this issue on May 23, 2026

    pythongh-149449: Fix use-after-free in _PyUnicode_GetNameCAPI

    b942917

  8. eendebakpt commented on May 23, 2026

    @eendebakpt
    eendebakpt
    on May 23, 2026
    Contributor

    I can confirm the issue on main and on earlier releases.

  9. eendebakpt
    added a commit that references this issue on May 24, 2026

    pythongh-149449: Fix use-after-free in unicodedata _ucnhash_CAPI capsule

    ca33d17
  10. kumaraditya303
    added a commit that references this issue on May 24, 2026

    gh-149449: Fix use-after-free in `_PyUnicode_GetNameCAPI` (#150323)

    43c60ec
  11. kumaraditya303
    closed this as completedon May 24, 2026
  12. kumaraditya303
    added 3 commits that reference this issue on May 25, 2026

    [3.14] gh-149449: Fix use-after-free in `_PyUnicode_GetNameCAPI` (GH-…

    ec4df59

    [3.15] gh-149449: Fix use-after-free in `_PyUnicode_GetNameCAPI` (GH-…

    c95aa3a

    [3.13] gh-149449: Fix use-after-free in `_PyUnicode_GetNameCAPI` (GH-…

    60d8437
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    interpreter-core(Objects, Python, Grammar, and Parser dirs)topic-unicodetype-crashA hard crash of the interpreter, possibly with a core dump

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions