Repository navigation
test_pyexpat: test_large_character_data_does_not_crash() fails with TypeError on Ubuntu BigMem buildbot #150397
Description
Activity
I can reproduce the issue on Fedora 44 using:
./python -m test test_pyexpat -m test_large_character_data_does_not_crash -v -u all -M 20G(need at least 8 GiB of memory, my laptop has 32 GiB)
The test was added by commit bc1be4f:
Author: ByteFlow <fakeshadow1337@gmail.com> Date: Sun May 10 21:42:04 2026 +0800 gh-148441: Avoid integer overflow in Expat's CharacterDataHandler (#148904) Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>I wrote a fix:
diff --git a/Lib/test/test_pyexpat.py b/Lib/test/test_pyexpat.py index 10dca684acc..5a42bbb5f08 100644 --- a/Lib/test/test_pyexpat.py +++ b/Lib/test/test_pyexpat.py @@ -827,7 +827,7 @@ def test_change_size_2(self): @support.requires_resource('cpu') @support.requires_resource('walltime') @support.bigmemtest(size=2**31, memuse=4, dry_run=False) - def test_large_character_data_does_not_crash(self): + def test_large_character_data_does_not_crash(self, size): # See https://github.com/python/cpython/issues/148441 parser = expat.ParserCreate() parser.buffer_text = True
Problem: with the fix, the test does crash! gdb trace on crash:
(gdb) where #0 0x00007ffff7dd4987 in __memmove_avx_unaligned_erms () from /lib64/libc.so.6 #1 0x00007fffe969d5a7 in my_CharacterDataHandler (userData=0x7fffe8efd850, data=0x7ffee81cf430 'A' <repeats 200 times>..., len=1048576) at ./Modules/pyexpat.c:410 #2 0x00007fffe84dc4c0 in doContent () from /lib64/libexpat.so.1 #3 0x00007fffe84dd5a5 in contentProcessor () from /lib64/libexpat.so.1 #4 0x00007fffe84d2bff in callProcessor () from /lib64/libexpat.so.1 #5 0x00007fffe84d8560 in XML_ParseBuffer () from /lib64/libexpat.so.1 #6 0x00007fffe969f3f4 in pyexpat_xmlparser_Parse_impl (self=self@entry=0x7fffe8efd850, cls=cls@entry=0x16a25a0, data=data@entry=b'<r>AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA(...)AAAAAAAAAAAAA', isfinal=1) at ./Modules/pyexpat.c:909 #7 0x00007fffe969f528 in pyexpat_xmlparser_Parse (self=<pyexpat.xmlparser at remote 0x7fffe8efd850>, cls=0x16a25a0, args=<optimized out>, nargs=2, kwnames=<optimized out>) at ./Modules/clinic/pyexpat.c.h:109 ...Reacted by Ivy XuYeah, that positional argument is missed ;) Thanks!
Problem: with the fix, the test does crash! gdb trace on crash:
Wait. How come it still crashes? it should not do so! is there some integer promotion somewhere that wasn't correct?
I am currently working so i can't take this one, but:
- First revert the patch. It's actually incomplete. My bad for this one.
- The let's investivate where we have other issues later... because now it looks like something else is wrong during the parsing.
Problem: with the fix, the test does crash! gdb trace on crash:
I'm not sure why it's still crashing. I applied the same fix but couldn't reproduce the crash locally on either WSL2 (Linux) or Windows 11.
Oh. I can no longer reproduce the
test_large_character_data_does_not_crash()crash!Reacted by Ivy Xu
aarch64 Ubuntu 24.04 BigMem 3.x: https://buildbot.python.org/#/builders/1832/builds/1366
Linked PRs