Repository navigation
Calling _PyBytes_Resize() on 1-byte bytes may raise error #87193
Description
Activity
animalize commented
on Jan 26, 2021 animalizemannequinMannequinAuthorPyBytes_FromStringAndSize() uses a global cache for 1-byte bytes:
https://github.com/python/cpython/blob/v3.10.0a4/Objects/bytesobject.c#L147if (size == 1 && str != NULL) { struct _Py_bytes_state *state = get_bytes_state(); op = state->characters[*str & UCHAR_MAX]; if (op != NULL) { Py_INCREF(op); return (PyObject *)op; } }
_PyBytes_Resize() will raise an error when (refcount != 1):
https://github.com/python/cpython/blob/v3.10.0a4/Objects/bytesobject.c#L3029Then this code will raise an exception:
obj1 = PyBytes_FromStringAndSize("a", 1); obj2 = PyBytes_FromStringAndSize("a", 1); ret = _PyBytes_Resize(&obj2, 2); // ret is -1
BTW, 0-byte bytes comes from a global singleton, but _PyBytes_Resize() processes it before checking refcount:
https://github.com/python/cpython/blob/v3.10.0a4/Objects/bytesobject.c#L3021if (Py_SIZE(v) == 0) { if (newsize == 0) { return 0; } *pv = _PyBytes_FromSize(newsize, 0); Py_DECREF(v); return (*pv == NULL) ? -1 : 0; } if (Py_REFCNT(v) != 1) { goto error; }
_PyBytes_Resize() doc:
Only use this to build up a brand new bytes object; don’t use this if the bytes may already be known in other parts of the code. It is an error to call this function if the refcount on the input bytes object is not one.4 remaining items
IMO, it's a docs issue:
PyBytes_From*should say that it returns a new reference, not necessarily to a brand new object.#117160 fixes this for cached 1-byte bytes objects, and also adds support of arbitrary bytes objects with refcount > 1. Note that resizing the empty bytes object was already supported even if it is always shared.
I am not sure whether this should be backported.
Reacted by Erlend E. Aasland- added a commit that references this issue
on Mar 25, 2024 I'd say your fix is a new feature, and shouldn't be backported.
- added 2 commits that reference this issue
on Apr 2, 2024
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
Linked PRs