Repository navigation
When using python -bb, struct.calcsize raises a warning when used with str argument after being used with bytes (might be a larger problem with dicts) #85943
Description
Activity
tadeu commented
on Sep 13, 2020 tadeumannequinMannequinAuthorHere is the inconsistent behavior, when running with
python -bb(or justpython -b), caused by an internal cache:>>> import struct >>> struct.calcsize(b'!d') # cache for '!d' uses bytes 8 >>> struct.calcsize('!d') # so there's a warning when trying to use str Traceback (most recent call last): File "<stdin>", line 1, in <module> BytesWarning: Comparison between bytes and string >>> struct.calcsize('>d') # cache for '>d' uses str 8 >>> struct.calcsize(b'>d') # so now the warning is inverted, it shows up when trying to use bytes Traceback (most recent call last): File "<stdin>", line 1, in <module> BytesWarning: Comparison between bytes and string >>> struct.calcsize('>d') # no problem when using str 8Note that this might be caused by a possible larger problem when dealing with keys of different string types in dicts under
python -b(orpython -bb):$ python Python 3.8.5 (tags/v3.8.5:580fbb0, Jul 20 2020, 15:57:54) [MSC v.1924 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> d={} >>> d['a']=1 >>> d[b'a']=2 >>> d['a'] 1 >>> d[b'a'] 2 $ python -bb Python 3.8.5 (tags/v3.8.5:580fbb0, Jul 20 2020, 15:57:54) [MSC v.1924 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> d={} >>> d['a']=1 >>> d[b'a']=2 Traceback (most recent call last): File "<stdin>", line 1, in <module>I'm not sure if this warning is intentional, since in Python 3 there seems to be no special reason for dicts to try to compare 'a' with b'a' (other than possible implementation details).
Note: this is from an issue found here: pytest-dev/pytest-xdist#596
The-Compiler commented
on Sep 13, 2020 The-CompilermannequinMannequinTaking the freedom of adding people involved in the
structmodule to the nosy list.tadeu commented
on Sep 13, 2020 tadeumannequinMannequinAuthorI'm not sure if this warning is intentional, since in Python 3 there seems to be no special reason for dicts to try to compare 'a' with b'a' (other than possible implementation details).
Okay, there's one special reason, it's the fact that 'a' and b'a' have the same hash. I'm not sure about the expected behavior, though.
The-Compiler commented
on Sep 13, 2020 The-CompilermannequinMannequinAh, also see https://bugs.python.org/issue21071#msg292409 where the same thing was mentioned as part of another issue as well.
After some discussions in the Python IRC channel, I guess it's acceptable for dicts to raise a ByteWarning here - after all, there *is* a comparison between str/bytes going on here. It might be an implementation detail, but so is e.g. b'a' in ['a'] and I'd certainly expect that to give me a warning/error with -b/-bb.
So I guess if struct continues to accept bytes as format string, it should probably decode them to ASCII or something before interacting with the cache?
- moved this from Todo to In Progress in Struct, memoryview and array issues 🏗️
on Jul 13, 2026 - moved this from In Progress to Done in Struct, memoryview and array issues 🏗️
on Jul 17, 2026 - added 3 commits that reference this issue
on Jul 17, 2026
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
Linked PRs