Skip to content

gh-93065: Fix HAMT to iterate correctly over 7-level deep trees #93066

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
May 23, 2022
Merged

gh-93065: Fix HAMT to iterate correctly over 7-level deep trees #93066

merged 2 commits into from
May 23, 2022

Conversation

1st1
Copy link
Member

@1st1 1st1 commented May 22, 2022

No description provided.

@ghost
Copy link

ghost commented May 22, 2022 •
edited by ghost
Loading

All commit authors signed the Contributor License Agreement.
CLA signed

@1st1 1st1 added needs backport to 3.9 only security fixes needs backport to 3.10 only security fixes needs backport to 3.11 only security fixes labels May 22, 2022
@1st1 1st1 requested review from pablogsal and ambv May 22, 2022 06:18
@1st1 1st1 mentioned this pull request May 22, 2022
Closed
@molaxx
Copy link
Contributor

molaxx commented May 22, 2022 •
edited
Loading

Hey @1st1 as I found and fixed this bug (MagicStack/immutables#85) can I open this PR? I'd really like to have a merge in CPython on my name.

@1st1
Copy link
Member Author

1st1 commented May 22, 2022

@molaxx Eli, the commit is a bit more involved now as I added a comment explaining the fix + a test. I'd prefer to keep this PR and merge it.

That said, this change is yours, so I changed the author of the commit to your name and also added you to Misc/ACKS.

Now since you're the author, you need to accept our CLA here: #93066 (comment)

@molaxx
Copy link
Contributor

molaxx commented May 22, 2022

@1st1 Thanks!

@1st1 1st1 force-pushed the fixiter branch 4 times, most recently from acf6ba7 to 0ed77a3 Compare May 22, 2022 20:54
@molaxx @1st1
pythongh-93065: Fix HAMT to iterate correctly over 7-level deep trees.
4216b2b 
Also while there, clarify a few things about why we reduce
the hash to 32 bits.

Co-authored-by: Yury Selivanov <yury@edgedb.com>
vstinner
vstinner approved these changes May 23, 2022
View reviewed changes
Copy link
Member

@vstinner vstinner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for adding a comment explaining the 8 constant!

ambv
ambv reviewed May 23, 2022
View reviewed changes
Misc/NEWS.d/next/Core and Builtins/2022-05-21-23-21-37.gh-issue-93065.5I18WC.rst Outdated

The bug was discovered and fixed by Eli Libman. See [1] for more details.

[1] https://github.com/MagicStack/immutables/issues/84
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't look great in the actual change log later:
Screen Shot 2022-05-23 at 19 38 32

I'll change it to an inline link and land.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After:
Screen Shot 2022-05-23 at 20 19 33

@ambv
Copy link
Contributor

ambv commented May 23, 2022

Backporting to all security branches as potential crashers are DoS.

@ambv ambv merged commit c1f5c90 into python:main May 23, 2022
@miss-islington
Copy link
Contributor

Thanks @1st1 for the PR, and @ambv for merging it 🌮🎉.. I'm working now to backport this PR to: 3.7, 3.8, 3.9, 3.10, 3.11.
🐍🍒⛏🤖

@bedevere-bot bedevere-bot removed the needs backport to 3.11 only security fixes label May 23, 2022
@bedevere-bot
Copy link

GH-93145 is a backport of this pull request to the 3.11 branch.

@bedevere-bot
Copy link

GH-93146 is a backport of this pull request to the 3.10 branch.

@bedevere-bot bedevere-bot removed the needs backport to 3.10 only security fixes label May 23, 2022
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request May 23, 2022
@1st1 @miss-islington
pythongh-93065: Fix HAMT to iterate correctly over 7-level deep trees (…
3eb4b54 
…pythonGH-93066)

Also while there, clarify a few things about why we reduce the hash to 32 bits.

Co-authored-by: Eli Libman <eli@hyro.ai>
Co-authored-by: Yury Selivanov <yury@edgedb.com>
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
(cherry picked from commit c1f5c90)

Co-authored-by: Yury Selivanov <yury@edgedb.com>
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request May 23, 2022
@1st1 @miss-islington
pythongh-93065: Fix HAMT to iterate correctly over 7-level deep trees (…
dc2c9fe 
…pythonGH-93066)

Also while there, clarify a few things about why we reduce the hash to 32 bits.

Co-authored-by: Eli Libman <eli@hyro.ai>
Co-authored-by: Yury Selivanov <yury@edgedb.com>
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
(cherry picked from commit c1f5c90)

Co-authored-by: Yury Selivanov <yury@edgedb.com>
@bedevere-bot bedevere-bot removed the needs backport to 3.9 only security fixes label May 23, 2022
@bedevere-bot
Copy link

GH-93147 is a backport of this pull request to the 3.9 branch.

@miss-islington
Copy link
Contributor

Sorry, @1st1 and @ambv, I could not cleanly backport this to 3.8 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker c1f5c903a7e4ed27190488f4e33b00d3c3d952e5 3.8

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request May 23, 2022
@1st1 @miss-islington
pythongh-93065: Fix HAMT to iterate correctly over 7-level deep trees (…
3a99e8e 
…pythonGH-93066)

Also while there, clarify a few things about why we reduce the hash to 32 bits.

Co-authored-by: Eli Libman <eli@hyro.ai>
Co-authored-by: Yury Selivanov <yury@edgedb.com>
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
(cherry picked from commit c1f5c90)

Co-authored-by: Yury Selivanov <yury@edgedb.com>
@miss-islington
Copy link
Contributor

Sorry @1st1 and @ambv, I had trouble checking out the 3.7 backport branch.
Please backport using cherry_picker on command line.
cherry_picker c1f5c903a7e4ed27190488f4e33b00d3c3d952e5 3.7

ambv pushed a commit to ambv/cpython that referenced this pull request May 23, 2022
@1st1 @ambv
[3.8] pythongh-93065: Fix HAMT to iterate correctly over 7-level deep…
563ad16 
… trees (pythonGH-93066)

Also while there, clarify a few things about why we reduce the hash to 32 bits.

Co-authored-by: Eli Libman <eli@hyro.ai>
Co-authored-by: Yury Selivanov <yury@edgedb.com>
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
(cherry picked from commit c1f5c90)

Co-authored-by: Yury Selivanov <yury@edgedb.com>
@bedevere-bot
Copy link

GH-93148 is a backport of this pull request to the 3.8 branch.

ambv pushed a commit to ambv/cpython that referenced this pull request May 23, 2022
@1st1 @ambv
[3.7] pythongh-93065: Fix HAMT to iterate correctly over 7-level deep…
df44b36 
… trees (pythonGH-93066)

Also while there, clarify a few things about why we reduce the hash to 32 bits.

Co-authored-by: Eli Libman <eli@hyro.ai>
Co-authored-by: Yury Selivanov <yury@edgedb.com>
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
(cherry picked from commit c1f5c90)

Co-authored-by: Yury Selivanov <yury@edgedb.com>
@bedevere-bot
Copy link

GH-93149 is a backport of this pull request to the 3.7 branch.

@1st1
Copy link
Member Author

1st1 commented May 23, 2022

Thanks so much Lukasz!!

ambv pushed a commit that referenced this pull request May 24, 2022
@miss-islington
gh-93065: Fix HAMT to iterate correctly over 7-level deep trees (GH-9…
c771cbe 
…3066) (GH-93145)

Also while there, clarify a few things about why we reduce the hash to 32 bits.

Co-authored-by: Eli Libman <eli@hyro.ai>
Co-authored-by: Yury Selivanov <yury@edgedb.com>
Co-authored-by: Łukasz Langa <lukasz@langa.pl>

(cherry picked from commit c1f5c90)
ambv pushed a commit that referenced this pull request May 24, 2022
@miss-islington
gh-93065: Fix HAMT to iterate correctly over 7-level deep trees (GH-9…
a4bea26 
…3066) (GH-93146)

Also while there, clarify a few things about why we reduce the hash to 32 bits.

Co-authored-by: Eli Libman <eli@hyro.ai>
Co-authored-by: Yury Selivanov <yury@edgedb.com>
Co-authored-by: Łukasz Langa <lukasz@langa.pl>

(cherry picked from commit c1f5c90)
ambv pushed a commit that referenced this pull request May 24, 2022
@miss-islington
gh-93065: Fix HAMT to iterate correctly over 7-level deep trees (GH-9…
95c9c2b 
…3066) (#93147)

Also while there, clarify a few things about why we reduce the hash to 32 bits.

Co-authored-by: Eli Libman <eli@hyro.ai>
Co-authored-by: Yury Selivanov <yury@edgedb.com>
Co-authored-by: Łukasz Langa <lukasz@langa.pl>

(cherry picked from commit c1f5c90)
ambv added a commit that referenced this pull request May 24, 2022
@ambv
[3.8] gh-93065: Fix HAMT to iterate correctly over 7-level deep trees (…
6d4927a 
…GH-93066) (#93148)

Also while there, clarify a few things about why we reduce the hash to 32 bits.

Co-authored-by: Eli Libman <eli@hyro.ai>
Co-authored-by: Yury Selivanov <yury@edgedb.com>
Co-authored-by: Łukasz Langa <lukasz@langa.pl>

(cherry picked from commit c1f5c90)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ambv ambv ambv approved these changes

@vstinner vstinner vstinner approved these changes

@pablogsal pablogsal Awaiting requested review from pablogsal

Assignees

@ambv ambv

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@1st1 @molaxx @ambv @miss-islington @bedevere-bot @vstinner