Repository navigation
Support linking against zlib-rs and libbzip2-rs #139314
Description
Activity
(not an official cpython maintainer, but I am the person who did the work to push for faster pip install and zlib-ng)
Eventually (not proposing this yet!), perhaps we could use zlib-rs for Windows builds to provide greater memory safety.
FYI: Python 3.14 already has switched to zlib-ng for the official cpython release on Windows. The adoption is driven by performance.
The blocker at this point is getting Linux and Mac builds to use zlib-ng. Windows was done first because all the libraries are vendored and it was easier to do. Other OS are problematic.
-
So far, nobody has been able to modify the custom builds scripts from the python interpreter to detect and use zlib-ng instead of zlib on Linux. That's the reason why python 3.14 on Linux is still using zlib instead of zlib-ng.
-
zlib-ng is not available in most main distributions like Ubuntu/Debian/Centos (you can't apt install/yum install to get them), which has been a problem for testing and doing feature detection. This is shifting slowly. You may have a similar problem with rust libraries that are available from cargo but not through Linux distributions.
-
pkgs.org is a good place to see what packages are available on Linux https://pkgs.org/search/?q=zlib-ng
Fedora was the first and only distributions to have swapped to zlib-ng.
I see there is a centos 10 with zlib-ng-compat/zlib-ng-compat-devel just released. Maybe centos just swapped to zlib-ng 😄
If you can figure out how to do library detection to use whichever is available among zlib-ng/zlib-rs/etc, by all means, you should go ahead! :D
-
FYI: Python 3.14 already has switched to zlib-ng for the official cpython release on Windows. The adoption is driven by performance.
Oh I'm well aware! And happy about that.
The blocker at this point is getting Linux and Mac builds to use zlib-ng.
I think I can adapt the above PR to be more generic. I think we could add a configure flag
--with-zlib=[no,zlib,zlib-ng,zlib-rs], which would allow specifying the zlib library to link against (normal zlib remaining the default).nowould mean to disable searching for zlib. We can do something similar for bzip2 and other compression libraries and allow users to disable compression libraries (for #137812) at configure time, as well as specify which backend to use if they do want the library.https://peps.python.org/pep-0775/ and #91246 were withdrawn/closed so I suppose that means the
=nooption is still required even forzlib. regardless, most people should never use it. WASI seemed to be the reason for that.I approved your draft PR but if you want to rework into a more generic form of =value rather than combo of mutually exclusive --with flags, that'd make sense for these. Another option should be "auto" which allows our configure step to choose the implementation to use based on what we find present. Long term goal: choose the nicest found by default for some value of nicest.
tangentially, FYI - Some things also appear to use https://github.com/ebiggers/libdeflate for performance - but it is lower level and not a drop-in zlib API replacement. I am not suggesting we ever do anything with that.
- added a commit that references this issue
on Aug 14, 2026
Feature or enhancement
The Trifecta Tech Foundation has sponsored work on Rust re-implementations of zlib and libbzip2.
These implementations are as fast or faster than their C counterparts (including zlib-chromium and zlib-ng). zlib-rs has seen adoption in Firefox, cargo, and many other major tools, including uv. These implementations are also "drop-in" replacements - they just require slight adjustment to linker arguments.
I think adding support for linking against zlib-rs and libbzip2-rs should be easy to do and maintain, and enable safer builds of CPython.
Eventually (not proposing this yet!), perhaps we could use zlib-rs for Windows builds to provide greater memory safety.
Related issue discussing supporting zlib-ng (mostly Unix build support is left): #91349
I have a draft PR with an implementation below:
Linked PRs
zlib-rsandlibbzip2-rs#155119