Repository navigation
port shebang of tools from python2 to python3 #85792
Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2020-08-24.16:28:45.369>
labels = ['library', '3.10']
title = 'port shebang of tools from python2 to python3'
updated_at = <Date 2020-12-01.11:51:42.061>
user = 'https://github.com/shihai1991'bugs.python.org fields:
activity = <Date 2020-12-01.11:51:42.061>
actor = 'christian.heimes'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Demos and Tools', 'Library (Lib)']
creation = <Date 2020-08-24.16:28:45.369>
creator = 'shihai1991'
dependencies = []
files = []
hgrepos = []
issue_num = 41626
keywords = ['patch']
message_count = 8.0
messages = ['375855', '379244', '379247', '379279', '379969', '379970', '382238', '382241']
nosy_count = 7.0
nosy_names = ['terry.reedy', 'christian.heimes', 'ned.deily', 'eric.araujo', 'petr.viktorin', 'corona10', 'shihai1991']
pr_nums = ['21948', '23581']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue41626'
versions = ['Python 3.10']
Activity
shihai1991 commentedon Aug 24, 2020
After the EOL of python2, the shebang of tools should be ported from python2 to python3.
some files like: https://github.com/python/cpython/blob/master/Objects/typeslots.py#L1
merwok commentedon Oct 21, 2020
This is a minor thing, and not a particularly visible bug, but it could create confusion about how to run scripts so could be fixed.
1) For stdlib modules, I think the intended usage is
python3 -m encodings.rot_13; I don’t expect anyone to runpython3 /usr/lib/python3.10/encodings/rot_13.pyto do a decoding in the terminal!So for these easter eggs / hidden gems in the stdlib, I would delete the shebang and remove the executable bit. There are 18 such modules, ignoring test, lib2to3, turtledemo and a few others named below.
1.1) There may be some exceptions: for Lib/idlelib/pyshell.py for example, maybe the IDLE entry point is just a symlink to that file, and removing the executable bit would break it. So this can’t be a mass update but each case should be reviewed.
2) Some of the files in the PR are used during development of CPython: Tools/clinic/clinic.py Python/makeopcodetargets.py Parser/asdl_c.py Objects/typeslots.py Modules/_sha3/cleanup.py Lib/ctypes/macholib/fetch_macholib Mac/BuildScript/build-installer.py
I think some of these work with any Python, but some need the locally built interpreter to give correct results.
It could avoid confusion and mistakes to remove the shebangs and executable bits from the scripts that need a local Python, and make dure their documentation mentions
./python path/to/tool.2.1) One recent example, Tools/peg_generator/pegen/__main__.py is documented to work with Python 3.8+, and confusingly is both a __main__ module (to support python3 -m pegen and an executable script with shebang!)
2.2) In Tools/ssl, we have
/usr/bin/env python3in make_ssl_data.py and./pythonin multissltests.py. Are these working?!3) The files in Tools/scripts are documented as «useful for building Python» or «generally useful scripts». They nearly all executable and nearly all use
#!/usr/bin/env python3, so fixing the fewpythonreferences there seems ok. Likewise for Tools/pynche/pynche.3.1) For Tools/gdb/libpython.py I don’t know how it’s meant to be used; if it’s not distributed but only for debugging a locally built CPython, then removing the shebang seems correct.
(Oh and by the way, Tools/scripts/fixps.py is a tool to replace
/usr/local/bin/pythonshebangs with/usr/bin/env python… You can judge if that is useful and/or correct.)So if things are changed (let’s get a couple opinions here), maybe this needs separate PRs for 1/2/3, and make sure to get reviews to ensure we don’t break IDLE / Mac installers / pegen / etc.
ned-deily commentedon Oct 21, 2020
To address some of the concerns:
The shebang line in Mac/BuildScript/build-installer.py can be safely removed.
"Lib/idlelib/pyshell.py for example, maybe the IDLE entry point is just a symlink to that file"
It's not, at least for unix-y builds. In [install-prefix], an "idle*" script is installed of the form:
#![install-prefix]/bin/python3.[n]
So there is an absolute link to the correct interpreter and the shebang in pyshell.py is not used.
terryjreedy commentedon Oct 22, 2020
Thoughts I wrote on the PR that belong here. (Thanks for the reminder, Éric.):
*The use of #! in both stdlib and tools seems rather inconsistent.
-- I was not really aware of idlelib.pyshell. Running with an explicit binary seems particularly important for IDLE. About once a month on SO, some beginner posts about not being able to import a module they downloaded when running IDLE (maybe only sometimes). Nearly always, they have 2 pythons, such as from Anaconda and python.org.
I may want or need to deprecate using pyshell as entry instead of idle.py/idlew.py/idle.bat as I want to move startup code from pyshell to either idle.py or a separate startup only file. I am not sure how to get from here to there, partly because I don't really know what 'here' is in practice.
With 2.7 put to bed, the line is hardly needed to select between latest 2.x and latest 3.x.
The response of py.exe to shebang lines needs to be detailed and considered. I believe it only knows about python.org installs, so it will only start the latest python.org install. I don't know what it does if 32- and 64- bit versions are both present. I also don't know what it runs if the 'default' installed version is not the latest installed version. Does py.exe know what the default is, or does that just affect what 'python' runs?
shihai1991 commentedon Oct 30, 2020
Sorry for my delay~
I update Objects/typeslots.py in #PR21913 recently and using f-format in this script(it means that just local python or python3 can run). So I will accept your suggestion from this file after this PR merged:)
Thanks, I will take a look.
Agree. To corner case, a single sepereate PR is good.
shihai1991 commentedon Oct 30, 2020
typo error: PR21913->PR21931
shihai1991 commentedon Dec 1, 2020
I create PR23581 to remove the inner Tools' shebang.
But I am not remove Tools/ssl/make_ssl_data.py, Tools/gdb/libpython.py in PR23581.
tiran commentedon Dec 1, 2020
make_ssl_data.py is an internal tool that generates internal header files. It's rarely used. The tool is not bound to any Python version, too. Feel free to change the shebang or leave it as it is. I don't care.
terryjreedy commentedon Aug 23, 2023
As near as I can tell, encodings.rot_13 is currently the only$ /Lib module with a shebang line with 'python' rather than 'python3'. #108273 proposes to modify this one file. Does this seem appropriate?
$ idlelib/idletest/example_noext has the same line (with a harmless bug), but it is a dummy file for testing whether IDLE recognizes an extensionless file with a python shebang line as python source. I intend to change it for consistency, but as part of a PR making other changes and changing other files and backporting them.
As noted above, idlelib.pyshell has a 'python3' shebang line. It has not been the official initial startup file for at least a decade. idlelib.idle does not have such a line, so it seems to be unneeded. I will think about removing that line (and exec bit?) in 3.13.
There seems to be as many /Lib files with
if __name__ == '__main__':without a shebang line as with.encukou commentedon Mar 19, 2024
I think that today, it's best to close this issue without action. python2/python3 confusion should be history now.
I'll close next month if there are no objections.
merwok commentedon Mar 19, 2024
This wasn’t only about 2 vs 3 ‑ there were also notes about stdlib modules having executable bit and shebang needlessly
encukou commentedon Mar 20, 2024
OK, I'll only close the PR for internal tools.
merwok commentedon May 28, 2024
That last part is handled in #118673
pythongh-85792: Update shebang to python3
sandy-lcq commentedon Mar 5, 2025
There are still several files using shebang "/usr/bin/env python", add a pull request to fix them.
pythongh-85792: Update shebang to python3