Skip to content

IndexError on REPL startup in _pyrepl/reader.py with Python 3.13.0b1+ and a custom sys.ps1 set in a PYTHONSTARTUP file #118835

Description

@mgedmin

Bug report

Bug description:

I compiled Python 3.13 (commit d86b494, the current tip of the 3.13 branch), and installed it into ~/opt/python313. When I run python3.13 with no arguments, I get this:

Python 3.13.0b1+ (heads/3.13:d86b494117, May  9 2024, 15:23:21) [GCC 13.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/mg/opt/python313/lib/python3.13/_pyrepl/__main__.py", line 47, in <module>
    interactive_console()
    ~~~~~~~~~~~~~~~~~~~^^
  File "/home/mg/opt/python313/lib/python3.13/_pyrepl/__main__.py", line 44, in interactive_console
    return run_interactive(mainmodule)
  File "/home/mg/opt/python313/lib/python3.13/_pyrepl/simple_interact.py", line 138, in run_multiline_interactive_console
    statement, contains_pasted_code = multiline_input(more_lines, ps1, ps2)
                                      ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mg/opt/python313/lib/python3.13/_pyrepl/readline.py", line 301, in multiline_input
    return reader.readline(), reader.was_paste_mode_activated
           ~~~~~~~~~~~~~~~^^
  File "/home/mg/opt/python313/lib/python3.13/_pyrepl/reader.py", line 646, in readline
    self.prepare()
    ~~~~~~~~~~~~^^
  File "/home/mg/opt/python313/lib/python3.13/_pyrepl/historical_reader.py", line 285, in prepare
    super().prepare()
    ~~~~~~~~~~~~~~~^^
  File "/home/mg/opt/python313/lib/python3.13/_pyrepl/reader.py", line 521, in prepare
    self.calc_screen()
    ~~~~~~~~~~~~~~~~^^
  File "/home/mg/opt/python313/lib/python3.13/_pyrepl/completing_reader.py", line 259, in calc_screen
    screen = super().calc_screen()
  File "/home/mg/opt/python313/lib/python3.13/_pyrepl/reader.py", line 299, in calc_screen
    self.cxy = self.pos2xy()
               ~~~~~~~~~~~^^
  File "/home/mg/opt/python313/lib/python3.13/_pyrepl/reader.py", line 474, in pos2xy
    p, l2 = self.screeninfo[y]
            ~~~~~~~~~~~~~~~^^^
IndexError: list index out of range

and it exits with status code 1.

My terminal is currently 126 columns by 45 lines, and I have a custom PYTHONSTARTUP environment file pointing to ~/.python, which you can see in this GH repo. It sets up some conveniences for the old REPL (colorful prompt, rlcompleter-based tab-completion, persistent readline history).

Running PYTHONSTARTUP= python3.13 fixes the IndexError.

By commenting out bits of the file I can reproduce the IndexError with just this:

import sys
sys.ps1 = '\001\033[0;32m\002>>> \001\033[0m\002'
del sys

The \001/\002 characters are hints for readline to inform it that everything between them is a terminal escape sequence that doesn't advance the cursor, without which readline cannot correctly compute the width of the prompt and makes wrong assumptions about line wrapping.

The IndexError goes away if I remove all \001 and \002 characters from sys.ps1.

CPython versions tested on:

3.13

Operating systems tested on:

Linux

Linked PRs

Activity

  1. added
    type-bugAn unexpected behavior, bug, or error
    on May 9, 2024
  2. mgedmin commented on May 9, 2024

    @mgedmin
    Author

    Since I heard that the new REPL code was based on PyPy's, I checked PyPy (7.3.16) and it had no issues with my PYTHONSTARTUP file. I could see that sys.ps1 did include the '\x01'/'\x02' characters.

  3. The-Compiler commented on May 23, 2024

    @The-Compiler
    Contributor

    FWIW I also immediately ran into this when testing 3.13b1. Given that Python 3.13 has its own colored prompt anyways, I guarded it by a:

    if sys.version_info != (3, 13, 0, 'beta', 1):
  4. danielhollas commented on May 31, 2024

    @danielhollas
    Contributor

    Just verified that this is still an issue on the latest main.

  5. danielhollas commented on Jun 1, 2024

    @danielhollas
    Contributor

    The problem is somewhere in the _pyrepl/reader.py::Reader.process_prompt method,
    which for the following prompt returns length -7. 😱

    >>> ps1 = '\001\033[0;32m\002>>> \001\033[0m\002'
    >>> Reader.process_prompt(ps1)
    ('\x1b[0;32m>>> \x1b[0m', -7)

    I'll take a closer look tomorrow.

  6. added a commit that references this issue on Jun 3, 2024
  7. added a commit that references this issue on Jun 3, 2024
  8. added a commit that references this issue on Jun 3, 2024
  9. added a commit that references this issue on Jun 3, 2024
  10. added a commit that references this issue on Jun 5, 2024
  11. added a commit that references this issue on Jul 11, 2024
  12. added a commit that references this issue on Jul 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    3.13only security fixestopic-replRelated to the interactive shelltype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions