Skip to content

doctest fails to compare tracebacks with notes #111159

Closed
Closed
doctest fails to compare tracebacks with notes#111159
Assignees
sobolevn
Labels
stdlibStandard Library Python modules in the Lib/ directorytype-bugAn unexpected behavior, bug, or error

Description

@sobolevn
sobolevn
opened on Oct 21, 2023

Bug report

Minimal reproducer:

"""
Module doctest

>>> raise_with_note()
Traceback (most recent call last):
  ...
ValueError: Text
Note
"""

def raise_with_note():
    err = ValueError('Text')
    err.add_note('Note')
    raise err

Now, run it, you will get a very strange error:

» ./python.exe -m doctest ex.py
**********************************************************************
File "/Users/sobolev/Desktop/cpython/ex.py", line 4, in ex
Failed example:
    raise_with_note()
Expected:
    Traceback (most recent call last):
      ...
    ValueError: Text
    Note
Got:
    Traceback (most recent call last):
      File "/Users/sobolev/Desktop/cpython/Lib/doctest.py", line 1374, in __run
        exec(compile(example.source, filename, "single",
      File "<doctest ex[0]>", line 1, in <module>
        raise_with_note()
      File "/Users/sobolev/Desktop/cpython/ex.py", line 14, in raise_with_note
        raise err
    ValueError: Text
    Note
**********************************************************************
1 items had failures:
   1 of   1 in ex
***Test Failed*** 1 failures.

Things that do not help:

  • IGNORE_EXCEPTION_DETAIL
  • ELLIPSIS
  • Coping the whole traceback

Related to how exceptions with notes are represented in

cpython/Lib/doctest.py

Line 1396 in 7237fb5

exc_msg = traceback.format_exception_only(*exception[:2])[-1]

I am working on this issue right now :)

Refs #111157

Linked PRs

Activity

  1. sobolevn
    added
    type-bugAn unexpected behavior, bug, or error
    stdlibStandard Library Python modules in the Lib/ directory
    on Oct 21, 2023
  2. sobolevn
    self-assigned this
    on Oct 21, 2023
  3. sobolevn
    added 2 commits that reference this issue on Oct 21, 2023

    pythongh-111159: Fix `doctest` output comparison for exceptions with …

    ad6ff28

    pythongh-111159: Fix `doctest` output comparison for exceptions with …

    30669a8
  4. iritkatriel
    added a commit that references this issue on Oct 21, 2023

    gh-111159: Fix `doctest` output comparison for exceptions with notes (#…

    fd60549
  5. miss-islington
    added 2 commits that reference this issue on Oct 21, 2023

    pythongh-111159: Fix `doctest` output comparison for exceptions with …

    47994b0

    pythongh-111159: Fix `doctest` output comparison for exceptions with …

    ee6345c
  6. iritkatriel
    added 2 commits that reference this issue on Oct 21, 2023

    [3.11] gh-111159: Fix `doctest` output comparison for exceptions with…

    cf28c61

    [3.12] gh-111159: Fix `doctest` output comparison for exceptions with…

    f6cde99
  7. terryjreedy
    closed this as completedon Oct 21, 2023

  8. scoder commented on Oct 31, 2023

    @scoder
    scoder
    on Oct 31, 2023
    Contributor

    With this change in place, I now get a (reproducible) doctest failure in lxml that seems rather random:
    https://github.com/lxml/lxml/blob/7356acba91bcf37b2c1c6ab5b7c9e37a4f1145b9/doc/xpathxslt.txt

    Doctest: xpathxslt.txt ... 
======================================================================
ERROR: txt (xpathxslt)
Doctest: xpathxslt.txt
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/python3.13/lib/python3.13/unittest/case.py", line 58, in testPartExecutor
    yield
  File "/opt/python3.13/lib/python3.13/unittest/case.py", line 636, in run
    self._callTestMethod(testMethod)
  File "/opt/python3.13/lib/python3.13/unittest/case.py", line 589, in _callTestMethod
    if method() is not None:
       ^^^^^^^^
  File "/opt/python3.13/lib/python3.13/doctest.py", line 2257, in runTest
    failures, tries = runner.run(
                      ^^^^^^^^^^^
  File "/opt/python3.13/lib/python3.13/doctest.py", line 1537, in run
    return self.__run(test, compileflags, out)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/python3.13/lib/python3.13/doctest.py", line 1402, in __run
    exc_msg_index = next(
                    ^^^^^
StopIteration

    I'm not using "notes" anywhere in the file (and wouldn't know what I'd use them for), so this breaks existing tests.

  9. scoder
    reopened this on Oct 31, 2023

  10. sobolevn commented on Oct 31, 2023

    @sobolevn
    sobolevn
    on Oct 31, 2023
    MemberAuthor

    Thanks, I will have a look!

  11. scoder commented on Oct 31, 2023

    @scoder
    scoder
    on Oct 31, 2023
    Contributor

    I think the problem is with subclasses of SyntaxError. If they live in packages, the startswith() comparison in the newly added code can fail.

  12. sobolevn
    added a commit that references this issue on Oct 31, 2023

    pythongh-111159: Fix `SyntaxError` doctests for non-builtin exception…

    9fa1722
  13. iritkatriel
    added a commit that references this issue on Nov 4, 2023

    gh-111159: Fix `SyntaxError` doctests for non-builtin exception class…

    18c9548
  14. miss-islington
    added 2 commits that reference this issue on Nov 4, 2023

    pythongh-111159: Fix `SyntaxError` doctests for non-builtin exception…

    8c2ac1d

    pythongh-111159: Fix `SyntaxError` doctests for non-builtin exception…

    1ace32d
  15. iritkatriel
    added 2 commits that reference this issue on Nov 4, 2023

    [3.11] gh-111159: Fix `SyntaxError` doctests for non-builtin exceptio…

    425efc1

    [3.12] gh-111159: Fix `SyntaxError` doctests for non-builtin exceptio…

    3b839cc

  16. hugovk commented on Nov 9, 2023

    @hugovk
    hugovk
    on Nov 9, 2023
    Member

    Anything left for this issue or can it be closed? Thanks!

  17. sobolevn
    closed this as completedon Nov 9, 2023
  18. aisk
    added 2 commits that reference this issue on Feb 11, 2024

    pythongh-111159: Fix `doctest` output comparison for exceptions with …

    c35ed3f

    pythongh-111159: Fix `SyntaxError` doctests for non-builtin exception…

    fe3f358
  19. Glyphack
    added 2 commits that reference this issue on Sep 2, 2024

    pythongh-111159: Fix `doctest` output comparison for exceptions with …

    7986ef7

    pythongh-111159: Fix `SyntaxError` doctests for non-builtin exception…

    91518ad
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

stdlibStandard Library Python modules in the Lib/ directorytype-bugAn unexpected behavior, bug, or error

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions