Repository navigation
t-strings: AST unparsing does not reproduce original formatting #133581
Description
Bug report
Bug description:
The AST unparsing of Interpolation objects does not reproduce formatting within interpolations.
This is true both for ast.unparse in Python:
>>> ast.unparse(ast.parse('t"{a + b}"'))
"t'{a + b}'"
And for the C unparse that is used by from __future__ import annotations:
>>> ns = {}
>>> exec("""
... from __future__ import annotations
... def f(x: t"{a + b}"): ...
... """, ns)
>>> ns["f"].__annotations__
{'x': "t'{a + b}'"}
In general, we can't recover the original source exactly, but in this case we do have the original source available:
>>> ast.parse('t"{a + b}"').body[0].value.values[0].str
'a + b'
Let's use it.
CPython versions tested on:
3.14
Operating systems tested on:
No response
Activity
JelleZijlstra commentedon May 7, 2025
This leads to AST roundtrip failures (showing up in test cases in #133553).
JelleZijlstra commentedon May 7, 2025
cc @lysnikolaou
lysnikolaou commentedon May 7, 2025
Hmm, what exactly is failing in the AST roundtrip tests?
test_unparseonly checks ASTs, which should be the same for all of these, right?JelleZijlstra commentedon May 7, 2025
The AST changes on roundtrip because the AST for an interpolation object includes the raw text, which changes.
lysnikolaou commentedon May 7, 2025
Oh right, I hit that while working on the PEP PR and thought I'd implemented skipping checking
ast.Interpolation.strintest_unparse.Not sure whether we should be using it for
test_unparse, that'd be breaking assumptions for the formatting of the code. For example,a + broundtrips toa + b. Why should t-string be different in that regard?JelleZijlstra commentedon May 7, 2025
Because in t-strings the original source code is part of the AST, and in the rest of the grammar it isn't.
In general I feel
ast.unparse(ast.parse(...))should roundtrip as much as possible, to the extent that the original source code is still recoverable from the AST.gh-133581: Improve AST unparsing of t-strings (#133635)
pythongh-133581: Improve AST unparsing of t-strings (pythonGH-133635)
[3.14] gh-133581: Improve AST unparsing of t-strings (GH-133635) (#13…
pythongh-133581: Fix refleak in AST unparsing
gh-133581: Fix refleak in t-string AST unparsing (#133724)
pythongh-133581: Fix refleak in t-string AST unparsing (pythonGH-133724)
[3.14] gh-133581: Fix refleak in t-string AST unparsing (GH-133724) (#…
lysnikolaou commentedon May 9, 2025
This has been fixed.
pythongh-133581: Improve AST unparsing of t-strings (python#133635)
pythongh-133581: Fix refleak in t-string AST unparsing (python#133724)
pythongh-133581: Improve AST unparsing of t-strings (python#133635)
pythongh-133581: Fix refleak in t-string AST unparsing (python#133724)