Self-documenting f-string in conversion specifier throws ValueError

[JelleZijlstra]

Bug report

Bug description:

Since Python 3.12, the compiler throws a ValueError when compiling a string like f"{x:{y=}}":

$ ./python.exe 
Python 3.14.0a0 (heads/main:81a654a342, Jun 28 2024, 07:45:17) [Clang 15.0.0 (clang-1500.3.9.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> f"{x:{y=}}"
ValueError: field 'value' is required for Constant

Note this is a compile-time error; you also see it if you call compile() or ast.parse(). I would not expect the compiler to ever throw ValueError.

On 3.11, this works as I'd expect:

$ python
Python 3.11.9 (main, May  7 2024, 09:02:19) [Clang 15.0.0 (clang-1500.3.9.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> f"{x:{y=}}"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'x' is not defined
>>> import ast
>>> ast.dump(ast.parse('f"{x:{y=}}"'))
"Module(body=[Expr(value=JoinedStr(values=[FormattedValue(value=Name(id='x', ctx=Load()), conversion=-1, format_spec=JoinedStr(values=[Constant(value='y='), FormattedValue(value=Name(id='y', ctx=Load()), conversion=114)]))]))], type_ignores=[])"

I don't have a use case for this, but came across it while exploring f-string syntax.

cc @ericvsmith for f-strings and @pablogsal because this feels related to PEP 701

CPython versions tested on:

3.12, CPython main branch

Operating systems tested on:

macOS

Linked PRs

• gh-121130: Fix f-string format specifiers with debug expressions #121150
• [3.13] gh-121130: Fix f-string format specifiers with debug expressions (GH-121150) #121868
• [3.12] gh-121130: Fix f-string format specifiers with debug expressions (GH-121150) #122063

[pablogsal]

Hummm, I think this is a bug in 3.11 or at least some undefined situation. For example, with x=34 and y=20 in 3.11 this prints:

>>> f"{x:{y=}}"
'yyyyyyyyyyyyyyyyyy34'

[pablogsal]

There is not even an = sign there and I guess nobody here would have predicted that result so unless I am missing anything we should not allow this (and raise a proper error). @ericvsmith what do you think?

[JelleZijlstra]

@pablogsal I think that's the expected result, note that the self-documenting string is passed to x.__format__:

>>> (34).__format__("y=20")
'yyyyyyyyyyyyyyyyyy34'

I don't know when this would be practically useful but I think it's at least consistent behavior.

[pablogsal]

Oh, I see! It's certainly surprising but I can see how this happens now much better! Thanks

[pablogsal]

CC @lysnikolaou WDYT?

[pablogsal]

We can fix this in the lexer, but this violates some assumptions (that only the topmost level can have debug expressions) so the fix is quite involved unless I am missing something.

[pablogsal]

I will try to give this a go to see if there is some easy way to handle this

[ericvsmith]

It's definitely an odd use case, but I think it should work.

When testing, I often use datetime, since it's format spec is (mostly) pass-through:

>>> import datetime
>>> y = 3
>>> f'{datetime.datetime.now():{y=}}'
'y=3'