3.12 regression: cannot create weak reference to 'typing.TypeVar' object

[encukou]

This works in Python 3.11, but fails in 3.12b3:

import weakref
import typing
weakref.ref(typing.TypeVar('T'))

I'm assuming it's an unintentional side effect of the PEP-695 implementation? Unfortunately I don't have time to investigate right now, let me know if I should.

Reported in cloudpipe/cloudpickle#507

Linked PRs

• gh-106403: Restore weakref support for TypeVar and friends #106418
• [3.12] gh-106403: Restore weakref support for TypeVar and friends (GH-106418) #106635

[JelleZijlstra]

Fixing.

The new typing.TypeAliasType is also not weakrefable:

>>> type T = int
>>> weakref.ref(T)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: cannot create weak reference to 'typing.TypeAliasType' object

But neither is the existing UnionType:

>>> weakref.ref(int | str)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: cannot create weak reference to 'types.UnionType' object

So maybe that's fine?

[ogrisel]

Thanks @JelleZijlstra!

So maybe that's fine?

I suspect that this might cause problems with cloudpickle when executing interactively defined functions with type annotations in their signatures on remote Python workers (e.g. in Dask, Ray or PySpark / databricks).

I did not have time to investigate the details myself yet but @musicinmybrain tried to test your fix in cloudpickle in cloudpipe/cloudpickle#507 (comment) and there are still segfaults to debug.

[JelleZijlstra]

I merged my PR that re-adds weakref support for TypeVars. I haven't tried to debug the segfaults in the cloudpickle tests, but let me know if those can be minified to something that shows the issue is in CPython.

[Yhg1s]

Closing this as the original report is (believed to be) fixed. If the other cloudpickle issues are caused by bugs in 3.12 it probably deserves its own issue.