Problems with PyImport_GetImporter()

[serhiy-storchaka]

Bug report

1. PyImport_GetImporter() can return NULL with set or not set error. The latter happens only if sys.path_hooks or sys.path_importer_cache was deleted, or in more obscure cases: when string allocation for strings "path_hooks" or "path_importer_cache" fail, or the sys module was not yet created. These cases so obscure, that the user code most likely do not expect this. The only place where PyImport_GetImporter() is used in Python itself expects an error to be set if NULL is returned.
2. PyImport_GetImporter() can crash (in debug build) or raise SystemError if sys.path_hooks is not a list or sys.path_importer_cache is not a dict. Note that both are set to None in _PyImport_FiniExternal() (which is called in Py_FinalizeEx() and Py_EndInterpreter()).

Crash is unacceptable, so asserts should be replaced with runtime checks and raising an exception (RuntimeError looks suitable).

And I think that PyImport_GetImporter() should set RuntimeError when it fails to get sys.path_hooks or sys.path_importer_cache. It is the most common error in similar cases.

Linked PRs

• gh-109521: Fix obscure cases handling in PyImport_GetImporter() #109522
• [3.12] gh-109521: Fix obscure cases handling in PyImport_GetImporter() (GH-109522) #109777
• [3.11] gh-109521: Fix obscure cases handling in PyImport_GetImporter() (GH-109522) #109781

[serhiy-storchaka]

@ncoghlan, as the author of PyImport_GetImporter(), what can you say about this? Was there a case for PyImport_GetImporter() returning NULL without setting error, or was it just an oversight?