Repository navigation
zipimport cannot do a namespace import when a directory has no python files, but it contains nested directories with python files inside of a zip file. #121111
Description
Activity
The issue was originally identified in this issue as well: SeaHOH/memimport#3
(I'd like to work on this issue, please assign it to me.)
It seems thatzipimport.zipimporterdoesn't like any subdirectory directory that doesn't contain an__init__.pyfile:A quick repro:
$ mkdir x $ touch x/a.py $ zip -r x.zip ximport zipimport import importlib.util x = zipimport.zipimporter("x.zip") spec = x.find_spec("x") mod = importlib.util.module_from_spec(spec) x.exec_module(mod) # Error!
(As an additional note, typeshed does not have the
exec_modulemethod onzipimporteryet. See this issue).Although, this works just fine when you do it the normal way:
import sys sys.path.append("./x.zip") import x
Ah, FWIW, I was being dumb - my example here is unable to import a submodule anyway:
import sys sys.path.append("./x.zip") import x
(I don't think this is something that could be added, and out of the scope of this issue anyway.) Here's the actual bug:
Using the following commands as setup:
$ mkdir a a/b a/b/c $ touch a/__init__.py a/b/c/__init__.py $ zip -r a.zip a $ rm -rf aThe following works using
import,import sys sys.path.append("a.zip") import a.b.c
but the following does not, using
zipimport:import zipimport importer = zipimport.zipimporter("./a.zip") importer.get_code("a") importer.get_code("a.b") # Error!
Ah, FWIW, I was being dumb - my example here is unable to import a submodule anyway:
import sys sys.path.append("./x.zip") import x
(I don't think this is something that could be added, and out of the scope of this issue anyway.) Here's the actual bug:
Using the following commands as setup:
$ mkdir a a/b a/b/c $ touch a/__init__.py a/b/c/__init__.py $ zip -r a.zip a $ rm -rf aThe following works using
import,import sys sys.path.append("a.zip") import a.b.c
but the following does not, using
zipimport:import zipimport importer = zipimport.zipimporter("./a.zip") importer.get_code("a") importer.get_code("a.b") # Error!
it is odd that
import a.b.cworks but on my zip filefrom a.b.c import <method here>does not.I didn't test that - does that fail? (Note that you need an
__init__.py)Here is how to repro:
#!/usr/bin/env python3 # test.py import zipfile with zipfile.ZipFile('a.zip', 'w') as zf: zf.writestr('a/__init__.py', b'') zf.writestr('a/b/c/__init__.py', b'') import sys sys.path.append("a.zip") import a.b.c print(a.b)
$ ./test.py Traceback (most recent call last): File "./test.py", line 12, in <module> import a.b.c ModuleNotFoundError: No module named 'a.b'
When add this line in create zip file:
zf.mkdir('a/b/') # need py311
$ ./test.py <module 'a.b' (<_frozen_importlib_external.NamespaceLoader object at 0x0000000000D371D0>)>
It looks rather like a new feature, so no.
There is a simple workaround -- always add explicit directory entries to the ZIP file.
zipfile,shutil.make_archive()andzipapphave been doing this for years.It seems that the original issue has been resolved.
Reacted by Peter Bierma and AraHaanJust a quick comment to say that the
make testfails on my setuptestNamespacePackageExplicitDirectories (test.test_zipimport.CompressedZipImportTestCase.testNamespacePackageExplicitDirectories) ... ERROR testNamespacePackageImplicitDirectories (test.test_zipimport.CompressedZipImportTestCase.testNamespacePackageImplicitDirectories) ... ERROR testNamespacePackageExplicitDirectories (test.test_zipimport.UncompressedZipImportTestCase.testNamespacePackageExplicitDirectories) ... ERROR testNamespacePackageImplicitDirectories (test.test_zipimport.UncompressedZipImportTestCase.testNamespacePackageImplicitDirectories) ... ERROR ====================================================================== ERROR: testNamespacePackageExplicitDirectories (test.test_zipimport.CompressedZipImportTestCase.testNamespacePackageExplicitDirectories) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/dierus01/work/ce-sw/repos/cpython/Lib/test/test_zipimport.py", line 352, in testNamespacePackageExplicitDirectories self._testPackage(initfile=None) ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^ File "/home/dierus01/work/ce-sw/repos/cpython/Lib/test/test_zipimport.py", line 377, in _testPackage mod = importlib.import_module(f'a.b') File "/home/dierus01/work/ce-sw/repos/cpython/Lib/importlib/__init__.py", line 88, in import_module return _bootstrap._gcd_import(name[level:], package, level) ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "<frozen importlib._bootstrap>", line 1387, in _gcd_import File "<frozen importlib._bootstrap>", line 1360, in _find_and_load File "<frozen importlib._bootstrap>", line 1319, in _find_and_load_unlocked ModuleNotFoundError: No module named 'a.b'; 'a' is not a package ....Did I miss anything?
These tests are passed on my computer and on all buildbots. It seems that something is wrong with your setup. Did you forget to rebuild Python?
I blew the build directory and the error went away. Apologies for the confusion.
Bug report
Bug description:
When on the filesystem packages like discord.py (
pip install discord.py) works just fine (from discord.ext.commands import Bot).However, when you use the same code used to create the python312.zip file to put that package inside of a zip file (the aiohttp dependency would need to be loaded from the user site-packages folder as it contains c extensions files inside of it's package) then
zipimport's_is_dirbugs out and says that it cannot import modulediscord.extbecause it most likely thinks it is a file (it is not as it clearly contains nested directories).The fix:
The simple fix is if the first check returns
false, is to do a recursive subdirectory check and see if it has subdirectories of any nesting levels with python files inside (__init__.py[c], etc) and to returntrueif that is the case.Temporary solution:
Manually create an empty
__init__.pywhen the interpreter sees this sort of issue and contribute it back to the package owner.While it might sound like a good suggestion at first, some maintainers like to randomly block people from their github from filing issues that they identify as a
skill issueso the best fix is the one inzipimportas there is no control over what package authors do, but there are use cases for zipping everything into a single zip file as well (embedding). Also fixingzipimportwould be a simple patch once and done chore as well as opposed to filing an issue with every package that a person could place into a zip file and use.CPython versions tested on:
3.12, 3.13, CPython main branch
Operating systems tested on:
Linux, macOS, Windows
Linked PRs
zipimportincorrectly dealing with namespace packages #121161