Skip to content

make test doesn't work w/ a WASI cross-build via (at least) Tools/wasm/wasi.py #114013

Closed

Description

@brettcannon

Bug report

Bug description:

For some reason, make test is appending python.wasm to the command to run even though python.wasm was specified as part of the host runner.

`make test` output
❯ make -C cross-build/wasm32-wasi/ test
make: Entering directory '/home/brettcannon/Repositories/python/cpython/cross-build/wasm32-wasi'
The following modules are *disabled* in configure script:
_sqlite3

The necessary bits to build these optional modules were not found:
_bz2                  _ctypes               _hashlib
_lzma                 _ssl                  _testimportmultiple
_testmultiphase       _uuid                 readline
xxlimited             xxlimited_35          zlib
To find the necessary bits, look in configure.ac and config.log.

Could not build the ssl module!
Python requires a OpenSSL 1.1.1 or newer

Checked 109 modules (76 built-in, 0 shared, 20 n/a on wasi-wasm32, 1 disabled, 12 missing, 0 failed on import)
_PYTHON_HOSTRUNNER='/home/linuxbrew/.linuxbrew/bin/wasmtime run --wasm max-wasm-stack=8388608 --wasm threads=y --wasi threads=y --dir /home/brettcannon/Repositories/python/cpython::/ --env PYTHONPATH=/cross-build/wasm32-wasi/build/lib.wasi-wasm32-3.13-pydebug /home/brettcannon/Repositories/python/cpython/cross-build/wasm32-wasi/python.wasm' _PYTHON_PROJECT_BASE=/home/brettcannon/Repositories/python/cpython/cross-build/wasm32-wasi _PYTHON_HOST_PLATFORM=wasi-wasm32 PYTHONPATH=/home/brettcannon/Repositories/python/cpython/cross-build/wasm32-wasi/build/lib.wasi-wasm32-3.13-pydebug:../../Lib _PYTHON_SYSCONFIGDATA_NAME=_sysconfigdata_d_wasi_wasm32-wasi /home/brettcannon/Repositories/python/cpython/cross-build/build/python  -m test --fast-ci --timeout=
+ /home/brettcannon/Repositories/python/cpython/cross-build/build/python -u -W default -bb -m test --fast-ci --timeout= --python '/home/linuxbrew/.linuxbrew/bin/wasmtime run --wasm max-wasm-stack=8388608 --wasm threads=y --wasi threads=y --dir /home/brettcannon/Repositories/python/cpython::/ --env PYTHONPATH=/cross-build/wasm32-wasi/build/lib.wasi-wasm32-3.13-pydebug /home/brettcannon/Repositories/python/cpython/cross-build/wasm32-wasi/python.wasm python.wasm' --dont-add-python-opts
== CPython 3.13.0a2+ (heads/main-dirty:3aa4b839e4, Jan 12 2024, 14:07:13) [Clang 17.0.6 ]
== Linux-5.15.133.1-microsoft-standard-WSL2-x86_64-with-glibc2.36 little-endian
== Python build: debug
== cwd: /home/brettcannon/Repositories/python/cpython/cross-build/wasm32-wasi/build/test_python_worker_13486æ
== CPU count: 8
== encodings: locale=UTF-8 FS=utf-8
== resources: all,-cpu
== cross compiled: Yes
== host python: /home/linuxbrew/.linuxbrew/bin/wasmtime run --wasm max-wasm-stack=8388608 --wasm threads=y --wasi threads=y --dir /home/brettcannon/Repositories/python/cpython::/ --env PYTHONPATH=/cross-build/wasm32-wasi/build/lib.wasi-wasm32-3.13-pydebug /home/brettcannon/Repositories/python/cpython/cross-build/wasm32-wasi/python.wasm python.wasm
python.wasm: can't open file '//python.wasm': [Errno 44] No such file or directory
== host platform: <command failed with exit code 2>

And for comparison ...

`./python.sh -m test` in `cross-build/wasm32-wasi`
❯ ./python.sh -m test
== CPython 3.13.0a2+ (heads/main-dirty:3aa4b839e4, Jan 12 2024, 23:19:57) [Clang 16.0.0 ]
== wasi-0.0.0-wasm32-32bit little-endian
== Python build: debug
== cwd: /build/test_python_worker_728488æ
== CPU count: 1
== encodings: locale=UTF-8 FS=utf-8
== resources: all test resources are disabled, use -u option to unskip tests
== host runner: /home/linuxbrew/.linuxbrew/bin/wasmtime run --wasm max-wasm-stack=8388608 --wasm threads=y --wasi threads=y --dir /home/brettcannon/Repositories/python/cpython::/ --env PYTHONPATH=/cross-build/wasm32-wasi/build/lib.wasi-wasm32-3.13-pydebug /home/brettcannon/Repositories/python/cpython/cross-build/wasm32-wasi/python.wasm

CPython versions tested on:

CPython main branch

Operating systems tested on:

Other

Linked PRs

Activity

  1. added
    type-bugAn unexpected behavior, bug, or error
    buildThe build process and cross-build
    on Jan 13, 2024
  2. brettcannon commented on Jan 13, 2024

    @brettcannon
    MemberAuthor

    My build was from doing py Tools/wasm/wasi.py build -- --with-pydebug -C. Maybe specifying the path to python.wasm in HOSTRUNNER when doing the build is wrong? The --python argument is effectively (which is probably also what the "host python" line is printing out).

    Specifically, the last part of setting HOSTRUNNER may be wrong/unnecessary:

    cpython/Tools/wasm/wasi.py

    Lines 215 to 218 in 3aa4b83

    HOST_DIR=CHECKOUT,
    ENV_VAR_NAME="PYTHONPATH",
    ENV_VAR_VALUE=f"/{sysconfig_data}",
    PYTHON_WASM=working_dir / "python.wasm")

  3. changed the title [-]`make test` doesn't work w/ a WASI cross-build[/-] [+]`make test` doesn't work w/ a WASI cross-build via (at least) `Tools/wasm/wasi.py`[/+] on Jan 13, 2024
  4. added a commit that references this issue on Jan 16, 2024
  5. moved this from Todo to In Progress in WASI to tier 2on Jan 16, 2024
  6. self-assigned this
    on Jan 16, 2024
  7. added a commit that references this issue on Jan 16, 2024
  8. moved this from In Progress to Done in WASI to tier 2on Jan 16, 2024
  9. added a commit that references this issue on Jan 22, 2024
  10. added a commit that references this issue on Feb 11, 2024
  11. added a commit that references this issue on Sep 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

OS-wasibuildThe build process and cross-buildtype-bugAn unexpected behavior, bug, or error

Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions