Updates for Pyodide builds after pyodide-build was unvendored
#2090
Conversation
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
| pyodide_root = ( | ||
| CIBW_CACHE_PATH | ||
| / f".pyodide-xbuildenv-{pyodide_version}/{pyodide_version}/xbuildenv/pyodide-root" | ||
| / f".pyodide-xbuildenv-{pyodide_build_version}/{pyodide_version}/xbuildenv/pyodide-root" |
There was a problem hiding this comment.
This will not respect overrides (like latest or pip installing a new version), but will always be the one hard-coded. Was that intentional?
There was a problem hiding this comment.
Yes, this is intentional, since we don't have any ways to override here (no environment variables or the like, etc.). We should be fine with the hardcoded version. We know that users can still override pyodide-build using their constraint files if they like, but that won't be of much use (unless there's a bug in pyodide-build or if they want to use a fork or for any other reasons to use a constraint). I plan to rebase #2002 after this gets reviewed/merged.
There was a problem hiding this comment.
Ah, just saw your edit. How do we ensure that we use the version of pyodide-build that users install themselves, e.g., in before-build?
There was a problem hiding this comment.
Constraint flags apply when it is setting up the environment. If a user installs it themselves, that will happen after the constrained version has already set up a new environment. Do we need the pyodide-build version in the env path?
There was a problem hiding this comment.
I am not sure if I understood that correctly 😅 If I got it right,
the constrained version has already set up a new environment
Is "environment" here the cibuildwheel venv that will be used for the build? If these changes constrain the pyodide-build version in this environment and constraints files will have no effect, I'd definitely like to try out a different method. Could you give me a pointer on how I could proceed?
Also, a side note – after thinking a bit more about this, I think it would be great not to hardcode the Pyodide version, too (not in this PR, though). In case we release a new version of pyodide-build, say, X, that drops compatibility with the Pyodide version Y listed in build-platforms.toml, overriding the version here with constraints to use X would make it unable to work with Pyodide version Y.
There was a problem hiding this comment.
I mean the xbuildenv gets set up with the constrained versions, but before-build runs after the xbuildenv is setup, so it can't affect the version used to setup the xbuildenv. After that I believe you can change it, so it's only the pyodide-build version used for the xbuildenv setup that's a little harder to control.
|
Thanks for the review and merge! I'm sorry I didn't see this sooner, and more notifications got in the way later. I think we can think of more solutions as part of #2002 in case users want more flexible constraints. But most of the discussion seems to point to the fact that |
Description
This PR has been split off from #2002, where it has been attempted to allow the use of a build-time flag to build and test a Pyodide version that is not available with cibuildwheel, since the cibuildwheel version couples the Pyodide version available within it.
This PR is, rather, more limited in its approach. The key changes here are:
pyodide-buildis now hosted in its own repository, and updates to it are not made in sync with updates to the Pyodide version, allowingpyodide-buildto move faster with changes to it. Hence, thepyodide-buildversion (currently at 0.29.0) is now what is used to fetch the Pyodide xbuildenv for Pyodide builds.0.26.4, which now uses Python 3.12.7 (but we reflect just 3.12 and don't reflect the patch version). The Emscripten version stays the same at 3.1.58 and there is no ABI break.