Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

uv pip install under certain circumstances generates long paths (\\?\) on Windows under Python 3.13t when it does not do so under other Python versions #6948

Closed
burgholzer opened this issue Sep 2, 2024 · 9 comments
Assignees
Labels
bug Something isn't working windows Specific to the Windows platform

Comments

@burgholzer
Copy link

When run under free-threading Python 3.13 on Windows, uv under certain circumstances emits long paths (prefixed with \\?\) which seem to trip up certain compilers (such as MSCV).
A lot more context is available in pypa/cibuildwheel#1975

This issue is limited to Python 3.13t. It does not happen on any regular Python version (including 3.13).
It is also purely limited to Windows and works on all other operating systems.
CI runs in pypa/cibuildwheel#1975 have also shown that this is limited to uv (as early as 0.2.35, but also tested just now with 0.4.2) and does not happen when using plain old pip.

An example of a 3.13t run producing a long path can e.g. be found here: https://github.com/cda-tum/mqt-qcec/actions/runs/10634832174/job/29568141103?pr=432#step:7:1526 (look at the PYTHON_EXECUTABLE, Python_ROOT_DIR and the related variants in the log)
It's a bit harder to show the output for a successful run as up until #6903 was merged, uv swallows the build system output.

It also seems to be a combination of a couple of specific things that lead to uv emitting long paths here.
The CI for our dependency project (https://github.com/cda-tum/mqt-core/tree/shared-libs) succeeds without problems on 3.13t: https://github.com/cda-tum/mqt-core/actions/runs/10634400958/job/29481632914#step:8:5742
Only once the depending project (https://github.com/cda-tum/mqt-qcec/tree/use-mqt-core-package) declares the git dependency on mqt-core in its build time requirements, the issues start to come up when trying to build mqt-qcec.

Maybe the root cause somewhat has to do with the double build isolation going on here as the mqt-qcec build is built in an isolated environment, but then the mqt-core git dependency is being built in another isolated environment (supposedly spawned from the first isolated env).

A way to reproduce the failures and see the long paths (adapted from https://github.com/cda-tum/mqt-qcec/blob/a15710a4faec2ccd62f28d6575800a8632a6ec2a/.github/workflows/cd.yml#L88-L107C14):

git clone https://github.com/cda-tum/mqt-qcec
cd mqt-qcec
git switch use-mqt-core-package
nuget.exe install python-freethreaded -Version 3.13.0-rc1 -FallbackSource https://api.nuget.org/v3/index.json -OutputDirectory '<wherever you want this>'
uv venv --python "<wherever you want this>\python-freethreaded.3.13.0-rc1\tools\python.exe"
uv pip install -vvv .

Sorry that I couldn't get the example any smaller or more isolated. I really tried but could not fabricate a simpler scenario that triggers the issues.

@charliermarsh
Copy link
Member

Thanks! Am I correct that this call is ultimately failing?

DEBUG Calling `scikit_build_core.build.build_wheel("\\\\?\\C:\\Users\\runneradmin\\AppData\\Local\\uv\\cache\\built-wheels-v3\\git\\c4f0338be623b94c\\e17ea8269bf6b83a\\.tmpiVR21p", {}, None)`

I think we should avoid introducing these in that context. I'll take a look.

@charliermarsh
Copy link
Member

I admittedly have no idea why this would affect the free-threaded Pythons, but not Python 3.13 without free-threading.

@burgholzer
Copy link
Author

Thanks! Am I correct that this call is ultimately failing?

DEBUG Calling `scikit_build_core.build.build_wheel("\\\\?\\C:\\Users\\runneradmin\\AppData\\Local\\uv\\cache\\built-wheels-v3\\git\\c4f0338be623b94c\\e17ea8269bf6b83a\\.tmpiVR21p", {}, None)`

I think we should avoid introducing these in that context. I'll take a look.

Yeah. At least that's what I read from the logs. Everything after that in https://github.com/cda-tum/mqt-qcec/actions/runs/10634832174/job/29568141103?pr=432#step:7:1526 is the error output.

Just saw that 0.4.3 was released. So I should be able to get you a "successful" build log on a different Python version as well now.

@burgholzer
Copy link
Author

Alright, new runs with 0.4.3 here: https://github.com/cda-tum/mqt-qcec/actions/runs/10673641135?pr=432

Specifically, compare:

@charliermarsh
Copy link
Member

I got a different failure here: https://github.com/astral-sh/uv/actions/runs/10673961604/job/29583752151?pr=6949. Does that look... correct?

@burgholzer
Copy link
Author

At least it seems to be able to build the mqt-core build time dependency as well as the mqt-qcec package itself in that log.
It seems to me that it then fails when trying to install numpy from source (which is a regular install dependency of mqt-qcec). That seems to be a known issue on the numpy side according to their tracking issue for 3.13t support (numpy/numpy#26157)

I believe if you install with --no-deps the install of mqt-qcec should succeed.

@charliermarsh
Copy link
Member

Ok cool that actually worked. (A subsequent test failed, unrelated.) I'll merge that change and link it to this issue.

charliermarsh added a commit that referenced this issue Sep 3, 2024
@charliermarsh
Copy link
Member

Should be fixed in the next release via #6949.

@charliermarsh charliermarsh self-assigned this Sep 3, 2024
@charliermarsh charliermarsh added bug Something isn't working windows Specific to the Windows platform labels Sep 3, 2024
@burgholzer
Copy link
Author

Nice 🎉 thanks for looking into this so quickly! Love what you and your team are building! 🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working windows Specific to the Windows platform
Projects
None yet
Development

No branches or pull requests

2 participants