Setuptools symlinks#20092
Open
tomeuv wants to merge 1 commit into
Open
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/20092
Note: Links to docs will display an error until the docs builds have been completed. This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
|
|
The following ciflow label(s) have been added but CI has not been triggered yet because the workflows are awaiting approval:
Once a maintainer approves the workflows (scroll to the bottom of the PR page), the corresponding CI jobs will be triggered automatically. Please ping one of the reviewers if you do not have access to approve and run workflows. |
This PR needs a
|
Contributor
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
This PR updates release guidance, dependency installation behavior, and CI workflow references, while addressing packaging issues caused by directory symlinks in manifests.
Changes:
- Update documentation and contributor guidance to reference
release/1.3/1.3.1instead ofviable/strict. - Adjust packaging/installation tooling (manifest filtering, pip
--no-cache-dir, model/artifact download robustness). - Migrate many GitHub Actions workflows from
pytorch/test-infra@mainto@release/2.12.
Reviewed changes
Copilot reviewed 45 out of 45 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| version.txt | Updates the project version string. |
| setup.py | Filters manifest entries for non-editable builds to avoid copying directory symlinks. |
| requirements-examples.txt | Adds torchaudio/torchvision requirements for examples. |
| pyproject.toml | Adjusts dependency specs and removes cortex_m optional extra from published metadata. |
| install_requirements.py | Adds --no-cache-dir to pip installs for dev + optional example requirements. |
| extension/llm/tokenizers | Updates submodule commit pointer. |
| extension/benchmark/apple/Benchmark/README.md | Updates clone instructions to release/1.3. |
| extension/android/executorch_android/android_test_setup.sh | Improves download robustness and makes golden artifacts URL discoverable/configurable. |
| examples/models/parakeet/tokenizer_utils.cpp | Replaces UTF-8 literal with byte-escaped constant for C++20 compatibility. |
| examples/models/moshi/mimi/install_requirements.sh | Updates torchcodec install source/version. |
| docs/source/using-executorch-ios.md | Updates SwiftPM docs and clone instructions for 1.3.1/release/1.3. |
| docs/source/using-executorch-building-from-source.md | Updates clone instructions to release/1.3. |
| docs/source/raspberry_pi_llama_tutorial.md | Updates clone instructions to release/1.3 and fixes formatting. |
| docs/source/getting-started.md | Updates iOS SwiftPM example and clone instructions to release/1.3. |
| docs/README.md | Updates clone instructions to release/1.3. |
| backends/cortex_m/requirements-cortex-m.txt | Clarifies why direct URL deps are kept out of pyproject metadata. |
| backends/cortex_m/passes/init.py | Updates missing-deps installation guidance for Cortex-M backend. |
| backends/arm/scripts/install_models_for_test.sh | Pins model-gym clone to a tag and shallow clones. |
| backends/arm/requirements-arm-models-test.txt | Pins ng_model_gym dependencies since it’s installed with --no-deps. |
| CONTRIBUTING.md | Updates guidance to rebase onto release/1.3 instead of viable/strict. |
| .github/workflows/windows-msvc.yml | Pins reusable workflow ref to pytorch/test-infra@release/2.12. |
| .github/workflows/update-viablestrict.yml | Pins update action to pytorch/test-infra@release/2.12. |
| .github/workflows/test-pico2-build.yml | Pins reusable workflows to pytorch/test-infra@release/2.12. |
| .github/workflows/periodic.yml | Pins reusable workflow to pytorch/test-infra@release/2.12. |
| .github/workflows/nightly.yml | Pins actions/workflows to pytorch/test-infra@release/2.12. |
| .github/workflows/docker-builds.yml | Pins actions to pytorch/test-infra@release/2.12. |
| .github/workflows/doc-build.yml | Pins reusable workflows to pytorch/test-infra@release/2.12. |
| .github/workflows/cuda.yml | Pins reusable workflows to pytorch/test-infra@release/2.12. |
| .github/workflows/cuda-windows.yml | Pins reusable workflows to pytorch/test-infra@release/2.12. |
| .github/workflows/cuda-perf.yml | Pins reusable workflows/actions to pytorch/test-infra@release/2.12. |
| .github/workflows/claude-code.yml | Pins reusable workflow to pytorch/test-infra@release/2.12. |
| .github/workflows/build-wheels-windows.yml | Pins reusable workflows to pytorch/test-infra@release/2.12. |
| .github/workflows/build-wheels-macos.yml | Pins reusable workflows to pytorch/test-infra@release/2.12. |
| .github/workflows/build-wheels-linux.yml | Pins reusable workflows to pytorch/test-infra@release/2.12. |
| .github/workflows/build-wheels-aarch64-linux.yml | Pins reusable workflows to pytorch/test-infra@release/2.12. |
| .github/workflows/build-cadence-runner.yml | Pins reusable workflow to pytorch/test-infra@release/2.12. |
| .github/workflows/apple.yml | Increases iOS device farm timeout. |
| .github/workflows/android-release-artifacts.yml | Pins reusable workflow to pytorch/test-infra@release/2.12. |
| .github/workflows/_test_riscv.yml | Pins reusable workflow to pytorch/test-infra@release/2.12. |
| .github/workflows/_test_cadence.yml | Pins reusable workflows to pytorch/test-infra@release/2.12. |
| .github/workflows/_link_check.yml | Switches to full-depth checkout and removes manual base-ref fetch. |
| .github/workflows/_android.yml | Pins setup action to pytorch/test-infra@release/2.12. |
| .ci/scripts/unittest-macos-cmake.sh | Scopes TorchInductor cache to the job to avoid stale PCH issues. |
| .ci/scripts/test_wheel_package_qnn.sh | Changes torch install index and disables pip cache. |
| .ci/scripts/test_model_e2e.sh | Updates torchcodec install source/version. |
Comments suppressed due to low confidence (5)
pyproject.toml:1
torch>=2.12.0a0andtorchao>=0.17.0are likely non-existent/out-of-range versions for many environments (anda0also implies a nightly/alpha). This can break dependency resolution for end users and published package metadata; consider setting a stable minimum supported PyTorch version (non-alpha) and/or moving “nightly-only” requirements behind an optional extra or a separate constraints/requirements file.
requirements-examples.txt:1- The minimum versions
torchaudio >= 2.11.0andtorchvision >= 0.27.0appear to be far ahead of commonly available releases and may not exist for most platforms/Python versions, which would make the examples requirements unsatisfiable. Consider using known-released minimums aligned with the supported Torch version for this repo, or express compatibility via atorch-aligned range (e.g., matching major/minor pairs).
examples/models/moshi/mimi/install_requirements.sh:1 --index-urloverrides PyPI, which can break installs if torchcodec (or its dependencies) aren’t fully mirrored on the PyTorch index for the user’s platform/Python. Consider using--extra-index-url https://download.pytorch.org/whl/cpuso PyPI remains available, or constrain this to environments where you know the wheel is hosted on that index.
docs/source/using-executorch-ios.md:1- The documented nightly branch format uses a dot separator (
swiftpm-<version>.<year_month_date>), but the example uses a hyphen (swiftpm-1.4.0-20260601). Update either the format text or the example so they match, to prevent users from copying an invalid branch pattern.
setup.py:1 - This currently drops any manifest entry that isn’t an
isfile()at analysis time, which is broader than “bare directory symlinks” and could mask unexpected packaging issues by silently removing entries. Consider narrowing the filter to the specific problematic case (symlink-to-directory), and optionally emitting a warning/log for removed entries so missing package data is easier to diagnose.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
12
to
15
| f"(missing: {missing}). Install them with " | ||
| "`pip install --no-dependencies -r " | ||
| "backends/cortex_m/requirements-cortex-m.txt`, or run " | ||
| "`examples/arm/setup.sh --i-agree-to-the-contained-eula`." |
| # Install torch based on the pinned PyTorch version, preferring the PyTorch test index | ||
| "$PIPBIN" install torch=="${TORCH_VERSION}" --extra-index-url "https://download.pytorch.org/whl/test" | ||
| # Install torch based on the pinned PyTorch version. | ||
| "$PIPBIN" install --no-cache-dir torch=="${TORCH_VERSION}" --index-url "https://download.pytorch.org/whl/cpu" |
| steps: | ||
| - uses: actions/checkout@v4 | ||
| with: | ||
| fetch-depth: 0 |
…le mode Recent setuptools includes bare directory symlinks (e.g. src/executorch/backends -> ../../backends) from version control in manifest_files. These exist for editable mode but break regular installs: build_package_data passes them to copy_file, which calls os.path.isfile() and gets False for a symlink-to-directory. Override analyze_manifest() to filter out non-regular-file entries after the parent populates manifest_files, guarded by editable_mode.
1edf104 to
6fbf6d4
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #20091
Test plan
Run the command in the bug report with the problematic Python version as reported.
cc @digantdesai @freddan80 @per @zingo @oscarandersson8218 @mansnils @Sebastian-Larsson @robell @rascani