Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

utils/pypi: specify dependencies needed to update resources #16772

Merged
merged 2 commits into from
Mar 4, 2024

Conversation

cho-m
Copy link
Member

@cho-m cho-m commented Feb 29, 2024

  • Have you followed the guidelines in our Contributing document?
  • Have you checked to ensure there aren't other open Pull Requests for the same change?
  • Have you added an explanation of what your changes do and why you'd like us to include them?
  • Have you written new tests for your changes? Here's an example.
  • Have you successfully run brew style with your changes locally?
  • Have you successfully run brew typecheck with your changes locally?
  • Have you successfully run brew tests with your changes locally?

Some formulae currently fail brew update-python-resources (and thus brew [bump|bump-formula-pr]) since some Python packages end up looking for system libs, various commands, and so on.

This PR provides initial feature support to specify these extra formulae dependencies. May need some follow up PRs to handle other scenarios (e.g. PATH, maybe build env similar to superenv, etc)

Users may not want arbitrary formulae getting installed on their system, so error message is default. We can enable this on bot for autobumps.


Example for certbot current behavior:

brew update-python-resources certbot --verbose
==> Retrieving PyPI dependencies for "certbot==2.9.0 certbot-apache certbot-nginx"...
  error: subprocess-exited-with-error

  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [48 lines of output]
      Traceback (most recent call last):
...
      OSError: ctypes.util.find_library() did not manage to locate a library called 'augeas'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

Adding this change and diff:

diff --git a/pypi_formula_mappings.json b/pypi_formula_mappings.json
index 09643275510..b82a7e353d0 100644
--- a/pypi_formula_mappings.json
+++ b/pypi_formula_mappings.json
@@ -112,6 +112,7 @@
     "exclude_packages": ["certifi"]
   },
   "certbot": {
+    "dependencies": ["augeas"],
     "extra_packages": ["certbot-apache", "certbot-nginx"],
     "exclude_packages": ["certifi", "cryptography"]
   },

We get:

brew update-python-resources certbot --verbose
Error: Missing formulae needed to update "certbot" resources: augeas

And then opt-ing into autoinstallation of dependencies:

brew update-python-resources certbot --verbose --install-dependencies
==> Installing `augeas`...
/opt/homebrew/bin/brew install --formula augeas
==> Downloading https://ghcr.io/v2/homebrew/core/augeas/manifests/1.14.1-1
Already downloaded: /Users/cho-m/Library/Caches/Homebrew/downloads/338b8e0bd10e0c97d2f41e197eb4f04ceb79e8449df0331ebe7e8a4925ce4111--augeas-1.14.1-1.bottle_manifest.json
==> Fetching augeas
==> Downloading https://ghcr.io/v2/homebrew/core/augeas/blobs/sha256:72892294927f45da15836ea628404d5ea93597344d93dfe1ba3889dc9c1daf68
Already downloaded: /Users/cho-m/Library/Caches/Homebrew/downloads/3579d890e27d4d805fd2549df841457a4645eb3432c0a099152a6fa0ab6739c3--augeas--1.14.1.arm64_sonoma.bottle.1.tar.gz
==> Pouring augeas--1.14.1.arm64_sonoma.bottle.1.tar.gz
==> Caveats
Lenses have been installed to:
  /opt/homebrew/share/augeas/lenses/dist
==> Summary
🍺  /opt/homebrew/Cellar/augeas/1.14.1: 492 files, 4MB
==> Running `brew cleanup augeas`...
Disable this behaviour by setting HOMEBREW_NO_INSTALL_CLEANUP.
Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`).
==> Retrieving PyPI dependencies for "certbot==2.9.0 certbot-apache certbot-nginx"...
==> Retrieving PyPI dependencies for excluded "certifi cryptography"...
==> Getting PyPI info for "acme==2.9.0"
...

Copy link
Member

@MikeMcQuaid MikeMcQuaid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks again @cho-m!

Library/Homebrew/utils/pypi.rb Outdated Show resolved Hide resolved
Library/Homebrew/utils/pypi.rb Outdated Show resolved Hide resolved
cho-m added 2 commits March 2, 2024 22:01
Also allow optionally installing these dependencies. By default, only
`[email protected]` formulae will be automatically installed.

Signed-off-by: Michael Cho <[email protected]>
Also same feature for update-python-resources
@cho-m cho-m force-pushed the pypi-specify-dependencies branch from 35acaae to 216f571 Compare March 3, 2024 03:02
Copy link
Member

@MikeMcQuaid MikeMcQuaid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work as usual @cho-m!

@MikeMcQuaid MikeMcQuaid merged commit 8d463b1 into Homebrew:master Mar 4, 2024
27 checks passed
@cho-m cho-m deleted the pypi-specify-dependencies branch March 5, 2024 02:05
@github-actions github-actions bot added the outdated PR was locked due to age label Apr 5, 2024
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Apr 5, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
outdated PR was locked due to age
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants