Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tap_auditor: validate pypi_formula_mappings.json #16800

Closed
wants to merge 1 commit into from

Conversation

cho-m
Copy link
Member

@cho-m cho-m commented Mar 3, 2024

  • Have you followed the guidelines in our Contributing document?
  • Have you checked to ensure there aren't other open Pull Requests for the same change?
  • Have you added an explanation of what your changes do and why you'd like us to include them?
  • Have you written new tests for your changes? Here's an example.
  • Have you successfully run brew style with your changes locally?
  • Have you successfully run brew typecheck with your changes locally?
  • Have you successfully run brew tests with your changes locally?

Follow up after #16772 to do schema validation and valid formulae checks.

@Bo98
Copy link
Member

Bo98 commented Mar 3, 2024

Note that json_schemer is a part of the pr_upload gem group so you'll need to add it to audit too. See how rexml is handled in the Gemfile.

@cho-m cho-m force-pushed the validate-pypi_formula_mappings branch 2 times, most recently from 8601aac to 254c900 Compare March 5, 2024 01:50
@cho-m
Copy link
Member Author

cho-m commented Mar 5, 2024

After adding some bad values to JSON, the audit output looks like:

brew audit --tap homebrew/core --only 'json_files,tap_formula_lists'
homebrew/core
  * pypi_formula_mappings.json schema validation failed with following errors:
    * object property at `/aiven-client/foo` is not defined and schema does not allow additional properties
    * value at `/animdl/exclude_packages/1` is not a string
  * pypi_formula_mappings.json references
    formulae or casks that are not found in the homebrew/core tap.
    Invalid formulae or casks: brew
Error: 2 problems in 1 tap detected.

I only created parts of JSON Schema needed to run check, but can choose to fill out the other metadata. I added it to catch things like typos and unwanted empty lists.

Can also consider restricting to homebrew/core if there is any chance external taps may be using JSON in other ways.

@MikeMcQuaid
Copy link
Member

@cho-m love the idea!

@cho-m cho-m force-pushed the validate-pypi_formula_mappings branch from 254c900 to 254355d Compare March 25, 2024 18:08
@cho-m
Copy link
Member Author

cho-m commented Mar 25, 2024

Still fine tuning schema. I plan to update homebrew/core side to align normalized names similar to how we name resources.

"package_name": {
"description": "Normalized PyPI package name with optional extras",
"type": "string",
"pattern": "^[a-z0-9][a-z0-9-]*(\\[[a-z0-9][a-z0-9-]*(,[a-z0-9][a-z0-9-]*)*\\])?$"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a simplification of https://packaging.python.org/en/latest/specifications/name-normalization/. It doesn't handle trailing - or consecutive - as those make the regex messier.

Copy link

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

@github-actions github-actions bot added the stale No recent activity label Apr 16, 2024
@github-actions github-actions bot closed this Apr 23, 2024
@github-actions github-actions bot added the outdated PR was locked due to age label May 23, 2024
@github-actions github-actions bot locked as resolved and limited conversation to collaborators May 23, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
outdated PR was locked due to age stale No recent activity
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants