Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement pre-commit. #6

Merged
merged 4 commits into from
Apr 6, 2024
Merged

Implement pre-commit. #6

merged 4 commits into from
Apr 6, 2024

Conversation

owenlittlejohns
Copy link
Member

Description

This PR begins the process of applying automated checking to all commits via pre-commit hooks. Currently it checks a small number of things:

  • Trailing whitespace
  • Blank lines at the end of the file.
  • Valid JSON and YAML syntax.
  • ruff linting.

Why haven't I added black or mypy? So far all these checks have left the service code untouched. I think we should totally add both mypy and black, but just wanted to get the ball rolling here. I've largely been inspired by @flamingbear wanting things like this for a while, and by the xarray implementation of this.

Jira Issue ID

N/A - this is an IP-ish bit of fun.

Local Test Steps

  • Pull this branch.
  • Follow the new instructions in the README, including the optional pre-commit run --all-files.
  • To really test this, make some minor change, use git add then git commit. You should see the checks all run when you are trying to use git commit.

PR Acceptance Checklist

  • Jira ticket acceptance criteria met.
  • CHANGELOG.md updated to include high level summary of PR changes.
  • VERSION updated if publishing a release.
  • Tests added/updated and passing. (Well, the hooks are all passing, but that's not quite the same thing)
  • Documentation updated (if needed).

@@ -16,7 +16,7 @@ ENV PYTHONDONTWRITEBYTECODE=1
COPY tests/pip_test_requirements.txt .
RUN conda run --name hoss pip install --no-input -r pip_test_requirements.txt

# Copy test directory containing Python unittest suite, test data and utilities
# Copy test directory containing Python unittest suite, test data and utilities
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Woop - the trailing whitespace check picked this up and automatically fixed it (note - that fix still needs to be added/committed, so we still get to review the magical changes)

</Dataset>
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

...and this one was from the blank line check. Good evidence that one is working, too!

@@ -7,7 +7,6 @@

from harmony.util import config
from harmony.message import Message
from pathlib import PurePosixPath
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

...and lastly this one was found by ruff (an unused import).

The really good news: with our other testing (like pycodestyle) the service code was already in good shape for the ruff checks.

@flamingbear
Copy link
Member

❤️! I was going to do this, but didn't get to it. You should be able to add black (or darker) and it won't change any existing code, only new changes.

@@ -11,6 +11,6 @@ A short description of the changes in this PR.
## PR Acceptance Checklist
* [ ] Jira ticket acceptance criteria met.
* [ ] `CHANGELOG.md` updated to include high level summary of PR changes.
* [ ] `VERSION` updated if publishing a release.
* [ ] `docker/service_version.txt` updated if publishing a release.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sudha-murthy spotted this in her PR for DAS-2106.

@owenlittlejohns
Copy link
Member Author

@flamingbear - Yeah, that's true about black; I just added it. We're committed now!!! Sorry for taking so long to get this implemented.

I still have avoided mypy, because it descends into loads of modules when it's checking. That's a future quest.

Be on the look-out for similar PRs in a couple of other repos. I've got the yen for doing this everywhere now.

- repo: https://github.com/psf/black-pre-commit-mirror
rev: 24.3.0
hooks:
- id: black-jupyter
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a new thing since v21.something of black. The name is a little misleading - it adds checking of Jupyter notebooks, alongside existing functionality. It won't just check Jupyter notebooks.

Copy link
Collaborator

@lyonthefrog lyonthefrog left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran the above test, and I believe it ran as expected. I see that:

  • When I run pre-commit run --all-files, it automatically makes all the required format changes for me to manually git add.
  • When I make a commit, it appears to only check the files I've changed and doesn't automatically change them when they fail.

I think I was right next to this conversation between you and @flamingbear a couple days ago, but why don't we just go through and do a quick sweep of all the files in each repo to go ahead and get all our code up to par? Was there some concern about git history?

tests/pip_test_requirements.txt Show resolved Hide resolved
@owenlittlejohns
Copy link
Member Author

I think I was right next to this conversation between you and @flamingbear a couple days ago, but why don't we just go through and do a quick sweep of all the files in each repo to go ahead and get all our code up to par? Was there some concern about git history?

@lyonthefrog - good question. My main hesitation was that it would be a bunch of changes all at once and maybe difficult to review, where as updating as we go would clean things up in a digestible way. That said, @flamingbear won me over with the ability to add commits to ignore in the blame, so I am doing exactly this. (I did a bunch of work on it yesterday, but without internet, so it hasn't been fully tested/pushed yet). Here's the updated plan.

@lyonthefrog lyonthefrog self-requested a review April 5, 2024 17:02
@owenlittlejohns
Copy link
Member Author

https://giphy.com/gifs/reactionseditor-3oKIPf3C7HqqYBVcCk

W504 to allow for breaking of some long lines. PEP8 suggests
breaking the line before a binary operatore is more "Pythonic".
* E203, E701: This repository uses black code formatting, which deviates
from PEP8 for these errors.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

@flamingbear flamingbear left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀 Love this. Checked out great.

@owenlittlejohns owenlittlejohns merged commit e468c0a into main Apr 6, 2024
3 checks passed
@owenlittlejohns owenlittlejohns deleted the implement-pre-commit branch April 6, 2024 16:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants