Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DIG-1669: Automate incrementing schema version when schemas get updated #235

Merged
merged 7 commits into from
Jun 13, 2024
Merged

DIG-1669: Automate incrementing schema version when schemas get updated #235

merged 7 commits into from
Jun 13, 2024

Conversation

SonQBChau
Copy link

@SonQBChau SonQBChau commented Jun 10, 2024 •
edited
Loading

What's new

  • The update-schema.yaml action now only triggers on pull requests.
  • Removed SHA ref and replaced it with schema_version to avoid circular updates and self-references.
  • schema_version.txt is a standalone file.
  • Any changes to models or schemas will regenerate the schema and increment the schema version.
  • Should not trigger on regular pull requests.

How to test

  • Only code review is needed; this GitHub action only triggers on pull requests, so I will test it out in the next PR.

@codecov Codecov
Copy link

codecov bot commented Jun 10, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 33.33333% with 4 lines in your changes missing coverage. Please review.

Project coverage is 96.99%. Comparing base (385c2b5) to head (dc1f66e).
Report is 2 commits behind head on develop.

❗ Current head dc1f66e differs from pull request most recent head 88f6bd4

Please upload reports for the commit 88f6bd4 to get more accurate results.

Files Patch % Lines
chord_metadata_service/mohpackets/utils.py 20.00% 4 Missing ⚠️
Additional details and impacted files 
@@             Coverage Diff             @@
##           develop     #235      +/-   ##
===========================================
+ Coverage    96.72%   96.99%   +0.27%     
===========================================
  Files           35       35              
  Lines         3569     3557      -12     
===========================================
- Hits          3452     3450       -2     
+ Misses         117      107      -10

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@mshadbolt mshadbolt changed the title Sonchau/schema action DIG-1669: Automate schema incrementing when schemas get updated Jun 10, 2024
@mshadbolt mshadbolt changed the title DIG-1669: Automate schema incrementing when schemas get updated DIG-1669: Automate incrementing schema version when schemas get updated Jun 10, 2024
@SonQBChau SonQBChau marked this pull request as ready for review June 10, 2024 20:44
@SonQBChau SonQBChau requested a review from daisieh June 10, 2024 20:44
@SonQBChau
Copy link
Author

The number in schema_version.txt is just a random number. It could be 1, but I put 440 because katsu version is 4.4.0.

daisieh
daisieh reviewed Jun 12, 2024
View reviewed changes
Copy link
Member

@daisieh daisieh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like the schema version value to be meaningful in some way. Could it be the commit sha of schema.json or schema.yml? Once those have been updated, their shas should stay consistent if we're not overwriting them in a github action, right?

.github/workflows/update-schema.yaml Outdated
- name: Install PyYAML
run: pip install PyYAML

- name: Convert schema.json to schema.yml
run: python -c 'import json, yaml; json.load(open("chord_metadata_service/mohpackets/docs/schema.json")); print(yaml.dump(json.load(open("chord_metadata_service/mohpackets/docs/schema.json"))))' > chord_metadata_service/mohpackets/docs/schema.yml

- name: Update schemas with SHA
- name: Increment schema version
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason we can't use the SHA of the updated schema.yml (or schema.json, whichever) document as the schema version?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem is the current branch SHA will get deleted once it merges. Sure I can put the SHA in, but if someone were to go back and find it, they won't be able to.

Copy link
Author

@SonQBChau SonQBChau Jun 12, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unless we want to keep all the branchs PR around

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not the branch SHA, the file sha. That stays around.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For example, in this deleted branch https://github.com/CanDIG/htsget_app/blob/9d00ade5811bf48727b7b73516c30d06796d5f81/tests/test_htsget_server.py and in the stable branch it was merged into https://github.com/CanDIG/htsget_app/blob/stable/tests/test_htsget_server.py, the sha for test_htsget_server.py is 6429207.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I can try that

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe that's not the file sha, I mean the commit sha.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

6429207 is the sha for the commit, whether it is in the deleted merged PR branch or in the stable branch it was merged into. So if a change was made to schema.json, which is triggering this schema version update, there should be a corresponding commit sha for that.

@SonQBChau SonQBChau marked this pull request as draft June 12, 2024 20:36
@SonQBChau
Copy link
Author

schema_version now uses commit version instead of auto-increasing number

@SonQBChau SonQBChau marked this pull request as ready for review June 12, 2024 22:21
daisieh
daisieh approved these changes Jun 13, 2024
View reviewed changes
Copy link
Member

@daisieh daisieh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You probably don't want to force the schema_version to an int in get_schema_version(); otherwise this looks good.

chord_metadata_service/mohpackets/utils.py Outdated Show resolved Hide resolved
chord_metadata_service/mohpackets/utils.py Outdated Show resolved Hide resolved
@SonQBChau SonQBChau merged commit 8a0136d into develop Jun 13, 2024
5 checks passed
@SonQBChau SonQBChau deleted the sonchau/schema_action branch June 13, 2024 15:57
@candigbot candigbot mentioned this pull request Jun 13, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@daisieh daisieh daisieh approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@SonQBChau @daisieh