-
Notifications
You must be signed in to change notification settings - Fork 12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Migrate sssom validation to new LinkML validation framework #532
Conversation
@pkalita-lbl Thanks a ton for your help, I made issues for everything and will resume work here once they are resolved. Thank you! |
Using the latest version of LinkML to get latest validator features.
In this commit we 1. Add a new parameter, fail_on_error, to all validation methods (if true, throws and exception when errors are encountered) 2. We generate ValidationResult instances fall all custom checks (reusing the LinkML validation framework) 3. Add a few missing docstrings 4. Add a new validation check, check_strict_curie_format, which looks for curies with a | character in it
Previously I used dataclasses. asdict to turn a dataclass instance to a dictionary to pass to the validation framework. @pkalita-lbl explained to me why this was not a good idea (essentially, dataclass instances dont look like what the validation framework expects), and recommended the use of linkml_runtime.dumpers.json_dumper instead. Tests work fine!
.. to ensure that it recognises the argument fail_on_error.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The validation-related changes look good to me!
@hrshdhgd I assigned this to you for review. I think it is done, but I am a bit scared of how the changes will impact people relying on validation downstream, so I prefer to have your eyes on it. Thank you! Ideally this should be a careful review beyond "lgtm" as I feel I cant afford messing this up! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So the idea here is if validation encounters a SSSOM tsv file does not contain (say) a valid column value, the row is dropped and warning is logged. Correct?
It will never error out by the presence of invalid elements?
@hrshdhgd just to be sure I need a positive approval from you for this PR, and real certainty that all is understood and correct. Can you comment on the exact pieces of code you have questions about? |
That is exactly what I'm trying to do as per my previous comment. |
I hope this PR is only affecting the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, it seems like parse
would have no impact with this change of code.
Blocked by:
Note to future self, only thing that stands in the way of this PR now is #2164, if that is dealt with
print_linkml_report(report, False)
to print_linkml_report(report, fail_on_error)