-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Miljoeportal #302
base: main
Are you sure you want to change the base?
Miljoeportal #302
Conversation
…processing a list of file paths.
…s dsk crawled data.
…case of worker termination errors.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code generally looks reasonable.
Though it is hard to review without the data.
However I believe we are missing metadata on these datasets?
To pull data run: | ||
|
||
```bash | ||
$ uv run data-processing/scripts/miljoeportal/download.py $PATH_TO_SAVE_RAW_DATA $PATH_TO_SAVE_EXTRACTED_DATA4 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
$ uv run data-processing/scripts/miljoeportal/download.py $PATH_TO_SAVE_RAW_DATA $PATH_TO_SAVE_EXTRACTED_DATA4 | |
$ python data-processing/scripts/miljoeportal/download.py $PATH_TO_SAVE_RAW_DATA $PATH_TO_SAVE_EXTRACTED_DATA4 |
I think we should be consistent throughout (I am perfectly fine with consistently using uv
Returns: | ||
bool: If the pull was succesful or not. | ||
""" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Returns: | |
bool: If the pull was succesful or not. | |
""" | |
Returns: | |
If the pull was succesful or not. | |
""" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(generally if you already have the type hint I would not add it in docs
Returns: | ||
list[str]: _description_ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Returns: | |
list[str]: _description_ |
|
||
Args: | ||
file_path (Path): Path to where to save raw files | ||
id_ (str): Assess |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
?
Adding the Danmarks Miljøportal data. #301
Merging this will also close #299 and #298