Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Marin County Scraper #80

Merged
merged 50 commits into from
Sep 3, 2020
Merged

Add Marin County Scraper #80

merged 50 commits into from
Sep 3, 2020

Conversation

kwonangela7
Copy link
Collaborator

@kwonangela7 kwonangela7 commented Jun 10, 2020

  • Please check in-line comments - specifically around the testing data I used.
  • Er, I shouldn't have made any edits to the data_model.json file. I guess I added spaces lol. I'll change that...

I'll work on passing CircleCI tests, and will take another look through my code to see if there's anything else I wanted to point out, but I think for now I've highlighted the most notable comments.

@kwonangela7 kwonangela7 self-assigned this Jun 11, 2020
Copy link
Collaborator

@Mr0grog Mr0grog left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Started this before we talked tonight, so we already mentioned a few things.

Can you also please make sure to use 4-space (not tab) indentation, so this is consistent with the other files? (If your editor has a plugin for editorconfig, you can install that and it should autoconfigure its indentation settings, etc. for this project.)

covid19_sfbayarea/data/marin_scraper.py Outdated Show resolved Hide resolved
covid19_sfbayarea/data/marin_scraper.py Outdated Show resolved Hide resolved
covid19_sfbayarea/data/marin_scraper.py Outdated Show resolved Hide resolved
covid19_sfbayarea/data/marin_scraper.py Outdated Show resolved Hide resolved
covid19_sfbayarea/data/marin_scraper.py Outdated Show resolved Hide resolved
covid19_sfbayarea/data/marin_scraper.py Outdated Show resolved Hide resolved
covid19_sfbayarea/data/marin_scraper.py Outdated Show resolved Hide resolved
covid19_sfbayarea/data/marin_scraper.py Outdated Show resolved Hide resolved
covid19_sfbayarea/data/marin_scraper.py Outdated Show resolved Hide resolved
covid19_sfbayarea/data/marin_scraper.py Outdated Show resolved Hide resolved
@kwonangela7
Copy link
Collaborator Author

kwonangela7 commented Aug 22, 2020

@Mr0grog Hi Rob, whenever you get the chance, would love it if you could review this for the second time!

(edit: I think we're probably good, since Elaine went ahead and reviewed it)

As I mentioned in the comment, currently I'm not able to scrape the test csv data so I've just commented that line out for now.

Edit: as mentioned below, I'll just be able to scrape the time series for positive tests + cumulative number of positive tests. I guess Marin County might have removed the count of negative tests altogether?

Copy link
Collaborator

@elaguerta elaguerta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's broken right now bc Marin has deleted 'inmates' from the website. I didn't flag every instance that "inmates" is used. Code looks good to me!

covid19_sfbayarea/data/marin.py Outdated Show resolved Hide resolved
covid19_sfbayarea/data/marin.py Outdated Show resolved Hide resolved
covid19_sfbayarea/data/marin.py Show resolved Hide resolved
covid19_sfbayarea/data/marin.py Outdated Show resolved Hide resolved
data_models/README.md Show resolved Hide resolved
data_models/README.md Show resolved Hide resolved
@kwonangela7
Copy link
Collaborator Author

kwonangela7 commented Aug 29, 2020

Thank you @elaguerta and @ldtcooper for your comments! @elaguerta unfortunately the original negative/positive breakdown for tests is gone, but I'll go ahead and populate the date/positive/cumul_pos since that's what I can scrape from the graph you linked. Thanks for the update!

@kwonangela7
Copy link
Collaborator Author

kwonangela7 commented Aug 29, 2020

@elaguerta I think I covered all of your requested changes, and made sure that I used 4 space indents like Rob requested above. I also added some discussion points in our data fetch channel, and edited the metadata accordingly.

Let me know if you think it's ready to merge!

@elaguerta
Copy link
Collaborator

Looks good to me. Thanks for your contributions!

@elaguerta elaguerta merged commit df5bff8 into master Sep 3, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants