CDC data duplicated on U.S. Department of Health & Human Services

[jbrown-xentity]

Total list of site-wide duplicates. 804 records, all but 10 are CDC records.
CDC has their own organization: https://catalog.data.gov/organization/centers-for-disease-control-and-prevention
However, it looks like all the CDC data is replicated in US Department of Health: https://catalog.data.gov/organization/hhs-gov
In fact, it looks like US Department of Health has more records: https://catalog.data.gov/dataset/?q=%22https%3A%2F%2Fdata.cdc.gov%2Fapi%2Fviews%2F%22&sort=views_recent+desc&ext_location=&ext_bbox=&ext_prev_extent=-150.46875%2C-80.17871349622823%2C151.875%2C80.17871349622823

Expected behavior

Department harvest sources are cleaned such that they don't provide duplicates

Actual behavior

Multiple harvest sources (one from agency, one from department) causing duplicates.

Sketch

Validate with CDC that Department of Health is getting their updated data, and remove CDC harvest source.
https://catalog.data.gov/harvest/https-data-cdc-gov-data-json
Both the harvest sources are being harvested daily...

[hkdctol]

Reaching out to HHS to track down some current contacts to work on this.

[hkdctol]

Pinged HHS again; moving to blocked for now

[hkdctol]

Still blocked but HHS confirms they are investigating the issue

[hkdctol]

Pinging HHS again--to confirm that we can delete CDC because HHS data.json covers it

[hkdctol]

HHS confirms that CDC harvest source and organization can be deleted, but there's a harvest job right now so will delete next week.

[hkdctol]

I tried to clear CDC data.json as a harvest source, but got an error:

[FuhuXia]

The error is from web server timeout. The clearing process has finished regardless. I see CDC has 0 datasets.
https://catalog.data.gov/organization/centers-for-disease-control-and-prevention

[FuhuXia]

CDC harvest source is cleared and deleted.

[hkdctol]

Organization deleted.