Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

mamluk studies journal ingestion #2

Open
verbalhanglider opened this issue May 6, 2017 · 1 comment
Open

mamluk studies journal ingestion #2

verbalhanglider opened this issue May 6, 2017 · 1 comment
Assignees
@c-blair
Milestone
ingest 291 PDFs i...

Comments

@verbalhanglider
Copy link
Contributor

The metadata in extracted from the 291 pdf files needs to be evaluated by stakeholders and a consensus should be reached that

A. it looks acceptable
B. what needs to be added or not added for any records that do not have a value in a particular field
@verbalhanglider verbalhanglider changed the title metadata extracted needs to be evaluated mamluk studies journal ingestion May 6, 2017
verbalhanglider pushed a commit that referenced this issue Jul 6, 2017
issue #2; added looking for keyword field in pdf metadata
8ab8568
verbalhanglider pushed a commit that referenced this issue Jul 6, 2017
issue #2: started metadata generator class
4abb5fe
verbalhanglider pushed a commit that referenced this issue Jul 12, 2017
issue #2; started generating required elements.
4b035fe
verbalhanglider pushed a commit that referenced this issue Jul 14, 2017
issue #2: first draft dublin core record creation
393028f 
I've created a mapper class and am feeding an input dict to each mapper
to generate a dublin core ElementTree with the required elements with
the appropriate values

TODO: need to get stakeholder approval of this mapping before moving
forward
TODO: fold existing SAF creation code base into this one
verbalhanglider pushed a commit that referenced this issue Jul 17, 2017
issue #2: refactored extractor.py
7161d5a 
split major loops in main() into two separate functions. also, moved all
functions that were being re-typed into single functions being called
multiple times.
verbalhanglider pushed a commit that referenced this issue Jul 18, 2017
issue #2: encapsulate extraction of data from input
10643d9 
I've created a separate function for each field being extracted from the
data that needs to be read/normalized in its own way. This should make
understanding the errors in the data a lot easier rather than the
previous way where it was all being done in one giant main() function
@verbalhanglider
Copy link
Contributor Author

SAFs for full volumes have been generated. Waiting on upload and DOIs for those per stakeholder request to add DOIs for volumes to individual articles for SAF generation of individual articles.

verbalhanglider pushed a commit that referenced this issue Nov 7, 2017
issue #2: first draft dublin core record creation
b1b1283 
I've created a mapper class and am feeding an input dict to each mapper
to generate a dublin core ElementTree with the required elements with
the appropriate values

TODO: need to get stakeholder approval of this mapping before moving
forward
TODO: fold existing SAF creation code base into this one
verbalhanglider pushed a commit that referenced this issue Nov 7, 2017
issue #2: refactored extractor.py
862538f 
split major loops in main() into two separate functions. also, moved all
functions that were being re-typed into single functions being called
multiple times.
verbalhanglider pushed a commit that referenced this issue Nov 7, 2017
issue #2: encapsulate extraction of data from input
96d5dfc 
I've created a separate function for each field being extracted from the
data that needs to be read/normalized in its own way. This should make
understanding the errors in the data a lot easier rather than the
previous way where it was all being done in one giant main() function
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@c-blair c-blair

Labels
None yet
Projects
None yet
Milestone
ingest 291 PDFs into knowledgespace.u...
Development

No branches or pull requests
2 participants
@verbalhanglider @c-blair