Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reference Implementation and Data Structure #4

Open
miili opened this issue Nov 3, 2022 · 1 comment
Open

Reference Implementation and Data Structure #4

miili opened this issue Nov 3, 2022 · 1 comment

Comments

@miili
Copy link
Member

miili commented Nov 3, 2022

I agree with the meta data but the data layout is not clear to me.

From our experience the real bottleneck handling (raw and processed) DAS data is I/O. I think that hdf5 is well suited for the job, but the layout of the data is very important. Please see a proposed object oriented structure here:

Where the DASFrame shall be limited to a number of samples in time, not number of channels (similar to miniseed). The methods guarantee that one can easily scan a data set without loading the data. Also one must consider memmaping the data.
@miili miili changed the title Reference Structure Reference Implementation and Data Structure Nov 3, 2022
@andreas-wuestefeld
Copy link
Collaborator

I must admit your data model is too abstract for my brain to process. I am but a humble geophysicist, not a programmer...

how does DASframe limite number of channels? Are you suggesting to switch matrix dimension?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@miili @andreas-wuestefeld