-
Notifications
You must be signed in to change notification settings - Fork 20
ESGFInterfaceGroups
| Wiki Reorganisation |
|---|
| This page has been classified for reorganisation. It has been given the category REVISE. |
| This page contains useful content but needs revision. It may contain out of date or inaccurate content. |
8 interface documentation groups have been identified. Each group has a lead who is responsible for coordinating discussions and writing the documentation. We have to balance this work with ongoing development and operations of ESGF so the groups are ranked into high and lower priority. Lower priority groups do not imply no work should be done on them but that they are not on the critical path for current deployments and immediate plans. High priority groups should report on progress at the June 21st ESGF teleconference. Lower priority groups can report briefly on June 21st or can postpone to a future teleconference. Anyone interested in a group should put themselves down as an interested party and/or contact the group lead.
| Interface Document Groups | Lead | Priority |
|---|---|---|
| [ Registry Group ](https://github.com/ESGF/esgf.github.io/wiki/ESGFInterfaceGroups | RegistryGroup) | Gavin |
| [ Search Group](https://github.com/ESGF/esgf.github.io/wiki/ESGFInterfaceGroups | SearchGroup) | Luca |
| [ Security Group ](https://github.com/ESGF/esgf.github.io/wiki/ESGFInterfaceGroups | SecurityGroup) | Rachana |
| [ Threeds / Publishing Group ](https://github.com/ESGF/esgf.github.io/wiki/ESGFInterfaceGroups | ThreddsGroup) | Bob |
| [ Information Architecture (Data Model) Group ](https://github.com/ESGF/esgf.github.io/wiki/ESGFInterfaceGroups | InformationArchitecture) | Stephen |
| [ Metadata Group ](https://github.com/ESGF/esgf.github.io/wiki/ESGFInterfaceGroups | MetadataGroup) | Mark |
| [ QC / DOI Group](https://github.com/ESGF/esgf.github.io/wiki/ESGFInterfaceGroups | QCGroup) | Martina |
| [ Notification Group ](https://github.com/ESGF/esgf.github.io/wiki/ESGFInterfaceGroups | NotificationGroup) | Gavin |
(note: group pages are initially empty, see below for suggested tasks) See the roadmap page for specific activities taking place and their POCs (point of contacts)
- High Priority Interfaces These are interfaces that require urgent work as they impact current deployments of ESGF or imediate future plans. Multiple implementations exist or are being developed and therefore there is a good chance of rapid progress on a common interface.
- Lead - Gavin
- Interested Parties - Nate, Phil, Luca, Feiyi,Rick, Neill
Page Registry Group
The registry is a concept that spans both P2P and Gateway architectures and is therefore vital to establishing a migration path to the P2P system. It is also part of the latest Gateway releases so impacts current test deployments and future operational deployments of the CMIP5 archive. Items that need addressing include:
- bootstraping
- interoperability between the central registry and P2P registry designs
- maintaining security
- schema
- Lead - Luca
- Interested Parties - Mark Morgan, Eric, Stephan Kindermann, Feiyi, John Harney
Page Search Group
Until recently faceted search has been embedded in the Gateway UI. There is now 2 independend implementations of faceted search: Gateway and P2P. Also there are a couple of search interfaces built on top of other THREDDS harvesters at IPSL and DKRZ. All of these implementations should expose a common interface to faceted search. This interface could be based on SOLr's capabilities but shouldn't rely on SOLr-specific formats and protocols in the long term. Another source could be opensearch ( http://www.opensearch.org ) which has an extensible mechanism for search parameters. We also need to standardise how searchable information is federated and how search results for the same data from different sources will be represented (i.e. replicas). There are 2 basic models for federated search, harvesting or distributed query.
- Lead - Rachana
- Interested Parties - Phil, Luca, Nate, Gavin, Neill
Page Security Group
A security interface control document already exists but needs extending to cover some features required for the P2P development that are currently only implemented in the Gateway:
- Group interface: registration, discovery, etc.
- LAS integration
- Lead - Roland & Bob
- Interested Parties - Roland, Stephen Pascoe, Stephan Kindermann, Henrik Wiberg, Bob Drach, Hans Ramthum, Luca, John Harney, Feiyi Wang
Page Thredds Group
THREDDS XML is a flexible XML schema with many features that are not used in ESGF. It is not clear whether these features are compatible with ESGF software. For instance use of the element to inherit file-dataset properties from their container. There are multiple tools being developed to process ESGF THREDDS catalogues and each of these rely on the implied profile of THREDDS XML created by esgpublisher. These implementations include:
- Gateway
- P2P Index node
- IPSL internal datanode interface
- IS-ENES Portal developed at DKRZ
New applications of the ESGF stack are being developed that will rely on variations to the CMIP5 THREDDS XML encoding. These developments need to be kept consistent with any formalisation of the ESGF THREDDS profile:
- CORDEX data
- Observation data
There is also a need to document the publication interface as there are now 2 independent implementations of the server:
- Gateway
- P2P Index node
ESGF THREDDS XML interacts closely with the ESGF search interface and therefore the relationship between THREDDS elements and ESGF facets needs defining.
- Lower Priority Interfaces These are important for future integration of ESGF components but agreement is not critical to current deployments.
- Lead - Stephen Pascoe
- Interested Parties - Luca, Sebastian, Gavin
Page Information Architecture Group
What is an ESGF dataset? How do they relate to data files? What's the difference between a dataset it's location (given it may be in many places at once). What does dataset versioning mean? How do we organise facets and associate them with datasets? How do we associate complex metadata (e.g. Metafor CIM) with datasets. How do we associate services with datasets (e.g. OPeNDAP, gridftp, LAS) Lots of thorny issues here.
- Lead - Mark Morgan
- Interested Parties - Nate, Stephen Pascoe, Charlotte Pascoe?, Luca, Cecelia, Sylvia, Sebastian
Page: Metadata Group Display of Metafor CIM metadata and it's association with ESGF datasets is an important requirement for CMIP5 and presently there is no alternative to the Gateway implementation. However, the Metafor services portal will expose APIs for accessing CIM metadata. This could be a future integration point for both the Gateway and P2P systems.
- Lead - Martina
- Interested Parties - Kevin Marsh?, Sebastien?, Luca, Sebastian
Page QC Group
Good documentation exists on Quality Control for CMIP5. It would be useful to generalise this to other data activities such as observations and CORDEX and to place it on the ESGF wiki. Some issues are still ill defined. E.g. how a DOIs relate to datasets. What is meant by the "Simulation Level". How DOI landing pages will be implemented. How DOIs interacts with version changes.
- Lead - Gavin
- Interested Parties - Gavin ,
Page NotificationGroup
The notification and metrics system should be documented at some point.
Please also see the RoadMap page for more status on the implementations of the interfaces we have come up with.