From 5e94ebabd06e61d21a658d823e2f92520e24d4c6 Mon Sep 17 00:00:00 2001 From: Matthew Evans Date: Wed, 20 Dec 2023 12:50:14 +0000 Subject: [PATCH 01/15] Update changelog and spec version for rc.2 --- CHANGELOG.md | 36 ++++++++++++++++++++++++++++++++++++ optimade.rst | 6 +++--- 2 files changed, 39 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ed9584ba0..7c31e8b5a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,41 @@ # Changelog +## v1.2.0-rc.2 (December 2023) + +This is the second release candidate of v1.2.0 of the OPTIMADE API specification. +Implementation details may still be modified before the final release. + +Note: The OpenAPI schemas distributed in `./schemas` have not yet been modified with the new features 1.2.0. + +This minor release adds significant but optional new functionality to the specification, as well as providing several clarifications to existing behaviour. + +### Changes relative to rc.1 + +- **Partial data** ([#467](https://github.com/Materials-Consortia/OPTIMADE/pull/467)): Adds the mechanism format for streaming, paginating or slicing individual properties within entries. +- **Per entry/property metadata** ([#463](https://github.com/Materials-Consortia/OPTIMADE/pull/463): Added a mechanism for providing metadata specific to a given entry or property +- **Database metadata field** ([#424](https://github.com/Materials-Consortia/OPTIMADE/pull/424): Added an additional metadata field `database` for describing a given dataset +- **Compatibility with JSON:API v1.1** ([#461](https://github.com/Materials-Consortia/OPTIMADE/pull/461) +- **Symmetry operation specification and enhanced space group fields** ([#480](https://github.com/Materials-Consortia/OPTIMADE/pull/480) +- Typo, formatting and snippet fixes + +### New features (duplicated from rc.1) + +- **Property definitions** ([#376](https://github.com/Materials-Consortia/OPTIMADE/pull/376)). +A new section titled [Property Definitions](https://github.com/Materials-Consortia/OPTIMADE/blob/develop/optimade.rst#property-definitions) has been added to the specification which significantly extends the way in which implementations can define and describe the custom properties they serve, including URIs, unit definitions, API support levels (for querying and sorting) as well as full support for JSON Schema constructs for describing the JSON representation of the property. +- **Files endpoint** ([#360](https://github.com/Materials-Consortia/OPTIMADE/pull/360)). +The `/files` endpoint and corresponding [`files` entry +type](https://github.com/Materials-Consortia/OPTIMADE/blob/develop/optimade.rst#files-entries) has been added to provide a robust way of linking entries to arbitrary file-based data relevant to the entry, such as alternative crystal structure representation formats, input or output files from computational procedures, or experimental data files. +- **Boolean values** ([#348](https://github.com/Materials-Consortia/OPTIMADE/pull/348)). +[Boolean values](https://github.com/Materials-Consortia/OPTIMADE/blob/develop/optimade.rst#comparisons-of-boolean-values) were overlooked in the first version of the filter grammar as no OPTIMADE fields required them. +This functionality has been introduced for boolean fields using the syntax `TRUE` and `FALSE`. +Only strict equality (`=`) and inequality (`!=`) comparisons on individual fields are supported. +- **Fuzzy comparisons on lists** ([#415](https://github.com/Materials-Consortia/OPTIMADE/pull/415)) +String comparisons like `CONTAINS`, `STARTS WITH` and `ENDS WITH` are now compatible with list filter operations like `HAS`, `HAS ALL` etc. +- **Backoff time** ([#411](https://github.com/Materials-Consortia/OPTIMADE/pull/411)): +- **Database licenses** ([#414](https://github.com/Materials-Consortia/OPTIMADE/pull/414)): +- **Symmetry data** ([#405](https://github.com/Materials-Consortia/OPTIMADE/pull/405)): + + ## v1.2.0-rc.1 (December 2022) This is the first release candidate of v1.2.0 of the OPTIMADE API specification. diff --git a/optimade.rst b/optimade.rst index b2f37d25a..17eb411c2 100644 --- a/optimade.rst +++ b/optimade.rst @@ -1,6 +1,6 @@ -========================================= -OPTIMADE API specification v1.2.0~develop -========================================= +====================================== +OPTIMADE API specification v1.2.0-rc.2 +====================================== .. comment From 9e4a0ac27de381cef20c52be3a096f05dfeed5a6 Mon Sep 17 00:00:00 2001 From: Matthew Evans <7916000+ml-evs@users.noreply.github.com> Date: Thu, 21 Mar 2024 09:45:56 +0000 Subject: [PATCH 02/15] Apply suggestions from code review --- CHANGELOG.md | 6 +----- optimade.rst | 6 +++--- 2 files changed, 4 insertions(+), 8 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7c31e8b5a..b48557959 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,11 +1,7 @@ # Changelog -## v1.2.0-rc.2 (December 2023) +## v1.2.0 (March 2024) -This is the second release candidate of v1.2.0 of the OPTIMADE API specification. -Implementation details may still be modified before the final release. - -Note: The OpenAPI schemas distributed in `./schemas` have not yet been modified with the new features 1.2.0. This minor release adds significant but optional new functionality to the specification, as well as providing several clarifications to existing behaviour. diff --git a/optimade.rst b/optimade.rst index 17eb411c2..8644ca9f0 100644 --- a/optimade.rst +++ b/optimade.rst @@ -1,6 +1,6 @@ -====================================== -OPTIMADE API specification v1.2.0-rc.2 -====================================== +================================= +OPTIMADE API specification v1.2.0 +================================= .. comment From bc49bd9ec92d62bc1f024492b1b545ab9f4bac5d Mon Sep 17 00:00:00 2001 From: Matthew Evans Date: Mon, 25 Mar 2024 10:44:16 +0000 Subject: [PATCH 03/15] Add biased selection of recent contributors to AUTHORS list --- AUTHORS | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/AUTHORS b/AUTHORS index 54a6389ad..255687cbb 100644 --- a/AUTHORS +++ b/AUTHORS @@ -5,14 +5,19 @@ The OPTIMADE API has been developed with contributions by the participants of th - the CECAM in Lausanne, Switzerland (virtually) 2020-06-08 to 2020-06-12 - the CECAM in Lausanne, Switzerland (virtually) 2021-06-07 to 2021-06-11 - the CECAM in Lausanne, Switzerland 2022-05-30 to 2022-06-03 +- the CECAM in Lausanne, Switzerland 2023-06-05 to 2023-06-09 -Contributors to the development of the OPTIMADE specification and implementation in alphabetical order: +Initial and major contributors to the development of the OPTIMADE specification and implementation in alphabetical order: Casper Andersen +Oskar Andersson Thomas Archer Rickard Armiento Rossella Aversa +Daniel Beltrán Johan Bergsma Evgeny Blokhin +Tara Boland +Sara Bonella Kamal Choudhary Pauline Colinet Gareth Conduit @@ -21,6 +26,7 @@ Davide Di Stefano Alexander Dorsk Claudia Draxl Shyam Dwaraknath +Kristjan Eimre Suleyman Er Marco Esters Matthew Evans @@ -39,11 +45,13 @@ Jens Hummelshoej Karsten W. Jacobsen Ankit Kariryaa Boris Kozinsky +Adam Krajewski Snehal Kumbhar Mohan Liu Nicola Marzari Andrius Merkys Fawzi Mohamed +Jens Jørgen Mortensen Andrew Morris Arash Mostofi Corey Oses @@ -68,3 +76,4 @@ Chris Wolverton Michael Wu Yibin Xu Xiaoyu Yang +Jusong Yu From f6616124e3dd3c0ca3808fc560345d726afc4fe4 Mon Sep 17 00:00:00 2001 From: Matthew Evans Date: Mon, 25 Mar 2024 11:24:30 +0000 Subject: [PATCH 04/15] Add citation for latest preprint in README --- README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 25e6abfa0..a2e0c23c3 100644 --- a/README.md +++ b/README.md @@ -46,9 +46,10 @@ Alternatively, the software using the file could itself be licensed in a way com ## How to cite -If you use OPTIMADE to access or host data, we kindly ask that you cite our paper: +If you use OPTIMADE to access or host data, we kindly ask that you cite our papers accompaying the version 1.0 and 1.2 releases: - Andersen *et al*, OPTIMADE, an API for exchanging materials data, *Sci. Data* **8**, 217 (2021) [10.1038/s41597-021-00974-z](https://doi.org/10.1038/s41597-021-00974-z) +- Evans *et al*, Developments and applications of the OPTIMADE API for materials discovery, design, and data exchange, *arXiv preprint* (2024) [10.48550/arXiv.2402.00572](https://doi.org/10.48550/arXiv.2402.00572) To cite an individual version of the specification, please use the versioned records on Zenodo: From f00aa83e06c8f03b422e07f015e581036d3b1888 Mon Sep 17 00:00:00 2001 From: Matthew Evans Date: Mon, 25 Mar 2024 11:24:39 +0000 Subject: [PATCH 05/15] Update CHANGELOG and add implementation notes --- CHANGELOG.md | 73 +++++++++++++++++++++------------------------------- 1 file changed, 30 insertions(+), 43 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index b48557959..74e7513e2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,60 +3,47 @@ ## v1.2.0 (March 2024) -This minor release adds significant but optional new functionality to the specification, as well as providing several clarifications to existing behaviour. +This release adds significant but optional new functionality to the specification, as well as providing several clarifications to existing behaviour. -### Changes relative to rc.1 +Minor OPTIMADE releases are always intended to be backwards-compatible for clients, meaning that any client code written for v1.1 should continue to work. +Although the majority of features added in thius release are optional for servers, there are a couple of mandatory format additions that are discussed below in the notes for implementations. +We have clarified our approach to versioning explicitly in the specification +under [Versioning of this standard](https://github.com/Materials-Consortia/OPTIMADE/blob/v1.2.0/optimade.rst#versioning-of-this-standard). -- **Partial data** ([#467](https://github.com/Materials-Consortia/OPTIMADE/pull/467)): Adds the mechanism format for streaming, paginating or slicing individual properties within entries. -- **Per entry/property metadata** ([#463](https://github.com/Materials-Consortia/OPTIMADE/pull/463): Added a mechanism for providing metadata specific to a given entry or property -- **Database metadata field** ([#424](https://github.com/Materials-Consortia/OPTIMADE/pull/424): Added an additional metadata field `database` for describing a given dataset -- **Compatibility with JSON:API v1.1** ([#461](https://github.com/Materials-Consortia/OPTIMADE/pull/461) -- **Symmetry operation specification and enhanced space group fields** ([#480](https://github.com/Materials-Consortia/OPTIMADE/pull/480) -- Typo, formatting and snippet fixes +For a more academic summary of the changes, please see our preprint: [arXiv:2402.00572](https://doi.org/10.48550/arXiv.2402.00572). -### New features (duplicated from rc.1) +### New features + +- **Property definitions** ([#376](https://github.com/Materials-Consortia/OPTIMADE/pull/376)): A new section titled [Property Definitions](https://github.com/Materials-Consortia/OPTIMADE/blob/v1.2.0/optimade.rst#property-definitions) has been added to the specification which significantly extends the way in which implementations can define and describe the custom properties they serve, including URIs, unit definitions, API support levels (for querying and sorting) as well as full support for JSON Schema constructs for describing the JSON representation of the property. +- **Namespace prefixes for definitions** ([#473](https://github.com/Materials-Consortia/OPTIMADE/pull/473)): The mechanism for providers to define custom properties under their own namespace/prefix has been extended to allow implementations to share common definitions. These so-called definition namespaces can be registered as providers in their own right and can serve property definitions in the new format. +- **Partial data** ([#467](https://github.com/Materials-Consortia/OPTIMADE/pull/467)): Adds a mechanism and format for streaming, paginating or slicing individual properties within entries. +- **Per entry/property metadata** ([#463](https://github.com/Materials-Consortia/OPTIMADE/pull/463): Added a mechanism for providing metadata specific to a given entry or property. +- **Files endpoint** ([#360](https://github.com/Materials-Consortia/OPTIMADE/pull/360)): The `/files` endpoint and corresponding [`files` entry type](https://github.com/Materials-Consortia/OPTIMADE/blob/v1.2.0/optimade.rst#files-entries) has been added to provide a robust way of linking entries to arbitrary file-based data relevant to the entry, such as alternative crystal structure representation formats, input or output files from computational procedures, or experimental data files. +- **Symmetry operation specification and space group fields** ([#480](https://github.com/Materials-Consortia/OPTIMADE/pull/480), [#405](https://github.com/Materials-Consortia/OPTIMADE/pull/405), [#464](https://github.com/Materials-Consortia/OPTIMADE/pull/464)): Several fields have been added to the `structures` entry type to fully describe symmetry. Symmetry operations can be provided explicitly in `space_group_symmetry_operations_xyz`, as well as space group specifications in various forms: `space_group_symbol_hall`, `space_group_symbol_hermann_mauguin`, `space_group_symbol_hermann_mauguin_extended`, `space_group_it_number`. +- **Database licenses** ([#414](https://github.com/Materials-Consortia/OPTIMADE/pull/414)): Several fields have been added to programmatically describe the licensing status of data served by OPTIMADE APIs. A database-wide license can be provided as a set of [SPDX identifiers](https://spdx.org/licenses/) in `available_licenses`, with a related field `available_licenses_for_entries` specifying specifically which licenses are available for data contained within the database. The global field `license` can be used point to a human-readable license page (external or otherwise) that explains any caveats to the licensing arrangement. +- **Database metadata field** ([#424](https://github.com/Materials-Consortia/OPTIMADE/pull/424)): Added an additional metadata field `database` for providing a human-readable description of a given database. +- **Backoff time** ([#411](https://github.com/Materials-Consortia/OPTIMADE/pull/411)): Databases can now request ahead of time that clients apply a particular rate limit or back-off time via the `request_delay` metadata field. -- **Property definitions** ([#376](https://github.com/Materials-Consortia/OPTIMADE/pull/376)). -A new section titled [Property Definitions](https://github.com/Materials-Consortia/OPTIMADE/blob/develop/optimade.rst#property-definitions) has been added to the specification which significantly extends the way in which implementations can define and describe the custom properties they serve, including URIs, unit definitions, API support levels (for querying and sorting) as well as full support for JSON Schema constructs for describing the JSON representation of the property. -- **Files endpoint** ([#360](https://github.com/Materials-Consortia/OPTIMADE/pull/360)). -The `/files` endpoint and corresponding [`files` entry -type](https://github.com/Materials-Consortia/OPTIMADE/blob/develop/optimade.rst#files-entries) has been added to provide a robust way of linking entries to arbitrary file-based data relevant to the entry, such as alternative crystal structure representation formats, input or output files from computational procedures, or experimental data files. +### Patches + +- **Fuzzy comparisons on lists** ([#415](https://github.com/Materials-Consortia/OPTIMADE/pull/415)) +String comparisons like `CONTAINS`, `STARTS WITH` and `ENDS WITH` are now compatible with list filter operations like `HAS`, `HAS ALL` etc. - **Boolean values** ([#348](https://github.com/Materials-Consortia/OPTIMADE/pull/348)). -[Boolean values](https://github.com/Materials-Consortia/OPTIMADE/blob/develop/optimade.rst#comparisons-of-boolean-values) were overlooked in the first version of the filter grammar as no OPTIMADE fields required them. +[Boolean values](https://github.com/Materials-Consortia/OPTIMADE/blob/v1.2.0/optimade.rst#comparisons-of-boolean-values) were overlooked in the first version of the filter grammar as no OPTIMADE fields required them. This functionality has been introduced for boolean fields using the syntax `TRUE` and `FALSE`. Only strict equality (`=`) and inequality (`!=`) comparisons on individual fields are supported. -- **Fuzzy comparisons on lists** ([#415](https://github.com/Materials-Consortia/OPTIMADE/pull/415)) -String comparisons like `CONTAINS`, `STARTS WITH` and `ENDS WITH` are now compatible with list filter operations like `HAS`, `HAS ALL` etc. -- **Backoff time** ([#411](https://github.com/Materials-Consortia/OPTIMADE/pull/411)): -- **Database licenses** ([#414](https://github.com/Materials-Consortia/OPTIMADE/pull/414)): -- **Symmetry data** ([#405](https://github.com/Materials-Consortia/OPTIMADE/pull/405)): - +- **Compatibility with JSON:API v1.1** ([#461](https://github.com/Materials-Consortia/OPTIMADE/pull/461)): references to JSON:API v1.0 have been updated to v1.1. +- Typo, formatting and code snippet fixes -## v1.2.0-rc.1 (December 2022) +### Notes for implementations -This is the first release candidate of v1.2.0 of the OPTIMADE API specification. -It should contain all of the new features in the specification, but their implementation may be modified in the final release. +As discussed above, the majority of this release involves the addition of more expressive metadata fields for property definitions, definition namespaces shared across providers, and licensing information, as well as mechanisms for serving files and partial data responses (for large entries, e.g., giant structures). +We hope that the machine-readable schemas and property definitions now available at [schemas.optimade.org](https://schemas.optimade.org) will make implementing the specification much easier. -Note: The OpenAPI schemas distributed in `./schemas` have not yet been modified with the new features 1.2.0. +The mandatory format changes required to support v1.2 are limited to the following: -This minor release adds significant but optional new functionality to the specification, as well as providing several clarifications to existing behaviour. - -### New features - -- **Property definitions** ([#376](https://github.com/Materials-Consortia/OPTIMADE/pull/376)). -A new section titled [Property Definitions](https://github.com/Materials-Consortia/OPTIMADE/blob/develop/optimade.rst#property-definitions) has been added to the specification which significantly extends the way in which implementations can define and describe the custom properties they serve, including URIs, unit definitions, API support levels (for querying and sorting) as well as full support for JSON Schema constructs for describing the JSON representation of the property. -- **Files endpoint** ([#360](https://github.com/Materials-Consortia/OPTIMADE/pull/360)). -The `/files` endpoint and corresponding [`files` entry -type](https://github.com/Materials-Consortia/OPTIMADE/blob/develop/optimade.rst#files-entries) has been added to provide a robust way of linking entries to arbitrary file-based data relevant to the entry, such as alternative crystal structure representation formats, input or output files from computational procedures, or experimental data files. -- **Boolean values** ([#348](https://github.com/Materials-Consortia/OPTIMADE/pull/348)). -[Boolean values](https://github.com/Materials-Consortia/OPTIMADE/blob/develop/optimade.rst#comparisons-of-boolean-values) were overlooked in the first version of the filter grammar as no OPTIMADE fields required them. -This functionality has been introduced for boolean fields using the syntax `TRUE` and `FALSE`. -Only strict equality (`=`) and inequality (`!=`) comparisons on individual fields are supported. -- **Fuzzy comparisons on lists** ([#415](https://github.com/Materials-Consortia/OPTIMADE/pull/415)) -String comparisons like `CONTAINS`, `STARTS WITH` and `ENDS WITH` are now compatible with list filter operations like `HAS`, `HAS ALL` etc. -- **Backoff time** ([#411](https://github.com/Materials-Consortia/OPTIMADE/pull/411)): -- **Database licenses** ([#414](https://github.com/Materials-Consortia/OPTIMADE/pull/414)): -- **Symmetry data** ([#405](https://github.com/Materials-Consortia/OPTIMADE/pull/405)): +- `/info/` endpoints MUST now have a top-level `id` and `type` field. This is for compliance with JSON:API and their previous omission should be treated as a specification bug. +- In cases where a server implementation treats filters on non-prefixed but unknown OPTIMADE fields as errors, implementations MUST update their known property list to handle new fields added to OPTIMADE in this version, such that they can continue to follow the expected behaviour for [Handling unknown property names](https://github.com/Materials-Consortia/OPTIMADE/blob/v1.2.0/optimade.rst#handling-unknown-property-names). ## v1.1.0 (July 8, 2021) From 648f229433dfb5f5e7242ab1f7c40558150e31ff Mon Sep 17 00:00:00 2001 From: Matthew Evans Date: Mon, 25 Mar 2024 11:33:48 +0000 Subject: [PATCH 06/15] Add new links out to schemas and specification --- CHANGELOG.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 74e7513e2..59f60df70 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,6 +12,8 @@ under [Versioning of this standard](https://github.com/Materials-Consortia/OPTIM For a more academic summary of the changes, please see our preprint: [arXiv:2402.00572](https://doi.org/10.48550/arXiv.2402.00572). +In addition to the specification document itself, machine-readable schemas from this repository can now be found hosted at [schemas.optimade.org](https:///schemas.optimade.org), and HTML builds of the specification can be found at [specification.optimade.org](https://specification.optimade.org). + ### New features - **Property definitions** ([#376](https://github.com/Materials-Consortia/OPTIMADE/pull/376)): A new section titled [Property Definitions](https://github.com/Materials-Consortia/OPTIMADE/blob/v1.2.0/optimade.rst#property-definitions) has been added to the specification which significantly extends the way in which implementations can define and describe the custom properties they serve, including URIs, unit definitions, API support levels (for querying and sorting) as well as full support for JSON Schema constructs for describing the JSON representation of the property. From 0cfd885456349fe5eb270c78a22abd0ad1b0eedb Mon Sep 17 00:00:00 2001 From: Matthew Evans Date: Mon, 25 Mar 2024 11:36:47 +0000 Subject: [PATCH 07/15] Update AUTHORS --- AUTHORS | 1 + 1 file changed, 1 insertion(+) diff --git a/AUTHORS b/AUTHORS index 255687cbb..93223d3a7 100644 --- a/AUTHORS +++ b/AUTHORS @@ -48,6 +48,7 @@ Boris Kozinsky Adam Krajewski Snehal Kumbhar Mohan Liu +Zi-Kui Liu Nicola Marzari Andrius Merkys Fawzi Mohamed From 73520d09c7c6ac97f102357e519b5959fa3bfc02 Mon Sep 17 00:00:00 2001 From: Matthew Evans Date: Mon, 25 Mar 2024 11:40:16 +0000 Subject: [PATCH 08/15] Add link to future spec builds --- README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index a2e0c23c3..c49e110fe 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,8 @@ The Open Databases Integration for Materials Design (OPTIMADE) consortium aims t This repository contains the specification of the OPTIMADE API. -* [optimade.rst](optimade.rst): The API specification. +* [optimade.rst](optimade.rst): The API specification document. +* [specification.optimade.org](https://specification.optimade.org): HTML builds of the different specification versions. * [schemas.optimade.org](https://schemas.optimade.org): Machine-readable schemas for property and server definitions. * [AUTHORS](AUTHORS): List of contributors. * [CHANGELOG](CHANGELOG.md): The release notes for each version of the specification. From 9a3af4589e2d47bbfea5d36f1fc5cb7725cacbd5 Mon Sep 17 00:00:00 2001 From: Matthew Evans <7916000+ml-evs@users.noreply.github.com> Date: Tue, 26 Mar 2024 23:36:10 +0000 Subject: [PATCH 09/15] Apply suggestions from code review Co-authored-by: Andrius Merkys Co-authored-by: Antanas Vaitkus --- CHANGELOG.md | 13 ++++++------- README.md | 2 +- 2 files changed, 7 insertions(+), 8 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 59f60df70..10763152a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,9 +6,8 @@ This release adds significant but optional new functionality to the specification, as well as providing several clarifications to existing behaviour. Minor OPTIMADE releases are always intended to be backwards-compatible for clients, meaning that any client code written for v1.1 should continue to work. -Although the majority of features added in thius release are optional for servers, there are a couple of mandatory format additions that are discussed below in the notes for implementations. -We have clarified our approach to versioning explicitly in the specification -under [Versioning of this standard](https://github.com/Materials-Consortia/OPTIMADE/blob/v1.2.0/optimade.rst#versioning-of-this-standard). +Although the majority of features added in this release are optional for servers, there are a couple of mandatory format additions that are discussed below in the notes for implementations. +We have clarified our approach to versioning explicitly in the specification under [Versioning of this standard](https://github.com/Materials-Consortia/OPTIMADE/blob/v1.2.0/optimade.rst#versioning-of-this-standard). For a more academic summary of the changes, please see our preprint: [arXiv:2402.00572](https://doi.org/10.48550/arXiv.2402.00572). @@ -19,8 +18,8 @@ In addition to the specification document itself, machine-readable schemas from - **Property definitions** ([#376](https://github.com/Materials-Consortia/OPTIMADE/pull/376)): A new section titled [Property Definitions](https://github.com/Materials-Consortia/OPTIMADE/blob/v1.2.0/optimade.rst#property-definitions) has been added to the specification which significantly extends the way in which implementations can define and describe the custom properties they serve, including URIs, unit definitions, API support levels (for querying and sorting) as well as full support for JSON Schema constructs for describing the JSON representation of the property. - **Namespace prefixes for definitions** ([#473](https://github.com/Materials-Consortia/OPTIMADE/pull/473)): The mechanism for providers to define custom properties under their own namespace/prefix has been extended to allow implementations to share common definitions. These so-called definition namespaces can be registered as providers in their own right and can serve property definitions in the new format. - **Partial data** ([#467](https://github.com/Materials-Consortia/OPTIMADE/pull/467)): Adds a mechanism and format for streaming, paginating or slicing individual properties within entries. -- **Per entry/property metadata** ([#463](https://github.com/Materials-Consortia/OPTIMADE/pull/463): Added a mechanism for providing metadata specific to a given entry or property. -- **Files endpoint** ([#360](https://github.com/Materials-Consortia/OPTIMADE/pull/360)): The `/files` endpoint and corresponding [`files` entry type](https://github.com/Materials-Consortia/OPTIMADE/blob/v1.2.0/optimade.rst#files-entries) has been added to provide a robust way of linking entries to arbitrary file-based data relevant to the entry, such as alternative crystal structure representation formats, input or output files from computational procedures, or experimental data files. +- **Per entry/property metadata** ([#463](https://github.com/Materials-Consortia/OPTIMADE/pull/463)): Added a mechanism for providing metadata specific to a given entry or property. +- **Files endpoint** ([#360](https://github.com/Materials-Consortia/OPTIMADE/pull/360)): The [`files` entry type](https://github.com/Materials-Consortia/OPTIMADE/blob/v1.2.0/optimade.rst#files-entries) has been added to provide a robust way of linking entries to arbitrary file-based data relevant to the entry, such as alternative crystal structure representation formats, input or output files from computational procedures, or experimental data files. - **Symmetry operation specification and space group fields** ([#480](https://github.com/Materials-Consortia/OPTIMADE/pull/480), [#405](https://github.com/Materials-Consortia/OPTIMADE/pull/405), [#464](https://github.com/Materials-Consortia/OPTIMADE/pull/464)): Several fields have been added to the `structures` entry type to fully describe symmetry. Symmetry operations can be provided explicitly in `space_group_symmetry_operations_xyz`, as well as space group specifications in various forms: `space_group_symbol_hall`, `space_group_symbol_hermann_mauguin`, `space_group_symbol_hermann_mauguin_extended`, `space_group_it_number`. - **Database licenses** ([#414](https://github.com/Materials-Consortia/OPTIMADE/pull/414)): Several fields have been added to programmatically describe the licensing status of data served by OPTIMADE APIs. A database-wide license can be provided as a set of [SPDX identifiers](https://spdx.org/licenses/) in `available_licenses`, with a related field `available_licenses_for_entries` specifying specifically which licenses are available for data contained within the database. The global field `license` can be used point to a human-readable license page (external or otherwise) that explains any caveats to the licensing arrangement. - **Database metadata field** ([#424](https://github.com/Materials-Consortia/OPTIMADE/pull/424)): Added an additional metadata field `database` for providing a human-readable description of a given database. @@ -30,11 +29,11 @@ In addition to the specification document itself, machine-readable schemas from - **Fuzzy comparisons on lists** ([#415](https://github.com/Materials-Consortia/OPTIMADE/pull/415)) String comparisons like `CONTAINS`, `STARTS WITH` and `ENDS WITH` are now compatible with list filter operations like `HAS`, `HAS ALL` etc. -- **Boolean values** ([#348](https://github.com/Materials-Consortia/OPTIMADE/pull/348)). +- **Boolean values** ([#348](https://github.com/Materials-Consortia/OPTIMADE/pull/348)): [Boolean values](https://github.com/Materials-Consortia/OPTIMADE/blob/v1.2.0/optimade.rst#comparisons-of-boolean-values) were overlooked in the first version of the filter grammar as no OPTIMADE fields required them. This functionality has been introduced for boolean fields using the syntax `TRUE` and `FALSE`. Only strict equality (`=`) and inequality (`!=`) comparisons on individual fields are supported. -- **Compatibility with JSON:API v1.1** ([#461](https://github.com/Materials-Consortia/OPTIMADE/pull/461)): references to JSON:API v1.0 have been updated to v1.1. +- **Compatibility with JSON:API v1.1** ([#461](https://github.com/Materials-Consortia/OPTIMADE/pull/461)): References to JSON:API v1.0 have been updated to v1.1. - Typo, formatting and code snippet fixes ### Notes for implementations diff --git a/README.md b/README.md index c49e110fe..a660bcf13 100644 --- a/README.md +++ b/README.md @@ -47,7 +47,7 @@ Alternatively, the software using the file could itself be licensed in a way com ## How to cite -If you use OPTIMADE to access or host data, we kindly ask that you cite our papers accompaying the version 1.0 and 1.2 releases: +If you use OPTIMADE to access or host data, we kindly ask that you cite our papers accompanying the version 1.0 and 1.2 releases: - Andersen *et al*, OPTIMADE, an API for exchanging materials data, *Sci. Data* **8**, 217 (2021) [10.1038/s41597-021-00974-z](https://doi.org/10.1038/s41597-021-00974-z) - Evans *et al*, Developments and applications of the OPTIMADE API for materials discovery, design, and data exchange, *arXiv preprint* (2024) [10.48550/arXiv.2402.00572](https://doi.org/10.48550/arXiv.2402.00572) From debb673d3fafc131a90106337e3b4aafb6d65d99 Mon Sep 17 00:00:00 2001 From: Matthew Evans <7916000+ml-evs@users.noreply.github.com> Date: Tue, 2 Apr 2024 19:24:39 +0100 Subject: [PATCH 10/15] Apply suggestions from code review --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 10763152a..a3f3d6b4f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -43,7 +43,7 @@ We hope that the machine-readable schemas and property definitions now available The mandatory format changes required to support v1.2 are limited to the following: -- `/info/` endpoints MUST now have a top-level `id` and `type` field. This is for compliance with JSON:API and their previous omission should be treated as a specification bug. +- `/info/` endpoints MUST now have a top-level `id` and `type` field, e.g., the `/info/structures` MUST now serve `{"id": "structures", "type": "info"}`. This is for compliance with JSON:API and their previous omission should be treated as a specification bug. - In cases where a server implementation treats filters on non-prefixed but unknown OPTIMADE fields as errors, implementations MUST update their known property list to handle new fields added to OPTIMADE in this version, such that they can continue to follow the expected behaviour for [Handling unknown property names](https://github.com/Materials-Consortia/OPTIMADE/blob/v1.2.0/optimade.rst#handling-unknown-property-names). From b42630d8ba1fa84b3b9565cac85741c07a338b29 Mon Sep 17 00:00:00 2001 From: Matthew Evans <7916000+ml-evs@users.noreply.github.com> Date: Tue, 2 Apr 2024 19:26:05 +0100 Subject: [PATCH 11/15] Update CHANGELOG.md --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a3f3d6b4f..842ab62a1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -15,7 +15,7 @@ In addition to the specification document itself, machine-readable schemas from ### New features -- **Property definitions** ([#376](https://github.com/Materials-Consortia/OPTIMADE/pull/376)): A new section titled [Property Definitions](https://github.com/Materials-Consortia/OPTIMADE/blob/v1.2.0/optimade.rst#property-definitions) has been added to the specification which significantly extends the way in which implementations can define and describe the custom properties they serve, including URIs, unit definitions, API support levels (for querying and sorting) as well as full support for JSON Schema constructs for describing the JSON representation of the property. +- **Property definitions** ([#376](https://github.com/Materials-Consortia/OPTIMADE/pull/376)): A new section titled [Property Definitions](https://github.com/Materials-Consortia/OPTIMADE/blob/v1.2.0/optimade.rst#property-definitions) has been added to the specification which significantly extends the way in which implementations can define and describe the custom properties they serve, including URIs, unit definitions, API support levels (for querying and sorting) as well as full support for JSON Schema constructs for describing the JSON representation of the property. These definitions form the basis of a new machine-readable form of the OPTIMADE specification found in the main repository and served at [schemas.optimade.org](https://schemas.optimade.org). - **Namespace prefixes for definitions** ([#473](https://github.com/Materials-Consortia/OPTIMADE/pull/473)): The mechanism for providers to define custom properties under their own namespace/prefix has been extended to allow implementations to share common definitions. These so-called definition namespaces can be registered as providers in their own right and can serve property definitions in the new format. - **Partial data** ([#467](https://github.com/Materials-Consortia/OPTIMADE/pull/467)): Adds a mechanism and format for streaming, paginating or slicing individual properties within entries. - **Per entry/property metadata** ([#463](https://github.com/Materials-Consortia/OPTIMADE/pull/463)): Added a mechanism for providing metadata specific to a given entry or property. From 621a53193de36df74f2f6105966f40b9f884e569 Mon Sep 17 00:00:00 2001 From: Matthew Evans <7916000+ml-evs@users.noreply.github.com> Date: Thu, 4 Apr 2024 14:56:49 +0100 Subject: [PATCH 12/15] Apply suggestions from code review Co-authored-by: Andrius Merkys Co-authored-by: Antanas Vaitkus --- CHANGELOG.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 842ab62a1..9301315aa 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,6 +1,6 @@ # Changelog -## v1.2.0 (March 2024) +## v1.2.0 (April 2024) This release adds significant but optional new functionality to the specification, as well as providing several clarifications to existing behaviour. @@ -27,14 +27,14 @@ In addition to the specification document itself, machine-readable schemas from ### Patches -- **Fuzzy comparisons on lists** ([#415](https://github.com/Materials-Consortia/OPTIMADE/pull/415)) +- **Fuzzy comparisons on lists** ([#415](https://github.com/Materials-Consortia/OPTIMADE/pull/415)): String comparisons like `CONTAINS`, `STARTS WITH` and `ENDS WITH` are now compatible with list filter operations like `HAS`, `HAS ALL` etc. - **Boolean values** ([#348](https://github.com/Materials-Consortia/OPTIMADE/pull/348)): [Boolean values](https://github.com/Materials-Consortia/OPTIMADE/blob/v1.2.0/optimade.rst#comparisons-of-boolean-values) were overlooked in the first version of the filter grammar as no OPTIMADE fields required them. This functionality has been introduced for boolean fields using the syntax `TRUE` and `FALSE`. Only strict equality (`=`) and inequality (`!=`) comparisons on individual fields are supported. - **Compatibility with JSON:API v1.1** ([#461](https://github.com/Materials-Consortia/OPTIMADE/pull/461)): References to JSON:API v1.0 have been updated to v1.1. -- Typo, formatting and code snippet fixes +- Typo, formatting and code snippet fixes. ### Notes for implementations From b15908d7013cbea4291feba890341842c9c79001 Mon Sep 17 00:00:00 2001 From: Matthew Evans <7916000+ml-evs@users.noreply.github.com> Date: Thu, 4 Apr 2024 16:59:52 +0100 Subject: [PATCH 13/15] Set version to rc.2 --- optimade.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/optimade.rst b/optimade.rst index 8644ca9f0..17eb411c2 100644 --- a/optimade.rst +++ b/optimade.rst @@ -1,6 +1,6 @@ -================================= -OPTIMADE API specification v1.2.0 -================================= +====================================== +OPTIMADE API specification v1.2.0-rc.2 +====================================== .. comment From be4bf87a2f781c36c0e83e2d3db4982a1a6201ce Mon Sep 17 00:00:00 2001 From: Matthew Evans <7916000+ml-evs@users.noreply.github.com> Date: Mon, 8 Apr 2024 11:23:42 +0100 Subject: [PATCH 14/15] Improve license description Co-authored-by: Rickard Armiento --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9301315aa..2715369c1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -21,7 +21,7 @@ In addition to the specification document itself, machine-readable schemas from - **Per entry/property metadata** ([#463](https://github.com/Materials-Consortia/OPTIMADE/pull/463)): Added a mechanism for providing metadata specific to a given entry or property. - **Files endpoint** ([#360](https://github.com/Materials-Consortia/OPTIMADE/pull/360)): The [`files` entry type](https://github.com/Materials-Consortia/OPTIMADE/blob/v1.2.0/optimade.rst#files-entries) has been added to provide a robust way of linking entries to arbitrary file-based data relevant to the entry, such as alternative crystal structure representation formats, input or output files from computational procedures, or experimental data files. - **Symmetry operation specification and space group fields** ([#480](https://github.com/Materials-Consortia/OPTIMADE/pull/480), [#405](https://github.com/Materials-Consortia/OPTIMADE/pull/405), [#464](https://github.com/Materials-Consortia/OPTIMADE/pull/464)): Several fields have been added to the `structures` entry type to fully describe symmetry. Symmetry operations can be provided explicitly in `space_group_symmetry_operations_xyz`, as well as space group specifications in various forms: `space_group_symbol_hall`, `space_group_symbol_hermann_mauguin`, `space_group_symbol_hermann_mauguin_extended`, `space_group_it_number`. -- **Database licenses** ([#414](https://github.com/Materials-Consortia/OPTIMADE/pull/414)): Several fields have been added to programmatically describe the licensing status of data served by OPTIMADE APIs. A database-wide license can be provided as a set of [SPDX identifiers](https://spdx.org/licenses/) in `available_licenses`, with a related field `available_licenses_for_entries` specifying specifically which licenses are available for data contained within the database. The global field `license` can be used point to a human-readable license page (external or otherwise) that explains any caveats to the licensing arrangement. +- **Database licenses** ([#414](https://github.com/Materials-Consortia/OPTIMADE/pull/414),[#497](https://github.com/Materials-Consortia/OPTIMADE/pull/497)): Several fields have been added to programmatically describe the licensing status of data served by OPTIMADE APIs. A database-wide license can be provided as a set of [SPDX identifiers](https://spdx.org/licenses/) in `available_licenses`, with a related field `available_licenses_for_entries` specifying specifically which licenses apply to individual entries and other non-substantial extracts from the database for cases where these differs from the database-wide licenses. The global field `license` can be used point to a human-readable license page (external or otherwise) that explains any caveats to the licensing arrangement. - **Database metadata field** ([#424](https://github.com/Materials-Consortia/OPTIMADE/pull/424)): Added an additional metadata field `database` for providing a human-readable description of a given database. - **Backoff time** ([#411](https://github.com/Materials-Consortia/OPTIMADE/pull/411)): Databases can now request ahead of time that clients apply a particular rate limit or back-off time via the `request_delay` metadata field. From c238d51964be610009c96bcebf47a6f3c931fba2 Mon Sep 17 00:00:00 2001 From: Matthew Evans <7916000+ml-evs@users.noreply.github.com> Date: Mon, 8 Apr 2024 12:28:50 +0100 Subject: [PATCH 15/15] Apply suggestions from code review Co-authored-by: Antanas Vaitkus --- CHANGELOG.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2715369c1..744781bef 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,7 +3,7 @@ ## v1.2.0 (April 2024) -This release adds significant but optional new functionality to the specification, as well as providing several clarifications to existing behaviour. +This release adds significant but optional new functionality to the specification; it also provides several clarifications to existing behaviour. Minor OPTIMADE releases are always intended to be backwards-compatible for clients, meaning that any client code written for v1.1 should continue to work. Although the majority of features added in this release are optional for servers, there are a couple of mandatory format additions that are discussed below in the notes for implementations. @@ -21,7 +21,7 @@ In addition to the specification document itself, machine-readable schemas from - **Per entry/property metadata** ([#463](https://github.com/Materials-Consortia/OPTIMADE/pull/463)): Added a mechanism for providing metadata specific to a given entry or property. - **Files endpoint** ([#360](https://github.com/Materials-Consortia/OPTIMADE/pull/360)): The [`files` entry type](https://github.com/Materials-Consortia/OPTIMADE/blob/v1.2.0/optimade.rst#files-entries) has been added to provide a robust way of linking entries to arbitrary file-based data relevant to the entry, such as alternative crystal structure representation formats, input or output files from computational procedures, or experimental data files. - **Symmetry operation specification and space group fields** ([#480](https://github.com/Materials-Consortia/OPTIMADE/pull/480), [#405](https://github.com/Materials-Consortia/OPTIMADE/pull/405), [#464](https://github.com/Materials-Consortia/OPTIMADE/pull/464)): Several fields have been added to the `structures` entry type to fully describe symmetry. Symmetry operations can be provided explicitly in `space_group_symmetry_operations_xyz`, as well as space group specifications in various forms: `space_group_symbol_hall`, `space_group_symbol_hermann_mauguin`, `space_group_symbol_hermann_mauguin_extended`, `space_group_it_number`. -- **Database licenses** ([#414](https://github.com/Materials-Consortia/OPTIMADE/pull/414),[#497](https://github.com/Materials-Consortia/OPTIMADE/pull/497)): Several fields have been added to programmatically describe the licensing status of data served by OPTIMADE APIs. A database-wide license can be provided as a set of [SPDX identifiers](https://spdx.org/licenses/) in `available_licenses`, with a related field `available_licenses_for_entries` specifying specifically which licenses apply to individual entries and other non-substantial extracts from the database for cases where these differs from the database-wide licenses. The global field `license` can be used point to a human-readable license page (external or otherwise) that explains any caveats to the licensing arrangement. +- **Database licenses** ([#414](https://github.com/Materials-Consortia/OPTIMADE/pull/414),[#497](https://github.com/Materials-Consortia/OPTIMADE/pull/497)): Several fields have been added to programmatically describe the licensing status of data served by OPTIMADE APIs. A database-wide license can be provided as a set of [SPDX identifiers](https://spdx.org/licenses/) in `available_licenses`, with a related field `available_licenses_for_entries` specifying specifically which licenses apply to individual entries and other non-substantial extracts from the database for cases where these differ from the database-wide licenses. The global field `license` can be used point to a human-readable license page (external or otherwise) that explains any caveats to the licensing arrangement. - **Database metadata field** ([#424](https://github.com/Materials-Consortia/OPTIMADE/pull/424)): Added an additional metadata field `database` for providing a human-readable description of a given database. - **Backoff time** ([#411](https://github.com/Materials-Consortia/OPTIMADE/pull/411)): Databases can now request ahead of time that clients apply a particular rate limit or back-off time via the `request_delay` metadata field.