Try to fix up everything to support pydantic v2

Use enum `.value` for comparison (check if this should be used elsewhere in the code for any Enum classes/types). Disable ignoring warning about pydantic v2 deprecations in order to use pydantic v2 methods in remaining places. Consider completely removing the special function used for setting `nullable=true` in JSON schema for 'SHOULD' OPTIMADE support fields. It seems `nullable` is no longer a valid field in OpenAPI v3.1.0, but also, the properties/fields this is added to already support the value being `null`.
Materials-Consortia · Oct 14, 2023 · 9e50994 · 9e50994
1 parent 8cc0dc4
commit 9e50994
Show file tree

Hide file tree

Showing 8 changed files with 40 additions and 25 deletions.
diff --git a/openapi/index_openapi.json b/openapi/index_openapi.json
@@ -1480,7 +1480,8 @@
               }
             ],
             "uniqueItems": true,
-            "title": "Included"
+            "title": "Included",
+            "description": "A list of unique included OPTIMADE entry resources."
           },
           "links": {
             "anyOf": [

diff --git a/openapi/openapi.json b/openapi/openapi.json
@@ -2616,7 +2616,8 @@
               }
             ],
             "uniqueItems": true,
-            "title": "Included"
+            "title": "Included",
+            "description": "A list of unique included OPTIMADE entry resources."
           },
           "links": {
             "anyOf": [
@@ -3437,7 +3438,8 @@
               }
             ],
             "uniqueItems": true,
-            "title": "Included"
+            "title": "Included",
+            "description": "A list of unique included OPTIMADE entry resources."
           },
           "links": {
             "anyOf": [
@@ -3529,7 +3531,8 @@
               }
             ],
             "uniqueItems": true,
-            "title": "Included"
+            "title": "Included",
+            "description": "A list of unique included OPTIMADE entry resources."
           },
           "links": {
             "anyOf": [
@@ -4122,6 +4125,7 @@
             ],
             "title": "Last Modified",
             "description": "Date and time representing when the entry was last modified.\n\n- **Type**: timestamp.\n\n- **Requirements/Conventions**:\n    - **Support**: SHOULD be supported by all implementations, i.e., SHOULD NOT be `null`.\n    - **Query**: MUST be a queryable property with support for all mandatory filter features.\n    - **Response**: REQUIRED in the response unless the query parameter `response_fields` is present and does not include this property.\n\n- **Example**:\n    - As part of JSON response format: `\"2007-04-05T14:30:20Z\"` (i.e., encoded as an [RFC 3339 Internet Date/Time Format](https://tools.ietf.org/html/rfc3339#section-5.6) string.)",
+            "nullable": true,
             "x-optimade-queryable": "must",
             "x-optimade-support": "should"
           },
@@ -4251,6 +4255,7 @@
             ],
             "title": "Dimension Types",
             "description": "List of three integers.\nFor each of the three directions indicated by the three lattice vectors (see property `lattice_vectors`), this list indicates if the direction is periodic (value `1`) or non-periodic (value `0`).\nNote: the elements in this list each refer to the direction of the corresponding entry in `lattice_vectors` and *not* the Cartesian x, y, z directions.\n\n- **Type**: list of integers.\n\n- **Requirements/Conventions**:\n    - **Support**: SHOULD be supported by all implementations, i.e., SHOULD NOT be `null`.\n    - **Query**: Support for queries on this property is OPTIONAL.\n    - MUST be a list of length 3.\n    - Each integer element MUST assume only the value 0 or 1.\n\n- **Examples**:\n    - For a molecule: `[0, 0, 0]`\n    - For a wire along the direction specified by the third lattice vector: `[0, 0, 1]`\n    - For a 2D surface/slab, periodic on the plane defined by the first and third lattice vectors: `[1, 0, 1]`\n    - For a bulk 3D system: `[1, 1, 1]`",
+            "nullable": true,
             "x-optimade-queryable": "optional",
             "x-optimade-support": "should"
           },
@@ -4296,6 +4301,7 @@
             ],
             "title": "Lattice Vectors",
             "description": "The three lattice vectors in Cartesian coordinates, in \u00e5ngstr\u00f6m (\u00c5).\n\n- **Type**: list of list of floats or unknown values.\n\n- **Requirements/Conventions**:\n    - **Support**: SHOULD be supported by all implementations, i.e., SHOULD NOT be `null`.\n    - **Query**: Support for queries on this property is OPTIONAL.\n      If supported, filters MAY support only a subset of comparison operators.\n    - MUST be a list of three vectors *a*, *b*, and *c*, where each of the vectors MUST BE a list of the vector's coordinates along the x, y, and z Cartesian coordinates.\n      (Therefore, the first index runs over the three lattice vectors and the second index runs over the x, y, z Cartesian coordinates).\n    - For databases that do not define an absolute Cartesian system (e.g., only defining the length and angles between vectors), the first lattice vector SHOULD be set along *x* and the second on the *xy*-plane.\n    - MUST always contain three vectors of three coordinates each, independently of the elements of property `dimension_types`.\n      The vectors SHOULD by convention be chosen so the determinant of the `lattice_vectors` matrix is different from zero.\n      The vectors in the non-periodic directions have no significance beyond fulfilling these requirements.\n    - The coordinates of the lattice vectors of non-periodic dimensions (i.e., those dimensions for which `dimension_types` is `0`) MAY be given as a list of all `null` values.\n        If a lattice vector contains the value `null`, all coordinates of that lattice vector MUST be `null`.\n\n- **Examples**:\n    - `[[4.0,0.0,0.0],[0.0,4.0,0.0],[0.0,1.0,4.0]]` represents a cell, where the first vector is `(4, 0, 0)`, i.e., a vector aligned along the `x` axis of length 4 \u00c5; the second vector is `(0, 4, 0)`; and the third vector is `(0, 1, 4)`.",
+            "nullable": true,
             "x-optimade-queryable": "optional",
             "x-optimade-support": "should",
             "x-optimade-unit": "\u00c5"
@@ -4474,7 +4480,8 @@
               }
             ],
             "uniqueItems": true,
-            "title": "Included"
+            "title": "Included",
+            "description": "A list of unique included OPTIMADE entry resources."
           },
           "links": {
             "anyOf": [
@@ -4566,7 +4573,8 @@
               }
             ],
             "uniqueItems": true,
-            "title": "Included"
+            "title": "Included",
+            "description": "A list of unique included OPTIMADE entry resources."
           },
           "links": {
             "anyOf": [

diff --git a/optimade/adapters/base.py b/optimade/adapters/base.py
@@ -156,7 +156,7 @@ def ingest_from(cls, data: Any, format: Optional[str] = None) -> Any:
 
         return cls(
             {
-                "attributes": cls._type_ingesters[format](data).dict(),
+                "attributes": cls._type_ingesters[format](data).model_dump(),
                 "id": "",
                 "type": "structures",
             }
@@ -170,7 +170,7 @@ def _get_model_attributes(
         for res in starting_instances:
             nested_attributes = name.split(".")
             for nested_attribute in nested_attributes:
-                if nested_attribute in getattr(res, "__fields__", {}):
+                if nested_attribute in getattr(res, "model_fields", {}):
                     res = getattr(res, nested_attribute)
                 else:
                     res = None

diff --git a/optimade/models/index_metadb.py b/optimade/models/index_metadb.py
@@ -1,8 +1,7 @@
-# pylint: disable=no-self-argument
 from enum import Enum
 from typing import Dict, Literal, Union
 
-from pydantic import BaseModel, Field  # pylint: disable=no-name-in-module
+from pydantic import BaseModel, Field
 
 from optimade.models.baseinfo import BaseInfoAttributes, BaseInfoResource
 from optimade.models.jsonapi import BaseResource

diff --git a/optimade/models/responses.py b/optimade/models/responses.py
@@ -1,7 +1,7 @@
 # pylint: disable=no-self-argument
 from typing import Any, Dict, List, Optional, Union
 
-from pydantic import Field, model_validator
+from pydantic import model_validator
 
 from optimade.models.baseinfo import BaseInfoResource
 from optimade.models.entries import EntryInfoResource, EntryResource
@@ -66,18 +66,24 @@ class InfoResponse(Success):
 
 
 class EntryResponseOne(Success):
-    data: Union[EntryResource, Dict[str, Any], None] = Field(None)  # type: ignore[assignment]
-    included: Optional[Union[List[EntryResource], List[Dict[str, Any]]]] = Field(  # type: ignore[assignment]
-        None, uniqueItems=True
+    data: Union[EntryResource, Dict[str, Any], None] = None  # type: ignore[assignment]
+    included: Optional[Union[List[EntryResource], List[Dict[str, Any]]]] = StrictField(  # type: ignore[assignment]
+        None,
+        description="A list of unique included OPTIMADE entry resources.",
+        uniqueItems=True,
     )
 
 
 class EntryResponseMany(Success):
-    data: Union[List[EntryResource], List[Dict[str, Any]]] = Field(  # type: ignore[assignment]
-        ..., uniqueItems=True
+    data: Union[List[EntryResource], List[Dict[str, Any]]] = StrictField(  # type: ignore[assignment]
+        ...,
+        description="List of unique OPTIMADE entry resource objects.",
+        uniqueItems=True,
     )
-    included: Optional[Union[List[EntryResource], List[Dict[str, Any]]]] = Field(  # type: ignore[assignment]
-        None, uniqueItems=True
+    included: Optional[Union[List[EntryResource], List[Dict[str, Any]]]] = StrictField(  # type: ignore[assignment]
+        None,
+        description="A list of unique included OPTIMADE entry resources.",
+        uniqueItems=True,
     )
 
 

diff --git a/optimade/models/structures.py b/optimade/models/structures.py
@@ -279,7 +279,7 @@ def structure_json_schema_extra(
 
     1. Constrained types in pydantic do not currently play nicely with
     "Required Optional" fields, i.e. fields must be specified but can be null.
-    The two contrained list fields, `dimension_types` and `lattice_vectors`,
+    The two constrained list fields, `dimension_types` and `lattice_vectors`,
     are OPTIMADE 'SHOULD' fields, which means that they are allowed to be null.
 
     2. All OPTIMADE 'SHOULD' fields are allowed to be null, so we manually set them
@@ -292,7 +292,8 @@ def structure_json_schema_extra(
     nullable_props = (
         prop
         for prop in schema["required"]
-        if schema["properties"][prop].get("x-optimade-support") == SupportLevel.SHOULD
+        if schema["properties"][prop].get("x-optimade-support")
+        == SupportLevel.SHOULD.value
     )
     for prop in nullable_props:
         schema["properties"][prop]["nullable"] = True
@@ -487,7 +488,7 @@ class StructureResourceAttributes(EntryResourceAttributes):
         pattern=CHEMICAL_FORMULA_REGEXP,
     )
 
-    dimension_types: Optional[  # type: ignore[valid-type]
+    dimension_types: Optional[
         Annotated[List[Periodicity], Field(min_length=3, max_length=3)]
     ] = OptimadeField(
         None,
@@ -535,7 +536,7 @@ class StructureResourceAttributes(EntryResourceAttributes):
         queryable=SupportLevel.MUST,
     )
 
-    lattice_vectors: Optional[  # type: ignore[valid-type]
+    lattice_vectors: Optional[
         Annotated[List[Vector3D_unknown], Field(min_length=3, max_length=3)]
     ] = OptimadeField(
         None,
@@ -563,7 +564,7 @@ class StructureResourceAttributes(EntryResourceAttributes):
         queryable=SupportLevel.OPTIONAL,
     )
 
-    cartesian_site_positions: Optional[List[Vector3D]] = OptimadeField(  # type: ignore[valid-type]
+    cartesian_site_positions: Optional[List[Vector3D]] = OptimadeField(
         None,
         description="""Cartesian positions of each site in the structure.
 A site is usually used to describe positions of atoms; what atoms can be encountered at a given site is conveyed by the `species_at_sites` property, and the species themselves are described in the `species` property.

diff --git a/pyproject.toml b/pyproject.toml
@@ -137,6 +137,7 @@ dummy-variable-rgx = "^(_+|(_+[a-zA-Z0-9_]*[a-zA-Z0-9]+?))$"
 plugins = "pydantic.mypy"
 ignore_missing_imports = true
 follow_imports = "skip"
+check-untyped-defs = true
 
 [tool.pytest.ini_options]
 filterwarnings = [
@@ -154,7 +155,6 @@ filterwarnings = [
     "ignore:.*has an unrecognised prefix.*:",
     "ignore:.*pkg_resources is deprecated as an API*:DeprecationWarning",
     "ignore:.*Deprecated call to `pkg_resources.declare_namespace*:DeprecationWarning",
-    "ignore:.*:pydantic.warnings.PydanticDeprecatedSince20",
 ]
 testpaths = "tests"
 addopts = "-rs"

diff --git a/tests/server/entry_collections/test_entry_collections.py b/tests/server/entry_collections/test_entry_collections.py
@@ -21,6 +21,6 @@ def test_get_attribute_fields():
 
     for entry_name, attributes_model in entry_name_attributes.items():
         assert (
-            set(attributes_model.__fields__.keys())
+            set(attributes_model.model_fields.keys())
             == ENTRY_COLLECTIONS[entry_name].get_attribute_fields()
         )