Skip to content

Commit

Permalink
PEP 728: Improve specification
Browse files Browse the repository at this point in the history 
- Specify that extra_items and closed are also supported with the
  functional syntax.
- Rewrite the rules for `closed=True` and inheritance. I attempted
  to make `closed=True` exactly equivalent to `extra_items=Never` in
  terms of inheritance. The semantics as specified in the previous
  version of the PEP felt harder to understand and less consistent.
- Fix some incorrect comments regarding expected type checker errors.
- Clarify section on assignability with Mapping
- Add section on runtime behavior. I tried to make the intended runtime
  behavior simple to implement and understand. This makes the runtime
  simpler but may make life more complicated for tools consuming the
  metadata.
  • Loading branch information
@JelleZijlstra
JelleZijlstra committed Dec 13, 2024
1 parent 9f2b319 commit 0c40686
Showing 1 changed file with 78 additions and 42 deletions.
120 changes: 78 additions & 42 deletions peps/pep-0728.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ Another possible use case for this is a sound way to
class Movie(TypedDict):
name: str
director: str

class Book(TypedDict):
name: str
author: str
Expand Down Expand Up @@ -194,12 +194,12 @@ to the ``extra_items`` argument. For example::

class Movie(TypedDict, extra_items=bool):
name: str

a: Movie = {"name": "Blade Runner", "novel_adaptation": True} # OK
b: Movie = {
"name": "Blade Runner",
"year": 1982, # Not OK. 'int' is not assignable to 'bool'
}
}

Here, ``extra_items=bool`` specifies that items other than ``'name'``
have a value type of ``bool`` and are non-required.
Expand All @@ -214,12 +214,12 @@ the ``extra_items`` argument::
def f(movie: Movie) -> None:
reveal_type(movie["name"]) # Revealed type is 'str'
reveal_type(movie["novel_adaptation"]) # Revealed type is 'bool'

``extra_items`` is inherited through subclassing::

class MovieBase(TypedDict, extra_items=int | None):
name: str

class Movie(MovieBase):
year: int

Expand All @@ -234,38 +234,46 @@ Here, ``'year'`` in ``a`` is an extra key defined on ``Movie`` whose value type
is ``int``. ``'other_extra_key'`` in ``b`` is another extra key whose value type
must be assignable to the value of ``extra_items`` defined on ``MovieBase``.

``extra_items`` is also supported with the functional syntax::

Movie = TypedDict("Movie", {"name": str}, extra_items=int | None)

The ``closed`` Class Parameter
------------------------------

When ``closed=True`` is set, no extra items are allowed. This is a shorthand for
When ``closed=True`` is set, no extra items are allowed. This is equivalent to
``extra_items=Never``, because there can't be a value type that is assignable to
:class:`~typing.Never`.
:class:`~typing.Never`. It is a runtime error to use the ``closed`` and
``extra_items`` parameters in the same TypedDict definition.

Similar to ``total``, only a literal ``True`` or ``False`` is supported as the
value of the ``closed`` argument; ``closed`` is ``False`` by default, which
preserves the previous TypedDict behavior.
value of the ``closed`` argument. Type checkers should reject any non-literal value.

Passing ``closed=False`` explicitly requests the default TypedDict behavior,
where arbitrary other keys may be present and subclasses may add arbitrary items.
It is a type checker error to pass ``closed=False`` if a superclass has
``closed=True`` or sets ``extra_items``.

The value of ``closed`` is not inherited through subclassing, but the
implicitly set ``extra_items=Never`` is. It should be an error to use the
default ``closed=False`` when subclassing a closed TypedDict type::
If ``closed`` is not provided, the behavior is inherited from the superclass.
If the superclass is TypedDict itself or the superclass does not have ``closed=True``
or the ``extra_items`` parameter, the previous TypedDict behavior is preserved:
arbitrary extra items are allowed. If the superclass has ``closed=True``, the
child class is also closed.

class BaseMovie(TypedDict, closed=True):
name: str

class MovieA(BaseMovie): # Not OK. An explicit 'closed=True' is required
class MovieA(BaseMovie): # OK, still closed
pass

class MovieB(BaseMovie, closed=True): # OK
class MovieB(BaseMovie, closed=True): # OK, but redundant
pass

Setting both ``closed`` and ``extra_items`` when defining a TypedDict type
should always be a runtime error::

class Person(TypedDict, closed=False, extra_items=bool): # Not OK. 'closed' and 'extra_items' are incompatible
name: str
class MovieC(BaseMovie, closed=False): # Type checker error
pass

As a consequence of ``closed=True`` being equivalent to ``extra_items=Never``.
The same rules that apply to ``extra_items=Never`` should also apply to
As a consequence of ``closed=True`` being equivalent to ``extra_items=Never``,
the same rules that apply to ``extra_items=Never`` also apply to
``closed=True``. It is possible to use ``closed=True`` when subclassing if the
``extra_items`` argument is a read-only type::

Expand All @@ -275,7 +283,7 @@ The same rules that apply to ``extra_items=Never`` should also apply to
class MovieClosed(Movie, closed=True): # OK
pass

class MovieNever(Movie, extra_items=Never): # Not OK. 'closed=True' is preferred
class MovieNever(Movie, extra_items=Never): # OK, but 'closed=True' is preferred
pass

This will be further discussed in
Expand All @@ -286,6 +294,10 @@ is assumed to allow non-required extra items of value type ``ReadOnly[object]``
during inheritance or assignability checks. This preserves the existing behavior
of TypedDict.

``closed`` is also supported with the functional syntax::

Movie = TypedDict("Movie", {"name": str}, closed=True)

Interaction with Totality
-------------------------

Expand Down Expand Up @@ -315,7 +327,7 @@ function parameters still apply::

class Movie(TypedDict, extra_items=int):
name: str

def f(**kwargs: Unpack[Movie]) -> None: ...

# Should be equivalent to:
Expand Down Expand Up @@ -356,7 +368,7 @@ unless it is declared to be ``ReadOnly`` in the superclass::

class Parent(TypedDict, extra_items=int | None):
pass

class Child(Parent, extra_items=int): # Not OK. Like any other TypedDict item, extra_items's type cannot be changed

Second, ``extra_items=T`` effectively defines the value type of any unnamed
Expand Down Expand Up @@ -384,14 +396,14 @@ For example::

class MovieBase(TypedDict, extra_items=int | None):
name: str
class AdaptedMovie(MovieBase): # Not OK. 'bool' is not assignable to 'int | None'

class AdaptedMovie(MovieBase): # Not OK. 'bool' is required
adapted_from_novel: bool

class MovieRequiredYear(MovieBase): # Not OK. Required key 'year' is not known to 'Parent'
year: int | None

class MovieNotRequiredYear(MovieBase): # Not OK. 'int | None' is not assignable to 'int'
class MovieNotRequiredYear(MovieBase): # Not OK. 'int | None' is not consistent with 'int'
year: NotRequired[int]

class MovieWithYear(MovieBase): # OK
Expand Down Expand Up @@ -478,7 +490,7 @@ checks::
class MovieDetails(TypedDict, extra_items=int | None):
name: str
year: NotRequired[int]

details: MovieDetails = {"name": "Kill Bill Vol. 1", "year": 2003}
movie: Movie = details # Not OK. While 'int' is assignable to 'int | None',
# 'int | None' is not assignable to 'int'
Expand All @@ -502,7 +514,7 @@ possible for an item to have a :term:`narrower <typing:narrow>` type than the

class Movie(TypedDict, extra_items=ReadOnly[str | int]):
name: str

class MovieDetails(TypedDict, extra_items=int):
name: str
year: NotRequired[int]
Expand All @@ -522,19 +534,19 @@ enforced::

class MovieExtraStr(TypedDict, extra_items=str):
name: str

extra_int: MovieExtraInt = {"name": "No Country for Old Men", "year": 2007}
extra_str: MovieExtraStr = {"name": "No Country for Old Men", "description": ""}
extra_int = extra_str # Not OK. 'str' is not assignable to extra items type 'int'
extra_str = extra_int # Not OK. 'int' is not assignable to extra items type 'str'

A non-closed TypedDict type implicitly allows non-required extra keys of value
type ``ReadOnly[object]``. Applying the assignability rules between this type
and a closed TypedDict type is allowed::

class MovieNotClosed(TypedDict):
name: str

extra_int: MovieExtraInt = {"name": "No Country for Old Men", "year": 2007}
not_closed: MovieNotClosed = {"name": "No Country for Old Men"}
extra_int = not_closed # Not OK.
Expand Down Expand Up @@ -578,17 +590,19 @@ arguments of this type when constructed by calling the class object::
Interaction with Mapping[KT, VT]
--------------------------------

A TypedDict type can be assignable to ``Mapping[KT, VT]`` types other than
``Mapping[str, object]`` as long as all value types of the items on the
TypedDict type is :term:`typing:assignable` to ``VT``. This is an extension of this
A TypedDict type is assignable to a type of the form ``Mapping[str, VT]``
when all value types of the items in the TypedDict
are :term:`typing:assignable` to ``VT``. For the purpose of this rule, a
TypedDict that does not have ``extra_items=`` or ``closed=`` set is considered
to have an item with a value of type ``object``. This is an extension of this
assignability rule from the `typing spec
<https://typing.readthedocs.io/en/latest/spec/typeddict.html#assignability>`__:

* A TypedDict with all ``int`` values is not :term:`typing:assignable` to
* A (non-closed) TypedDict with all ``int`` values is not :term:`typing:assignable` to
``Mapping[str, int]``, since there may be additional non-``int`` values
not visible through the type, due to :term:`typing:structural`
assignability. These can be accessed using the ``values()`` and
``items()`` methods in ``Mapping``,
``items()`` methods in ``Mapping``.

For example::

Expand All @@ -598,6 +612,10 @@ For example::
extra_str: MovieExtraStr = {"name": "Blade Runner", "summary": ""}
str_mapping: Mapping[str, str] = extra_str # OK

class MovieExtraInt(TypedDict, extra_items=int):
name: str

extra_int: MovieExtraInt = {"name": "Blade Runner", "year": 1982}
int_mapping: Mapping[str, int] = extra_int # Not OK. 'int | str' is not assignable with 'int'
int_str_mapping: Mapping[str, int | str] = extra_int # OK

Expand All @@ -611,7 +629,7 @@ and ``items()`` on such TypedDict types::
Interaction with dict[KT, VT]
-----------------------------

Note that because the presence of ``extra_items`` on a closed TypedDict type
Because the presence of ``extra_items`` on a closed TypedDict type
prohibits additional required keys in its :term:`typing:structural`
:term:`typing:subtypes <subtype>`, we can determine if the TypedDict type and
its structural subtypes will ever have any required key during static analysis.
Expand All @@ -636,8 +654,8 @@ For example::
def f(x: IntDict) -> None:
v: dict[str, int] = x # OK
v.clear() # OK
not_required_num_dict: IntDictWithNum = {"num": 1, "bar": 2}

not_required_num_dict: IntDictWithNum = {"num": 1, "bar": 2}
regular_dict: dict[str, int] = not_required_num_dict # OK
f(not_required_num_dict) # OK

Expand All @@ -652,10 +670,28 @@ because such dict can be a subtype of dict::

class CustomDict(dict[str, int]):
pass

not_a_regular_dict: CustomDict = {"num": 1}
int_dict: IntDict = not_a_regular_dict # Not OK

Runtime behavior
----------------

At runtime, it is an error to pass both the ``closed`` and ``extra_items``
arguments in the same TypedDict definition, whether using the class syntax or
the functional syntax. For simplicity, the runtime does not check other invalid
combinations involving inheritance.

For introspection, the ``closed`` and ``extra_items`` arguments are mapped to
two new attributes on the resulting TypedDict object: ``__closed__`` and
``__extra_items__``. These attributes reflect exactly what was passed to the
TypedDict constructor, without considering superclasses.

If ``closed`` is not passed, the value of ``__closed__`` is None. If ``extra_items``
is not passed, the value of ``__extra_items__`` is the new sentinel object
``typing.NoExtraItems``. (It cannot be ``None``, because ``extra_items=None`` is a
valid definition that indicates all extra items must be ``None``.)

How to Teach This
=================

Expand Down

0 comments on commit 0c40686

Please sign in to comment.