How to handle Callable with covariant argument as class member variable.

[binishkaspar]

Imagine building a web app which has different handler for different endpoints

from dataclasses import dataclass
from typing import Callable, Generic, List, Optional, TypeVar


@dataclass
class Request:
    raw: Optional[str] = None


RT = TypeVar('RT', bound=Request, covariant=True)


@dataclass
class NamedRequest(Request):
    name: Optional[str] = None


def handler1(req: Request) -> None:
    print('Req', req.raw)


def handler2(req: NamedRequest) -> None:
    print('Req', req.raw)


@dataclass
class Endpoint:
    handler: Callable[[RT], None]

    def assign_handler(self, h: Callable[[RT], None]) -> 'Endpoint':
        self.handler = h
        return self


@dataclass
class Webapp:
    endpoints: List[Endpoint]


Webapp([
    # Case 1
    Endpoint(handler1),

    # Case 2 - throws typing error
    Endpoint(handler2),

    # Case 2 - doesn't throw typing error
    Endpoint(None).assign_handler(handler2),
])

Here you can see the case 2 is throwing typing error but the same type doesn't throw typing error when passed as function argument (case 3).

I understand, we can solve this by introducing Generic but that add bit more complexity,

@dataclass
class EndpointG(Generic[RT]):
    handler: Callable[[RT], None]


@dataclass
class WebappG:
    endpoints: List[EndpointG]


# Works as expected
WebappG(
    [
        # Case 1
        EndpointG(handler1),
        # Case 2
        EndpointG(handler2),
    ]
)

# Throws typing error
WebappG(
    [
        # Case 1
        EndpointG(handler1),
    ] + [
        # Case 2 - throws typing error
        EndpointG(handler2),
    ]
)

To solve this typing error, I need to cast and merge list to WebappG. Is there any other better way to handle this use case?

I hope this can be solved with ParamSpec but it's available only on Python 3.10. Is there any other better way to handle this use case?

[srittau]

@dataclass
class Endpoint:
    handler: Callable[[RT], None]

    def assign_handler(self, h: Callable[[RT], None]) -> 'Endpoint':
        self.handler = h
        return self

One note: RT as a type var has no meaning here, since Endpoint is not generic and RT is only used once in both handler and assign_handler(). The code above is basically identical to:

@dataclass
class Endpoint:
    handler: Callable[[Request], None]

    def assign_handler(self, h: Callable[[Request], None]) -> 'Endpoint':
        self.handler = h
        return self

And of course Endpoint.handler is incompatible with a callable expecting a NamedRequest as it's not safe to call that using a Request.

(Also, I would recommend to change the return type of the callable to Any, as it is ignored and you don't care if something is returned.)

Using a generic is the best solution, in my opinion. Using the following should work:

WebappG(
    [
        EndpointG(handler1),
        EndpointG(handler2),
    ]
)

So should the following if you want to construct the list:

endpoints: list[EndpointG[Any]] = [EndpointG(handler1)]
endpoints.extend([EndpointG(handler2)])

WebappG(endpoints)

[binishkaspar]

Thank you!

[JelleZijlstra]

Not a full answer, but I wanted to respond to this part:

I hope this can be solved with ParamSpec but it's available only on Python 3.10.

ParamSpec is actually available for all Python >=3.5 through the typing_extensions project. I think we skipped Python 2 but we can add Python 2 support if people are willing to contribute the code.

In general this is true for all non-syntactic changes to typing: we backport them and typecheckers will support them regardless of Python version.

[binishkaspar]

Thank you. I didn't realise ParamSpec is available from 3.5.