Subclass registration with generic decorator subclass fails

[Darkdragon84]

• cattrs version: 24.1.0
• Python version: 3.11.9
• Operating System: Ubuntu 22.04

Description

I am using the decorator pattern to generate a decorator class Decorated from an abstract base class Base.

from abc import abstractmethod
from attrs import define


@define
class Base:
    @abstractmethod
    def meth(self): ...


@define
class Decorated(Base):
    base: Base

    def meth(self):
        pass

Since Base has many more subclasses and is widely used as an annotation I want to register all subclasses of Base with my converter.

from cattrs import Converter
from cattrs.strategies import include_subclasses

converter = Converter()
include_subclasses(Base, converter)

This works fine if Decorated is not a generic class, but as soon as I make the Base typed field generic,

from typing import Generic, TypeVar

T = TypeVar("T", bound=Base)

@define
class Decorated(Base, Generic[T]):
    base: T

    def meth(self):
        pass

include_subclasses(Base, converter)

the subclass registration for Base fails with

Traceback (most recent call last):
  File "/home/valentin/python/pCloudPython/misc/cattrs_playground/decorator_pattern_subclass_registration.py", line 38, in <module>
    include_subclasses(Base, converter)
  File "/home/valentin/.pyenv/versions/3.11.9/envs/os11/lib/python3.11/site-packages/cattrs/strategies/_subclasses.py", line 75, in include_subclasses
    _include_subclasses_without_union_strategy(
  File "/home/valentin/.pyenv/versions/3.11.9/envs/os11/lib/python3.11/site-packages/cattrs/strategies/_subclasses.py", line 117, in _include_subclasses_without_union_strategy
    dis_fn = converter._get_dis_func(subclass_union, overrides=overrides)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/valentin/.pyenv/versions/3.11.9/envs/os11/lib/python3.11/site-packages/cattrs/converters.py", line 976, in _get_dis_func
    return create_default_dis_func(
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/valentin/.pyenv/versions/3.11.9/envs/os11/lib/python3.11/site-packages/cattrs/disambiguators.py", line 61, in create_default_dis_func
    overrides = [
                ^
  File "/home/valentin/.pyenv/versions/3.11.9/envs/os11/lib/python3.11/site-packages/cattrs/disambiguators.py", line 62, in <listcomp>
    getattr(converter.get_structure_hook(c), "overrides", {}) for c in classes
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/valentin/.pyenv/versions/3.11.9/envs/os11/lib/python3.11/site-packages/cattrs/converters.py", line 574, in get_structure_hook
    self._structure_func.dispatch(type)
  File "/home/valentin/.pyenv/versions/3.11.9/envs/os11/lib/python3.11/site-packages/cattrs/dispatch.py", line 134, in dispatch_without_caching
    res = self._function_dispatch.dispatch(typ)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/valentin/.pyenv/versions/3.11.9/envs/os11/lib/python3.11/site-packages/cattrs/dispatch.py", line 76, in dispatch
    return handler(typ)
           ^^^^^^^^^^^^
  File "/home/valentin/.pyenv/versions/3.11.9/envs/os11/lib/python3.11/site-packages/cattrs/converters.py", line 1290, in gen_structure_attrs_fromdict
    return make_dict_structure_fn(
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/valentin/.pyenv/versions/3.11.9/envs/os11/lib/python3.11/site-packages/cattrs/gen/__init__.py", line 772, in make_dict_structure_fn
    return make_dict_structure_fn_from_attrs(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/valentin/.pyenv/versions/3.11.9/envs/os11/lib/python3.11/site-packages/cattrs/gen/__init__.py", line 349, in make_dict_structure_fn_from_attrs
    raise StructureHandlerNotFoundError(
cattrs.errors.StructureHandlerNotFoundError: Missing type for generic argument T, specify it when structuring.

This happens at registration time, not at (un)structuring time.

Is this expected/intended? Can this be mitigated?

[Tinche]

Hello, sorry for the delay, I have a baby nowdays.

The problem is I don't think include_subclasses is a good fit for generic classes. cattrs can structure a Decorated[int] for you easily, but it can't really structure a Decorated[T] because it doesn't know what T is. (Even if I made provisions to treat it as Decorated[Any], the behavior wouldn't be all that useful.)

I don't really know how to improve the situation.

[Darkdragon84]

Hi! First of all congratulations!! Thanks for the reply and sorry for my late reply. I understand very well, I have two little kids too 😁

🤔 I probably don't understand the mechanics enough, but why is it not possible to just register the generic class with unknown T and sort of register a fully specified unstructure hook upon unstructuring a concrete fully specified type, e.g. Decorated[int]? The general Decorated case could be caught by the existing handler (which doesn't know T) yet, which in turn dispatches a concrete version for Decorated[int].

Is the problem, that subclass registration should be closed in itself, i.e. there should be nothing unknown and left undone once it's called?

[Tinche]

Not sure what exactly you're suggesting. The problem here is structuring, unstructuring should work just fine (and this is the error you see - StructureHandlerNotFoundError). The strategy fails because it's attempting to set up both structuring and unstructuring. If you need just unstructuring, we can probably accomodate this.

When speaking about structuring, the problem is this: assume the strategy worked, what would converter.structure(payload, Base) return in the Decorated case? A Decorated[Any]?

You could do something like this:

converter = Converter()

converter.register_structure_hook(
    Decorated, make_dict_structure_fn(Decorated[Any], converter)
)
include_subclasses(Base, converter)

print(converter.structure({"base": 1}, Base))

This will perform no structuring/validation on the contents of the base field (that's the default Any behavior) but structure everything else.