wrapt-decorated method does not strip self due to positional-or-keyword `self` and ParamSpec invariance

[injust]

Summary

ty isn't happy when wrapt-decorated methods are used as an attrs converter. This is something that basedpyright is okay with.

The pass_through decorator comes from https://wrapt.readthedocs.io/en/latest/typing.html#function-decorators.

Environment: Python 3.14.5 with attrs==26.1.0 and wrapt==2.2.1

---

from typing import TYPE_CHECKING, Any, ParamSpec, TypeVar, reveal_type

import wrapt
from attrs import define, field

if TYPE_CHECKING:
    from collections.abc import Callable

    P = ParamSpec("P")
    R = TypeVar("R")


@wrapt.decorator
def pass_through(wrapped: Callable[P, R], instance: Any, args: tuple[Any, ...], kwargs: dict[str, Any]) -> R:
    return wrapped(*args, **kwargs)


class Util:
    def converter(self, s: str) -> int:
        return int(s)

    @pass_through
    def decorated_converter(self, s: str) -> int:
        return int(s)


util = Util()
reveal_type(util.converter)
reveal_type(util.decorated_converter)


@define
class Foo:
    val: int = field(converter=util.converter)


@define
class Bar:
    val: int = field(converter=util.decorated_converter)

---

info[revealed-type]: Revealed type
  --> test.py:28:13
   |
28 | reveal_type(util.converter)
   |             ^^^^^^^^^^^^^^ `bound method Util.converter(s: str) -> int`
   |

info[revealed-type]: Revealed type
  --> test.py:29:13
   |
29 | reveal_type(util.decorated_converter)
   |             ^^^^^^^^^^^^^^^^^^^^^^^^ `BoundFunctionWrapper[(self, s: str), int]`
   |

error[no-matching-overload]: No overload of function `field` matches arguments
  --> test.py:39:16
   |
39 |     val: int = field(converter=util.decorated_converter)
   |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
info: First overload defined here
  --> .venv/lib/python3.14/site-packages/attrs/__init__.pyi:69:1
   |
69 | / @overload
70 | | def field(
71 | |     *,
72 | |     default: None = ...,
73 | |     validator: None = ...,
74 | |     repr: _ReprArgType = ...,
75 | |     hash: bool | None = ...,
76 | |     init: bool = ...,
77 | |     metadata: Mapping[Any, Any] | None = ...,
78 | |     converter: None = ...,
79 | |     factory: None = ...,
80 | |     kw_only: bool | None = ...,
81 | |     eq: bool | None = ...,
82 | |     order: bool | None = ...,
83 | |     on_setattr: _OnSetAttrArgType | None = ...,
84 | |     alias: str | None = ...,
85 | |     type: type | None = ...,
86 | | ) -> Any: ...
   | |_____________^ First overload defined here
   |
info: Possible overloads for function `field`:
info:   (*, default: None = ..., validator: None = ..., repr: bool | ((Any, /) -> str) = ..., hash: bool | None = ..., init: bool = ..., metadata: Mapping[Any, Any] | None = ..., converter: None = ..., factory: None = ..., kw_only: bool | None = ..., eq: bool | None = ..., order: bool | None = ..., on_setattr: ((Any, Attribute[Any], Any, /) -> Any) | list[(Any, Attribute[Any], Any, /) -> Any] | _NoOpType | None = ..., alias: str | None = ..., type: type | None = ...) -> Any
info:   [_T](*, default: None = ..., validator: ((Any, Attribute[_T], _T, /) -> Any) | Sequence[(Any, Attribute[_T], _T, /) -> Any] | None = ..., repr: bool | ((Any, /) -> str) = ..., hash: bool | None = ..., init: bool = ..., metadata: Mapping[Any, Any] | None = ..., converter: ((Any, /) -> Any) | Converter[Any, Any] | list[((Any, /) -> Any) | Converter[Any, Any]] | tuple[((Any, /) -> Any) | Converter[Any, Any], ...] | None = ..., factory: (() -> _T) | None = ..., kw_only: bool | None = ..., eq: bool | ((Any, /) -> Any) | None = ..., order: bool | ((Any, /) -> Any) | None = ..., on_setattr: ((Any, Attribute[Any], Any, /) -> Any) | list[(Any, Attribute[Any], Any, /) -> Any] | _NoOpType | None = ..., alias: str | None = ..., type: type | None = ...) -> _T
info:   [_T](*, default: _T, validator: ((Any, Attribute[_T], _T, /) -> Any) | Sequence[(Any, Attribute[_T], _T, /) -> Any] | None = ..., repr: bool | ((Any, /) -> str) = ..., hash: bool | None = ..., init: bool = ..., metadata: Mapping[Any, Any] | None = ..., converter: ((Any, /) -> Any) | Converter[Any, Any] | list[((Any, /) -> Any) | Converter[Any, Any]] | tuple[((Any, /) -> Any) | Converter[Any, Any], ...] | None = ..., factory: (() -> _T) | None = ..., kw_only: bool | None = ..., eq: bool | ((Any, /) -> Any) | None = ..., order: bool | ((Any, /) -> Any) | None = ..., on_setattr: ((Any, Attribute[Any], Any, /) -> Any) | list[(Any, Attribute[Any], Any, /) -> Any] | _NoOpType | None = ..., alias: str | None = ..., type: type | None = ...) -> _T
info:   [_T](*, default: _T | None = ..., validator: ((Any, Attribute[_T], _T, /) -> Any) | Sequence[(Any, Attribute[_T], _T, /) -> Any] | None = ..., repr: bool | ((Any, /) -> str) = ..., hash: bool | None = ..., init: bool = ..., metadata: Mapping[Any, Any] | None = ..., converter: ((Any, /) -> Any) | Converter[Any, Any] | list[((Any, /) -> Any) | Converter[Any, Any]] | tuple[((Any, /) -> Any) | Converter[Any, Any], ...] | None = ..., factory: (() -> _T) | None = ..., kw_only: bool | None = ..., eq: bool | ((Any, /) -> Any) | None = ..., order: bool | ((Any, /) -> Any) | None = ..., on_setattr: ((Any, Attribute[Any], Any, /) -> Any) | list[(Any, Attribute[Any], Any, /) -> Any] | _NoOpType | None = ..., alias: str | None = ..., type: type | None = ...) -> Any

Found 3 diagnostics

Version

ty v0.0.43 via uv check

[carljm]

Thanks for the very interesting report!

The key clue here is the revealed type BoundFunctionWrapper[(self, s: str), int] -- we aren't stripping the self argument. Which makes it an invalid attrs field converter, since a converter must be able to accept just one argument. So attrs is kind of incidental to the core problem here. Here's a version without attrs just showing the error on the assignment in the final line:

from collections.abc import Callable
from typing import Any, ParamSpec, TypeVar, reveal_type

import wrapt

P = ParamSpec("P")
R = TypeVar("R")

@wrapt.decorator
def passthrough(
    wrapped: Callable[P, R],
    instance: Any,
    args: tuple[Any, ...],
    kwargs: dict[str, Any],
) -> R:
    return wrapped(*args, **kwargs)

class Util:
    @passthrough
    def convert(self, value: str) -> int:
        return int(value)

convert = Util().convert
reveal_type(convert)  # ty reveals a signature with `self`, pyright without

converter: Callable[[str], int] = convert  # ty errors here, pyright does not

The place where wrapt wants us to strip the self argument is in these overloads. The reason we don't is that a) Concatenate creates a positional-only parameter, and b) FunctionWrapper is invariant in its ParamSpec type parameter. Because of that we don't consider FunctionWrapper[(self, s: str), int] assignable to FunctionWrapper[Concatenate[_T2, _P2], _R2] in either of those first two overloads. We end up trying to match FunctionWrapper[(self, s: str)] against FunctionWrapper[(t2, /, s: str)]; the former signature is assignable to the latter, but the latter signature is not assignable to the former (due to the positional-only parameter), so given invariance (which requires equivalence, that is, assignability both ways) we consider it a non-match, and therefore we fall back to the third overload, which does not strip the initial parameter.

It may be a little clearer if we also eliminate wrapt and simplify to this example:

from collections.abc import Callable
from typing import Concatenate, Generic, ParamSpec, TypeVar, reveal_type

P = ParamSpec("P")
Q = ParamSpec("Q")
R = TypeVar("R", covariant=True)
T = TypeVar("T")

class Wrapper(Generic[P, R]):
    pass

def wrap(function: Callable[P, R]) -> Wrapper[P, R]:
    raise NotImplementedError

def remove_first(
    wrapper: Wrapper[Concatenate[T, Q], R],
    first: T,
) -> Wrapper[Q, R]:
    raise NotImplementedError

class Util:
    def convert(self, value: str) -> int:
        return int(value)

wrapped = wrap(Util.convert)
reveal_type(wrapped)

bound = remove_first(wrapped, Util())
reveal_type(bound)

Playground

The problem goes away if we make self explicitly positional-only: https://play.ty.dev/3141cbf7-5913-4d05-ac8a-cad2f4cc71ee (and this is also true for the original example with wrapt and attrs). So I guess that provides a practical workaround.

Ultimately I think ty is correct here, and pyright is not correctly implementing the specified invariance of ParamSpec. I think the best option here would be to explicitly support variance of ParamSpec, and then wrapt could possibly make FunctionWrapper covariantcontravariant in its paramspec.

But I also think that perhaps we should always consider the self argument of a method to be implicitly defined as positional-only? And that would make this example work unmodified.

@AlexWaygood @JelleZijlstra very interested in your thoughts here.

[carljm]

This example (taken from the typing spec examples for Concatenate) shows that ty does generally support using Concatenate in this way to add/remove parameters, even if they are not explicitly marked as positional-only. The issue only occurs when the ParamSpec is invariant in a container type, as in the wrapt case.

[AlexWaygood]

But I also think that perhaps we should always consider the self argument of a method to be implicitly defined as positional-only? And that would make this example work unmodified.

We already do this for protocol methods, but I think we held off from doing this in general for the reason that it does not accurately reflect the runtime, where it's of course possible to pass self by keyword if you really want to:

>>> class Foo:
...     def method(self):
...         print('wooooo')
...
>>> Foo.method(self=Foo())
wooooo

whether that's something ty should/needs to support, though, is another question.

Ultimately I do agree that ty's behaviour seems correct here right now, to be honest. But I do wonder if full-scope bidirectional inference could solve this? In this playground example you posted above, the wrapped variable is solved as Wrapper[(self, value: str), int], which is correct if you don't consider any context from later statements in that scope. But an equally correct inference for that variable would be Wrapper[(self, /, value: str), int] (the callable type (self, /, value: str), int is just a less precise supertype of (self, value: str), int, so it's valid to infer wrapped as having type Wrapper[(self, /, value: str), int] just like it's valid to infer [42] as having type list[object]). With full-scope bidirectional inference, would we be able to use the type context from the later remove_first call to bidirectionally infer that we need to infer wrapped as having a more general type? Or is that a crazy idea?

[carljm]

Yes, I think that's true, but I'm not sure that full-scope bidi inference for all user generic types (as opposed to just for builtin collection literals) is even remotely within the realm of performance feasibility.

[AlexWaygood]

Another thing to note is that if you explicitly annotate wrapper, of course, we are already able to use the type context to solve the ParamSpec to the type that avoids the error. But that doesn't really help the original use case of the wrapt decorator, because there's of course no way to provide type context to specify what you want the type of a decorated function to be solved as being post-decoration

[injust]

The problem goes away if we make self explicitly positional-only: play.ty.dev/3141cbf7-5913-4d05-ac8a-cad2f4cc71ee (and this is also true for the original example with wrapt and attrs). So I guess that provides a practical workaround.

Interesting! But unfortunately, the example I provided is simplified, and my actual code uses a class method. Seems like the positional-only self workaround doesn't work in that case.

[JelleZijlstra]

Since you asked for my thoughts:

• I'm fine with treating self as implicitly positional-only; it's not always strictly correct but highly unlikely to cause real problems. In typeshed we already don't annotate self as positional-only, even for C methods where it in fact is.
• ParamSpec variance is almost ready to be accepted I believe; I need to do a round of cajoling to get it across the finish line. Perhaps ty should already implement it?

[carljm]

• Perhaps ty should already implement it?

I just realized we already do: marking Wrapper contravariant in its ParamSpec fixes the standalone repro. So no action needed there in ty; we just need to get it into the spec in order to make a case to wrapt that it's worth marking the typevar this way.

[carljm]

Interesting! But unfortunately, the example I provided is simplified, and my actual code uses a class method. Seems like the positional-only self workaround doesn't work in that case.

Looks like with a classmethod, the problem then becomes this overload in wrapt.

When we match the classmethod callable against Callable[_P, _R] | Callable[Concatenate[type[_T], _P], _R], it can match either arm of the union. Pyright chooses the second arm of the union (the "more precise" match), and thus strips the cls parameter already at decoration time, not at descriptor-access time. We match the first arm of the union (Callable[_P, _R]) and thus don't strip the first argument.

Wrapt's stubs here are not accurate to its runtime behavior: at runtime the cls parameter is actually removed by later descriptor access, not eagerly by decoration. Nonetheless, taking wrapt's stubs at face value, I think pyright's generic solver is behaving better here by choosing the more precisely matching union arm. This is already tracked in #2870.