PEP 695 – Type Parameter Syntax
source link: https://peps.python.org/pep-0695/
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
Specification
Type Parameter Declarations
Here is a new syntax for declaring type parameters for generic classes, functions, and type aliases. The syntax adds support for a comma-delimited list of type parameters in square brackets after the name of the class, function, or type alias.
Simple (non-variadic) type variables are declared with an unadorned name.
Variadic type variables are preceded by *
(see PEP 646 for details).
Parameter specifications are preceded by **
(see PEP 612 for details).
# This generic class is parameterized by a TypeVar T, a
# TypeVarTuple Ts, and a ParamSpec P.
class ChildClass[T, *Ts, **P]: ...
There is no need to include Generic
as a base class. Its inclusion as
a base class is implied by the presence of type parameters, and it will
automatically be included in the __mro__
and __orig_bases__
attributes
for the class. The explicit use of a Generic
base class will result in a
runtime error.
class ClassA[T](Generic[T]): ... # Runtime error
A Protocol
base class with type arguments may generate a runtime
error. Type checkers should generate an error in this case because
the use of type arguments is not needed, and the order of type parameters
for the class are no longer dictated by their order in the Protocol
base class.
class ClassA[S, T](Protocol): ... # OK
class ClassB[S, T](Protocol[S, T]): ... # Recommended type checker error
Type parameter names within a generic class, function, or type alias must be unique within that same class, function, or type alias. A duplicate name generates a syntax error at compile time. This is consistent with the requirement that parameter names within a function signature must be unique.
class ClassA[T, *T]: ... # Syntax Error
def func1[T, **T](): ... # Syntax Error
Class type parameter names are mangled if they begin with a double
underscore, to avoid complicating the name lookup mechanism for names used
within the class. However, the __name__
attribute of the type parameter
will hold the non-mangled name.
Upper Bound Specification
For a non-variadic type parameter, an “upper bound” type can be specified
through the use of a type annotation expression. If an upper bound is
not specified, the upper bound is assumed to be object
.
class ClassA[T: str]: ...
The specified upper bound type must use an expression form that is allowed in type annotations. More complex expression forms should be flagged as an error by a type checker. Quoted forward references are allowed.
The specified upper bound type must be concrete. An attempt to use a generic
type should be flagged as an error by a type checker. This is consistent with
the existing rules enforced by type checkers for a TypeVar
constructor call.
class ClassA[T: dict[str, int]]: ... # OK
class ClassB[T: "ForwardReference"]: ... # OK
class ClassC[V]:
class ClassD[T: dict[str, V]]: ... # Type checker error: generic type
class ClassE[T: [str, int]]: ... # Type checker error: illegal expression form
Constrained Type Specification
PEP 484 introduced the concept of a “constrained type variable” which is constrained to a set of two or more types. The new syntax supports this type of constraint through the use of a literal tuple expression that contains two or more types.
class ClassA[AnyStr: (str, bytes)]: ... # OK
class ClassB[T: ("ForwardReference", bytes)]: ... # OK
class ClassC[T: ()]: ... # Type checker error: two or more types required
class ClassD[T: (str, )]: ... # Type checker error: two or more types required
t1 = (bytes, str)
class ClassE[T: t1]: ... # Type checker error: literal tuple expression required
If the specified type is not a tuple expression or the tuple expression includes complex expression forms that are not allowed in a type annotation, a type checker should generate an error. Quoted forward references are allowed.
class ClassF[T: (3, bytes)]: ... # Type checker error: invalid expression form
The specified constrained types must be concrete. An attempt to use a generic
type should be flagged as an error by a type checker. This is consistent with
the existing rules enforced by type checkers for a TypeVar
constructor call.
class ClassG[T: (list[S], str)]: ... # Type checker error: generic type
Runtime Representation of Bounds and Constraints
The upper bounds and constraints of TypeVar
objects are accessible at
runtime through the __bound__
and __constraints__
attributes.
For TypeVar
objects defined through the new syntax, these attributes
become lazily evaluated, as discussed under Lazy Evaluation below.
Generic Type Alias
We propose to introduce a new statement for declaring type aliases. Similar
to class
and def
statements, a type
statement defines a scope
for type parameters.
# A non-generic type alias
type IntOrStr = int | str
# A generic type alias
type ListOrSet[T] = list[T] | set[T]
Type aliases can refer to themselves without the use of quotes.
# A type alias that includes a forward reference
type AnimalOrVegetable = Animal | "Vegetable"
# A generic self-referential type alias
type RecursiveList[T] = T | list[RecursiveList[T]]
The type
keyword is a new soft keyword. It is interpreted as a keyword
only in this part of the grammar. In all other locations, it is assumed to
be an identifier name.
Type parameters declared as part of a generic type alias are valid only when evaluating the right-hand side of the type alias.
As with typing.TypeAlias
, type checkers should restrict the right-hand
expression to expression forms that are allowed within type annotations.
The use of more complex expression forms (call expressions, ternary operators,
arithmetic operators, comparison operators, etc.) should be flagged as an
error.
Type alias expressions are not allowed to use traditional type variables (i.e.
those allocated with an explicit TypeVar
constructor call). Type checkers
should generate an error in this case.
T = TypeVar("T")
type MyList = list[T] # Type checker error: traditional type variable usage
We propose to deprecate the existing typing.TypeAlias
introduced in
PEP 613. The new syntax eliminates its need entirely.
Runtime Type Alias Class
At runtime, a type
statement will generate an instance of
typing.TypeAliasType
. This class represents the type. Its attributes
include:
__name__
is a str representing the name of the type alias__type_params__
is a tuple ofTypeVar
,TypeVarTuple
, orParamSpec
objects that parameterize the type alias if it is generic__value__
is the evaluated value of the type alias
All of these attributes are read-only.
The value of the type alias is evaluated lazily (see Lazy Evaluation below).
Type Parameter Scopes
When the new syntax is used, a new lexical scope is introduced, and this scope includes the type parameters. Type parameters can be accessed by name within inner scopes. As with other symbols in Python, an inner scope can define its own symbol that overrides an outer-scope symbol of the same name. This section provides a verbal description of the new scoping rules. The Scoping Behavior section below specifies the behavior in terms of a translation to near-equivalent existing Python code.
Type parameters are visible to other type parameters declared elsewhere in the list. This allows type parameters to use other type parameters within their definition. While there is currently no use for this capability, it preserves the ability in the future to support upper bound expressions or type argument defaults that depend on earlier type parameters.
A compiler error or runtime exception is generated if the definition of an earlier type parameter references a later type parameter even if the name is defined in an outer scope.
# The following generates no compiler error, but a type checker
# should generate an error because an upper bound type must be concrete,
# and ``Sequence[S]`` is generic. Future extensions to the type system may
# eliminate this limitation.
class ClassA[S, T: Sequence[S]]: ...
# The following generates no compiler error, because the bound for ``S``
# is lazily evaluated. However, type checkers should generate an error.
class ClassB[S: Sequence[T], T]: ...
A type parameter declared as part of a generic class is valid within the class body and inner scopes contained therein. Type parameters are also accessible when evaluating the argument list (base classes and any keyword arguments) that comprise the class definition. This allows base classes to be parameterized by these type parameters. Type parameters are not accessible outside of the class body, including class decorators.
class ClassA[T](BaseClass[T], param = Foo[T]): ... # OK
print(T) # Runtime error: 'T' is not defined
@dec(Foo[T]) # Runtime error: 'T' is not defined
class ClassA[T]: ...
A type parameter declared as part of a generic function is valid within the function body and any scopes contained therein. It is also valid within parameter and return type annotations. Default argument values for function parameters are evaluated outside of this scope, so type parameters are not accessible in default value expressions. Likewise, type parameters are not in scope for function decorators.
def func1[T](a: T) -> T: ... # OK
print(T) # Runtime error: 'T' is not defined
def func2[T](a = list[T]): ... # Runtime error: 'T' is not defined
@dec(list[T]) # Runtime error: 'T' is not defined
def func3[T](): ...
A type parameter declared as part of a generic type alias is valid within the type alias expression.
type Alias1[K, V] = Mapping[K, V] | Sequence[K]
Type parameter symbols defined in outer scopes cannot be bound with
nonlocal
statements in inner scopes.
S = 0
def outer1[S]():
S = 1
T = 1
def outer2[T]():
def inner1():
nonlocal S # OK because it binds variable S from outer1
nonlocal T # Syntax error: nonlocal binding not allowed for type parameter
def inner2():
global S # OK because it binds variable S from global scope
The lexical scope introduced by the new type parameter syntax is unlike
traditional scopes introduced by a def
or class
statement. A type
parameter scope acts more like a temporary “overlay” to the containing scope.
The only new symbols contained
within its symbol table are the type parameters defined using the new syntax.
References to all other symbols are treated as though they were found within
the containing scope. This allows base class lists (in class definitions) and
type annotation expressions (in function definitions) to reference symbols
defined in the containing scope.
class Outer:
class Private:
pass
# If the type parameter scope was like a traditional scope,
# the base class 'Private' would not be accessible here.
class Inner[T](Private, Sequence[T]):
pass
# Likewise, 'Inner' would not be available in these type annotations.
def method1[T](self, a: Inner[T]) -> Inner[T]:
return a
The compiler allows inner scopes to define a local symbol that overrides an outer-scoped type parameter.
Consistent with the scoping rules defined in PEP 484, type checkers should generate an error if inner-scoped generic classes, functions, or type aliases reuse the same type parameter name as an outer scope.
T = 0
@decorator(T) # Argument expression `T` evaluates to 0
class ClassA[T](Sequence[T]):
T = 1
# All methods below should result in a type checker error
# "type parameter 'T' already in use" because they are using the
# type parameter 'T', which is already in use by the outer scope
# 'ClassA'.
def method1[T](self):
...
def method2[T](self, x = T): # Parameter 'x' gets default value of 1
...
def method3[T](self, x: T): # Parameter 'x' has type T (scoped to method3)
...
Symbols referenced in inner scopes are resolved using existing rules except that type parameter scopes are also considered during name resolution.
T = 0
# T refers to the global variable
print(T) # Prints 0
class Outer[T]:
T = 1
# T refers to the local variable scoped to class 'Outer'
print(T) # Prints 1
class Inner1:
T = 2
# T refers to the local type variable within 'Inner1'
print(T) # Prints 2
def inner_method(self):
# T refers to the type parameter scoped to class 'Outer';
# If 'Outer' did not use the new type parameter syntax,
# this would instead refer to the global variable 'T'
print(T) # Prints 'T'
def outer_method(self):
T = 3
# T refers to the local variable within 'outer_method'
print(T) # Prints 3
def inner_func():
# T refers to the variable captured from 'outer_method'
print(T) # Prints 3
When the new type parameter syntax is used for a generic class, assignment
expressions are not allowed within the argument list for the class definition.
Likewise, with functions that use the new type parameter syntax, assignment
expressions are not allowed within parameter or return type annotations, nor
are they allowed within the expression that defines a type alias, or within
the bounds and constraints of a TypeVar
. Similarly, yield
, yield from
,
and await
expressions are disallowed in these contexts.
This restriction is necessary because expressions evaluated within the new lexical scope should not introduce symbols within that scope other than the defined type parameters, and should not affect whether the enclosing function is a generator or coroutine.
class ClassA[T]((x := Sequence[T])): ... # Syntax error: assignment expression not allowed
def func1[T](val: (x := int)): ... # Syntax error: assignment expression not allowed
def func2[T]() -> (x := Sequence[T]): ... # Syntax error: assignment expression not allowed
type Alias1[T] = (x := list[T]) # Syntax error: assignment expression not allowed
Accessing Type Parameters at Runtime
A new read-only attribute called __type_params__
is available on generic classes,
functions, and type aliases. This attribute is a tuple of the
type parameters that parameterize the class, function, or alias.
The tuple contains TypeVar
, ParamSpec
, and TypeVarTuple
instances.
Type parameters declared using the new syntax will not appear within the
dictionary returned by globals()
or locals()
.
Variance Inference
This PEP eliminates the need for variance to be specified for type parameters. Instead, type checkers will infer the variance of type parameters based on their usage within a class. Type parameters are inferred to be invariant, covariant, or contravariant depending on how they are used.
Python type checkers already include the ability to determine the variance of type parameters for the purpose of validating variance within a generic protocol class. This capability can be used for all classes (whether or not they are protocols) to calculate the variance of each type parameter.
The algorithm for computing the variance of a type parameter is as follows.
For each type parameter in a generic class:
1. If the type parameter is variadic (TypeVarTuple
) or a parameter
specification (ParamSpec
), it is always considered invariant. No further
inference is needed.
2. If the type parameter comes from a traditional TypeVar
declaration and
is not specified as infer_variance
(see below), its variance is specified
by the TypeVar
constructor call. No further inference is needed.
3. Create two specialized versions of the class. We’ll refer to these as
upper
and lower
specializations. In both of these specializations,
replace all type parameters other than the one being inferred by a dummy type
instance (a concrete anonymous class that is type compatible with itself and
assumed to meet the bounds or constraints of the type parameter). In
the upper
specialized class, specialize the target type parameter with
an object
instance. This specialization ignores the type parameter’s
upper bound or constraints. In the lower
specialized class, specialize
the target type parameter with itself (i.e. the corresponding type argument
is the type parameter itself).
4. Determine whether lower
can be assigned to upper
using normal type
compatibility rules. If so, the target type parameter is covariant. If not,
determine whether upper
can be assigned to lower
. If so, the target
type parameter is contravariant. If neither of these combinations are
assignable, the target type parameter is invariant.
Here is an example.
class ClassA[T1, T2, T3](list[T1]):
def method1(self, a: T2) -> None:
...
def method2(self) -> T3:
...
To determine the variance of T1
, we specialize ClassA
as follows:
upper = ClassA[object, Dummy, Dummy]
lower = ClassA[T1, Dummy, Dummy]
We find that upper
is not assignable to lower
using normal type
compatibility rules defined in PEP 484. Likewise, lower
is not assignable
to upper
, so we conclude that T1
is invariant.
To determine the variance of T2
, we specialize ClassA
as follows:
upper = ClassA[Dummy, object, Dummy]
lower = ClassA[Dummy, T2, Dummy]
Since upper
is assignable to lower
, T2
is contravariant.
To determine the variance of T3
, we specialize ClassA
as follows:
upper = ClassA[Dummy, Dummy, object]
lower = ClassA[Dummy, Dummy, T3]
Since lower
is assignable to upper
, T3
is covariant.
Auto Variance For TypeVar
The existing TypeVar
class constructor accepts keyword parameters named
covariant
and contravariant
. If both of these are False
, the
type variable is assumed to be invariant. We propose to add another keyword
parameter named infer_variance
indicating that a type checker should use
inference to determine whether the type variable is invariant, covariant or
contravariant. A corresponding instance variable __infer_variance__
can be
accessed at runtime to determine whether the variance is inferred. Type
variables that are implicitly allocated using the new syntax will always
have __infer_variance__
set to True
.
A generic class that uses the traditional syntax may include combinations of type variables with explicit and inferred variance.
T1 = TypeVar("T1", infer_variance=True) # Inferred variance
T2 = TypeVar("T2") # Invariant
T3 = TypeVar("T3", covariant=True) # Covariant
# A type checker should infer the variance for T1 but use the
# specified variance for T2 and T3.
class ClassA(Generic[T1, T2, T3]): ...
Compatibility with Traditional TypeVars
The existing mechanism for allocating TypeVar
, TypeVarTuple
, and
ParamSpec
is retained for backward compatibility. However, these
“traditional” type variables should not be combined with type parameters
allocated using the new syntax. Such a combination should be flagged as
an error by type checkers. This is necessary because the type parameter
order is ambiguous.
It is OK to combine traditional type variables with new-style type parameters if the class, function, or type alias does not use the new syntax. The new-style type parameters must come from an outer scope in this case.
K = TypeVar("K")
class ClassA[V](dict[K, V]): ... # Type checker error
class ClassB[K, V](dict[K, V]): ... # OK
class ClassC[V]:
# The use of K and V for "method1" is OK because it uses the
# "traditional" generic function mechanism where type parameters
# are implicit. In this case V comes from an outer scope (ClassC)
# and K is introduced implicitly as a type parameter for "method1".
def method1(self, a: V, b: K) -> V | K: ...
# The use of M and K are not allowed for "method2". A type checker
# should generate an error in this case because this method uses the
# new syntax for type parameters, and all type parameters associated
# with the method must be explicitly declared. In this case, ``K``
# is not declared by "method2", nor is it supplied by a new-style
# type parameter defined in an outer scope.
def method2[M](self, a: M, b: K) -> M | K: ...
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK