Understanding Incompatible Types in Assignment in Python with mypy Errors

Learn why assigning lists with different element types raises errors in Python's mypy, and how covariant and invariant generics play a role in this behavior.
---

Visit these links for original content and any more details, such as alternate solutions, latest updates/developments on topic, comments, revision history etc. For example, the original title of the Question was: Incompatible types in assignment (expression has type "List[str]", variable has type "List[Union[str, int]]")

If anything seems off to you, please feel free to write me at vlogize [AT] gmail [DOT] com.
---
Understanding Incompatible Types in Assignment in Python with mypy Errors

If you’ve encountered an error when running Python code through mypy that states "Incompatible types in assignment (expression has type 'List[str]', variable has type 'List[Union[str, int]]')", you might find yourself puzzled. This error typically arises when trying to assign a list of one type to a list declared with a broader type. In this guide, we'll dive into the underlying principles of Python typing that lead to this error, and how to resolve it by properly understanding variance in generics.

The Problem: What Causes the Error?

Consider the following code snippet that produces the mypy error:

[[See Video to Reveal this Text or Code Snippet]]

The error message indicates that attempting to assign list1 (which is declared as list[str]) to list2 (declared as list[str | int]) is invalid. At first glance, one might think that since list[str] can only contain strings, it should be a valid subset of list[str | int], which can contain both strings and integers. So, why is this assignment unfriendly?

Understanding Generic Types: Covariance, Contravariance, and Invariance

To fully grasp this issue, we need to explore how generic types in Python work, especially their classifications:

Covariant: A generic type g is covariant if g[u] is a subtype of g[v] when u is a subtype of v. This allows substitution in a type-safe manner.

Contravariant: In this case, g[v] is a subtype of g[u], which implies that a less specific type can be assigned to a more specific one.

Invariant: If neither g[u] nor g[v] is a subtype of the other, the generic type is invariant.

Lists are Invariant

In Python, lists are considered invariant. This means that you cannot substitute one concrete type of list for another safely. Mutation (changing elements inside the list) could lead to inconsistent states, where a list that expects diverse element types could end up with unexpected behavior.

The Solution: Using Sequences

The key to resolving this issue lies in utilizing immutable sequences. Unlike lists, immutable sequences (such as tuples and strings) naturally allow substitution due to their non-mutability. In fact, Sequence is covariant. This means you can safely assign Sequence[str] to Sequence[str | int], as long as you are only reading from it.

Here’s how you can rewrite the previous example using Sequence:

[[See Video to Reveal this Text or Code Snippet]]

By using Sequence, you can bridge the typing gap. Here’s why:

Reading from Sequences: You can safely read from list2, knowing that the type of an element will either be str or int.

Flexible Type Handling: Since list1 doesn't change and its elements are guaranteed to be one of the specified types, Python can ensure that operations safely handle data at runtime.

Conclusion

Understanding the principles of covariance and invariance in Python typing is crucial for proper usage of generics in your code. The error you encounter in mypy serves as a valuable lesson in type safety within Python's type system.

With this understanding, you can avoid similar errors in the future and write more robust, type-safe Python code. Always remember to choose your data structures wisely, especially when dealing with types that are mutable like lists.