Python must be familiar to everyone, and you may be tired of even arguments about its usefulness or usefulness. However, as a popular language, it still has its uniqueness. Today we will talk about Python again. At the end of the article, there is a technical exchange group. Welcome to participate. Welcome to collect and like.

Python is a dynamic strongly typed language

The book fluent Python mentioned that if a language rarely implicitly converts types, it means that it is a strongly typed language. For example, Java, C + + and python are strongly typed languages.

△ strong type embodiment of Python

At the same time, if a language often implicitly converts types, it indicates that it is a weakly typed language, and PHP, JavaScript and Perl are weakly typed languages.

△ dynamic weakly typed language: JavaScript

Of course, the above simple example comparison does not accurately say that Python is a strongly typed language, because Java also supports integer and string addition operations, and Java is a strongly typed language. Therefore, fluent Python also defines static types and dynamic types: the language for checking types at compile time is static type language, and the language for checking types at run time is dynamic type language. Static languages require type declarations (some modern languages use type derivation to avoid partial type declarations).

To sum up, it is obvious that Python is a dynamic strongly typed language and there is no dispute.

Type Hints

Python in PEP 484 (Python Enhancement Proposals)[ https://www.python.org/dev/peps/pep-0484/ ]Type Hints is proposed in. It further strengthens the feature that Python is a strongly typed language, which was first introduced in Python 3.5. Using Type Hints allows us to write Python code with types, which looks more in line with the strongly typed language style.

Two greeting functions are defined here:

• The common wording is as follows:

• The common wording is as follows:

name = "world"

def greeting(name):
    return "Hello " + name

greeting(name)

• Type Hints is added as follows:

name: str = "world"

def greeting(name: str) -> str:
    return "Hello " + name

greeting(name)

Take PyCharm as an example. In the process of writing code, the IDE will check the type of parameters passed to the function according to the type annotation of the function. If the argument type is found to be inconsistent with the formal parameter type label of the function, the following prompt will appear:

Type Hints writing method of common data structures

The above shows the usage of Type Hints through a greeting function. Next, we will learn more about the writing method of Type Hints of common Python data structures.

Default parameters

Python functions support default parameters. The following is the writing method of Type Hints for default parameters. You only need to write the type between variables and default parameters.

def greeting(name: str = "world") -> str:
    return "Hello " + name

greeting()

Custom type

For custom types, Type Hints can also be well supported. Its writing is no different from Python built-in types.

class Student(object):
    def __init__(self, name, age):
        self.name = name
        self.age = age


def student_to_string(s: Student) -> str:
    return f"student name: {s.name}, age: {s.age}."

student_to_string(Student("Tim", 18))

When a type is labeled as a custom type, the IDE can also check the type.

Container type

When we want to add a type annotation to a built-in container type, a syntax error will be raised because the type annotation operator [] represents a slicing operation in Python. Therefore, you cannot directly use the built-in container type as an annotation. You need to import the corresponding container type annotation from the typing module (usually the initial capital form of the built-in type).

from typing import List, Tuple, Dict

l: List[int] = [1, 2, 3]

t: Tuple[str, ...] = ("a", "b")

d: Dict[str, int] = {
    "a": 1,
    "b": 2,
}

However, PEP 585[https://www.python.org/dev/peps/pep-0585/ ]To solve this problem, we can directly use Python's built-in types without syntax errors.

l: list[int] = [1, 2, 3]

t: tuple[str, ...] = ("a", "b")

d: dict[str, int] = {
    "a": 1,
    "b": 2,
}

Type alias

Some complex nested types are long to write. If they are repeated, it will be painful and the code will not be neat enough.

config: list[tuple[str, int], dict[str, str]] = [
    ("127.0.0.1", 8080),
    {
        "MYSQL_DB": "db",
        "MYSQL_USER": "user",
        "MYSQL_PASS": "pass",
        "MYSQL_HOST": "127.0.0.1",
        "MYSQL_PORT": "3306",
    },
]

def start_server(config: list[tuple[str, int], dict[str, str]]) -> None:
    ...

start_server(config)

This can be solved by aliasing the type, similar to variable naming.

Config = list[tuple[str, int], dict[str, str]]


config: Config = [
    ("127.0.0.1", 8080),
    {
        "MYSQL_DB": "db",
        "MYSQL_USER": "user",
        "MYSQL_PASS": "pass",
        "MYSQL_HOST": "127.0.0.1",
        "MYSQL_PORT": "3306",
    },
]

def start_server(config: Config) -> None:
    ...

start_server(config)

This makes the code look much more comfortable.

Variable parameters

A very flexible feature of Python functions is that they support variable parameters. Type Hints also supports the type annotation of variable parameters.

def foo(*args: str, **kwargs: int) -> None:
    ...

foo("a", "b", 1, x=2, y="c")

The IDE can still check it out.

generic paradigm

The use of dynamic languages requires the support of generics. Type Hints also provides a variety of solutions for generics.

TypeVar

TypeVar can receive any type.

from typing import TypeVar

T = TypeVar("T")

def foo(*args: T, **kwargs: T) -> None:
    ...

foo("a", "b", 1, x=2, y="c")

Union

If you don't want to use generics and only want to use several specified types, you can use Union. For example, defining the concat function only wants to receive str or bytes types.

from typing import Union

T = Union[str, bytes]

def concat(s1: T, s2: T) -> T:
    return s1 + s2

concat("hello", "world")
concat(b"hello", b"world")
concat("hello", b"world")
concat(b"hello", "world")

The check prompt of IDE is as follows:

Difference between TypeVar and Union

TypeVar can not only receive generics, but also be used like Union. You only need to pass the type range you want to specify as parameters in turn during instantiation. Unlike Union, functions declared with TypeVar must have the same multi parameter types, and union does not impose restrictions.

from typing import TypeVar

T = TypeVar("T", str, bytes)

def concat(s1: T, s2: T) -> T:
    return s1 + s2

concat("hello", "world")
concat(b"hello", b"world")
concat("hello", b"world")

The following is the IDE prompt when using TypeVar as a qualified type:

Optional

Type Hints provides optional as the shorthand form of Union[X, None], indicating that the marked parameter is either of type X or None. Optional[X] is equivalent to Union[X, None].

from typing import Optional, Union

# None => type(None)
def foo(arg: Union[int, None] = None) -> None:
    ...


def foo(arg: Optional[int] = None) -> None:
    ...

Any

Any is a special type that can represent all types. Functions that do not specify the return value and parameter type implicitly use any by default, so the following two greeting functions are written equivalently:

from typing import Any

def greeting(name):
    return "Hello " + name


def greeting(name: Any) -> Any:
    return "Hello " + name

When we want to use Type Hints to write static types without losing the unique flexibility of dynamic languages, we can use Any.

When Any type value is assigned to a more precise type, type checking is not performed. The following code IDE will not have an error prompt:

from typing import Any

a: Any = None
a = []  # Dynamic language characteristics
a = 2

s: str = ''
s = a  # Any type value is assigned to a more precise type

Callable objects (functions, classes, etc.)

Any Callable type in Python can be annotated with Callable. In the following code annotation, Callable[[int], str], [int] represents the parameter list of Callable types, and str represents the return value.

from typing import Callable

def int_to_str(i: int) -> str:
    return str(i)

def f(fn: Callable[[int], str], i: int) -> str:
    return fn(i)

f(int_to_str, 2)

Self reference

When we need to define a Tree structure, we often need self reference. When executed to__ init__ Method, the Tree type has not been generated, so it cannot be labeled directly like using the built-in type str. you need to use the string form "Tree" to reference the non generated object.

class Tree(object):
    def __init__(self, left: "Tree" = None, right: "Tree" = None):
        self.left = left
        self.right = right

tree1 = Tree(Tree(), Tree())

The IDE can also check self reference types.

This form can be used not only for self reference, but also for pre reference.

Duck type

A notable feature of Python is its extensive application of duck types. Type Hints provides a Protocol to support duck types. When defining a class, you only need to inherit the Protocol to declare an Interface type. When you encounter the annotation of the Interface type, as long as the received object implements all methods of the Interface type, you can pass the check of the type annotation, and the IDE will not report an error. The Stream here does not need to explicitly inherit the Interface class, but only needs to implement the close method.

from typing import Protocol

class Interface(Protocol):
    def close(self) -> None:
        ...

# class Stream(Interface):
class Stream:
    def close(self) -> None:
        ...

def close_resource(r: Interface) -> None:
    r.close()

f = open("a.txt")
close_resource(f)

s: Stream = Stream()
close_resource(s)

Since both the file object and Stream object returned by the built-in open function implement the close method, it can pass the check of Type Hints, while the string "s" does not implement the close method, so the IDE will prompt the type error.

Other ways of writing Type Hints

In fact, there is not only one way to write Type Hints. Python also implements two other ways to be compatible with different people's preferences and the migration of old code.

Write with comments

Take a look at an example of the tornado framework (tornado/web.py). It is applicable to the modification of existing projects. The code has been written, and the type annotation needs to be added later.

Write in a separate file (. pyi)

You can create a new. pyi file with the same name as. py in the same directory as the source code, and the IDE can also automatically check the type. The advantage of this is that the original code can be completely decoupled without any change. The disadvantage is that it is equivalent to maintaining two copies of code at the same time.

Type Hints practice

Basically, the common writing methods of Type Hints in daily coding have been introduced to you. Let's take a look at how to apply Type Hints in actual coding.

dataclass -- data class

dataclass is a decorator that can decorate classes to add magic methods to classes, such as__ init__ () and__ repr__ () etc., it is in PEP 557[https://www.python.org/dev/peps/pep-0557/ ]Defined in.

from dataclasses import dataclass, field


@dataclass
class User(object):
    id: int
    name: str
    friends: list[int] = field(default_factory=list)


data = {
    "id": 123,
    "name": "Tim",
}

user = User(**data)
print(user.id, user.name, user.friends)
# > 123 Tim []

The above code written using dataclass is equivalent to the following code:

class User(object):
    def __init__(self, id: int, name: str, friends=None):
        self.id = id
        self.name = name
        self.friends = friends or []


data = {
    "id": 123,
    "name": "Tim",
}

user = User(**data)
print(user.id, user.name, user.friends)
# > 123 Tim []

Note: the data class does not check the field type.

It can be found that using dataclass to write classes can reduce a lot of duplicate template code and make the syntax clearer.

Pydantic

Pydantic is a third-party library based on Python Type Hints. It provides data validation, serialization and document functions. It is a library worthy of learning and reference. Here is a sample code using pydantic:

from datetime import datetime
from typing import Optional

from pydantic import BaseModel


class User(BaseModel):
    id: int
    name = 'John Doe'
    signup_ts: Optional[datetime] = None
    friends: list[int] = []


external_data = {
    'id': '123',
    'signup_ts': '2021-09-02 17:00',
    'friends': [1, 2, '3'],
}
user = User(**external_data)

print(user.id)
# > 123
print(repr(user.signup_ts))
# > datetime.datetime(2021, 9, 2, 17, 0)
print(user.friends)
# > [1, 2, 3]
print(user.dict())
"""
{
    'id': 123,
    'signup_ts': datetime.datetime(2021, 9, 2, 17, 0),
    'friends': [1, 2, 3],
    'name': 'John Doe',
}
"""

Note: Pydantic will perform a mandatory check on the field type.

Pydantic is written very similar to dataclass, but it does more extra work and provides very convenient methods such as. dict().

Let's take another example of Pydantic data validation. When the parameters received by the User class do not meet the expectations, a ValidationError exception will be thrown. The exception object provides a. json() method to view the cause of the exception.

from pydantic import ValidationError

try:
    User(signup_ts='broken', friends=[1, 2, 'not number'])
except ValidationError as e:
    print(e.json())
"""
[
  {
    "loc": [
      "id"
    ],
    "msg": "field required",
    "type": "value_error.missing"
  },
  {
    "loc": [
      "signup_ts"
    ],
    "msg": "invalid datetime format",
    "type": "value_error.datetime"
  },
  {
    "loc": [
      "friends",
      2
    ],
    "msg": "value is not a valid integer",
    "type": "type_error.integer"
  }
]
"""

All error reporting information is saved in a list, and the error reporting of each field is saved in a nested dict, where loc identifies the abnormal field and error reporting location, msg is the error reporting prompt information, and type is the error reporting type, so that the whole error reporting reason is clear at a glance.

MySQLHandler

MySQLHandler[https://github.com/jianghushinian/python-scripts/blob/main/scripts/mysql_handler_type_hints.py ]It is my encapsulation of the pymysql library to support calling the execute method using the with syntax and replacing the query result from tuple with object. It is also an application of Type Hints.

class MySQLHandler(object):
    """MySQL handler"""

    def __init__(self):
        self.conn = pymysql.connect(
            host=DB_HOST,
            port=DB_PORT,
            user=DB_USER,
            password=DB_PASS,
            database=DB_NAME,
            charset=DB_CHARSET,
            client_flag=CLIENT.MULTI_STATEMENTS,  # execute multi sql statements
        )
        self.cursor = self.conn.cursor()

    def __del__(self):
        self.cursor.close()
        self.conn.close()

    @contextmanager
    def execute(self):
        try:
            yield self.cursor.execute
            self.conn.commit()
        except Exception as e:
            self.conn.rollback()
            logging.exception(e)

    @contextmanager
    def executemany(self):
        try:
            yield self.cursor.executemany
            self.conn.commit()
        except Exception as e:
            self.conn.rollback()
            logging.exception(e)

    def _tuple_to_object(self, data: List[tuple]) -> List[FetchObject]:
        obj_list = []
        attrs = [desc[0] for desc in self.cursor.description]
        for i in data:
            obj = FetchObject()
            for attr, value in zip(attrs, i):
                setattr(obj, attr, value)
            obj_list.append(obj)
        return obj_list

    def fetchone(self) -> Optional[FetchObject]:
        result = self.cursor.fetchone()
        return self._tuple_to_object([result])[0] if result else None

    def fetchmany(self, size: Optional[int] = None) -> Optional[List[FetchObject]]:
        result = self.cursor.fetchmany(size)
        return self._tuple_to_object(result) if result else None

    def fetchall(self) -> Optional[List[FetchObject]]:
        result = self.cursor.fetchall()
        return self._tuple_to_object(result) if result else None

Run time type check

Type Hints is called Hints instead of Check because it is only a type prompt rather than a real Check. The type Hints usage demonstrated above is actually the function of the IDE to help us complete the type Check. However, in fact, the type Check of the IDE does not determine whether an error is reported during code execution, but can only perform the function of syntax Check prompt in the static period.

To enforce type checking during code execution, we need to write our own code or introduce a third-party library (such as Pydantic described above). Now I pass a type_ The check function implements the runtime dynamic check type for your reference:

from inspect import getfullargspec
from functools import wraps
from typing import get_type_hints


def type_check(fn):
    @wraps(fn)
    def wrapper(*args, **kwargs):
        fn_args = getfullargspec(fn)[0]
        kwargs.update(dict(zip(fn_args, args)))
        hints = get_type_hints(fn)
        hints.pop("return", None)
        for name, type_ in hints.items():
            if not isinstance(kwargs[name], type_):
                raise TypeError(f"expected {type_.__name__}, got {type(kwargs[name]).__name__} instead")
        return fn(**kwargs)

    return wrapper


# name: str = "world"
name: int = 2

@type_check
def greeting(name: str) -> str:
    return str(name)

print(greeting(name))
# > TypeError: expected str, got int instead

Just type the greeting function_ Check decorator to realize run-time type check.

appendix

If you want to continue to learn more about using Python Type Hints, here are some open source projects I recommend for your reference:

• Pydantic [https://github.com/samuelcolvin/pydantic]

• FastAPI [https://github.com/tiangolo/fastapi]

• Tornado [https://github.com/tornadoweb/tornado]

• Flask [https://github.com/pallets/flask]

• Chia-pool [https://github.com/Chia-Network/pool-reference]

• MySQLHandler [https://github.com/jianghushinian/python-scripts/blob/main/scripts/mysql_handler_type_hints.py]

Technical exchange

Welcome to reprint, collect, gain, praise and support!

At present, a technical exchange group has been opened, with more than 2000 group friends. The best way to add notes is: source + Interest direction, which is convenient to find like-minded friends

• Method ① send the following pictures to wechat, long press identification, and the background replies: add group;
• Mode ②. Add micro signal: dkl88191, remarks: from CSDN
• WeChat search official account: Python learning and data mining, background reply: add group