What does Python mean by closures?

Keywords: Python IPython Programming Attribute

Closure is a common concept in programming language, but to be honest, the term is a bit obscure. After checking the information on the Internet for half a day, I still don't know what to do.

What I wonder is: what kind of problem is this thing used to solve? Or what is his role?

First of all, function

After consulting many materials, the summary has the following functions:

This local variable can still be accessed outside the function of a local variable
Global variables can be avoided to reduce the possible impact (many articles call this: save the current running environment)
You can turn a multi parameter function into a single parameter function (most articles call this: you can get different results based on local variables in the external scope)

Here are some examples to illustrate these functions.

1. Visit

In [33]: def test_1():
    ...:     name = 'Tom'
    ...:

In [35]: name
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-35-9bc0cb2ed6de> in <module>
----> 1 name

NameError: name 'name' is not defined

In [48]: def test_2():
    ...:     name = 'Tom'
    ...:     def test_3():
    ...:         print(name)
    ...:     return test_3

In [49]: test = test_2()

In [50]: test
Out[50]: <function __main__.test_2.<locals>.test_3()>

In [51]: print(test)
<function test_2.<locals>.test_3 at 0x000001F9195D5EE8>

In [52]: test()
Tom

As you can see in the first example, there is no way to get the value of name;

In the second example, you can get the value of name, which is not directly accessed.

The principle of implementation is to use the scope of local variables. Since the local variable name cannot be accessed outside, then access it from the function and indirectly get the value.

In the second example, test is called_ At 2 (), a closure is generated: test_3(). This place is actually a typical form of closure, but it still seems obscure. So I think, in terms of function, the logic is smoother.

So some people call closure as follows:

A closure is a function that carries state, and its state can be completely hidden from the outside world.

2. Global parameters can be avoided

If the function's function is to append/pop a list and return the result after N steps, we can do this:

In [54]: nums = [1 ,2, 3, 4]

In [55]: def append_num(x):
    ...:     nums.append(x)
    ...:

In [56]: append_num(5)

In [57]: nums
Out[57]: [1, 2, 3, 4, 5]

In this requirement, the most important step is to save the intermediate results of execution, so global variables are needed.

But obviously, this implementation is not elegant. When the project is large or many people work together, global variables may have disastrous consequences. At the same time, the operation of global variables in the function will lead to uncontrollable process.

At this point, closures are a good choice. In other words, closures can store intermediate variables in similar requirements, as follows:

In [59]: def append_nums(nums):
    ...:     def append_x(x):
    ...:         nums.append(x)
    ...:         return nums
    ...:     return append_x
    ...:

In [60]: tmp = append_nums([1])

In [61]: tmp.append_x(2)
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-61-f86fca8c96cf> in <module>
----> 1 tmp.append_x(2)

AttributeError: 'function' object has no attribute 'append_x'

In [62]: tmp(2)
Out[62]: [1, 2]

In [63]: tmp(3)
Out[63]: [1, 2, 3]

In [64]: tmp
Out[64]: <function __main__.append_nums.<locals>.append_x(x)>

As you can see, after declaring tmp, you can directly pass parameters to tmp as you would call a function. At the same time, each execution is append on the result of the last execution, which achieves our goal and looks elegant at the same time.

If you look at tmp again, you will find that tmp itself is a function, so the call is the same as the function.

Of course, there is also an obvious benefit: different num can be declared, and they do not interfere with each other.

In [65]: tmp_1 = append_nums([3])

In [66]: tmp_1(0)
Out[66]: [3, 0]

Does it feel like the relationship between class and object~

In general, closures are more appropriate and simpler than classes when there is only one internal method.

At the same time, if compared with the example in 1, does the sense of reusability improve a lot?

There's a more common example of a game of chess and cards

Take a similar board game as an example. Suppose the size of the chessboard is 50 * 50, and the upper left corner is the origin of the coordinate system (0,0). I need a function to receive two parameters, namely direction and step, which control the movement of the chessboard. In addition to the direction and step length, the new coordinates of a chess piece's motion also depend on the original coordinate points. By using closure, the original coordinates of the chess piece can be maintained.

origin = [0, 0] # Origin of coordinate system 
legal_x = [0, 50] # Legal coordinates in x-axis direction 
legal_y = [0, 50] # Legal coordinates of y-axis direction 

def create(pos): 
 def player(direction,step): 
  # First of all, we should judge the validity of the parameter direction and step. For example, direction can't walk sideways, step can't be negative, etc 
  # Then we need to judge the validity of the newly generated x, y coordinates. Here we mainly want to introduce the closure, so we won't write it in detail. 
  new_x = pos[0] + direction[0]*step 
  new_y = pos[1] + direction[1]*step 
  pos[0] = new_x 
  pos[1] = new_y 
  #be careful! Can't write POS = [new] here_ x, new_ y] , for the reasons mentioned above 
  return pos 
 return player 
  
player = create(origin) # Create the chess player with the starting point as the origin 
print player([1,0],10) # Move 10 steps to the positive direction of x axis 
print player([0,1],20) # Move 20 steps to the positive direction of y axis 
print player([-1,0],10) # Move 10 steps to the negative direction of x axis

3. Reduce parameters

This can be seen in the above example:

After using the closure feature, declaring variables, and then using it, you only need to pass the parameters of the functions in it.

Let's talk about definition

Wikipedia:

In computer science, Closure is the abbreviation of Lexical Closure, which refers to the function of free variable. This referenced free variable will exist with this function, even if it has left the environment in which it was created. So, there is another way of saying that a Closure is an entity composed of a function and its associated reference environment.

What are the functions that reference free variables?

def print_msg():
    msg = "python"
    def printer():
        print(msg)
    return printer

test = print_msg()
test()

The printer in this example is actually a function that references free variables / external temporary variables.

Generally speaking, MSG belongs to print_ The local variable of MSG can only be printed again_ Called when msg() is executed.

But in the above example, when test is declared, it is print_ When msg() executes, but when test() executes, we still get the value of MSG. This is a feature of closures. Now let's see if the example in 1 is clearer?

When do I use the closure? In addition to the above examples, there is a more common one: decorator. More specifically, the decorator is an application scenario of closures.

def make_wrap(func):
    def wrapper(*args):
        print("before function")
        func(*args)
        print("after function")
    return wrapper

@make_wrap
def print_msg(msg):
    print(msg)

>>> print_msg("Hello")
before function
Hello
after function

Finally, some pits

1. Closures cannot modify local variables of external functions

In [69]: def outer():
    ...:     x = 1
    ...:     def inner():
    ...:         x = 2
    ...:         print(x)
    ...:     print(x)
    ...:     inner()
    ...:     print(x)
    ...:

In [70]: outer()
1
2
1

2. Closures cannot directly access local variables of external functions

In [71]: def outer():
    ...:     x = 1
    ...:     def inner():
    ...:         x = x + 2
    ...:         return x
    ...:     return inner
    ...:

In [72]: f = outer()

In [73]: f()
---------------------------------------------------------------------------
UnboundLocalError                         Traceback (most recent call last)
<ipython-input-73-c43e34e6d405> in <module>
----> 1 f()

<ipython-input-71-4b6c486c99d4> in inner()
      2     x = 1
      3     def inner():
----> 4         x = x + 2
      5         return x
      6     return inner

UnboundLocalError: local variable 'x' referenced before assignment

That is, the inner function can access x, but cannot change its value. The reason is that Python was designed in this way..

Python is designed in this way. It thinks that in the function body, if there is an assignment operation to a variable, it will prove that the variable is a local variable, and it will only read data from the local variable. In this way, we can avoid getting the value of global variable without knowing it, which leads to some wrong data.
Fluent Python

So what do we have to do? There are two methods in Python 3

In [74]: def outer():
    ...:     x = 1
    ...:     def inner(x=x):
    ...:         x = x + 2
    ...:         return x
    ...:     return inner
    ...:

In [75]:

In [75]: f = outer()

In [76]: f()
Out[76]: 3

In [80]: def outer():
    ...:     x = 1
    ...:     def inner():
    ...:         nonlocal x
    ...:         x = x + 1
    ...:         return x
    ...:     return inner
    ...:

In [81]: outer()()
Out[81]: 2

3. In the circulatory body, there is no concept of scope

In [82]: nums = []

In [83]: for i in range(3):
    ...:     def f():
    ...:         return i
    ...:     nums.append(f)
    ...:

In [84]: nums
Out[84]: [<function __main__.f()>, <function __main__.f()>, <function __main__.f()>]

In [85]: nums[1]
Out[85]: <function __main__.f()>

In [86]: nums[1]()
Out[86]: 2

In [87]: nums[0]()
Out[87]: 2

In [88]: nums[2]()
Out[88]: 2

The ideal result is 0,1,2, and the actual result is 2,2,2.

This is actually a feature of Python. It's very common. Lazy loading. It is not executed when it is declared, but only when it is called.

Of course, it's also a good solution:

# Direct assignment
In [90]: nums = []

In [91]: for i in range(3):
    ...:     def f():
    ...:         return i
    ...:     nums.append(f())
    ...:
    ...:

In [92]: nums[0]
Out[92]: 0

In [93]: nums[1]
Out[93]: 1

In [94]: nums[2]
Out[94]: 2

# As in the above example, you can use the following method
In [97]: nums = []

In [98]: for i in range(3):
    ...:     def f(i=i):
    ...:         return i
    ...:     nums.append(f)
    ...:

In [99]: nums[0]
Out[99]: <function __main__.f(i=0)>

In [100]: nums[0]()
Out[100]: 0

In [101]: nums[1]()
Out[101]: 1

In [102]: nums[2]()
Out[102]: 2