Understanding Decorators in Python
2012-05-02 23:07 by Rollen Holt, 31365 Reading, 8 Comments, Collection, editThis article starts with a problem on stack overflow, if you use the following code:
@makebold @makeitalic def say(): return "Hello"
Print out the following output:
<b><i>Hello<i></b>
What would you do? The final answer is:
def makebold(fn):
def wrapped():
return "<b>" + fn() + "</b>"
return wrapped
def makeitalic(fn):
def wrapped():
return "<i>" + fn() + "</i>"
return wrapped
@makebold
@makeitalic
def hello():
return "hello world"
print hello() ## Return to < b > < I > Hello world </i ></b>
Now let's look at some of the most basic ways to understand Python's decorators. English discussion reference Here.
Decorator is a well-known design pattern, which is often used in scenarios with tangential requirements. The more classical ones are insertion logs, performance testing, transaction processing and so on. Decorator is an excellent design to solve this kind of problem. With decorator, we can extract a large number of identical codes which are not related to the function itself and continue to reuse them. Generally speaking, the function of the decorator is to add additional functions to existing objects.
1.1. How does demand come about?
The definition of decorator is abstract. Let's take a small example.
def foo():
print 'in foo()'
foo()
This is a very boring function, right. But suddenly there's a more boring person, we call him Jun B, said I want to see how long it took to implement this function, all right, then we can do this:
import time
def foo():
start = time.clock()
print 'in foo()'
end = time.clock()
print 'used:', end - start
foo()
Very good. The function looks impeccable. But Mr. B, who is suffering from egg pain, suddenly doesn't want to see this function. He is more interested in another function called foo2.
What shall I do? If you copy the newly added code into foo2, you'll have a big problem. Isn't the difficulty of copying something the most annoying? And what if Jun B continues to look at other functions?
1.2. Changing with unchangeability is also changing.
Remember, functions are first-class citizens in Python, so we can consider redefining a function timeit, passing the reference to foo to him, calling foo in timeit and timing, so that we can achieve the goal of not changing the definition of foo, and no matter how many functions Mr. B has seen, we do not need to modify the definition of function!
import time
def foo():
print 'in foo()'
def timeit(func):
start = time.clock()
func()
end =time.clock()
print 'used:', end - start
timeit(foo)
It doesn't seem to be a problem logically. Everything's fine and working well!... Wait a minute, we seem to have modified the code for the calling part. Originally, we called foo(), which changed to timeit(foo). In this way, if foo is called at N, you have to modify the code at N. Or more extremely, consider that the code invoked somewhere can't modify the situation, such as: this function is for someone else to use.
1.3. Minimize changes!
In this case, let's find a way not to modify the calling code; if we don't modify the calling code, it means that calling foo() needs to produce the effect of calling timeit(foo). We can think of assigning timeit to foo, but timeit seems to have a parameter... Find a way to unify the parameters. If timeit(foo) does not directly produce the call effect, but returns a function that is consistent with the foo parameter list... That's fine. Assign the return value of timeit(foo) to foo, and then call foo() code without modifying at all!
#-*- coding: UTF-8 -*-
import time
def foo():
print 'in foo()'
# Define a timer, pass in one, and return another method with a timer attached
def timeit(func):
# Define a built-in wrapper function that wraps the incoming function with a timer function
def wrapper():
start = time.clock()
func()
end =time.clock()
print 'used:', end - start
# Returns the wrapped function
return wrapper
foo = timeit(foo)
foo()
In this way, a simple timer is ready! We just need to add foo = timeit(foo) before we call foo after defining foo. This is the concept of decorator. It looks like foo is decorated by timeit. In this example, functions enter and exit with a timer called an Aspect, which is called Aspect-Oriented Programming. Compared with the top-down execution of traditional programming habits, it is like inserting a piece of logic horizontally in the process of function execution. In specific business areas, it can reduce a lot of duplicate code. Aspect-oriented programming still has quite a lot of terminology. There is no more introduction here. If you are interested, you can look for relevant information.
This example is only used for demonstration, and does not consider the foo with parameters and return values. The task of improving it is left to you:)
The above code seems to be no longer streamlined, so Python provides a grammatical sugar to reduce the amount of character input.
import time
def timeit(func):
def wrapper():
start = time.clock()
func()
end =time.clock()
print 'used:', end - start
return wrapper
@timeit
def foo():
print 'in foo()'
foo()
Focus on @timeit in line 11, adding this line to the definition is exactly the same as writing foo = timeit(foo) separately. Never assume that @ has another magic. In addition to a little less character input, there is an additional benefit: it looks more decorative.
-------------------
To understand python's decorator, we must first understand that functions are also treated as objects in Python. This is very important. Let's start with an example:
def shout(word="yes") :
return word.capitalize()+" !"
print shout()
# Output:'Yes!'
# As an object, you can assign functions to any other object variable
scream = shout
# Note that we don't use parentheses because we're not calling functions.
# We assign the function shout to scream, which means you can call shout through scream.
print scream()
# Output:'Yes!'
# Also, you can delete the old name shout, but you can still access the function through scream.
del shout
try :
print shout()
except NameError, e :
print e
#Output: "name'shout'is not defined"
print scream()
# Output:'Yes!'
Let's leave this topic aside for a moment. Let's look at another interesting property of python: you can define functions in functions:
def talk() :
# You can define another function in talk.
def whisper(word="yes") :
return word.lower()+"...";
# ... and use it immediately
print whisper()
# Every time you call'talk', whisper defined in talk is also called.
talk()
# Output:
# yes...
# But "whisper" does not exist alone:
try :
print whisper()
except NameError, e :
print e
#Output: "name'whisper'is not defined"*
Function reference
From the above two examples, we can conclude that since a function is an object, so:
1. It can be assigned to other variables.
2. It can be defined in another function.
That is to say, a function can return a function. Look at the following example:
def getTalk(type="shout") :
# Let's define another function.
def shout(word="yes") :
return word.capitalize()+" !"
def whisper(word="yes") :
return word.lower()+"...";
# Then we go back to one of them.
if type == "shout" :
# We did not use (), because we were not calling the function.
# We are returning this function.
return shout
else :
return whisper
# And how to use it?
# Assign the function to a variable
talk = getTalk()
# Here you can see that talk is actually a function object:
print talk
#Output: <function shout at 0xb7ea817c>
# This object is one of the objects returned by the function:
print talk()
# Or you can call it directly as follows:
print getTalk("whisper")()
#Output: yes...
Also, since we can return a function, we can pass it as a parameter to the function:
def doSomethingBefore(func) :
print "I do something before then I call the function you gave me"
print func()
doSomethingBefore(scream)
#Output:
#I do something before then I call the function you gave me
#Yes !
Here you can understand the decorator sufficiently. Others can be regarded as encapsulators. That is, it allows you to execute code before and after decoration without changing the content of the function itself.
Hand decoration
So how to do manual decoration?
# Decorator is a function and its parameter is another function.
def my_shiny_new_decorator(a_function_to_decorate) :
# Another function is defined internally: a wrapper.
# This function encapsulates the original function, so you can execute some code before or after it.
def the_wrapper_around_the_original_function() :
# Put in some code that you want before the real function executes
print "Before the function runs"
# Execute the original function
a_function_to_decorate()
# Put in some code you want after the original function is executed
print "After the function runs"
#At this point, "a_function_to_decrorate" has not yet been executed, and we return the wrapper function we created.
#The wrapper contains functions and the code they execute before and after, and it's ready.
return the_wrapper_around_the_original_function
# Now imagine that you created a function that you would never touch again far away.
def a_stand_alone_function() :
print "I am a stand alone function, don't you dare modify me"
a_stand_alone_function()
#Output: I am a stand alone function, don't you dare modify me
# Well, you can encapsulate it to implement behavior extensions. You can simply throw it to a decorator.
# The decorator will dynamically encapsulate it and the code you want, and return a new available function.
a_stand_alone_function_decorated = my_shiny_new_decorator(a_stand_alone_function)
a_stand_alone_function_decorated()
#Output:
#Before the function runs
#I am a stand alone function, don't you dare modify me
#After the function runs
Now you may ask that every time a_stand_alone_function is called, the actual call is a_stand_alone_function_decorated. The implementation is also simple, and a_stand_alone_function can be reassigned with my_shiny_new_decorator.
a_stand_alone_function = my_shiny_new_decorator(a_stand_alone_function) a_stand_alone_function() #Output: #Before the function runs #I am a stand alone function, don't you dare modify me #After the function runs # And guess what, that's EXACTLY what decorators do !
Decorator Disclosure
In the previous example, we can use the grammar of decorator:
@my_shiny_new_decorator
def another_stand_alone_function() :
print "Leave me alone"
another_stand_alone_function()
#Output:
#Before the function runs
#Leave me alone
#After the function runs
Of course, you can also accumulate decoration:
def bread(func) :
def wrapper() :
print "</''''''\>"
func()
print "<\______/>"
return wrapper
def ingredients(func) :
def wrapper() :
print "#tomatoes#"
func()
print "~salad~"
return wrapper
def sandwich(food="--ham--") :
print food
sandwich()
#Output: -- ham--
sandwich = bread(ingredients(sandwich))
sandwich()
#outputs :
#</''''''\>
# #tomatoes#
# --ham--
# ~salad~
#<\______/>
Using python decorator grammar:
@bread
@ingredients
def sandwich(food="--ham--") :
print food
sandwich()
#Output:
#</''''''\>
# #tomatoes#
# --ham--
# ~salad~
#<\______/>
The order of ornaments is very important. It should be noted that:
@ingredients
@bread
def strange_sandwich(food="--ham--") :
print food
strange_sandwich()
#Output:
##tomatoes#
#</''''''\>
# --ham--
#<\______/>
# ~salad~
Finally, answer the questions mentioned above:
# Decorator makebold is used to convert to bold
def makebold(fn):
# The result returns the function.
def wrapper():
# Insert some code before and after execution
return "<b>" + fn() + "</b>"
return wrapper
# Decorator makeitalic for conversion to Italic
def makeitalic(fn):
# The result returns the function.
def wrapper():
# Insert some code before and after execution
return "<i>" + fn() + "</i>"
return wrapper
@makebold
@makeitalic
def say():
return "hello"
print say()
#Output: < b > < I > Hello </i ></b>
# Equate to
def say():
return "hello"
say = makebold(makeitalic(say))
print say()
#Output: < b > < I > Hello </i ></b>
Built-in ornaments
There are three built-in decorators, staticmethod, classmethod and property, which are used to transform instance methods defined in classes into static methods, class methods and class attributes, respectively. Because functions can be defined in modules, static methods and class methods are not very useful unless you want full object-oriented programming. And attributes are not indispensable. Java also lives well without attributes. From my personal Python experience, I haven't used property, and the frequency of using static method and classmethod is very low.
class Rabbit(object):
def __init__(self, name):
self._name = name
@staticmethod
def newRabbit(name):
return Rabbit(name)
@classmethod
def newRabbit2(cls):
return Rabbit('')
@property
def name(self):
return self._name
The property defined here is a read-only property, and if it needs to be writable, a setter needs to be defined again:
@name.setter
def name(self, name):
self._name = name
functools module
The functools module provides two decorators. This module was added after Python 2.5. Generally speaking, it should be used higher than this version. But my usual working environment is 2.4 T-T
2.3.1. wraps(wrapped[, assigned][, updated]):
This is a very useful ornament. Friends who have read the previous article on reflection should know that functions have several special attributes, such as function names. After being decorated, the function name foo in the example above will become wrapper of wrapper. If you want to use reflection, it may lead to unexpected results. This decorator can solve this problem by preserving the special attributes of decorated functions.
import time
import functools
def timeit(func):
@functools.wraps(func)
def wrapper():
start = time.clock()
func()
end =time.clock()
print 'used:', end - start
return wrapper
@timeit
def foo():
print 'in foo()'
foo()
print foo.__name__
First, notice line 5. If you comment on this line, foo. _name_ will be'wrapper'. In addition, I believe you have noticed that this decorator even has a parameter. In fact, he has two other optional parameters. The attribute names in assigned will be replaced by assignments, while the attribute names in updated will be merged by updates. You can get their default values by looking at the source code of functools. For this decorator, it is equivalent to wrapper = functools.wraps(func)(wrapper).
2.3.2. total_ordering(cls):
This decorator is useful for certain occasions, but it was added after Python 2.7. Its purpose is to implement at least one class of _lt_, _le_, _gt_, _ge_ plus other comparison methods, which is a class decorator. If you find it difficult to understand, you might as well take a closer look at the source code of this ornament:
def total_ordering(cls):
54 """Class decorator that fills in missing ordering methods"""
55 convert = {
56 '__lt__': [('__gt__', lambda self, other: other < self),
57 ('__le__', lambda self, other: not other < self),
58 ('__ge__', lambda self, other: not self < other)],
59 '__le__': [('__ge__', lambda self, other: other <= self),
60 ('__lt__', lambda self, other: not other <= self),
61 ('__gt__', lambda self, other: not self <= other)],
62 '__gt__': [('__lt__', lambda self, other: other > self),
63 ('__ge__', lambda self, other: not other > self),
64 ('__le__', lambda self, other: not self > other)],
65 '__ge__': [('__le__', lambda self, other: other >= self),
66 ('__gt__', lambda self, other: not other >= self),
67 ('__lt__', lambda self, other: not self >= other)]
68 }
69 roots = set(dir(cls)) & set(convert)
70 if not roots:
71 raise ValueError('must define at least one ordering operation: < > <= >=')
72 root = max(roots) # prefer __lt__ to __le__ to __gt__ to __ge__
73 for opname, opfunc in convert[root]:
74 if opname not in roots:
75 opfunc.__name__ = opname
76 opfunc.__doc__ = getattr(int, opname).__doc__
77 setattr(cls, opname, opfunc)
78 return cls