[python magic method] iterator (_iter_ and _next_)

Keywords: Python

Articles Catalogue

There are many functions in python that start and end with which can complete many complex logic codes and improve the simplicity of the code. This paper mainly summarizes the magic methods used by the iterator, and mainly explains them with code examples.

_ iter_ and _next__

In fact, we need to introduce a concept called iterator. It is common that when we use the for statement, python actually uses the built-in function iter on the object after for, such as:

a = [1, 2, 3]
for i in a:
    do_something()

In fact, python has undergone a similar transformation as follows:

a = [1, 2, 3]
for i in iter(a):
    do_something()

What does ITER return, then, is an iteration object that maps mainly to the _iter_ function in the class, which returns an object that implements _next_. Pay attention to understanding this sentence, such as:

class B(object):
    def __next__(self):
        raise StopIteration

class A(object):
    def __iter__(self):
        return B()

We can see that class A implements a _iter_ function, returning an instance object of B(), in which the _next_ function is implemented.

The following concepts are introduced:
Iterable: Iterable: Iterable object, a class, implements _iter_, then it is considered that it has the ability to iterate. Usually this function must return an object that implements _next_. If it implements itself, you can return self, of course, this return value is not necessary.
Iterator: Iterator: Iterator (also Iterable) implements both _iter_ and _next_ objects. The absence of any of these objects is not an Iterator. For example, in the above example, A() can be an Iterable, but A() and B() can not be considered as Iterator, because A only implements _iter_ while B only implements __ next ().

We can use the types in collections to verify:

class B(object):
    def __next__(self):
        raise StopIteration

class A(object):
    def __iter__(self):
        return B()


from collections.abc import *

a = A()
b = B()
print(isinstance(a, Iterable))
print(isinstance(a, Iterator))

print(isinstance(b, Iterable))
print(isinstance(b, Iterator))

The result is:

True
False
False
False

Let's make a slight change to class B:

class B(object):
    def __next__(self):
        raise StopIteration

    def __iter__(self):
        return None

class A(object):
    def __iter__(self):
        return B()


from collections.abc import *

a = A()
b = B()
print(isinstance(a, Iterable))
print(isinstance(a, Iterator))

print(isinstance(b, Iterable))
print(isinstance(b, Iterator))

The result is:

True
False
True
True

Real Iterator

The above is just a few demonstrations. Here is a specific explanation:
When the ITER function is called, an iteration object is generated, requiring _iter_ to return an object that implements _next_. We can access the next element of the object through the next function, and throw a StopIteration exception if you don't want to continue iterating (the for statement captures this). Exceptions, and automatically terminate for), the following implements a function similar to range function.

class MyRange(object):
    def __init__(self, end):
        self.start = 0
        self.end = end

    def __iter__(self):
        return self

    def __next__(self):
        if self.start < self.end:
            ret = self.start
            self.start += 1
            return ret
        else:
            raise StopIteration

from collections.abc import *

a = MyRange(5)
print(isinstance(a, Iterable))
print(isinstance(a, Iterator))

for i in a:
    print(i)

The result is:

True
True
0
1
2
3
4

Next, we use the next function to simulate once:

class MyRange(object):
    def __init__(self, end):
        self.start = 0
        self.end = end

    def __iter__(self):
        return self

    def __next__(self):
        if self.start < self.end:
            ret = self.start
            self.start += 1
            return ret
        else:
            raise StopIteration

a = MyRange(5)
print(next(a))
print(next(a))
print(next(a))
print(next(a))
print(next(a))
print(next(a)) # Actually, it's done here. We're running once to check for exceptions.

One obvious advantage is that every time I generate data, I use one. What does that mean? For example, I need to traverse [0, 1, 2, 3...] to 1 billion. If I use a list, it will all be loaded into memory, but if I use an iterator, I can see that when I use it (i.e., in the Calling next) generates the corresponding number, which saves memory, which is a lazy way to load.

summary

  1. Iterator and Iterable in collection.abs can be used with the isinstance function to determine whether an object is iterative or not.
  2. ITER is actually mapped to _iter_ function
  3. As long as the object that implements _iter_ is an Iterable object, normally an object that implements _next_ should be returned (although this requirement is not mandatory). If you implement _next_, you can also return to yourself.
  4. At the same time, Iterator and Iterator are implemented. Of course, it is also an iterative object, in which _next_ should throw a StopIteration exception after the iteration is completed.
  5. The for statement automatically handles this StopIteration exception to end the for loop

Generator-related documentation is already available Here.

Posted by mattwade on Wed, 21 Aug 2019 04:23:33 -0700