Supplementary: Common modules, built-in functions, exception handling

Keywords: Python xml shell JSON

I. shutil module

Advanced File, Folder and Compressed Packet Processing Module

shutil.copyfileobj(fsrc, fdst[, length])
Copy the contents of a file to another file

import shutil
shutil.copyfileobj(open('old.xml','r'), open('new.xml', 'w'))

shutil.copyfile(src, dst)
Copy file

shutil.copymode('f1.log', 'f2.log') #The target file must exist

shutil.copy(src, dst)
Copy files and permissions

import shutil  
shutil.copy('f1.log', 'f2.log')

shutil.copy2(src, dst)
Copy files and status information

import shutil  
shutil.copy2('f1.log', 'f2.log')

shutil.ignore_patterns(*patterns)
shutil.copytree(src, dst, symlinks=False, ignore=None)
Recursive de-copy folders

import shutil
  
shutil.copytree('folder1', 'folder2', ignore=shutil.ignore_patterns('*.pyc', 'tmp*')) #The target directory cannot exist.#Note that the folder 2 directory parent directory should have writable permissions, ignore means excluding 

shutil.rmtree(path[, ignore_errors[, onerror]])
Recursive Delete Files

import shutil  
shutil.rmtree('folder1')

shutil.move(src, dst)
To move files recursively, it is similar to the mv command, in fact, it is renamed.

import shutil  
shutil.move('folder1', 'folder3')

shutil.make_archive(base_name, format,...)

Create compressed packages and return file paths, such as zip, tar

Create compressed packages and return file paths, such as zip, tar

    • base_name: The file name of the compressed package, or the path of the compressed package. If only the file name is saved to the current directory, otherwise it is saved to the specified path.
      For example, data_bak=> is saved to the current path
      For example: / TMP / data_bak = > save to / tmp/
    • format: Compressed package type, "zip", "tar", "bztar", "gztar"
    • root_dir: The folder path to be compressed (default current directory)
    • owner: User, default current user
    • Group: group, default current group
    • logger: Used for logging, usually logging.Logger objects
#1,take /data Pack the files below to place the current program directory
import shutil
ret = shutil.make_archive("data_bak", 'gztar', root_dir='/data')
   
#2,take /data Packing and placing the files below /tmp/Catalog
import shutil
ret = shutil.make_archive("/tmp/data_bak", 'gztar', root_dir='/data') 

Shuil processes compressed packages by calling ZipFile and TarFile modules.

import zipfile

# compress
z = zipfile.ZipFile('laxi.zip', 'w')
z.write('a.log')
z.write('data.data')
z.close()

# decompression
z = zipfile.ZipFile('laxi.zip', 'r')
z.extractall(path='.')
z.close()
zipfile compression and decompression
import tarfile

# compress
>>> t=tarfile.open('/tmp/egon.tar','w')
>>> t.add('/test1/a.py',arcname='a.bak')
>>> t.add('/test1/b.py',arcname='b.bak')
>>> t.close()


# decompression
>>> t=tarfile.open('/tmp/egon.tar','r')
>>> t.extractall('/egon')
>>> t.close()
tarfile compression and decompression

 

2. Sheve module

The shell module is simpler than the pickle module, with only one open function that returns dictionary-like objects that are readable and writable; the key must be a string, and the value can be a data type supported by python

import shelve

f=shelve.open(r'sheve.txt')
# f['stu1_info']={'name':'egon','age':18,'hobby':['piao','smoking','drinking']}
# f['stu2_info']={'name':'gangdan','age':53}
# f['school_info']={'website':'http://www.pypy.org','city':'beijing'}

print(f['stu1_info']['hobby'])
f.close()

 

3. xml module

XML is a protocol for data exchange between different languages or programs, similar to json, but JSON is easier to use. However, in ancient times, in the dark age before JSON was born, people could only choose to use xml. So far, many traditional companies, such as many systems in the financial industry, still mainly use xml.

The format of xml is as follows, which distinguishes data structures by <> nodes:

<?xml version="1.0"?>
<data>
    <country name="Liechtenstein">
        <rank updated="yes">2</rank>
        <year>2008</year>
        <gdppc>141100</gdppc>
        <neighbor name="Austria" direction="E"/>
        <neighbor name="Switzerland" direction="W"/>
    </country>
    <country name="Singapore">
        <rank updated="yes">5</rank>
        <year>2011</year>
        <gdppc>59900</gdppc>
        <neighbor name="Malaysia" direction="N"/>
    </country>
    <country name="Panama">
        <rank updated="yes">69</rank>
        <year>2011</year>
        <gdppc>13600</gdppc>
        <neighbor name="Costa Rica" direction="W"/>
        <neighbor name="Colombia" direction="E"/>
    </country>
</data>
xml data

The XML protocol is supported in all languages. In python, the following modules can be used to manipulate xml:

# print(root.iter('year')) #Full text search
# print(root.find('country')) #stay root Find only one child node
# print(root.findall('country')) #Find all the child nodes in root
import xml.etree.ElementTree as ET
 
tree = ET.parse("xmltest.xml")
root = tree.getroot()
print(root.tag)
 
#ergodic xml File
for child in root:
    print('========>',child.tag,child.attrib,child.attrib['name'])
    for i in child:
        print(i.tag,i.attrib,i.text)
 
#Traversal only year node
for node in root.iter('year'):
    print(node.tag,node.text)
#---------------------------------------

import xml.etree.ElementTree as ET
 
tree = ET.parse("xmltest.xml")
root = tree.getroot()
 
#modify
for node in root.iter('year'):
    new_year=int(node.text)+1
    node.text=str(new_year)
    node.set('updated','yes')
    node.set('version','1.0')
tree.write('test.xml')
 
 
#delete node
for country in root.findall('country'):
   rank = int(country.find('rank').text)
   if rank > 50:
     root.remove(country)
 
tree.write('output.xml')
#stay country Internal addition ( append)node year2
import xml.etree.ElementTree as ET
tree = ET.parse("a.xml")
root=tree.getroot()
for country in root.findall('country'):
    for year in country.findall('year'):
        if int(year.text) > 2000:
            year2=ET.Element('year2')
            year2.text='new year'
            year2.attrib={'update':'yes'}
            country.append(year2) #to country Add child nodes under nodes

tree.write('a.xml.swap')
import xml.etree.ElementTree as ET
 
 
new_xml = ET.Element("namelist")
name = ET.SubElement(new_xml,"name",attrib={"enrolled":"yes"})
age = ET.SubElement(name,"age",attrib={"checked":"no"})
sex = ET.SubElement(name,"sex")
sex.text = '33'
name2 = ET.SubElement(new_xml,"name",attrib={"enrolled":"no"})
age = ET.SubElement(name2,"age")
age.text = '19'
 
et = ET.ElementTree(new_xml) #Generating Document Objects
et.write("test.xml", encoding="utf-8",xml_declaration=True)
 
ET.dump(new_xml) #Print-generated format
How to generate xml documents

 

4. configparser module

[mogu] #section
name="mogu"  #k , v Form
password="123"
salary=3.1
is_cool=True

[xiaoming] #section
name="xiaoming"
password="123456"
salary=3.2
is_sb=True
user.ini(ini data format)
import configparser

config=configparser.ConfigParser()
config.read('user.ini')

#Look at all the titles
res=config.sections() #['mogu', 'xiaoming']
print(res)

#View title'mogu'All under key=value Of key
options=config.options('mogu')
print(options) # ['name', 'password', 'salary', 'is_cool']

#View title'mogu'All under key=value Of(key,value)format
item_list=config.items('mogu')
print(item_list) #[('name', '"mogu"'), ('password', '"123"'), ('salary', '3.1'), ('is_cool', 'True')]

#View title section1 lower user Value=>String format
val=config.get('mogu','salary')
print(val) # '3.1'

#View title section1 lower age Value=>Integer format
val1=config.getint('mogu','password')
print(val1) #123

#View title section1 lower is_admin Value=>Boolean Value Format
val2=config.getboolean('mogu','is_cool')
print(val2) #True

#View title section1 lower salary Value=>Floating-point format
val3=config.getfloat('mogu','salary')
print(val3) #3.1

rewrite

#Delete the entire title section2
config.remove_section('xiaoming')

#Delete title'mogu'One of the following k1 and k2
config.remove_option('mogu','name')
config.remove_option('mogu','salary')

#Determine whether a title exists
print(config.has_section('mogu'))

#Judgement heading'mogu'Is there any under? user
print(config.has_option('mogu',''))


#Add a title
config.add_section('zhangsan')

#In title zhangsan Add below name=xiaosan,age=58 Configuration
config.set('zhangsan','name','xiaosan')
config.set('zhangsan','age','58') #Report errors,Must be a string


#Finally, write the modified content to the file,Complete final modifications
config.write(open('user.ini','w'))
import configparser

config = configparser.ConfigParser()
config["DEFAULT"] = {'ServerAliveInterval': '45',
                     'Compression': 'yes',
                     'CompressionLevel': '9'}

config['bitbucket.org'] = {}
config['bitbucket.org']['User'] = 'hg'
config['topsecret.server.com'] = {}
topsecret = config['topsecret.server.com']
topsecret['Host Port'] = '50022'  # mutates the parser
topsecret['ForwardX11'] = 'no'  # same here
config['DEFAULT']['ForwardX11'] = 'yes'
with open('example.ini', 'w') as configfile:
    config.write(configfile)
Create an ini configuration document based on the above

 

V. subprocess module

import subprocess
cmd=input('Enter the command:').strip()
obj=subprocess.Popen(
    '%s'%cmd,
    shell=True,
    stdout=subprocess.PIPE,
    stderr=subprocess.PIPE
)

res=obj.stdout.read()
print(res.decode('gbk'))

print('='*30)
res1=obj.stderr.read()
print(res1.decode('gbk'))
 1 import  subprocess
 2 
 3 '''
 4 sh-3.2# ls /Users/egon/Desktop |grep txt$
 5 mysql.txt
 6 tt.txt
 7 Thing.txt
 8 '''
 9 
10 res1=subprocess.Popen('ls /Users/jieli/Desktop',shell=True,stdout=subprocess.PIPE)
11 res=subprocess.Popen('grep txt$',shell=True,stdin=res1.stdout,
12                  stdout=subprocess.PIPE)
13 
14 print(res.stdout.read().decode('utf-8'))
15 
16 
17 #Equivalent to the above,But the advantage is that,One data stream can interact with another data stream,The results can be obtained by the crawler and handed over to the crawler. grep
18 res1=subprocess.Popen('ls /Users/jieli/Desktop |grep txt$',shell=True,stdout=subprocess.PIPE)
19 print(res1.stdout.read().decode('utf-8'))
20 
21 
22 #windows lower:
23 # dir | findstr 'test*'
24 # dir | findstr 'txt$'
25 import subprocess
26 res1=subprocess.Popen(r'dir C:\Users\Administrator\PycharmProjects\test\Function preparation',shell=True,stdout=subprocess.PIPE)
27 res=subprocess.Popen('findstr test*',shell=True,stdin=res1.stdout,
28                  stdout=subprocess.PIPE)
29 
30 print(res.stdout.read().decode('gbk')) #subprocess Using the default encoding of the current system, the result is bytes Type, in windows Under need gbk Decode

 

6. Built-in Functions

The invisible parts of the above pictures can be opened on the new tab.

For details, please refer to: http://www.runoob.com/python3/python3-built-in-functions.html

 

VII. Abnormal handling

I. What is an exception?

An exception is a signal that an error occurs while the program is running (an exception occurs when the program is wrong, and if the program does not handle it, the exception is thrown and the program terminates). In python, the exception triggered by the error is as follows

Errors fall into two categories

#Demonstration 1 of Grammatical Errors
if
#Demonstration 2 of Grammatical Errors
def test:
    pass
#Demonstration 3 of Grammatical Errors
class Foo
    pass
#Model 4 of Grammatical Errors
print(haha
1. Syntax errors (such errors can not pass the grammar detection of python interpreter at all, they must be corrected before the program is executed)
#TypeError:int Types are not iterative
for i in 3:
    pass
#ValueError
num=input(">>: ") #input hello
int(num)
#NameError
aaa
#IndexError
l=['bb','aa']
l[3]
#KeyError
dic={'name':'mogu'}
dic['age']
# AttributeError
class Foo:pass
Foo.x
#ZeroDivisionError:Unable to complete the calculation
res1=1/0
res2=1+'str'
2. Logical errors

 

Types of Abnormalities

Different exceptions in Python can be identified by different types (python unifies classes and types, types are classes). An exception identifies an error.

AttributeError   # Attempting to access a tree that an object does not have, for example foo.x,however foo No attribute x
IOError          # input/Output exception; basically unable to open file
ImportError      # Unable to introduce modules or packages; basically a path problem or name error
IndentationError # Syntax errors (subclasses); incorrect alignment of code
IndexError       # Subscript indexes exceed sequence boundaries, such as when x There are only three elements, but they try to access them. x[5]
KeyError         # Attempt to access keys that do not exist in the dictionary
KeyboardInterrupt# Ctrl+C Be pressed
NameError        # Use a variable that has not been assigned to an object
SyntaxError      # Python Code is illegal and cannot be compiled(Personally, I think it's a grammatical error and a mistake in writing.
TypeError        # Incoming Object Types and Requirements
UnboundLocalError# An attempt to access a local variable that has not yet been set is basically due to another global variable with the same name, which you think is being accessed.
ValueError       # Pass in a value that the caller does not expect, even if the value type is correct
Frequently used anomalies
ArithmeticError
AssertionError
AttributeError
BaseException
BufferError
BytesWarning
DeprecationWarning
EnvironmentError
EOFError
Exception
FloatingPointError
FutureWarning
GeneratorExit
ImportError
ImportWarning
IndentationError
IndexError
IOError
KeyboardInterrupt
KeyError
LookupError
MemoryError
NameError
NotImplementedError
OSError
OverflowError
PendingDeprecationWarning
ReferenceError
RuntimeError
RuntimeWarning
StandardError
StopIteration
SyntaxError
SyntaxWarning
SystemError
SystemExit
TabError
TypeError
UnboundLocalError
UnicodeDecodeError
UnicodeEncodeError
UnicodeError
UnicodeTranslateError
UnicodeWarning
UserWarning
ValueError
Warning
ZeroDivisionError
More anomalies

 

3. Abnormal handling

In order to ensure the robustness and fault tolerance of the program, that is, the program will not crash when encountering errors, we need to deal with exceptions.

If the conditions for the occurrence of errors are predictable, we need to deal with them with if: prevent them before they occur

AGE=10
while True:
    age=input('>>: ').strip()
    if age.isdigit(): #Only in age Integer time in string form,The following code is error-free,This condition is predictable.
        age=int(age)
        if age == AGE:
            print('you got it')
            break
if evades anomalies

 

If the conditions for the occurrence of an error are unpredictable, try...except is needed: after the error occurs, it is handled.

#The basic grammar is
try:
    //Code blocks detected
except Exception types:
    try Once an exception is detected, the logic for this location is executed
#Give an example
try:
    f=open('a.txt')
    g=(line.strip() for line in f)
    print(next(g))
    print(next(g))
    print(next(g))
    print(next(g))
    print(next(g))
except StopIteration:
    f.close()

More uses:

#1 Exception classes can only be used to handle specified exceptions, but cannot handle non-specified exceptions.
s1 = 'hello'
try:
    int(s1)
except IndexError as e: # Exceptions are not caught and the program reports errors directly
    print e

#2 Multiple branches
s1 = 'hello'
try:
    int(s1)
except IndexError as e:
    print(e)
except KeyError as e:
    print(e)
except ValueError as e:
    print(e)

#3 Universal anomaly Exception
s1 = 'hello'
try:
    int(s1)
except Exception as e:
    print(e)

#4 Multi-branch anomaly and omnipotent anomaly
#4.1 If the effect you want is that whatever exceptions occur, we throw them away, or use the same code logic to deal with them, then Sao Nian, do it boldly, there is only one. Exception That's enough.
#4.2 If you want the effect that we need to customize different processing logic for different exceptions, then we need to use multiple branches.

#5 It can also be a multi-branch later one. Exception
s1 = 'hello'
try:
    int(s1)
except IndexError as e:
    print(e)
except KeyError as e:
    print(e)
except ValueError as e:
    print(e)
except Exception as e:
    print(e)

#6 Other Abnormal Institutions
s1 = 'hello'
try:
    int(s1)
except IndexError as e:
    print(e)
except KeyError as e:
    print(e)
except ValueError as e:
    print(e)
#except Exception as e:
#    print(e)
else:
    print('try If there are no exceptions in the inner code block, execute me')
finally:
    print('Whether unusual or not,The module will be executed.,Usually cleaning is done.')

#7 Active triggering anomaly
try:
    raise TypeError('error in type')
except Exception as e:
    print(e)

#8 Custom exception
class EgonException(BaseException):
    def __init__(self,msg):
        self.msg=msg
    def __str__(self):
        return self.msg

try:
    raise EgonException('error in type')
except EgonException as e:
    print(e)

#9 Assertion:assert condition
assert 1 == 1  
assert 1 == 2

#10 summary try..except

1: Separate error handling from real work
2: Code is easier to organize, clearer, and complex tasks are easier to achieve.
3: Undoubtedly, it's safer than accidentally crashing a program due to minor negligence.

 

4. When to use exception handling

First try...except is an exception handling logic that you attach to your program. It has nothing to do with your main job. The addition of this kind of thing will lead to worse readability of your code.

It is only when the conditions of the error are unpredictable that try...except should be added.

Posted by sniped22 on Sat, 18 May 2019 21:03:46 -0700