Python monitoring server edge -- psutil

Keywords: Python network shell Unix

In addition to installing some commonly used monitoring software, the monitoring of the server sometimes needs to run some shell or Python scripts; under the shell, you can use the system's own shell commands such as ps/free/top/df, and python can call subprocess and other modules to run shell commands, but it is more troublesome to do so. Here is a better third-party module to use: psutil.
psutil is a cross platform library for retrieving information about running processes and system utilization (CPU, memory, disk, network, sensor) in Python. It is mainly used for system monitoring, analysis, limiting process resources and running process management. It implements many functions provided by UNIX command-line tools, such as ps, top, lsof, netstat, ifconfig, who, df, kill, free, nice, ionice, iostat, iotop, uptime, pidof, tty, taskset, pmap. psutil currently supports the following platforms:

  • Linux
  • Windows
  • OSX,
  • FreeBSD, OpenBSD, NetBSD
  • Sun Solaris
  • AIX

...
And other 32-bit and 64 bit architectures with Python 2.6 to 3.6. They can also run on PyPy.

install

The examples in this paper are all running in Python version 3.6;

# pip3 install psutil

Common modules

Get psutil version information

In [1]: import psutil
In [2]: psutil.version_info
Out[2]: (5, 4, 3

PS: if you need Python learning materials, you can click the link below to get them by yourself
note.youdao.com/noteshare?id=2dce86d0c2588ae7c0a88bee34324d76

Get CPU Information

In [3]: psutil.cpu_count() # Logical CPU cores
Out[3]: 4
In [4]: psutil.cpu_count(logical=False) # Physical CPU cores
Out[4]: 2
In [5]: psutil.cpu_times() # CPU user, system, idle time
Out[5]: scputimes(user=240773.0, nice=0.0, system=96416.32, idle=1161930.41)
In [9]: psutil.cpu_percent(percpu=True) # Get the utilization rate of each CPU, similar to the TOP command
Out[9]: [43.3, 22.0, 42.0, 23.0]
In [10]: top = [psutil.cpu_percent(interval=i, percpu=True) for i in range(10)] #Set the refresh interval per second, and count the results of ten times
In [11]: top
Out[11]:
[[40.8, 19.7, 38.5, 20.7],
[25.7, 5.9, 13.0, 5.0],
[35.0, 15.6, 30.0, 14.4],
[23.7, 7.0, 18.3, 7.4],
[38.5, 17.0, 34.2, 17.5],
[37.2, 19.6, 36.3, 20.0],
[29.6, 16.6, 28.8, 16.8],
[37.7, 19.0, 35.4, 18.7],
[30.8, 16.3, 26.9, 16.5],
[44.2, 27.9, 41.5, 28.6]]

Get memory information

In [13]: psutil.virtual_memory() #Get memory statistics, in bytes, I have 8G memory here
Out[13]: svmem(total=8589934592, available=1891045376, percent=78.0, used=6053986304, free=15130624, active=1878392832, inactive=1875914752, wired=2299678720)
In [14]: psutil.swap_memory() # Get swap statistics
Out[14]: sswap(total=2147483648, used=1340866560, free=806617088, percent=62.4, sin=126090076160, sout=3524710400)

Get disk information

In [17]: psutil.disk_partitions() #Get partition information
Out[17]: [sdiskpart(device='/dev/disk1', mountpoint='/', fstype='hfs', opts='rw,local,rootfs,dovolfs,journaled,multilabel')]
In [20]: psutil.disk_usage('/') # Get partition usage, 25.4% used here
Out[20]: sdiskusage(total=499055067136, used=126482944000, free=372309979136, percent=25.4)
In [22]: psutil.disk_io_counters() #Disk IO
Out[22]: sdiskio(read_count=7364142, write_count=6510641, read_bytes=282106464256, write_bytes=261763244544, read_time=2608778, write_time=1095259)

Access to network information

In [23]: psutil.net_if_stats() # Get network interface status
Out[23]:
{'awdl0': snicstats(isup=True, duplex=<NicDuplex.NIC_DUPLEX_UNKNOWN: 0>, speed=0, mtu=1484),
'bridge0': snicstats(isup=True, duplex=<NicDuplex.NIC_DUPLEX_UNKNOWN: 0>, speed=0, mtu=1500),
'en0': snicstats(isup=True, duplex=<NicDuplex.NIC_DUPLEX_UNKNOWN: 0>, speed=0, mtu=1500),
'en1': snicstats(isup=True, duplex=<NicDuplex.NIC_DUPLEX_FULL: 2>, speed=0, mtu=1500),
'en2': snicstats(isup=True, duplex=<NicDuplex.NIC_DUPLEX_FULL: 2>, speed=0, mtu=1500),
'gif0': snicstats(isup=False, duplex=<NicDuplex.NIC_DUPLEX_UNKNOWN: 0>, speed=0, mtu=1280),
'lo0': snicstats(isup=True, duplex=<NicDuplex.NIC_DUPLEX_UNKNOWN: 0>, speed=0, mtu=16384),
'p2p0': snicstats(isup=True, duplex=<NicDuplex.NIC_DUPLEX_UNKNOWN: 0>, speed=0, mtu=2304),
'stf0': snicstats(isup=False, duplex=<NicDuplex.NIC_DUPLEX_UNKNOWN: 0>, speed=0, mtu=1280),
'utun0': snicstats(isup=True, duplex=<NicDuplex.NIC_DUPLEX_UNKNOWN: 0>, speed=0, mtu=2000),
'utun1': snicstats(isup=True, duplex=<NicDuplex.NIC_DUPLEX_UNKNOWN: 0>, speed=0, mtu=1352)}
In [25]: psutil.net_if_stats().get("en0") #Get the status of a single network card en0
Out[25]: snicstats(isup=True, duplex=<NicDuplex.NIC_DUPLEX_UNKNOWN: 0>, speed=0, mtu=1500)
In [26]: psutil.net_if_addrs() # Get address information of all network cards
Out[26]:
{'awdl0': [snic(family=<AddressFamily.AF_LINK: 18>, address='36:7d:f3:80:6e:4e', netmask=None, broadcast=None, ptp=None),
snic(family=<AddressFamily.AF_INET6: 30>, address='fe80::347d:f3ff:fe80:6e4e%awdl0', netmask='ffff:ffff:ffff:ffff::', broadcast=None, ptp=None)],
'bridge0': [snic(family=<AddressFamily.AF_LINK: 18>, address='4a:00:02:c0:33:70', netmask=None, broadcast=None, ptp=None)],
'en0': [snic(family=<AddressFamily.AF_INET: 2>, address='192.168.0.101', netmask='255.255.255.0', broadcast='192.168.0.255', ptp=None),
snic(family=<AddressFamily.AF_LINK: 18>, address='ac:bc:32:91:32:8b', netmask=None, broadcast=None, ptp=None),
snic(family=<AddressFamily.AF_INET6: 30>, address='fe80::1476:ce7e:210a:2e32%en0', netmask='ffff:ffff:ffff:ffff::', broadcast=None, ptp=None)],
'en1': [snic(family=<AddressFamily.AF_LINK: 18>, address='4a:00:02:c0:33:70', netmask=None, broadcast=None, ptp=None)],
'en2': [snic(family=<AddressFamily.AF_LINK: 18>, address='4a:00:02:c0:33:71', netmask=None, broadcast=None, ptp=None)],
'lo0': [snic(family=<AddressFamily.AF_INET: 2>, address='127.0.0.1', netmask='255.0.0.0', broadcast=None, ptp=None),
snic(family=<AddressFamily.AF_INET6: 30>, address='::1', netmask='ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff', broadcast=None, ptp=None),
snic(family=<AddressFamily.AF_INET6: 30>, address='fe80::1%lo0', netmask='ffff:ffff:ffff:ffff::', broadcast=None, ptp=None)],
'p2p0': [snic(family=<AddressFamily.AF_LINK: 18>, address='0e:bc:32:91:32:8b', netmask=None, broadcast=None, ptp=None)],
'utun0': [snic(family=<AddressFamily.AF_INET6: 30>, address='fe80::583c:77a0:6b93:b045%utun0', netmask='ffff:ffff:ffff:ffff::', broadcast=None, ptp=None)],
'utun1': [snic(family=<AddressFamily.AF_INET: 2>, address='10.5.200.244', netmask=None, broadcast=None, ptp='10.5.200.244')]}
# Get the address of en0 network card, including mac and ipv6 addresses
In [40]: for addr in psutil.net_if_addrs().get("en0"):
...: print(addr.address)
192.168.0.101
ac:bc:32:91:32:8b
fe80::1476:ce7e:210a:2e32%en0
In [43]: psutil.net_io_counters() # Get the number of network read and Write Bytes / packets
Out[43]: snetio(bytes_sent=174614221, bytes_recv=586279725, packets_sent=863903, packets_recv=873583, errin=0, errout=0, dropin=0, dropout=0)
In [45]: psutil.net_connections() # Get the network connection information. Note that root permission is required here.

Get process information:

In [46]: psutil.pids() # Get all process ID s
In [47]: psutil.Process(61) # Get the process information of the specified PID
Out[47]: psutil.Process(pid=61, name='dsAccessService', started='2018-02-26 09:57:04')
In [49]: psutil.Process(45573).exe() # Get the exe path of the process
Out[49]: '/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/Resources/Python.app/Contents/MacOS/Python'
In [50]: psutil.Process(45573).name() # Get process name
Out[50]: 'Python'
In [52]: psutil.Process(45573).cmdline() # Get command for process startup
Out[52]:
['/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/Resources/Python.app/Contents/MacOS/Python',
'/usr/local/bin/ptipython']
In [56]: psutil.Process(45573).num_threads() # Get the number of threads in the process
Out[56]: 3
In [57]: psutil.Process(45573).environ() # Get process environment variable information

Conclusion:

Use the psutil module to do a more comprehensive monitoring of the system. If you are considering using Python as a monitoring system or script tool, you can consider this module.

Posted by cwetherbee on Fri, 29 Nov 2019 05:48:58 -0800