Log file analysis

Keywords: Linux


log file

Log function

:
It is used to record various events during the operation of the system and program

By reading the log, it is helpful to diagnose and solve system faults

Classification of log files

Kernel and system log = =: managed by the system service syslog system. The log format is basically similar to the configuration file / etc/rsyslog.conf

User log:
Record relevant information of system user login and logout

Program log:

When the program is allowed, any error will be recorded

Save location of logs: under the directory of / var/log

Introduction to main log files

/var/log/messages:record Linux Kernel messages and public log information of various applications, including startup, IO Error, network error, program failure, etc. for applications or services that do not use an independent log file, relevant event recording information can generally be obtained from the log file

/var/log/cron:record crond Event information generated by the scheduled task

/var/log/dmesg:record Linux Various event information during system boot

/var/log/maillog:Record email activity entering or sending out the system

/var/log/secure:Record security event information related to user authentication

/var/log/lastlog:Record the latest login event of each user in binary format

/var/log/wtmp:Record each user's login, logout, system startup and shutdown events in binary format

/var/run/btmp:Log failed and incorrect login attempts and authentication events in binary format

/var/run/utmp:Details of each user currently logged in

Kernel and system log

It is uniformly managed by the system service reyslog, and the configuration file is saved in / etc/rsylog.conf

*. info # indicates that the information of all levels above info level is written to the corresponding log file
mail.none # means that the information of an event is not written to the log file (e.g. mail here)

Device field description:
auth				#Log generated during user authentication

authpriv 		    #Authentication information of ssh, ftp and other login information
daemon				#- logs generated by some Daemons
ftp					#Log generated by ETP
lpr	       			#Print related activities
rmark       		#Internal information of rsyslog service, time id
news         		#Messages generated by network news transmission protocol (nntp).
syslog		 		#system log
uucp		 		#unix to unix copy related communication between two unix
console		 		#Messages for the system console.
cron         		#The log generated by the system executing scheduled tasks.
kern         		#System kernel log
localo~      		#White definition program use
rnail        		#Mail log
user		 		#User process

Linux Priority of system kernel log messages (the lower the number level is, the higher the priority is, and the more important the message is)

0		EMERG(Emergency): a situation that will make the host system unavailable. Such as system crash
1		ALERT(Warning): measures must be taken immediately to solve the problem. If the database is damaged
2		CRIT(Serious): a more serious situation. If the hard disk is wrong, it may hinder some functions of the program
3		ERR(Error): an error occurred while running. It's not very urgent. Repair it as soon as possible
4		WARNING(Reminder): important events that may affect system functions and need to be reminded to users. Not an error, such as the disk is used			85%etc.
5		NOTICE(Note): events that do not affect normal functions but need attention. No processing required
6		INFO(Information): general information. Normal system information
7		DEBUG(Debugging): program or system debugging information, etc. Contains detailed development information, which is used when debugging the program
		none: No priority,No log messages are logged.

General format of logging

more /var/log/messages

Nov 27 15:10:35 zzz NetworkManager[8821]: <info>  [1637997035.9410] dhcp4 (ens33):   plen 24 (255.255.255.0)

Time stamp: Nov 27 15:10:35	Date and time when the message was sent
 Host name: zzz	The name of the computer that generated the message
 Subsystem Name: NetworkManager[8821]:	The name of the application that sent the message
 Message fields:<info>  [1637997035.9410]	
Details of the message: dhcp4 (ens33):   plen 24 (255.255.255.0)

Analysis tools

The last command is used to query the user records that have successfully logged in to the system

[root@zzz log]# last
root     pts/1        192.168.232.1    Sat Nov 27 16:50   still logged in   
root     pts/4        192.168.232.1    Sat Nov 27 13:38   still logged in   
root     pts/3        :0               Sat Nov 27 13:37   still logged in   
root     pts/2        :0               Sat Nov 27 13:36   still logged in 

The lastb command is used to query the user record of login failure

[root@zzz log]# lastb
root     ssh:notty    zzz              Sat Nov 27 17:08 - 17:08  (00:00)    
root     ssh:notty    zzz              Sat Nov 27 17:08 - 17:08  (00:00)    
root     ssh:notty    192.168.232.1    Wed Nov  3 17:08 - 17:08  (00:00)    
root     :0           :0               Sun Oct 31 17:02 - 17:02  (00:00) 

Program log analysis

It is managed independently by the corresponding application and written by the developer

For example, the Web service log is located in / var/log/httpd/

access_log ## records customer access events

error_log ## log error events

Log management policy

It is necessary to backup and archive logs in time

Extend log retention

Control log access: the log may contain various sensitive information, such as account, password, etc

Centralized log management: send the log files of the server to the unified log file server to facilitate the unified collection, sorting and analysis of log information, and eliminate the accidental loss, malicious tampering and deletion of log information

Configure log server to collect logs

Sending server: client 192.168.232.130

Collection server: server 192.168.232.129

1. Turn off the firewall and deep Protection for the two servers first

setenforce stop firewalld
systemctl stop firewalld
systemctl disable firewalld

2. Go to the client / etc/rsyslog.conf for configuration

Cancel the comments of TCP and UDP and open the port module

Then write the configuration information to

$template myFormat, "%timestamp% %hostname% %syslogseverity-text% %syslogtag% %msg%\n"
$ActionFileDefaultTemplate myFormat
*.info;mail.none;authpriv.none;cron.none              @@192.168.232.129:514

Restart service

[root@send ~]# systemctl restart rsyslog

View port information

[root@send ~]# netstat -anutp | grep 514

Client setup completed

Next, set the server

1. Enter / etc/rsyslog.conf for configuration

Cancel the comments of TCP and UDP and open the port module

[the external chain image transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the image and upload it directly (img-y0zPaQXk-1638082066553)(C:\Users\zhuquanhao\Desktop \ screenshot command set \ linux \ log file management \ 6.png)]

$AllowedSender tcp, 192.168.232.0/24  #Allow hosts in the 192.168.232.0 network segment to transmit with tcp protocol

$template Remote,"/data/log/%fromhost-ip%/%fromhost-ip%_%$YEAR%-%$MONTH%-%$DAY%.log"  #Define the template, accept the log file path, and distinguish the logs of different hosts

:fromhost-ip, !isequal, "127.0.0.1" ?Remote  #Filter out the logs of the server

2. Create / data/log / directory

mkdir /data/log/

3. Restart the service

[root@recive log]# systemctl restart rsyslog

4.cd to the / data/log / directory, ls take a look

5. Send a log file at the client to see if the server can synchronize the log information

client:

[root@send ~]# logger "hello world"

Server:

journalctl log management tool

The log management tool journalctl is a proprietary log management tool on CentOS 7. The tool reads information from the message file.

System D uniformly manages the startup logs of all units. The advantage is that you can view all logs (kernel log and application log) with only one command of journalctl. The log configuration file is / etc/ systemd/journald.conf

View the latest log file, starting from the end

[root@send ~]# journalctl -r

View kernel log

[root@send ~]# journalctl -k

View the system startup log

[root@send ~]# journalctl -b 0

View the log of the last startup

[root@send ~]# journalctl -b -1

The log with 20 lines specified at the end is displayed and updated dynamically

[root@send ~]# journalctl -n 20 -f

View the log of a service (such as firewall)

[root@send ~]# journalctl -u firewalld

View the log of the specified process

[root@send ~]# journalctl _PID=1

View the log of the specified user

[root@send ~]# journalctl _UID=0 --since today

View error log

[root@send ~]# journalctl -xe

Posted by ladams02 on Sat, 27 Nov 2021 23:04:17 -0800