Text 3 swordsman - awk

Keywords: awk

catalogue

1, awk working principle

2, Common built-in variables in awk (can be used directly)

3, Examples

3.1 output text content by line

3.2 output specified line content

3.3 output odd and even lines

3.4 special usage

3.5 how to extract and put it back

3.6 extracting IP address

4, Summary

1, awk working principle

  Read the text line by line. By default, space is used as the separator. Save the separated fields in the built-in variable, and execute the editing command according to the mode or condition.

  The sed command is often used to process a whole line, while awk prefers to divide a line into multiple "fields" and then process it. The reading of awk information is also read line by line. The execution result can be printed and displayed through the function of print. When using the awk command, you can use the logical operators "& &" to represent "and", "||" to represent "or" and "!" to represent "not"; you can also perform simple mathematical operations, such as +, -, *, /,%, ^ to represent addition, subtraction, multiplication, division, remainder and power respectively.
2, Command format

awk option  'Mode condition{operation}' File 1 file 2
awk -f|-v Script file 1 file 2

2, Common built-in variables in awk (can be used directly)

FS: column separator. Specifies the field separator for each line of text. The default is space or tab stop. It works the same as "- F"
NF: the number of fields in the currently processed row.
NR: line number (ordinal number) of the currently processed line
$0: the entire line content of the currently processed line.
$n: the nth field (column n) of the current processing line
FILENAME: the name of the file being processed.
Rs: line separator. When awk reads data from a file, it will cut the data into many records according to the definition of RS, while awk only reads one record at a time for processing. The default value is' \ n '

3, Examples

3.1 output text content by line

[root@localhost ~]# df |awk '{print $5}'
Used%
10%
0%
0%
1%
0%
18%
1%
0%
1%

There is a special case. If a string is used, no matter what is printed, it is a character. The position added after it is the number of lines.

[root@localhost ~]# awk '{print "hellow"}'
1
hellow
2
hellow
3
hellow
4
hellow
^C
[root@localhost ~]# awk '{print "hellow"}' /etc/passwd
hellow
hellow
hellow
hellow
hellow
hellow
hellow
hellow
hellow
hellow

3.2 output specified line content

awk 'NR==1, NR==3{print}'  /etc/passwd       #Output lines 1-3
awk  '(NR>=1) &&  (NR<=3)  {print}'  /etc/passwd       #Output lines 1-3
awk  'NR==1 || NR==3 {print}'    /etc/passwd       #Output the contents of line 1 and line 3

For example:

[root@localhost ~]# cat /etc/passwd|awk 'NR==1,NR==3{print}'
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
[root@localhost ~]# cat /etc/passwd|awk '(NR>=1)&&(NR<=3){print}'
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
[root@localhost ~]# cat /etc/passwd|awk 'NR==1||NR==3{print}'
root:x:0:0:root:/root:/bin/bash
daemon:x:2:2:daemon:/sbin:/sbin/nologin

3.3 output odd and even lines

awk '(NR%2)==1{print}'  /etc/passwd       #Output odd rows
awk '(NR%2)==0{print}'  /etc/passwd       #Output even rows
[root@localhost ~]# cat /etc/passwd|awk '(NR%2)==1{print}'
root:x:0:0:root:/root:/bin/bash
daemon:x:2:2:daemon:/sbin:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
games:x:12:100:games:/usr/games:/sbin/nologin

3.4 special usage

BEGIN mode means: before processing the specified text, you need to execute the actions specified in BEGIN mode; awk then process the specified text, and then execute the actions specified in END mode. In the END {} statement block, statements such as print results are often placed.

awk 'BEGIN {x=0};/\/bin\/bash$/{x++};END {print x}' /etc/passwd
#Count the number of lines ending in / bin/bash, equivalent to
grep  -c "/bin/bash$"  /etc/passwd

3.5 how to extract and put it back

[root@localhost lgy]# cat 23.txt |awk -F "[.]" '{print $1}' >>23.txt
[root@localhost lgy]# cat 23.txt 
1 www.kgc.com
2 mail.kgc.com
3 ftp.kgc.com
4 linux.kgc.com
5 blog.kgc.com
www
mail
ftp
linux
blog

3.6 extracting IP address

[root@localhost ~]# ifconfig ens33|sed -n '2p'|awk '{print $2}'
192.168.189.100
[root@localhost ~]# hostname -I |awk '{print $1}'
192.168.189.100

  Exercise:

1. Count the number of occurrences of each file system type in the / etc/fstab file

[root@localhost ~]# cat /etc/fstab |sed -n '9,$p'|awk '{print $3}'|sort|uniq -c
      1 swap
      3 xfs


2. Count the number of occurrences of each word in the / etc/fstab file

[root@localhost ~]# egrep -o "\<[[:alpha:]]+\>" /etc/fstab |sort |uniq -c |sort -n
      1 aada
      1 Accessible
      1 anaconda
      1 and
      1 are
      1 blkid
      1 boot
      1 Created
      1 disk
      1 etc
      1 filesystems


3. Extract the string YD$ C@M05MB %All numbers in 9 & bdh7dq + yvixp3vpw

[root@localhost ~]# echo "Yd$C@M05MB%9&Bdh7dq+YVixp3vpw"|grep -Eo "[0-9]+"
05
9
7
3

4. Extract the host name and put it into the original file

[root@localhost ~]# awk -F"[/.]" '{print $3}' 1.txt >>1.txt
[root@localhost ~]# cat 1.txt
http://mail.kgc.com/index.html
http://www.kgc.com/test.html
http://study.kgc.com/index.html
http://blog.kgc.com/index.html
http://www.kgc.com/images/logo.jpg
http://blog.kgc.com/20080102.html
http://www.kgc.com/images/kgc.jpg
mail
www
study
blog
www
blog
www

4, Summary

After learning awk, I mainly learn how to filter.

Posted by roby2411 on Wed, 27 Oct 2021 00:16:43 -0700