sed stream editor: a powerful tool of Shell script

Keywords: ftp Linux network Apache

One of the core ideas of Linux is that everything is a file. Among these files, the text file plays an important role. All the changes of system settings, the adjustment of software parameters, and the most basic command-line operation are the operation of "text". To manipulate text, you can't do without a text editor.

In Linux system, text editor can be divided into two categories: one is full screen editor, such as vi, vim, nano, etc., which opens a text file and completes the modification and saving of the text in full screen mode; the other is stream editor, such as sed, grep, awk, etc., which does not directly open the file, and completes the operation of the text through a series of commands on the command line, including Find, insert, replace, delete, append, and so on.

sed is an excellent stream editor. It processes text in behavioral units. Its general processing flow is as follows: first read the first line of the file to be processed to the buffer (a block of memory), and then operate the line according to the given command. These operations include find, replace, delete, etc. after the operation is completed, output from the buffer to the screen Display, complete a cycle. After that, the second line of text is read to the buffer to start the second loop. Cycle like this until all behaviors have been processed. At this time, the content displayed on the screen comes from the buffer and does not overwrite the original file. The buffer mentioned above is called pattern space. If you need to save the modified content, there are two ways: one is to use the - i parameter to directly modify the original file; the other is to use redirection to save the modified text in another file.

Although sed supports many text operation functions, the most commonly used one is the replacement function, so this article mainly introduces the use of the replacement function. For more functions and usage, please refer to the user manual.

The command format of sed tool is:

sed option subcommand file 1, file 2, file 3

Common options:

-e. edit the sed action in command line mode. Default value
 -f read the sed action to be executed from the file.
-i. modify the contents of the original file directly without buffer.
-n print only the matched lines or the lines processed by subsequent instructions.
-r. add support extension expression

Common subcommands:
s / target text / new text / in each line, replace the first [target text] matched with [new text]
s / target text / new text / g in each line, replace all matched [target text] with [new text]
s / target text / new text / Ng in each line, replace all [target text] after the nth matching with [new text]
s / target text / new text / N in each line, replace the nth [target text] matched with [new text]

File or file list. sed supports processing multiple files at a time. The two files need to be separated by commas.

sed application example

The test text is as follows:

root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin

lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
operator:x:11:0:operator:/root:/sbin/nologin
games:x:12:100:games:/usr/games:/sbin/nologin
ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin

nobody:x:99:99:Nobody:/:/sbin/nologin
systemd-network:x:192:192:systemd Network Management:/:/sbin/nologin
dbus:x:81:81:System message bus:/:/sbin/nologin
polkitd:x:999:998:User for polkitd:/:/sbin/nologin
sshd:x:74:74:Privilege-separated SSH:/var/empty/sshd:/sbin/nologin
postfix:x:89:89::/var/spool/postfix:/sbin/nologin
chrony:x:998:996::/var/lib/chrony:/sbin/nologin
saslauth:x:997:76:Saslauthd user:/run/saslauthd:/sbin/nologin
apache:x:48:48:Apache:/usr/share/httpd:/sbin/nologin
  1. Replace the first match in the current row:
$ sed 's/root/admin/' test_file
admin:x:0:0:root:/root:/bin/bash
operator:x:11:0:operator:/admin:/sbin/nologin
  1. Replace the second match in the current row:
$ sed 's/root/admin/2' test_file 
root:x:0:0:admin:/root:/bin/bash
  1. Replace all matches in the current row:
$ sed 's/root/admin/g' test_file
admin:x:0:0:admin:/admin:/bin/bash
operator:x:11:0:operator:/admin:/sbin/nologin
  1. Replace all the second and later contents matched in the current row:
sed 's/root/admin/2g' test_file 
root:x:0:0:admin:/admin:/bin/bash

To be continued...

Published 13 original articles, won praise 0, visited 151
Private letter follow

Posted by DeGauss on Fri, 17 Jan 2020 21:15:47 -0800