preface
sed (Stream EDitor) is a powerful and simple text parsing and conversion tool. It can read the text, edit the text content (delete, replace, add, move, etc.) according to the specified conditions, and finally output all lines or only some lines processed. sed can also realize quite complex text processing operations without interaction. It is widely used in Shell scripts to complete various automatic processing tasks.
1, Workflow
The workflow of sed mainly includes three processes: reading, executing and displaying.
-
Read:
sed reads a line from the input stream (file, pipe, standard input) and stores it in a temporary buffer (also known as pattern space). -
Execution:
By default, all sed commands are executed sequentially in the pattern space. Unless the address of the line is specified, the SED command will be executed successively on all lines. -
Display:
Send the modified content to the output stream. After sending data, the mode space will be cleared.
Before all file contents are processed, the above process will be repeated until all contents are processed.
Note: by default, all sed commands are executed in the mode space, so the input file will not change unless the output is stored by redirection.
2, Common usage of sed command
- There are usually two formats for calling the sed command
sed [option] 'operation' parameter
sed [options] - f scriptfile parameter
"Parameter" refers to the target file of the operation. When there are multiple operation objects, the files are separated by commas ",";
The scriptfile represents the script file, which needs to be specified with the "- f" option. When the script file appears before the target file, it means that the input target file is processed through the specified script file.
1. Common sed command options
| option | meaning |
|---|---|
| -e or – expression= | Indicates that the input text file is processed with the specified command or script. |
| -f or – file= | Indicates that the input text file is processed with the specified script file. |
| -h or – help | Displays help. |
| -n. – quiet or silent | Indicates that only the processed results are displayed. |
| -i | Edit the text file directly. |
2. Common operations
Operation "is used to specify the action behavior of file operation, that is, the sed command. Generally, it is in the format of "[n1[,n2]]" operation parameter. n1 and n2 are optional and represent the number of lines selected for operation. For example, the operation needs to be between 5 and 20 lines
| parameter | meaning |
|---|---|
| a | Add: add a line of specified content under the current line. |
| c | Replace to replace the selected row with the specified content. |
| d | Delete, deletes the selected row. |
| i | Insert, inserts a row of specified content above the selected row. |
| p | Print. If a line is specified at the same time, it means to print the specified line; If no line is specified, it means that all contents are printed; If there are non printing characters, they are output in ASCII code. It is usually used with the "- n" option. |
| s | Replace, replace specified character |
| y | Character conversion. |
3. Basic usage examples:
3.1 output qualified text (p indicates normal output)
[root@c7-1 home]# sed -n 'p' test.txt
#Output all contents, equivalent to cat test.txt
[root@c7-1 home]# sed -n '3p' test.txt // Output third line
The home of Football on BBC Sport online.
[root@c7-1 home]# sed -n '3,5p' test.txt // Output lines 3 to 5
The home of Football on BBC Sport online.
the tongue is boneless but it breaks bones.12!
google is the best tools for search keyword.
[root@c7-1 home]# sed -n 'p;n' test.txt // Output all odd rows, n means to read the next row of data
he was short and fat.
The home of Football on BBC Sport online.
google is the best tools for search keyword.
PI=3.141592653589793238462643383249901429
Actions speak louder than words
#woooooood
I bet this place is really spooky late at night!
[root@c7-1 home]# sed -n 'n;p' test.txt // Output all even rows
He was wearing a blue polo shirt with black pants.
the tongue is boneless but it breaks bones.12!
The year ahead will test our political establishment to the limit.
a wood cross!
#woood #
# AxyzxyzxyzxyzC
Misfortunes never come alone/single.
[root@c7-1 home]# sed -n '1,5{p;n}' test.txt // Output odd lines between lines 1 to 5
he was short and fat.
The home of Football on BBC Sport online.
google is the best tools for search keyword.
[root@c7-1 home]# sed -n '10,${n;p}' test.txt // Output an even number of lines from line 10 to the end of the file
#woooooood
I bet this place is really spooky late at night!
#When executing the command "sed -n '10,${n;p}' test.txt", the first line read is the tenth line of the file,
#The second line read is the 11th line of the file, and so on,
#Therefore, the even lines output are lines 11 and 13 of the file until the end of the file, including blank lines.
The above is sed Basic usage of commands, sed When the command is combined with a regular expression, the format is slightly different, and the regular expression is“/"surround.
- Examples of using the sed command with regular expressions:
[root@c7-1 home]# sed -n '/the/p' test.txt // Output rows containing the the tongue is boneless but it breaks bones.12! google is the best tools for search keyword. The year ahead will test our political establishment to the limit. [root@c7-1 home]# sed -n '4,/the/p' test.txt // Output from line 4 to the first line containing the the tongue is boneless but it breaks bones.12! google is the best tools for search keyword. [root@c7-1 home]# sed -n '/the/=' test.txt // Output the line number of the line containing the, and the equal sign (=) is used to output the line number 4 5 6 [root@c7-1 home]# sed -n '/^PI/p' test.txt // Output lines starting with PI PI=3.141592653589793238462643383249901429 [root@c7-1 home]# sed -n '/[0-9]$/p' test.txt // Output lines ending in numbers PI=3.141592653589793238462643383249901429 [root@c7-1 home]# sed -n '/\<wood\>/p' test.txt // Output a line containing the word wood, \ <, \ > representing the word boundary a wood cross!
3.2 delete the qualified text (d)
- In the following commands, the nl command is used to calculate the number of lines of the file. Combined with this command, you can more intuitively check and see the execution results of the command.
[root@c7-1 home]# nl test.txt |sed '3d' // Delete line 3
1 he was short and fat.
2 He was wearing a blue polo shirt with black pants.
4 the tongue is boneless but it breaks bones.12!
[root@c7-1 home]# nl test.txt | sed '3,5d' // Delete lines 3 to 5
1 he was short and fat.
2 He was wearing a blue polo shirt with black pants.
6 The year ahead will test our political establishment to the limit.
[root@c7-1 home]# nl test.txt | sed '/cross/d'
1 he was short and fat.
2 He was wearing a blue polo shirt with black pants.
3 The home of Football on BBC Sport online.
4 the tongue is boneless but it breaks bones.12!
5 google is the best tools for search keyword.
6 The year ahead will test our political establishment to the limit.
7 PI=3.141592653589793238462643383249901429
9 Actions speak louder than words
//Delete the row containing cross, and the original row 8 will be deleted; if you want to delete the row without cross, use the! Symbol to indicate the reverse operation, such as' / Cross /! d '
[root@c7-1 home]# sed '/^[a-z]/d' test.txt // Delete lines beginning with lowercase letters He was
[root@c7-1 home]# sed '/\.$/d' test.txt // Delete lines ending with '.'
[root@c7-1 home]# sed '/^$/d' test.txt // Delete all blank lines
3.3 replace qualified text
- The s (string replacement), c (whole line / block replacement) and y (character conversion) command options are required when using the sed command for replacement. The common usage is as follows.
sed 's/the/THE/' test.txt //Replace THE first THE in each line with THE sed 's/l/L/2' test.txt //Replace the second L in each line with L sed 's/the/THE/g' test.txt //Replace all THE in THE file with THE sed 's/o//g' test.txt // Delete all o in the file (replace with empty string) sed 's/^/#/' test.txt //Insert # number at the beginning of each line sed '/the/s/^/#/' test.txt //Insert # number at the beginning of each line containing the sed 's/$/EOF/' test.txt //Insert the string EOF at the end of each line sed '3,5s/the/THE/g' test.txt //Replace all THE in lines 3 to 5 with THE sed '/the/s/o/O/g' test.txt //Replace o with O in all rows containing the
3.4 migrating qualified text
- When using the sed command to migrate qualified text, the following parameters are commonly used
| parameter | meaning |
|---|---|
| H | copy to clipboard |
| g,G | Overwrites / appends the data in the clipboard to the specified row |
| w | Save as file |
| r | Read the specified file |
| a | Append specified content |
sed '/the/{H;d};$G' test.txt //Move the line containing the to the end of the file, {;} is used for multiple operations
sed '1,5{H;d};17G' test.txt //After transferring lines 1 to 5 to line 17
sed '/the/w out.file' test.txt //Save the line containing the as the file out.file
sed '/the/r /etc/hostname' test.txt //Add the contents of the file / etc/hostname after each line that contains the
sed '3aNew' test.txt //Insert a New line after line 3 with the content New
sed '/the/aNew' test.txt //After each line containing the, insert a New line with the content New
sed '3aNew1\nNew2' test.txt //Insert multiple lines after line 3, and the middle \ n indicates line feed
3.5 editing files using scripts
- Use sed script to store multiple editing instructions in a file (one editing instruction per line) and call it through "- f" option.
sed '1,5{H;d};16G' test.txt //After transferring lines 1 to 5 to line 16
The above operations can be performed in script file mode instead:
[root@c7-1 home]# vi opt.list
1,5H
1,5d
17G
[root@c7-1 home]# sed -f opt.list test.txt
3.6 sed direct operation file example
- Write a script to adjust the vsftpd service configuration. It is required to prohibit anonymous users, but allow local users (write is also allowed).
[root@c7-1 home]# vim local_ftp.sh #!/bin/nash #Requirement: used to adjust the vsftpd service configuration. Anonymous users are prohibited. However, local users are allowed (write is also allowed) #1. Specify sample file path and configuration file path SAMPLE="/usr/share/doc/vsftpd-3.0.2/EXAMPLE/INTERNET_SITE/vsftpd.conf " CONFIG="/etc/vsftpd/vsftpd.conf" #2. Back up the original configuration file and check whether the backup file named / etc/vsftpd/vsftpd.conf.bak exists. If not, use the cp command to enter > Line file backup [ ! -e "$CONFIG.bak" ] && cp $SAMPLE $CONFIG.bak sed -e '/anonymous_enable/s/YES/NO/g' $SAMPLE > $CONFIG sed -i -e '/^local_enable/s/NO/YES/g' -e '/^write_enable/s/NO/YES/g' $CNFIG grep "listen" $CONFIF || sed -i '$alistn' $CONFIG #Start the vsftpd service and set it to start automatically systemctl restart vsftpd systemctl enable vsftpd