Linux three swordsmen - sed

describe

sed - stream editor for filtering and transforming text；
sed is the abbreviation of stream editor, that is, stream editor, which reads the whole file and modifies each line by default;

Mode of use

Command syntax
sed [option] '[line locator] instruction' file name
Command | sed [options] '[line locator] command'

Common options

option	describe
-h	display help information
-n	Only the processed row information is displayed
-r	Use extended regular expressions in locators
-i	Directly write the processed information to the file
-e	It is allowed to process multiple 'locators + instructions'

Common instructions

option	describe
p	Print positioning line information
d	Delete positioning line information
s	Replace positioning line assignment information
c	Replace the whole line information of the positioning line
=	Print current line number information
a	Adds the specified information to the row next to the positioning row
i	Adds the specified information to the row above the positioning row
r	Add r read specified file information after locating line
w	Writes the positioning line to w the specified file
n	The next row of the positioning row is processed

Common locator
- 1,3 – lines 1-3
- 1 ~ 2 – start from line 1 and process every other line, i.e. odd lines
- 2,3 + – starting from line 2 and the next 3 lines, i.e. lines 2-5
- 3! – Other lines other than line 3
- /Regular expression / – the row that matches the regular expression
- /Regular expression 1 /, / regular expression 2 / – lines from matching regular expression 1 to matching regular expression 2

Example
Test file information

[root@centos-36_2 tmp]# cat test_sed 
 1      The Redis server has been running, ignore this step.
 2
 3      2021-07-29 11:31:34-tor.py-478-NOTIFY: Received info
 4      2021-07-29 13:15:15-kln.py-478-INFO: Received NotifyREQ
 5
 6      2021-07-29 13:15:55-tor.py-550-NOTIFY: RPD IRA-RSP CORE-ID TLVs: 
 7      CoreIpAddress: 172.16.36.218
 8      CoreName: broad_Aux
[root@centos-36_2 tmp]#

Run directly without any options and positioning information (all lines are processed by default)

[root@centos-36_2 tmp]# sed 'p' test_sed 
 1      The Redis server has been running, ignore this step.
 1      The Redis server has been running, ignore this step.
 2
 2
 3      2021-07-29 11:31:34-tor.py-478-NOTIFY: Received info
 3      2021-07-29 11:31:34-tor.py-478-NOTIFY: Received info
 4      2021-07-29 13:15:15-kln.py-478-INFO: Received NotifyREQ
 4      2021-07-29 13:15:15-kln.py-478-INFO: Received NotifyREQ
 5
 5
 6      2021-07-29 13:15:55-tor.py-550-NOTIFY: RPD IRA-RSP CORE-ID TLVs: 
 6      2021-07-29 13:15:55-tor.py-550-NOTIFY: RPD IRA-RSP CORE-ID TLVs: 
 7      CoreIpAddress: 172.16.36.218
 7      CoreIpAddress: 172.16.36.218
 8      CoreName: broad_Aux
 8      CoreName: broad_Aux
[root@centos-36_2 tmp]#

Locate odd lines (it can be seen that odd lines are printed twice)

[root@centos-36_2 tmp]# sed '1~2p' test_sed 
 1      The Redis server has been running, ignore this step.
 1      The Redis server has been running, ignore this step.
 2
 3      2021-07-29 11:31:34-tor.py-478-NOTIFY: Received info
 3      2021-07-29 11:31:34-tor.py-478-NOTIFY: Received info
 4      2021-07-29 13:15:15-kln.py-478-INFO: Received NotifyREQ
 5
 5
 6      2021-07-29 13:15:55-tor.py-550-NOTIFY: RPD IRA-RSP CORE-ID TLVs: 
 7      CoreIpAddress: 172.16.36.218
 7      CoreIpAddress: 172.16.36.218
 8      CoreName: broad_Aux
[root@centos-36_2 tmp]#

Add the - n option to display only the positioning line (you can see that only the 3rd and 6th lines are displayed)

[root@centos-36_2 tmp]# sed -n '3p;6p' test_sed 
 3      2021-07-29 11:31:34-tor.py-478-NOTIFY: Received info
 6      2021-07-29 13:15:55-tor.py-550-NOTIFY: RPD IRA-RSP CORE-ID TLVs: 
[root@centos-36_2 tmp]#

Locate the specified row (the row containing not / Dot / Dot / not) through a regular expression

[root@centos-36_2 tmp]# sed -n '/N[Oo][Tt]/p' test_sed 
 3      2021-07-29 11:31:34-tor.py-478-NOTIFY: Received info
 4      2021-07-29 13:15:15-kln.py-478-INFO: Received NotifyREQ
 6      2021-07-29 13:15:55-tor.py-550-NOTIFY: RPD IRA-RSP CORE-ID TLVs: 
[root@centos-36_2 tmp]#

Locate range lines (lines starting with '3' and ending with 'REQ') through regular expressions

[root@centos-36_2 tmp]# sed -n '/^ 3/,/REQ$/p' test_sed
 3      2021-07-29 11:31:34-tor.py-478-NOTIFY: Received info
 4      2021-07-29 13:15:15-kln.py-478-INFO: Received NotifyREQ
[root@centos-36_2 tmp]#

Locate rows (rows containing ip addresses) by extending regular expressions

[root@centos-36_2 tmp]# sed -rn '/([0-9]{1,3}\.){3}[0-9]{1,3}/p' test_sed
 7      CoreIpAddress: 172.16.36.218
[root@centos-36_2 tmp]#

Delete positioning line (delete the line containing NOTIFY)

[root@centos-36_2 tmp]# sed '/NOTIFY/d' test_sed 
 1      The Redis server has been running, ignore this step.
 2
 4      2021-07-29 13:15:15-kln.py-478-INFO: Received NotifyREQ
 5
 7      CoreIpAddress: 172.16.36.218
 8      CoreName: broad_Aux
[root@centos-36_2 tmp]#

Whole line replacement (lines including Ip, whole replacement)

[root@centos-36_2 tmp]# sed -n '/Ip/c CoreIpAddress: 1.1.1.1' test_sed 
CoreIpAddress: 1.1.1.1
[root@centos-36_2 tmp]#

All lines find and replace the specified character once (replace the first number of all lines with NUM)

[root@centos-36_2 tmp]# sed 's/[0-9]/NUM/' test_sed 
 NUM    The Redis server has been running, ignore this step.
 NUM
 NUM    2021-07-29 11:31:34-tor.py-478-NOTIFY: Received info
 NUM    2021-07-29 13:15:15-kln.py-478-INFO: Received NotifyREQ
 NUM
 NUM    2021-07-29 13:15:55-tor.py-550-NOTIFY: RPD IRA-RSP CORE-ID TLVs: 
 NUM    CoreIpAddress: 172.16.36.218
 NUM    CoreName: broad_Aux
[root@centos-36_2 tmp]#

Find all lines replace all specified characters (replace numbers in all lines with NUM)

[root@centos-36_2 tmp]# sed 's/[0-9]/NUM/g' test_sed 
 NUM    The Redis server has been running, ignore this step.
 NUM
 NUM    NUMNUMNUMNUM-NUMNUM-NUMNUM NUMNUM:NUMNUM:NUMNUM-tor.py-NUMNUMNUM-NOTIFY: Received info
 NUM    NUMNUMNUMNUM-NUMNUM-NUMNUM NUMNUM:NUMNUM:NUMNUM-kln.py-NUMNUMNUM-INFO: Received NotifyREQ
 NUM
 NUM    NUMNUMNUMNUM-NUMNUM-NUMNUM NUMNUM:NUMNUM:NUMNUM-tor.py-NUMNUMNUM-NOTIFY: RPD IRA-RSP CORE-ID TLVs: 
 NUM    CoreIpAddress: NUMNUMNUM.NUMNUM.NUMNUM.NUMNUMNUM
 NUM    CoreName: broad_Aux
[root@centos-36_2 tmp]#

Replace the specified character with the specified line number (replace NOTIFY on line 3 with ERROR)

[root@centos-36_2 tmp]# sed  -n '3s/NOTIFY/ERROR/p' test_sed 
 3      2021-07-29 11:31:34-tor.py-478-ERROR: Received info
[root@centos-36_2 tmp]#

Filter the line to replace the specified character (replace the IP address in the line containing CoreIp with 1.1.1.1)

[root@centos-36_2 tmp]# sed -rn '/CoreIp/s/([0-9]{1,3}\.){3}[0-9]{1,3}/1\.1\.1\.1/p' test_sed
 7      CoreIpAddress: 1.1.1.1
[root@centos-36_2 tmp]#

The role of the & sign when replacing a specified character (the & sign in the replaced character is the character before replacement)

[root@centos-36_2 tmp]# sed -n 's/INFO/&(info)/p' test_sed 
 4      2021-07-29 13:15:15-kln.py-478-INFO(info): Received NotifyREQ
[root@centos-36_2 tmp]# 
[root@centos-36_2 tmp]# sed -n 's/k.*\.py/&(file)/p' test_sed 
 4      2021-07-29 13:15:15-kln.py(file)-478-INFO: Received NotifyREQ
[root@centos-36_2 tmp]#

Print the line number of the search line (search the line with '[0-9] {3}.' and print its line number)

[root@centos-36_2 tmp]# sed -nr '/[0-9]{3}\./p' test_sed 
 7      CoreIpAddress: 172.16.36.218
[root@centos-36_2 tmp]# sed -nr '/[0-9]{3}\./=' test_sed 
7
[root@centos-36_2 tmp]#

Print the total number of document rows ($= indicates the line number of the last line, that is, the total number of document rows)

[root@centos-36_2 tmp]# sed -n '$=' test_sed 
8
[root@centos-36_2 tmp]#

Insert information on the previous row of the specified row (search the row containing 'Core', and insert information above)

[root@centos-36_2 tmp]# sed  '/Core/i test line' test_sed 
 1      The Redis server has been running, ignore this step.
 2
 3      2021-07-29 11:31:34-tor.py-478-NOTIFY: Received info
 4      2021-07-29 13:15:15-kln.py-478-INFO: Received NotifyREQ
 5
 6      2021-07-29 13:15:55-tor.py-550-NOTIFY: RPD IRA-RSP CORE-ID TLVs: 
test line
 7      CoreIpAddress: 172.16.36.218
test line
 8      CoreName: broad_Aux
[root@centos-36_2 tmp]#

Insert information on the next row of the specified row (search the row containing 'Core', and insert information below)

[root@centos-36_2 tmp]# sed  '/Core/a test line' test_sed 
 1      The Redis server has been running, ignore this step.
 2
 3      2021-07-29 11:31:34-tor.py-478-NOTIFY: Received info
 4      2021-07-29 13:15:15-kln.py-478-INFO: Received NotifyREQ
 5
 6      2021-07-29 13:15:55-tor.py-550-NOTIFY: RPD IRA-RSP CORE-ID TLVs: 
 7      CoreIpAddress: 172.16.36.218
test line
 8      CoreName: broad_Aux
test line
[root@centos-36_2 tmp]#

After reading the file and inserting the specified line (locate the line with IpAddress and insert the 'addline' document information after it)

[root@centos-36_2 tmp]# cat addline 
AAA
BBB 
[root@centos-36_2 tmp]# sed '/IpAddress/r addline' test_sed 
 1      The Redis server has been running, ignore this step.
 2
 3      2021-07-29 11:31:34-tor.py-478-NOTIFY: Received info
 4      2021-07-29 13:15:15-kln.py-478-INFO: Received NotifyREQ
 5
 6      2021-07-29 13:15:55-tor.py-550-NOTIFY: RPD IRA-RSP CORE-ID TLVs: 
 7      CoreIpAddress: 172.16.36.218
AAA
BBB
 8      CoreName: broad_Aux
[root@centos-36_2 tmp]#

Write to the file after filtering the specified line (locate the line containing 'NO/No' and write to the file 'no_sed')

[root@centos-36_2 tmp]# sed -n '/N[oO]/w no_sed' test_sed 
[root@centos-36_2 tmp]# cat no_sed 
 3      2021-07-29 11:31:34-tor.py-478-NOTIFY: Received info
 4      2021-07-29 13:15:15-kln.py-478-INFO: Received NotifyREQ
 6      2021-07-29 13:15:55-tor.py-550-NOTIFY: RPD IRA-RSP CORE-ID TLVs: 
[root@centos-36_2 tmp]#

Filter the next row of the specified row for processing (delete and replace the next row of the filtered row)

[root@centos-36_2 tmp]# sed '/NOTIFY/{n;d;}' test_sed
 1      The Redis server has been running, ignore this step.
 2
 3      2021-07-29 11:31:34-tor.py-478-NOTIFY: Received info
 5
 6      2021-07-29 13:15:55-tor.py-550-NOTIFY: RPD IRA-RSP CORE-ID TLVs: 
 8      CoreName: broad_Aux
[root@centos-36_2 tmp]# 
[root@centos-36_2 tmp]# sed -n '/NOTIFY/{n;s/o/RRR/p;}' test_sed
 4      2021-07-29 13:15:15-kln.py-478-INFO: Received NRRRtifyREQ
 7      CRRRreIpAddress: 2.2.2.2
[root@centos-36_2 tmp]#

The information processed by sed is directly written to the source file (the source file will not be modified by default, the - I option will directly modify the source file, and i.bak generates a backup file while modifying the source file)

[root@centos-36_2 tmp]# cat test_sed 
 1      The Redis server has been running, ignore this step.
 2
 3      2021-07-29 11:31:34-tor.py-478-NOTIFY: Received info
 4      2021-07-29 13:15:15-kln.py-478-INFO: Received NotifyREQ
 5
 6      2021-07-29 13:15:55-tor.py-550-NOTIFY: RPD IRA-RSP CORE-ID TLVs: 
 7      CoreIpAddress: 172.16.36.218
 8      CoreName: broad_Aux
[root@centos-36_2 tmp]# 
[root@centos-36_2 tmp]# sed -ri.bak '/IpAddress/s/([0-9]{1,3}\.){3}[0-9]{1,3}/2\.2\.2\.2/' test_sed 
[root@centos-36_2 tmp]# cat test_sed
 1      The Redis server has been running, ignore this step.
 2
 3      2021-07-29 11:31:34-tor.py-478-NOTIFY: Received info
 4      2021-07-29 13:15:15-kln.py-478-INFO: Received NotifyREQ
 5
 6      2021-07-29 13:15:55-tor.py-550-NOTIFY: RPD IRA-RSP CORE-ID TLVs: 
 7      CoreIpAddress: 2.2.2.2
 8      CoreName: broad_Aux
[root@centos-36_2 tmp]# 
[root@centos-36_2 tmp]# ll | grep test_sed
\-rw-r--r-- 1 root    root     325 8 November 17:13 test_sed
\-rw-r--r-- 1 root    root     331 8 November 17:12 test_sed.bak
[root@centos-36_2 tmp]#

Filter lines at the same time for multipoint processing (replace lines 3 and 4 respectively at the same time; replace line 3 with two items at the same time)

[root@centos-36_2 tmp]# sed -ne '3p' -e '4p'  test_sed
 3      2021-07-29 11:31:34-tor.py-478-NOTIFY: Received info
 4      2021-07-29 13:15:15-kln.py-478-INFO: Received NotifyREQ
[root@centos-36_2 tmp]# 
[root@centos-36_2 tmp]# sed -ne '3s/NOTIFY/INFO/p' -e '4s/INFO/ERROR/p'  test_sed
 3      2021-07-29 11:31:34-tor.py-478-INFO: Received info
 4      2021-07-29 13:15:15-kln.py-478-ERROR: Received NotifyREQ
[root@centos-36_2 tmp]# 
[root@centos-36_2 tmp]# sed -ne '3s/NOTIFY/ABC/' -e '3s/info/OOO/p' test_sed
 3      2021-07-29 11:31:34-tor.py-478-ABC: Received OOO
[root@centos-36_2 tmp]#

Use sed through redirection (the same method is used for sed, but the data source is different)

[root@centos-36_2 tmp]# ll | grep test
\-rw-r--r-- 1 root    root     436 8 June 13:54 test_grep
\-rw-r--r-- 1 root    root     325 8 November 17:13 test_sed
\-rw-r--r-- 1 root    root     331 8 November 17:12 test_sed.bak
[root@centos-36_2 tmp]# ll | grep test | sed -n '2p'
\-rw-r--r-- 1 root    root     325 8 November 17:13 test_sed
[root@centos-36_2 tmp]#

Posted by roldahayes on Sun, 26 Sep 2021 00:32:52 -0700

Programmer Group

Linux three swordsmen - sed

Linux three swordsmen - sed

describe

Mode of use

Hot Keywords