File packaging and compression
Self feeling: it's easy to learn how to pack and compress files. Although there are many compression names, several are not commonly used, and several commonly used options are basically universal. It will produce a good result. Just learn the option of one command and remember the names of the remaining commands.
This article focuses on the tar command. Most of the files on the Internet are packed and compressed.
Common compressed file extensions in linux: *. GZ *. Bz2 *. XZ *. tar *. tar.gz *. tar.bz2 *. tar.xz *. tgz *. Z *. tar.Z
0.compress/uncompress
compress does not retain the source file. The compressed file suffix is. Z
-d decompress, equivalent to uncompress, can be redirected to retain the source file
-c output pressure information to the screen, which can be redirected to keep the source file
-v display details
zcat command
You can view the contents of. Z and. gz compressed files
This command is too old, and it is rare to see *. Z compressed files. You can also extract *. Z compressed files using the gzip command.
1.gzip/gunzip
gzip compressed file will delete the source file and generate a compressed file with. gz as the suffix.
gzip [option]....FILE.....
-c output the compressed file data stream to the screen, you can use redirection to save to the file, to keep the source file.
-d decompress, equivalent to gunzip
-k keeps the source file, valid only in versions later than CentOS 8
-After v compression, the compression ratio can be displayed
-t used to test the integrity of the compressed package
-#Compress according to the specified compression ratio. 6 by default, 1-9 supported
zcat, zmore, zless commands
View the contents of the compressed file without extracting the compressed file with the. gz suffix. zcat FILE.GZ
zgrep command
You can search for keywords directly when decompressing
zgrep keyword / PATH/TO/COMPRESS.gz
Example:
1 #Demonstrate searching for keywords without decompressing 2 [root@CentOS8 tmp]# zgrep http services.gz 3 # http://www.iana.org/assignments/port-numbers 4 http 80/tcp www www-http # WorldWideWeb HTTP 5 http 80/udp www www-http # HyperText Transfer Protocol 6 http 80/sctp # HyperText Transfer Protocol 7 https 443/tcp # http protocol over TLS/SSL 8 https 443/udp # http protocol over TLS/SSL 9 https 443/sctp # http protocol over TLS/SSL 10 gss-http 488/tcp 11 gss-http 488/udp 12 13 14 #-c Options are used in combination with pipe characters, mainly for saving source files 15 [root@CentOS8 tmp]# gzip -c services > services.gz 16 [root@CentOS8 tmp]# ll 17 total 824 18 -rwx------. 1 root root 1379 May 2 08:53 ks-script-drjaqc7p 19 -rw-r--r-- 1 root root 692241 May 3 17:35 services 20 -rw-r--r-- 1 root root 142549 May 3 18:02 services.gz
2.bzip2/bunzip2
The bzip2 compressed file will delete the source file and generate a compressed file with. bz2 as the suffix.
bzip2 [option]....FILE.....
-c output the compressed file data stream to the screen, you can use redirection to save to the file, to keep the source file.
-k keeps the source file after compression, and generates the compressed file at the same time.
-d decompress, equivalent to bunzip2
-#Compress according to the specified compression ratio. 6 by default, 1-9 supported
-q silent compression
-After v compression, the compression ratio is displayed
-r compress the files in the directory separately
bzcat/bzmore/bzless command
View the contents of the compressed file without decompressing the compressed file with the. bz2 suffix, which is the same as zcat.
bzcat FILE.BZ2
bzgrep command
Search the contents of the compressed package without understanding the compressed file with. bz2 suffix
bzgrep keyword / PATH/TO/COMPRESS.bz2
Example:
1 #Keep the source file compressed and display the compression ratio information after compression 2 [root@CentOS8 tmp]# bzip2 -kv services 3 services: 5.334:1, 1.500 bits/byte, 81.25% saved, 692241 in, 129788 out. 4 [root@CentOS8 tmp]# ll 5 total 952 6 -rwx------. 1 root root 1379 May 2 08:53 ks-script-drjaqc7p 7 -rw-r--r-- 1 root root 692241 May 3 18:15 services 8 -rw-r--r-- 1 root root 129788 May 3 18:15 services.bz2 9 -rw-r--r-- 1 root root 142549 May 3 18:02 services.gz 10 11 [root@CentOS8 tmp]# bunzip2 services.bz2
3.xz/unxz
xz compressed file will delete the source file and generate a compressed file with. xz as the suffix.
xz [option]....FILE.....
-c output the compressed file data stream to the screen, you can use redirection to save to the file, to keep the source file.
-k keeps the source file after compression, and generates the compressed file at the same time.
-d decompress, equivalent to unxz
-#Compress according to the specified compression ratio. 6 by default, 1-9 supported
-q silent compression
-After v compression, display compression ratio and other information
xzcat/xzmore/xzless command
View the contents of the compressed file without decompressing the compressed file with the. xz suffix, which is the same as zcat. xzcat FILE.XZ
xzgrep command
Search the contents of the compressed package without decompression
xzgrep keyword / PATH/TO/COMPRESS.xz
Example:
#Keep source file compression, and output compression ratio and other information [root@CentOS8 tmp]# xz -kv services services (1/1) 100 % 103.4 KiB / 676.0 KiB = 0.153 [root@CentOS8 tmp]# ll total 1056 -rw-r--r-- 1 root root 692241 May 3 18:15 services -rw-r--r-- 1 root root 129788 May 3 18:15 services.bz2 -rw-r--r-- 1 root root 142549 May 3 18:02 services.gz -rw-r--r-- 1 root root 105872 May 3 18:15 services.xz [root@CentOS8 tmp]# rm services #Delete source file rm: remove regular file 'services'? y [root@CentOS8 tmp]# xz -dk services.xz #Keep the source file uncompressed [root@CentOS8 tmp]# ll total 1056 -rw-r--r-- 1 root root 692241 May 3 18:15 services -rw-r--r-- 1 root root 129788 May 3 18:15 services.bz2 -rw-r--r-- 1 root root 142549 May 3 18:02 services.gz -rw-r--r-- 1 root root 105872 May 3 18:15 services.xz #Search for compressed packages using xzgrep [root@CentOS8 tmp]# xzgrep http services.xz # http://www.iana.org/assignments/port-numbers http 80/tcp www www-http # WorldWideWeb HTTP http 80/udp www www-http # HyperText Transfer Protocol http 80/sctp # HyperText Transfer Protocol https 443/tcp # http protocol over TLS/SSL https 443/udp # http protocol over TLS/SSL https 443/sctp # http protocol over TLS/SSL #Use-l Option to display long format information [root@CentOS8 tmp]# xz -l services.xz Strms Blocks Compressed Uncompressed Ratio Check Filename 1 1 103.4 KiB 676.0 KiB 0.153 CRC64 services.xz
4.zip/unzip
zip is a compression command packed with compression. It is also a general compression command for all operating systems. Support windows, UNIX, Linux.
zip [OPTION]... Compressed filename. zip source file to be compressed
Note: if you want to use zip to compress the directory, you need to indicate the file name under the directory. If you do not indicate, it will refer to the compressed directory itself. Example: to use zip pam.d.zip /etc/pam.d/*
5.tar
Tools for packaging. Can package multiple files into one file. Tarpack or expand does not delete the source file
1. Create the file tar -c -f -[jJz] /PATH/TO/SOMEFILE.tar FILE
Note: - [jJz] is usually only used when creating compressed files. The decompression tar will automatically call the corresponding compression tool to realize decompression
2. Expand the file tar -x -f -[jJz] /PATH/TO/SOMEFILE.tar -C /PATH/TO/DIR
Note: in fact, when expanding, you don't need to add the - j/-z/-J option. tar will automatically call the corresponding decompression tool to decompress.
3. Do not expand the list tar -t -f -[jJz] /PATH/TO/SOMEFILE.tar
Note: in fact, you don't need to add the - j-z-j option when expanding. tar will automatically call the corresponding decompression tool to decompress.
-c create package file
-x expand package file
-TList the list of files to package without expanding
-f specifies the path of the package file to create or expand
-j call bzip2 compression tool
-J calls xz compression tool
-z calls gzip compression tool
-Z calls the compress compression tool to compress
Simultaneous occurrence of - j-j-z-z is not allowed
-v display process
-p keep the original properties of the source file, only the administrator has permission
-P keeps the absolute path, which allows the bain data to contain the root directory. The default is to automatically cancel the root directory
--xattrs keeps the extended properties of the source file
-C saves the expanded file to the specified file, which defaults to the current directory.
--Exclude = / path / to / {media| somefile} exclude some directories or files from being compressed
Example:
1 #Use-z compress 2 [root@CentOS8 tmp]# tar -zcvf etc.gz /etc/* 3 tar: Removing leading `/' from member names 4 /etc/adjtime 5 tar: Removing leading `/' from hard link targets 6 /etc/aliases 7 /etc/alternatives/ 8 ... 9 #Use - j compression and keep source file properties 10 [root@CentOS8 tmp]# tar -jcvpf etc.bz2 /etc/* 11 tar: Removing leading `/' from member names 12 /etc/adjtime 13 tar: Removing leading `/' from hard link targets 14 /etc/aliases 15 /etc/alternatives/ 16 /etc/alternatives/libnssckbi.so.x86_64 17 ... 18 #Use - J compression, and keep the source file attributes and facl attributes. The - v option is not used here 19 [root@CentOS8 tmp]# tar --xattrs -Jcpf etc.xz /etc/* 20 tar: Removing leading `/' from member names 21 tar: Removing leading `/' from hard link targets 22 [root@CentOS8 tmp]# ll 23 total 11508 24 -rw-r--r-- 1 root root 3654689 May 3 19:05 etc.bz2 25 -rw-r--r-- 1 root root 5053462 May 3 19:03 etc.gz 26 -rw-r--r-- 1 root root 3069780 May 3 19:07 etc.xz 27 28 #Use the - C option to specify the extraction directory 29 [root@CentOS8 tmp]# mkdir test 30 [root@CentOS8 tmp]# tar xf etc.gz -C /tmp/test
6.cpio
Old packing and unpacking tools
-o output mode, packing, packing and sending the file name of standard input to standard output
-i input mode, unpacking
-TVIEW to see the list of included files in the package file passed in by standard input
cpio -tv < NAME.cpio
-A append files to the existing archive
-v display detailed process
-d if a folder is created automatically, it is often used with - i to create a directory automatically when decompressing
cpio -idv < etc.cpio
-F filename replaces standard input or output with the specified filename
-I filename decompress the specified archive file name
-O filename output to the specified archive file name
1 #The example shows that cpio does not distinguish between absolute path and relative path 2 [root@CentOS8 /]# find ./etc | cpio -H newc -o > /tmp/etc.cpio 3 40756 blocks 4 [root@CentOS8 /]# find /etc | cpio -H newc -o > /tmp/etc2.cpio 5 40757 blocks 6 [root@CentOS8 /]# cd /tmp 7 [root@CentOS8 tmp]# ls 8 boot.cpio etc2.cpio etc.cpio 9 [root@CentOS8 tmp]# cpio -tv < etc.cpio 10 drwxr-xr-x 78 root root 0 May 3 18:14 etc 11 lrwxrwxrwx 1 root root 19 May 2 08:51 etc/mtab -> ../proc/self/mounts 12 -rw-r--r-- 1 root root 615 May 2 08:50 etc/fstab 13 -rw------- 1 root root 0 May 2 08:50 etc/crypttab 14 -rw-r--r-- 1 root root 69 May 3 17:31 etc/resolv.conf 15 . . . 16 [root@CentOS8 tmp]# cpio -tv < etc2.cpio 17 drwxr-xr-x 78 root root 0 May 3 18:14 /etc 18 lrwxrwxrwx 1 root root 19 May 2 08:51 /etc/mtab -> ../proc/self/mounts 19 -rw-r--r-- 1 root root 615 May 2 08:50 /etc/fstab 20 -rw------- 1 root root 0 May 2 08:50 /etc/crypttab 21 -rw-r--r-- 1 root root 69 May 3 17:31 /etc/resolv.conf 22 drwxr-xr-x 8 root root 0 May 2 08:51 /etc/dnf 23 #Comparison found, using find /etc generate cpio The root directory should be kept in the file. If you unzip it, you will overwrite the files in the corresponding root directory. It's very dangerous 24 #And use find ./etc Generated cpio The root directory is not reserved in the file. 25 26 #Unzip etc.cpio 27 [root@CentOS8 tmp]# ls 28 boot.cpio etc2.cpio etc.cpio 29 [root@CentOS8 tmp]# mkdir test 30 [root@CentOS8 tmp]# cd test 31 [root@CentOS8 test]# cpio -id <../etc.cpio 32 40756 blocks 33 [root@CentOS8 test]# ls 34 etc
About time cost:
As can be seen from the following small examples, it takes a long time to compress the same file with gzip, bzip2, xz. xz takes the longest time and gzip takes the shortest time.
1 [root@CentOS8 tmp]# rm -fr * 2 [root@CentOS8 tmp]# ls 3 [root@CentOS8 tmp]# cp /etc/services . 4 [root@CentOS8 tmp]# ls 5 services 6 [root@CentOS8 tmp]# time gzip -k services ;time bzip2 -k services ;time xz -k services 7 8 real 0m0.016s #Use gzip Time consumed by compression0.016s 9 user 0m0.013s 10 sys 0m0.003s 11 12 real 0m0.036s #Use bzip2 Time consumed by compression0.036s 13 user 0m0.025s 14 sys 0m0.010s 15 16 real 0m0.184s #Use xz Time consumed by compression0.184s 17 user 0m0.107s 18 sys 0m0.077s 19 [root@CentOS8 tmp]# ll 20 total 1052 21 -rw-r--r-- 1 root root 692241 May 3 18:36 services 22 -rw-r--r-- 1 root root 129788 May 3 18:36 services.bz2 23 -rw-r--r-- 1 root root 142549 May 3 18:36 services.gz 24 -rw-r--r-- 1 root root 105872 May 3 18:36 services.xz
To summarize:
Common compressed file extensions in linux: *. GZ *. Bz2 *. XZ *. tar *. tar.gz *. tar.bz2 *. tar.xz *. tgz *. Z *. tar.Z
The common compression commands are: gzip bzip2 xz. The best compression ratio is xz, but the time consumed is also the highest. Time cost and CPU performance need to be considered.
Several options are basically common: - d decompress - v display detailed process - c output content to screen - k keep source file
tar can package files and directories, and gzip bzip2 xz can be called for compression when packaging
Compression: tar -[zjJ]cv -f filename. Suffix file or directory to be compressed
Query: tar - [zjJ]tv -f filename. Suffix
Unzip: tar - [zjJ]xv -f filename. Suffix - C / path / to / media
The cpio command needs to be used with the find command when compressing. Pay attention to using relative path