Shell script pit avoidance Guide

Hello, I'm Zhang Jintao.

You must be familiar with Shell. We usually think that Shell is the interface between us and the system, executing commands and returning output, such as bash, zsh, etc. Occasionally, some people confuse Shell with Terminal, but this has little to do with this article and will be omitted for the time being.

As a programmer, we may use Shell every day. Occasionally, we will organize some commands and write a Shell script to improve our work efficiency.

However, in a seemingly simple Shell script, there may be a deep pit. Here, I'll give you two simple and similar Shell scripts. You might as well take a look at the output of these two codes:

#!/bin/bash
set -e -u
i=0
while [ $i -lt 6 ]; do
  echo $i
  ((i++))
done

The answer is that only one 0 will be output.

#!/bin/bash
set -e -u
let i=0
while [ $i -lt 6 ]; do
  echo $i
  ((i++))
done

The answer is no output, just exit.

If you can explain the output of the above two paragraphs of code clearly, you can probably skip the rest of this article.

Let me first break down the main knowledge points involved in this code.

Variable declaration

There are many ways to declare variables, but their behavior is different.

We must first have a basic understanding: bash has no type system and all variables are strings. For this reason, if the variable is used for arithmetic operation, the arithmetic operator cannot be written directly as in other programming languages. This allows bash to be interpreted as an operation on strings, not numbers.

Direct declaration

(MoeLove)➜  ~ foo=1+1
(MoeLove)➜  ~ echo $foo
1+1

Direct declaration is the simplest, but as mentioned earlier, direct declaration will be treated as a string by default, and arithmetic operation cannot be performed during declaration.

declare statement

(MoeLove)➜  ~ declare foo=1+1
(MoeLove)➜  ~ echo $foo
1+1

In addition to directly declaring variables, the more common method is to declare variables with declare. However, by default, the declared variables are processed as string s, and normal arithmetic operations cannot be performed.

declare integer attribute

When declare declares a variable, you can add an integer attribute through the - i parameter. When the variable is assigned, arithmetic operation will be performed.

(MoeLove)➜  ~ declare -i bar=1+1
(MoeLove)➜  ~ echo $bar
2

However, it should be noted that after adding the integer attribute, if the string is assigned to it, the parsing will fail, that is, set the value to 0:

(MoeLove)➜  ~ bar=test
(MoeLove)➜  ~ echo $bar
0

let declaration

Alternatively, we can declare variables through the let command. This method allows arithmetic operations during declaration, and also supports assigning other values to this variable.

(MoeLove)➜  ~ let baz=1+1
(MoeLove)➜  ~ echo $baz
2
(MoeLove)➜  ~ baz=moelove.info
(MoeLove)➜  ~ echo $baz
moelove.info

while loop

while list-1; do list-2; done

The while syntax in Bash is like this. After the while keyword is a sequence (list), which can be one or more expressions / statements,

It should be noted that when the return value of list-1 is 0, list-2 will always be executed, and the last return value of the while statement is the return value of the last execution of list-2, or 0 if no statement is executed.

Arithmetic calculation in bash

You must often use this part. Let me introduce several common methods:

Arithmetic extension

There are seven extensions in Bash, and arithmetic extension is only one of them. Specifically, it calculates the value of an expression in a form like $((expression)). For example:

(MoeLove)➜  ~ echo $((3+7))
10
(MoeLove)➜  ~ x=3;y=7
(MoeLove)➜  ~ echo $((x+y))
10

expr command

expr is a command provided by coreutils software package, which can evaluate expressions or compare sizes.

(MoeLove)➜  ~ x=3;y=7
(MoeLove)➜  ~ expr $x + $y
10
# Compare size
(MoeLove)➜  ~ expr 2 \< 3
1
(MoeLove)➜  ~ expr 2 \< 1
0

bc command

By definition, bc is actually a computing language that supports arbitrary precision and interactive execution. It is much more powerful than expr mentioned above, especially it also supports floating-point operations. For example:

General floating point calculation

(MoeLove)➜  ~ echo "scale=2;7/3"|bc
2.33
(MoeLove)➜  ~ echo "7/3"|bc
2

Note: scale needs to be specified manually. It represents the number of digits after the decimal point. By default, the value of scale is 0.

Built in function

bc also has some built-in functions, which can facilitate us to perform some fast calculations. For example, sqrt() can be used to quickly calculate the square root.

(MoeLove)➜  ~ echo "scale=2;sqrt(9)" |bc
3.00
(MoeLove)➜  ~ echo "scale=2;sqrt(6)" |bc
2.44

script

In addition, bc also supports a simple syntax, which can support declaring variables, writing loops and judgment statements. For example, we can print numbers within 20 that can be divided by 3:

(MoeLove)➜  ~ echo "for(i=1; i<=20; i++) {if (i % 3 == 0) i;}" |bc
3
6
9
12
15
18

bash debugging

In fact, there is no built-in debugger in the bash shell. In many cases, repeated operation and printing are used for debugging. But this approach is not efficient enough.

Here is an intuitive and convenient way to debug shell code. Here is a sample shell code.

(MoeLove)➜  ~ cat compare.sh 
#!/bin/bash
read -p "Please enter any number: " val
real_val=66
if [ "$val" -gt "$real_val" ]
then
   echo "The input value is greater than or equal to the preset value"
else
   echo "The input value is smaller than the preset value"
fi

Add execution permission to it, or use bash to execute:

(MoeLove)➜  ~ bash compare.sh 
Please enter any number: 33
 The input value is smaller than the preset value

Detailed mode

By adding the - v option, you can turn on the detailed mode to view the executed commands. Of course, we can also enable this mode by directly adding the - v option on shebang or by adding set -v

(MoeLove)➜  ~ bash -v compare.sh
which () {  ( alias;
 eval ${which_declare} ) | /usr/bin/which --tty-only --read-alias --read-functions --show-tilde --show-dot "$@"
}
#!/bin/bash
read -p "Please enter any number: " val
 Please enter any number: 33
real_val=66
if [ "$val" -gt "$real_val" ]
then
   echo "The input value is greater than or equal to the preset value"
else
   echo "The input value is smaller than the preset value"
fi
 The input value is smaller than the preset value

Use xtrace mode

We can enter the xtrace mode by adding the - x parameter, which is used to debug the variable value of the execution phase.

(MoeLove)➜  ~ bash -x compare.sh
+ read -p 'Please enter any number: ' val
 Please enter any number: 33
+ real_val=66
+ '[' 33 -gt 66 ']'
+ echo The input value is smaller than the preset value
 The input value is smaller than the preset value

Identify undefined variables

In the following example, I deliberately misspelled a character. After executing the script, you will find that there are no errors, but the result is not what we expected. Most of these may be manual errors, so we need to check whether there are unbound variables.

(MoeLove)➜  ~ cat add.sh 
#!/bin/bash
five=5
ten=10
total=$((five+tne))
echo $total
(MoeLove)➜  ~ bash add.sh
5
(MoeLove)➜  ~ bash -u add.sh
add.sh: line 4: tne: unbound variable

Add the - u option to check whether the variable is undefined / bound.

Combined use

The above are several common ways of use. Of course, it can also be used in combination. For example, for the problem with undefined variables above, you can directly see the content of the specific code with - vu.

(MoeLove)➜  ~ bash -vu add.sh
which () {  ( alias;
 eval ${which_declare} ) | /usr/bin/which --tty-only --read-alias --read-functions --show-tilde --show-dot "$@"
}
#!/bin/bash
five=5
ten=10
total=$((five+tne))
add.sh: line 4: tne: unbound variable

Output debugging information to the specified file

Here, I open the debug.log file on a specific FD. Note that this FD needs to be associated with bash_ The xtracefd configuration is consistent. In addition, I modified the variable content of PS4. Its default value is + which looks messy and has no valid information. I set PS4='$LINENO:' to display the line number.

Then set -x at the location where debugging is needed and set +x at the end, so that only the logs of the part I need to debug will be recorded in the debugging log.

(MoeLove)➜  ~ cat compare.sh 
#!/bin/bash
exec 6> debug.log 
PS4='$LINENO: ' 
BASH_XTRACEFD="6" 
read -p "Please enter any number: " val
real_val=66
set -x
if [ "$val" -gt "$real_val" ]
then
   echo "The input value is greater than or equal to the preset value"
else
   echo "The input value is smaller than the preset value"
fi
set +x

echo "End"
(MoeLove)➜  ~ bash compare.sh 
Please enter any number: 88
 The input value is greater than or equal to the preset value
End
(MoeLove)➜  ~ cat debug.log 
8: '[' 88 -gt 66 ']'
10: echo $'\350\276\223\345\205\245\345\200\274\345\244\247\344\272\216\347\255\211\344\272\216\351\242\204\350\256\276\345\200\274'
14: set +x

Here is a simple way to set options through set. Other ways, such as using trap and debugging, are also recommended. We won't start here.

Back to the beginning

Let's use the debugging method just introduced to execute the first two scripts and answer the questions.

first

(MoeLove)➜  ~ bash -xv demo1.sh
#!/bin/bash
set -e -u
+ set -e -u
i=0
+ i=0
while [ $i -lt 6 ]; do
  echo $i
  ((i++))
done
+ '[' 0 -lt 6 ']'
+ echo 0
0
+ (( i++ ))

As can be seen from the above debugging results, the script exits after outputting 0 and executing ((i + +). Why? This is mainly due to the set -e option added at the top of the script.

This option will exit directly when the first non-zero value is encountered. Let's explain:

(MoeLove)➜  ~ i=0
(MoeLove)➜  ~ ((i++))
0
(MoeLove)➜  ~ echo $?
1

It can be seen that after executing ((i + +), the return value is actually 1, so the exit condition of set -e is triggered, and the script exits.

the second

(MoeLove)➜  ~ bash -xv demo2.sh
#!/bin/bash
set -e -u
+ set -e -u
let i=0
+ let i=0

The main difference between the second and the first is the assignment of variables. The return value of let i=0 is 1, so the exit condition of set -e will be triggered. We try to modify the second script and execute it again:

[tao@moelove ~]$ cat demo2-1.sh
#!/bin/bash
set -e -u
let i=1
while [ $i -lt 6 ]; do
  echo $i
  ((i++))
done

[tao@moelove ~]$ bash demo2-1.sh 
1
2
3
4
5

Modify let i=0 to let i=1 to execute successfully as expected.

summary

In this article, we mainly talked about variable declaration, loop, mathematical operation and bash shell debugging in bash shell. Has it inspired you? Welcome to leave a message.

  • Note: This article only discusses Bash Shell

Posted by christillis on Tue, 23 Nov 2021 23:49:36 -0800