background

In the work, because the log files generated by our project are very important, and the local disk space is limited for a long time, so considering the backup scheme, we originally intended to save them on nas, but due to various reasons, we communicated with the operation and maintenance department to suggest saving them on oss.

Because linux natively supports shell, in order to reduce dependence, we consider using shell script to upload OSS directly. We found some information on the Internet, as shown in:

• https://bbs.aliyun.com/read/233456.html

pit

However, there is a hole in the script when it is tried, and it is specially recorded as follows:

1. Character comparison prompt exception

The above screenshot character comparison will prompt:

./oss.sh: 13: ./oss.sh: [get: not found
./oss.sh: 16: ./oss.sh: [put: not found
./oss.sh: 32: ./oss.sh: [put: not found

It should be changed to the format above

2. When splicing the url, the bucket is also brought in. 3. The splicing signature is not correct. After a long study, it is found that "ා / bin/sh" should not be used, but "# / bin/bash" should be used. This is a big hole...

Modified version

The modified version is shown below, which needs to be self fetched:

#!/bin/bash

host="oss-cn-shanghai.aliyuncs.com"
bucket="bucket name"
Id="AccessKey ID"
Key="Access Key Secret"
# Parameter 1, PUT: upload, GET: Download
method=$1
# Parameter 2: local source file path when uploading and oss source file path when downloading
source=$2
# Parameter 3: OSS target file path when uploading and local target file path when downloading
dest=$3

osshost=$bucket.$host

# Check method
if test -z "$method"
then
    method=GET
fi

if [ "${method}"x = "get"x ] || [ "${method}"x = "GET"x ]
then
    method=GET
elif [ "${method}"x = "put"x ] || [ "${method}"x = "PUT"x ]
then
    method=PUT
else
    method=GET
fi

#Verify upload target path
if test -z "$dest"
then
    dest=$source
fi

echo "method:"$method
echo "source:"$source
echo "dest:"$dest

#Check whether the parameter is empty
if test -z "$method" || test -z "$source" || test -z "$dest"
then
    echo $0 put localfile objectname
    echo $0 get objectname localfile
    exit -1
fi

if [ "${method}"x = "PUT"x ]
then
    resource="/${bucket}/${dest}"
    contentType=`file -ib ${source} |awk -F ";" '{print $1}'`
    dateValue="`TZ=GMT date +'%a, %d %b %Y %H:%M:%S GMT'`"
    stringToSign="${method}\n\n${contentType}\n${dateValue}\n${resource}"
    signature=`echo -en $stringToSign | openssl sha1 -hmac ${Key} -binary | base64`
    echo $stringToSign
    echo $signature
    url=http://${osshost}/${dest}
    echo "upload ${source} to ${url}"
    curl -i -q -X PUT -T "${source}" \
      -H "Host: ${osshost}" \
      -H "Date: ${dateValue}" \
      -H "Content-Type: ${contentType}" \
      -H "Authorization: OSS ${Id}:${signature}" \
      ${url}
else
    resource="/${bucket}/${source}"
    contentType=""
    dateValue="`TZ=GMT date +'%a, %d %b %Y %H:%M:%S GMT'`"
    stringToSign="${method}\n\n${contentType}\n${dateValue}\n${resource}"
    signature=`echo -en ${stringToSign} | openssl sha1 -hmac ${Key} -binary | base64`
    url=http://${osshost}/${source}
    echo "download ${url} to ${dest}"
    curl --create-dirs \
      -H "Host: ${osshost}" \
      -H "Date: ${dateValue}" \
      -H "Content-Type: ${contentType}" \
      -H "Authorization: OSS ${Id}:${signature}" \
      ${url} -o ${dest}
fi

Execute command:

#upload
$ ./oss.sh put a.gz c.gz

#download
$ ./oss.sh get c.gz d.gz