Mounting Aliyun OSS to Local Use Based on OSSFS

Keywords: git sudo github curl

Summary:
Limitations

Compared with the local file system, the function and performance provided by ossfs have some limitations. Specifically include:

  • Random or additional writing of a file results in rewriting of the entire file.
  • Metadata operations, such as list directory, have poor performance because they require remote access to oss servers
  • rename operations on files/folders are not atomic
  • When multiple clients mount the same oss bucket, they depend on the user to coordinate the behavior of each client. For example, avoid multiple clients writing the same file and so on.
  • Hardlink is not supported.
  • Not suitable for high concurrent read/write scenarios, which will increase the load of the system

1. Install the corresponding dependency libraries before installing ossfs:

Ubuntu 14.04:

sudo apt-get install automake autotools-dev g++ git libcurl4-gnutls-dev \
                     libfuse-dev libssl-dev libxml2-dev make pkg-config
CentOS 7.0:

sudo yum install automake gcc-c++ git libcurl-devel libxml2-devel \
                 fuse-devel make openssl-devel

Install ossfs source code, available from github

git clone https://github.com/aliyun/ossfs.git
cd ossfs
./autogen.sh
./configure
make
sudo make install

Configuration of corresponding files
Set the bucket name, access key/id information and store it in the / etc/passwd-ossfs file. Note that the permissions of this file must be set correctly. It is recommended to set it to 640.

echo my-bucket:my-access-key-id:my-access-key-secret > /etc/passwd-ossfs
chmod 640 /etc/passwd-ossfs

Mount the corresponding bucket of oss to the local mount point
ossfs my-bucket my-mount-point -ourl=my-oss-endpoint

example:
//Will my-bucketthis bucket Mount to/tmp/ossfs Under the catalogue, AccessKeyId yes faint, AccessKeySecret yes123,oss endpoint yes http://oss-cn-hangzhou.aliyuncs.com

echo my-bucket:faint:123 > /etc/passwd-ossfs
chmod 640 /etc/passwd-ossfs
mkdir /tmp/ossfs
ossfs my-bucket /tmp/ossfs -ourl=http://oss-cn-hangzhou.aliyuncs.com

Uninstall oss

uninstall bucket:

umount /tmp/ossfs # root user
fusermount -u /tmp/ossfs # non-root user

Problems encountered:

ossfs test /ossfsfile -ourl=http://oss-cn-hangzhou.aliyuncs.com
[ERR] curl.cpp:CheckBucket(2625): Check bucket failed, OSS response: <?xml version="1.0" encoding="UTF-8"?>
<Error>
  <Code>RequestTimeTooSkewed</Code>
  <Message>The difference between the request time and the current time is too large.</Message>
  <RequestId>58F6D89ECBB6F0795271EEBA</RequestId>
  <HostId>test.oss-cn-hangzhou.aliyuncs.com</HostId>
  <MaxAllowedSkewMilliseconds>900000</MaxAllowedSkewMilliseconds>
  <RequestTime>2017-04-19T04:21:16.000Z</RequestTime>
  <ServerTime>2017-04-19T03:25:18.000Z</ServerTime>
</Error>

[ERR] curl.cpp:CheckBucket(2625): Check bucket failed, OSS response: <?xml version="1.0" encoding="UTF-8"?>
<Error>
  <Code>RequestTimeTooSkewed</Code>
  <Message>The difference between the request time and the current time is too large.</Message>
  <RequestId>58F6D89ECBB6F0795271EEC1</RequestId>
  <HostId>test.oss-cn-hangzhou.aliyuncs.com</HostId>
  <MaxAllowedSkewMilliseconds>900000</MaxAllowedSkewMilliseconds>
  <RequestTime>2017-04-19T04:21:16.000Z</RequestTime>
  <ServerTime>2017-04-19T03:25:18.000Z</ServerTime>
</Error>

[CRT] s3fs.cpp:s3fs_check_service(3690): invalid credentials - result of checking service.
ossfs: Check OSS service failed. Run with -f option for more details.

The most important sentence of the above mistake is:

The difference between the request time and the current time is too large.

This means that the request time is too different from the current time.
Try to view local time

[root@storm1 tmp]# date
Wed Apr 19 16:47:47 CST 2017

Sure enough, the difference of one hour only needs to update the synchronization time.

ntpdate cn.pool.ntp.org
19 Apr 15:52:39 ntpdate[71442]: step time server 51.15.41.135 offset -3357.358058 sec

Mount again

ossfs test /ossfsfile -ourl=http://oss-cn-hangzhou.aliyuncs.com

Attempt to add debug information logs when you encounter errors

ossfs test /ossfsfile -ourl=http://oss-cn-hangzhou.aliyuncs.com -o dbglevel=debug -f -d > /tmp/fs.log 2>&1

View the log information and resolve it.

Posted by Digwood on Sun, 07 Jul 2019 16:23:42 -0700