How to mount Amazon S3 storage to AWS Lightsail Instance

Following are the instructions on how to mount Amazon S3 storage to an AWS Lightsail Instance using S3FS and Fuse. Prerequisites sudo yum remove fuse fuse-s3fs Install dependencies: sudo yum...

Following are the instructions on how to mount Amazon S3 storage to an AWS Lightsail Instance using S3FS and Fuse.

Prerequisites

sudo yum remove fuse fuse-s3fs

Install dependencies: sudo yum install openssl-devel gcc libstdc++-devel gcc-c++ fuse fuse-devel curl-devel libxml2-devel mailcap git automake

Run the following command to assume root permissions all the time: sudo su  

Install S3FS-Fuse

Go to your home directory.

cd /home/ec2-user

Clone Fuse

git clone https://github.com/s3fs-fuse/s3fs-fuse.git

Go to cloned directory:

cd s3fs-fuse

./configure You'll see something like this:

Compile Fuse using make command: make

And install: make install

Add symlink: ln -s /usr/local/bin/s3fs /usr/bin/s3fs

Install Linux Fuse

cd /home/ec2-user

Visit https://github.com/libfuse/libfuse/releases and look for the latest package, in my case: fuse-3.1.0.tar.gz is the latest, located at https://github.com/libfuse/libfuse/releases/download/fuse-3.1.0/fuse-3.1.0.tar.gz, let's mark this path down and use wget to download it.

wget https://github.com/libfuse/libfuse/releases/download/fuse-3.1.0/fuse-3.1.0.tar.gz

Extract tar file, move to extracted directory, then follow by installing Fuse and exporting config path: tar xzf fuse-3.1.0.tar.gz cd fuse-3.1.0 ./configure --prefix=/usr/local make make install export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig ldconfig modprobe fuse Edit /etc/ld.so.conf

vim /etc/ld.so.conf

Add /usr/local/lib/ to the top, it should look something like this:

Run ldconfig again: ldconfig  

Create Access Key File

Create a file that will hold your AWS access key id and secret access key

echo AWS_ACCESS_KEY_ID:AWS_SECRET_ACCESS_KEY > /etc/passwd-s3fs
chmod 600 /etc/passwd-s3fs

Setup Auto Mount (fstab)

mkdir /tmp/cache
mkdir /mnt/s3mnt
chmod 777 /tmp/cache /mnt/s3mnt

Edit /etc/fstab

This is if you need to auto-mount your bucket when your Amazon Linux machine starts

vim /etc/fstab

Add this line:

s3fs#mybucket /mnt/s3mnt fuse _netdev,allow_other 0 0

Just make sure to provide your AWS bucket name instead of 

Save the file

Start NETFS service

service netfs start

Or enable it to start on boot: sudo chkconfig --level 3 netfs on Now re-mount:

mount -a

TEST

Now, you should be able to cd into your s3 bucket

cd /mnt/s3mnt

You'll see the same content as is currently in your Amazon S3 bucket

If you run into any issues, turn on the logging to see what the problem is

s3fs <YOURBUCKETNAME> /mnt/s3mnt -o passwd_file=/etc/passwd-s3fs -d -d -f -o f2 -o curldbg

Limitations

Generally, S3 cannot offer the same performance or semantics as a local file system. More specifically:

- random writes or appends to files require rewriting the entire file

- metadata operations such as listing directories have poor performance due to network latency

- eventual consistency can temporarily yield stale data(Amazon S3 Data Consistency Model)

- no atomic renames of files or directories

- no coordination between multiple clients mounting the same bucket

- no hard links

ENJOY!