Following are the instructions on how to mount Amazon S3 storage to an AWS Lightsail Instance using S3FS and Fuse.
Prerequisites
sudo yum remove fuse fuse-s3fs
Install dependencies:
sudo yum install openssl-devel gcc libstdc++-devel gcc-c++ fuse fuse-devel curl-devel libxml2-devel mailcap git automake
Run the following command to assume root permissions all the time:
sudo su
Install S3FS-Fuse
Go to your home directory.
cd /home/ec2-user
Clone Fuse
git clone https://github.com/s3fs-fuse/s3fs-fuse.git
Go to cloned directory:
cd s3fs-fuse
./configure
You’ll see something like this:
Compile Fuse using make command:
make
And install:
make install
Add symlink:
ln -s /usr/local/bin/s3fs /usr/bin/s3fs
Install Linux Fuse
cd /home/ec2-user
Visit https://github.com/libfuse/libfuse/releases and look for the latest package, in my case: fuse-3.1.0.tar.gz is the latest, located at https://github.com/libfuse/libfuse/releases/download/fuse-3.1.0/fuse-3.1.0.tar.gz, let’s mark this path down and use wget to download it.
wget https://github.com/libfuse/libfuse/releases/download/fuse-3.1.0/fuse-3.1.0.tar.gz
Extract tar file, move to extracted directory, then follow by installing Fuse and exporting config path:
tar xzf fuse-3.1.0.tar.gz cd fuse-3.1.0 ./configure --prefix=/usr/local make make install export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig ldconfig modprobe fuse
Edit /etc/ld.so.conf
vim /etc/ld.so.conf
Add /usr/local/lib/ to the top, it should look something like this:
Run ldconfig again:
ldconfig
Create Access Key File
Create a file that will hold your AWS access key id and secret access key
echo AWS_ACCESS_KEY_ID:AWS_SECRET_ACCESS_KEY > /etc/passwd-s3fs chmod 600 /etc/passwd-s3fs
Setup Auto Mount (fstab)
mkdir /tmp/cache mkdir /mnt/s3mnt chmod 777 /tmp/cache /mnt/s3mnt
Edit /etc/fstab
This is if you need to auto-mount your bucket when your Amazon Linux machine starts
vim /etc/fstab
Add this line:
s3fs#mybucket /mnt/s3mnt fuse _netdev,allow_other 0 0
Just make sure to provide your AWS bucket name instead of <bucket>
Save the file
Start NETFS service
service netfs start
Or enable it to start on boot:
sudo chkconfig --level 3 netfs on
Now re-mount:
mount -a
TEST
Now, you should be able to cd into your s3 bucket
cd /mnt/s3mnt
You’ll see the same content as is currently in your Amazon S3 bucket
If you run into any issues, turn on the logging to see what the problem is
s3fs <YOURBUCKETNAME> /mnt/s3mnt -o passwd_file=/etc/passwd-s3fs -d -d -f -o f2 -o curldbg
Limitations
Generally, S3 cannot offer the same performance or semantics as a local file system. More specifically:
- random writes or appends to files require rewriting the entire file
- metadata operations such as listing directories have poor performance due to network latency
- eventual consistency can temporarily yield stale data(Amazon S3 Data Consistency Model)
- no atomic renames of files or directories
- no coordination between multiple clients mounting the same bucket
- no hard links
ENJOY!