Techno Blog Just another Technology Blog

22May/110

Mount Amazon s3 Bucket on Linux

This article outlines the process for mounting your Amazon s3 bucket as a filesystem on Linux.

Information on how to create a Amazon Simple Storage Service (Amazon S3) account is beyond the scope of this article. See http://aws.amazon.com/s3/ for more information.

  1. Install fuse, load module, and verify
  2. $ yum install fuse
    
    $ modprobe fuse
    
    $ lsmod | grep fuse
    fuse                   83057  0
    

  3. Get s3tools repository
  4. $ cd /etc/yum.repos.d/
    
    $ wget http://s3tools.org/repo/RHEL_5/s3tools.repo
    

  5. Install Amazon S3 Tools
  6. $ yum install s3cmd fuse-s3fs python-hashlib
    

    Amazon S3 Full Access
    Provides full access to all buckets via the AWS Management Console.
    In addition to creating an Amazon account, it is necessary to create a user with appropriate permissions to at least readonly access to S3. In my case I attached a user policy to my user which allows full access to s3.
    {
      "Statement": [
        {
          "Effect": "Allow",
          "Action": "s3:*",
          "Resource": "*"
        }
      ]
    }
    

  7. Create s3 Bucket
  8. $ s3fs -C -c <bucket> -p <accessKeyId> -s <secretAccessKey>
    

    The "accessKeyId" and "secretAccessKey" are generate when the user is created in the Amazon Web Services (AWS) Management Console

  9. Format s3 Bucket
  10. $ s3fs -C -f <bucket> -p <accessKeyId> -s <secretAccessKey>
    

  11. Create Mount Point
  12. $ mkdir -m 755 /mnt/s3drive
    

  13. Add entry to /etc/fstab
  14. $ echo -e "s3fs\t\t/mnt/s3drive\t\tfuse\tsuid,dev,exec,noauto,users,cachedir=/tmp,bucket=<bucket>\t0 0" >> /etc/fstab
    

    Adding the "users" option allows non root users to mount this. Adding "noauto" prevents the mount from being mounted on boot.

  15. If a non root user is going to be mounting this, it is necessary to add that user to the "fuse" group
  16. $ usermod -a -G fuse <user>
    

  17. $ /bin/cat <<EOF > ~/.s3rc
    export AWS_ACCESS_KEY_ID=<accessKeyId>
    export AWS_SECRET_ACCESS_KEY=<secretAccessKey>
    EOF
    
    $ chmod 600 ~/.s3rc
    
    $ echo ". .s3rc" >> ~/.bashrc
    

  18. Mount s3 Drive
  19. $ mount /mnt/s3drive
    
    $ df -hP /mnt/s3drive
    Filesystem            Size  Used Avail Use% Mounted on
    fuse                  2.0T     0  2.0T   0% /mnt/s3drive
    

  20. Unmount when finished
  21. $ umount /mnt/s3drive
    

  22. If by chance you made a mistake or no longer need your s3 bucket, you can delete it from here as well.
  23. $ s3fs -C -k <bucket> -p <accessKeyId> -s <secretAccessKey>
    
    $ s3fs -C -d <bucket> -p <accessKeyId> -s <secretAccessKey>
    

Related Posts