How to sync an entire Amazon S3 bucket locally

Edit on GitHub
Feb 23, 2017
Amazon Web Services (AWS)
2 minutes

tl;dr

1brew install awscli # MacOS
2# apt update && apt install awscli -y # Ubuntu
3# pip install --upgrade --user awscli # Unix
4aws configure
5aws s3 ls 
6aws s3 sync s3://my-bucket .

Ubuntu install

this is on a fresh Ubuntu system.

try first:

1apt install awscli -y

1echo `python --version` `pip --version`

install pip

1curl -O https://bootstrap.pypa.io/get-pip.py && python get-pip.py
2pip --version

install AWS CLI

1pip install awscli --upgrade --user

install (Unix)

1apt install python-pip -y
2pip install --upgrade pip
3pip install --upgrade --user awscli

to upgrade, run the same command above. To uninstall, pip uninstall awscli

configuration

1aws configure

AWS Access Key ID [None]: AKIAIOSFODNN7EXAMPLE
AWS Secret Access Key [None]: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
Default region name [None]: us-west-2
Default output format [None]: ENTER

For Amazon S3, region doesn’t matter. But you must enter a value, so pick whichever you like from here.

Default output format can be either json, text, or table. If you don’t specify an output format, json will be used. To skip having to specify anything hit ENTER.

You can change or update these later. The access_key and ID are stored in ~/.aws/credentials and the region and default format are saved in ~/.aws/config. You can also have multiple profiles with different configurations.

List and sync buckets

1# List all buckets
2aws s3 ls

1# Sync the 'my-bucket' bucket on S3 to current directory
2aws s3 sync s3://my-bucket .

Notes