If you’re looking to replicate an S3 bucket across regions, there are a few things to keep in mind. The first is that you’ll need to create a new bucket in each region. Next, you’ll need to set up replication between the buckets. Finally, you’ll need to configure your applications to use the replicated buckets. To create a new bucket in a different region:
- Go to the AWS Management Console and open the S3 console.
- In the navigation pane on the left, select Buckets and then click Create Bucket.
- In the Create Bucket dialog box, type a name for your new bucket and then click Create.
- The newly created bucket will be displayed in the list of buckets on the right side of the S3 console window.
- To copy your existing data into your new bucket, select it and then click Copy Data (or Copy Files).
- To set up replication between your buckets, select your original bucket and then click Properties (or Click Replication Settings). In the Properties dialog box, under Replication Settings, click Enable Replication and enter an optional description for your replication configuration (for example, “Replicate my data between two regions”). Then click OK to close the dialog box. To set up applications to use replicated buckets:
- Go to each application’s settings page (for example, by clicking its name in your browser’s address bar).
- Under General Settings or Preferences (depending on which application you’re using), locate and open its S3 section (usually at http://
/s3/). 3a If you’re using Amazon SES for sending email messages from applications hosted on Amazon EC2 instances running Amazon Simple Email Service (Amazon SES) or Amazon CloudFront distributions: Under Email Delivery Settings , make sure that Use Amazon SES For Email Delivery is selected . This will enable ..
If you want to store multiple copies of objects in your S3 buckets in different regions, S3 can be set up to automatically replicate objects from a source bucket into replica buckets around the world, increasing performance and access latency of your applications.
It should be noted that if the objects you’re replicating are publically available over the internet, such as a media bucket for uploaded content, you can simply use AWS’s CloudFront CDN to cache content on edge locations everywhere.
How Does Replication Work?
Replication basically runs whenever you upload or update an object in a bucket. You create a replication rule, which defines the source and destination bucket. Objects uploaded into the source bucket will be uploaded into the destination bucket. For replication to work, you’ll need Versioning enabled. This will increase storage costs slightly, depending on how much you overwrite data.
There are some limitations. AWS doesn’t replicate deleted objects, so you’ll need to delete each copy manually. And, while replication usually happens fairly quickly, it can take up to 15 minutes or more if you’re replicating large objects.
When an object is replicated, a metadata tag is set in the source object. All of the replicas will have the REPLICA tag, and the source object will be tagged COMPLETED. This means you can have bdirectional replication—two buckets with replication rules pointing at each other. S3 won’t replicate uploaded objects with the replica tag, otherwise you’d be stuck in an infinite replication loop.
This is called multi-master replication, but there’s no system in place for ensuring that objects uploaded later will be replicated later. In this setup, it’s possible for data to get out of sync if you’re updating two objects simultaneously, and it’s possible to be operating on stale data as it syncs from one master to another.
Setting Up Replication
From the S3 Management Console, navigate to the source bucket. You’ll need to turn on Versioning under “Properties:”
Once that’s turned on, you can add a new replication rule under Management > Replication > Add Rule.
By default, it will replicate all objects, though you can set a filter if you’d like.
Under Destination, select the destination bucket. You can also configure the storage class for the replica objects, as well as setting the owner if you’re transferring across accounts. There’s also an option for “S3 Replication Time Control,” which guarantees objects will be replicated within 15 minutes, backed by a service level agreement.
On the next screen you’ll need to select or create an IAM role for the replication. The permissions are handled automatically, just enter a name.
Click “Next,” and click “Save.” The rule should be active immediately; you can test uploading an object, and you should see it replicated to the destination bucket, then you’ll see the replication status tag change to COMPLETED.