With the latest version of AzCopy (version 10), you get a new feature which allows you to migrate Amazon S3 buckets to Azure blob storage. In this blog post, I will show you how you can copy objects, folders, and buckets from Amazon Web Services (AWS) S3 to Azure blob storage using the AzCopy command-line utility. This makes it easy to migrate S3 storage to Azure or create a simple backup of your AWS S3 bucket on Azure.
AzCopy will use the Put Block from URL API, which allows you to directly copy files from AWS directly to Azure. This means you will not use a lot of bandwidth from your computer. You can even copy large objects or buckets from S3 to Azure.
Configure access and authorize AzCopy with Azure and AWS
First, you will need to install AzCopy to your machine. After that, you will need to authorize AzCopy with Microsoft Azure and AWS. To authorize with AWS S3, you have to use an AWS access key and a secret access key.
AWS access key and secret access key, and then set these environment variables:
| OS | Command |
|---|---|
| Windows | set AWS_ACCESS_KEY_ID= |
| set AWS_SECRET_ACCESS_KEY= |
|
| Linux | export AWS_ACCESS_KEY_ID= |
| export AWS_SECRET_ACCESS_KEY= |
|
| macOS | export AWS_ACCESS_KEY_ID= |
| export AWS_SECRET_ACCESS_KEY= |
Copy an AWS S3 object to Azure blob
You can copy a simple object using the following command:
azcopy cp "https://s3.amazonaws.com/tomsbucket/tomsobject" "https://tomsstorageaccount.blob.core.windows.net/tomscontainer/tomsblob"
Copy and migrate Amazon S3 folder to Azure
You can copy a folder from the Amazon S3 bucket to the Azure blob storage:
azcopy cp "https://s3.amazonaws.com/tomsbucket/tomsfolder" "https://tomsstorageaccount.blob.core.windows.net/tomscontainer/tomsfolder" --recursive=true
Copy an Amazon S3 bucket to Azure blob storage
You can also copy one or multiple Amazon S3 buckets to Azure:
azcopy cp "https://s3.amazonaws.com/tomsbucket" "https://tomsstorageaccount.blob.core.windows.net/tomscontainer" --recursive=true
I hope this gives you a quick idea of how you can migrate data from Amazon AWS S3 storage to Azure using AzCopy. If you want to know more, check out the official Microsoft Docs about how to copy data from Amazon S3 buckets by using AzCopy.
Tags: Amazon, Amazon S3, AWS, AWS S3, AzCopy, Azure, Azure Blob Storage, Azure Storage, Cloud Storage, copy, Microsoft, Microsoft Azure, Migrate, Migrate Amazon S3 to Azure, Migration, Move, S3, S3 Storage Last modified: June 18, 2019
I’m receiving error:
RESPONSE Status: 403 Forbidden
Content-Length: [200]
Content-Type: [application/xml]
Date: [Mon, 14 Oct 2019 05:54:26 GMT]
Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]
X-Ms-Error-Code: [CannotVerifyCopySource]
X-Ms-Request-Id: [83102e66-101e-0021-3c53-82cbf1000000]
X-Ms-Version: [2018-11-09]
So even though the S3 files are available to the public, I must request the owner of the public repository to supply me with S3 keys for no reason whatsover….sigh
Pity this doesn’t work yet. I recognize I can copy the files to my local computer then re-upload, but like you said, that doesn’t save bandwidth, and my local laptop doesn’t have the space anyway.
I am comfortable with using a GUI tool called Gs Richcopy 360 to migrate AWS S3 to Azure blob , or from windows server to both , it is easy and quick
Thanks Randy for suggesting Gs Richcopy , it works very well with me
I know this is a really old post AND that my following request is certainly a unique use case but…
Wondering if there is a way using azcopy (or at this point I’ll settle for any tool) that can preserve the last modified timestamp on the blob as it is copied from S3 to Azure storage account. Getting it into Azure hasn’t been the problem; it’s preserving the timestamp that’s kicking my butt.