Windows

Azure – Copy data from Windows Virtual Machine to Azure Storage using AzCopy

AzCopy is a command-line utility designed for copying data from and to Microsoft Azure Blob, File and Table storage with optimal performance.

This WIKI will explain how to copy to and from Windows Virtual Machine to Azure blob storage. Download the latest AzCopy tool on Windows on your Virtual Machine

Below is the basic syntax of AzCopy:

AzCopy /Source:<source> /Dest:<destination> [Options]

Installing AzCopy on Windows Machine

1. Using the above link, once you have downloaded the AzCopy tool. The setup file will show as “MicrosoftAzureStorageTools”

download_azcopy

2. Double click on the “MicrosoftAzureStorageTools” installer and the set-up will continue. AzCopy will be installed in a default location – “C:\Program Files (x86)\Microsoft SDKs\Azure\”. You can change the default location, during the installation process.

3. Once the installation is done, you can find the “AzCopy.exe” under the installed folder. The default location is: “C:\Program Files (x86)\Microsoft SDKs\Azure\AzCopy” OR “%ProgramFiles%\Microsoft SDKs\Azure\AzCopy”. If desired, you can add the AzCopy installation location to your system path.

 azcopy_istalled

Upload Blobs To Azure Blob

Upload all blobs in a folder

Below is the syntax:

AzCopy /Source:C:\myfolder /Dest:https://myaccount.blob.core.windows.net/mycontainer /DestKey:key /S

Example:

I have created a folder called “CopyData” that has three empty text files. I will be uploading these three files to an Azure Blob container called “test-azcopy”

 folder_details

Now we will use AzCopy to copy this folder (recursively) to Azure Blob container

Command: .\AzCopy.exe /Source:C:\Users\manju\Desktop\CopyData /Dest:https://manjugtestdisks.blob.core.windows.net/test-azcopy /DestKey:<insert_storage_key> /S

(Note: I have removed my storage access key – for obvious reasons). “/S” parameter will recursively copy the folder and its sub folders.

 AzCopy_Finished

Now go back to the blob container to verify if the folder/files are copied. You may use any tool you want to do this. (Azure Portal, Azure Storage Explorer etc.,)

 copy_verify

We can upload single blob to Azure Blob. Below is the syntax:

Upload a single blob to a virtual directory

Syntax:

AzCopy /Source:C:\myfolder /Dest:https://myaccount.blob.core.windows.net/mycontainer/vd /DestKey:key /Pattern:abc.txt

Upload a single blob
Syntax:

AzCopy /Source:C:\myfolder /Dest:https://myaccount.blob.core.windows.net/mycontainer /DestKey:key /Pattern:”abc.txt”

Download Blobs from Azure Blob Storage

Download all blobs from a container

Below is the syntax:

AzCopy /Source:https://myaccount.blob.core.windows.net/mycontainer /Dest:C:\myfolder /SourceKey:key /S

Example:

Earlier we had uploaded files from a local folder – “CopyData” to Azure Blob container “test-azcopy”. Now we shall download the same three files to a local folder called “DownloadData”.

Command:

.\AzCopy.exe /Source:https://manjugtestdisks.blob.core.windows.net/test-azcopy /Dest:C:\Users\manju\Desktop\DownloadData /SourceKey:<insert_storage_access_key> /S

 download_data

Similarly, we can download single blobs, below is the syntax

Download a single blob
Syntax:

AzCopy /Source:https://myaccount.blob.core.windows.net/mycontainer /Dest:C:\myfolder /SourceKey:key /Pattern:”abc.txt”

Download a single blob from the secondary region

Syntax:
AzCopy /Source:https://myaccount-secondary.blob.core.windows.net/mynewcontainer /Dest:C:\myfolder /SourceKey:key /Pattern:abc.txt

Download blobs with a specific prefix

Syntax:
AzCopy /Source:https://myaccount.blob.core.windows.net/mycontainer /Dest:C:\myfolder /SourceKey:key /Pattern:a /S

Copy blobs in Blob storage

Copy a single blob from one container to another within the same storage account

Syntax:
AzCopy /Source:https://myaccount.blob.core.windows.net/mycontainer1 /Dest:https://myaccount.blob.core.windows.net/mycontainer2 /SourceKey:key /DestKey:key /Pattern:abc.txt

Copy a single blob from one storage account to another

Syntax:
AzCopy /Source:https://sourceaccount.blob.core.windows.net/mycontainer1 /Dest:https://destaccount.blob.core.windows.net/mycontainer2 /SourceKey:key1 /DestKey:key2 /Pattern:abc.txt

Copy a single blob from the secondary region to the primary region

Syntax:
AzCopy /Source:https://myaccount1-secondary.blob.core.windows.net/mynewcontainer1 /Dest:https://myaccount2.blob.core.windows.net/mynewcontainer2 /SourceKey:key1 /DestKey:key2 /Pattern:abc.txt

Copy a single blob and its snapshots from one storage account to another

Syntax:
AzCopy /Source:https://sourceaccount.blob.core.windows.net/mycontainer1 /Dest:https://destaccount.blob.core.windows.net/mycontainer2 /SourceKey:key1 /DestKey:key2 /Pattern:abc.txt /Snapshot

Copy all blobs in a container to another storage account

Syntax:
AzCopy /Source:https://sourceaccount.blob.core.windows.net/mycontainer1 /Dest:https://destaccount.blob.core.windows.net/mycontainer2 /SourceKey:key1 /DestKey:key2 /S

Download files from File storage

Download a single file
Syntax:
AzCopy /Source:https://myaccount.file.core.windows.net/myfileshare/myfolder1/ /Dest:C:\myfolder /SourceKey:key /Pattern:abc.txt

Download all files in a directory
Syntax:
AzCopy /Source:https://myaccount.file.core.windows.net/myfileshare/ /Dest:C:\myfolder /SourceKey:key /S

Upload Files to File storage

Upload a single file
Syntax:
AzCopy /Source:C:\myfolder /Dest:https://myaccount.file.core.windows.net/myfileshare/ /DestKey:key /Pattern:abc.txt

Upload all files in a folder
Syntax:
AzCopy /Source:C:\myfolder /Dest:https://myaccount.file.core.windows.net/myfileshare/ /DestKey:key /S

Upload files matching a specific pattern
Syntax:
AzCopy /Source:C:\myfolder /Dest:https://myaccount.file.core.windows.net/myfileshare/ /DestKey:key /Pattern:ab* /S

Copy files in File storage

Copy from one file share to another
Syntax:
AzCopy /Source:https://myaccount1.file.core.windows.net/myfileshare1/ /Dest:https://myaccount2.file.core.windows.net/myfileshare2/ /SourceKey:key1 /DestKey:key2 /S

Copy from an Azure File share to Blob storage
Syntax:
AzCopy /Source:https://myaccount1.file.core.windows.net/myfileshare/ /Dest:https://myaccount2.blob.core.windows.net/mycontainer/ /SourceKey:key1 /DestKey:key2 /S

Copy a blob from Blob storage to an Azure File share
Syntax:
AzCopy /Source:https://myaccount1.blob.core.windows.net/mycontainer/ /Dest:https://myaccount2.file.core.windows.net/myfileshare/ /SourceKey:key1 /DestKey:key2 /S

Synchronously copy files
Syntax:
AzCopy /Source:https://myaccount1.file.core.windows.net/myfileshare1/ /Dest:https://myaccount2.file.core.windows.net/myfileshare2/ /SourceKey:key1 /DestKey:key2 /S /SyncCopy

Export data from Table storage

Export a table
Syntax:
AzCopy /Source:https://myaccount.table.core.windows.net/myTable/ /Dest:C:\myfolder\ /SourceKey:key

Split an export from Table storage into multiple files
Syntax:
AzCopy /Source:https://myaccount.table.core.windows.net/mytable/ /Dest:C:\myfolder /SourceKey:key /S /SplitSize:100

Import data into Table storage

Import a table
Syntax:
AzCopy /Source:C:\myfolder\ /Dest:https://myaccount.table.core.windows.net/mytable1/ /DestKey:key /Manifest:”myaccount_mytable_20140103T112020.manifest” /EntityOperation:InsertOrReplace

Check out my Powershell Contributions to Microsoft Technet Script Centre

Advertisements

Windows – Convert windows partition from GPT to RAW

This blog will teach you how to convert a “GPT” partition to “RAW” using Windows Command Prompt.

  1. Bring up the windows disk partition tool
    C:\Users\manju>diskpart
  2. List the existing disks
    DISKPART> list disk
  3. Select the disk that you want to change to “RAW” type
    DISKPART> select disk 3
  4. Now type “clean” to convert any partition (GPT in this example) to “RAW”
    DISKPART> clean
  5. Verify by listing the disks
    DISKPART> list disk

 

Screenshot:

convert_GPT_2_RAW