How to save your old virtual machines to low cost archive storage

I had an interesting request this week with a customer who had some decommissioned Azure virtual machines but for compliance reasons needed to retain the data on the disks for 7 years.

Even on Standard HDD storage a 1TB managed disk might cost around €35 per month which adds up to quite a lot over 7 years. The query was could the archive tier of a general purpose v2 storage account be used?

This would cost less than €1 per TB per month, quite a difference!

Of course the individual files and folders could just be uploaded and stored to a storage account as blob objects but the customer was ideally looking to dump the entire disks as VHD files into archive storage so that they could be restored in future if ever required.

There is an immediate challenge with this as the access tier feature of storage accounts (hot, cool, archive) only supports block blobs but not page blobs which is what VHDs use for random access storage. The other downside to storing large VHD files is that the rehydration costs could work out quite expensive if you need to rehydrate an entire disk from archive storage rather than just the individual files that you require.

In this case, the customer wanted to proceed with archiving the full disks so with the challenge set, off I went to work out and automate the process.

Notes:
Page blobs have a maximum size of 8 TiB
Block blobs have a maximum size of 4.75 TiB

Therefore this process is not suitable for virtual machine disks larger than the maximum block blob size of 4.75 TiB.

Alternative option: I was aware that there will soon be a feature of Azure Backup that will allow you to backup Azure VMs directly to archive tier storage. This feature is currently in a limited public preview that you need to apply for and the limitations are not clearly documented at this point.

The process

  1. Create a snapshot of your managed disk
  2. Copy the disk snapshot to a page blob VHD file in a storage account
  3. Delete disk snapshot as it is no longer required
  4. Copy the page blob to a new block blob VHD file in storage account using AzCopy
  5. Delete page blob as it is no longer required
  6. Set the block blob to the archive tier

If you need to restore from the archive tier the process would be:

  1. Rehydrate the block blob from the archive tier to the hot tier
  2. Once rehydrated, copy the block block to a new page blob VHD file using AzCopy
  3. Create a new managed disk resource from this page blob
  4. Create a new VM from this managed disk or mount as a data disk to an existing VM

The code

Below you will find the PowerShell script that I put together for this process. Just supply the parameters for your storage account and virtual machine.

Please use this script at your own risk and don’t delete any data unless you are sure you have working copies available.

Hopefully this script will be of help to some readers out there, please feel free to reach out if you have any questions.


#Provide the name of your resource group where snapshot will be created
$vmResourceGroupName = ""
#Provide the name of your resource group where the blobs will be stored
$storageaccountResourceGroupName = ""
#Provide the name of the virtual machine where the snapshot will be made
$vmName = ""
#Provide storage account name where you want to copy the snapshot.
$storageAccountName = ""
#Name of the storage container where the downloaded snapshot will be stored
$storageContainerName = ""
#Provide the name of the VHD files for blob storage
$pageblobVHDFileName = "pageblobsnapshot.vhd"
$archiveVHDFileName = "blockblobarchive.vhd"
#Set virtual machine context where snapshots will be taken
$vmContext = Get-AzVM ResourceGroupName $VMResourceGroupName Name $vmName
#Set the location of virtual machine for snapshot creation
$location = $vmContext.Location
#Set snapshot config settings
#Use the below command for the OS disk
$snapshotConfig = New-AzSnapshotConfig SourceUri $vmContext.StorageProfile.OsDisk.ManagedDisk.Id Location $location CreateOption copy
#For data disks use the below command where [x] is the ID of the data disk
#$snapshotConfig = New-AzSnapshotConfig -SourceUri $vmContext.StorageProfile.DataDisks[0].ManagedDisk.Id -Location $location -CreateOption copy
#Set the name of the snapshot resource
#Use the below command for the OS disk
$snapshotName = $vmContext.StorageProfile.OsDisk.name+"-snapshot"
#For data disks use the below command where [x] is the ID of the data disk
#$snapshotName = $vmContext.StorageProfile.DataDisks[0].name+"-snapshot"
#Create snapshot
New-AzSnapshot Snapshot $snapshotConfig SnapshotName $snapshotName ResourceGroupName $vmResourceGroupName
#Generate the SAS for the snapshot – valid for 24 hours
$snapshotSAS = Grant-AzSnapshotAccess ResourceGroupName $vmResourceGroupName SnapshotName $SnapshotName DurationInSecond 86400 Access Read
#Get the storage account context for the VHD blobs
$storageAccountContext = (Get-AzStorageAccount ResourceGroupName $storageaccountResourceGroupName AccountName $storageAccountName).context
#Generate the SAS for the storage account – valid for 24 hours
$storageaccountSAS = New-AzStorageAccountSASToken Context $storageAccountContext Service Blob ResourceType Service,Container,Object Permission "rw" ExpiryTime (Get-Date).AddDays(1)
#Copy snapshot to page blob
azcopy copy $snapshotSAS.accessSAS "https://$storageAccountName.blob.core.windows.net/$storageContainerName/$pageblobVHDFileName$storageaccountSAS"
#Revoke SAS for the snapshot
Revoke-AzSnapshotAccess ResourceGroupName $vmResourceGroupName SnapshotName $SnapshotName
#Delete snapshot
Remove-AzSnapshot SnapshotName $snapshotName ResourceGroupName $vmResourceGroupName Force
#Copy page blob to block blob
azcopy copy "https://$storageAccountName.blob.core.windows.net/$storageContainerName/$pageblobVHDFileName$storageaccountSAS" "https://$storageAccountName.blob.core.windows.net/$storageContainerName/$archiveVHDFileName$storageaccountSAS" blobtype=blockblob
#Delete page blog
Remove-AzStorageBlob Container $storageContainerName Blob $pageblobVHDFileName Context $storageAccountContext
#Set block blob to archive tier
$archiveblob = Get-AzStorageBlob Container $storageContainerName Blob $archiveVHDFileName Context $storageAccountContext
$archiveblob.BlobClient.SetAccessTier("Archive", $null, "Standard")

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.