I had an interesting request this week with a customer who had some decommissioned Azure virtual machines but for compliance reasons needed to retain the data on the disks for 7 years.
Even on Standard HDD storage a 1TB managed disk might cost around €35 per month which adds up to quite a lot over 7 years. The query was could the archive tier of a general purpose v2 storage account be used?
This would cost less than €1 per TB per month, quite a difference!
Of course the individual files and folders could just be uploaded and stored to a storage account as blob objects but the customer was ideally looking to dump the entire disks as VHD files into archive storage so that they could be restored in future if ever required.
There is an immediate challenge with this as the access tier feature of storage accounts (hot, cool, archive) only supports block blobs but not page blobs which is what VHDs use for random access storage. The other downside to storing large VHD files is that the rehydration costs could work out quite expensive if you need to rehydrate an entire disk from archive storage rather than just the individual files that you require.
In this case, the customer wanted to proceed with archiving the full disks so with the challenge set, off I went to work out and automate the process.
Notes:
Page blobs have a maximum size of 8 TiB
Block blobs have a maximum size of 4.75 TiB
Therefore this process is not suitable for virtual machine disks larger than the maximum block blob size of 4.75 TiB.
Alternative option: I was aware that there will soon be a feature of Azure Backup that will allow you to backup Azure VMs directly to archive tier storage. This feature is currently in a limited public preview that you need to apply for and the limitations are not clearly documented at this point.
The process
- Create a snapshot of your managed disk
- Copy the disk snapshot to a page blob VHD file in a storage account
- Delete disk snapshot as it is no longer required
- Copy the page blob to a new block blob VHD file in storage account using AzCopy
- Delete page blob as it is no longer required
- Set the block blob to the archive tier
If you need to restore from the archive tier the process would be:
- Rehydrate the block blob from the archive tier to the hot tier
- Once rehydrated, copy the block blob to a new page blob VHD file using AzCopy
- Create a new managed disk resource from this page blob
- Create a new VM from this managed disk or mount as a data disk to an existing VM
The code
Below you will find the PowerShell script that I put together for this process. Just supply the parameters for your storage account and virtual machine.
Please use this script at your own risk and don’t delete any data unless you are sure you have working copies available.
Hopefully this script will be of help to some readers out there, please feel free to reach out if you have any questions.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #Provide the name of your resource group where snapshot will be created | |
| $vmResourceGroupName = "" | |
| #Provide the name of your resource group where the blobs will be stored | |
| $storageaccountResourceGroupName = "" | |
| #Provide the name of the virtual machine where the snapshot will be made | |
| $vmName = "" | |
| #Provide storage account name where you want to copy the snapshot. | |
| $storageAccountName = "" | |
| #Name of the storage container where the downloaded snapshot will be stored | |
| $storageContainerName = "" | |
| #Provide the name of the VHD files for blob storage | |
| $pageblobVHDFileName = "pageblobsnapshot.vhd" | |
| $archiveVHDFileName = "blockblobarchive.vhd" | |
| #Set virtual machine context where snapshots will be taken | |
| $vmContext = Get-AzVM -ResourceGroupName $VMResourceGroupName -Name $vmName | |
| #Set the location of virtual machine for snapshot creation | |
| $location = $vmContext.Location | |
| #Set snapshot config settings | |
| #Use the below command for the OS disk | |
| $snapshotConfig = New-AzSnapshotConfig -SourceUri $vmContext.StorageProfile.OsDisk.ManagedDisk.Id -Location $location -CreateOption copy | |
| #For data disks use the below command where [x] is the ID of the data disk | |
| #$snapshotConfig = New-AzSnapshotConfig -SourceUri $vmContext.StorageProfile.DataDisks[0].ManagedDisk.Id -Location $location -CreateOption copy | |
| #Set the name of the snapshot resource | |
| #Use the below command for the OS disk | |
| $snapshotName = $vmContext.StorageProfile.OsDisk.name+"-snapshot" | |
| #For data disks use the below command where [x] is the ID of the data disk | |
| #$snapshotName = $vmContext.StorageProfile.DataDisks[0].name+"-snapshot" | |
| #Create snapshot | |
| New-AzSnapshot -Snapshot $snapshotConfig -SnapshotName $snapshotName -ResourceGroupName $vmResourceGroupName | |
| #Generate the SAS for the snapshot – valid for 24 hours | |
| $snapshotSAS = Grant-AzSnapshotAccess -ResourceGroupName $vmResourceGroupName -SnapshotName $SnapshotName -DurationInSecond 86400 -Access Read | |
| #Get the storage account context for the VHD blobs | |
| $storageAccountContext = (Get-AzStorageAccount -ResourceGroupName $storageaccountResourceGroupName -AccountName $storageAccountName).context | |
| #Generate the SAS for the storage account – valid for 24 hours | |
| $storageaccountSAS = New-AzStorageAccountSASToken -Context $storageAccountContext -Service Blob -ResourceType Service,Container,Object -Permission "rw" -ExpiryTime (Get-Date).AddDays(1) | |
| #Copy snapshot to page blob | |
| azcopy copy $snapshotSAS.accessSAS "https://$storageAccountName.blob.core.windows.net/$storageContainerName/$pageblobVHDFileName$storageaccountSAS" | |
| #Revoke SAS for the snapshot | |
| Revoke-AzSnapshotAccess -ResourceGroupName $vmResourceGroupName -SnapshotName $SnapshotName | |
| #Delete snapshot | |
| Remove-AzSnapshot -SnapshotName $snapshotName -ResourceGroupName $vmResourceGroupName -Force | |
| #Copy page blob to block blob | |
| azcopy copy "https://$storageAccountName.blob.core.windows.net/$storageContainerName/$pageblobVHDFileName$storageaccountSAS" "https://$storageAccountName.blob.core.windows.net/$storageContainerName/$archiveVHDFileName$storageaccountSAS" –blob-type=BlockBlob | |
| #Delete page blog | |
| Remove-AzStorageBlob -Container $storageContainerName -Blob $pageblobVHDFileName -Context $storageAccountContext | |
| #Set block blob to archive tier | |
| $archiveblob = Get-AzStorageBlob -Container $storageContainerName -Blob $archiveVHDFileName -Context $storageAccountContext | |
| $archiveblob.BlobClient.SetAccessTier("Archive", $null, "Standard") |
azurealan.ie 
Thanks so much for this post. This seems like a great method for archiving. Running through the script, I’ve been able to recreate a VM with a system disk, however, when I mount recreated data disks, they show on the VM as being empty. Windows disk manager says they need to be initialised.
Was wondering if you have come across similar and what the issue might be. Any thoughts appreciated!
LikeLike
Hi Daniel, I’m not sure off hand. Are they regular NTFS volumes? Gen1 or Gen2 VMs? Encrypted with SSE only? If you can share more specific details I can try and re-create the issue. Feel free to reach out via Twitter or LinkedIn and we can get a conversation going.
LikeLike
Hello Alan,
Thank you so much for you knowledge. You script help me a lot. 🙂
But I’m having some issues in the final of the script when the azcopy convert the page blob file to block blob. Follow the error:
azcopy : Error: wrong number of arguments, please refer to the help page on usage of this command
No linha:1 caractere:1
+ azcopy copy “https://abelohorizontelab6681.blob.core.windows.net/bkp- …
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (Error: wrong nu…of this command:String) [], RemoteException
+ FullyQualifiedErrorId : NativeCommandError
Could you tell what can be happening?
LikeLike
Hi Marcus,
I think this is a copy and paste issue from the webpage. Can you try copying the script from this link below and see if it works?
Alan
https://gist.githubusercontent.com/alankinane/621a4d5e80f869ba701212bdccc9f546/raw/d4b4e402e2fa6fe0e560e5bd4eb5f71806ab44fe/archive-managed-disks.ps1
LikeLike
Hello Alan,
Thank you so much once again. The process worked but in the final step (set the Access tier to Archive) some error happens.
Believe me, I tried to solve the problem but I don’t know what is happening. The code is correct. I checked in a link from Microsoft Docs and is the same code.
Error:
You cannot call a method on a null-valued expression
No line:3 caractere:1
+ $archiveblob.BlobClient.SetAccessTier(“Archive”, $null, “Standard”)
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : InvokeMethodOnNull
LikeLike