It’s fairly typical for customers to want to make a web application highly available be utilising multiple Azure regions. By placing a priority based load balancing service such as Traffic Manager or Azure Front Door you can easily route your traffic to a secondary (failover or DR) region in the event of a service outage. Using PaaS services such as App Service for your web app and Azure SQL for the backend database this process is pretty easy to implement and the Azure SQL database can be asynchronously geo-replicated between regions.
However, what if you are using private endpoints to send traffic over a private link connection from the web app to the Azure SQL database? How can this process be implemented for failover without requiring any changes to DNS services or database connection strings?
The solution is a combination of using an Azure SQL failover group along with registering multiple private endpoints for both of your Azure SQL database servers. Let me explain.
A private endpoint allows the customer to assign a network interface with a private IP address to an Azure PaaS resource. This allows you to completely restrict public access to your PaaS resource and add an additional layer of security to your solution.
This private endpoint, which will be used for Azure SQL in our case, is assigned a private IP address from your virtual network address space effectively joining the resource to your virtual network. The App Service (Standard tier or above) can then integrated with this virtual network which allows outbound traffic to flow privately from the App Service to the Azure SQL database server. This endpoint can be registered in an Azure private DNS zone allowing you to resolve the public FQDN name of your Azure SQL database server to a private IP address on your virtual network.
That’s great, so you end up with something like this.
How many private DNS zones do I need?
If we set this up for another Azure region (our DR site) then we have two distinct setups, with an App Service, an Azure SQL server and a private endpoint for the SQL server registered to a private DNS zone in each region. This solution will work just fine in isolation but there is no awareness of an Azure SQL failover group. If you try to resolve the FQDN of your Azure SQL failover group you will get a public IP address which means you are not using your private link connection.
If you were to register both of your private endpoints to a single Private DNS zone (this is a global service after all) instead of using separate DNS zones then you can get around this issue as the failover group FQDN will propagate to your DNS zone and correctly resolve to the active server endpoint. There is a potential issue with this set up however. This will work fine for complete failover solutions where you are failing over both the App Service and Azure SQL to your DR region but what if you only want to failover the App Service and leave Azure SQL active in the primary region or vice versa?
Failing over a database service can lead to some data loss so ideally you don’t want to initiate a failover unless you really have to. With this scenario, the App Service would be in a different region to your database. App Service VNET integration currently does not support VNET peering and unless you want to go setting up VPN gateways there is no way for the App Service to access the private endpoint of the Azure SQL server in the DR region.
Thankfully, the solution is pretty straightforward. Private Endpoints are able to connect to PaaS services across multiple regions and you can add multiple private endpoints to the same resource.
Therefore, by deploying separate private DNS zones to each region and adding a second private endpoint for each Azure SQL server but deploying it to the virtual network in the other Azure region we can make the service available to both regions via a private link.
Here’s an architectural overview of the solution:
Real world example
In my environment, I have a primary region located in the North Europe region. This contains an App Service hosting my web application and an Azure SQL server and database. My DR site is in West Europe and contains an App Service, and an Azure SQL server. I have a virtual network in both locations and my App Service is integrated with the virtual network in its own region. I have geo-replication in place to replicate my Azure SQL database from North Europe to West Europe.
I will register two private endpoints for this primary Azure SQL Server in North Europe. The first endpoint is placed on a subnet of my virtual network in North Europe. Then I will add a second private endpoint (for the same database server) but this endpoint will be registered to a subnet on the virtual network in my DR region in West Europe.
The process is duplicated for my failover Azure SQL server located in West Europe, i.e. two private endpoints there also, one in either virtual network.
I will have two private DNS zones called privatelink.database.windows.net – one per region. Note: You will need to make sure you are using different resource groups here in order to re-use the same zone name.
These private DNS zones contain the DNS records for both the primary and failover Azure SQL servers. Remember, the App Service is integrated with the virtual network and the private DNS zone has a virtual network link in place so that the App Service can resolve these DNS records.
Here is my primary region virtual network with the two private endpoints registered.
…and the private DNS zone for my primary region.
Then here is the virtual network in my DR region with the two private endpoints.
…and the private DNS zone of my DR region.
These networks are not peered so technically it’s possible to use the same network address space here but this is not recommended as it can get confusing and also things may change in future where peering is required and then you won’t be able to achieve this if the address ranges overlap.
Once all of this is in place you can now resolve the public FQDN and connect to both the primary and failover database servers from the web app in either region over a private link connection.
Most importantly, the Azure SQL failover group (alantest.database.windows.net) will also resolve and connect to the active database server.
..and the same from my DR site. Note the slightly higher latency due to the active Azure SQL server being in a different region.
Finally, if we initiate a failover on the Azure SQL service notice how the private endpoint address is updated automatically by the failover group endpoint and we can still connect to the service in the DR site.
This is the most complete solution for achieving multi-region private link connections to your database servers. By using this method you can achieve high availability even with a partial service outage, i.e. by allowing either the web app or Azure SQL service to failover independently and not being forced to failover both services together. It’s always good to have options and not to be forced into a particular DR strategy particularly when data loss and downtime is an important factor.