At the Microsoft Build conference this year, Microsoft announced Nested Virtualization for Azure Virtual Machines, and last week Microsoft announced the availability of these Azure VMs. Nested Virtualization allows you to run a hypervisor inside a virtual machine running on a hypervisor, which means you can run Hyper-V within a Hyper-V virtual machine or an Azure virtual machine. Kind of like Inception for virtual machines.
You can use Nested Virtualization since Windows Server 2016 or the same release of Windows 10, for more details on this, check out my blog post: Nested Virtualization in Windows Server 2016 and Windows 10
With the release of the Azure Dv3 and Ev3 VM sizes:
- D2-64 v3 instances are the latest generation of General Purpose Instances. D2-64 v3 instances are based on the 2.3 GHz Intel XEON ® E5-2673 v4 (Broadwell) processor and can achieve 3.5GHz with Intel Turbo Boost Technology 2.0. D2-64 v3 instances offer the combination of CPU, memory, and local disk for most production workloads.
- E2-64 v3 instances are the latest generation of Memory Optimized Instances. E2-64 v3 instances are based on the 2.3 GHz Intel XEON ® E5-2673 v4 (Broadwell) processor and can achieve 3.5GHz with Intel Turbo Boost Technology 2.0. E2-64 v3 instances are ideal for memory-intensive enterprise applications.
With the upgrade to new Intel Broadwell processors, Microsoft enabled Nested Virtualization. This will enable a couple of different scenarios if you create a virtual machine running Windows Server 2016 or Windows Server 2019.
- You can run Hyper-V Containers (Windows Containers with additional isolation) inside an Azure VM. With future releases, we will also be able to run Linux Containers in Hyper-V Containers running on a Windows Server OS.
- You can quickly spin up and shut down new demo and test environments, and you only pay when you use them (pas-per-use)
How to Setup Nested Virtualization in Azure
Deploy Azure VM
To set up Nested Virtualization inside an Azure virtual machine, you first need to create a new virtual machine using one of the new instance sizes like Ev3 or Dv3 and Windows Server 2016. I also recommend installing all the latest Windows Server patches to the system.
Optional: Optimize Azure VM Storage
This step is optional, but if you want to better performance and more storage for your nested virtual machines to run on, this makes sense.
In my case, I attached two additional data disks to the Azure VM. Of course, you can choose more or different sizes. Now you can see two new data disk inside your Azure virtual machine. Do not format them, because we are going to create a new storage spaces pool and a single virtual disk, so we get the performance form both disks at the same time. In the past, this was called disk striping.
With that, you can create a new Storage Spaces Storage Pool and a new Virtual Disk inside the VM using the storage layout “Simple” which configures it as striping.
I also formatted the disk and set the drive letter to V, and this will be the volume where I will place my nested virtual machines.
Install Hyper-V inside the Azure VM
The next step would be to install the Hyper-V role in your Azure virtual machine. You can use PowerShell to do this since this is a regular Windows Server 2016. This command will install Hyper-V and restart the virtual machine.
Install-WindowsFeature -Name Hyper-V -IncludeManagementTools -Restart
After the installation you have Hyper-V installed and enabled inside your Azure virtual machine, now you need to configure the networking for the Hyper-V virtual machines. For this, we will use NAT networking.
Configure Networking for the Nested Environment
To allow the nested virtual machine to access the internet, we need to set up Hyper-V networking in the right way. For this, we use the Hyper-V internal VM Switch and NAT networking. I described this here: Set up a Hyper-V Virtual Switch using a NAT Network
Create a new Hyper-V Virtual Switch
First, create an internal Hyper-V VM Switch
New-VMSwitch -SwitchName "NATSwitch" -SwitchType Internal
Configure the NAT Gateway IP Address
The Internal Hyper-V VM Switch creates a virtual network adapter on the host (Azure virtual machine), this network adapter will be used for the NAT Gateway. Configure the NAT gateway IP Address using New-NetIPAddress cmdlet.
New-NetIPAddress –IPAddress 172.21.21.1 -PrefixLength 24 -InterfaceAlias "vEthernet (NATSwitch)"
Configure the NAT rule
After that, you have finally created your NAT network, and you can now use that network to connect your virtual machines and use IP addresses from 172.21.21.2-172.21.21.254.
New-NetNat –Name MyNATnetwork –InternalIPInterfaceAddressPrefix 172.21.21.0/24
Now you can use these IP Addresses to assign this to the nested virtual machines. You can also set up a DHCP server in one of the nested VMs to assign IP addresses automatically to new VMs.
Optional: Create NAT forwards inside Nested virtual machines
To forward specific ports from the Host to the guest VMs, you can use the following commands.
This example creates a mapping between port 80 of the host to port 80 of a virtual machine with an IP address of 172.21.21.2.
Add-NetNatStaticMapping -NatName "MyNATnetwork" -Protocol TCP -ExternalIPAddress 0.0.0.0 -InternalIPAddress 172.21.21.2 -InternalPort 80 -ExternalPort 80
This example creates a mapping between port 82 of the virtual machine host to port 80 of a virtual machine with an IP address of 172.21.21.3.
Add-NetNatStaticMapping -NatName "MyNATnetwork" -Protocol TCP -ExternalIPAddress 0.0.0.0 -InternalIPAddress 172.16.0.3 -InternalPort 80 -ExternalPort 82
Optional: Configure the default virtual machine path
Since I have created an extra volume for my nested virtual machines, I configure this as the default path for virtual machines and virtual hard disks.
Set-VMHost -VirtualHardDiskPath V:\VMs -VirtualMachinePath V:\VMs
Create Nested Virtual Machines inside the Azure VM
Now you can start to create virtual machines inside the Azure VM. You can, for example, use an existing VHD/VHDX or create a new VM using an ISO file as you would do on a hardware Hyper-V host.
Some crazy stuff to do
There is a lot more you could do, not all of it makes sense for everyone, but it could help in some cases.
- Running Azure Stack Development Kit – Yes, you can run the Azure Stack Development Kit, if you use large enough Azure virtual machine.
- Configure Hyper-V Replica and replicate Hyper-V VMs to your Azure VM running Hyper-V.
- Nested a Nested virtual machine in an Azure VM – You could enable nesting on a VM running inside the Azure VM so you could do a VM inside a VM, inside a VM. Just follow my blog post to create a nested Virtual Machine: Nested Virtualization in Windows Server 2016 and Windows 10
In my opinion, Nested Virtualization is mostly helpful if you run Hyper-V Containers. But it also works great, if you want to run some virtual machines inside an Azure VM, to run a lab or to test something.
