With the release of Windows Server 2012 Microsoft offers a new way to store Hyper-V Virtual Machine on a shared storage. In Windows Server 2008 and Windows Server 2008 R2 Hyper-V Microsoft did only offer block-based shared-storage like Fiber channel or iSCSI. With Windows Server 2012 Hyper-V Microsoft allows you to used file-based storage to run Hyper-V Virtual Machine from via the new SMB 3.0 protocol. This means Hyper-V over SMB allows you to store virtual machines on a SMB file share. In the past years I did a lot of Hyper-V implementations working with iSCSI or Fiber channel storage, and I am really happy with the new possibilities SMB 3.0 offers.
The common problem of block storage is that the Hyper-V host has to handle the storage connection. That means if you use iSCSI or fiber channel you have to configure the connection to the storage on the Hyper-V host for example multipath, iSCSI initiator or DSM software. With Hyper-V over SMB you don’t have to configure anything special because SMB 3.0 is built-in to Windows and supporting features like SMB Multichannel are activated and used by default. Of course you have to do some design considerations but this is much less complex than an iSCSI or Fiber Channel implementation.
How did they make it work
The first thing which was important was speed. SMB 3.0 offers a huge performance increase over the SMB 2.x protocol and you totally have to think about it in a different way. There are also a lot of other features like SMB Direct (RDMA), SMB Multichannel or Transparent Failover and many more which help in terms of performance, security and availability, but more on this supporting features in the next post.
Why Hyper-V over SMB?
Well I already mentioned a lot of reasons why you should use Hyper-V over SMB, but if you think about it there are there main reasons why you should use it.
Costs – Windows Server 2012 Hyper-V allows you to build cluster up to 64 nodes and if you build a clusters this size with fiber channel storage this will be quiet an investment in terms of fiber channel hardware such as HBAs, Switches and cables. By using Hyper-V over SMB you can reduce cost for infrastructure dramatically. Sure maybe you have already invested in a fiber channel storage and a fiber channel infrastructure and you don’t have to change that. For example if you have 100 Hyper-V hosts you may have about 200 HBAs and you also need fiber channel switches. What you could do with Hyper-V over SMB, you could create a Scale-Out File Server Cluster with 8 nodes which are attached to the fiber channel and present the storage to the Hyper-V hosts by using a SMB file share. This would save you a lot money.
Flexibility – Another point which I already mentioned is flexibility. By using Hyper-V over SMB you are removing the Storage dependency from the Hyper-V host and add the storage configuration to the Virtual Machine. In this case you don’t have to configure zoning or iSCSI initiators which is making life for Virtualization Administrators much easier. Here are two examples how IT teams can reduce complexity by using Hyper-V over SMB. First in small IT departments you may not have a dedicated storage team and if you have to add an new Hyper-V host or if you have to reconfigure your storage this can be a lot of difficult work for some people who haven’t much experience with the storage. In enterprise scenario you may have a dedicated Storage and a dedicated Virtualization team and in the most cases they have to work really closely together. For example if the Virtualization team adds another Hyper-V host, the Storage team has to configure the Storage for the host on the Storage site. If the Storage team makes changes to the Storage the Virtualization team eventually has to make changes to the Hyper-V hosts. This dependencies can be reduced by adding a layer between Storage and the Hypervisors and in this case this could be a Scale-Out File Server.
Technology – The third point in my list is technology. Microsoft is not really mention this point but since I have worked with different options like iSCSI, fiber channel or SMB I am a huge fan of SMB 3.0. Fiber channel is a great but expensive technology and people who have worked with iSCSI know that there can be a lot of issues in terms of performance. SMB 3.0 has some great supporting features which can help you increase performance, RDMA which is a technology which can increase networking performance by multiple times and SMB Multichannel which allows you to use multiple network adapters for failover and load balancing are working very well and let you make the most out of your hardware. Another part can be security if you think about encrypting iSCSI networks via IPsec you know that this can be something complex, with SMB Encryption there is a very easy solution for that on the SMB scenario.
I hope I could give you a quick introduction to Hyper-V over SMB and why it’s a good idea consider this in your deployment plans. In the next post I will quickly summarize the supporting features in SMB 3.0.Tags: Hyper-V, iSCSI, Microsoft, SMB, SMB 3.0, Storage, Virtulization, Windows Server, Windows Server 2012, Windows Server 2012 R2 Last modified: September 2, 2018
What if I also run Hyper-v on the 2 node file servers cluster? Is there any issues with this configuration? In my situation, I have 2 servers that connect to SAN over HBA cards and 6 server that don’t, but the 2 servers that connect to shared storage also have much resources and i dont want to use only for file server.
I want to have highest HA to my enviroment so can you suggest me? If it is not support to nested hyperv and file server, can i have 1 CSV for hyperv cluster and another CSVs for file server clusters.
Well it’s not supported to use a so called “loopback” configuration where the hyper-v machines access a SMB file share which is running on the same servers as the vms. I would really try to not mix fileshares and hyper-v servers.
Of course you could do some crazy things and mixed but I really would recommend it.
Thanks Thomas for great explanation!
I had trouble understanding the benefits first but your example of 100 Hyper-V host machines helped me to understand.
Keep up the good work.
“dependencies can be reduced by adding a layer between Storage and the Hypervisors and in this case this could be a Scale-Out File Server” – a little bit more about this, please. I’m confused how adding SOFS can reduce dependencies.
I am facing Bit issue with Hyper v with SMB 3.0 shared.
Actually newly created VM’s has to reside in SMB share which is mapped to Hyper v. I am encountering “General access error” which is described in one of your blog, post following steps also i am not able to resolve the issue.
Let me know how to ensure that setup is correct
Has the user you are using access to the share?
Is there possibility to run two VMs as WS2016 on Hyper-V and use RDMA for a quick sharing between VMs ?
When that VMs are on the same Hyper-V will be speed of transfer files higher with RDMA than without RDMA?
[…] I’ll give you a couple of links to start your research. The first is a quick introduction to Hyper-V with SMB 3 written by Thomas Maurer. TechNet has a great article that can show you how to get Hyper-V on SMB […]