Tag: Cluster

Cluster Functional Level and Cluster Upgrade Version

Learn about Windows Server Cluster Functional Levels

A couple of weeks ago, I released a blog post about Hyper-V VM Configuration versions to give an overview about the version history of Hyper-V virtual machines. After that I had the chance to work with John Marlin (Microsoft Senior Program Manager High Availability and Storage) on a similar list of Windows Server Cluster Functional Levels.

Why Cluster Functional Levels are important

With Windows Server 2016, Microsoft introduced a new feature called Cluster OS Rolling Upgrade or Cluster Rolling Upgrade. This feature allows you to upgrade the operating system of the cluster nodes to a new version, without stopping the cluster. With mixed-OS mode, you can have for example 2012 R2 and 2016 nodes in the same cluster. Keep in mind that this should only be temporary, while you are upgrading the cluster. You can basically upgrade node by node, and after all nodes are upgraded, you then upgrade the Cluster functional Level to the latest version.

List of Windows Server Cluster Functional Levels

Since the feature Cluster OS Rolling Upgrade was first introduced with Windows Server 2016, you never really knew about Cluster Functional Levels before. However, it already existed since Windows Server NT4.

Windows Server VersionCluster Functional Level
Windows Server 201910.x
Windows Server 20169
Windows Server 2012 R28
Windows Server 20127
Windows Server 2008 R26
Windows Server 20085
Windows Server 2003 R24
Windows Server 20033
Windows Server 20002
Windows Server NT41

Tips and PowerShell

If you want to know more about Cluster OS Rolling Upgrade, you can check out the Microsoft Docs. Together with John, I created a quick list of some tips for you, and some of the important PowerShell cmdlets.

To check which Cluster Functional Levels your cluster is running on, you can use the following PowerShell cmdlet:

Get-Cluster | Select ClusterFunctionalLevel

If you have upgraded all nodes in the cluster, you can use the Update-ClusterFunctionalLevel to update the Cluster Functional Level. Also make sure that you upgrade the workloads running in that cluster, for example upgrade the Hyper-V Configuration Version or in a Storage Spaces Direct Cluster, the Storage Pool version (Update-StoragePool).

Update-ClusterFunctionalLevel

In Windows Server 2019 the Clustering team introduced a new PowerShell cmdlet to check how many nodes of the cluster are running on which level. Get-ClusterNodeSupportedVersion helps you to identify the Cluster Functional Level and the Cluster Upgrade Version.

Get-ClusterNodeSupportedVersion

Cluster Functional Level Get-ClusterNodeSupportedVersion

This means that the functional level is 10 (Windows 2019).  The Upgrade version column is what you can upgrade to/with, meaning 11.1 or Windows 2019 only.

Cluster Functional Level and Cluster Upgrade Version

This means your Cluster Functional Level is 10.

If you are running System Center Virtual Machine Manager, the Cluster OS rolling upgrade can be fully automated as well. Check out the Microsoft Docs for Perform a rolling upgrade of a Hyper-V host cluster to Windows Server 2016 in VMM.

To find out more about information Cluster operating system rolling upgrade, like how-to, requirements and limitations, check out the Microsoft Windows Server Docs page.



Windows Server

Configure CSV Cache in Windows Server 2012 R2

In Windows Server 2012 Microsoft introduced CSV Cache for Windows Server 2012 Hyper-V and Scale-Out File Server Clusters. The CSV Block Cache is basically a RAM cache which allows you to cache read IOPS in the Memory of the Hyper-V or the Scale-Out File Server Cluster nodes. In Windows Server 2012 you had to set the CSV Block Cache and enable it on every CSV volume. In Windows Server 2012 R2 CSV Block cache is by default enabled for every CSV volume but the size of the CSV Cache is set to zero, which means the only thing you have to do is to set the size of the cache.

 
# Get CSV Block Cache Size
(Get-Cluster).BlockCacheSize
# Set CSV Block Cache Size to 512MB
(Get-Cluster).BlockCacheSize = 512

Microsoft recommends using 512MB as cache on a Hyper-V host. On a Scale-Out File Server node, things are a little bit different. In Windows Server 2012 Microsoft allowed you to use a cache size up to 20% of the server, in Windows Server 2012 R2 Microsoft changed this, so you can now finally use up to 80% of the RAM of a Scale-Out File Server but with a maximum of 64GB.

Back in the days of Windows Server 2012 I made a little benchmark of CSV Cache on my Hyper-V hosts.



Windows Server 2012 R2 Private CLoud Storage and Virtualization

Windows Server 2012 R2 Private Cloud Virtualization and Storage Poster and Mini-Posters

Yesterday Microsoft released the Windows Server 2012 R2 Private Cloud Virtualization and Storage Poster and Mini-Posters. This includes overviews over Hyper-V, Failover Clustering, Scale-Out File Server, Storage Spaces and much more. These posters provide a visual reference for understanding key private cloud storage and virtualization technologies in Windows Server 2012 R2. They focus on understanding storage architecture, virtual hard disks, cluster shared volumes, scale-out file servers, storage spaces, data deduplication, Hyper-V, Failover Clustering, and virtual hard disk sharing.

Bedsides the overview poster, Microsoft Includes the following Mini-Posters:

  • Virtual Hard Disk and Cluster Shared Volumes Mini Poster
  • Virtual Hard Disk Sharing Mini Poster
  • Understanding Storage Architecture Mini Poster
  • Storage Spaces and Deduplication Mini Poster
  • Scale-Out and SMB Mini Poster
  • Hyper-V and Failover Clustering Mini Poster

You can get the posters from the Microsoft download page.



Windows Server

Recommend Hotfixes and Updates for Hyper-V and Failover Clusters

I the last couple of releases I always posted the pages where you could get the list of Recommended Hotfixes and Updates for Windows Server 2012 Failover Clusters and List of Hyper-V and Failover Cluster Hotfixes for Windows Server 2012. I want to upgrade the post with the links for Windows Server 2008 R2, Windows Server 2012 and Windows Server 2012 R2. So you can find all updates from a single site.

Windows Server 2012 R2

Windows Server 2012

Windows Server 2008 R2

Feel free to share this page and I always recommend to get the latest hotfixes when you are deploying a new Hyper-V or Scale-Out File Server environment. And definitely check also Aidan Finns blog from time to time where he does some deeper look at the Knowledge Base articles for Hyper-V.

 



Add Windows-based File Server

Manage SOFS Cluster and File Shares from Virtual Machine Manager

In the past months I did several blog posts about Hyper-V over SMB and Storage Spaces. In small environment management of such a Scale-Out File Server Cluster can be a simple thing because you don’t have a lot of changes, you setup the thing once and this will work for some time. In larger enterprise fabric and storage management is a huge topic, now with Hyper-V over SMB you don’t have to do any zoning or configure iSCSI initiators but you still have to set the right permission on the file share. This is where System Center Virtual Machine Manager comes into play.

Virtual Machine Manager also you to not only manage your iSCSI or fiber channel storage appliances via SMI-S, you can also manage your Scale-Out File Server.

First you have to add the Scale-Out File Server to the SCVMM fabric management. You can simple add a resource and Add a Storage Device. This will open a wizard where you can not only select SAN or NAS storage, but you can also select Widows-based file server.

Add Windows-based File Server

Enter the FQDN of your Fileserver Cluster

Enter Fileserver FQDN

This will scan your File Server Cluster and will show you already existing file shares. You can now match Storage Classifications with the existing file shares.

File Server Fileshares and Classification

After you have connected your Scale-Out File Server you can now create new File Shares and Storage Spaces directly from the Virtual Machine Manager Console.

Create File Shares

After you have created the file share you now have to add the permission for the Hyper-V host to the File Share. Virtual Machine Manager does automatically take care of that if you add the File Share to the Hyper-V Host or if you have a Hyper-V Cluster to the Cluster Object.

Add File Share to Hyper-V host

Now you can start using the file shares for placing Virtual Machines on it. The File Shares classifications will also be available in the VM Clouds.

Cloud Storage Resouces

As you can see, System Center Virtual Machine Manager can make your life a lot easier and helps you manage your whole datacenter fabric, from Compute, network up to storage. In 2013 I did several presentations on Fabric Management with System Center Virtual Machine Manager and two of them are online. You should check out the following posts:

Fabric Management with System Center Virtual Machine Manager (German)

Fabric Management with System Center Virtual Machine Manager at the TechDays Basel (German)



SMB Scale-Out File Server

Hyper-V over SMB: Scale-Out File Server and Storage Spaces

On some community pages my blog post started some discussions why you should use SMB 3.0 and why you should use Windows Server as a storage solution. Let me be clear here, you don’t need Windows Server as a storage to make use of the Hyper-V over SMB 3.0 scenario, you can use storage form vendors like NetApp or EMC as well. But in my opinion you can get a huge benefit by using Windows Server in different scenarios.

  • First you can use Windows Server together with Storage Spaces, which will offer you a really great enterprise and scalable storage solution for low cost.
  • Second you can use Windows Server to mask your existing Storage, by building a layer between the Hyper-V hosts and your storage. So you easily extend your storage even with other vendors.

At the moment there are not a lot of vendors out there which offer SMB 3.0 in there storage solution. EMC was one of the first supporting SMB 3.0 and with ONTAP 8.2 Netapp is now supporting SMB 3.0 as well. But if you want to build a SMB layer for a storage which does not support SMB 3.0. to mask your storage so you can mix it with different vendors or using it with Windows Server 2012 Storage Spaces, the solution would be the Scale-Out File Server cluster. Microsoft offers file server cluster for a while now, but since this was an active/passive cluster, this was not really a great solution of a Hyper-V storage environment (even if a lot of small iSCSI storage boxes are active/passive as well).

Basically what the Scale-Out File Server let you do it so cluster up to 10 file servers which all will share CSVs (Cluster Shared Volumes) like you know from Hyper-V hosts and present SMB shares which are created on the CSV volumes. And the great thing about that, every node can offers the same share this will be a active/active solution up to 8 nodes. Together with SMB Transparent Failover the Hyper-V host does not really get any storage downtime if on of the SOFS nodes fails.

SMB Scale-Out File Server

For the storage guys out there think about the cluster nodes as your storage controllers. Most of the time you will have 2 controllers for fail-over and a little bit of manual load balancing where one LUN is offered by controller 1 and the other LUN is offered by controller 2. With the Scale-Out File Server you don’t really have that problem since the SMB share is offered on all hosts at the same time and up to 8 “controllers”. With Windows Server 2012 one Hyper-V host connected to one of the SOFS nodes and used multiple paths to this node by using SMB Multichannel, the other Hyper-V host connected automatically to the second SOFS node so both nodes are active at the same time. In case on of the SOFS nodes dies, the Hyper-V host fails over to the other SOFS node without any downtime for the Hyper-V Virtual Machines.

In Windows Server 2012 R2, Microsoft worked really hard to make this scenario even better. In Windows Server 2012 R2 a Hyper-V host can be connected to multiple SOFS node at the same time. Which means that VM1 and VM2 running on the same Hyper-V hosts can be offered by two different SOFS nodes.

Advantages of the Scale-Out File Server

  • Mask your storage and use different vendors
  • Scale up to 8 nodes (controllers)
  • Active/Active configuration
  • Transparent Failover
  • Supporting features like SMB Multichannel and SMB Direct
  • Easy entry point with SMB shares
  • Easy configuration, Hyper-V host and Cluster objects need access on the shares
  • Same Windows Server Failover Cluster Technology with the same management tools

Storage Spaces

As already mentioned you can use your already existing storage appliance as storage for your Scale-Out File Server CSVs or you could use Windows Server Storage Spaces which allow you to build great storage solution for a lot less money. Again, the Scale-Out File Server Cluster and Windows Server Storage Spaces are two separate things you don’t need a SOFS cluster for Storage Spaces and you don’t need Storage Spaces for a SOFS cluster, but of course both solutions work absolutely great together.

Windows Server Storage Spaces vs Traditional Storage

Microsoft first released there Software Defined Storage solution called Storage Spaces in Windows Server 2012 and this allows you basically to build your own storage solution based on a simple JBOD hardware solution. Storage spaces is a really cost-effective storage solution which allows companies to save up to 75% of storage costs in compare to traditional SAN storage.  It allows you to pool disks connected via SAS  (in Windows 8 and Windows 8.1 USB works as well for home users) and create different Virtual Disks (not VHDs) on these Storage Pools. The Virtual Disks, also called Storage Spaces, can have different resiliency levels like Simple, Mirror or Parity and you can also create multiple disks on one storage pool and even use thing provisioning. This sounds a lot like a traditional storage appliance right? True, this is not something totally different, this is something storage vendors do for a long time. But of course you pay a lot of money for this blackbox the storage vendors offer you. With Windows Server Storage Spaces Microsoft allows you to build our “own storage” on commodity hardware which will save you a lot of money.

Storage Space

This is not only just an “usable solution” this solution comes with some high-end storage features, which make the Storage Spaces and Windows File Server a perfect storage at low cost.

  • Windows Server Storage Spaces let you use cheap hardware
  • Offers you different types of resiliency, like Simple (Stripe), Mirror or Parity (also 3-way Mirror and Parity)
  • Offers you thin-provisioning
  • Windows Server File Server allows you to share the Storage via SMB, iSCSI or NFS.
  • Read-Cache – Windows Server CSV Cache offers you Memory based Read-Cache (up to 80% in Windows Server 2012 R2)
  • Continuous availability – Storage Pools and Disks can be clustered with the Microsoft Failover Cluster so if one server goes down the virtual disks and file shares are still available.
  • SMB copy offload – Offloading copy actions to the storage.
  • Snapshots – Create Snapshots and  clone virtual disks on a storage pool.
  • Flexible resiliency options – In Windows Server 2012 you could create a Mirror Spaces with a two-way or three-way mirror, a Parity Space with a single parity and a Simple Space with no data resiliency. New in R2 parity spaces can now be used in clustered pools and there is also a new dual parity option. (enhanced in 2012 R2)
  • Enhanced Rebuilding – Speed of rebuilding of failed disks is enhanced. (enhanced in 2012 R2)
  • Storage Tiering – Windows Server 2012 R2 allows you to use different kind of disks and automatically moves “hot-data” from SAS disks to fast SSD storage. (new in 2012 R2)
  • Write-Back Cache – This feature allows data to be written to SSD first and moves later to the slower SAS tier. (new in 2012 R2)
  • Data Deduplication – Data Deduplication was already included in Windows Server 2012 but it is enhanced in Windows Server 2012 R2, and allows you to use it together with Cluster Shared Volumes (CSV) and supports VDI virtual machines. (enhanced in 2012 R2)

You can get more information about Storage Spaces in Windows Server 2012 R2 in my blog post: What’s new in Windows Server 2012 R2 Storage Spaces

Combine Windows Server Storage Spaces and the Scale-Out File Server Cluster

As mentioned both of this techologies do not require each other, but if you combine them you get a really great solution. You can build your own storage based on Windows Server, which not only allows you to share storage via SMB 3,0 it also allows you to share storage via NFS or iSCSI.

Windows Server 2012 Storage Spaces and File Server

A lot of concerns I have heard, was about scale of Storage Spaces. But as I can see scale is absolutely no problem for Windows Server Storage Spaces.  First of all you can build up to 8 nodes in a single cluster which basically would mean you create a 8 node active/active solution. With SMB Multichannel you can use multiple NICs for example 10GbE, infiniband, or even faster network adapters. You can also make use of RDMA which brings latency down to a minimum.

Scale Windows Server Storage SpacesTo scale this even bigger you can go to way, you could setup a new Scale-Out File Server Cluster and create new file shares where virtual machines can be placed. Or you could extend the existing cluster with more servers and more shared SAS disks chassis which don’t have to be connected to the existing servers. This is possible because of  features like CSV Redirected mode hosts can access disks from other hosts even if they are not connected directly via SAS, instead the node is using the Ethernet connection between the hosts.

Scale Windows Server Storage Spaces 2

New features and enhancements in Windows Server 2012 R2 and System Center 2012 R2

With the 2012 R2 releases of Windows Server and System Center Microsoft made some great enhancements to Storage Spaces, Scale-Out File Server, SMB, Hyper-V and System Center. So if you have the chance to work with R2 make sure you check the following:

  • Flexible resiliency options – In Windows Server 2012 you could create a Mirror Spaces with a two-way or three-way mirror, a Parity Space with a single parity and a Simple Space with no data resiliency. New in R2 parity spaces can now be used in clustered pools and there is also a new dual parity option. (enhanced in 2012 R2)
  • Enhanced Rebuilding – Speed of rebuilding of failed disks is enhanced. (enhanced in 2012 R2)
  • Storage Tiering – Windows Server 2012 R2 allows you to use different kind of disks and automatically moves “hot-data” from SAS disks to fast SSD storage. (new in 2012 R2)
  • Write-Back Cache – This feature allows data to be written to SSD first and moves later to the slower SAS tier. (new in 2012 R2)
  • Data Deduplication – Data Deduplication was already included in Windows Server 2012 but it is enhanced in Windows Server 2012 R2, and allows you to use it together with Cluster Shared Volumes (CSV) and supports VDI virtual machines. (enhanced in 2012 R2)
  • Read-Cache – Windows Server CSV Cache offers you Memory based Read-Cache (up to 80% in Windows Server 2012 R2)
  • Management – Management of Hyper-V and Scale-Out File Servers as well as Storage Spaces right in System Center 2012 R2 Virtual Machine Manager.
  • Deployment – Deploy new Scale-Out File Server Clusters with and without Storage Spaces directly from System Center 2012 R2 Virtual Machine Manager via Bare-Metal Deployment.
  • Rebalancing of Scale-Out File Server clients – SMB client connections are tracked per file share (instead of per server), and clients are then redirected to the cluster node with the best access to the volume used by the file share. This improves efficiency by reducing redirection traffic between file server nodes.
  • Improved performance of SMB Direct (SMB over RDMA) – Improves performance for small I/O workloads by increasing efficiency when hosting workloads with small I/Os.
  • SMB event messages -SMB events now contain more detailed and helpful information. This makes troubleshooting easier and reduces the need to capture network traces or enable more detailed diagnostic event logging.
  • Shared VHDX files – Simplifies the creation of guest clusters by using shared VHDX files for shared storage inside the virtual machines.. This also masks the storage for customers if you are a service provider.
  • Hyper-V Live Migration over SMB – Enables you to perform a live migration of virtual machines by using SMB 3.0 as a transport. This allows you to take advantage of key SMB features, such as SMB Direct and SMB Multichannel, by providing high speed migration with low CPU utilization.
  • SMB bandwidth management – Enables you to configure SMB bandwidth limits to control different SMB traffic types. There are three SMB traffic types: default, live migration, and virtual machine.
  • Multiple SMB instances on a Scale-Out File Server – Provides an additional instance on each cluster node in Scale-Out File Servers specifically for CSV traffic. A default instance can handle incoming traffic from SMB clients that are accessing regular file shares, while another instance only handles inter-node CSV traffic.

(Source: TechNet: What’s New for SMB in Windows Server 2012 R2)

I hope I could help with this blog post to understand a little bit more about the Scale-Out File Server and Storage Spaces, and how you can create a great storage solution for your cloud Environment.

Btw the pictures and information are taken from people like Bryan Matthew (Microsoft), Jose Barreto (Microsoft) and Jeff Woolsey (Microsoft).

 

 



Install SCVMM HA Cluster

How to Install a Highly Available SCVMM Management Server

Since System Center Virtual Machine Manager is starting to get more and more important and starts to be a critical application for your environment especially if your are using Hyper-V Network Virtualization and SCVMM is your centralized policy store, you should install Virtual Machine Manager highly available. To do this Virtual Machine Manager uses the Failover Cluster feature integrated in Windows Server.

Before you begin check this important nodes

  • Not only the SCVMM Management Server should be high available, also the SQL Server where the SCVMM database is installed and the file share for the library share should be highly available.
  • You can have two or more SCVMM Management server in a cluster, but only one node will be active.
  • You will need to configure Distributed Key Management. You use distributed key management to store encryption keys in Active Directory Domain Services (AD DS) instead of storing the encryption keys on the computer on which the VMM management server is installed.

Here the quick steps what you do:

  • Install two server for the SCVMM Management servers (Cluster nodes) with Windows Server 2012 or Windows Server 2012 R2.
  • Install all the SCVMM prerequisites (ADK and SQL native client)
  • Create a SCVMM Service Account which has local admin rights on the SCVMM nodes.
  • Create a container in Active Directory Domain Services for the Distributed Key Management.
  • Set all IP addresses, you may also configure an independent Heartbeat network
  • Install the Failover Cluster feature on both server.

After you have done this steps you start to create a Failover Cluster with both nodes.

Create SCVMM Cluster

You built the SCVMM Cluster, now you have to install the SCVMM Service on the first node. You can start the SCVMM Installer and will automatically detected the SCVMM Cluster and will ask you if you want to install the SCVMM server as high available installation.

Install SCVMM HA Cluster

The installation is now more less the same as for a standalone Virtual Machine Manager Server, expect you have to use Distributed Key Management and one screen where you configure the SCVMM Cluster Role with a name and an IP address.

Install SCVMM HA Cluster Configuration

After you have installed the first node you can now run the setup on the second node. The setup does also see the cluster and the SCVMM Cluster role, and will ask you about the configuration. Many of the settings cannot be changed because they are the same on all nodes.

Add Node to SCVMM Cluster

After you have installed both nodes you can see the SCVMM Cluster Role in the Failover Cluster Manager.

VMM Failover Cluster Role

And you can of course also see all your Virtual Machine Management servers in the Virtual Machine Manager console.

VMM Console

I hope this helps you to install System center 2012 SP1 or System Center 2012 R2 Virtual Machine Manager as Highly Available installation. If you need more information check out TechNet: How to Install a Highly Available VMM Management Server