Tag: Scale

AzureStack Admin Portal

Microsoft Azure Stack – Azure Extension in your Datacenter

A couple of weeks ago, I had the chance to attend the Microsoft Azure Certified for Hybrid Cloud Airlift in Bellevue WA, which is close to the Microsoft campus in Redmond. I had the chance to spend the week there and talk with the Microsoft PG about different Azure Stack scenarios. Most of the discussions and presentations are under NDA, but there are a few things I can share, since they are publicly announced. I prepared this blog post already a couple of months ago, when I was talking to a lot of different customers about Azure Stack, and since then Microsoft also shared some new information about the release of Azure Stack Technical Preview 3.

The Azure Stack Announcement

Azure vs Azure Stack

Microsoft announced Azure Stack at Microsoft Ignite in May 2015. Back at this time Microsoft did only mention about the vision of Azure Stack and that it will bring cloud consistency between the Microsoft Azure Public Cloud and your Private Cloud. But Microsoft did not really announce exactly what Azure Stack will be and how it will be implemented in your Datacenter.

During the Microsoft World Wide Partner Conference (WPC 2016), Microsoft announced more information about the availability of Azure Stack. For more information, you can read the Microsoft blog posts, but I tried to summarize the most important parts.

Building a true Hybrid Cloud and Consistency with Microsoft Azure

Azure Stack

This is probably the most important part about Azure Stack today. Microsoft Azure Stack will bring Azure consistency between the Microsoft Azure Public Cloud and your Private Cloud or your Hosters Service Provider Cloud using the Azure Resource Manager. So you will be able to not only operate an Azure-like environment, like you could with Windows Azure Pack and System Center, you now get real consistency between Azure and Azure Stack. You not only get the exact look and feel from the Microsoft Azure Public Cloud, you also can use the same Azure Resource Templates and deployment methods as you can in the Public Cloud. This allows customers to really operate in a Hybrid Cloud environment, between the Microsoft Public Cloud, their own Private Cloud and also local Service Provider Clouds.

Bring the agility and fast-paced innovation of cloud computing to your on-premises environment with Azure Stack. This extension of Azure allows you to modernize your applications across hybrid cloud environments, balancing flexibility and control. Plus, developers can build applications using a consistent set of Azure services and DevOps processes and tools, then collaborate with operations to deploy to the location that best meets your business, technical, and regulatory requirements. Pre-built solutions from the Azure Marketplace, including open source tools and technologies, allow developers to speed up new cloud application development.

The Integrated System Approach

Azure Stack Integrated System

(picture by Microsoft)

Microsoft announced that Azure Stack will be available as an appliance from different hardware vendors in Mid 2017. The confirmed hardware providers delivering Azure Stack Appliance at this point in time will be: Dell EMC, HPE and Lenovo and later in 2017 we will also see an appliance from Cisco, Huawei and Avanade.

The big difference here is that Microsoft delivers the Azure Stack platform first in an appliance way, which is really different from the way they delivered Windows Azure Pack. Windows Azure Pack was based on System Center and Windows Server and every customer could design his own environment based on their needs.

This was great, but also had some huge challenges for customers. Clouds needed different designs, this ended up in very complex design workshops where we basically discussed the customer solutions. The installation and configuration of a Windows Azure Pack platform was also very complex and a lot of work which needed a lot of resources, knowledge and of course a lot of project costs. Before customers could start saving money, they had to invest money to get things up and running. Of course, system integrators like itnetX and others, built automation to spin up clouds based on Windows Azure Pack, but still the investment needed to be done.

The use of an appliance approach not only helps to spin up clouds faster, but also build environments on tested hardware, firmware and drivers. Another point here which makes a great case for an appliance solution, are management and operations. Management and operation of a cloud-like environment is not easy, doesn’t matter what software you are using. Keeping the platform stable, maintained and operational will end up in a lot of work, especially if every cloud looks different. The last thing I want to mention here is upgrading, if you want real Azure consistency, you need to keep up with the ultra-fast pace of the Azure Public Cloud, which is basically impossible or extremely expensive. An integrated system scenario can really help you keep things up-to-date, since updates and upgrades can be pre-tested before they are released for you to deploy. This will help you save a huge amount of testing since every environment looks the same.

Operating Azure Stack

Azure Stack Administration and Operation

As already mentioned, Azure Stack will be delivered as an integrated system. OEMs, will help you to set up and install your Azure Stack appliance in your datacenter, but they will not adequately manage the integrated system. You will need to have some Cloud Operator managing and operating your Azure Stack. With this, all the host will be sealed, and administrators do not have access to the hosts or Hyper-V Manager or Failover Cluster Manager to manage the systems. Instead, Administrators or Cloud Operators will manage the system for a management portal.

Azure Stack Platform

Since this is an integrated system, you don’t even need to care what it is running in the background. But still, for a lot of us, it is still exciting to see how Azure Stack is built. In the back Azure Stack runs on “common” rack mount servers from HPE, Dell, Lenovo, and Cisco, for HPE this is the DL380 Gen9. From the software stack, it is running Windows Server 2016, and the Software Define Datacenter features such as Storage Spaces Direct, the new Windows Server 2016 Software-Defined Networking Stack a Hyper-V. In the release version of Azure Stack, we will see a Hyper-Converged Storage Spaces Direct architecture starting from 4 nodes. On top of this Microsoft used code from Azure to bring the Azure Resource Manager, Azure Resource Providers and the Azure Portal to the Azure Stack.

POC – Azure Stack Development Kit

Azure Stack Development Kit

Very early in the development process of Azure Stack, Microsoft releases Technical Previews to customers so they could test Azure Stack on one node deployments. This is called the Azure Stack POC, and you can download it today on a single physical server, and it was only designed for non-productive, non-HA environments. Microsoft officially announced that they would rename the Azure Stack POC to Azure Stack Development Kit after the General Availability Mid 2017. This is a great solution to quickly spin up a test environment of Azure Stack without having to invest in hardware.

Azure Stack HCI

In March 2019, Microsoft announced a new hyper-converged virtualization solution call Azure Stack HCI, check it out here on my blog.

Azure Marketplace Syndication

Azure Stack Marketplace Syndication

You will be able to create your own Marketplace items in Azure Stack, building your own templates and images and offer them to your customers. One of the greatest editions Microsoft made in the Azure Stack Technical Preview 3 is the Azure Marketplace Syndication. This allows you to get Marketplace items from Azure and offer them in your Azure Stack offering to your customers. With that you don’t need to build all Marketplace items by yourself.

Identity Management

Azure Stack has to be integrated into your datacenter. In terms of Identity, Microsoft allows you to use two ways to integrate. First, and from my site the preferred option, is Azure AD (AAD) which allows you to integrate with an existing Azure Active Directory. Azure AD can be synced and connected with your on-premise Active Directory and this will allow you to log in to Azure as well as Azure Stack. The other option Microsoft is offering is using ADFS to bring identities to your Azure Stack.

Azure Stack use cases

Since Azure Stack is consistent with Microsoft Azure, the question comes up, why are we not just using Azure. There are many good reasons to use Azure, but there are also some challenges with that. Azure Stack can make sense in a couple of scenarios.

  • Data Sovereignty – In some cases data cannot be stored outside of a specific country. With Azure Stack, customers have the option to deploy in even their own datacenter or on a service provider within the same country.
  • Latency – Even Microsoft offers a solution to reduce network latency to Azure, with using Azure Express Route, in some scenarios latency is still a big issue. With Azure Stack can customers place Azure very close to the location where resources are accessed from.
  • Disconnected Scenarios – In some scenarios you really want to benefit from the consistent deployment model, and for example use Azure Resource Manager (ARM), but not everywhere on earth do you have access to Azure or sometimes you have a very bad connection. Think about cruise ships or other scenarios where you need to run IT infrastructure but you are not able to connect to Azure.
  • Private Instance of Azure – For some companies shared infrastructures can be challenging, even security standards in Azure are extremely high, it is not always an option. With Azure Stack, companies can basically spin up their completely own instance of Azure.
  • Differentiation – Service Providers or even Enterprise companies cannot only use the Azure Marketplace, but they can also build their own solutions for the Azure Stack and make them available to their customers.

Pricing and Licensing

As mentioned Microsoft will offer Azure Stack from 5 different OEMs. HPE, Dell and Lenovo will deliver a solution at GA in mid-CY17, Cisco and Huawei will be available later. The hardware needs to be bought directly from the OEM or Partner. Some of them also offer a flexible investment model like the HPE Flexible Capacity. For the pricing model, Microsoft decided to deliver the licensing of Azure Stack on a pay-per-use base. This meets of course the cloud economics and there will be no upfront licensing costs for customers. Services will be typically metered on the same units as Azure, but prices will be lower, since customers operate their own hardware and facilities. For scenarios where customers are unable to have their metering information sent to Azure, Microsoft will also offer a fixed-price “capacity model” based on the number of cores in the system.

Azure Stack will be offered in two different models, Pay-as-you-use model and Capacity model. The pay-as-you-use model is licensed by Microsoft via the Enterprise Agreement (EA) or Cloud Service Provider (CSP) programs. The capacity model is available via EA only. It is purchased as an Azure Plan SKU via normal volume licensing channels. For typical use cases, Microsoft expects the pay-as-you-use model to be the “most economical” option.

The Azure Stack pricing models

Azure Stack will be offered in two different models, Pay-as-you-use model and Capacity model. The pay-as-you-use model is licensed by Microsoft via the Enterprise Agreement (EA) or Cloud Service Provider (CSP) programs. The capacity model is available via EA only. It is purchased as an Azure Plan SKU via normal volume licensing channels. For typical use cases, Microsoft expects the pay-as-you-use model to be the “most economical” option.

Pay-as-you-use model

For the pay-as-you-use model you will you can take advantage of the cloud economics and only pay for resources which are actually consumed, plus additional costs for the Azure Stack hardware and the operations.

Service prices:

  • Base virtual machine $0.008/vCPU/hour ($6/vCPU/month)
  • Windows Server virtual machine $0.046/vCPU/hour ($34/vCPU/month)
  • Azure Blob Storage $0.006/GB/month (no transaction fee)
  • Azure Table and Queue Storage $0.018/GB/month (no transaction fee)
  • Azure App Service (Web Apps, Mobile Apps, API Apps, Functions) $0.056/vCPU/hour ($42/vCPU/month)

Capacity model

For the capacity model, two packages are available which makes you license the physical cores of your Azure Stack system via an annual subscription. The packages are only available via Enterprise Agreement (EA).

  • App Service package ($400/core/year)
    Includes App Service, base virtual machines and Azure Storage
  • IaaS package ($144/core/year)
    Includes base virtual machines and Azure Storage

You will also need additional licenses if you deploy Windows Server and SQL Server virtual machines, like you would do if you are using your traditional Hyper-V servers.

What else will you need

  • Integrated System (hardware) – you will need to purchase the Azure Stack hardware from one of the OEM vendors
  • Support – you will need to purchase support from Microsoft for software support and a support package for the hardware from the hardware provider. If you already have Premier, Azure, or Partner support with Microsoft, your software support is included.
  • Service Providers – Service Provider can also license Azure Stack to others using the CSP (Cloud Solution Provider) channel.


At the Azure Stack GA release this summer, Microsoft will deliver hardware with provides from HPE, Dell and Lenovo. Later in 2017 Microsoft will also deliver Azure Stack with Cisco, Huawei and Avanade hardware. Azure Stack at GA will support 4-12 nodes, 1 single scale-unit and a single region.

Microsoft will also deliver some of the services at General Availability, and will add more and more services over time. At GA we will see:

  • Virtual Machines
  • Storage (Blob, Table and Queue)
  • Networking (Virtual Networks, S2S VPN, …)
  • App Service (in Preview)
  • SQL (in Preview)
  • MySQL (in Preview)

After GA, Microsoft  will continuously deliver additional capabilities through frequent updates. The first round of updates after GA are focused on two areas: 1) enhanced application modernization scenarios and 2) enhanced system management and scale. These updates will continue to expand customer choice of IaaS and PaaS technologies when developing applications, as well as improve manageability and grow the footprint of Azure Stack to accommodate growing portfolios of applications. Please be reminded that this will not just be a product you purchase, think about it as a service which will add features and functionality over time.

The choice for your datacenter

Windows Azure Pack

Microsoft is pushing Azure Stack since it will bring consistency to the Azure public cloud, which means your companies and people need to understand the advantages of using methods like DevOps and Infrastructure in code. This will help you to make the most out of Azure Stack and the Azure Resource Manager. If you already have Microsoft Azure know-how, this is great, because it will also apply to Azure Stack.

No worries, if you are not there yet, or for some reason, this doesn’t make sense to you, Microsoft still has a great solution to build traditional Virtualization platforms together with automation using System Center, Windows Server and if needed Windows Azure Pack. Both solutions, System Center and Windows Azure Pack, will be supported in the future and will get updates.

What's new in Hyper-V 2016

Windows Server 2016 Hyper-V Scale Numbers

Yesterday Microsoft announced the VMware to Hyper-V Migration offer with Windows Server 2016. The Hyper-V team also announced the new scale numbers in Windows Server 2016 Hyper-V. Microsoft announced a Hyper-V Host will support 24TB of RAM and up to 512 CPUs, and up to 16TB and 240 virtual CPUs per Virtual Machine. This are huge number and a huge improvement to the numbers to the numbers of Windows Server 2012 and Windows Server 2012 R2.

Windows Server 2012/2012 R2

Standard & Datacenter

Windows Server 2016 Standard & Datacenter
Physical (Host) Memory Support Up to 4 TB per physical server Up to 24 TB per physical server (6x)
Physical (Host) Logical Processor Support Up to 320 LPs Up to 512 LPs
Virtual Machine Memory Support Up to 1 TB per VM Up to 16 TB per VM (16x)
Virtual Machine Virtual Processor Support Up to 64 VPs per VM Up to 240 VPs per VM (3.75x)

Hyper-V vs. VMware vSphere – Scale

Windows Server 2012 RC Logo

I am still working on my diploma thesis and this is another comparison between Hyper-V and VMware.

This time it is about scale, in the  TechEd session “VIR311 – Compete to Win | Part I: Comparing Core Virtualization Platforms” from Matt McSpirit (@mattmcspirit) did some great work comparing the two hypervisors. I know scale is not everything but Microsoft made huge improvements to Hyper-V 2012.

System Resource Windows Server 2012 Release Candidate Hyper-V VMware vSphere Hypervisor 5.0 VMware vSphere 5.0 Enterprise Plus
Host Logical Processors 320 160 160
Physical Memory 4 TB 32 GB 2 TB
Virtual CPUs per Host 2048 2048 2048
VM Virtual CPUs per VM 64 8 32
Memory per VM 1 TB 32 GB 1 TB
Maximum Virtual Disk 64 TB 2 TB 2 TB
Active VMs per Host 1024 512 512
Cluster Maximum Nodes 64 N/A 32
Maximum VMs 8000 N/A 3000

Check out my Blog post Hyper-V 2012 – Hey I Just Met You And This Is Crazy for more information about the latest version of Hyper-V.

Hyper-V 2012 – Hey I Just Met You And This Is Crazy

Windows Server 2012 RC Logo

Okay I admit it, the title is more a reference to a song than a true fact, because my first contact with the latest Hyper-V release was last September.

Last September Microsoft showed the newest release of Hyper-V at the build conference. Back then I wrote a blog post about the new version of Microsoft Hypervisor Hyper-V called “Hyper-V: Version 3 kills them all“.

Now Microsoft released the Windows Server 2012 and Microsoft Hyper-V 2012 Release Candidate last week. They made a lot of changes since September 2011 and I tried to show this in another blog post (What’s new in Windows Server 2012 Release Candidate Hyper-V: Scale). With these changes I decided to upgrade my blog post from September 2011 with the latest changes made with Windows Server 2012 Release Candidate.

Windows Server 2012 the Cloud OS

Windows Server 2012 Management

First let’s start with Windows Server 2012 as the base of Microsoft cloud strategy. Windows Server 2012 is probably the most significant release of the Windows Server platform ever. Microsoft’s focus in Windows Server 2012 was to make it easy for all to build public, private or hybrid cloud solutions. Microsoft has used the experience and learning from their own Cloud services like Hotmail, Messenger, Office 365, Bing, Windows Azure, and Xbox Live. There are a lot of improvements to manageability, security, scalability, extensibility, predictability and reliability which will also improve the possibilities with Hyper-V. The Power of Many, The Simplicity of One – In technical terms Microsoft made a lot of improvements how you can manage a lot of servers and services, Storage, Networking and PowerShell. Of course there is a lot more, but this are the parts I think are the most important. And here are some keywords to the improvements in Windows Server 2012:

  • Storage improvements – SMB 3.0, SMB transparent Failover, data de-duplication, Storage Spaces, online filesystem repairs, 64TB NTFS volumes, ReFS volumes, etc.
  • NIC Teaming
  • NIC Naming and CDN (consistent device naming)
  • PowerShell v3 – You can now just do everything in PowerShell and even more with 23000 PowerShell cmdlets.
  • Server Dashboard – The new Dashboard lets you manage all servers, or even better, all Services from one place.
  • Multi-tenant – everything seems to be made for that
  • Performance Counters
  • IP address management (IPAM)

Hyper-V Host improvements

Windows Server 2012 Hyper-V Manager

The Hyper-V Host gets a lot of improvements in terms of features and scale.

  • up to 320 logical CPUs
  • supports up to 4 TB RAM
  • no more vCPU:pCPU ration limit
  • up to 2048 Virtual CPUs per Virtual Machine

Hyper-V Virtual Machine improvements

Windows Server 2012 Hyper-V Virtual Machine

Microsoft did a lot to extend the existing Virtual Machine hardware to support even high workload Virtual Machines. Most of the time you were talking with VMware consultants, they tried to say that Hyper-V is not made for Enterprise workloads. Now with the release of Hyper-V 2012 scale should not be an argument anymore.

Hyper-V Networking improvements

Windows Server 2012 Hyper-V Networking

Hyper-V got a lot of improvements in terms of networking. Microsoft realized that networking features are really important if you start to create private and public cloud scenarios and now even create a mix of public and private cloud scenarios without creating a lot of work for the IT teams to reconfigure Virtual Machines.

  • QoS and flexible bandwidth allocation
  • Single-root I/O virtualization or SR-IOV (Direct Access to the physical Network adapter)
  • Network Virtualization
  • PVLAN support
  • Dynamic Virtual Machine Queue (D-VMQ)
  • Receive Side Coalescing (RSC)
  • DHCP Guard
  • Router Guard
  • Port mirroring
  • Port ACLs
  • Trunk mode Allows directing traffic from a group of VLANs to a specific VM
  • IPsec Task offload
  • Integrated Network Adapter Teaming
  • Better Network Adapter Naming and Consistent Network Device Naming
  • Hyper-V Extensible Switch (for example Cisco Nexus 1000v)
  • Data Center Bridging (DCB) – eliminates loss due to queue overflow and to be able to allocate bandwidth on links
  • Network Metering

Hyper-V Clustering improvements

Hyper-V gets also a lot of Cluster improvements. Microsoft is working on Cloud solutions which will give great availability to low cost. For example Hyper-V Replica or Shared-Nothing Live Migration, which allows you to move a Virtual Machine from one Hyper-V host to another host over the Ethernet without the need for a shared storage or a Cluster.

  • supporting up to 4000 VMs per cluster
  • supporting up to 64 Cluster nodes
  • improved Cluster Manager Console
  • Application Monitoring – Application health detection inside the virtual machine
  • New Placement policies – Virtual Machine Priority and enhanced placement
  • Storage Live Migration
  • VM Failover Prioritization
  • Cluster Wide Task Scheduling
  • Hyper-V Replica supporting clustering – replicate a Virtual Machine from one Cluster to another Cluster or Standalone Hyper-V Host
  • No need for Block Storage – you can use SMB Shares
  • Support for Storage Spaces
  • Automated Node Draining – like Maintenance mode in SCVMM
  • Cluster Aware Updating (CAU)
  • Cluster Shared Volume Improvements – BitLocker support, a lot of performance improvements, Self-Healing
  • CSV Block Cache
  • CSV 2.0 (No Redirected I/O for Backup
  • CSV 2.0 Block Level I/O redirection
  • CSV enabled volumes now appear as “CSVFS”
  • No Active Directory dependencies
  • Live Migration Queuing
  • Migrate multiple Virtual Machine at the same time
  • Anti-Affinity VM Rules
  • Dynamic Quorum
  • Guest Clustering via ISCSI, SMB or Fibre Channel

Hyper-V Storage improvements


A I mentioned earlier Microsoft made a lot of improvements in terms of storage in Windows Server 2012 and Hyper-V can take advantage of those which are quiet impressive. For example with the new features in SMB 3.0 you can now use SMB file shares to store your Virtual Machines.

  • New Virtual Disk format (VHDX supports up to 64 TB Virtual Disks)
  • Offloaded Data Transfer – ODX (Open Diagnostic Data Exchange)
  • Live merging of VHDs and Snapshots
  • RDMA
  • SMB 3.0 – Transparent Failover
  • SMB 3.0 Direct
  • SMB 3.0 Multichannel
  • Native 4 KB sector disks support
  • Data De-duplication
  • Virtual Fibre Channel inside the Virtual Machines
  • VM boot from SAN
  • Storage Spaces (Pool Disks or LUNs)
  • New File system ReFS

Hyper-V Management Improvements

Hyper-V Powershell

As everywhere in Windows Server 2012 PowerShell is the key. And the new Server Manager Dashboard Microsoft enables to create Server Groups to manage multiple servers from a single console.

  • PowerShell cmdlets for Hyper-V
  • PowerShell Workflows – commands and tasks across servers
  • Hyper-V Extensible Switch – lets vendors to create “plugins”.
  • Server Manager Dashboard – lets you manage multiple Hyper-V hosts from a single console.
  • System Center Virtual Machine Manager 2012 SP1 – not a part of Windows Server 2012 but will add great management solutions.
  • Improved VM Import
  • Local Hyper-V Administrator Group
  • Client Hyper-V

Hyper-V Live Migration and Disaster Recovery

Windows Server 2012 Hyper-V Replica

Now I think this is maybe the greatest new feature. You can migrate Virtual Machines from one Hyper-V Host to another without Shared Storage or Cluster configuration. This feature is called Shared-Nothing Live Migration. Microsoft also included a new feature called Hyper-V Replica which includes the option to replicate Virtual Machine to another host which can be hosted in the same datacenter, secondary datacenter or even in the cloud.

  • Improved Live Migration
  • Unlimited Simultaneous live migrations
  • Live Storage Migration
  • Shared-Nothing Live Migration – Live Migration to another Hosts (Not clustered) over Ethernet
  • Hyper-V Replica – Replicated Virtual Machines to another Hyper-V host on-premise or public cloud over LAN or WAN connections.

You can get more information and the download link about Windows Server 2012 Release Candidate and Hyper-V Server 2012 Release Candidate.

The Windows Server Team and especially the Hyper-V Team did a great job, and I am sure Hyper-V will gain significant market share in the future.