Windows Server 2016 Technical Preview 4 has been released and bring nested virtualization into the game. At the moment it is restricted to Windows 10 10565+ and Windows Server 2016 TP4+ (including nano server) builds and from these operating system/hypervisor as physical host as well.

This gives us a big opportunity for self studying new technologies such as software define storage, software define network and so on with less hardware requirements.

I did decide to try deploying an hyper converged hyper-v cluster to play with storage space direct aka S2D. This feature is fully supported with nano server TP4 and requires at least 4 nodes, each block being copied 3 times for domain awareness being on hosts and disks.

As a reminder here is an hyper converged hyper-v cluster pattern:

I will then create 4 virtual machines running nano server, enable nested virtualization on these machines to install hyper-v role and then create a cluster with these 4 nodes.

Storage will be provided by hosts themselves using S2D (storage space direct) and we will then be able to create nested VM from this cluster.

Prerequisites to use this script are :

– an existing domain with DHCP available (will be used by management network)

– 300 80 GB free space ( 5 fixed dynamic data disks of 10GB will be attached to each nano host for S2D), ideally on a partition using ReFS

– 16GB memory, nested virtualization with hyper-v requires 4GB fixed memory per virtualized host

– an existing virtual switch which will be used to connect management network

Networks on cluster will be defined as follow with X being 11 for nano1, 12 for nano2, 13 for nano3, 14 for nano4:

Management Network: Provided by your DHCP

Heartbeat Network:  10.10.10.X/27

Storage Network: 10.11.11.x/27

LiveMigration Network: 10.12.12.X/27

a virtual switch called “vswitch” is configured on each nano host and is connected to the Management Network through the “VM” network card.

each nano host will have the following virtual machine configuration and will have nested virtualization enabled:

(note: My Management network is internal but served by a pfsense providing external connectivity, dhcp and so on)

To use the script, start it from an elevated powershell window: ./hypervconverged.ps1 –domain “YourexisitingDomain”

You will be prompted to enter the desired local administrator password

in addition you can add “–includeSOFS” parameter if you want to also deploy a scale out file server on the cluster and/or “-includeVMM” if the cluster will be later on managed by Virtual Machines Manager 2016 TP4.

“@edit1: Updated script revision to 2: added support for VMM, changed fixed disks to dynamic disks to decrease space required.”

Once the script has finished running, you will need to configure your quorum yourself, I didn’t include it on purpose since I’ll be using cloud witness myself.

The script does not contain any error handling (such as try/catch/finally) and is for lab/tests purposes only.

script content :

######################################################################################
#Description: Create a nested hyper converged hyper-v cluster using storagespace direct
#Author: Alain VETIER @ MICROSOFT
#Email: alainv@microsoft.com
#Date: 11/23/2015
#Revison 2
#based on the following articles:
#https://technet.microsoft.com/en-us/library/mt126167.aspx
#https://technet.microsoft.com/en-us/library/dn550728.aspx
#https://technet.microsoft.com/en-us/library/hh831718.aspx
#
#usage: run from an elevated ps shell on your physical host:
# .hypervconverged.ps1 -domain (-domain being your existing domain with DHCP available)
# add -includeSOFS if you also want a scale out file server role to be dpeloyed on the cluster
# add -includeVMM if the cluster is going to be managed by Virtual Machine Manager 2016
#########################################################################################
# This sample script is not supported under any Microsoft standard support program or service. 
# The sample script is provided AS IS without warranty of any kind. Microsoft further disclaims 
# all implied warranties including, without limitation, any implied warranties of merchantability 
# or of fitness for a particular purpose. The entire risk arising out of the use or performance of 
# the sample scripts and documentation remains with you. In no event shall Microsoft, its authors, 
# or anyone else involved in the creation, production, or delivery of the scripts be liable for any 
# damages whatsoever (including, without limitation, damages for loss of business profits, business 
# interruption, loss of business information, or other pecuniary loss) arising out of the use of or 
# inability to use the sample scripts or documentation, even if Microsoft has been advised of the 
# possibility of such damages
########################################################################################

[CmdletBinding()]
Param (

#Password for Local Administrator
[parameter(Mandatory=$true)]
[securestring]$adminpass,

#Existing domain to join
[parameter(Mandatory=$true)]
[string]$domain,

#Include SOFS role or not?
[switch]$includeSOFS,

#Managed by VMM ?
[switch]$includeVMM
)

#Variables that you can change
#$iso = "path yo your Windows Server 2016 TP4 iso"
#$dir = "a folder you want to use for staging nano servers images"
#$vmpath = "path to your VMs on your host"
#switch = "name of the virtual switch that you want to use for management"
#cluster = name of your hyper-v cluster
#sofs = name of your scale out file server role
#pswitch = name of the private switch which will be created

#set variables
$nanos = @('NANO1','NANO2','NANO3','NANO4')                                     
$iso = "c:sourceswindowsen_windows_server_2016_technical_preview_4_x64_dvd_7258292.iso"
$dir = "d:nano"
$vmpath = "D:VMs"
$switch = "internal"
$pswitch= "PrivateS2D"
$cluster = "HYPVCVGD"
$sofs = "SOFSS2D"

#check prerequisite features
$Install = Add-WindowsFeature Hyper-V-Powershell, RSAT-Clustering-Powershell

#Create Private virtual switch
New-VMSwitch -Name $pswitch -SwitchType Private

#Create staging directory
md $dir
cd $dir
Mount-DiskImage -ImagePath $iso | Out-Null
$drive = (Get-DiskImage -ImagePath $iso | Get-Volume).DriveLetter
cp ${drive}:nanoserverConvert-WindowsImage.ps1 $dir
cp ${drive}:nanoserverNanoServerImageGenerator.psm1 $dir


Import-Module .NanoServerImageGenerator.psm1

#Creates nano hosts
foreach ($nano in $nanos)
{
djoin.exe /provision /domain $domain /machine $nano /savefile .odjblob.$nano | Out-Null
if ($includeVMM)
    {
    New-NanoServerImage -MediaPath ${drive}: -BasePath .base -TargetPath $vmpath$nano$nano.vhdx -Clustering -Storage -GuestDrivers -Packages Microsoft-NanoServer-Compute-Package,Microsoft-Windows-Server-SCVMM-Compute-Package,Microsoft-Windows-Server-SCVMM-Package -DomainBlobPath .odjblob.$nano -MaxSize 5GB -AdministratorPassword $adminpass | Out-Null
    }
else {
    New-NanoServerImage -MediaPath ${drive}: -BasePath .base -TargetPath $vmpath$nano$nano.vhdx -compute -Clustering -Storage -GuestDrivers -DomainBlobPath .odjblob.$nano -MaxSize 5GB -AdministratorPassword $adminpass | Out-Null
    }
New-VM -Name $nano -VHDPath $vmpath$nano$nano.vhdx -Generation 2 -BootDevice VHD -SwitchName $switch -Path $vmpath -MemoryStartupBytes 4GB -Verbose | Out-Null
Set-VMProcessor -VMName $nano -ExposeVirtualizationExtensions $true -Count 2 -Verbose
Set-VMNetworkAdapter -VMName $nano -MacAddressSpoofing On -Verbose
1..5 | % { New-VHD -Path $vmpath$nanoData$psitem.vhdx -Dynamic -SizeBytes 10GB; Add-VMHardDiskDrive -VMName $nano -ControllerType SCSI -ControllerNumber 0 -ControllerLocation $psitem -Path $vmpath$nanoData$psitem.vhdx}
Start-VM $nano | Out-Null 
}

Start-Sleep -Seconds 30 
Dismount-DiskImage -ImagePath $iso | Out-Null

#Configure nano hosts: fix file server issue, set drive policy
Invoke-Command -ComputerName $nanos[0],$nanos[1],$nanos[2],$nanos[3] {
Set-StorageSetting -NewDiskPolicy OfflineShared
Enable-WindowsOptionalFeature -Online -FeatureName File-Services,CoreFileServer,Storage-replica -NoRestart | Out-Null
Start-Sleep -Seconds 30
Restart-Computer -force
}

Start-Sleep -Seconds 30

#Create cluster
New-Cluster -Name $cluster -Node $nanos -NoStorage | Out-Null

Start-Sleep -Seconds 30

#enable storage space direct
Get-Cluster $cluster | Enable-ClusterS2D -S2DCacheMode Disabled | Out-Null

if ($includeSOFS)
{
#Create a Scale-Out File Server
Add-ClusterScaleoutFileServerRole -name $sofs -cluster $cluster | Out-Null
}

# Create storage pool & vdisk
Invoke-Command -ComputerName $cluster {
New-StoragePool -StorageSubSystemFriendlyName *cluster* -FriendlyName S2D -ProvisioningTypeDefault Fixed -PhysicalDisk (Get-PhysicalDisk | ? CanPool -eq $true) | Out-Null
New-Volume -StoragePoolFriendlyName S2D -FriendlyName vDisk1 -Size 50GB -FileSystem CSVFS_ReFS -PhysicalDiskRedundancy 2 | Out-Null
md c:clusterstorageVolume1VMs
}

# add network cards to nodes and configure network
# Since everything is nested on a single host, we could use only a single NIC because network in the end will be going only through internal physical host netwok
# but for demonstration purpose we will will best practice and create multiple NIC to "isolate" traffic in the cluster


#Add network cards to VM
foreach ( $nano in $nanos) {
Add-VMNetworkAdapter -VMName $nano -SwitchName $pswitch -Name HeartBeat -DynamicMacAddress | Out-Null
Add-VMNetworkAdapter -VMName $nano -SwitchName $pswitch -Name Storage -DynamicMacAddress | Out-Null
Add-VMNetworkAdapter -VMName $nano -SwitchName $pswitch -Name LiveMigration -DynamicMacAddress | Out-Null
Add-VMNetworkAdapter -VMName $nano -SwitchName $switch -Name VM -DynamicMacAddress | Out-Null
Set-VMNetworkAdapter -VMName $nano -Name VM -MacAddressSpoofing On
}

#Configure nodes network
Invoke-Command -ComputerName $nanos[0],$nanos[1],$nanos[2],$nanos[3] {
Rename-NetAdapter -Name "Ethernet" -NewName "Management"
Rename-NetAdapter -Name "Ethernet 2" -NewName "HeartBeat"
Rename-NetAdapter -Name "Ethernet 3" -NewName "Storage"
Rename-NetAdapter -Name "Ethernet 4" -NewName "LiveMigration"
Rename-NetAdapter -Name "Ethernet 5" -NewName "VM"

if ($env:COMPUTERNAME -like 'NANO1')
{
$ip = '11'    
}
if ($env:COMPUTERNAME -like 'NANO2')
{
$ip = '12'    
}
if ($env:COMPUTERNAME -like 'NANO3')
{
$ip = '13'    
}
if ($env:COMPUTERNAME -like 'NANO4')
{
$ip = '14'    
}

New-NetIPAddress -InterfaceAlias "HeartBeat" -IPAddress 10.10.10.$ip -PrefixLength 27
New-NetIPAddress -InterfaceAlias "Storage" -IPAddress 10.11.11.$ip -PrefixLength 27
New-NetIPAddress -InterfaceAlias "LiveMigration" -IPAddress 10.12.12.$ip -PrefixLength 27

Set-DnsClient -InterfaceAlias "HeartBeat" -RegisterThisConnectionsAddress $false
Set-DnsClient -InterfaceAlias "Storage" -RegisterThisConnectionsAddress $false
Set-DnsClient -InterfaceAlias "LiveMigration" -RegisterThisConnectionsAddress $false

New-VMSwitch -Name vSwitch -NetAdapterName VM -AllowManagementOS $false | out-null
Set-VMHost -VirtualHardDiskPath c:clusterstorageVolume1Vms -VirtualMachinePath c:clusterstoragevolume1Vms

}

Start-Sleep -Seconds 30

##Configure cluster network

#identify cluster networks

$Managementnet = (Get-ClusterNetworkInterface -Node NANO1 -Cluster $cluster | where Name -like "*Management").Network
$Clusternet = (Get-ClusterNetworkInterface -Node NANO1 -Cluster $cluster | where Name -like "*HeartBeat*").Network
$Storagenet = (Get-ClusterNetworkInterface -Node NANO1 -Cluster $cluster | where Name -like "*Storage").Network
$LiveMigrationnet = (Get-ClusterNetworkInterface -Node NANO1 -Cluster $cluster | where Name -like "*LiveMigration").Network

#Rename Cluster Networks
(Get-ClusterNetwork –Name $Managementnet.Name -Cluster $cluster).Name = "Management"
(Get-ClusterNetwork –Name $clusternet.Name -Cluster $cluster).Name = "HeartBeat"
(Get-ClusterNetwork –Name $storagenet.Name -Cluster $cluster).Name = "Storage"
(Get-ClusterNetwork –Name $livemigrationnet.Name -Cluster $cluster).Name = "LiveMigration"

#isolate Traffics
(Get-ClusterNetwork -Name "Management" -Cluster $cluster).Role = 3
(Get-ClusterNetwork -Name "HeartBeat" -Cluster $cluster).Role = 1
(Get-ClusterNetwork -Name "Storage" -Cluster $cluster).Role = 0
(Get-ClusterNetwork -Name "LiveMigration" -Cluster $cluster).Role = 1

#Configure LiveMigration on a dedicated network
Get-ClusterResourceType -Name "Virtual Machine" -Cluster $cluster| Set-ClusterParameter -Name MigrationExcludeNetworks -Value ([String]::Join(";",(Get-ClusterNetwork -Cluster $cluster | Where-Object {$_.Name -ne "LiveMigration"}).ID))

In a future post I’ll show how to manage this cluster and its storage from Virtual Machine Manager 2016 Technical Preview 4.