Background

At some recent customer engagement I was facing the problem of creating a large number (100+) of Provisioning Services 6.1 based XenApp 6.5 target devices with their Write Cache placed on local SSD storage of a VMware vSphere 5.1 host. One might think this would not be a complicated task, so just make a copy of your master target device machine with a second disk placed on the SSD storage, convert it to a template and use this template with the Provisioning Services Streamd VM setup wizard to create the machines on vSphere. Easy hmm ? Not so, since I learned that the PVS wizard will not allow me to use templates that include disks hosted on local vSphere hypervisor storage. This behavior cannot be changed the easy way.

Creating the machines by hand via VMware vCenter is not feasible to their high number. Moreover they have to be equally distributed over all respective hosts since Dynamic Resource Scheduling (as it is called by VMware) will not work because of the local storage. Second step would be creating corresponding target device entries within the PVS device collections which would also be time consuming for such a high number of machines.

So that’s the time when it comes to scripting a solution. Using three different SDKs (VMware Power CLI, Provisioning Services MCLI and a PowerShell module for getting access to Active Directory management) the needed functionality can be achieved. Scripting the whole process also gave me the possibility to include one step which would have to be done manually anyway: moving the XenApp worker computer objects to two Active Directory groups for controlling the reboot cycle of the XenApp machines. Due to availability reasons it was planned that always 50% of the servers of one PVS site (with two sites created for two datacenters) will reboot every 2-days. So dividing the XenApp machines for this type of reboot behavior could also be easily achieved by some PowerShell scripting.

The following process had to be implemented:

1. Query the XenApp vSphere cluster for the host with the fewest number of VMs
2. Determine the next free name for the new virtual machine based on a defined naming convention
3. Create the virtual machine with disk on local SSD storage
4. Create the target device entry within PVS via MCLI
5. Create the computer object within the correct Active Directory OU
6. Add the computer object to the correct reboot group

Sounds not to complicated, however there were some things to consider which could easily be found in any other customer deployment. The vSphere cluster used was a stretched cluster including hosts from two datacenters in an active/active setup. In order to equally distribute the target devices over the two datacenter the location of the queried hosts had to be determined. However this could not be easily done by some naming convention of the hosts but rather by a custom vSphere attribute the customer has set for the hosts to display the location. So querying this attribute was the only possibility to get the correct location for the vSphere hosts. Another challenge was to cope with different Active Directory response times for creating the computer objects via MCLI. The subsequent commands for adding the computer object to the reboot groups often failed, so I had to implement some short cmdlet that checks for the existence of the newly created object and pause the script as long as the object is not yet responded to exist.

Scripting the whole process

Prerequisites that have to be installed on the machine running the scripts are:

  • VMware PowerCLI
  • Provisioning Services Console (for having access to MCLI commands)
  • Windows Server 2008R2 Feature – Active Directory Module for Windows PowerShell

The script is quite long so I will only cover some important snippets here, feel free to try and adapt the full script attached below.

Query the XenApp vSphere cluster for the host with the fewest number of VMs

First we get all hosts from the cluster, sort them according to their location (DC1 or DC2) and calculate the number of machines currently deployed on each host:

$hosts = $cluster | <span class="code-keyword">Get-VMHost</span> -State Connected | Select Name,@{N="NumVM";E={@(($_ | <span class="code-keyword">Get-Vm</span> )).Count}},@{l="Location";e={$_.customfields | ?{$_.key -eq 'Location'} | select -ExpandProperty value}} | Sort Location</em></code>

$HostVMsDC1 = $hosts | where { $_.Location -eq "DC1"}
$HostVMsDC2 = $hosts | where { $_.Location -eq "DC2"}

Here is an example for getting the host in DC1 with the least amount of machines on it. Based on that we need to get the name of the local datastore for virtual disk creation. To only query for local datastores the attribute MultipleHostAccess is used which is always set to false for local datastores. $UsedHost finally denotes the host we are using for VM creation in DC1:

$LeastHost = $HostVMsDC1 | Sort NumVM | Select -First 1
$Datastore = <span class="code-keyword">Get-VMHost</span> -Name $LeastHost.Name | <span class="code-keyword">Get-Datastore</span> | ? {($_.Type -eq "VMFS") -and ($_.ExtensionData.Summary.MultipleHostAccess -eq $False)} | Sort Name | Select -First 1
$UsedHost = <span class="code-keyword">Get-VMHost</span> -Name $LeastHost.Name

Create the virtual machine

$taskTab[(<span class="code-keyword">New-VM</span> -Name $VM_Name -VMHost $UsedHost -Template $template -Datastore $Datastore -Location $folder).ID]=$VM_Name

After creation we increase the counter for number of virtual machines on this host by one (we do not query the number of machines for all hosts each time due to performance reasons):

$HostVMsDC1 | Where { $_.Name -eq $LeastHost.Name } | Foreach { $_.NumVM++ }

We will also need the MAC address for target device creation within PVS:

$MAC = <span class="code-keyword">Get-NetworkAdapter</span> -VM $VM_Name | ForEach-Object {$_.MacAddress} | % {$_ -replace ':',"-"}

Add the target device to its device collection and create a computer object

<span class="code-keyword">Mcli-Add</span> Device -r deviceName=$VM_Name, collectionName=$collection, siteName=$site_1, deviceMac=$MAC, description=$description, copyTemplate=1
<span class="code-keyword">Mcli-Run</span> AddDeviceToDomain -p deviceName=$VM_Name, organizationUnit=$OU

Adding 50% of each machines from one datacenter to different reboot groups

The custom made cmdlet Test-XADComputer (see attached script) tests for existence of an Active Directory computer object. In case the object does not yet exists the whole script sleeps for an amount of time and continues when the object is finally created and reported to exist:

do {
         sleep 5
         <span class="code-keyword">Write-Host</span> "." 
    } until (<span class="code-keyword">Test-XADComputer</span> $VM_Name)

The newly created machines for one datacenter are equally distributed to the reboot groups so that after configuring Citrix reboot policies filtered for these two groups only 50% of the servers will reboot each day. The Get-ADComputer and Add-PrincipalGroupMembership cmdlets are part of the Active Directory PowerShell module:

if($vm_count % 2) 
    {
    <span class="code-keyword">Get-ADComputer</span> -identity $VM_Name | <span class="code-keyword">Add-ADPrincipalGroupMembership</span> -MemberOf $Reboot2
    <span class="code-keyword">Write-Host</span> "Computer object added to Reboot group $Reboot2"
    }
    else 
    {
    <span class="code-keyword">Get-ADComputer</span> -identity $VM_Name | <span class="code-keyword">Add-ADPrincipalGroupMembership</span> -MemberOf $Reboot1
    <span class="code-keyword">Write-Host</span> "Computer object added to Reboot group $Reboot1"
    }

I hope this script can be useful if you encounter similar difficulties during implementations and show what can be achieved by leveraging the truly powerful SDKs of our (and also other) products 😉

Here you can view the whole script, parameters in bold need to be adapted for your environment:

@" 
========================================================================================
Title:         CreateTargetDevices with disk on local SSD storage and imports the devices
               into Citrix PVS
Author:        Thomas Fuhrmann
Version:       1.6
Usage:         .\CreateTargetDevices.ps1 
Date:          04/06/2013 
========================================================================================
"@ 

############
# Feature "Active Directory Module for Windows Powershell" has to be installed on the machine where the script is executed
############

Import-Module ActiveDirectory

Add-PSSnapin -Name "VMware.VimAutomation.Core"  
Add-PSSnapin -Name McliPSSnapIn

$ErrorActionPreference = 'Stop'

############
# Function for testing if AD Computer Object exists
############

function Test-XADComputer() {

[CmdletBinding(ConfirmImpact="Low")]

Param (

    [Parameter(Mandatory=$true,
                Position=0,
                ValueFromPipeline=$true,
                HelpMessage="Check if AD Computer object exists")]
                
    [Object] $Identity

)

trap [Exception] {

   return $false
   
   }

   $auxObject = Get-ADComputer -Identity $Identity
   
   return $true

}

##############################################################
# PARAMETERS
##############################################################

#
# vCenter Server hostname and hostname of one PVS Server
#

$vchost = <strong>"vCenterServerHostname"</strong>
$pvshost =<strong>"PVSServerHostname"</strong>

#
# Parameters which have to be edited according to environment - check for correct silo name
#

$template = Get-Template -Name <strong>"Name of Vmware Target Device Template"</strong>
$folder=Get-Folder -Location <strong>"Name of VMware Folder"</strong>

$description= <strong>"Description of the vDisk"</strong>

$OU=<strong>"OU for creation of Comnputer object"</strong>

$Reboot1=<strong>"AD Reboot Group 1 for XenApp Servers"</strong>
$Reboot2=<strong>"AD Reboot Group 2 for XenApp Servers"</strong>

$clustername=<strong>"vSphere Clustername"</strong>
$vcenteruser=<strong>"vCenter Username"</strong>

# Prefixes ($vm_prefix)
#
# $start_vm denotes first possible target device
# example: first VM would be named VM00020

$vm_prefix = <strong>"VM00"</strong>
$start_vm = <strong>20</strong>

#
# Parameters which have to be edited according to PVS Device Collection
#

$collection= <strong>"PVS Device Collection Name"</strong>
$site_1= <strong>"PVS Site 1"</strong>
$site_2= <strong>"PVS Site 1"</strong>

################################################################

###########
# Establish connection to vCenter and PVS
###########

Connect-VIserver $vchost -user $vcenteruser
Mcli-Run SetupConnection -p server=$pvshost, port=54321

###########
# Set cluster where to deploy the target devices
###########

$cluster=Get-Cluster -Name $clustername

###########
# Input for number of target devices to be created (only even numbers supportet)
###########

$num_vms_total = 1

while ($num_vms_total % 2)
{
$num_vms_total = Read-Host "Number of Target Devices to create ? (Even number needed)"
if ($num_vms_total -le 0)
{
Write-Host "Please enter number greater than 0"
$num_vms_total = 1
}
}

#############
# Getting list of available hosts
#############
   
Write-Host "`nCollecting Host list"

#############
# Getting list of connected hosts including location and number of VMs
#############

# $hosts contains the following columns:
#
# | Name | Number of VMs on Host | Location
#

$hosts = $cluster | Get-VMHost  -State Connected | Select Name,@{N="NumVM";E={@(($_ | Get-Vm )).Count}},@{l="Location";e={$_.customfields | ?{$_.key -eq 'Location'} | select -ExpandProperty value}} | Sort Location

#############
# Getting hosts per datacenter location
#############

$HostVMsDC1 = $hosts | where { $_.Location -eq "DC1"}
$HostVMsDC2 = $hosts | where { $_.Location -eq "DC2"} 

$taskTab = @{} 

$vm_count = 0 
$counter = 0

##########
# First target device hostname to try
##########

$VM_Name = $vm_prefix + 0 + $start_vm

#########
# DC1
#########
  
While ($vm_count -lt ($num_vms_total/2)){ 
    Write-Host "Finding host in Datacenter DC1 with least amount of VMs on it"
    $LeastHost = $HostVMsDC1 | Sort NumVM | Select -First 1 
    
    ##########
    # Get local datastore of host with least amount of VMs on it 
    ##########
    
    Write-Host "Finding corresponding SSD datastore`n"
    $Datastore = Get-VMHost -Name $LeastHost.Name | Get-Datastore  | ? {($_.Type -eq "VMFS") -and ($_.ExtensionData.Summary.MultipleHostAccess -eq $False)} | Sort Name | Select -First 1
    $UsedHost = Get-VMHost -Name $LeastHost.Name
    
    
    ###########
    # Determine first free hostname for target device creation
    ###########
    
    
    while ((Get-VM -Name $VM_Name -ErrorAction SilentlyContinue) -ne $null) 
    {
    $counter++
        if (($counter + $start_vm) -lt 100)
        {
        $VM_Name = $vm_prefix + 0 + ($counter + $start_vm)
        }
        else
        {
        $VM_Name = $vm_prefix + ($counter + $start_vm)
        }
    }
    
    ###########
    # Creation of the target devices for site DC1
    ###########
    
    
    Write "Create virtual machine $VM_Name on $($LeastHost.Name) using template $template"
   
    $taskTab[(New-VM -Name $VM_Name -VMHost $UsedHost -Template $template -Datastore $Datastore -Location $folder).ID]=$VM_Name 
    $HostVMsDC1 | Where { $_.Name -eq $LeastHost.Name } | Foreach { $_.NumVM++ } 
    $MAC = Get-NetworkAdapter -VM $VM_Name  | ForEach-Object {$_.MacAddress} | % {$_ -replace ':',"-"}

    ###########
    # Add target device to device collection $collection and add to AD domain in Organizational Unit $OU
    ###########

    Mcli-Add Device -r deviceName=$VM_Name, collectionName=$collection, siteName=$site_1, deviceMac=$MAC, description=$description, copyTemplate=1
    Write-Host "Added device to collection $collection on site $site_1`n"
    Mcli-Run AddDeviceToDomain -p deviceName=$VM_Name, organizationUnit=$OU
    
    
    ##########
    # Add half of target devices per site to reboot group in AD
    ##########
    
    
    
    do {
         sleep 5
         Write-Host "." 
    } until (Test-XADComputer $VM_Name)
    
    if($vm_count % 2) 
    {
    Get-ADComputer -identity $VM_Name | Add-ADPrincipalGroupMembership -MemberOf $Reboot2
    Write-Host "Computer object added to Reboot group $Reboot2"
    }
    else 
    {
    Get-ADComputer -identity $VM_Name | Add-ADPrincipalGroupMembership -MemberOf $Reboot1
    Write-Host "Computer object added to Reboot group $Reboot1"
    }
    
    $vm_count ++ 
 }
  
$taskTab = @{} 
 
#########
# DC2
#########
  
While ($vm_count -lt $num_vms_total){ 
    Write-Host "Finding host in Datacenter DC2 with least amount of VMs on it"
    $LeastHost = $HostVMsDC2 | Sort NumVM | Select -First 1 
    
    ##########
    # Get local datastore of host with least amount of VMs on it 
    ##########
    
    Write-Host "Finding corresponding SSD datastore`n"
    $Datastore = Get-VMHost -Name $LeastHost.Name | Get-Datastore  | ? {($_.Type -eq "VMFS") -and ($_.ExtensionData.Summary.MultipleHostAccess -eq $False)} | Sort Name | Select -First 1
    $UsedHost = Get-VMHost -Name $LeastHost.Name
    
    
    ###########
    # Determine first free hostname for target device creation
    ###########
    
    
    while ((Get-VM -Name $VM_Name -ErrorAction SilentlyContinue) -ne $null) 
    {
    $counter++
        if (($counter + $start_vm) -lt 100)
        {
        $VM_Name = $vm_prefix + 0 + ($counter + $start_vm)
        }
        else
        {
        $VM_Name = $vm_prefix + ($counter + $start_vm)
        }
    }
    
    ###########
    # Creation of the target devices for site DC2
    ###########
    
    
    Write "Create virtual machine $VM_Name on $($LeastHost.Name) using template $template"
   
    $taskTab[(New-VM -Name $VM_Name -VMHost $UsedHost -Template $template -Datastore $Datastore -Location $folder).ID]=$VM_Name 
    $HostVMsDC2 | Where { $_.Name -eq $LeastHost.Name } | Foreach { $_.NumVM++ } 
    $MAC = Get-NetworkAdapter -VM $VM_Name  | ForEach-Object {$_.MacAddress} | % {$_ -replace ':',"-"}

    ###########
    # Add target device to device collection $collection and add to AD domain in Organizational Unit $OU
    ###########

    Mcli-Add Device -r deviceName=$VM_Name, collectionName=$collection, siteName=$site_2, deviceMac=$MAC, description=$description, copyTemplate=1
    Write-Host "Added device to collection $collection on site $site_2`n"
    Mcli-Run AddDeviceToDomain -p deviceName=$VM_Name, organizationUnit=$OU
    sleep 5
    
    ##########
    # Add half of target devices per site to reboot group in AD
    ##########
    
     do {
         sleep 5
         Write-Host "." 
    } until (Test-XADComputer $VM_Name)
    
    if($vm_count % 2) 
    {
    Get-ADComputer -identity $VM_Name | Add-ADPrincipalGroupMembership -MemberOf $Reboot2
    Write-Host "Computer object added to Reboot group $Reboot2"
    }
    else 
    {
    Get-ADComputer -identity $VM_Name | Add-ADPrincipalGroupMembership -MemberOf $Reboot1
    Write-Host "Computer object added to Reboot group $Reboot1"
    }
    
    $vm_count ++ 
 }
  
 
  Disconnect-VIServer -Confirm:$false

Thomas Fuhrmann
Principal Consultant
Citrix Consulting Central Europe

And to satisfy the legal guys:

Disclaimer Notice

This software / sample code is provided to you “AS IS” with no representations, warranties or conditions of any kind. You may use, modify and distribute it at your own risk. CITRIX DISCLAIMS ALL WARRANTIES WHATSOEVER, EXPRESS, IMPLIED, WRITTEN, ORAL OR STATUTORY, INCLUDING WITHOUT LIMITATION WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NONINFRINGEMENT. Without limiting the generality of the foregoing, you acknowledge and agree that (a) the software / sample code may exhibit errors, design flaws or other problems, possibly resulting in loss of data or damage to property; (b) it may not be possible to make the software / sample code fully functional; and (c) Citrix may, without notice or liability to you, cease to make available the current version and/or any future versions of the software / sample code. In no event should the software / code be used to support of ultra-hazardous activities, including but not limited to life support or blasting activities. NEITHER CITRIX NOR ITS AFFILIATES OR AGENTS WILL BE LIABLE, UNDER BREACH OF CONTRACT OR ANY OTHER THEORY OF LIABILITY, FOR ANY DAMAGES WHATSOEVER ARISING FROM USE OF THE software / SAMPLE CODE, INCLUDING WITHOUT LIMITATION DIRECT, SPECIAL, INCIDENTAL, PUNITIVE, CONSEQUENTIAL OR OTHER DAMAGES, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. Although the copyright in the software / code belongs to Citrix, any distribution of the code should include only your own standard copyright attribution, and not that of Citrix. You agree to indemnify and defend Citrix against any and all claims arising from your use, modification or distribution of the code.