I didn’t write a blog for a long time, but thought it’s time to start sharing some technical information around XenClient Enterprise. The ones who know me, are well aware of my excitement about XenClient (Express)however I’m even more excited about the enterprise product which is based on Citrix recent acquisition – Virtual Computer. Many people are approaching me and indicate, they would love to convert the exiting XenClient Express virtual machine images, so they don’t need to recreate their golden images.

To be be perfectly honest, I don’t really like that idea – Pretty much as I don’t like the idea to P2V (Convert physical to virtual) an existing installation of an operating system. The main reason for that is that you can run into issues which you definitely won’t when creating a fresh and clean OS install. P2V may suffer due to lots a unnecessary code in the image, such as hardware vendor tools and drivers, which are not required in a virtualized operating system image (Isn’t that one of the major benefits, that we do a strict separation of the hardware and the OS?). If you convert an existing VM, it can besides incompatible virtualization support drivers and tools also be that the platform treats the OS differently e.g. using para-virtualization versus pure HVM (hardware virtual machine).  Having said thatXenClient Express and XenClient Enterprise do share the same “engine” (even if Xen version is different – 3.4 vs. 4.0), so generally speaking there’s a chance to you in fact can re-use your existing VM’s.

Note: This is not a method Citrix is providing TechSupport for and if something goes wrong the standard procedure would be to create an image from scratch….  

The operation at a glance

In order to re-use existing VM’s from XenClient Express there are quite a few steps required.

This is the overview of what needs to be done:

  • Prepare the VHD on the XenClient Express
  • Remove the VHD junctions on Dynamic image mode
  • Coalesce the VHD chain
  • Export the VHD
  • Import the VHD on the Synchronizer for XenClient Enterprise
  • Uninstall PV tools and Services
  • Prepare and publish the VM
  • Assign the VM
  • Download and install the VM on the XenClient Enterprise
  • Note: Due to an error I observed last minute, I removed the script for the preparation of Dynamic Images – I’ll update this blog shortly (Consequently the instructions are for static image conversion)

    Prepare the Virtual Machine on the XenClient Express (2.x)

    The first thing you need to do is getting acces to a shell on your XenClient Express. On the Citrix Receiver for XenClient (GUI of the XenClient) press <CTRL+Shift+t> and provide the root password, you entered during the installation. If you want to these steps remotely, you may turn on the SSH daemon by entering the following command in the terminal:

    <span style="color: #008000">@xenclient-dom0:~# xec set enable-ssh true</span>

    After that you can access Dom0 remotely.

    Now let’s first see what VM’s we have on that XenClient:

    <span style="color: #008000">root@xenclient-dom0:~# xec-vm</span>

    <span style="color: #008000">ID| Name | UUID | State</span>
    <span style="color: #008000">--+-------------+-------------------------------------–+------</span>
    <span style="color: #008000">1 | uivm | 00000000-0000-0000-0000-000000000001 | running</span>
    <span style="color: #008000"> | Win7 Static | cf42005e-ce9a-4084-802a-11e8a6702114 | stopped</span>
    <span style="color: #008000">root@xenclient-dom0:~#</span>

    The UIVM is the Citrix Receiver GUI, a specialist service VM preinstalled on XenClient. The one we want to convert to XenClient Enterprise is the “Win7 Static”

    In the next step we’re going to understand which VHD file (or better chain) is being assigned to the ”Win7 Static”:

    <span style="color: #008000">root@xenclient-dom0:~# more /config/vms/cf42005e-ce9a-4084-802a-11e8a6702114.db | grep disks "path": "\/storage\/disks\/25217f3d-fc42-4e51-9050-81e49ac0f324.vhd",</span>

    So, we got the VHD file the VM runs from. Just to verify I’m going to read the VHD header of that file:
    <span style="color: #008000">root@xenclient-dom0:~# vhd-util read -p -n /storage/disks/25217f3d-fc42-4e51-9050-81e49ac0f324.vhd</span>

    <span style="color: #008000">VHD Footer Summary:</span>
    <span style="color: #008000">-------------------</span>
    <span style="color: #008000">Cookie : conectix</span>
    <span style="color: #008000">Features : (0x00000002) &lt;RESV&gt;</span>
    <span style="color: #008000">File format version : Major: 1, Minor: 0</span>
    <span style="color: #008000">Data offset : 512</span>
    <span style="color: #008000">Timestamp : Thu Aug 9 07:43:25 2012</span>
    <span style="color: #008000">Creator Application : 'tap'</span>
    <span style="color: #008000">Creator version : Major: 1, Minor: 3</span>
    <span style="color: #008000">Creator OS : Unknown!</span>
    <span style="color: #008000">Original disk size : 80000 MB (83886080000 Bytes)</span>
    <span style="color: #008000">Current disk size : 80000 MB (83886080000 Bytes)</span>
    <span style="color: #008000">Geometry : Cyl: 40156, Hds: 16, Sctrs: 255</span>
    <span style="color: #008000"> : = 79998 MB (83884277760 Bytes)</span>
    <span style="color: #008000">Disk type : Differencing hard disk</span>
    <span style="color: #008000">Checksum : 0xffffed3e|0xffffed3e (Good!)</span>
    <span style="color: #008000">UUID : 6db7e14d-b63f-4952-832a-fdc3b7a520fa</span>
    <span style="color: #008000">Saved state : No</span>
    <span style="color: #008000">Hidden : 0</span>
    <span style="color: #008000"> VHD Header Summary:</span>
    <span style="color: #008000">-------------------</span>
    <span style="color: #008000">Cookie : cxsparse</span>
    <span style="color: #008000">Data offset (unusd) : 18446744073709</span>
    <span style="color: #008000">Table offset : 1536</span>
    <span style="color: #008000">Header version : 0x00010000</span>
    <span style="color: #008000">Max BAT size : 153600</span>
    <span style="color: #008000">Block size : 2097152 (2 MB)</span>
    <span style="color: #008000">Parent name : a1ca53ef-f591-4be3-aad1-f874c4e33a8b.vhd</span>
    <span style="color: #008000">Parent UUID : da4c63c6-1579-494e-953b-905117b6db61</span>
    <span style="color: #008000">Parent timestamp : Thu Aug 9 07:43:24 2012</span>
    <span style="color: #008000">Checksum : 0xffffd865|0xffffd865 (Good!)</span>

    If you want to see the while chain you may use the following command:

    <span style="color: #008000">root@xenclient-dom0:~# vhd-util scan -f -m'/storage/disks/*.vhd' -p vhd=a1ca53ef-f591-4be3-aad1-f874c4e33a8b.vhd capacity=83886080000 size=9787777536 hidden=0 parent=none vhd=25217f3d-fc42-4e51-9050-81e49ac0f324.vhd capacity=83886080000 size=643617280 hidden=0 parent=a1ca53ef-f591-4be3-aad1-f874c4e33a8b.vhd</span>

    The “vhd-util” can merge the chain with the two vhd files into a single vhd file:

    root@xenclient-dom0:/storage/disks# vhd-util coalesce -n 25217f3d-fc42-4e51-9050-81e49ac0f324.vhd -o coalesce-out.vhd -p

    In this example the coalasce-out.vhd file is the one you’re going to export and contains all diskblocks form the “Win7 Static” virtual machine.

    There’s even an easier way to do. David from the Cambridge team made a script which is specifically helpful if your chain is rather long because it will repeat the coalesce job until no parent is indicated in the VHD header. Here’s the script:

    ________ snip _________

    if test $# -ne 1
     echo "usage: $0 &lt;vm-uuid&gt;" 1&gt;&amp;2
     exit 1
    diskp="$(xec-vm -u "$1" --disk 1 get phys-path)"
    if test $? -ne 0
     echo "failed to get vhd path" 1&gt;&amp;2
     exit 1
    while true
     parentp="$(vhd-util query -p -n "$diskp")"
     if echo "$parentp" | egrep -q ' has no parent$'
     echo "coalescing $diskp to $parentp"
     vhd-util coalesce -n "$diskp"
     mv "$parentp" "$diskp"

    ________ snip _________

    The usage of the script is simple: ./ <UUID_of_the_VM>

    You may have to chmod +x the script to make it executable on the Dom0.

    If you prefere you can get the script form here

    To bring the script ont the XenClient you just use SCP e.g. WinSCP.

    The cli on a Linux / OSX terminal would be:

    <span style="color: #008000">Walters-iMac:Desktop whofstet$ scp root@ root@'s password: 100% 437 0.4KB/s 00:00</span>

    Once it is downloaded you may set the executable flag as indicated.

    <span style="color: #008000">root@xenclient-dom0:~# chmod +x root@xenclient-dom0:~# ls -la -rwxr-xr-x. 1 root root 437 Aug 11 21:58</span>


    Prepare the Virtual Machine on the XenClient Express (2.x) – Dynamic Images

    In order to prepare VM’s which were authored as Dynamic Image, things a re bit more complicated.

    Dynamic Image mode, breaks the VHD in three parts, the “System Disk”, the “User Disk” und the “Application Disk”. This is different than with the XenClient Enterprise Shared Image mode – In XenClient we used “VHD Junctions” where in the XenClient Enterprise world we just folder redirection which on the operating system level. Consequently we need to combine the “linked” files which is basically requires to copy the content of the three VHD’s into a single one.

    Again, thanks to David we got a script (ruby) which helps on that.

    Here’s the script to sanitise the VHD junctions. 

    This script isn’t substituting the  script, we also will run the after the resque.rb has done it’s job.

    Here’s my output from the Dynamic Image:

    root@xenclient-dom0:~# xec-vm
    ID | Name | UUID | State
    1 | uivm | 00000000-0000-0000-0000-000000000001 | running
    | Windows 7 | 35b41d19-4e10-460d-aff0-5ed8a7b742f7 | stopped
    root@xenclient-dom0:~# more rescue.rb –vm-uuid 35b41d19-4e10-460d-aff0-5ed8a7b742f7

    So I executed:

    ./rescue.rb –vm-uuid 35b41d19-4e10-460d-aff0-5ed8a7b742f7

    The scripts is quite chatty and will end with message like this:

    [RESCUE][INFO][26407] 12-08-22 16:18:27 – Completed, VHD location:
    [RESCUE][INFO][26407] 12-08-22 16:18:27 – /storage/disks/772dadc7-857a-4e7c-a2ab-93cdc2b6ff32.vhd

    root@xenclient-dom0:~# ./ 3a3329fb-f7c7-490c-af84-6b11207269bd coalescing /storage/disks/8a7b5cf3-f626-4090-b7a8-ffcd2f2623d4.vhd

    to /storage/disks/cc6d22c7-b778-4bc2-b8de-3ee60af6a243.vhd
    coalescing /storage/disks/8a7b5cf3-f626-4090-b7a8-ffcd2f2623d4.vhd to /storage/disks/ecc8bb23-d499-4137-a402-45cfaa9cf5d9.vhd

    Now, we can coalesce the VHD chains by:

    root@xenclient-dom0:~# cd /storage/disks/
    root@xenclient-dom0:/storage/disks# /root/ 35b41d19-4e10-460d-aff0-5ed8a7b742f7
    coalescing /storage/disks/772dadc7-857a-4e7c-a2ab-93cdc2b6ff32.vhd to /storage/disks/cc6d22c7-b778-4bc2-b8de-3ee60af6a243.vhd
    coalescing /storage/disks/772dadc7-857a-4e7c-a2ab-93cdc2b6ff32.vhd to /storage/disks/ecc8bb23-d499-4137-a402-45cfaa9cf5d9.vhd

    Export the Virtual Machine

    Exporting the virtual machine, means copying the single VHD which resulted form the preparing process.

    You got two main ways how you can copy that file:

    • Using a USB storage device
    • Using the network (FTP or SCP)

    Using storage devices, is typically a bit more complicated. Mainly becasue the Dom0 system can NOT write to NTFS formatted storage.

    The steps would be:

    • Plug the drive into the USB port – No VM must run or focus has to be on the UIVM
    • run dmesg
    • indetify the device as it’s being recogonized e.g. sdb
    • create a Linux partition using fdisk
    • run mkfs.ext3 /dev/sdb1 (1st partition in device sdb – verify with the results from dmesg)
    • Mount the filesystem with mount /dev/sdb1 /mnt
    • Now you can copy the VHD e.g. cp /storage/disk/coalesce.out /mnt/exported.vhd
    Using network you got the option for scp and ftp

    Here’s an example of transferring the VHD file to a FTP server:

    <span style="color: #008000">ftpput -v -u user_name -p password ftp_server_ip exported.vhd /storage/disk/coalesce-out.vhd</span>

    Here’s an example of transferring the VHD file using SCP:
    <span style="color: #008000">root@xenclient-dom0:~# scp /storage/disks/coalesce-out.vhd whofstet@imac.xd.local:Desktop/exported.vhd</span>

    <span style="color: #008000">The authenticity of host 'imac.xd.local (' can't be established. RSA key fingerprint is a0:fd:16:e9:a4:ad:a8:d8:a4:9d:77:88:8a:dc:5d:22. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'imac.xd.local' (RSA) to the list of known hosts. Password: coalesce-out.vhd 100% 10 0.0KB/s 00:00</span>

    Import the Virtual Machine

    As you may have noticed, XenClient Enterprise doesn’t allow access in any form to the Dom0. So, they only practical way is to leverage the Synchronizer for XenClient Exterprise.

    If you don’t have a Snchronizer running yet – Go Get it, it’s worth I promis!

    These are the required steps:

    • Copy the exported VHD file to the

    Explorer view Import Directory

    • Import the VHD file from the console

    • Create a virtual machine from the VHD file

    Once the import is completed, the VM can be started (will automatically by default).

    Before we really can use the image we need to get rid off the installed XenClient drivers and services. This can sometimes be tricky, specially if you updated the XenClient tools a few times. Sometimes the uninstaller may fail to remove all components which may conflict with drivers being installed with the XenClient Enterprise support tools.

    This is not a big deal, if run into an issue just delete the VM an create a new from your exported VHD file 🙂

    Now your actually done with cleaning, you can proceed to the next step which is to:
    • Prepare / Deploy the image (During that phase the XenClient drivers and tools will be injected into the image)
    • Assign the new version to users/groups

    The assigned user(s) XenClient Enterprise engine is by default checking every 10 Minutes with the backend, for updates. The new image will be downloaded and installed.

    Note: There’s a engine side preparing taking place when new image / update to an existing image is being installed, the GUI of XenClient provides the information and a percentage competed.

    Again, as I mentioned previously, this is not supported and I prefere to start with a clean image.

    TIP: You always export an image and store it somewhere, if want to have an additional insurance.

    I hope you enjoy the new possibilities of the XenClient Enterprise – Have Fun!

    +++ There’s one more thing: Don’t forget to register for Synergy Barcelona – We got great sessions and a three hour hands on lab about XenClient Enterprise! +++

    Cheers, walter.hofstetter[at]