In an earlier article, I described the new capabilities in App Streaming 6.5 to use mounted VHDs to implement layer of cake on XenDesktop.  This is great stuff for separating operating system image from application image and greatly assists administrator management of computer systems in a virtualized + pooled XenDesktop world.   Some comments in the prior article and similar queries from sales engineer and customer channels ask if the same “mounting” of execution content is possible in a XenApp world.  This is such an interesting question that it deserves it’s own post.

First let’s back up and talk about why mounting application images is needed and specifically why MOUNTING is more efficient than copying data into the execution machine.  Given the XenDesktop execution is based on a virtual machine and given that the virtual machine “resets” to initial state on each logon, copying RadeCache data into the execution machine at runtime is an expensive operation that occurs at each logon.  It would populate the space, but the copies would have to repeated each time the user logs on to the hosted desktop.

A more efficient method declares disk writes “evil” and uses VHD Mounting to place the streaming execution content into the execution machine as redirected READS.   The disk accesses into the RadeCache get reflected to the mounted VHD, effectively moving disk IRPS from C: to the network stack.  It also greatly reduces the number of disk writes as all of the cache operations for the streamed content will already be “present”, saving the read from App Hub and Write to local disk C:.  There is a cost that network read operations are likely slower than disk SAN reads, but the bet is that eliminating the writes is well worth the costs of moving to a slower network based disk read infrastructure.

This technology exists now and ships with the 6.5 streaming client to enable this mounting of execution content for XenDesktop execution.  Here’s a graphic that shows this visually.

Mounting App Streaming content into the RadeCache

In the comments of the prior post, Serge de Klerk asks

Does anyone know if the mounting of .vhd’s would work on Win2003 XenApp servers?

It is a very valid question and is one that I have received from a few sources.   The quick and official answer is “no, this feature is intended only for XenDesktop virtualized execution”.   The long answer is “yes, but you’ll have to do some manual steps”.  The longer answer is that it’s possible this is working now even though we didn’t expect it.

As a quick intro, why use mounted execution for XenApp servers.  The answer is that many XenApp admins are now applying virtualized machine execution and ” layer of cake” to their construction of XenApp servers in the same manner that this is being used for XenDesktop.  The same NEED exists in both places.  In version 6.5, we set out to solve it for XenDestkop, but XenApp admins are coming back and saying “HEY!  We need that too”.  In XenApp though, there’s an extra thing to worry about – there are multiple users per machine.

VHD Mounts dissected

A VHD file is mounted into a sub-directory of a Windows based computer.  Here, it looks like a directory, but from the view of the Windows machine, it is a mount point looking at a LOCAL and PHYSICAL disk.  This disk is “valid” for backing store for .EXE load and execution.

From the view of the network server the VHD file is just a file.  A file like any other file and a file that is large and perhaps has a lot of reading occurring and precisely zero writing, but ultimately, it’s just a file.  A user opens the file and then the file is accessed.

The streaming system tells the VHD Mounting device driver to connect a disk volume at a mount point with a given VHD file located at some UNC accessible network location.  In the case of Windows 7, the VHD driver is the Microsoft driver included with Windows.  In the case of Windows XP, the streaming client uses the Citrix one that does the same thing.  A side note is that the Citrix streaming group “liberated” this driver from Citrix Provisioning Services so we were able to avoid writing one from scratch.  When can we stop supporting Windows XP?

Here’s what it looks like in the RadeCache space with the output of a DIR command.

The question is, will this work with XenApp?  Or is it purely a XenDesktop feature?

Okay – On to the point of this post

Consider for a moment that when the App Streaming system mounted the VHD, the local machine gets a junction point, which is effectively the same thing as a drive letter.  The remote machine (the network server) sees file access from the machine and more specifically, the USER.

The streaming system in mounting the VHD “borrows” the user’s ability to get to the network server as a given in accessing the VHD.

SO, will this work on XenApp?  The answer is generally “no”, but here it gets pretty interesting.

Consider a XenApp server with 100 users…

  1. User A logs onto the XenApp Server and runs the streamed application.
  2. User B logs onto the XenApp Server and runs the streamed application
  3. Lather rinse and repeat until you get all 100 users onto the machine.

User A logs off.  This is the user whose access rights were borrowed so the streaming system could mount the streaming execution content from the App Hub.  What happens to users B and up?

Basic OS theory says that when user A logs off, the  privilege to access the network server vaporizes and the VHD mount point becomes invalid.

I have been telling people “this won’t work” for months now and I’m even pondering potential solutions to this problem.

Where things get unexpected

In writing this post, I’ve been doing some experiments.  My experiments say that it “works”, while my arguably solid knowledge of Windows internals tells me that it shouldn’t.

Using my Windows 7 64-bit notebook as test platform, I conducted the “1, 2, 3” experiment above.  As expected, when user A runs a streamed app, the streaming execution cache indeed is present for all other users on the machine.  Where I have become surprised is that when user A logs off, the VHD mount points are STILL THERE!  This is something that I’ve so far “known” wouldn’t work.  So confident that it won’t work that it was never worth testing, but sit down and test it and … the execution content is still present.  Hum.

Update (31-Oct-11): The answer appears to be that the kernel mode driver duplicates the user mode handle by creating a kernel object that points to the same file.   When the user logs off, the user mode reference to the file object decrements, but the kernel reference stays in place, which keeps the file open from the perspective of the execution machine.  What is interesting is that I expected the network server client to force close all user files when the user logs off.  This appears to not occur and that allows the kernel driver to continue with successful file access even when the user has left.

Getting back to Serge’s question

> Does anyone know if the mounting of .vhd’s would work on Win2003 XenApp servers?

Never assume

My experiment above was on Windows 7 64-bit.  This implies that all of my prior statements that this won’t work are not true.  But, that is incomplete.

I’ve been telling people that it won’t work because of assumptions I have about how VHD files are mounted in Windows.  The reality is that it doesn’t work the way I have figured.  It APPEARS that the Windows VHD device driver “holds on” to the open file handle for the network server, and holds on to this handle EVEN AFTER the user logs off.  That is, the user that first triggered the VHD to be mounted does not need to still be logged in in order for the VHD driver to continue accessing stuff that the first user assisted the driver in accessing.  Interesting!

Multiple endings

Like a good novel, this blog post appears to have multiple potential endings.  I had planned to end this post with guidance to the XenApp admin that while the streaming system can’t keep the VHD mount alive, that you the admin can.  Create the mount points outside the scope of App Streaming and the system will have these “directories” already present at runtime.  The streaming system accepts “magic” for how the mount points got created and if they exist, it will use them to run streamed content.  This is by design and planned.  This means that if the admin takes action to create a mount point, then that mount point will be used.    This will make VHD mounted streaming content work on XenApp even though when we set out to build version 6.5, we really were only targeting XenDesktop virtualized execution.

The reality of the ending is that it is POSSIBLE that this is working today, despite never really being intended to be so.  It will take a round of testing on additional platforms to prove it out.  In the question referenced from the earlier blog, Serge asks about XenApp on Server 2003.  Notice there are lot of things that are different here than in the 1 hour testing I have done for this blog post.  Server 2003 will use the Citrix VHD driver, not the Microsoft one.  I’m going to assume for a moment that this VHD driver is installed when the 6.5 streaming client is installed on top of Server 2003, but even that isn’t a valid assumption.  Second assumption is that server version of OS behaves the same as client version.  That is, on Server 2008 (32, 64), Server 2008 R2 and other flavors, do the mount points persist as they are showing themselves to do so on my Windows 7 64-bit machine?  I don’t know the answer.  Given already surprising results here, making assumptions is no longer a good idea.

Update (31-Oct-11): Variety of people have reported to me offline that the Server 2003 case is also successful, as well as Server 2008 R0.

Wrapping it up

The official answer is that Citrix never intended for the VHD mounting feature in the 6.5 streaming client to be used on XenApp servers.   We officially don’t expect this to work.

The less official answer is that if the VHD mounts persist, then it should WORK.  And I’ll do my best to help you be successful…

Anyone want to volunteer for some testing and report back results?

Joe Nord

Product Architect Application Streaming and Profile Manager

Citrix Systems