Deleted files are an interesting topic in isolation systems.  How do you represent a file as “gone”, when it is still present?  During profiling of an installation program, the isolation system has to isolate additions to the file system; this part is easy and pretty easy to understand using the “layers of glass” metaphor.  You look down from above and the highest version of the file is the version that you see.  What is more interesting to consider is that the isolation system also has to represent DELETED files; and deleted registry content.  These require placing a marker at higher levels that say to mask vision to the content that really does exist.  Deleting files is much more interesting than adding files!

Consider a view from above, looking down through the isolation system goggles:

I used the IIC version of this graphic because it was handy, but consider the above with only 2 layers as this is the environment during profiling.  The PHYSICAL or “REAL” disk/registry is off limits for writes.  When the installer during profiling, or when the program at runtime writes to the isolate space, the changes are reflected as writes to the higher layers of isolation (the first “writable” space, which is always on the top).  This works great for adding content because the presence of the content at the higher layer “masks” vision to the layers below.

For deleting things, this gets interesting.  Consider a program at installation which erases content from the physical machine.  Looking down from above, the installer needs to BELIEVE that the erase it commanded, really happened.  Here, the isolation system has to represent “deleted” files and registry content without really deleting the stuff from the true disk.  Failure to do this would require a VM snapshot thing and resetting of the environment before/after profiling and we didn’t want that.  Instead, you can profile over and over all day long without hurting the true physical image – at least for the spaces that are isolated, which is pretty much everything during profiling.

A deleted file is represented as a real file, at a higher layer of isolation.  The real file is changed to have 0 size, but still has the same filename as the lower layer file.  The filesize reduction is just to conserve disk space, but in concept the deleted file is the same file as the lower layer; promotion from physical to isolated occurs commonly for files “changed”.  To make the file “deleted”, the isolation system tags the file with a NTFS Alternate Data Stream, “CITRIX_DELETED_FILE_MARKER”.   If the file exists on the layer of glass, but has zero size and the required tag, then the file “disappears” from the perspective of isolated processes.  The file still exists and is even visible outside of isolation; though it still has no contents.

References:

Forked file systems (Mac)

Scary reference – ADS are evil IMO, They are not evil, they just aren’t widely understood.

Microsoft KB describing how to use ADS

Sysinternals tool for displaying Streams

The fact that ADS are rarely used by “normal” Windows applications makes them a PERFECT candidate for usage in the isolation system.  The isolation system can attach a stream to the file and feel pretty confident that this won’t hurt the applications ability to manipulate that file, yet the information associated with that file hangs out and even follows the file from place to place when it is copied.  Note: this isn’t exactly true – keep reading.

Profiling

During profiling, the isolation system sets the marker on files that the installation program erases.  In concept, this means that the marker is present on the execution machine at runtime because the captured layer of glass is transported to the execution machine.  In reality, this is true only in concept.  In reality, we use CAB files to hold the layer of glass and CAB files do not maintain streams.  Hum.  Problem.  What did we do?  Answer: When the application installer completes processing, the profiler goes through and “deletes the deleteds”; removing the files with zero size and the stream attached.  This means that the deleted file will not be present in the captured image, but it also means that the marker will not be present at runtime.  Problem?

First, why is this a risk?  If the file existed on the profiling machine (at true disk location), and if the installer erased it; then on the execution machine, if the file exists on the true disk, then at application runtime, the erased file will re-appear.  As the theory goes, the application at runtime will get unhappy.

Is that a problem?

At the time, I wore the FSFD dev hat and took note that darn near zero application installers actually erase things from isolated spaces during profiling.  Those that do, likely will also tollerate that the file comes back.  We also took note that even if the file did re-appear, we could always erase it again using a pre-launch script.  Bottom line: Application Streaming has shipped this way from the beginning and I have never seen a failure attributed to this behavior.

But wait, there are more problems

At runtime, the application may decide to erase things.  In general, the only spaces isolated are the \windows directory and below and the \program files spaces or more precisely, these are the locations where an ill-behaved program may want to erase things; things that could effect the execution of the program.

It could also be that the application erases some of its configuration data from the directory to which it was installed (a bad application). In either of these cases, a deleted file marker will be generated and will land in the per-user layer of isolation.  This will mask vision to the file on the true disk.  Since this is part of the per-user space, all should be happy because the per-user space will follow the user from machine to machine as part of roaming profiles or even better, Citrix User Profile Manager!  The Win32 CopyFile API DOES maintain Streams!

Problem: Roaming profiles do not maintain ADS Streams, so the deleted markers will disappear on movement from machine to machine.  4 years in the field, has this been a problem?  Nope.

I have asked the Citrix UPM team to maintain Streams on copy of the user profile content to/from central store which is a bit of a curiosity because it is different behavior than Roaming Profiles.   I’m not sure if it occurs yet and not even sure if they will actually implement the change.  Why?  Fundamentally, the ADS streams haven’t been missed yet, they will likely be missed less in the future; except for all them MAC users.  Stick with me.

What file systems support ADS Streams?

Excellent question!  The answer is “NTFS”.  The Mac file system REQUIRES streams, but we rarely get the mac file system as the foundation of a Windows machine.  FAT32 definitely doesn’t support streams and I’m not sure about CDFS.

Since deleted file markers are needed for isolation processing, the Application Streaming file system code restricts it’s filtering to only NTFS disk volumes.  In some ways, this is a plus.  You don’t want to isolate USB drives and these tend to be FAT32, so they don’t get isolated.  But, the FSFD knows the USB drive is “removable” and it assumes all removable media is for “data”, so it doesn’t mess with it.  In this way, the NTFS restriction actually gets in the way a bit.  Note: you can convince the FSFD to look at the USB drive as I outlined here.

What about more elaborate file systems?

Some customers have enterprise storage which is network based, but appears to the execution machine as local.  These disk volumes COULD be NTFS, but often they have a custom file system.  Do these support NTFS style ADS streams?  Answer: It depends.  Let’s say they don’t.  How to solve…  Answer: Convert the representation of “deleted files” from ADS Stream to something a bit more common.  Has this happened? Nope.  Is it really necessary?  I’m not sure.  Experience of the last few years says it is rare enough that the failure case probably won’t come up.  With this wide post though, something tells me someone is going to say: “I’ve seen the failure case”.

In a follow up post, I’ll describe how the registry “deleted” markers are implemented.

Enjoy.

Joe Nord

Product Architect – Application Streaming

Citrix Systems