Every now and then I’m asked where the Provisioning Services Write Cache should be located and how big it might become. So I thought it might be worth to share some thoughts.
Provisioning Services supports several options for storing the PVS Write Cache, which contains all disk writes of a target device when using a write-protected vDisk (Standard Image). Options for the write cache location are:
- Cache on Provisioning Server
- Cache on Target Device RAM
- Cache on Target Device Hard Drive
Storing the write cache on the target side is beneficial as it keeps the write “close” to the target and minimizes the load on the Provisioning Servers, but it requires more resources on the target side as when storing the cache on the PVS side. A big benefit of writing the write cache to a target side hard disk is that this disk can also be used for data, which needs to be persistent (we will talk about this later).
For virtual desktops I’ve seen all three options within customer environments, while I’d say the majority of customers tend to use a cache drive directly attached to the virtual desktop. In large enterprises I’ve seen quite some customers who looked into the target side RAM option, as server memory was far cheaper than shared storage and local storage was not an option.
For virtual XenApp servers the best practice is to use a target side hard drive for storing the write cache and this is also what the vast majority of customers does.
Personally I prefer using a target side hard disk for storing the write cache for virtual desktops and XenApps.
As I’d love to hear your opinion about this topic, I’ve setup a short survey which should take you no more than 1 minute to complete: http://www.surveymonkey.com/s/3YWKNJQ
The local disk option
As stated earlier many customers choose to store the write cache on a hard disk attached to the PVS target device. When talking about virtual desktops or XenApps we have two options to provide these write cache drives.
Option 1 – write cache disk on shared storage
- High level of performance (typically)
- Easy to scale
- Central monitoring and management
- Virtual targets can be moved between hypervisor hosts for load balancing and/ or management reasons
- Available disk space on hypervisor hosts is wasted
Option 2 – write cache disk on local disk
- Cheap (compared to shared storage)
- Low complexity
- Can be a performance bottleneck / hard to scale
- Virtual targets are “tied” to a hypervisor host
- Hypervisor hosts must have local drives
So I think there is not the silver bullet and based on my experience customers haven’t found one either, as I see both options used at an almost equal share. Personally I prefer to use local disks for provisioned XenApp servers whenever possible from an performance and management point of view. For virtual desktops I prefer to use shared storage, as many customers have standardized on blade servers which have just two local disks in most cases. These disk subsystems are typically not able to cope with the amount of IOPS caused by he 50 or 100 desktops on a particular server. But this really depends the customer use case and the available hardware.
Please tell me what your opinion is by completing the survey: http://www.surveymonkey.com/s/3YWKNJQ
Estimating the size of the PVS write cache is almost impossible, as it highly depends on the user and application pattern. From a technical point of view the write cache can not grow bigger than the vDisk it is based on. So if you have a 20GB vDisk the maximum size of the write cache is 20GB per target. In 99.9% of the cases you will not see such big write cache files, but to be able to estimate the size we have to understand what makes the write cache grow.
Basically the write cache will store all writes which would have gone to the hard disk. So if a users tends to copy large files locally to his/her desktop the write cache will grow at the same pace as the files are transferred. If there’s any application which caches files or portions of a central DB locally for better performance, then the write cache will grow again.
But there are some items which will always hit the write cache and these are split into two areas again. On one hand there is the user space, which contains items such as the user profile or internet/application related temp files. The user related write cache disk space needs to be multiplied by the amount of users logged on to a particular system.
On the other hand we have the system space, which contains items such as logs or system temp / cache files, but we will also find files which are modified by the OS or any service for whatever reason. The system related write cache disk space is typically lager for server operating systems than for workstations.
Now there are options to keep the write cache size to a minimum. The best option is to redirect. In case you opted for writing the write cache into the target RAM or back to the Provisioning Servers, you don’t have too much options for redirection. All you can do is:
- Keep the user profile small by redirecting profile folders such as Desktop, My Documents, Application Data and so on. This helps for user driven file copies (to have it local) and for many application temp file scenarios.
- Using App-V Shared Cache or use the latest code for Citrix Application Streaming, which has the VHD Mount feature (similar to App-V Shared Cache). No streaming cache is build locally.
In case you opted for “Cache on Target Device Hard Drive” you can redirect some additional items. Here it is important to understand that items located on the local drive outside of the write cache file are persistent and are not wiped upon reboot of the target.
- Windows Pagefile. In fact the PVS Target Device driver detects if a local drive is available and redirects the pagefile automatically.
- Windows Event Log. While the eventlog is typically quite small (maybe 100MB or so) many customers redirect it for supportability and traceability reasons.
- Citrix related logs. Same as Windows Event Log.
- Anti Virus pattern. In case the virus scanner allows redirecting the pattern file, doing so saves some write cache space but it also saves some network traffic as it is not required to load the pattern from scratch after every reboot.
- App-V / Application Streaming Cache in case a shared cache concept cannot be used.
- EdgeSight DB. If fact you have to redirect the EdgeSight DB to a persistent drive or use a DB Proxy Agent, as otherwise ES data gets lost upon reboot of the target.
The local disk
When redirecting the write cache and other items on a persistent local disk we have to consider two things:
- Persistence. We have to keep in mind that by writing files to a persistent drive we contradict the central management approach of PVS. So these files and folders have to be maintained at a target device level rather than a vDisk level. So before something is redirected we should spend a minute and think about a strategy how we can maintain or possible delete these items.
- Size of the drive. The local drive needs to be big enough to hold all the redirected items and to allow some growth. The PVS Target Device driver will check if the local drive has more than 500MB of available disk space, before it uses it. If this check is unsuccessful the write cache will be written back to the Provisioning Server. But even if the check is successful 500MB is not too much. So in case your write cache grows bigger than that, but no more disk space is available Windows will throw a “No disk space on drive C available” error message and more or less stop responding.
So lets do a sample calculation for a write cache drive to illustrate the decision process. I assume we’re using a XenApp server with 50 concurrent users. Every user has a 20MB user profile and created a 50MB workspace (i.e. temp files):
1.0GB user profiles (50 x 20MB)
2.5GB user workspace (50 x 50MB)
1.0GB system workspace (assumed, different for every environment)
= 4.5GB write cache file
4.5GB write cache file
0.1GB Windows Event Logs
0.1GB Citrix Logs
4.0GB Dedicated Dump File (see UPDATE section below)
2.0GB buffer (some room to grow)
= 16 GB local disk size required per XenApp target
While 16GB per may sound a lot, keep in mind that without PVS you would have to host a 50GB disk (or whatever the size of you vDisk is) per XenApp and you probably would not have such a great central image management.
[UPDATE] My colleague Ronald Grass just made me aware of a change we introduced in PVS 5.6 SP1. Since then a dedicated dump file is created automatically within the write cache partition. The dedicateddumpfile.sys is essentially a mirror of pagefile.sys that is required to capture a dump on a drive other than the system partition. Check out CTX129292 for more information and how to disable it.
You have reached the end of the blog. In case you haven’t completed the survey yet, now is a good time to do so: http://www.surveymonkey.com/s/3YWKNJQ
I’ll share the results of the survey within my next blog.