Working as an architect often resembles a Choose Your Own Adventure book. You are talking with your customer, trying to understand their needs and offer them several options to choose from. Sometimes you might have a very strong preference for one of these options and sometimes you leave it up to customer preference, just providing them with a list of pros and cons. Your only goal, after all, is to make sure that the story ends with a happy ending and not in a trap door with deadly performance spikes.

Citrix Provisioning Services (PVS) would be one of the bolder “Choose Your Own Design” books. There are many different design decisions that you need to make – and surprisingly a large number of these decisions are not black or white. Some of them are very simple, like which write cache type you should use (RAM cache w/ overflow to disk is the right answer in 99% of the cases). Some can vary depending on your environment, as in memory sizing on PVS servers. (See earlier blog posts Cache Manager and How to properly size your memory.)

Then there are some design decisions that don’t have a clear winner and are often based on personal preference. For example, how are you planning to deliver the bootstrap file? Do you want to use BDM or TFTP? If BDM, ISO file or boot partition? For TFTP, how are you going to make sure that it’s highly available? There are quite a lot of good blog posts about this topic, but no clear winners.

Another example of a design decision with multiple valid answers is the design of vDisk stores. There is no best answer on which one to choose. To help, I am going to pull from my experience with all of them and provide a list of pros and cons.

Thanks to Citrix Insight Services, we have a much better understanding of what our customers are using and we can provide them with tailored recommendations – and I highly recommend all customers leverage CIS capabilities to keep their environment running well and help us determine what features are most important. With reports from almost 3000 PVS farms, we know that 85% of our customers are using decentralized architecture, while 15% use a centralized approach. So, I’m going to spend more time on a decentralized architectures. This is probably related to outdated best practices from the past. While it’s true that CIFS\SMB historically had issues with PVS, that is no longer the case and centralized stores are becoming more of a norm.

Typical Deployments

In general, there are two types of configuration:

• Centralized – File based (SMB\NFS)
• Decentralized – Local or Network storage

Centralized approach consists of a single store that is shared by multiple PVS servers. It is usually based on SMB shares with some occurrences of NFS and very few cases of block-based storage. Centralized stores are very common in the U.S., but might require more testing. You need to make sure that the vDisks are being properly cached on all PVS servers and confirm that failover is working as expected.

This can be challenging on some NAS devices, as their SMB implementation might not be perfect. If you are planning to use centralized store, you should spend enough time testing this configuration. Storage related issues are mostly gone now, but additional testing never hurts as any issue with a central store can mean that a whole site is unavailable. I’ve sometimes used virtualized Windows file servers to resolve these problems, but it’s not a very elegant solution and increases the complexity. Troubleshooting might be also more complicated; you’re adding an additional layer to the solution that might not work as expected.

From a performance perspective, the PVS server will offload most of the IOPS from a storage. After all, PVS servers are loading disks from storage only once and serve them from memory. Be aware though that merge operations have a more aggressive IOPS profile and you must size accordingly.

• PRO: Requires less storage (each vDisk is stored only once for a group of PVS servers)
• PRO: Works well with versioning as long as you’re careful with merging
• CON: Additional component can increase complexity (HA, performance)
• CON: Additional testing is recommended during implementation

Centralized vDisk store

Decentralized solution provides each PVS server with its own storage for vDisks. It can be local disks (in case of physical PVS servers, which is still quite a common scenario in Japan) or disks provisioned on a shared storage that are directly mapped to each PVS server (used with virtualized PVS servers). I’ve seen great and bad implementations of this architecture. The problem with a decentralized solution is that you need to make sure that all stores are synchronized – and while you can do this manually, that’s not manageable in larger environments.

There are different methods that you can use to replicate vdisks and each of them have specific challenges.

• You want to make sure that replication can handle versioning data.
• You want to make sure that replication does not have a negative impact on the ability of the PVS server to stream target devices.
• You want to make sure that a replication glitch will not delete any of your files. That can be challenging, especially with continuous replication – which is strongly discouraged for PVS replication.

One of my favorite implementations involves on-demand replication using DFS-R (using DFSRDIAG command). DFS-R is not automatically replicating all changes, but it needs to be triggered manually or is automated overnight.

Another important consideration is how replication can impact the caching ability of the PVS server. Different replication technologies behave differently and I’m planning to write another “PVS Internals” article focused on this topic. Small teaser – the most common method has some negative side effects that most people are not aware of, so stay tuned for PVS Internals #4.

I have preferred decentralized architectures in the past, as I’m a big believer in centralized configuration and decentralized execution, but this often requires automation/scripting skills and systematic approach to image management.

• PRO: Performance can be distributed across multiple separate storages (local)
• PRO: Easier to configure
• CON: Requires replication (manual or automated)
• CON: You need to make sure that all stores are identical
• CON: RIMS/PVS replication issues are common

Decentralized vDisk Stores

Hybrid Architectures

Most people are only familiar with these two approaches, but there are also some hybrid architectures. And surprisingly, combining two basic methods can yield the best results and might be the solution that you’re looking for. Hybrid architectures are especially useful for very large enterprise customers, while smaller and medium environments should stick with one of the two base methods.

Multiple Shared Stores is using a centralized approach. However, it consists of multiple shared stores that are replicated. We’ve used this method successfully for multi-site deployments in the past, but it’s not a very common method. I haven’t seen it in a very long time, but again your experience might vary and I would appreciate if you could leave some comments.

• PRO: Useful for multi-site deployments, but there are better solutions
• CON: Combines disadvantages of centralized and decentralized approach

Multiple Shared Stores

Local Development, Distributed Production is one of my favorite approaches, especially if you have a complex environment with multiple sites or want to scale-out your deployment. This method provides you with advantages of versioning, which is amazing for rapid build and testing of images, while providing you with scale-out capabilities and robust distributed solution.

In this architecture, you delegate one of the PVS servers as a maintenance/build/development server. This PVS server will have a dedicated store with versioning enabled that is attached only to this single machine. Once you are ready to release your updated image to production, you merge all changes into a single VHDX file and copy it over to the production store. Production store can be either centralized or decentralized, whatever is your preference.

While this solution might seem complicated, the flow is rather simple and very robust. Replication between development server and the rest of the environment can be fully automated and is reliable, as files do not change frequently and do not differ between servers. With a proper design and hub-and-spoke distribution architecture, you can support very complex environments.

• PRO: If done properly, provides centralized configuration and decentralized execution
• PRO: No issues with versioning
• PRO: Simple replication
• PRO: Highly scalable solution
• CON: Complex to setup
• CON: Management overhead for smaller environments

Hybrid Architecture

Summary

When designing your vDisk store architecture, you can choose if you want centralized, decentralized or hybrid solutions. Each of them have advantages and disadvantages and might be usable for different requirements. Small and medium customers should consider using simple centralized or decentralized architectures, while large enterprises could benefit from hybrid implementations.

In the next blog post, I will look at some of the common replication techniques and their advantages and disadvantages.

I would also like to thank Nick Rintalan, Grzegorz Iskra and Martin Latteier for their support and ideas. It’s easy to write blog posts like this if you have a superstar team to support you!