This is the 2nd blog of a three part blog series on architecting an on-demand VM provisioning system in the cloud. In the first blog, I discussed the conceptual architecture for how you could design the system using a Cloud Provider (such as SoftLayer or Amazon EC2) or using your own virtual machine infrastructure (such as XenServer or Hyper-V).

In the next two blogs, I’m going to take a deeper dive by discussing a few real-world examples for cloud projects we have done at Citrix. This blog will be dedicated to discussing the high-level architecture of the Virtual Demo Center. In Part 3 – I will discuss another Citrix project called the Solution Center Cloud. Before I get into the architecture of the Virtual Demo Center, it’s important to first understand what this system is and how it’s used.

What is the Virtual Demo Center?
The Virtual Demo Center (VDC) is a web-based system that allows Citrix sales teams to provision complete Citrix demo environments in the cloud. Since it is web-based, environments can be provisioned “on-demand” from any location, and they can be accessed from any location that has an internet connection. This fits in nicely with the mobile nature of the sales teams as they roam from site to site giving demos. Complete demo environments are provisioned within 30 to 60 minutes depending on how many virtual machines are included within the environment. Environments run a defined time period as specified by the user, at which point the environment is automatically cancelled by the system. A screen shot of the VDC portal is shown below.

The VDC leverages SoftLayer as the cloud provider. In case you are new to SoftLayer, they are a rapidly growing cloud provider that has datacenters in Dallas, Seattle, and Washington DC. Since SoftLayer supports spinning up virtual machines in three datacenters, we configured the VDC to allow users to do the same; users typically would choose the datacenter closest to them to give their demo environment the lowest latency and best performance. SoftLayer uses XenServer internally for their host infrastructure, making them a nice fit for creating Citrix environments comprised of virtual machines and VPX virtual appliances.

What does the Virtual Demo Center provision?
The VDC currently provisions three types of Citrix demo environments, each of them consisting of different sets of virtual machines. We created templates for each of the virtual machines that comprise an environment. The templates contain all of the Citrix software pre-installed and pre-configured to allow them to be ready to use right after a provision is completed; no additional configuration is required by the end-user.

It takes about 10 to 20 minutes to provision a single virtual machine from a template on SoftLayer, although we do provision multiple instances concurrently to save time. All told, a complete environment provision typically takes 30 to 60 minutes as mentioned above. All environments are placed on isolated vlans so that users do not step on each other’s toes.

The demo environments that can be provisioned are outlined in the table below:

Demo Environment Virtual Machines
XenDesktop 4 (with XenApp 6 and App-V application delivery) • Active Directory Domain Controller
• Desktop Deliver Controller
• XenApp 6 (with App-V components)
• Virtual Desktop Agent 1 (Windows 7)
• Virtual Desktop Agent 2 (Windows 7)
• Virtual Desktop Agent 3 (Windows XP)
XenDesktop 4 (with XenApp 5 application delivery) • Active Directory Domain Controller
• Desktop Deliver Controller
• XenApp 5
• Virtual Desktop Agent 1 (Windows 7)
• Virtual Desktop Agent 2 (Windows 7)
• Virtual Desktop Agent 3 (Windows XP)
XenApp 5 (stand-alone environment) • Active Directory Domain Controller
• XenApp 5 (server 1)
• XenApp 5 (server 2)

Virtual Demo Center Physical Architecture
You should now have a better idea for what the VDC is and the demo environments we can provision from it. It’s now time to go into the architectural details of the VDC system itself.

The physical architecture of the VDC is shown in the picture below.

There are a couple key points about SoftLayer to help explain the context of this picture. The first point is that SoftLayer has the concept of a CloudLayer and HardwareLayer. The CloudLayer represents virtual machines that are provisioned as cloud computing instances (CCIs). When you provision a CCI, you have no visibility into the specific XenServer the virtual machine is placed on. SoftLayer will internally locate an available XenServer to host the virtual machine that has the appropriate hardware specs (RAM, CPU, etc.) required for the virtual machine. The HardwareLayer is not used within the current representation of the Virtual Demo Center, but it refers to physical XenServers that are purchased by you that reside up on SoftLayer. With the HardwareLayer, you can manage all aspects of the XenServer itself since you essentially rent the entire box.

The second key point is that the API provided by SoftLayer for performing actions on the CloudLayer is a local API only at this time. This means you have a couple of options for using the SoftLayer API: (1) Host your program that leverages the API on a CCI that resides in the SoftLayer cloud or (2) Host your program on a local machine that resides in your local environment but then have a VPN tunnel open to SoftLayer for the API calls to go through. We personally didn’t like the idea of having an “always on” VPN tunnel to run the VDC portal environment locally at Citrix, so we chose the first option. But for creating the components that leverage the API, we developed those on our own local development machines and then configured a VPN tunnel to test our components. The Citrix Portal Environment that you see in the picture above (Portal Server and SQL Server) are both virtual machines that run as CCIs within SoftLayer to give us direct API access.

The components that make up the architecture are discussed in the following table:

Component Description
NetScaler All users go through a NetScaler to access the web portal. NetScaler wasn’t required for the solution, but we definitely recommend it to help protect the portal infrastructure machines that run in the cloud.
When you provision a VM in the cloud, that VM is given both a public and private IP address. Since the VM is on the public internet, it is very prone to being attacked. In fact, we experienced many attacks very early on during this project. To alleviate some of these attacks, we disabled the public interfaces on all of the portal infrastructure machines. The NetScaler was then configured with a LB vserver to point to our portal website. We then configured public DNS to redirect all requests to our portal to go through the NetScaler. Having a NetScaler in front has definitely helped protect the site!
Web Portal The web portal is the GUI that users leverage for provisioning environments, viewing environment details, and cancelling environments. I showed a screen shot of the web portal at the top of this blog.
The key thing to note about the web portal is that it only communicates with the SQL Server database. It does not communicate with the Windows Service or SoftLayer API directly.
To handle authentication for the web portal, we tied it to the MyCitrix web service as noted below.
MyCitrix Web Service The VDC leverages MyCitrix for authentication. If you are a current Citrix customer, MyCitrix (www.mycitrix.com) should be very familiar to you since you use this site to obtain your Citrix software and product licenses.
MyCitrix offers a web service for performing authentication requests against the MyCitrix credential database. Our web portal just ties into this web service directly. To clarify the above picture a bit, the MyCitrix web service resides on the Citrix network, not on a virtual machine (CCI) within SoftLayer. The web portal makes a SSL connection to the web service for performing authentication.
There are a lot of options that can be chosen on how to perform authentication for the web portal. In the initial prototypes we built for the VDC, we actually leveraged an Active Directory virtual machine that was part of our Citrix Portal Environment on SoftLayer – so this can work very well also.
SQL Server The SQL Server hosts the database for the VDC. The VDC database has about 10+ tables but I’ll just focus on three key ones here that similar systems will likely have as well: Templates, Environments, and Instances.
The Templates table contains the metadata for the templates stored on SoftLayer (e.g. template id). The way to think about this table is that this table defines the resources that are available to be provisioned by the system. If you are designing a similar system, your table schema may be different but the overall purpose is the same – the system needs to know what is available to be provisioned.
When a user makes a provisioning request from the web portal, the request is stored in the Environments and Instances tables. The Environments table contains the high-level metadata for the entire environment to be provisioned (portal username, demo environment type, time to provision environment, time to cancel environment, provision status for entire environment, etc.). The Instances table contains the meta-data for each instance within the environment (template id to use for provisioning instance, instance id, instance status, time completed, time cancelled, etc.) Essentially, a single demo environment request will produce one entry within the Environments table and multiple entries in the Instances table.
There’s a ton more here but this should give you the gist. I may plan another blog that just goes deeper into our database schema since there’s a lot to it – we have views and stored procedures as well. If you are designing a similar provisioning solution, your database probably will have tables similar to the ones mentioned above.
Windows Service The Windows Service runs in the background and performs the actual operations on SoftLayer (e.g. ordering vlans and instances, cancelling vlans and instances, checking the status of instances, etc.). The Windows Service executes its various functions every 3 minutes so that users do not have to wait long for their requests to get started.
The Windows Services learns about what it needs to do by making calls to the SQL Server database. Each major function of the service is constructed roughly the same way:
(1) Perform SQL query to find out if it should perform an action
(2) If records were returned, call SoftLayer APIs to perform the action
(3) Retrieve result from SoftLayer and update data within database
The Windows Service also connects to a SMTP Service to send emails to users when critical actions have been completed. Emails are sent when the following actions are performed by the service:
• Environment has completed provisioning
• Environment is about to be cancelled within specified time period
• Environment has been cancelled and can no longer be accessed
SMTP Service The SMTP Service is not depicted in the picture above, but it resides on the Portal Server as well. This service is used by the Windows Service to send emails to users when critical actions have been performed. See the Windows Service section above for more information.
SoftLayer APIs SoftLayer provides their own API for interacting with the Cloud Computing Instances (CCIs) provisioned on the SoftLayer cloud. This API is discussed here – http://sldn.softlayer.com/wiki/index.php/The_SoftLayer_API.
I mentioned earlier that SoftLayer uses XenServer as their host infrastructure. The SoftLayer API is really just a wrapper to the XenServer API. They expose functions that allow you to order CCIs, cancel CCIs, gather status of CCIs, etc. I also mentioned earlier that the SoftLayer API is not a publicly accessible API. You either need to run your program on a CCI to gain direct access, or create a VPN tunnel to SoftLayer to allow your program to run in your own local environment.
The Windows Service is the only component in the VDC environment that calls the SoftLayer APIs. The Windows Service runs on the Portal Server, which is a CCI running on SoftLayer.
VM Templates The demo environments that are provisioned by the VDC all come from templates that were pre-built by our team. To create a template on SoftLayer, you just provision a new CCI, install whatever software you want on it, make any other tweaks to it, and then use the SoftLayer portal (or API) to create a template from that CCI. A template as defined by SoftLayer is really just a copy of the virtual disk for a virtual machine.
When a CCI is provisioned from a template, SoftLayer makes a copy of the virtual disk, applies a hostname to the virtual machine, and performs a number of other sysprepping actions and scripting actions to make the CCI available for use.
The demo environments that we built for the VDC contain a varying number of templates to help provision those environments. Please see the section above on What does the Virtual Demo Center provision? to understand the specific templates built for each environment.
Virtual Machines The virtual machines are the Cloud Computing Instances (CCIs) created from the templates on the SoftLayer CloudLayer. Each CCI has a public and private IP address. The public IP address allows end-users to connect directly to the virtual machine from the internet. The private IP address allows the virtual machines to communicate internally on their own private network as part of a dedicated vlan.
When an entire demo environment has completed provisioning, we gather the credentials for the virtual machines and provide them in an email we send to users regarding their environment being ready for use. The VDC web portal also displays the same information about how to connect to the virtual machines.
Since the demo environments consist of XenDesktop and XenApp workloads, users connect to the Web Interface site hosted on one of the virtual machines to launch a published desktop or published application. The user then has a direct ICA connection to the published resource.
If the user wants to connect directly to the virtual machines to perform some admin functions, they can RDP to the public interface of the virtual machines. The VDC web portal exposes the public IP address for all virtual machines in the environment to allow the user to perform admin operations on any virtual machine.

Summary
This blog provides a high-level overview of the Virtual Demo Center architecture. Since the Virtual Demo Center has been launched in February, it has made quite a splash with the sales teams since it simplifies their jobs quite a bit. There’s a lot of talk about cloud computing in general, but you rarely hear about specific examples of how it’s being used. My goal here was to showcase one real-world example of how we are using cloud computing at Citrix to help improve our way of doing business.

Although we are using SoftLayer as the cloud provider for the Virtual Demo Center, this same architecture could really work with any other cloud provider such as Amazon EC2. The conceptual architecture for having a Web Portal, Windows Service, and Database would be the same; just the details would differ. For example, you would need to create the desired templates on the cloud provider’s system, re-work the Windows Service to use the API required by that vendor, update the database schema a bit to reflect some of the different attributes used by the new API, and rework the portal interface as needed.

In the next blog, I will wrap up this blog series by discussing a different real-world cloud computing example at Citrix called the Solution Center Cloud. Stay tuned!

Blogs in this series:
Part 1 Conceptual architecture
• Part 2 – Real-world example 1 – Virtual Demo Center (this one)
Part 3 – Real-world example 2 – Solution Center Cloud

Ed York – Architect – Worldwide Technical Readiness
Ask-the-Architect Site: http://community.citrix.com/p/product-automation#home
Follow Me on twitter: http://twitter.com/citrixedy