With the recent release of the Cloud Provider Pack, our app orchestration technology (a.k.a. Project Rainmaker) is now available for immediate download and use. You can find more information about the many benefits you get from this technology in several recent blogs. Suffice to say – we have reinvented the way to manage complex multi-tenant, multi-farm deployments of XenApp with a minimum of effort on the administrator’s part.

This article is the second in a series of blogs by the engineering team responsible for the creation of the app orchestration technology, in which we will dive deeply into the architecture, deployment, and customization of the various components that comprise this groundbreaking new technology. (The first is available here.) Today, I will cover the architecture of the app orchestration system, and introduce all of the components that make up this system.

App Orchestration Component Diagram

Let’s break this picture down and investigate the individual components.

Configuration Server

This server hosts the primary components of app orchestration. Central to the entire picture is the App Orchestration Engine, and the REST API that it exposes. All components communicate with the engine exclusively via this REST API.

App Orchestration Engine

This component is the “brains” behind the app orchestration technology. Whenever a configuration change is made, it is written to the app orchestration database, and then the App Orchestration Engine determines all of the actions that need to happen across all of the Organizational Units, Farms, Workload Machines, and Web Interface machines in order to apply that change. These actions are scheduled as “Workflows” and are added to dispatch queues for the various agents.

When a configuration change is made, it completes immediately. The administrator’s intent is stored in the database, allowing the agents to apply the changes asynchronously. We call this approach “Desired-State”, since we always keep track of the administrator’s end-goal. This allows the system to perform multiple operations across different products and agents in the correct sequence and over extended periods of time, always moving the underlying systems toward the end-goal. If there are any failures, they can be corrected and the system will recover automatically and continue to march toward the desired configuration.

The Desired-State approach allows the app orchestration technology to be highly resistant to outages and failures. If the Configuration Server is offline, user sessions and normal XenApp farm operation are not impacted. The only impact is that configuration changes cannot be done during that outage. Still, for DR reasons and higher availability, it is recommended that at least two Configuration Servers are created per deployment.


This is the end-point where all of the other app orchestration components communicate with the App Orchestration Engine. The REST API uses HTTP(S) exclusively.

In order to perform all of the operations that we automate, sometimes the agents require Active Directory domain administrator credentials. While these are protected by access control so that only Agents can read them via the API, they are still transmitted over this channel, and over the network. For this reason is it critical that production deployments use HTTPS to protect those credentials during transmission. The App Studio setup program makes it easy to configure the REST API and Web Console Service to use SSL.

There will be further blogs on the topic of the API; stay tuned.

Web Console Service

Citrix App Studio is the name of the UI component that you use to configure app orchestration. The UI is written in HTML 5 and has been tested on multiple browsers to ensure compatibility. The Web Console Service component hosts the web UI.

The Web Console Service and the web UI are stateless. This allows administrators to connect to the web UI on any Configuration Server and perform changes.

Agent – Configuration Server

The Agent appears multiple times in the component diagram, and serves multiple roles. The Agent is the “brawn” behind the power of app orchestration – it performs the actions that the App Orchestration Engine has determined are necessary in order to bring the system to the desired configuration.

On the configuration server, the Agent is responsible for interfacing with Active Directory for operations such as creating and monitoring Import OUs, and moving Workload Machines to the correct OUs when they are allocated. All communication with AD is done via Active Directory Web Services (ADWS).

The agent is also responsible for communicating with Workload Machines which have not yet been allocated, and therefore are not part of any XenApp farm; for example, during the machine import process. It does so using PowerShell Remoting (WinRM) and executing the app orchestration scripts that are pre-installed there.

Web Interface Server

For any deployment, ultimately the users must be able to launch the applications they subscribe to, and that means they must access a Web Interface site (note: StoreFront support will come at a later date).

App orchestration manages multiple farms and automatically allocates farms to tenants based on isolation requirements and capacity limits. To simplify use of Web Interface in these complex scenarios, app orchestration will also create and configure Web Interface sites automatically based on per-tenant isolation requirements.

Agent – Web Interface Server

On the Web Interface server, the Agent is responsible for creating sites and pointing those sites to the farms for each tenant. The sites are created using basic settings. App orchestration does not yet interface with Access Gateway or Netscaler, so once the sites are created, you may need to do additional manual configuration to enable users to access the sites externally.

XenApp Controllers

App orchestration manages XenApp 6.5 farms only. In order to create the correct level of isolation for hosting users in a CSP environment, XenApp farms are strictly split into two different types of servers: Controllers, and Workload Machines (aka Session Hosts). Controllers never host sessions, and Workload Machines never receive session data about other Workload Machines.

XenApp Controllers managed by app orchestration must have the Agent installed. It is also very important to realize that app orchestration enforces the desired state that it obtains from the App Orchestration Engine. This means that importing a farm which already has configuration, such as existing session hosts or apps, may result in that configuration being overwritten or corrupted. Do not import farms containing existing configuration into App Studio.

Agent – XenApp Controllers

On the XenApp controllers, the Agent is responsible for creating and managing worker groups and published applications, and for managing the “Drain” mode of session hosts. It maintains the relationships between users, apps, worker groups, and organizational units so that multi-tenancy and isolation are achieved, matching the desired configuration. It is also responsible for “Joining” and “Un-joining” workload machines to the farm; it does this by connecting to those machines using PowerShell Remoting (WinRM) and executing the app orchestration scripts that are pre-installed there.

XenApp Workload Machines – aka Session Hosts

Of course, the primary measure of user capacity in XenApp is the number of servers which can host sessions. App orchestration will automatically manage allocation, deallocation, isolation, draining, joining and unjoining those machines to farms, and moving them to the correct organizational units.

App Orchestration Scripts

Since user density per session host is a critically important number for CSPs, app orchestration was designed so that it has zero CPU and memory footprint on the machines hosting sessions. There are no added services, and no workflows are directed to a session host running sessions. The only component that app orchestration adds to the session hosts is a collection of scripts and programs that are invoked remotely via PowerShell Remoting, when the machine is either being prepared to host sessions or being decommissioned.

All other actions that are orchestrated around session hosts are done by Agents, either on the XenApp Controllers or on the Configuration Server itself.


Though long, this article has just scratched the surface of how app orchestration works. Look forward to a series of blog articles over the next few weeks providing deeper dives into all of the various components and functionality provided.

This blog is part of a series on app orchestration. For the rest of the articles in this series, please refer to these articles:

  • Concepts
  • Architecture (this article)
  • Provisioning machines (coming soon)
  • Managing Tenants (coming soon)
  • Managing Advertisements (coming soon)
  • Managing Subscriptions (coming soon)
  • Patching Workload Machines (coming soon)
  • Understanding Workflows (coming soon)
  • Troubleshooting (coming soon)
  • Integration with CloudPortal Services Manager (coming soon)