In my previous blog, I describe scalability testing I performed to validate a Citrix VDI-in-a-Box and HP ProLiant reference architecture. The reference architecture defines a turnkey-like appliance that simplifies the task of VDI deployment. Using this reference architecture as a basis for sizing, it’s relatively easy to determine the number and type of appliances that you’ll need to fit your user capacity and workload requirements.

In previous testing, I validated 50- and 100-user configurations using performance monitoring scripts and Login VSI, a VDI testing tool that captures response times while simulating user workloads. Under a Medium Login VSI workload (representing a typical knowledge worker), my results showed linear scalability without any significant degradation in response times.

Since these 50- and 100-user test runs did not reach Login VSIMax (the maximum user capacity at which performance starts to degrade), I decided to repeat the test runs with larger user capacities to push the limits of the reference architecture configurations. This blog describes my second series of tests with the same test configurations and user capacities of 70 and 120 users. I also ran two additional stress tests to investigate performance impacts as the number of users scaled. All in all, my second round of testing included four test runs:

  • 70-user configuration test. Since Login VSIMax was not reached in the original 50-user test, I wanted to see the point at which performance degraded consistently for the 50-user configuration.
  • 120-user configuration test. Using the 100-user configuration (with twice the CPU and memory resources of the 50-user configuration), I repeated the 100-user test with 120 users to see when Login VSIMax would occur.
  • Desktop restart test. In this 75-user test, I assessed the impact of restarting four desktops midstream during a test.
  • Storage impact test. I booted 50 desktops in a period of 10 minutes (simulating a boot storm) to examine the impact on the storage subsystem.

Test Results

Generally these tests used the same test environment and methodology as the initial 50- and 100-user test runs (see the first blog about my scalability testing for details). In addition to Login VSI, I used a set of scripts to capture performance metrics during each test run. In this way, I could observe the impact of the testing on system resources (CPU, memory, storage, and network I/O). Detailed results for the four test runs follow.

70 User Results (61 VSIMax)

The following graphs show Login VSIMax, CPU utilization, memory consumption, calculated read and write IOPS, and network traffic (received and sent), for the 70-user test configuration. With 70 users total, the 50-user configuration reached VSIMax at 61 users, as shown in the graph below. The server configuration was an HP ProLiant DL380p Gen8 Server with a single Intel Xeon E5-2680 CPU @ 2.70GHz (8 cores, 16 threads) and 128 GB of RAM (instead of 96 GB, which I used in the initial 50-user test).

Login VSI Data (61 VSIMax, 70 sessions launched)

To establish Login VSIMax, the Login VSI workload script records response times of seven operations within each workload test loop. The analysis tool uses one of two methods for calculating Login VSIMax, Classic or Dynamic (see http://www.loginvsi.com/analyzing-results). In this testing, the Dynamic method (in which average response times must consistently exceed a dynamically calculated threshold) was used since this approach is more appropriate for analyzing application virtualization. The formula for determining the threshold is explained on the Login VSI site. Login VSIMax indicates the active Login VSI users logged on to the system when response times consistently exceed this threshold.

 

CPU (Single, 16 logical cores)

This metric records utilization of physical processors in the host computer. The “\Hyper-V Hypervisor Logical Processor(*)\% Total Run Time” performance counter indicates overall processor utilization of the Hyper-V server. This is more accurate than using the system’s “% Processor Time” counter that measures only host processor time.

The CPU graph shows that the single Xeon E5-2680 CPU sustained 100% utilization for a period of over 4 minutes, indicating that CPU resources became exhausted when supporting over 50 users under a Login VSI Medium workload.

Memory (128GB Total)

This metric is the amount of physical RAM, in megabytes, immediately available for allocation to a process or for system use. It is equal to the sum of memory assigned to the standby (cached), free, and zero page lists.

Storage IOPS  

Calculated IOPS represents the combined rate of read and write operations on the C: disk.

Network

Network Interface Bytes Total/second is the combined rate at which bytes are sent and received over each network adapter, including framing characters.

120-User Results (113 VSIMax)

Using the 100-user configuration, I performed a test run with 120 users to determine when Login VSIMax would occur. The following graphs show Login VSIMax, CPU utilization, memory consumption, calculated read and write IOPS, and network traffic (received and sent) for the 120-user test configuration.

Login VSI Data (113 VSIMax, 120 sessions launched)

As shown in the graph below, the 100-user configuration reached VSIMax at 113 users.

CPU (Dual, 32 logical cores)

The graph shows CPU utilization for the Hyper-V server. It indicates that this server, hosted on two Xeon E5-2680 CPUs, sustained close to 100% utilization for more than 10 minutes, indicating the exhaustion of CPU resources with 120 users under a Medium workload.

Memory (128GB Total)

This metric shows the amount of available physical RAM, in megabytes.

Storage IOPS  

Calculated IOPS represents the combined rate of read and write operations on the C: disk.

Network

This metric is the combined rate at which bytes are sent and received over each network adapter, including framing characters.

Desktop Restart Test

Using an HP ProLiant server configuration with 2 CPUs and 128GB RAM, I conducted a 75-user stress test to investigate the impact of restarting four desktops midstream. As in other tests, user desktops were launched and Login VSI simulated user workloads.  At the halfway point (when the workload for user 37 began), I restarted four desktop sessions. Restarting the four desktops had no significant impact on response times and VSIMax was not reached. Performance monitoring scripts indicated that subsystem resources, including server CPU cycles, were still readily available.

The following graphs show the Login VSI data, CPU utilization, memory consumption, calculated read and write IOPS, and network traffic (received and sent), respectively, for the desktop restart test.

Login VSI Data (no VSIMax)

As shown in the graph below, Login VSIMax was not reached during this test.

CPU (Dual, 32 logical cores)

The graph shows CPU utilization for the Hyper-V server.

Memory (128GB Total)

This metric indicates the amount of available physical RAM, in megabytes.

Storage IOPS

Calculated IOPS represents the combined rate of read and write operations on the C: disk.

Network

This metric is the combined rate at which bytes are sent and received over each network adapter, including framing characters.

Storage Impact Test

In a production Citrix VDI-in-a-Box configuration, VDI-in-a-Box Manager staggers desktop power-ups, launching a desktop every 30 seconds. In my initial 50-user scalability test, it took 25 minutes to power on the full 50 VMs. In this stress test, I wanted to evaluate the I/O capacity of the storage subsystem, so I generated an artificial boot storm by turning on 50 desktops in a period of just 10 minutes. Although this stress test does not represent a true production VDI-in-a-Box deployment, it provides valuable insight into the performance of the storage subsystem.

The graph below shows the calculated read and write IOPS for the storage impact test.

The Average Disk Queue Length is the average number of combined read and write requests queued for C: during the sample interval. The warning range indicates that there are more than two I/Os waiting on the physical disk.

Summary

While you should take into account all relevant configuration factors — user workload types, availability needs, etc. — when sizing an actual production deployment, this testing reinforced the validity of the initial 50- and 100-user configurations under a Medium Login VSI workload. The results of these stress tests indicated that the reference architecture configurations had user capacity to spare, and were not adversely impacted by desktops restarting midstream. The storage configuration also could support the I/O demands of the artificial boot storm. For more details on the configurations, testing, and results, see the reference architecture report.

In my next blog, I’ll describe user experience testing that I performed for this reference architecture, which served to corroborate the results of my scalability and capacity testing.

References