Citrix and SAP have collaborated for nearly 20 years to build solutions that enable our joint customers to transform and run their businesses. In fact, more than 40 percent of SAP customers, including SAP, use Citrix solutions to improve their app environments and accelerate time to value of their SAP investment.

For more background on the partnership and how we are working together to enable our customers along their cloud journey, please join us for the Citrix Cloud Summit on October 8, where Steve Shute, EVP and COO Customer Success, will talk about boosting security, scalability, and productivity in the cloud.

As organizations begin to provide access to SAP systems via a Fiori web-based client, delivering this access via Citrix provides additional security, performance, and management benefits.

Previous SAP clients like SAPGUI were traditional Win32 client/server applications. The client was lightweight, and the data processing happened on the back-end servers. SAP Fiori apps are developed using the SAPUI5 framework and delivered to users via an HTML5-capable web browser. These apps offer much better role-based workflows, aggregating and processing content from various data sources, but they can be more client-compute intensive.

This Citrix and SAP whitepaper provides insights and guidance for our joint customers on how to successfully deliver a great SAP Fiori experience with Citrix. This blog post provides additional details that focus on performance testing we carried out.

It’s important that real-world load testing forms part of any Citrix delivery platform, and the transition to SAP Fiori is no different. The intensity of SAP Fiori apps varies greatly, given the choice that developers have to create their applications.

We did some testing for ourselves, and, with the help of colleagues at Login VSI, we ran tests against an SAP Fiori demo environment provided to us by SAP. (For a high level overview of SAP Fiori running on Citrix technologies, check out this recent SAP product blog from Oliver Graeff).

Login VSI provides the ability to run a specific repeatable workload from one or more locations and can drive that workload within a native web browser as well as via a web browser published by Citrix Virtual Apps and Desktops. This enabled us to do direct comparisons and to try out some different compute options.

SAP provides specific recommendations relating to browser caching, and we wanted to see what the impact would be if these recommendations weren’t in place. Many existing Citrix environments are configured to minimize profile storage and will often discard web browser caches at logoff. However, retaining the browser cache is one of the top recommendations SAP outline to their developers to support performance.

Load Testing Platform

First, let’s look at the testing environment.

We used a U.S.-based SAP environment and provided a set of standard Fiori apps to a set of demo user accounts. The environment was runningSAPUI5/Fiori 3. The user accounts browsed to the URL and performed specific actions that were repeated in a consistent manner (Logon, Show Projects, Create New Project, Save Project, Delete Project). We used Login VSI to launch these repeatable sessions as well as track the duration of each action. During our testing, we made changes to where the workload executed and then measured the impact on action duration time.

The Login VSI Launches and the Google Chrome web browsers were all running on VMs in public cloud locations (this allowed us to make changes more easily during our testing). When conducting your own performance testing, replicate the physical location and logical connectivity of your endpoint devices and Citrix Virtual Apps and Desktops workloads in relation to where your SAP services reside.

We started with a baseline workload — a single user running a Google Chrome web browser directly on a Windows OS. We then had similarly configured Google Chrome web browser-based workloads, but this time running as “published apps” via Citrix Virtual Apps and Desktops. The launchers were always based in the U.S., but to simulate the impact of bad network conditions, we also ran some tests from VDAs located in Europe. The impact of both high latency or small bandwidth is more or less equal to the application performance.

We used Azure instance types, but for simplicity’s sake here, we’ll reference them as D1, D3 and F4. The actual instances were as follows:

  • D1_v2, which has 1 vCPU (2.1 to 2.4 GHz) and 3.5GB RAM
  • D3_v2, which has 4 vCPUs (2.1 to 2.4 GHz) and 14GB RAM
  • F4s_v2, which has 4 vCPUs (3.4 to 3.7 GHz) and 8GB RAM

Test Methodology

Each workload ran with either a single user or 15 users (fixed number). We changed the system specifications or other variables to see how quickly the workload would complete. The duration for each action, from a number of identical test runs, was averaged and plotted on a graph.

The emphasis of this testing was on changing specific variables and noting the impact. If you are doing your own performance testing, once you establish an indicative workload for your own use case, we’d advise doing some additional VSImax testing. VSImax will give you deeper understanding of the maximum viable user density for your target platforms.

Test Results and Initial Findings

Citrix delivered workloads do not adversely impact performance.

The first thing we noted was that a “single user on a VDA” compared with a “single user on a native endpoint browser” resulted in no discernible difference in performance. (This was when the browser was executing in the same location, with similar compute resource available.) This should give everyone confidence that when configured correctly, there is very little downside to the user experience when using Citrix Virtual App delivery. Indeed, once you understand where user experience challenges can occur with SAP Fiori workloads in general, virtual app delivery provides a number of additional options to overcome these challenges and improve performance.

VDA Placement

Even with web-based apps such as SAP Fiori, latency between the client app (web browser) and the back-end service can affect user experience. This confirms what we’ve been saying at Citrix for a long time — place your workloads and back-end services close together. Our choice of test in this instance was somewhat extreme (placing the VDA workloads in a different continent and introducing 120ms of latency). But if you have users who are globally dispersed, and a single SAP back-end service, you can imagine the benefits for more remotely located users by delivering them a virtualized web browser that is far closer to the back-end service than their endpoint device.

We ran the EU vs U.S. comparison tests with D1 and D3 instance sizes.

With the D1 instances, the EU located workloads were ~20 percent slower, with “Project Load” and “Login” being the actions particularly affected.

With the D3 instances, the impact was much more dramatic. “Project Load” was 80 percent slower, and “Login” was 60 percent slower. Why? All our D1 workload testing was CPU constrained. The CPU was under so much load that it effectively masked the latency issues.

In our Login VSI test, the Fiori actions are relatively simple. The more complex your Fiori applications are, the more you need to be mindful of latency between the browser and back end. Indeed, it may not just be latency; a lack of bandwidth could also be problematic. That’s yet another reason to host the web browser closer to the back-end SAP service.

Adding CPU Cores

We mentioned that the D1 instance was CPU constrained. The D1 instance in Azure only has a single core, so it was not surprising to see that when we jumped to a D3 instance with four cores we saw improvements of 30 percent to 50 percent faster completion time for some of the actions. Nobody should be running Windows with just a single core. The same applies for Fiori workloads.

Increasing Clock Speed

In the next test we compared a D3 with an F4. Both of these instances have four cores, but the F4 has a faster clock speed. When we tested with just a single user, the difference was negligible; however in our 15 user test, we saw substantial improvements in both the “Login” and “Project Load” actions, as well as improvements in the “Save” and “Delete” actions. On a system that is under load (which is what you’ll likely have in a production deployment), the faster clock speed helps to provide a better user experience.

Deleting the Browser Cache

In many environments, when a user logs off (or even when they close their browser), the browser cache is deleted. This configuration is often put in place to reduce profile size, helping with session logon times and minimizing storage costs. However, SAP strongly recommends retaining browser cache for SAP Fiori apps. To test the impact, we compared a normal standard configured web browser alongside the browser forced to launch in Incognito mode (which would not retain browser cache). We ran these comparison tests using the F4 instances.

Unfortunately, we don’t have a “stacked” graph to share, so please note that the “scale” and “colors” of the two charts below are different. (Click the image to view larger.)

Everything happened within 12 seconds with a normal browser, but nearly double with incognito mode. The lack of browser cache affected almost all aspects of the workload.

CPU Overviews

The charts below show the CPU load of the VDA VMs during the tests. As you can see, 15 users (even for the fairly basic workload that we created for SAP Fiori) is too much for a D3. While the actions within the workloads completed, they took a much longer time to complete, and the user experience was severely impacted because the CPU was maxed out. When we look at the F4 we can see that while the VM was busy, it never maxed out and was able to complete each users’ activities in a timely manner. The overall CPU time is actually pretty similar, but the user experience was vastly different. Finally, without retaining the browser cache, the browser has to do a lot more work with Fiori Apps. We see a similar maxed CPU scenario, which resulted in a poor user experience.

Conclusion

We learned quite a bit through our testing:

  • Reduced latency between the web browser and SAP service improves user experience. We recommend using Citrix Virtual Apps and Desktops in Citrix Cloud to allow you to quickly integrate additional resource locations wherever you have SAP services to help ensure the best launch times and Fiori app performance.
  • Don’t allow the CPU in your VDAs to max out. This is true in all use cases, but for SAP Fiori we found that it could be easy to let happen, and the difference between 95 percent utilization and 100 percent utilization makes a huge difference with launch times and user experience.
  • The number of CPUs in the VDA is important. Web browsers are multi-threaded, so having additional CPUs available, especially with multi-user operating systems, helps ensure any spare capacity is leveraged to complete user actions more quickly.
  • CPU clock speed becomes more relevant once all the CPUs are in use. When the system is under load, faster clock speeds help prevent CPUs from getting maxed for longer periods.
  • Do not delete the web browser cache. Historical Citrix recommendations to reduce profile storage and improve session logon time may not be the best configurations for an SAP Fiori workload. Profile streaming or placing the browser cache in a container-based profile that is mounted at logon (FSLogix) would be much better options to consider.

While not specifically part of our testing, we also recommend that customers always follow the SAP design guidance when developing their Fiori apps. Limit the number of live tiles on the launchpad, and try to be efficient with your coding to minimize the compute processing that has to happen within the browser. More information is available in the whitepaper.

Above all, before rolling out SAP Fiori, test your own workload and understand its performance profile. Every customer’s SAP Fiori implementation is unique and will consist of custom developed Fiori apps that will impact performance requirements differently.

Need help assessing, designing and deploying SAP Fiori (or any other workload for that matter)? Reach out to your local Citrix account manager. They can provide options to engage Citrix Consulting to help you with your project and drive successful outcomes.