The last 2 weeks have been a major clean up effort in my home as multiple carloads of old tech made its way to electronics recycling. (Any of you need a SCSI-2 card?) When we found multiple wifi routers, the wife and I tried to figure out which would be a better choice for those times when we plug into GigE to do large transfers to the file server. The basic question was how much we believed those marketing numbers and whether those marketing numbers provided translated to all features.
There is a dirty secret amongst us marketing types… Well, it’s not much of a secret. The maximum performance of a product does not necessary correlate to performance for all features across the board. Enough of you smart folks out there know this and demand details. Fortunately for me, I’ve got a whip smart engineering crew that is fussy about those details so when I am asked, I’ve been blessed with interesting numbers and details to share.
The most recent situation came up just yesterday with an application firewall. A customer having followed the Sony PSN situation decided that the incremental cost of an application firewall as well worth it compared to the potential ramifications of assuming his application was secure. The high profile nature certainly helped get budget approval. The performance issue came up when he was puzzled why his CPU utilization went up. A quick check with his support and he found out that application firewall performance is not the same as maximum performance.
When I get asked about performance of a NetScaler, I really encourage people to benchmark it for themselves with their desired configuration. It’s important because real life performance rarely matches ideal benchmark performance. I’ve got the benchmark numbers if you ask, but in the end it’s all about the performance you’ll really experience. Now I’ll admit there is some selfishness in the answer -- the NetScaler performs extremely well with real life workloads, especially complex application policies such as those for the application firewall. We consistently outperform our competition when it comes to complex policies and we’re continuing our investment there to make sure that doesn’t change.
In the case of of the Application Firewall, my peak performance number is 12Gbps on the 21500 platform. The overall HTTP performance of the 21500 is 50Gbps so you can see how complex rules impact performance. The key technical issue at hand is the depth of the packet inspection -- at 50Gbps, the NetScaler has to only inspect the headers of the packet. When the application firewall is enabled, the NetScaler has to inspect the complete packet end to end. This is obviously a much larger task and thus takes more effort to do and is true for any networking device.
There are optimizations to be had around HOW a packet is scanned. Using various methods you can use such as scanning a large number of rules in parallel so it only takes on pass through the packet to check all rules, and then identifying places where you can skip some number of characters based on a mismatch. Regardless of the optimization, there is still additional effort involved compared to just looking at the top of a packet and forwarding it. In other words, be wary of anyone that claims they can scan the contents of a packet in the same time it takes to merely forward a packet.
So as you’re looking at performance numbers and trying to see what makes sense in your environment, be sure you look at the right numbers. Don’t, for instance, compare two systems based on their similar overall throughput. Instead, compare the numbers for the feature(s) you want to focus on and make sure you’re getting similar systems based on that. Finally, benchmark it yourself with your configuration and your workload. That will give you a much stronger sense of what’s real, what’s not and what’s marketing.