Okay, so Yogi Berra wasn’t an IT administrator and has likely never heard of NetFlow. (But then again many IT admins have never heard of Yogi Berra) But Yogi possessed wisdom far beyond the game of baseball.
Quick primer: In the not too distant past (maybe two recessions ago), network managers struggled to answer the most basic questions concerning their own network. Who are the top talkers? Where is congestion? What applications are driving the most traffic? Pretty basic stuff considering that these same managers just invested $1M+ in shiny new Cisco gear. Whether they needed this information for more thoughtful network planning (probably what they told their VP) or after-the-fact troubleshooting (the more likely case), they were desperate for better visibility into the layer 2-3 network. Sounds mundane now, but it was a real problem then.
Along came NetFlow (for the uninitiated: an industry-standard protocol for collecting information on network traffic), which provided an efficient mechanism to capture and export network traffic information for easy consumption by correlation and analytical tools. NetFlow. JFlow. SFlow. Take your pick, the problem was solved – or at least, nearly so.
Fast forward to today and the era of application networks, where traffic is driven by the exploding number of web applications. Now it’s the application owners wrestling with how to understand performance and availability issues. Pretty much the same challenges, they’ve simply moved ‘up the stack,’ and have been inherited by another IT group. But wait, doesn’t NetFlow (or IPFIX) provide application-layer information? Well, yes and no – depends on your point of view. If you’re a network manager, a web application is just TCP port 80, and a user is simply an IP address. As long as packets keep moving, application issues probably aren’t keeping you up at night.
If you happen to be responsible for the company’s SharePoint site, however, you might need just a bit more information. Well, maybe a lot more information. What application URLs or resources are being hit the most? What are the response times for each URL? Which application users are creating havoc on my infrastructure? Which are my best performing servers? Necessary information, but a bit out of the reach for your typical NetFlow solution.
Enter the realm of application performance monitoring (APM). There are some really impressive products in this space (which happen to carry an equally impressive price tag). The APM market has many proprietary tools, including software agents for application servers and expensive appliances that hang nicely off span ports. Want to use your existing NetFlow collectors and analytic engines to grab that data? Good luck – it’s not going to happen (at least with existing tools). The bottom line is that you’re going to pay a price for those nice graphs and charts.
Want to get a full 360-degree view of the application environment, from the application client all the way to the web server? Then be prepared roll up your sleeves and do some serious data correlation, as it’s not always straightforward to get all the data you need from one source. There’s got to be a better way, right?
Looks like help is on the way. There is one technology common to nearly all datacenter architectures that already captures (and processes!) detailed, per-flow application data: the application delivery controller (ADC). Finally… the ADC vendors (Citrix included) are moving in the right direction. The ADC sees the end-to-end application environment, from the application user, through to the web and application tier. Makes sense to use some of that intelligence.
Why have the ADC vendors been locking up that data for so long? Good question. I suspect things are about to change. Get ready for ADC-centric application visibility tools to complement today’s NetFlow solutions. The big questions that application owners will be asking:
• Will these ADC-based methods be open so that I can use my existing monitoring tools, or proprietary – forcing me to abandon tried and true solutions?
• Will the solutions extend into the cloud so I can continue to use when some of my applications migrate outside of my datacenter?
Answers to these questions are on the way.
What would Yogi say about better visibility for application traffic? Perhaps, “You can observe a lot just by watching.”