To say that the excitement behind the Splunk/AppFlow combination is huge is an understatement – it turns out that some of our largest customers are also our common customers and they have been lining up dates right and left to take the combination for a whirl in their labs.
We’ve been learning a lot from the process too with regard to where people are finding the most use for Splunk with AppFlow and seeing how it differs from what we originally thought based on early market feedback during the development process.
Replacing Web Server Logs – I have to be honest, this completely came out of the blue for me since it felt like there was already so much infrastructure around server logs, but alas this has come up with a few large web sites. The idea is that since they are slurping the server logs into databases for analysis anyway, having AppFlow feed the Splunk database from the start not only cuts out the middle step but goes on to add rich timing information that is otherwise lost in W3C style logs. Once there, Splunk’s report writing tools make it possible to do some things that most log analysis tools don’t do for you like k-means clustering.
Contextual Data Capture – In a site that is pushing even tens of thousands of requests per second, it isn’t possible to have all the data recorders on all the time. Because AppFlow can generate data based on policy (including timing policies), it is possible to define specific situations that data should be generated. For a Splunk user that is aggregating log data across many different systems, adding the conditional AppFlow data means that peculiar situations can get more easily debugged without having to deal with the tsunami of data that comes from AppFlow on an ongoing basis. (AppFlow on a loaded system can easily push 100k log events/sec — that’s a lot of data.) This is great operationally when you’re trying to find a needle in a haystack and are trying to minimize the size of the haystack. For example, a customer was looking for cases where either the servers were taking unusually long to respond or an HTTP error code came through. Using the Virtual Server Based Expressions in the NetScaler policy language, it is possible to generate AppFlow records ONLY when either of those situations are true. Now a quick correlation analysis of other events in the system at the time those AppFlow records are generated and you can pick out the problem in short order.
Cloud Monitoring – Several people picked up on this detail quickly… tools that they could count on in their infrastructure didn’t work in the cloud because the underlying infrastructure didn’t expose the data necessary. With AppFlow, they could get the data from a NetScaler VPX and leverage their Splunk installation for correlation analysis with other syslog data. Finally, the visibility they needed to get in order to make a cloud deployment successful is possible.
Have you been playing with AppFlow? Got an interesting use case that you’d like to share? Send it over — we’d love to hear it. (We’ll happily keep you anonymous if you desire.)