Network taps, span ports and aggregators are soon becoming defunct with the Cloud revolution picking up steam and making IT pros rethink their application performance monitoring (APM) strategy. Unlike proprietary packet capture tools, AppFlow leverages in-place footprints within existing infrastructure, virtualized, cloud or otherwise, and delivers the APM data in an industry standard format that is modeled on the success of NetFlow/IPFIX. Steve Shah’s recent blogs Splunk and AppFlow & Changing the Market Landscape of APM call out the business drivers of NetScaler AppFlowTM. Here we will focus on the technical aspects of NetScaler AppFlowTM . The recent inundating requests from customers and sales teams alike prompted me to write this blog outlining the installation instructions to implement NetScaler AppFlowTM with Splunk and/or Solarwinds.
First we will take a look at configuring the NetScaler for generating AppFlow records. AppFlow is an advanced feature available on all NetScaler editions – standard, enterprise and platinum, for the NetScaler products. Here is a detailed knowledge base article that was published recently http://support.citrix.com/article/CTX130334 for NetScaler config. The NetScaler version required would be 220.127.116.11 or higher and remember all such new features like AppFlow & DataStream are only available in the nCore builds. Do remember to bind the appflow policies to the specific Vservers that you want AppFlow data to be generated.
For the NetScaler CLI enthusiasts here is how to do the same.
enable feature appflow
add appflow collector solarwinds_OR_splunk -IpAddress 10.217.100.100 ( port is optional default is UDP 4739)
add appflow action log_appflow -collectors solarwinds_OR_splunk
add appflow policy log_all true log_all_records ( this is the simplest policy, one of the more useful ones is in the support article mentioned above)
Now bind this policy to the vservers that needs to be monitored – > this step is critical without which no AppFlow records are generated.
bind lb vserver <vservername> -policyName log_all -priority 100
Check to make sure the vserver and the services bound have appflow enabled ( by default appflow is enabled).
show lb vserver <vservername> | grep Appflow should show “Appflow logging: ENABLED”
show service <servicename> | grep Appflow should also show “Appflow logging: ENABLED”
If appFlow is not enabled here is how one can enable this
set service <service> -appflowLog ENABLED
set lb vserver <vserver> -appflowLog ENABLED
Please set the desired appflow parameters to be included in the Appflow records. For a Proof of Concept please enable all fields.
show appflow param
IPFIX template refresh interval: 600 seconds
Appname refresh interval: 600 seconds
IPFIX flow record export interval: 600 seconds
IPFIX UDP Path MTU: 1472 bytes
HTTP URL logging: ENABLED
HTTP cookie logging: ENABLED
HTTP referer logging: ENABLED
HTTP method logging: ENABLED
HTTP host logging: ENABLED
HTTP user-agent logging: ENABLED
Log only client-side traffic: NO
To enable each field follow this example of enabling httpUserAgent.
set appflow param -httpUserAgent ENABLED
Set IPFIX flow record export interval to 60 seconds for faster AppFlow data export.
set appflow param -flowRecordInterval 60
Solarwinds AppFlow Tool configuration
Download from here Solarwinds AppFlow Free tool . To have this on the production ORION software please contact your Solarwinds Sales team.
On the startup screen make sure the UDP port matches the one on the NetScaler. Solarwinds Tool defaults to 2055. This is the only config change needed for Solarwinds.
A successful configuration and continuous reception of AppFlow data results in the following screen. The free tool functions for only one hour on each run.
This will be the main screen on Solarwinds once a decent number of AppFlow records are collected.
Splunk AppFlow Configuration
Download the Free Splunk App from here http://www.splunk.com/download?r=header . For further installation instructions please go to http://www.splunk.com/base/Documentation. Splunk App needed would be 4.2.2 or greater.
Now download the SplunkforCitrixNetScaler.tgz Appflow plugin from splunkbase.com here Splunk for Citrix NetScaler with AppFlow . This plugin works with the production Splunk App as well as the free one mentioned above.
To install the app, Click on App menu and choose “Manage apps” Now Click on “Install app from file” to arrive at the following screen. specify the location of the SplunkforCitrixNetScaler.tgz file for upload.
For Splunk Windows installs *only* the inputs.conf should be modified to look like this. The location of inputs.conf is in the image. Only lines 1 and 5 are modified from the original install to correct the Unix “/” to windows “\”
The configuration file (config.ini) is located in the app’s “default” directory, which is $SPLUNK_HOME/etc/apps/SplunkforCitrixNetScaler/default/config.ini (unix install) . The AppFlow dashboards and reports rely on the sourcetype=appflow. To change the UDP port for appflow one has to edit the config.ini and restart Splunk. By default Splunk and NetScaler both use UDP 4739 so no need to change to get it started.
Now Restart the Splunk Server to make sure the changed settings are applied. The successful install will result in the following screen. Be sure to click on “AppFlow” tab.
For more details on NetScaler Config Please refer to http://support.citrix.com/article/CTX128667 and http://support.citrix.com/proddocs/topic/netscaler-admin-guide-93/ns-ag-appflow-intr-wrapper-con.html.
For more information on NetScaler AppFlow click here http://www.citrix.com/English/ps2/products/subfeature.asp?contentID=2311581