Syslog is one of the most commonly used methods for receiving audit messages from various systems in a centralized location. Syslog enables monitoring and long-term retention of audit, error, and other system messages to be collected for all infrastructure in one location.
From a security point of view, it’s important to have a syslog repository that isn’t stored in the same system where the syslog message was generated. Why? The local repository can be manipulated if a system has been compromised.
You can divide syslog requirements for Citrix ADC into two parts — Citrix ADC functionality and Citrix ADC operating system. We’ll look at both in this post.
Syslog for Citrix ADC Functionality
Our product documentation discusses syslog for Citrix ADC functionality and recommends using advanced policy expressions when configuring policies for the Citrix ADC. Classical policy expressions are legacy and might not be supported in future firmware releases.
The process of configuring syslog messages so they’re sent to a remote syslog server can be summarized in these three steps:
- Create a syslog server to define the destination and how the syslog messages should be sent
- Create a syslog policy to define when the messages should be sent
- Bind the syslog policy to define what messages are sent for
If there are no specific requirements, it’s normal to send all log levels, except debug levels. Debug levels can be enabled if in-depth troubleshooting is needed but be careful. It could send substantial amounts of data and quickly fill up storage.
If there are requirements to log which access lists (ACL) are being hit, ACL logging needs to be enabled both for the syslog server where the logs are to be sent as well as the individual ACL rules to be logged.
It is also a good idea to enable user-created syslog messages to be sent. Otherwise you won’t see anything in the log if message actions are configured and used.
Here are the CLI commands you can use to configure the syslog messages to be sent to a retention syslog server and to the Citrix Application Delivery Management server:
add audit syslogAction retention-syslog-SRV -logLevel EMERGENCY ALERT CRITICAL ERROR WARNING NOTICE INFORMATIONAL -acl ENABLED -userDefinedAuditlog YES
add audit syslogAction CitrixADM-syslog-SRV -logLevel EMERGENCY ALERT CRITICAL ERROR WARNING NOTICE INFORMATIONAL -userDefinedAuditlog YES
add audit syslogPolicy retention-syslog-POL true retention-syslog-SRV
add audit syslogPolicy CitrixADM-syslog-POL true CitrixADM-syslog-SRV
bind audit syslogGlobal -policyName retention-syslog-POL -priority 100
bind audit syslogGlobal -policyName CitrixADM-syslog-POL -priority 110Syslog for Citrix ADC Operating System
One thing I see in many deployments is that there is no configuration for sending syslog messages to an external syslog server (specifically for anything happening within the operating system). All that’s visible is that the user logs in and runs the command “shell,” and nothing else is logged to a retention syslog server for auditing purposes. That means if something happens, there’s no trace of it outside the logs stored locally on the Citrix ADC, which in the event of a device failure are not always readily retrievable. Further, it is not best practice to explicitly trust the integrity of local logs.
Some of the configuration files that exist in /etc can be copied to /nsconfig and will become resilient during a reboot of the Citrix ADC; /etc/syslog.conf is one of those files.
Which syslog local you select depends on the configuration of the syslog server where the messages are sent. For a Citrix Application Delivery Management server, you can use all the available locals, 0 through 7. The star behind local7 defines which messages are sent to this syslog server. In this example, I am sending all messages.
Modify /nsconfig/rc.netscaler to restart the syslog daemon after boot to make sure the correct configuration file is used. If the Citrix ADC is configured as a high availability pair NSIP cannot be entered in clear text, instead it can be extracted from the ns.conf file as done in the example below.
One final thing to note is that all commands a user runs in shell will be run as the user “root,” which is also the username that will be seen in the syslog messages.
To enable syslog messages so events in the operating system go to an external syslog server, take the following steps:
- Copy /etc/syslog.conf to /nsconfig/syslog.conf
- Add the following lines to the file /nsconfig/syslog.conf
- local7.* @<IP Address retention syslog server>
- local7.* @<IP Address Citrix ADM server>
- Add the following lines to the file /nsconfig/rc.netscaler
- pkill syslogd
- nsip=$(grep -i ‘set ns config -IPAddress’ /nsconfig/ns.conf | cut -d ‘ ‘ -f 5)
- /usr/sbin/syslogd -b $nsip -n -v -v -8 -C &
- Force an HA synchronization and reboot both nodes in the HA Pair for the changes to take effect
Trusted syslog information is critical for troubleshooting, forensics, and more, and the information in this blog post should help you ensure thorough auditing of your Citrix ADC environment. Learn more about setting up audit logging in our product documentation.
