As a member of the U.S. Public Sector Consulting Services team, I have assisted many federal agencies in their deployment of NetScaler MPX FIPS-enabled appliances within their networks. In this blog post, I have identified some keys to success which will help you with your implementation efforts.
Just a quick refresher on what FIPS is and how it pertains to the Citrix NetScaler. FIPS is an acronym for Federal Information Processing Standards and the guideline is used by the U.S. government to certify cryptographic hardware. All federal government agencies are obligated to use FIPS 140-2-compliant devices to encrypt all application transactions and internet traffic, and the NetScaler MPX FIPS appliances address those requirements. For more information, take a look at CTX129543.
Below is a list of recommendations regarding setting up Citrix NetScaler SSL FIPS between a High Availability (HA) pair with some practical insight regarding configuration and management.
- Unsure if the Citrix NetScaler platform you are working with has been previously set up for SSL FIPS? Connect via an SSH client, such as PuTTY, and run the ‘show fips’ command from the NetScaler command line interface. This command will let you know if FIPS has been previously configured on the appliance. If the Hardware Security Module (HSM) isn’t initialized, then the NetScaler is not FIPS complaint and needs to be configured. Below is an example of an initialized HSM:
FIPS HSM Info:
HSM Label: FIPS-140-2
Initialization: FIPS-140-2 Level-2
HSM Serial Number: 2.1G1008-IC000007
HSM State: 2
Firmware Version: 1.1
Total Flash Memory: 1900428
Free Flash Memory: 1899720
Free SRAM Memory: 17201052
Total Crypto Cores: 3
Enabled Crypto Cores - When working with NetScaler FIPS SSL and you are unsure on the status of the HSM you may reset the HSM. Resetting the HSM will require you to save the running configuration and reboot the virtual instance or appliance. Backup all SSL FIPS keys and certificate key pairs before resetting the HSM.
- Outline a work plan before starting any NetScaler FIPS SSL configuration. This includes testing your ability to SSH between the NSIP addresses of the NetScaler appliances. The importance in drafting a blueprint prior to starting a configuration is due to the six-minute security timer that is built into FIPS and the HSM. If the Secure Information Management (SIM) key files are not replicated within the six-minute time window, then you are forced to reinitialize the SIM key transfer. Below is a table that outlines the steps to set up NetScaler FIPS SIM between a High Availability pair as part of a net new or greenfield project:
| Step | Command | Notes |
| 1 | reset ssl fips | This command resets the FIPS card to the default password. The default SSL FIPS security officer and user password for firmware 11.1 build may be found here: http://docs.citrix.com/en-us/netscaler/11/reference/netscaler-command-reference/ssl/ssl-fips.html (Run command(s) from primary node) |
| 2 | save ns config | Save the running configuration (Run command(s) from primary node) |
| 3 | reboot | Reboot the appliance (Run command(s) from primary node) |
| 4 | reset ssl fips | This command resets the FIPS card to the default password. The default SSL FIPS security officer and user password for firmware 11.1 build may be found here: http://docs.citrix.com/en-us/netscaler/11/reference/netscaler-command-reference/ssl/ssl-fips.html (Run command(s) from secondary node) |
| 5 | save ns config | Save the running configuration (Run command(s) from secondary node) |
| 6 | reboot | Reboot the appliance (Run command(s) from secondary node) |
| 7 | set ssl fips –initHSM Level-2 [-hsmLabel <string>] | Run the set SSL FIPS command (Run command(s) from primary node) |
| 8. | save ns config | Save the running configuration (Run command(s) from primary node) |
| 9. | reboot | Reboot the appliance (Run command(s) from primary node) |
| 10. | set ssl fips –initHSM Level-2 [-hsmLabel <string>] | Run the set SSL FIPS command (Run command(s) from secondary node) |
| 11. | save ns config | Save the running configuration (Run command(s) from secondary node) |
| 12. | reboot | Reboot the appliance (Run command(s) from secondary node) |
| 13. | init fipsSIMsource /nsconfig/ssl/source.cert | Initialize the source FIPS system (Run command(s) from primary node) |
| 14. | scp /nsconfig/ssl/source.cert nsroot@<IP_Address_of_Secondary_Appliance>:/nsconfig/ssl/ | This step correlates to the SIM configuration. Copy the source.cert file to the secondary node. The SIM files are stored in ‘/flash/nsconfig/ssl’ (Run command(s) from primary node) |
| 15. | init fipsSIMtarget /nsconfig/ssl/source.cert /nsconfig/ssl/target.key /nsconfig/ssl/target.secret | Initialize the target FIPS system (Run command(s) from secondary node) |
| 16. | scp /nsconfig/ssl/target.secret nsroot@< IP_Address_of_Primary_Appliance >:/nsconfig/ssl/ | This step correlates to the SIM configuration. Copy the target.secret file to the primary node. The SIM files are stored in ‘/flash/nsconfig/ssl’ (Run command(s) from secondary node) |
| 17. | enable fipsSIMsource /nsconfig/ssl/target.secret /nsconfig/ssl/source.secret | Enable the source FIPS system (Run command(s) from primary node) |
| 18. | scp /nsconfig/ssl/source.secret nsroot@< IP_Address_of_Secondary_Appliance >:/nsconfig/ssl/ | This step correlates to the SIM configuration. Copy the source.secret file to the secondary node. The SIM files are stored in ‘/flash/nsconfig/ssl’ (Run command(s) from primary node) |
| 19. | enable fipsSIMtarget /nsconfig/ssl/target.key /nsconfig/ssl/source.secret | Enable the target FIPS system (Run command(s) from secondary node) |
| 20. | create ssl fipsKey <fipsKeyName> -modulus
<positive_integer> [-exponent ( 3 | F4 )] |
Create the FIPS key on the source system (Run command(s) from primary node) |
| 21. | export ssl fipsKey <fipsKeyName> -key <string> | Export the FIPS key on the source system (Run command(s) from primary node). The Citrix NetScaler will propagate the FIPS key(s) automatically between a HA pair if the SIM was set up using the management GUI |
| 22. | import ssl fipsKey <fipsKeyName> -key <string> [-inform
( SIM | DER )] [-wrapKeyName <string>] [-iv <string>] |
Import the FIPS key to the target system (Run command(s) from secondary node) The Citrix NetScaler will propagate the FIPS key(s) automatically between an HA pair if the SIM was set up using the management GUI |
| 23. | show ssl fips | The results of this command will validate that the Citrix NetScaler SSL FIPS appliance(s) are configured. This command may run from both the primary and secondary nodes |
- Be mindful to document settings when configuring the HSM user password and the Security Officer password. There are limitations for the HSM password fields as the maximum length is 14 alphanumeric characters. Also, symbol characters are not allowed. Make sure this information is recorded securely in multiple locations. Without this information you are limited in managing the cryptographic keys stored on the HSM card.
- Use an SSH client such as PuTTy to configure the HSM, the SIM, and the FIPS key(s). Avoid the command line interface from within the NetScaler management web based GUI to issue FIPS commands. In certain scenarios, our team has observed the command line interface will not respond when attempting to issue FIPS command via a web browser.
- Configure the NetScaler for High Availability and Remote Procedure Call (RPC) node passwords prior to setting up the FIPS card, creating the FIPS key, and initializing the SIM feature.
- Use WinSCP to copy the target and source secret files verses typing or copying the Secure Copy (SCP) syntax. Just to recap from earlier, you have six minutes to completely configure the HSM and complete the transfer of the SIM files or you have to start the process all over again.
- Attempting to import a non-FIPS certificate key pair into the HSM may cause some unnecessary headaches such as certificate key pair mismatches or non-FIPS cipher supportability for the NetScaler FIPS edition platform. Follow the keep it simple principle, by generating the Certificate Signing Request (CSR) from the NetScaler using the FIPS key then simply submit the CSR file to a public or private Certificate Authority (CA).
- Be cognizant of NetScaler firmware builds and their supportability with the firmware version of the HSM. We recommend testing all firmware and HSM upgrades independently in a non-production environment.
- OpenSSL – a great tool for managing X.509 based certificates but the tool running from within the NetScaler shell has limited ability such as certificate chain validation once the certificates are in the HSM. Attempting to run the OpenSSL tool with the verify option will fail as the tool will be unable to find the certificate file or locate the FIPS crypto library.
- Use with caution – the ‘reset ssl fips’ command will invalidate all pre-existing certificate key pairs that reference any FIPS key(s) already present within the HSM. Running this command will impact all existing FIPS certificate key pair links and bindings on your virtual services or servers and may cause an unintended service interruption.
The same FIPS key can be transferred between the multiple datacenter sites that support the NetScaler FIPS platform. We recommend keeping all your NetScaler FIPS appliances on the same build version. This is useful when you wish to limit the number of certificate key pairs you wish to manage. For example, you have one NetScaler Gateway virtual web server that uses the same URL that leverages GSLB and you want to use the same FIPS SSL certificate between the two virtual servers. Simply set up the SIM between the primary nodes of each NetScaler at each datacenter site and ensure you allow TCP port 22 to support the SIM file exchange.
Marissa Schmidt wrote a blog post recently that outlines benefits, such as ‘Pay-As-You-Grow’ licensing of our new NetScaler MPX 14000 FIPS edition appliance. Finally, for more information on the Citrix NetScaler and FIPS please reference our product documentation here.
~Jonathan
