Citrix NetScaler recently added support for Safenet external hardware security modules (HSM.) The Safenet HSM is an separate network appliance that stores private keys used for SSL communication. The Safenet HSM is FIPS-compliant and can be used to store private keys for various network devices including the NetScaler appliance. In many cases the Safenet HSM module used with the NetScaler appliance is a substitute for the purchase of a Citrix NetScaler with the FIPS hardware module onboard.

The Safenet HSM is a hardware appliance that looks similar to the image shown here.

The Safenet HSM can be used with any NetScaler VPX, NetScaler MPX or NetScaler VPX instance on an SDX appliance running NetScaler version 11.1 build 55.10 or higher or NetScaler version 12.0 build 53.6 or higher. The Safenet HSM can not be used with any NetScaler FIPS appliances.

The Safenet HSM connects to network infrastructure via Ethernet. Safenet has no requirement to be placed on a separate VLAN because even though it can still be reached with SSH to perform some limited admin functions, anything that would give access to keys requires the administrator to connect with a special USB device attached to the Safenet HSM appliance or to the remote admin workstation with a special hardware device and a fixed IP address. A good practice would be to connect the Safenet HSM on a separate secure VLAN despite the fact that Safenet does not require a separate VLAN.

The Safenet HSM is accessed by the NetScaler and other network appliances via an encrypted IP protocol listening on TCP port 1792. Both the NetScaler and the HSM module need to be able to reach each other on TCP 1792.

The Safenet HSM module can be configured with multiple partitions, each assigned separate resources and space for private keys in order to facilitate multi-tenant environments or simply to separate keys for different applications or groups of applications. The number of partitions available on a specific Safenet HSM is determined by hardware version and licensing.

Communication between the NetScaler and the Safenet HSM is secured through the exchange of certificates between the NetScaler appliance (the client) and the Safenet HSM (the server.) After the certificates have been exchanged, the Safenet HSM module registers the NetScaler as a client and assigns a partition for the use of the NetScaler. The NetScaler appliance then registers the Safenet HSM and partition as a server.

When using the Safenet HSM as designed, the keys never exist anywhere outside the Safenet HSM module. This is critical to the security model and you need a security exception request in most organizations to import a key into the Safenet HSM that was created elsewhere. The commands used to create the keys create them directly on the Safenet HSM. If the Safenet HSMs are disconnected or if they suffer from a hardware failure, the NetScaler SSL session will fail.

Most users of Safenet HSM modules use them in an active-active high availability group with two or more members in order to overcome the possibility of a single HSM failure. High availability with Safenet HSM modules uses the client software to manage high availability. Each individual Safenet HSM module is unaware that it is part of a high availability group. The clients and the client software manage the high availability and keep the keys synchronized across the members of the HA group. In a Safenet HA configuration, the NetScaler appliance must exchange keys and register separately with each Safenet HSM appliance using the Safenet client software resident on the NetScaler appliance. When the NetScaler appliances are configured in a high availability pair, both the primary and secondary NetScaler appliances must register separately with each of the Safenet HSM appliances in the Safenet HA group.

In order to configure the NetScaler appliance to work with Safenet HSM, verify the following settings first:

  1. NetScaler version is 11.1 build 55.10 or higher or NetScaler version 12.0 build 53.6 or higher.
  2. NetScalers are not FIPS models.
  3. Create a separate partition for the NetScalers on each Safenet HSM appliance and make sure the partition passwords are the same for the partitions on each of the Safenet HSM appliances. (If the NetScalers are configured for high availability, both the primary and secondary NetScalers will connect to the same partition of each Safenet HSM in the Safenet HA group. The partition names do not have to match on each of the Safenet HA members, but the passwords need to be identical. Additionally, a bug in the NetScaler does not properly handle using $ as the starting or ending special character in a partition password, so we recommend using other special characters in the middle of the password.)
  4. Verify that each Safenet HSM is configured with Allow Cloning and Allow Network Replication (See following images)

Log into the first HSM and run hsm showPolicies

Log into the second HSM and run hsm showPolicies

If the settings are not turned on, use the set hsm command to change the settings to On.

Once the prerequistes are met, use the following commands:

Initial NetScaler Steps

  1. Change directory on NetScaler into /var/safenet: cd /var/safenet
  2. Run the install_client.sh script (be sure to install version 6.2.2 because version 6.0 does not allow HA functionality in the NetScaler implementation): ./install_client.sh -v 622
  3. Change directory on NetScaler into /var/safenet/config: cd /var/safenet/config
  4. Run the script safenet_config: sh safenet_config
  5. Verify that the file /etc/Chrystoki.conf was created: ls -l /etc/Chrystoki.conf
  6. Verify that the symbolic link /usr/lib/libCryptoki2_64.so was created: ls -l /usr/lib/libCryptoki2_64.so
  7. Change directory on NetScaler into /var/safenet/safenet/lunaclient/bin: cd /var/safenet/safenet/lunaclient/bin
  8. Run the vtl script to create a certificate for the NetScaler to use when communicating with the Safenet HSM: ./vtl createCert -n (NSIP)
  9. Verify that the file /etc/Chrystoki.conf contains references to the new certificate: more /etc/Chrystoki.conf
  10. Verify that the new certificate files were created and placed in the proper directory: ls –l var/safenet/safenet/lunaclient/cert/client/
  11. SCP the client certificate created to allow NetScaler communication to the Safenet HSM to the first Safenet HSM: scp /var/safenet/safenet/lunaclient/cert/client/(NSIP).pem (AccountName)@(HSM1-IP):
  12. SCP the client certificate created to allow NetScaler communication to the Safenet HSM to the second Safenet HSM: scp /var/safenet/safenet/lunaclient/cert/client/(NSIP).pem (AccountName)@(HSM2-IP):
  13. SCP the server certificate from the first Safenet HSM to allow communication to the NetScaler: scp (AccountName)@(HSM1-IP):server.pem  /var/safenet/safenet/lunaclient/cert/server/(HSM1-IP).pem
  14. SCP the server certificate from the first Safenet HSM to allow communication to the NetScaler: scp (AccountName)@(HSM2-IP):server.pem  /var/safenet/safenet/lunaclient/cert/server/(HSM2-IP).pem
  15. Verify that the server certificate files exist in the proper directory : ls –l /var/safenet/safenet/lunaclient/cert/server/
  16. Repeat steps 1-15 from this section for the secondary NetScaler in the NetScaler HA pair.

Initial Safenet HSM Steps

  1. SSH to the first Safenet HSM: ssh -l (AccountName) (HSM1-IP)
  2. Register the NetScaler as a client on the Safenet HSM: client register –client (ClientName) -ip (NSIP)
  3. List the partitions on the Safenet HSM and verify the name and partition serial number of the partition assigned for the NetScalers: partition list
  4. Assign the partition to the client that was registered in previous steps: client assignPartition -client (ClientName) -par (PartitionName)
  5. Exit the HSM module: exit
  6. SSH to the second Safenet HSM: ssh -l (AccountName) (HSM2-IP)
  7. Register the NetScaler as a client on the Safenet HSM: client register –client (ClientName) -ip (NSIP)
  8. List the partitions on the Safenet HSM and verify the name and partition serial number of the partition assigned for the NetScalers: partition list
  9. Assign the partition to the client that was registered in previous steps: client assignPartition -client (ClientName) -par (PartitionName)
  10. Exit the HSM module: exit
  11. Repeat steps 1-10 from this section for the secondary NetScaler in the NetScaler HA pair.

Additional NetScaler Steps

  1. Change directory into /var/safenet/safenet/lunaclient/bin: cd /var/safenet/safenet/lunaclient/bin
  2. Run the vtl script to register the first HSM as a server on the NetScaler: ./vtl addserver -n (HSM1-IP) -c /var/safenet/safenet/lunaclient/cert/server/(HSM-IP).pem
  3. Run the vtl script to register the second HSM as a server on the NetScaler: ./vtl addserver -n (HSM2-IP) -c /var/safenet/safenet/lunaclient/cert/server/(HSM-IP).pem
  4. Run the vtl script to verify that the servers were registered correctly: ./vtl listServer
  5. Run the vtl script to verify that the network trust links between the NetScaler and the Safenet HSM appliances are functioning correctly: ./vtl verify
  6. Run the lunacm shell to configure the HA for the Safenet HSM appliances: ./lunacm
  7. Create a new Safenet HA group with the first member being the first HSM appliance and the partition assigned on that partition for the NetScaler: haGroup creategroup -serialNumber (serial number from partition list) -l (Group Friendly Name) -p (Partition Password)
  8. Add the second HSM appliance and the partition created for the NetScaler to the Safenet HA group: hagroup addMember -group (Group Friendly Name) -serialNumber (serial number from partition list) -password (Partition Password)
  9. Verify the configuration of the HA group: hagroup listGroups
  10. Configure the NetScaler to use the new HA group only to avoid mistakes with adding keys to only one member of the group: hagroup HAOnly –enable
  11. Configure the HA group to synchronize keys across all members of the Safenet HA group: hagroup synchronize -group (Group Friendly Name) -password (Partition Password) -enable
  12. Exit the lunacm shell: exit
  13. Exit back to the NetScaler CLI: exit
  14. Save the NetScaler configuration: save ns config
  15. Go back to the NetScaler BSD shell: shell
  16. Verify the new HA configurations and all Safenet HSM registrations are in the Safenet configuration file: more /etc/Chrystoki.conf
  17. Copy the /etc/Chrystoki.conf file into the /var/safenet/config directory to allow the NetScaler to start the Safenet client processes automatically on reboot: cp /etc/Chrystoki.conf /var/safenet/config/
  18. Start the Safenet Gateway client process: sh /var/safenet/gateway/start_safenet_gw
  19. Verify the Safenet Gateway client process is running: ps –aux | grep safenet_gw
  20. Create the /var/safenet/safenet_is_enrolled file to signal the NetScaler to automatically start the Safenet client processes after reboot: touch /var/safenet/safenet_is_enrolled
  21. Reboot the NetScaler to verify the processes start automatically at boot time: reboot
  22. Log back into the NetScaler after reboot
  23. Exit the NetScaler CLI to the BSD shell: shell
  24. Verify the Safenet Gateway client process is running: ps –aux | grep safenet_gw
  25. Repeat steps 1-24 from this section for the secondary NetScaler in the NetScaler HA pair

Using the Safenet HSM from the NetScaler appliance

Use the following commands from a standalone NetScaler appliance or from the primary member of a NetScaler HA pair:

  1. Change directories into the /var/safenet/safenet/lunaclient/bin/ directory: cd /var/safenet/safenet/lunaclient/bin/
  2. Use the cmu script to generate a new private server key on the Safenet HSM HA pair: ./cmu gen -modulusBits=2048 -publicExponent=65537 -sign=T -verify=T –label=(HSMKeyName)
  3. List the handle IDs for all certificates on the Safenet HSM partition: ./cmu list
  4. Request a CSR based off the new keys created on the Safenet HSM appliance: ./cmu requestcertificate
  5. Verify the CSR file was created in the proper directory: ls /var/safenet/safenet/lunaclient/bin/
  6. Copy the CSR file to local workstation and submit to a trusted certificate authority to have a new certificate signed.
  7. Copy the certificate returned from the trusted certificate authority to /nsconfig/ssl on the NetScaler appliance.
  8. Exit the BSD shell back to the NetScaler CLI: exit
  9. Add the key on the Safenet HSM appliance as an HSM key on the NetScaler (This step can be performed from the NetScaler CLI or from the NetScaler GUI. This step will also give an error message due to the increased time taken to complete the step, but the key will be properly created. The error message can be ignored.): add ssl hsmkey (HSMKeyName) -hsmType SAFENET -serialNum (HSMPartitionSerialNumber) -password (Partition Password)
  10. Verify the command executed successfully: show run | grep –i hsm
  11. Add the new SSL certificate using the new HSM key and the new certificate: add ssl certkey (FriendlyName) -cert (CertFileName) -hsmkey (HSMKeyName)
  12. Verify that all commands executed properly: show run | grep –i hsm

At this point, the NetScaler can be configured with Load Balanced vServers using the certificate and HSM key in the same manner as any other certificate on the NetScaler appliance. The certificate can be linked to CA certificates and used as any other certificate.