I’ve decided to start blogging some of the answers I give to customers in the field—apparently some people find this information enlightening!
So, if you’re a NetScaler admin or architect, read on. Below I will discuss using the SYS verb in an AppExpert expression. This allows you say “yes” to some of the more bizarre queries that come in from your business, like, “We want to close down our website for the weekend, from 12 midnight Friday to 6am Monday!”
At antisocial times like this, you could log into your network, login to the NetScaler, and make a change. And be tired. Or … you could create a policy-based on time, enjoy your Friday evening and snooze through that 6am wakeup, confident that NetScaler AppExpert is doing all the work for you. Work smarter, not harder (it’s the Citrix way, you know)!
AppExpert is not a feature on NetScaler, it’s how NetScaler works. It’s the chassis upon which the features rest and how they are invoked. Other appliances might use some scripting language to configure the features – perhaps bringing the evaluation and the action parts into one piece.
By separating these two parts, tasks are simplified and a Network admin can configure complex layer 7 procedures with little to no scripting knowledge required. There is no steep learning curve.
Usually – your customer or business will come to you with a request: “We want X to happen when Y happens.” You need to break the customer requirement into two things:
- What do you want to do?
- When do you want to do it?
The WHAT is the feature you use on NetScaler. The WHEN is the AppExpert expression. Combine the WHAT and WHEN – and you get a NetScaler Policy – which is how most of the features on the appliance are configured. For example – a customer says they want to limit the number of times a user can hit a certain page, e.g. a reports page that takes a lot of back end DB resources to build. Their request is incomplete. To fulfill the requirements you need to ask the customer what they want to have happen when the limit is reached. The answer will give you all the information required to build your policy.
The answer could be “Oh – just drop the connection”, or “delete their session cookie so the app treats them like a brand new user.” Great – that means you can use Rewrite. Or they might say “send back an error page telling them they cannot request the report so many times in succession.” Got it – that means Responder.
Once you have the ‘What’, you build the action. Many of the actions are pre-built for you – for example the drop, or reset, delete. Lets build out this example. To delete a cookie, the action is simple:
add rewrite action Remove-Myapp-Cookie delete “HTTP.REQ.COOKIE.NAME_VALUE(\”sessionid\”)”
Now we look at the ‘When’. This is where AppExpert comes into it’s own. We just need to convert the English expression into a NetScaler AppExpert expression. Where rate limits are involved, you create the limit identifier – which is the number of times an event is allowed in a specific time window. I didnt want to get into Rate Limiting here.. but I’d really like to provide something “copy and pasteable” for those who want to try this stuff out – you know who you are! 🙂
The extra backslashes you see below in the config examples are only inserted for the CLI – you don’t need them when using the GUI. First, we set our selector – i.e. the URL being accessed, and the IP address of the client hitting it. If we don’t specify the client IP – then the 4th person to try to login within a 60 second window would get blocked. We could use an application session cookie instead for more accuracy – but lets keep it simple for now! Skip on past the Swiss Army knife if you only want to find out more about SYS.
add stream selector protect-login-page “HTTP.REQ.URL.STARTSWITH(\”/application/login\”)” CLIENT.IP.SRC
add ns limitIdentifier ThreeAttemptsOnly -threshold 3 -timeSlice 60000 -selectorName protect-login-page
The timeSlice is in Milliseconds – so 60,000 = 1 minute. The threshold is the number of allowed occurrences – so that’s 3. Therefore you use the following for your rewrite expression – using SYS.Check_Limit() as your verb:
add rewrite policy Prevent-More-Than-Three “SYS.Check_Limit(\”ThreeAttemptsOnly\”)” Remove-Myapp-Cookie
The remaining piece is to bind said Rewrite policy to the Request flow of a HTTP or HTTPS VServer. That bit you’ll need to do on your own.
Now that the scene is set, lets take a look at some other things I can do with SYS to decide when I want an action to occur. SYS is probably one of the most powerful of the AppExpert verbs – so lets run through some of the things you can do with this.
With “SYS.TIME” I can build expressions which will enforce an action to happen at a certain time, or between a specific time window. For example, return a “Sorry – Gone Fishin’ ” web page from the NetScaler when back end servers are taken down for maintenance, or during your application or database upgrade where you don’t want any app data to pollute your database. Does the NetScaler admin have to be present?
Our previous example used SYS.CHECK_LIMIT – which means you can invoke an action when a limit has been reached. This is extremely powerful in creating security policies which can be used for ANY protocol supported on the appliance. e.g. where NetScaler sees DNS resolution queries, it can decide to not responds or respond with a different IP upon seeing a limit being hit. It can stop brute force login attempts or anything like that to protect your back servers from being overwhelmed. The time window for Rate Limiting is milliseconds, so very granular and small windows can be defined.
SYS.HTTP_CALLOUT – another power security feature. You can pause ANY type of request, pull the Layer7 information from it, and send this over HTTP (or HTTPS) to a 3rd party, parse the response from the third party, and form your action based on the response. This can be used for security (blocking users etc.) or to access some application statistic about that that user to decide their experience from the NetScaler’s perspective. If they are considered a bad guy – they can be content switched to a honeypot server for example.
- Vserver Metrics:
Now – onto my favourite – SYS.VServer. This allows me to call an action based on some statistic from a VServer. For example, with SYS.Vserver(“bob”).HEALTH I can get the health of the VServer ( The number of “Up” services as a percentage”,) and decide an action if the percentage is greater or less than whatever number I want.
The action could be a responder policy with “no Action” specified, but with a custom log action which would trigger an Emergency SYSLOG action monitored by your Operations team. Less than 50% – time to call that On-Call number. And not the NetScaler team – the operations team as it’s their servers that are down.
We can also return the state of a VServer with SYS.VSERVER(“bob”).STATE. This does not have to be the VServer to which the policy is bound. I can use the state of ANOTHER Vserver to decide my action! For example, if I have a Web App with a DB back end, I can create a VServer to the DB.
Now, I check the health of the database by using the built in monitoring where I can send a SQL command to the Database server, parse the response, and based upon what I receive back, determine the health of the database. If there is something I don’t like, the Database Service goes down, the DB Vserver goes down, and suddenly NetScaler will start to serve back HTTP responses to users hitting the site with a message like “Sorry – due to something fishy with the Database Server, we’re pausing access to the web application – please have patience while we look into this.. yadda yadda.”
Do you need to learn any of the syntax for the expressions? No. That’s the magic part about all of this. The AppExpert expression builder will allow you to create (with examples) expressions, and even testdrive them in the evaluator before you apply them, as shown above.
If you have any cool expressions you’ve used, why not share them on the Citrix User Groups Community forums (CUGC ) – there’s nothing like examples from the wild to get the imagination going!