One of the most common issues we encounter in many Citrix environments is a lack of basic monitoring. Companies spend hundreds of thousands of dollars on software, networking, storage, and servers to build the virtual desktop and application environments then struggle to proactively support their systems because there is no effective monitoring solution in place.
Why is this? There are a couple of reasons that come to mind.
1. Budget
Management sometimes has a hard time spending money on monitoring solutions because they don’t think it affects the bottom line. Of course, an extended outage usually changes their minds. 🙂
2. Convoluted Monitoring Market
Many traditional monitoring toolsets don’t focus on Citrix monitoring itself. Sure they allow monitoring of basic server counters (CPU, memory,etc.) but gathering useful Citrix related statistics has always proved elusive. What should one monitor and what should the thresholds be?
Citrix’s answer was EdgeSight which has been around for some time for Platinum customers. However, it’s never been my favorite tool (sorry Citrix). It’s a “point product” (and a bit of a bear to get installed and working) and doesn’t really integrate with other monitoring toolsets.
Of course, other virtual desktop and application focused products now exist including LakeSide’s SysTrack Software, EG Innovations, and Liquidware Labs. Some of their advanced user experience metrics are great and are definitely worth a look, but we’ll table them for another blog entry.
In this blog article we want to keep it simple and focus on:(1) the most important things to be monitoring and, 2) a cheap mechanism for doing it.
Citrix Operations Guide – What to Monitor
Last year Citrix published an Operations Guide for availability and performance monitoring. In this guide, Citrix outlines all of the recommended counters, metrics, event IDs, services, ports etc. that they recommend should be monitored in a Citrix environment (XenApp, XenDesktop, XenServer, licensing services, Netscaler, etc.)
While this guide is great, we still need a mechanism and tool for monitoring in a meaningful fashion. Most of these values (on the Windows server side at least) can be obtained via Perfmon counters but capturing via Perform and playing back leaves a great deal to be desired for IT operations staff. So how do we do it?
Enter PRTG – How to Monitor
We’ve talked about PRTG before in our blogs for monitoring Citrix (Get Proactive! Effective Methods for Monitoring Atlantis Computing’s ILIO VDI Storage Platform). PRTG is a server/network monitoring tool from Paessler that is:
- Robust: provide alerts and does short and long term data trending, which we want.
- Extensible – while it provides numerous monitoring options “out-of-the-box”, PRTG provides the ability to write custom sensors (we use PoweShell for ours).
- Cheap – most companies can get away with a $5K or under investment, although an unlimited version is only around $10K).
Once we’ve built our sensors based on the Citrix Operations Guide (kudos to Lockstep’s Josh Sanders for putting the code together) and started monitoring our servers, we can have a nice overview of our complete Citrix environment along with custom sensors that match the recommended metrics, etc.
Citrix Overview – The Big Picture
PRTG allows to create multiple views of our Citrix environment. Above, we’ve created a simple view for monitor a sample Citrix environment with:
- XenApp Controllers – Dedicated XenApp XML Controllers and License servers.
- XenApp Worker Servers
- Web Interface Servers/StoreFront
- Provisioning Servers
- XenServer
- NetScaler VPX
Of course, green is good, yellow is a warning, and red is bad. In an alternate layout below, we use the “sunburst view” to show us the health of our Citrix environment based on component categories. This is a great way to drill down and see what’s going on.
Cool, huh? However, most of you are probably also interested in the individual sensors and what and how they’re gathering data. Take a look at some of examples below. This is not a complete list but provides a great example of what happens when we combine the Citrix Operations Guide, PowerShell and PRTG (we’ve also added some of our own custom monitors as well).
Custom PowerShell Sensor – Key XenApp Farm Performance Metrics
This sensor covers key farm metrics from the Operations Guide including Application Resolution Time, DataStore Connection Failures, busy XML threads, IMA work queues, License Server connections, failures, Terminal Server session counts, among others.
Short and long term trending options allow us to see this data over time:
Custom PowerShell Sensor – XenApp Server Performance Metrics
Traditional recommended Citrix system performance counters for individual XenApp Worker servers including processor time, processor queue length, memory stats, disk usage/response times/queue length, among others.
Custom PowerShell Sensor – Citrix Licensing Stats
Custom Citrix license server usage statistics for various Citrix products including XenServers, XenApp, Provisioning Server,etc. Determines how many licenses are available and the percentage in use.
Custom PowerShell Sensor – Citrix Web Interface Metrics
Number of ASP.Net requests queued and rejected.
Custom PowerShell Sensor – XenApp Load Metrics
Custom Citrix Farm and Server Load based on Citrix Load Evaluator policies.
And there’s more. But we’ll save them for subsequent blog posts…..
Do It Yourself!
Look like something that would be valuable to your environment? Cool, we’ll show you how.
In the next few weeks, we’ll be publishing subsequent articles that show you how to set this up yourself. We’ll share with you our PowerShell code along with the PRTG sensor code needed to set this up.
Either check back here or feel free to follow us on Twitter @lockstepgroup for our next article in this series.
More info on us: Lockstep Technology Group
[activecampaign form=10]
