I’d like to introduce you to a fairly new tool to add to your Citrix CloudPlatform administrative toolbox.
Meet the doctor. CCPDoctor–or ‘The Doc’ as I like to refer to it–is a powerful tool that can highlight potential issues in your CloudPlatform environment.
The doc works on all major code lines (as of this writing; 3.0.7, 4.2.1, 4.3, 4.5 and soon to be released 4.7). It also comes baked in with some sweet features that make certain tedious administrative tasks a breeze.
I’ll look to cover a couple core features as well as dive into a really cool feature that should make all admin’s lives easier. It all starts by downloading the CCPDoctor utility, unpacking it, and making sure you have the latest by running update.
Here are the steps to get you there.
wget http://ccpdoctor.com/CCPDoctor-latest.zip unzip CCPDoctor-latest.zip cd CCPDoctor ./CCPDoctor.pl —update [you’ll need to accept the license agreement before proceeding]
Now that we have the latest version of The Doc up and running I’ll walk you through two of the more popular functions in the CCPDoctor utility. The first is used to diagnose potential problems with-in your CloudPlatform deployment. This is achieved by executing the following:
./CCPDoctor.pl —troubleshoot or use -t for short.
[root@ms-node]# ./CCPDoctor.pl -t CloudPlatform Doctor [1.12] [Read only] [06-Oct-2015 02:44AM PDT] Log file location: /var/log/cloudstack/management/CCPDoctor.log ========== Checking CloudPlatform management server version 4.2.1 ========== OK: No new CCPDoctor updates were found (Online version: 1.12 Local version: 1.12) OK: integration.api.port is null, unauthenticated admin access to CloudPlatform is disabled on this system ...snip... ERROR: The Java process config file has heap size defined as -Xmx2g, which is lower than the recommended 8Gb (about half the total memory) This will lead to slower responses and the environment will be unstable going forward when the resources in the cloud increase. To fix this, edit /etc/cloudstack/management/tomcat6.conf and change -Xmx2g to -Xmx8g, and also add the parameter -Xms8g After changing it, restart the management service for this to take effect ('# service cloudstack-management stop' then start). -OR- Run './CCPDoctor.pl --troubleshoot --fix memory.sh' or './CCPDoctor.pl --troubleshoot --fix all' to automatically fix this (remember to restart management service for changes to take effect)
The output here shows OK for test that have passed the muster, WARNING next to potential issues, and ERROR denoting known issues. With ERROR and WARNING cases, you’ll receive help text stating the issue, impact, and how to remediate. These issues can be fixed manually and some have an option where the doctor can automatically fix for you. Another cool aspect of the auto fix feature is that you can tell the doctor to fix a specific issue or have the doctor fix all the issues it has uncovered. Demonstrated below:
./CCPDoctor.pl --troubleshoot --fix memory.sh or ./CCPDoctor.pl --troubleshoot --fix all
Another common use for the doctor is to run the tuning check. Tuning will point out common performance improvements, you could take, achieved through various configuration changes.
./CCPDoctor.pl —tune …sample output... WARNING: There is data older than 90 days that could be cleaned up Cleaning this up will improve the overall performance of the system Older data can be cleaned up using the following SQL commands: …snip... Please make sure to take a backup of your database before applying - OR - Run "./CCPDoctor.pl --tune --fix old_usage.py" or "./CCPDoctor.pl --tune --fix all" to automatically fix this.
In the same fashion as troubleshooting you’ll see output denoted by OK and WARNING. Warnings are accompanied by helpful text that will guide you on the tuning changes suggested and how they are intended to affect the system. Similar to troubleshoot, you can have the doctor fix specific issues or all. There will be some issues the doctor can’t automatically fix as they might require access to a remote machine’s configuration file… in this case, instructions will help guide you to addressing the issue.
Now for my favorite feature, because really when you can automate any time consuming brain numbing tasks, it’s a win… and that’s the collect logs feature. Not only can the doctor grab relevant logs files, it also can add database dumps. The real magic occurs when including a case id, if there was an incident filed with support. CCPDoctor can automatically collect log files, upload, and attach them to your support case. To add more ‘win’ icing to the party, logs can be collected automatically from system vm’s. No more fumbling with passwords and hopping through hosts, ssh’ing and scp’ing logs to intermediate locations only to have you compress and upload them yourself… one by one. CCPDoctor does this with a few additional keystrokes.
Example using collect logs feature: ./CCPDoctor.pl --collectlogs or shorthand with ./CCPDoctor.pl -c
[root@ms-node]# ./CCPDoctor.pl --collectlogs CloudPlatform Doctor [1.12] Log file location: /var/log/cloudstack/management/CCPDoctor.log =============== Collecting MS Logs ================ OK: No new CCPDoctor updates were found (Online version: 1.12 Local version: 1.12) Please enter your Citrix username: XXXXXX Please enter your password: Trying to verify the credentials, please wait... OK: Username and password were accepted OK: Saved username and encrypted password in the config.ini OK: BugTool Collecting mode is [Normal] OK: Found CloudStack installation on this system Collecting data from the system subsystem (using df, top, netstat, route, etc) Collecting data from the database subsystem (using mysqldump) Collecting data from the logs subsystem (management-server.log, catalina.out, access_log, etc) Compressing /tmp/cloud-bugtool_2015.10.06-02.39.01.HwTk9q/ccp-42-ms . . . please wait (this may take a while). OK: Diagnostic file location: /tmp/cloud-bugtool_2015.10.06-02.39.01.HwTk9q.zip Cleaning up...
Here’s an example specifying a case number and a specific system VM to collect logs from. After the log collection completes, your log files will be uploaded and attached to the case:
./CCPDoctor.pl —collectlogs —case 12345 —collectsvm 25 CloudPlatform Doctor [1.12] Log file location: /var/log/cloudstack/management/CCPDoctor.log =============== Collecting MS logs and logs from SVM ID 25 ================ OK: No new CCPDoctor updates were found (Online version: 1.12 Local version: 1.12) Trying to verify the credentials, please wait... OK: Username and encrypted password from config.ini were verified Collecting cloud.out from instance 25 Copying file /var/log/cloud.log-20151002, saving as /tmp/svm_25/_var_log_cloud.log-20151002 …snip... Cleaning up... Initiating the upload, please wait... Uploading 21 MB now..........11%..........23%..........34%..........46%..........58%..........69%..........81%..........92%....... OK: Now finalizing the upload... OK: All done.
I recommend exploring these and other features of the CCPDoctor utility.
Have a look at the help text, accessed using the /help flag at the command line, to uncover other useful features. Remember, here at Citrix we’re always interested in hearing feedback about the doctor, good and bad, email us at email@example.com. Happy optimizing, and safe journey through the clouds!