The Engineering team for the Citrix AppDNA application migration software is in the midst of researching the upcoming Microsoft Windows Blue release, focusing on discovering the types of application compatibility issues that will occur in a typical enterprise application portfolio. R&D activities are numerous, but include things like manual testing of applications in our test library, digesting field information from Citrix partners, and writing sample applications which demonstrate an issue being researched. The goal of this process is to discover application compatibility issues and then design a pattern that can be applied to the universe of applications which identifies those impacted by the issue. I thought the compatibility curious or like-minded application gurus would be interested in a peek behind the scenes.

At its core, AppDNA extracts and stores the DNA of an application without having to install or run the app. This subject has been written about at length so I won’t go into it here, but for those of you new to AppDNA just remember that the building blocks of an application have been extracted and stored. In fact, the DNA stored by AppDNA is so extensive that the process and scope of DNA extraction rarely changes.

Now it’s time to see how these AppDNA algorithms come together to identify compatibility issues across thousands of applications. I thought it would be interesting to focus on an algorithm that was recently written for Windows 8 and Windows Server 2012.

Kernel mode drivers signed by an untrusted publisher

On 64-bit versions of Windows 8 and Server 2012, or Windows 8 with Secure Boot enabled, drivers that are not signed by a trusted certificate authority will fail to load. Depending on how an application uses the driver it could fail to start or certain aspects of the application could simply fail to work properly. To develop an algorithm that detects this issue there are a couple of key checks that need to be performed. First, identify the applications that install kernel mode drivers. Second, ensure that those drivers are signed and that the signature was generated by a trusted certificate authority.

Identifying Kernel mode drivers is something that AppDNA has been doing for years and we have several well tested heuristic algorithms for doing such identification. Sometimes the installer DNA identifies a driver as being Kernel mode, sometimes a corresponding INF does as well. As a last check AppDNA looks to see if a binary is calling Kernel mode driver API’s. The DNA for installers, INF’s and API calls from binaries are in our database, so performing this check is a simple SQL Query.

For each Kernel mode driver identified, we then check to see if it is signed and if so by what certificate authority. Certificate information for all signed binaries is also stored in the AppDNA database. At this point, if a driver is unsigned then the algorithm will report failure. If the driver is signed then AppDNA holds on to the signing certificate authority for later.

In addition to storing DNA about applications, the AppDNA database also stores operating system DNA that customers can load much in the same way they load applications. As a part of the operating system DNA, the AppDNA database contains information describing the certificate chain on the OS image. Using the signing certificate authority and the certificate chain from the OS the algorithm simply walks the certificate chain until it identifies the certificate as either trusted or untrusted. Based on that result the algorithms can accurately report success or failure as needed. AppDNA even allows customers to load multiple OS images in at the same time so if an enterprise manages multiple OS images this algorithm can be run against all of those images in a single pass.

So there you have it, an automated process for detecting kernel mode drivers signed by an untrusted publisher. Using AppDNA, this algorithm can run across thousands of applications and multiple OS images in just a few minutes. So maybe AppDNA algorithms aren’t magic once you get an understanding of how they work, very cool but not magic.