When transforming the desktops of an organization form a distributed virtual world into a centralized virtual environment, application compatibility has always been one of the key focus areas. While we have spoken a lot about the related processes and helpful 3rd party tools, I personally never found an exhaustive answer about a tiny little but very important detail about the memory mapping of 32bit applications on a 64bit Windows OS.
This missing piece of information can be put into a simple question:
Can 32bit applications access and make use of memory beyond the 4GB boundary on a 64bit OS?
I’ve asked this question fellow Citrites as well as colleagues from Microsoft and I got answers… lots of answers… and all different, ranging from “Yes” to “No” to “It depends”.
As the answer to that question has a significant impact of the scalability and/or the sizing of 64bit XenApp servers, I finally decided to just test it. So this blog will share my findings, but before going into the details we should take a little step back and briefly summarize what we already know.
In order to run a 32bit Windows application on top of a 64bit Windows OS we need to make use of the WoW64 (Windows 32-bit on Windows 64-bit) subsystem, which is included in all 64bit version of Windows. This subsystem has been build to take care of the architectural and structural differences between 32-bit Windows and 64-bit Windows.
In my experience the WOW64 subsystem works pretty flawless, but it comes at a cost which is missed during the planning phases of many projects. This cost is a –sometimes– significant memory overhead. CTP Helge Klein summarized the background of this overhead in a nice blog a while ago, so I just quote him on that:
“Whenever a process accesses a disk, the network, the registry, or any other system resource, the corresponding API call invariably ends in kernel mode. The x64 kernel, of course, expects 64-bit data structures (pointers and the like). 32-bit applications, on the other hand, use 32-bit data structures. Thus conversion (called thunking) is required, which is performed by the WoW DLLs mentioned above, but it comes at a price. Not so much performance – CPU overhead is negligible in most cases – but RAM. Converting 32-bit data to 64-bits essentially doubles the amount of RAM required to store the data. However, since only some data structures need to be converted and use of those structures varies greatly between applications, it is simply not possible to give a good estimate of how much more RAM is needed to run a 32-bit app on x64 than on x86. As often, it depends…”
From my experience the memory overhead increases with the complexity of an applications (more complex = more overhead). Typical multipliers for the memory consumption of single processes range from 1.2 – 2.5 based on the memory consumption seen on a 32bit Windows OS. But to determine the exact multiplier for your apps, you need to test.
So why would I move my 32bit applications to 64bit then, given this memory overhead? The answer is simple. It’s the 4GB boundary that normal 32bit applications cannot cross. This means that even there may be more than 4GB of memory available in your server the applications won’t make use of it, as they simply cannot access memory beyond the 32bit address space (there are exceptions, but I don’t go into that now). So if you try you likely end with something like this (click on image to enlarge):
(Sample over 4 days, where you can see users login on in the morning and leaving in the evening. Memory consumption never exceeds 4GB + a little bit for the OS, which can address more than 4GB)
(Different server but same environment. You can see that there is lots of physical memory available but the pagefile is used heavily)
Back to the initial question
Coming back to the initial question: Can 32bit applications access and make use of memory beyond the 4GB boundary on a 64bit OS? Can WOW64 help here?
To answer this question I’ve build a vanilla Windows 2008 R2 SP1, installed the latest hotfixes available and equipped the server with 14GB of memory (could not build a larger machine because of memory constraints on my XenServer).
To consume memory in order to prove if 32bit apps are able to cross the 4GB boundary by means of WOW64 or not I’ve used the freeware tool MemAlloc.
MemAlloc does not only consume memory on demand, but it also offers some build in reporting about memory statistics seen from a process point of view.
The first thing we notice is that the MemAlloc process sees a total 4GB of memory. This has been the case on “normal” 32bit Windows OS as well (check this for details). So no changes here.
Now let’s start a couple of 1024MB consuming processes (I’ve used two users to “mimic” a multiuser XenApp server).
Now 4 processes have been started, which consume 4GB. The end of 32bit.
Let’s start some more.
Now I have 8 processes started, which consume a total of 8GB. The pagefile (which has been set to 2GB static) is not used at all. So we can conclude the answer to the initial question is:
Yes, 32bit applications can access memory pages beyond the 4GB boundary, when running on a 64bit OS. WOW64 does it’s job.. J What we also see, is that every application is handled independently, so there is no competition in between the 32bit processes.
Now let’s max out the system…
The show ended with 15 processes started, whereof one was not able to allocate 1024MB, because the physical memory as well as the pagefile (set to 2GB) was maxed out.
So I got a lovely error message (meaningful this time):
If you’d like to learn more nice little things about Windows, Citrix products and protocols or storage and networking, register yourself for Synergy in Barcelona and visit the Desktop Transformation Track.