I was recently on a panel in Stockholm moderated by Brian Madden and Alessandro Perilli where, with Mike Neil, Avi Kivity, and Lance Berc, I enjoyed an engaging discussion on the future of virtualization.  The event was a great opportunity for the audience to dig into different products and vendor strategies and Brian and Alessandro allowed none of us to wriggle away from the real issues.  Definitely the best panel I’ve been on – and all due to the two moderators. 

Brian is one of a rare breed that I am seldom lucky enough to meet.   He’s about 20 (probably permanently), super smart, and incredibly engaging and friendly. He dresses cool and trendy.  He is an excellent presenter and author. He also knows more about Citrix products than many people at Citrix, me included.  He has found and filled a key niche in the Citrix ecosystem for no-bullshit, hands-on, been-there-and-done-it technology and product expertise that has won him the respect of Citrix and its customers. As an independent advocate to our customer base he has developed a reputation for demanding answers, and getting them. 

So Brian’s latest piece predicting the end of Xen should be seen in precisely that light – he’s beating the hive with a stick to see whether bees will come buzzing out.   His logic is a little blunt – hence the stick analogy – but I value tremendously his approach  because it gives me an opportunity to clearly state the Citrix virtualization strategy, so that customers understand why an investment in Xen and XenServer is sound.  I guess it also drives clicks to brianmadden.com, which can’t be a bad thing, eh Brian?

Brian predicts the end of Xen based on the following claims:

  1. Citrix XenServer has “literally zero percent market share”
  2. VMware has dominant presence in enterprises,
  3. Microsoft Hyper-V will take over SME virtualization, so Citrix will abandon XenServer
  4. The open source community will abandon Xen in favor of KVM

His argument is superficially appealing, but unfortunately completely wrong.  That said, I’m grateful that he has highlighted the need for Citrix to clarify its virtualization strategy, and I hope that this long, but rather rapidly prepared response goes some way towards doing so. 

1. XenServer has zero market share:  Setting aside Brian’s challenging hyperbole, the XenServer business has been approximately doubling quarter on quarter since its first release – “approximately” only because the transition from XenSource to Citrix meant a new CRM system, new support system, new channel management, wrapping our heads around Citrix licensing and a ton of other overhead that took us a while to figure out, but which will accelerate our business in the back half of 2008. We have somewhere approaching 4,000 enterprise customers, and about 3000 trained channel partners.  (Watch for the inevitable follow up from Mike D at VMware who is still out to prove that he finished high school math).  VMware claims 100,000 customers.  Citrix has about 220,000 customers and about a hundred million users.  The XenServer market share is small, and growing as rapidly as any such product can given the current VMware brand status, and the fact that we started well behind them.  We had a few key blockers for enterprise adoption, four fifths of which are addressed in our forthcoming XenServer 4.2 release.

A recent analyst report that I saw gave us about 4% market share (by customers), which would seem a tad generous, but not far off.   But XenServer is just starting to show up in the market. HP and Dell have just started shipping XenServer embedded into their server hardware.  And with HP stating that XenServer HP Select Edition is HP’s preferred embedded virtualization platform there must be something in it.  Yes, this is embedded virtualization done right - In each of our announced OEM deals, our partner gets to offer their customer the tremendous value of  a powerful virtual infrastructure component “built in”, while adding their own differentiated value propositions on top.  And uniquely we charge one fixed price per server, not per core or socket.  We are proud to be a completely hidden component of a ProLiant server that is managed using HP’s powerful multi-hypervisor VMM 3.5 (also manages VMware and Microsoft), integrated with HP Lights Out management and Systems Insight Manager, and with a compelling graphical ProLiant Virtual Console, whose UI is like my Mac.  After all, a key HP value prop to their customers is out of the box manageability. XenServer is also available integrated into Dell PowerEdge servers and soon with Dell OpenManage, and it is embedded into the NEC Eco Center and other x86 servers, x86 servers from Lenovo, and Egenera PAN Manager for Egenera servers, which is also resold as Dell PAN Manager. Each offers a powerful value-added, XenServer-powered customer experience that is simply not available with VMware or other products. And then there’s Marathon EverRunVM and EverRun for XenServer - both of which offer five nines of availability for VMs. (Can others do this? No). Stratus Avance-- a perfect mid market appliance for business continuity on a pair of industry standard servers.  XenServer inside.  Avance is also sold by Dell (gosh, that’s right: XenServer is inside three key Dell enterprise virtualization offerings). (Can other vendors do this? No).  Finally, most recently and strategically profoundly importantly, Symantec’s Veritas Virtual Infrastructure (VVI) - an incredibly rich software offering from a major enterprise IT vendor that combines the incredible depth of the Veritas storage management suite with the powerful, open storage repository model of  XenServer to create a high-end enterprise offering that application level availability, and performance and management integrated with the industry’s leading storage management solution stack. (Can the competition do this? No). 

The army of enterprise class competitors to VMware is growing.  XenServer is inside and it is compatible with Hyper-V. And it’s not going away.  It’s just that we allow our partners to make money from virtualization, and we don’t demand to be the sole value proposition.  What a novel idea!

2. VMware owns the Enterprise: Ignoring Brian’s second hyperbole (we all know that VMware has signed up many large enterprises to their ELA), remember that only 10-15% of servers are virtualized, and there is a long way to go yet.  Evidence of imminent change in this regard came a week ago on an ACM sponsored panel with the director of IT for a major F50 who said “We have more Xen in production than VMware.  An open architecture gives us greater scalability and control, and we cannot afford to rely on a single vendor.”  There you have it – VMware, for all its success, has inherent weaknesses – the largest of which is its one size fits all, single vendor sells all mentality.  Its arrogance with customers and the channel, and the paucity of rich value-added ISV offerings around it all demonstrate their vulnerability.  It is VMware that will be the Netscape of Virtualization, and not Xen or Citrix.  We have never “wanted it all” and by having an approach that is founded on the notion of a rich, competitive choice of value-added offerings, we explicitly acknowledge that we will partner in key areas, and add value in others.  Citrix is after all a partner-centric vendor.

Our largest partner is of course Microsoft.  And we share with Microsoft and Hyper-V an entirely different view of virtualized infrastructure, one which is embodied in XenServer and Hyper-V: A virtualized infrastructure founded on fast, free, compatible and ubiquitously available hypervisors and a rich ecosystem of value added partners that address all customer needs. So Brian’s statement is simply a statement of current market status.  Sounds big and bold, but nothing really new.

3. Microsoft will take the rest of the market:    It is important to state yet again that we are not in a competition for server sockets with Microsoft.   If that were the case, why would we have helped Microsoft to make Hyper-V a better hypervisor, by developing the shims and drivers that will allow Linux to run with optimal performance on Hyper-V?   The founding thesis of XenSource, and the continued strategy at Citrix, is to promote fast, free, compatible and ubiquitous hypervisor based virtualization.  If the hypervisor is free, why worry about who delivers it?  Let the customer pick the implementation method that they want – the real money is in the up-sell with products that make virtualization valuable for customers. 

I personally view Hyper-V as one of the most compatible implementations of the Xen architecture out there, and we guarantee that XenServer and Hyper-V are 100% compatible at the VM level.  So why not drop Xen and go with Hyper-V?  Xen is ahead of Hyper-V in scalability, cross platform support and crucially, support from a rich ecosystem of vendors all of whom stand to benefit from commoditizing the hypervisor, and continuing to make Xen great.   Xen and XenServer offer the industry a ring through the nose of a bull - called Microsoft.  We have a rope through that ring and we tug it from time to time to ensure that the bull goes the right way.  But when the bull charges (as it will now with the RTM of Hyper-V) we will step nimbly aside and watch the effect on our friends at VMware.

Who will use Hyper-V?  Over time, a lot of customers, starting in SME.  Will Hyper-V eclipse XenServer’s footprint? Again, yes, over time and often for different use cases.  Microsoft still needs to extend the architecture to deal with live relo, offer better resource pooling than simply re-using Microsoft Cluster Server, and figure out how to deal with storage, but they will do a good job over the next few years. Do I feel threatened by that?  No.  Every implementation of the fast, free, compatible, ubiquitous hypervisor architecture offers us an opportunity to up-sell the customer with rich value-added features. Remember – we’re the guys who made the hypervisor free!  Not for nothing have we been beating Microsoft about the ears to get the darn thing to market – it will offer us a terrific opportunity for up-sell.  With what? 

  • XenServer Platinum today is already hypervisor agnostic. It can instantly boot and run a VM on Xen, Hyper-V, VMware and even bare metal, with a streamed VHD that offers better performance than local disk.  It even allows us to boot a thousand VMs from a single VHD, which is key to scalability for XenDesktop. 
  • Oh, and then there’s XenDesktop, our powerful VDI offering that presumes the availability of fast, free, compatible, ubiquitous hypervisors…  I see I’m starting to repeat myself.  XenDesktop is a key area of focus for Citrix, and the inclusion of XenServer as a technology feature is the right way to go about delivering desktops as a service.  No need to be a virtualization administrator – delivering desktops is hard enough already.   Doing it VMware style you have fights between the Desktop guy and the Virtual Center administrator about why a user’s desktop disappeared, and no way to figure out why it all went wrong.
  • XenApp (Presentation Server) on XenServer already offers a factor of five better performance than on “a leading virtualization vendor’s product” (they’d sue me if I used their name, and it doesn’t begin with M).
  • And yes, there’s more coming we haven’t spoken about yet.

Bottom line on this point: XenServer today and in the future offers a powerful, Hyper-V compatible enterprise virtual infrastructure that is simply a component of many of the industry’s most powerful virtualization offerings, including our own XenDesktop and soon XenApp.  XenServer at the platinum level today extends Hyper-V with powerful value-added features to address some of its key enterprise scalability needs, particularly in dynamic management of storage.  And it will soon be able to be managed by System Center VMM (or any other DMTF equipped management tool).

4. The open source community will abandon Xen for KVM:  Brian, please don’t take offence, but I suspect that for many Microsoft MVPs, the workings of the open source community may be a little bit of a mystery. First, there is no such thing as “the (singular) open source community”.  [The Xen community] is independent of any vendor, large, growing and vibrant. It is quite different from the [Linux kernel community], though we share a few developers.   Xen is a cross platform, type 1 hypervisor, OS neutral, that runs on ARM based PDAs, x86, PPC and SGI Itanium super computers.  Xen is in BIOS offerings from major vendors, available at point of sale on just about every x86 server, and embedded in network and storage chipsets and appliances. It has tremendous performance and leads in IOV hardware support and performance.  Xen is available from the major Linux Distros, but also in Sun xVM and Oracle VM in their virtualization offerings, and they are very committed to their products.  Xen is or soon will be available embedded in every x86 server.  Xen is used as a standard test on every AMD and Intel CPU, and is the foundation of the planet’s largest deployment of virtualization, at Amazon.  Xen is present in or runs every major cloud, including Google (who recently began to host the open source project to develop a Xen equivalent of VM Safe).  And Xen is quietly emerging as an embedded hypervisor on laptops and PCs. At the Xen Summit last week, we had representatives from 12 countries, 14 universities, and 47 vendors – totaling about 175 core developers.  The research and work in progress indicated that the Xen project is stronger than ever, and far deeper and more powerful than “virtualization in Linux” – a role that I’m sure KVM will evolve to fill very well.   The Xen ecosystem is participating in the benefits of secure class 1 hypervisor based virtualization, while still being able to take advantage of the incredible innovation in Linux (which some vendors use as a parent partition / driver domain).  And of course in spite of their marketing fluff, the Red Hat engineering team continues to be a significant contributor to Xen (and we love them for it)  and Red hat has a seat on the project’s advisory board. 

KVM is interesting, but rather late to the party – I think the world has already decided that hypervisor based virtualization is the way to go.  That said, KVM is arguably more convenient for a Linux distro – because they don’t have to get their hypervisor from xen.org and their kernel from kernel.org, and combine them.  I like KVM a lot.  Great for Linux based developers too. But there’s nothing really new in KVM – the technology has been in products such as Microsoft Virtual PC/Virtual Server, Parallels and so on for quite a while.  It’s just a VT/AMDV driver added to Linux to allow it to host additional VMs.  Great if your usage model is “first install Linux, then use your Linux skills to install VMs”.  None of our customers want that, though I’m sure the adopters of Xen in SLES 10 or RHEL 5 are comfortable with the model.  Unfortunately the KVM project isn’t yet addressing any of the other key requirements for virtual infrastructure (virtualization-aware shared storage, snapshotting, cloning, thin provisioning, HA, and much more) it is just another way to do CPU and memory virtualization … at a time when Xen already offers Linux a typical overhead of under 1% (SPECJBB), and a rich set of value-added features.  To be honest, I’d love to see the two projects working together to have a single extension code base to both core virtualization technologies, but the kernel.org folks are solely Linux focussed (good for them) whereas we tend to look at all OSes with a requirement for OS indepencence.  I also have a strong suspicion too that by the time the KVM folks are done, they will have discovered that they have to re-design many of the core Linux resource management algorithms to take account of VMs, so it’s going to be a long and slow road. (There was a new performance comparison of Xen vs KVM at the Summit, which couldn’t complete because the KVM test kept crashing, but I’ve seen earlier work that indicates that KVM performs very respectably).  Can the two projects co-exist? Of  course.   And Linux now contains the key paravirt_ops API for secure dynamic binding to a class 1 hypervisor – Xen, Hyper-V (we’ve built the shim) and of course VMware, who collaborated on the API development.  Ultimately the open source community will have the benefit of both approaches – type 1 and type 2. 

There is one intriguing aspect of KVM, namely its likely adoption by one or more Linux Distros. I’m looking forward to hearing how those vendors explain the value proposition of a product that contains two incompatible virtualization technologies, and no management framework other than a thin veneer of an API that is incompatible with the ([DMTF]) accepted industry standard for management of virtualization. [Added note, with thanks to follow-up poster: I forgot to mention that there is a project under way to provide the relevant DMTF profile support, but it’s a long way off, and there’s rumored to be an effort to address the VM incompatibility issue.  It would be a great step forward for open source if these projects would accelerate, and it would be a great way to accelerate the adoption of fast, free, compatible, ubiquitous virtualization by the Linux distros.]


  1. XenServer is growing strongly on its own, and XenServer is inside the industry’s most differentiated enterprise virtual infrastructure products from leading vendors.  It offers the greatest price performance, compatibility with Hyper-V, and a rich ecosystem of customer-focused value props for management, availability, dynamism and flexibility.  Get it in your next server, completely built in.
  2. VMware has a strong lead, customer confidence, significant presence, and expensive, over priced products that demand customers buy into a single vendor, proprietary, closed, monolithic architecture that has significant architectural and scalability drawbacks.  Next time your VMware sales guy calls, ask him for some of the free stuff, or better, just download XenServer for free.
  3. Microsoft Hyper-V is a fine implementation of the Xen architecture, compatible with XenServer.  It has a strong development plan.   XenServer today and in the future will extend Hyper-V to address advanced use cases, and to support specific Citrix/Microsoft product partnerships.  A good example is XenDesktop.  There will be others.  System Center VMM is a great product, and will be able to manage XenServer as well as Hyper-V and VMware.
  4. The Xen community is vibrant and growing.   Xen is widely used, massively deployed, core to the product and corporate strategies of some of the industry’s largest vendors.  Predicting its demise would be extremely naïve.  KVM is very “in”:  interesting, incompatible and incomplete.  The two projects can co-exist independently and happily. 

Now, back to my day job!  And thanks again Brian for banging on the bee hive.  Beer on me.