Where Does Virtualization End and Cloud Begin

There’s much heated debate over cloud computing. The fact is that cloud computing is a paradigm shift in how we deliver compute capacity to end users. Virtualization, on the other hand, while enabling many of the conveniences of cloud computing, is not required to actually build a cloud. Virtualization, also, is an actual technology that creates a paradigm shift, not just a label for a paradigm shift, itself, like cloud computing is. In other words, there is no product that can “cloud you”, while there are many products that can “virtualize your business.” That said, most well-designed clouds include virtualization technologies.

The virtualization technologies that help enable cloud computing paradigms include virtualized networks, virtualized storage, virtualized name spaces, IP addresses, and clustered resources; and probably the mostly referred to, virtualized server and desktop machines. I’ll assume, at this point, if you’re reading this, you’re familiar with virtualization as it generally pertains to servers and desktops.

The question is, where does virtualization end and cloud begin?

To answer that, we have to establish some generally agreed assertions about cloud computing. In general, a modern cloud infrastructure will provide at least:

  • Self-Service – A self-service user portal to deploy and destroy compute instances
  • Automation and Elasticity – An engine that automatically handles the deployment request, notifies the user when the resource is ready, and terminates the resource when the user indicates it is no longer needed
  • Metering and Reporting – An administrative tool that meters the consumption of resources in the cloud as well as the ability to report on that usage, these two may be separate or an integrated solution
  • Billing & Charge-back (optional) – Depending on whether the cloud is designed for monetary gain or for organizationally charged back resources, the cloud may have the ability to do automated billing and/or automated charge-back

While there may or may not be dozens or hundreds of additional attributes to any given cloud infrastructure, I think that most cloud architects would agree that the above list is relatively inclusive of the basic components of an Infrastructure as a Service (IaaS) cloud solution.

In general, I have seen two marketectural (Marketing and Architecture) approaches to the answer to our question.

The first approach seems to treat virtualization as a very traditional component of cloud computing. The virtualization component of the cloud simply virtualizes servers and desktops and provides access to storage and networking for them. This approach does not include any aspect of user self-service, nor does it include a metering system and only has the rudimentary monitoring capability that has always been included in the solution.

This first approach seems to be most common in the industry. To complete the base cloud infrastructure, a company or service provider must purchase additional add-ons, which are often product-specific, and appear designed to lock you in to a single virtualization infrastructure.

The second approach is one that expands the architectural capabilities of the virtualization component and treats the cloud component of an IaaS solution as a higher level of capability and abstraction. This is the approach that Red Hat has taken with Red Hat Enterprise Virtualization (RHEV) 3.0 and its announced, upcoming, CloudForms solution.

Red Hat sees the base components of an IaaS cloud as key enablers at the virtualization layer. So RHEV 3.0 includes a power user portal (self-service & elasticity) giving users the ability to:

  • View assigned servers and desktops
  • Create/Edit/Delete VMs
  • Run VM with all options (including attach CD, etc)
  • Create/Edit/Delete/preview Snapshot
  • Create/Edit/Delete Templates
  • View VM statistics and status
  • View resource usage and statistics
    • Including network, storage, CPU and memory
  • Access the VM Console

RHEV 3.0 also includes an Enterprise Edition of Jasper Reports to provide advanced metering and reporting capability.

The only base component of a cloud architecture not included in RHEV 3.0 is the billing and charge-back capability, as it is not necessary in all solutions (optional). Generally, when billing and/or charge-back are required, they are implemented at a higher level, where they can include additional aspects and components of the cloud in the calculation of the resources consumed. This could include software licenses, physical disk usage, network usage, etc., that may or may not be consumed directly through the virtualized environment. Integrating the billing component into the virtualization layer would preclude a successful, extensible and inclusive billing and charge-back solution. For companies that require this capability, Red Hat has partners with solutions like Tivoli’s Usage and Accounting Manager (TUAM).

If Red Hat is offering an entire IaaS cloud solution, at no additional charge, as components of its RHEV 3.0 offering, then where does CloudForms come in?

Red Hat views cloud computing as more than just virtualizing your infrastructure. Clouds should provide abstraction from both the hardware and the virtualization themselves. A cloud infrastructure should give the enterprise or service provider the capability to use any sort of back end infrastructure they choose, physical or virtual, and provide automated, rules-based, dynamic access to those resources. That is the intent of Red Hat’s future CloudForms offering.

CloudForms employs a concept of Deployables which can define entire workloads, including database servers, app servers, web servers, all in a single package that can be deployed from a single end-user interface. Whereas RHEV 3.0 is going to deliver an entire IaaS cloud solution at one of the lowest costs in the industry, CloudForms will enable things like the automated deployment of en entire Platform as a Service (PaaS) solution with just a few clicks.

In general, virtualization will end where your virtualization solution provider decides it will. Most virtualization solutions provide only the basics, and call the rest (self-service, metering, reporting, etc.) “cloud” at significant additional cost. Red Hat’s virtualization solution provides everything an enterprise or service provider needs to stand up an IaaS offering at one low cost. Future offerings like CloudForms will serve to further enhance RHEV 3.0′s capabilities and ensure that customers have choice and flexibility in their cloud architectures, not just one solution that can’t fit all.

Virtual Machine Disk Performance & Scalability Considerations

“With additional engineering, Red Hat has eliminated the shared file system and the virtual machine storage file requirements, and returned the performance and scalability of raw disk to the virtualized, enterprise datacenter.”

Most virtualization professionals are familiar with VMware’s .VMDK and Microsoft’s .VHD files. These are the default virtual machine storage containers for each of their respective virtualization solutions. Using files to store virtual disks provides some ease-of-use benefits, for example ease of manually moving an entire virtual machine from one location to another. For desktop, or workstation-based virtualization, there is rarely raw disk available for virtual machine disk storage. In the case of workstation-based virtualization, the ability to store the entire virtual machine’s disk within an existing file system is a significant benefit. For enterprise virtualization, though, are virtual disk files really the best way to virtualize storage?

In general, most virtualization professionals agree that enterprise storage, virtualized or not, should have, among other attributes, excellent performance and scalability. This brings into question the use of files as storage containers and shared file systems. Neither of these are high performance or scalable. It has been observed over the last several years that workloads like database, ERP, CRM, etc., often do not perform well in virtualized environments that employ these default methods.

Let’s first discuss why these approaches don’t scale or perform well. Then, let’s talk about Red Hat Enterprise Virtualization’s unique approach to virtual machine storage management, and why that approach makes it the ideal platform on which to virtualize high I/O, transactional and large-scale workloads – like databases, ERP, CRM and e-mail applications.

Enterprise datacenter storage has traditionally been presented as raw disk to a physical system. This provides for the greatest level of performance and scalability. Using files as storage containers isn’t inherently bad, but files don’t perform well under high I/O. VMware introduced files as storage to the enterprise datacenter, and administrators accepted them since they weren’t presented with any apparent choice. But files require a file system. Adding an additional layer of file system to the enterprise datacenter adds cost of administration and overhead – both administrative and system performance overhead. Another file system also requires additional file system expertise, troubleshooting tools, data recovery strategies, etc.

Both VMware and Microsoft have chosen virtualization strategies that require a shared file system – VMFS and CSV, respectively. We’ve already pointed out why files aren’t necessarily the best way to present virtual machine storage, now we’ve added a shared file system to the implementation. The purpose of the shared file system is to allow the simplicity of implementation for VMware’s vMotion and Microsoft’s LiveMigration technologies – the ability to move a running virtual instance from one virtualization host to another without downtime or disruption. There is no question that this technology is tremendously valuable in the enterprise datacenter.

By adding a share file system, not only does the virtualized datacenter incur the overhead of storing raw disk data in a file, as opposed to on a disk itself, but now the issue of file system locking and contention is introduced. When multiple servers share a file system, there must be a locking mechanism in place to arbitrate respective access to the file system from each server in the pool, or resource cluster, for that shared file system. There are methods of optimizing this sort of scenario, but locking cannot be avoided at some level within the shared file system. Once there are enough virtualization hosts all sharing the same file system, the contention for locking the file system for access becomes too great and the system performance becomes unacceptable.

From a scalability perspective, the larger a flat virtual machine storage file gets, the slower the overall access to it usually becomes. As discussed above, the more virtualized hosts placing virtual machine storage files on a shared file system, the greater the possibility is that there will be contention for accessing the resource. VMware’s published limits of virtualization hosts in a single cluster of resources is 32, but generally accepted best practice is never more than 8 to 12 hosts in a highly utilized DRS cluster. This is because they are all sharing a file system and must be able to lock the system for access. Microsoft’s published limit is 16 virtualized hosts, and best practices are likely even lower.

Do the technology benefits, like vMotion and LiveMigration, really require a shared file system? Do they really require that virtual machines utilize files (VMDK or VHD) as their storage containers? Or, was this approach chosen simply because it was the easiest to develop and deliver to market? With additional engineering, Red Hat has eliminated the shared file system and the virtual machine storage file requirements, and returned the performance and scalability of raw disk to the virtualized, enterprise datacenter.

RHEV Manager’s approach to storage management is far more advanced than the shared file system approach. This substantial architectural difference means that RHEV does not suffer from shared file system locking contention and makes it the ideal platform for I/O-intensive applications.

Additionally, not limiting  the virtualization management with a shared file system removes the restrictions on resource cluster scalability. As virtualized, enterprise datacenters grow, it’s important to be able to share resource pools with as many virtualized hosts as possible, not limit them to small, independently and separately managed pools of resource. Cloud computing is a great example use case for large scalability of resource pools, also drawing into question VMware’s viability as a cloud computing platform, with its dependence on the VMFS shared file system.

Red Hat Enterprise Virtualization uses direct mapping of virtual machines to a respective logical volume on shared storage. So, a running virtual instance in a RHEV environment is not confined by a file, but is stored on block-level disk. This provides each virtual instance in a RHEV environment with the I/O performance traditionally attributed to bare metal computing. The best part is that RHEV accomplishes this and still maintains feature offerings like LiveMigration. Yes, you can still migrate a running virtual machine instance in a RHEV environment from one KVM host to another without any downtime and without the requirements of a file for a storage container or a shared file system.

The approach also allows RHEV Manager to scale the number of KVM hosts in its resource clusters to orders of magnitude more than a VMware or Microsoft-based solution.

If you’ve unsuccessfully attempted to virtualize I/O-intensive workloads before, or if you have simply dismissed it as not feasible, consider Red Hat Enterprise Virtualization. Also keep RHEV in mind when you’re building out a large-scale solution that requires large pools of shared resources. When it comes to performance and scalability, RHEV is unmatched in the virtualization industry.