Virtual Machine Disk Performance & Scalability Considerations

“With additional engineering, Red Hat has eliminated the shared file system and the virtual machine storage file requirements, and returned the performance and scalability of raw disk to the virtualized, enterprise datacenter.”

Most virtualization professionals are familiar with VMware’s .VMDK and Microsoft’s .VHD files. These are the default virtual machine storage containers for each of their respective virtualization solutions. Using files to store virtual disks provides some ease-of-use benefits, for example ease of manually moving an entire virtual machine from one location to another. For desktop, or workstation-based virtualization, there is rarely raw disk available for virtual machine disk storage. In the case of workstation-based virtualization, the ability to store the entire virtual machine’s disk within an existing file system is a significant benefit. For enterprise virtualization, though, are virtual disk files really the best way to virtualize storage?

In general, most virtualization professionals agree that enterprise storage, virtualized or not, should have, among other attributes, excellent performance and scalability. This brings into question the use of files as storage containers and shared file systems. Neither of these are high performance or scalable. It has been observed over the last several years that workloads like database, ERP, CRM, etc., often do not perform well in virtualized environments that employ these default methods.

Let’s first discuss why these approaches don’t scale or perform well. Then, let’s talk about Red Hat Enterprise Virtualization’s unique approach to virtual machine storage management, and why that approach makes it the ideal platform on which to virtualize high I/O, transactional and large-scale workloads – like databases, ERP, CRM and e-mail applications.

Enterprise datacenter storage has traditionally been presented as raw disk to a physical system. This provides for the greatest level of performance and scalability. Using files as storage containers isn’t inherently bad, but files don’t perform well under high I/O. VMware introduced files as storage to the enterprise datacenter, and administrators accepted them since they weren’t presented with any apparent choice. But files require a file system. Adding an additional layer of file system to the enterprise datacenter adds cost of administration and overhead – both administrative and system performance overhead. Another file system also requires additional file system expertise, troubleshooting tools, data recovery strategies, etc.

Both VMware and Microsoft have chosen virtualization strategies that require a shared file system – VMFS and CSV, respectively. We’ve already pointed out why files aren’t necessarily the best way to present virtual machine storage, now we’ve added a shared file system to the implementation. The purpose of the shared file system is to allow the simplicity of implementation for VMware’s vMotion and Microsoft’s LiveMigration technologies – the ability to move a running virtual instance from one virtualization host to another without downtime or disruption. There is no question that this technology is tremendously valuable in the enterprise datacenter.

By adding a share file system, not only does the virtualized datacenter incur the overhead of storing raw disk data in a file, as opposed to on a disk itself, but now the issue of file system locking and contention is introduced. When multiple servers share a file system, there must be a locking mechanism in place to arbitrate respective access to the file system from each server in the pool, or resource cluster, for that shared file system. There are methods of optimizing this sort of scenario, but locking cannot be avoided at some level within the shared file system. Once there are enough virtualization hosts all sharing the same file system, the contention for locking the file system for access becomes too great and the system performance becomes unacceptable.

From a scalability perspective, the larger a flat virtual machine storage file gets, the slower the overall access to it usually becomes. As discussed above, the more virtualized hosts placing virtual machine storage files on a shared file system, the greater the possibility is that there will be contention for accessing the resource. VMware’s published limits of virtualization hosts in a single cluster of resources is 32, but generally accepted best practice is never more than 8 to 12 hosts in a highly utilized DRS cluster. This is because they are all sharing a file system and must be able to lock the system for access. Microsoft’s published limit is 16 virtualized hosts, and best practices are likely even lower.

Do the technology benefits, like vMotion and LiveMigration, really require a shared file system? Do they really require that virtual machines utilize files (VMDK or VHD) as their storage containers? Or, was this approach chosen simply because it was the easiest to develop and deliver to market? With additional engineering, Red Hat has eliminated the shared file system and the virtual machine storage file requirements, and returned the performance and scalability of raw disk to the virtualized, enterprise datacenter.

RHEV Manager’s approach to storage management is far more advanced than the shared file system approach. This substantial architectural difference means that RHEV does not suffer from shared file system locking contention and makes it the ideal platform for I/O-intensive applications.

Additionally, not limiting  the virtualization management with a shared file system removes the restrictions on resource cluster scalability. As virtualized, enterprise datacenters grow, it’s important to be able to share resource pools with as many virtualized hosts as possible, not limit them to small, independently and separately managed pools of resource. Cloud computing is a great example use case for large scalability of resource pools, also drawing into question VMware’s viability as a cloud computing platform, with its dependence on the VMFS shared file system.

Red Hat Enterprise Virtualization uses direct mapping of virtual machines to a respective logical volume on shared storage. So, a running virtual instance in a RHEV environment is not confined by a file, but is stored on block-level disk. This provides each virtual instance in a RHEV environment with the I/O performance traditionally attributed to bare metal computing. The best part is that RHEV accomplishes this and still maintains feature offerings like LiveMigration. Yes, you can still migrate a running virtual machine instance in a RHEV environment from one KVM host to another without any downtime and without the requirements of a file for a storage container or a shared file system.

The approach also allows RHEV Manager to scale the number of KVM hosts in its resource clusters to orders of magnitude more than a VMware or Microsoft-based solution.

If you’ve unsuccessfully attempted to virtualize I/O-intensive workloads before, or if you have simply dismissed it as not feasible, consider Red Hat Enterprise Virtualization. Also keep RHEV in mind when you’re building out a large-scale solution that requires large pools of shared resources. When it comes to performance and scalability, RHEV is unmatched in the virtualization industry.