Until now there hasn't been a proper gauge to check a virtual machine's speed or performance.
But VMware, which fancies itself as the lead driver in virtualization technologies, has created VMmark, a benchmark system to help customers measure the performance of various applications running in virtual machines.
While traditional benchmarks were developed to measure the performance of a single workload on a server, they don't effectively gauge the ability of a server to support multiple, simultaneous workloads on the same server.
VMark is the answer, said Andrea Eubanks, senior director of enterprise and technical marketing at VMware.
Eubanks said customers who have been running production workloads have long been asking VMware for the typical virtual machine density policy for certain enterprise applications, or the right number of virtual machines to run on a certain piece of hardware. Until now, server vendors could only help them roughly estimate.
With VMark, server vendors can publish a score that provides scaling information about the workloads each product can support, as well as on the overall performance of virtual machines running on a server. This helps customers determine how much hardware they need to buy based on their plans to deploy certain enterprise workloads.
Eubanks said VMark presently runs six basic workloads as virtual appliances, including a file server, mail server, Web server, a standby server, an OLTP database and a Java order-entry system.
Half run on Windows, half run on Linux. Those six workloads represent what VMware calls a "tile." Scoring is determined by how many tiles a machine can scale up to.
"Unfortunately, because of Microsoft's licensing restrictions, we're not able to package the Exchange e-mail server and the other workloads on Windows on virtual appliances, although it would massively simplify the benchmark tool if we could," Eubanks said.
VMark is a VMware benchmark, but Eubanks said the company has lent VMark to performance benchmark group Standard Performance Evaluation Corporation (SPEC) to be used as the basis for a standard benchmark for measuring virtualization performance.