Virtual Memory Management

Think of it as a way to tell lies using technology. But they aren’t lies to you, or anybody else for that matter. The reality of memory address management in virtualized systems is that the virtualization software — usually the hypervisor — gives each operating system (OS) a fictional story about where its memory actually resides. And then it keeps track of where that memory really is.

The reason your server is making stuff up has to do with the fact that most computer OSes were designed to run alone on the server hardware. The OS thinks its memory addresses start with zero, and goes up from there. The problem is that a server with perhaps 16 operating systems running on the same machine doesn’t have zero available as a physical address for all of those OSes. Instead, it has to allocate a section of physical memory for the use of each OS, and then tell the OS that it’s got address location zero, when in reality the physical address is something else entirely.

Exactly how this memory address translation happens differs depending on which virtualization software is being used. A simple way to look at it is to imagine it is a table that the hypervisor (a software process that supervises the operations of the virtualization system) maintains so when an OS asks for a memory address, it’s given a spot in the area of physical memory that’s been allocated for it. In theory, it’s pretty simple.

Of course, reality changes everything.

According to VMware’s Chief Platform Architect, Richard Brunner, the process by which the OS manages memory depends on a number of factors, including the OS in use, and the hardware environment on which VMware is running. The process of using look-up tables is only one way of doing it. “In yesterday’s machine, we used shadow page tables as the method on yesterday’s processors,” Brunner said. “We did that because yesterday’s processors did not understand virtualization and did not have the ability to do the translation.”

Brunner saidAMD and Intel are now making servers with processors that include their own ability to handle page tables in hardware, thus freeing the VMware hypervisor from the task. Either way, however, the process still involves VMware accepting a request from the OS for access to a specific memory location and translating that request to the actual memory location.

But it gets more complex. To save memory, VMware can tell whether it’s supporting two or more identical copies of the same OS. If it is, it can create a common memory location for the files that reside on the OS kernel, and simply refer all requests from any virtual machine to that one location. This means there may be only one copy of the Windows kernel running, and it’s shared by all the virtual Windows machines on the server. Those shadow tables in VMware or on the processor keep track of what’s being held in the protected kernel space, and they provide access to each copy of Windows as the particular DLL is needed.

The shared kernel memory is set to read-only. That way, one copy of Windows can’t change something another copy might need. If some event causes Windows to write to the material stored in the shared kernel, VMware creates a new copy of the kernel in a new memory space to accommodate the modified kernel.

Unlike other methods of providing virtual memory space, VMware can provide memory pages as the OS needs them. This way, the memory isn’t committed until it’s actually needed, and that provides space for other OSes if they need it.

Part of the process of making memory addressing more flexible involves what Brunner calls “over commitment.” This allows guest OSes (as VMware calls them) to request more memory than they actually have available. VMware temporarily allows the OS with the greater need to get access to the additional memory by using physical memory that other guests aren’t using. When the demand goes away, so too does the memory requirement.

However, not every company uses the same technique as VMware. Currently, IBM’s approach to virtual memory doesn’t support the use of the over commitment. But according to Bob Kovacs, a Virtual Server Architect from IBM, that capability is coming in the spring. Currently, IBM’s virtualization sets up fixed amounts of memory for each OS, and the hypervisor keeps a table of where the physical memory is actually located, and it translates memory requests from the virtual machine to the physical environment as needed.

According to Andrew Hillier, CTO of CiRBA, which sells a management system for virtualization, a different approach can be used when running Linux on a virtual machine. There, he said, the idea is to provide as little memory as possible to the OS. That way, the OS acts as if it’s running in a low-memory environment, and pages out everything it can to a swap file. However, in this case, the swap file is also running in memory, keeping performance up.

Hillier said operating Linux in as little memory as possible makes the OS very frugal. This results in more-efficient operation.

Hillier also noted that not every approach to virtualization uses a hypervisor. He noted Sun uses what it calls Solaris Zones, which means virtual machines are all using the same copy of Solaris, but in their own memory space. The OS itself keeps track of what’s running in which memory space, eliminating the need for a hypervisor.

With the exception of Solaris, where there’s only one copy of the OS running on a server, other approaches are simply creating an environment where it appears to the OS that it has the machine to itself. The reality is that many OSes can be supported by a single server, and in the process improve efficiency and reduce wasted storage. But you can do this only if you don’t tell the OS the truth about where it is.

This article was first published on ServerWatch.com.