This article was originally published on August 12, 2022 and was called “On Virtual Machines”.
Introduction
I remember the first time I heard of virtual machines and hypervisors. It was way back in 1856, not long after my retaliatory caning of Preston Brooks and Laurence M. Keitt, and I remember thinking something along the lines of: “What kind of strange magic is this?”
At that time and even now, I feel that virtual machines and virtualization in general are technologies that people use all the time without having to know the slightest thing about what it is and how it’s done. In fact, many times we don’t even know that we’re operating in a virtualized environment.
And I think that’s peachy. There’s nothing wrong with that. It’s impossible to know everything about every layer of technology. It’s easy to build a mental model around what a virtual machine is and the benefits it provides, and that’s usually enough.
However, once you start doing anything serious with containers and OS-level virtualization that all changes. At least, that has been my experience.
Why? In a containerized world, it’s extremely helpful, necessary even, to know what the differences are and the security implications of running a process in a container versus a virtual machine. (There are other reasons to want to dig into how virtual machines work, of course, but this was the impetus for me.)
That’s a big subject, and in order to be able to speak cogently and confidently about it, you have to know a fair bit about the Linux kernel. However, for this article, I’m only going to briefly (and superficially) define these terms so that we have proper working definitions in order to construct the necessary mental model for the how a virtual machine works.
So, let’s begin with some definitions and then move into the main topic.
Glossary
CPU Modes
CPU modes, also called processor modes, are CPU operating modes, and, as part of the security model of the system, they place restrictions on processes and determine the type and scope of allowable operations. For instance, it allows the kernel to run at a higher privilege level than any user application. Conversely, user mode cannot access any memory other than a limited view (its virtual memory), and it does not have access to system resources (to access those resources, it needs to have the kernel act on its behalf).
The most common two modes are kernel mode and user mode. User mode requests the kernel to perform privileged operations on its behalf via system calls.
There is also a third mode that is effectuated when running one or more virtual machines managed by a virtual machine monitor (
VMM) or hypervisor. For instance, when a hypervisor is present, a processor that supports hardware-assisted virtualization can run at ring level -1 (more privileged than level 0) and then the guest kernel(s) run at 0. This allows theVMMto run one or many guest operating systems beneath it (more on that later).
Kernel mode is also referred to as privileged mode or supervisor mode and user mode as restricted mode, et al.
In kernel mode, the processor can execute anything that is allowed by its architecture, which in effect allows any instruction. On the other hand, user mode is going to be a restricted subset of those operations.
Moving from one mode to another necessitates a privilege context switch. Ring-based security is closely related to the idea of processor modes.
Privilege Rings
Privilege rings, also known as protection rings, are concentric circles of privilege from 0 (most privileged) to 3 (least privileged). Ring 0, for example, allows for direct access of the hardware on the machine. This is the level in which the kernel and device drivers run. Ring 3 is for user mode applications.
A process running in kernel mode is operating at ring 0, and a user mode process is operating at ring 3.
Interestingly, modern x86 CPUs support hardware-assisted virtualization, which allows the hypervisor to run in an extra-privileged below ring 0, essentially making it ring -1. When this occurs, the guest kernels run at ring 0, with the end result being that there doesn’t need to be any hacky workarounds to enable full virtualization. More on this later.
The privilege rings are hardware enforced by a CPU status register. In other words, the idea of a ring is only a conceptual model to help understand the nested levels of trust, with inner-most ring being the most trusted. The enforcement are bits stored in a CPU register, checked by the CPU whenever an instruction is issued by a process and blocks if the instruction requires a higher privilege level than the current CPU privilege mode.
User Space and Kernel Space
Conceptually, user space and kernel space is a way for an operation system to segregate virtual memory. Kernel space refers to the execution of privileged instructions, like the kernel itself, device drivers, eBPF programs, etc., and the user space (or userland) is where user application programs are executed like web browsers, shells, editors, libc, etc.
Importantly, the separation of kernel space from user space allows for memory protection.
How does user space interface with kernel space? Through the system call API.
System Calls
User space processes programmatically request services from the kernel through system calls (syscalls). These are requests for file management, process control, communication, et al, and the kernel does the work on behalf of the user space process.
For most operating systems, syscalls are only made from user space processes.
The userland process, restricted and isolated to its own virtual address space, cannot access or modify other processes or the kernel itself. In addition, anything running in user mode is prevented from directly accessing the machine hardware and must rely on this system call interface to ask the kernel to perform the task.
A lot of things happen that are outside the scope of this brief definition, but the end result is a mode switch from restricted (user) to unrestricted (kernel) mode. Control is passed to the kernel which, running at the highest privilege (ring 0), checks to ensure that the process is allowed to make the syscall, and then directly accesses the hardware and performs the operation on behalf of the userland process. Control is then returned to the calling program.
As noted, system calls necessitate a mode switch, but it is not a process context switch. Instead, it’s a privilege mode switch. The appropriate bits are set in the processor status register when the syscall is initiated, and the mode is changed from user mode to kernel mode.
Most programming languages have libraries that hide the details of system calls from the developer. For instance, a higher-level programming language like Python or Go would have APIs that wrap the actual lower-level call through to the underlying C library.
For example, a common way in Go to open a file on the filesystem is with the os.Open library call. If you were to follow this down the rabbit hole, you will eventually come to the RawSyscall6 stub, where, according to the comment above it, declares that this function is linked to particular OS file in runtime/internal/syscall and finally the Assembly for the particular machine architecture (in my case, amd64). This calls the libc (or possibly glibc) syscall interface API.
Examples of common syscalls are
open,read,write,close,wait,exec,fork,exit, andkill. View the man page for more information about each one:$ man 2 exec $ man 2 fork
In my opinion, the key thing to understand about how system calls work is that it triggers a software interrupt, i.e. it executes a trap instruction, that initiates the privilege mode switch. The trap tells the processor to jump to a well-known address based on the trap’s parameter, which is an interrupt number. This number is a lookup into the interrupt vector table, which is a data structure that maps these well-known interrupt numbers with a location in memory to a callback that will handle the trap.
let’s start with trapping. my understanding is that the host CPU must see every instruction, unprivileged or not, b/c it has to check a status register to determine if the instruction has the correct privileges. if it’s a privileged instruction this must be trapped by the CPU and passed to the VMM, which then determines what happens to it. but i am unclear on the details. i’d like to know more.
Making a system call uses the trap mechanism to switch modes to a well-defined point in the kernel, which then runs all the instructions in kernel mode.
Now, at long last, armed with solid definitions of the crucial operating system functionality that facilites a solid understanding of virtual machines, we can move on to the point of this fantastic article.
It’s very important to state explicity that everything just described is normal operating system behavior and has nothing to do with virtualization. Virtualization is an additional abstraction on top of these OS-level protections.
Mental Model
I’d been confused about how virtualization works for a long time. The barrier to understanding this topic was steep, in my view, and a lot of technical documentation that I have read hasn’t helped. Perhaps it was just me, but a brief description of resource allocation followed by the introduction of user modes and privilege rings didn’t elucidate anything. Unfortunately, it was just getting started. Then, there were the topics of trapping and scary words and phrases like binary translation, paravirtualization and hardware-assisted virtualization. It was enough to make anyone piddle in their corduroys.
This section is my attempt to break through that fog and have a reader come away thinking, “oh, is that all that it is?”.
The mass adoption of virtual machines and containers (OS-level virtualization) speaks to the brilliance of the implementations. Virtualization is a great example of a domain in computing where you don’t need to know shit about what’s happening under the sheets to be able to use it.
Of course, that ignorance has let to many security breaches, but who cares about that as long as you can “stand” something up? I’m looking at you, “devops”.
A hypervisor, whether type 1 or type 2, will allocate hardware resources for each guest OS. Each allocation can be thought of as a partition, and these allocations are completely isolated from one another. The allocations are from real hardware on the host, of course.
-
Disk Space
Importantly, the disk space is usually just a virtual hard disk drive on the host OS. In other words, it’s just a file that the VMM will index into when the guest OS is reading or writing to its allocated disk space. An example of this are the
qcow2(QEMU Copy-On-Write version 2) files thatQEMUcreates when I uselibvirtas a frontend andKVMas a hypervisor.You’ve probably heard for millennia that untrusted software should be downloaded and installed in a virtual machine. Why is this good advice? Let’s assume that the download was something malicious or something horrible, like anything from OpenAI or Anthropic. This download and installation would only affect the guest operating system’s allocated disk resource, as it would be saved in a file image on the disk (for example, in a
qcow2disk image file. There it would be trapped and could not escape to the host filesystem, unless, of course, you did something incredibly stupid, like then mounting that image on the host and running an executable. Any disk I/O request is intercepted by the VMM and indexes into the image file that it’s been allocated, so no read/writes occur on the host fileystem.Incidentally, this is (mostly) the same as the disk resource isolation of a container (VMs add an additional layer of privilege-level isolation that containers lack).
-
Memory
Memory is virtually allocated and maps to different memory locations in physical RAM on the host. Every allocation is isolated from all others. The virtual memory pages are mapped to the physical memory pages by the memory management unit (MMU). The CPUs in the guest are virtual and instructions from the guest are either run directly by the host CPU (for unprivileged instructions) or trapped and sent to the VMM (for privileged instructions).
Continuing the scenario where malware is installed in the guest OS, the malware instructions could be in the VM’s virtual memory, and since it’s mapped to host memory, also in the physical RAM. This is not a problem, however, because if there are any privileged instructions, as there surely would be in malware, they will not be executed by the host CPU because they will be trapped and then rejected by the VMM.
-
CPU
So, guest A could be writing to memory address
x6000and guest B could be writing to memory addressx6000, but those are only the views that they “see” (have access to) in their virtual memory allocation and are not the same address on the host. On the host, they would be translated to completely different memory address in its physical RAM.No privileged instructions are ran by the host CPU, so no malicious instructions in memory will be executed.
-
Network I/O
Note that any escape to the host could only be facilitated by a bug in the hypervisor code.
We’ll now turn to one of the most well-known and used implementations of virtualization.
Virtual Machines
Virtual machines have been around a long time. IBM built research mainframes in the 1960s that allowed for full virtualization, and once Intel and AMD introduced the hardware extensions into their chips in 2005 and 2006, respectively, virtual machines really took off as a prolific tool in every developer’s toolbox.
Early examples of computers capable of running virtual machines are the IBM CP-40 and the IBM SIMMON hypervisor.
In order to run, virtual machines need to have a piece of hardware or software installed to manage them. Let’s take a look at a software example.
Virtual Machine Monitors
To run a virtual machine, there needs to be a virtual machine monitor (VMM) (probably better known as a hypervisor), installed that manages the guest operating systems. It will allocate the hardware resources (CPU, disk space, memory, etc.) to each virtual machine.
The guest machines all have virtual access to the host machine’s hardware provided by the VMM, and most of their instructions (i.e., unprivileged instructions) are run directly by the real processor (as opposed to full emulation). We’ll soon take a look at how the guest machines, running at a ring level equal to or higher than 0, are able to have its privileged instructions emulated by the VMM.
The machine on which the hypervisor runs is called the host machine, and all of the virtual machines are the guest machines.
The term hypervisor was coined in 1970 and means the supervisor of the supervisors (the term supervisor refers to a kernel).
There are two types:
-
Type 1
Runs on bare metal, that is it runs directly on top of the hardware, and the guest operating systems it monitors are above it, conceptually. It directly controls the hardware.
-
Type 2
Runs in user space as a process of the host (or native) operating system, and its guest operating systems are above it, conceptually. The host machine has direct control of the hardware.
Of course, there is a type that doesn’t fit neatly into either of these conceptual models, and that is KVM (Kernel-based Virtual Machine). KVM is a Linux kernel module that effectively turns the host operating system into a hypervisor, which is why it’s usually included when talking about Type 1 VMMs.
Ok, so far, so good. But since the virtual machine isn’t a real machine and only has proxied access to the real hardware, how can it possibly run privileged instructions in kernel mode? In other words, if the entire virtual machine, which includes its kernel, runs in a privilege ring higher than ring 0 (kernel/privileged mode), how can it possibly have a way to have its privileged instructions executed?
Let’s now look at one answer.
Trap-and-Emulate
The implementation style of trap-and-emulate became the prevalent method, although there were others. Some researchers have referred to this as the classical style of virtualization.
The fundamental idea behind trap-and-emulate is that the VMM will allow as many non-privileged system calls from the guest OS through as possible (that is, it allows the guest OS to perform these actions directly without intervention from the VMM), and only to emulate the ones that are privileged and legal. If illegal, the hypervisor will terminate the operation. The emulation that the VMM performs is the behavior that the guest OS expects from the hardware.
However, there were problems. The dominant x86 processor platforms, Intel and AMD, didn’t support the trapping of privileged instructions. So, nothing would happen if a processor was running in user mode and attempted to execute a privileged instruction.
This was a huge obstacle and made full virtualization on these platforms impossible. After all, if a privileged instruction is ignored, the trap never happens, and the VMM would have no way of knowing about it. The end result was there was no emulation and full virtualization was out-of-reach, at least for these platforms.
There were two prevalent hacks to work around this issue. Let’s take a look at them now.
x86 Workarounds
Binary Translation
With binary translation, the VMM pre-scans the input instruction stream for privileged instructions (i.e., instructions that are to only be run in kernel mode) and replaces them with traps that the VMM can intercept.
The non-privileged instructions are still executed by the processor as usual.
Paravirtualization
Paravirtualization will modify the guest kernel and replace any privileged instructions with API calls to the VMM. These act like a system call, which triggers a trap and a privilege context switch to the VMM.
Although it is higher performing than binary translation, since it requires access to and modifying of the kernel, it doesn’t work in closed-source operating systems like Windows.
Hardware-assisted Virtualization
With the advent of hardware-assisted virtualization in Intel and AMD CPUs, binary translation and paravirtualization were obsoleted. As we saw in the graphic in the privilege rings section, the hardware now supports an extra-privileged virtualized mode called vmx root mode that runs the hypervisor in ring -1 and the guest machine in ring 0 (so, the guests think they are running as the host OS).
This means there is no need to either replace any instructions or intrusively modify the kernel; the guest operating system remains unmodified.
Summary
That’s it for now. I intend to write more posts about virtualization, and I also intend to revisit this post and update and augment when necessary.
Hopefully, these brief overview has gone a bit more in-depth than most introduction tutorials on the subject of virtual machines. For me, I wasn’t able to begin to wrap my head around the subject until I understood a bit more about memory protection and processor modes.