When it comes to understanding the inner workings of modern processors, security researchers often find themselves in a tricky situation. The operating systems they rely on, like macOS or Linux, weren't designed for such intricate analysis. Patching these kernels by hand is a messy and unstable process, and it's becoming increasingly obsolete on Apple's platforms. This is where the team at MIT's CSAIL steps in with a brilliant solution.
Enter Fractal, an innovative operating system kernel crafted from scratch. Unlike traditional OSes, Fractal treats the hardware as its primary subject of study. Its first major application was a deep dive into the branch predictors of Apple's M1 processor, and the results were eye-opening.
A New Perspective on Hardware
Modern processors have numerous internal structures that store state information, such as branch predictors, caches, and translation lookaside buffers. To study how these structures behave across user and kernel code domains, researchers need to run identical experiments on both sides of this boundary. However, general-purpose operating systems complicate matters by managing privilege levels, address spaces, and scheduling, introducing noise into measurements.
Fractal flips this model on its head. It boots directly on bare metal, eliminating any interference from other software. By exposing specific primitives, it allows experiments to switch privilege levels at runtime while executing the same instructions in the same address space. This technique, dubbed multi-privilege concurrency, relies on a novel construct called the outer kernel thread, which operates within a user process's memory but with kernel privileges.
The outcome? An experimental setup with minimal background noise. Fractal produces clean, clear signals, free from the interruptions and address-space management that blur measurements on macOS or Linux.
Uncovering the Secrets of the M1
Using Fractal, the MIT team confirmed that Apple's M1 processor, which implements the ARM CSV2 specification, provides protection against speculative execution in the execute stage of indirect branch prediction. In simpler terms, user-mode programs cannot make the kernel speculatively execute chosen targets through the indirect branch predictor.
However, the team also discovered something unexpected. While the protection works for execution, the CPU still fetches the target into the instruction cache before this stage. This fetch can be observed through a side channel, allowing user code to influence the kernel's cache across the privilege boundary. The same pattern was observed between processes with different address space identifiers.
Additionally, the team presented the first evidence of Phantom speculation on Apple Silicon. Phantom is a type of misprediction where ordinary instructions, including no-ops, are misinterpreted by the CPU as branches, leading to unwanted speculative behavior. Fractal showed that Phantom fetches succeed across both privilege levels and address spaces on the M1, although the execute phase remains blocked.
Fractal also challenged a previous finding about the M1's conditional branch predictor. While earlier work suggested that cross-privilege training worked on Apple's performance cores but not its efficiency cores, the Fractal team demonstrated that the conditional branch predictor lacks privilege isolation altogether, on both core types. The earlier result was likely due to macOS silently migrating threads between cores during system calls.
A Tool for the Community
Fractal is designed to be more than just a one-off experiment. It supports x86_64, ARM64, and RISC-V architectures and consists of over 31,000 lines of code. The MIT team has equipped it with familiar POSIX system calls, a C library, and ports of standard tools like vim, GCC, and the dash shell, making it easy for researchers to adapt their existing experiment code.
The long-term vision for Fractal is to become a shared infrastructure for the entire microarchitecture research community, akin to tools like QEMU and FFmpeg in their respective fields. By reducing software noise and providing tighter control across privilege boundaries, Fractal makes complex hardware experiments more interpretable and reliable.
Assistant Professor Mengyuan Li from the University of Southern California, who was not involved in the paper, praised Fractal as "a strong architecture contribution" that transforms microarchitectural reverse-engineering into reusable research infrastructure. With Fractal, the community can look forward to more accurate and dependable results, thanks to its clarity and guarantees of running on the right core and system.
In my opinion, Fractal is a game-changer for processor research. It empowers researchers to delve deeper into the intricacies of hardware, uncovering hidden behaviors and potential vulnerabilities. The insights gained from Fractal's experiments not only enhance our understanding of modern processors but also contribute to making them more secure and efficient. It's an exciting development that showcases the power of innovative thinking in the field of computer science.
