How Building My Own OS Changed the Way I See Operating Systems

Copied

The other day, I started building my own OS by following Writing an OS in 1,000 Lines, a guide written by nuta.

As a university student majoring in computer science, I tend to regard the OS, one of the most important components of a computer, as something special among all kinds of software, and I had long admired the idea of building my own OS.

On the other hand, the operating systems in use today, starting with Linux, are extremely complex, and partly because of the difficulty of setting up a development environment, I had never managed to step into actually implementing an OS.

Then I learned about Writing an OS in 1,000 Lines on X (formerly Twitter).

After reading its preface, I thought it looked more approachable than any other build-your-own-OS book I had come across, so I started copying the code by hand.

As a result, just as the preface promised, I was able to build the following OS in about three days, and I have also made progress implementing extra features.

https://github.com/speed1313/cos

In the process of implementing an OS, I learned at the implementation level how an OS, and the kernel in particular, actually works, and the way I see the kernel changed quite a bit.

In this post, I describe how my “view of the OS” changed between before and after building one, something I can probably only write now, right after having started.

Attention Note that I use “OS” and “kernel” almost as synonyms.

I have only just started building an OS, so the following may contain mistakes. If you find any, please let me know.

What I Thought Before Building an OS

I pictured the OS exactly as in the diagram in Wikipedia’s Operating system article: an intermediary sitting between the Application and the Hardware.

Something that keeps running all the time, supporting applications from below.

I had a rough idea of the roles and mechanisms of an OS from reading an OS textbook and Patterson and Hennessy, but I could not imagine what it would look like when actually written as a program.

In particular, I had no idea at all how the interface with the hardware would be written.

I knew the mechanisms and roles of concepts such as paging, exception handling, process scheduling, and memory allocators, yet it was vague to me which part was handled by whom (the OS? the CPU?) and how.

What I Think After Building an OS

The Kernel Is (in a Sense) Just a Process

It is obvious in hindsight, given that operating systems existed even in the single-processor era, but the OS (the kernel) is not running all the time; the applications and the kernel take turns running. (Now that multiprocessors are widespread, there may also be designs where the kernel runs all the time, though.)

More precisely: the bootloader loads the boot image (e.g. an ELF file) into memory and sets the pc to the kernel’s start address, which starts the kernel. The kernel performs various initializations and starts running the first application (such as a shell). After that, the following cycle repeats: an event such as a system call or an exception triggers the CPU’s exception handling mechanism, which rewrites the pc to point to the part of the kernel that handles the context switch into the kernel; the kernel handles the exception in kernel mode according to the event; and once it finishes, control moves back to the application.

Therefore, viewed from the implementation side, switching between an application and the kernel is nothing more than changing values held by the CPU, such as the pc (program counter), the general-purpose registers, the sp (stack pointer), the start address of the page table, and the privilege mode.

I find this fact quite striking, because apart from the change of privilege mode, it is hardly any different from a context switch between applications.

The kernel can be described as a special process with the following characteristics:

  • It runs in supervisor mode
  • It can, or must, manage many things
  • It is the very first thing started by the bootloader

But in the end, it is still a process.

Assembly Language Is Indispensable

A kernel program performs operations that cannot be written in a high-level programming language, such as directly manipulating registers that hold the start address of the page table or control mode switching.

So it has no choice but to use assembly language.

For this reason, kernel programs either link in assembly files or, using inline assembly, a compiler extension, embed assembly partially into high-level language code like this:

void boot(void){
    __asm__ __volatile__(
        "mv sp, %[stack_top]\n" /* set stack pointer to the top of the stack */
        "j kernel_main\n" /* jump to kernel_main */
        :
        : [stack_top] "r"(__stack_top) /* __stack_top is stored in a general purpose register named stack_top */
    );
}

For more on this, the places in the Linux source code where inline assembly is used are a good reference.

CPUs Have So Many Features

Beyond simple instructions like add and jump, CPUs offer a rich set of features such as mode switching, exception handling, and paging.

For example, on a RISC-V CPU, an instruction like the one below registers the start address of the page table in the satp register, and the CPU then performs paging automatically.

"csrw satp, %[satp]\n"

Also, by using the ecall instruction as shown below, you can raise a system call exception and handle the system call in the exception handler, which implements a system call: switching from user mode to kernel mode to do the work.

__asm__ __volatile__("ecall"
	                 : "=r"(a0)
	                 : "r"(a0), "r"(a1), "r"(a2), "r"(a3)
	                 : "memory");

When building an OS, you make generous use of these CPU features to implement things like system calls, paging, and exception handling.

I used to think the kernel was amazing for doing so many things, but in fact it was wielding a powerful weapon: the CPU.

Exception Handling Is Very Important

Anything that transitions into kernel mode, such as a system call or a page fault, is implemented by moving the pc to the start address of the exception handler registered with the CPU.

Exceptions can be regarded as events: when an event occurs, the CPU hooks into the exception handler, which does different work for each kind of event, so the kernel can deal with a wide variety of events as they happen.

The Timer Was Important Too

Timer exceptions, raised by the timer built into the CPU, are indispensable for preemptive multiprocessing such as round-robin scheduling.

I used to wonder why OS textbooks listed timer exceptions alongside system calls and the like as examples of exceptions, but once I learned how important timer exceptions are, it finally made sense.

I underestimated the timer. I’m sorry.

When the Kernel Panics, Everything Stops

When an application panics, the kernel takes care of it, but when the kernel panics and stops, there is nothing to handle it and recover. So the whole system just stops.

Once the whole system stops, the only option is to reboot the computer, so in practice kernel panics should be avoided.

Come to think of it, when the attempt to bring Rust into Linux began, Linus Torvalds said:

I do think that the “run-time failure panic” is a fundamental issue.

LKML: Linus Torvalds: Re: [PATCH 00/13] [RFC] Rust support

calling for serious attention to panics.

Looking back after building an OS, I think he is exactly right about how important the handling of panics is in kernel development.

Reference: Rust for Linux uses its own alloc library

OS Development Is a Mixed Martial Art

The technologies you come into contact with when building an OS cover a wide range: compilers, CPUs, shells, file systems, network protocols, and more.

Building an OS is a mixed martial art where every bit of low-level, build-it-yourself experience you have gained in the past comes in handy.

Black Boxes All the Way Down

I now have a reasonable grasp of what the kernel does, but new black boxes have emerged: the bootloader, device drivers, and how the many features built into the CPU are implemented.

Conclusion

This time I shared my impressions after looking at a very conventional OS design, but apparently recent research is exploring interesting designs such as:

  • Designs that replace software context switches with hardware threads
  • Unikernels, in which the application and the kernel are combined into one

(Reference: 自作OSで学ぶマイクロカーネルの設計と実装 (Design and Implementation of Microkernels Learned by Building Your Own OS))

There are many ways to specify and implement an OS, and each of them has its own face.

I wonder what face the next OS I meet will have.

References