b0nker

In operating systems, a very important concept is virtualization. Its initial implementation was done on the CPU to make processes believe they were the only ones running on it and the only ones in memory. Developing such a notion entirely changed how things worked.

Now imagine we want to develop the idea further to virtualize entire systems including its file system, processes and even network such that from its perspective, 'it is the host system'. And that's where containers come in.

Containers: How do they work

Containers are processes that run natively on a machine but given an illusion of complete isolation. They are known by different names in different operating systems like Jails in FreeBSD or just Containers in Linux.

Containers are built using some low level mechanisms & syscalls. For Linux Containers, virtualization takes place using the following mechanisms:

• Namespaces
• Root Pivot
• Cgroups
• Overlayfs

Let's understand each of them a bit better.

Namespaces: How to give a process a fake view of the system

Namespaces are what makes it possible for programmers to have two different identifiers (variables, functions) from two different libraries with the same name exist on a codebase together.

On the topic of containers, it's a feature of the linux kernel that lets you isolate global system resources for individual processes.

We use namespaces to create a virtualization around PIDs, networks and mount points such that the process gets an illusion that it has its own dedicated environment.

Namespaces can be created using multiple syscalls including clone() or unshare(). Both do the same except clone does it through a child process whereas unshare alters the current process itself. Their comparison is similar to fork() & exec().

Root Pivot: How to give a process a fake file system

Once a process gets its own namespace, how do we point to its new root directory and not of the host machine.

The most common approach to this might be chroot(). However chroot() has no security measure that ensures a process can't access its old root again. Hence we need something better.

That's where pivot_root() comes in. The call takes in two arguments, where the new root should point at and the path where the old root should be put at. We can then safely unmount the old root and hence our system has a whole new root with no access to the old.

Cgroups: How do we limit our containers

When running such processes, we must ensure they don't hog up our host system's resources. To do this, we use cgroups. Unlike the other steps we followed, here we don't have a syscall rather we have a virtual file system.

Virtual file systems exist only in memory and not in your hard drive; they act as a real time view of the kernel and thus we can read/write to it to interact with the kernel directly. /proc & /sys are examples of a VFS.

cgroups are found at /sys/fs/cgroup/ and within it are multiple views into the kernel. We are most interested in a couple like memory.max to limit the max memory the container can take and pids.max to limit the number of processes allowed within a container. Running the command below creates a new cgroup with a max memory of around 100mb and maximum of 64 processes.

mkdir /sys/fs/cgroup/mycontainer
echo 100000000 > /sys/fs/cgroup/mycontainer/memory.max
echo 64 > /sys/fs/cgroup/mycontainer/pids.max
echo <your_pid> > /sys/fs/cgroup/mycontainer/cgroup.procs

OverlayFS: How to share a base filesystem between containers without corruption

Now imagine we want to run multiple containers but want to localize any changes made such that every container begins from the same base image and any extra writes would be local to that container. OverlayFS could be used here to achieve this properly.

It does so by having 2 parts:

• Lower: This is the base image where only read takes place. Nothing is written by a container here
• Upper: This is where all the writes take place. All containers have their own upper

The syscall for overlayfs also takes in two other paths which are for:

• Merged: The actual data the container will be looking at which contains both the upper and lower together.
• Work: A sort of buffer area for the kernel to do atomic operations to make the data available as expected.

b0nker

Using these very concepts b0nker was built. It's a lightweight container runtime built in rust that can spin up multiple containers simultaneously and exits each container gracefully once done. Bonker is the container struct and the containers themselves are referred to as 0ysters.

The source code for b0nker can be found at: github.com/owlpharoah/b0nkers

Usage

b0nker run <command> [args...]

Examples:

/b0nker main ➤ sudo -E ./target/release/b0nker run ls -al
    total 0
    drwxr-xr-x    1 0        0                0 Jul  4 07:57 .
    drwxr-xr-x    1 0        0                0 Jul  4 07:57 ..
    drwxr-xr-x    1 1000     1000            88 Jul  3 22:00 bin
    drwxrwxrwt    2 0        0               80 Jul  4 07:57 dev
    drwxr-xr-x    1 1000     1000            16 Jul  3 19:02 etc
    dr-xr-xr-x  308 0        0                0 Jul  4 07:57 proc
    drwxr-xr-x    1 1000     1000            16 Jul  3 19:02 sys
    drwxr-xr-x    1 1000     1000            16 Jul  3 19:02 tmp
    drwxr-xr-x    1 1000     1000            16 Jul  3 19:02 usr
    drwxr-xr-x    1 1000     1000            16 Jul  3 19:02 var
 
/b0nker main ➤ sudo -E ./target/release/b0nker run ps
    PID   USER     TIME  COMMAND
        1 0         0:00 ps

Benchmarks

The benchmarks over 100 runs of b0nker run true:

/b0nker main ➤ hyperfine \
  --warmup 20 \
  --runs 100 \
  --export-json benchmark.json \
  'sudo ./target/release/b0nker run true'
        Benchmark 1: sudo ./target/release/b0nker run true
        Time (mean ± σ):      37.0 ms ±   6.1 ms    [User: 7.1 ms, System: 8.8 ms]
        Range (min … max):    27.2 ms …  57.6 ms    100 runs
 
        "mean": 0.03696413268999999,
        "stddev": 0.006127194251732403,
        "median": 0.036267097580000004,
        "user": 0.007134180000000002,
        "system": 0.008807100000000002,
        "min": 0.027217842080000002,
        "max": 0.05759475708,

Stats:

• Mean: 37.0 ms
• Median: 36.3 ms
• Min: 27.2 ms
• Max: 57.6 ms

What's Next?

The next steps would be to include:

• network namespace
• OCI image support
• optimizing certain functions