aster.cloud aster.cloud
  • /
  • Platforms
    • Public Cloud
    • On-Premise
    • Hybrid Cloud
    • Data
  • Architecture
    • Design
    • Solutions
    • Enterprise
  • Engineering
    • Automation
    • Software Engineering
    • Project Management
    • DevOps
  • Programming
    • Learning
  • Tools
  • About
  • /
  • Platforms
    • Public Cloud
    • On-Premise
    • Hybrid Cloud
    • Data
  • Architecture
    • Design
    • Solutions
    • Enterprise
  • Engineering
    • Automation
    • Software Engineering
    • Project Management
    • DevOps
  • Programming
    • Learning
  • Tools
  • About
aster.cloud aster.cloud
  • /
  • Platforms
    • Public Cloud
    • On-Premise
    • Hybrid Cloud
    • Data
  • Architecture
    • Design
    • Solutions
    • Enterprise
  • Engineering
    • Automation
    • Software Engineering
    • Project Management
    • DevOps
  • Programming
    • Learning
  • Tools
  • About
  • Containers
  • Programming
  • Software Engineering

How To Run Untrusted Containers In Kubernetes

  • aster.cloud
  • June 25, 2022
  • 5 minute read

The IT world is adopting container-based infrastructures more and more every day. However, the advantages, disadvantages and even the limitations are not clear to everyone.

Considering that even large companies are approaching container-based infrastructures, it is a good idea to account for possible attack areas and the potential impact of a data breach.


Partner with aster.cloud
for your next big idea.
Let us know here.



From our partners:

CITI.IO :: Business. Institutions. Society. Global Political Economy.
CYBERPOGO.COM :: For the Arts, Sciences, and Technology.
DADAHACKS.COM :: Parenting For The Rest Of Us.
ZEDISTA.COM :: Entertainment. Sports. Culture. Escape.
TAKUMAKU.COM :: For The Hearth And Home.
ASTER.CLOUD :: From The Cloud And Beyond.
LIWAIWAI.COM :: Intelligence, Inside and Outside.
GLOBALCLOUDPLATFORMS.COM :: For The World's Computing Needs.
FIREGULAMAN.COM :: For The Fire In The Belly Of The Coder.
ASTERCASTER.COM :: Supra Astra. Beyond The Stars.
BARTDAY.COM :: Prosperity For Everyone.

Technologies like Docker (containerd) and LXC are not really isolated systems as they share the same Linux Kernel as the operating system they are hosted on.

For a potential attacker, having their container launched within a large company is a golden opportunity. Does technology allow us to defend ourselves easily?

Current container technology

It’s been repeated several times, Containers are a new way of packaging, sharing, and deploying applications, as opposed to monolithic applications where all features are packaged in a single software or possibly an operating system.

Right now, containers do not exploit anything new, but they are an evolution created above the Linux Namespace and Linux Control Groups. The namespace creates a virtual and isolated user space and provides the application with its system resources such as file system, network and processes. This abstraction allows the application to be started independently without interfering with other applications running on the same host.

So, thanks to namespaces and cgroup together, we can definitely start many applications running in the same host in an isolated environment.

Container vs Virtual Machine

It is clear that container technology solves a problem in terms of isolation, portability and streamlined architecture compared to a virtual machine environment. But let’s not forget that the virtual machine allows us to isolate our application, especially at the kernel level, and therefore the risk that a hacker can escape a container and compromise the system is much higher than escaping a virtual machine.

Most kernel Linux exploits might work for containers, and this could allow them to escalate and compromise not only the affected namespace but also other namespaces within the same operating system.

Read More  An Introduction To Kubernetes Security Using Falco

These security issues led researchers to try to create real separated namespaces from the host. Specifically called “sandboxed” and right now there are several solutions that provide these features: gVisor or, for example, Kata Containers.

Container Runtime in Kubernetes

We can go deeper into this type of technology within a container orchestrator, Kubernetes.

Kubernetes uses a component, kubelet, to manage containers. We can define it as the captain of the ship that takes charge of the specifications that are supplied to it and performs its operations punctually and precisely.

Kubelet takes pod specifications and makes them run as containers on the host machine to which they are assigned and can interact with any container runtime as long as it is OCI standard compliant (whose implementation is RunC)

How container runtime works
How container runtime works

RunC, originally embedded into the Docker architecture, was released in 2015 as a standalone tool. It has become a commonly used, standard, cross-functional container runtime that DevOps teams can use as part of container engines.

RunC provides all the functionality to interact with existing low-level Linux features. It uses namespaces and control groups to create and run container processes.

In the following paragraph, we will introduce the runtime class and the core elements. There is also the RuntimeClass handler whose default value will be RunC (for installations of Kubernetes that use containerd as the container runtimes).

Runtime Class

The Runtime Class, as its name suggests, allows us to operate with various container runtimes. In 2014, Docker was the only runtime container available on Kubernetes. Starting with version 1.3 of Kubernetes, compatibility with Rocket (RKT) was added and finally with Kubernetes 1.5, the Container Runtime Iterface (CRI) was introduced with the possibility of having a standard interface and all the container runtime you can interface directly, with this standard saving developers the trouble of adapting to all types of container runtimes and worrying about version maintenance.

Read More  How to Learn Effectively With Your Personal Programming Projects

The CRI, in fact, has allowed us to decouple the part of container runtime from Kubernetes and, above all, allowed technologies such as Kata Containers and gVisor to connect to a container runtime as containerd.

In Kubernetes 1.14, RuntimeClass was introduced again as a built-in cluster resource with a handler property at its core.

The handler indicates the program that receives the container creation request and corresponds to a container runtime.

kind: RuntimeClass
apiVersion: node.k8s.io/v1
metadata:
    name: #RuntimeClass Name
handler: #container runtime for example: runc
overhead:
    podFixed:
        memory: "" # 64Mi
        cpu: "" # 250m
scheduling:
  nodeSelector:
    <span class="token tag"><span class="token punctuation"><</span>key<span class="token punctuation">></span></span>: <span class="token tag"><span class="token punctuation"><</span>value<span class="token punctuation">></span></span> # container-rt: gvisor
  • The handler field points to specific container runtime or configuration to be used.
  • Declaring overhead allows the cluster (including the scheduler) to account for it when making decisions about Pods and resources. Through the use of these fields, you can specify the overhead of running pods utilizing this RuntimeClass and ensure these overheads are accounted for in Kubernetes.
  • The scheduling field is used to ensure that Pod are scheduled on the right nodes.

By default, our handler, if we have a cluster with Docker or containerd, is runc, but if we use gVisor it will be runsc.

Isolate Linux Host and Containers with gVisor in Kubernetes

Now we will see how can we have more than one container runtime in a Kubernetes cluster and choosing a more strict one for sensitive workload.

For this tutorial, I use a previous project where I carried out the installation of a Kubernetes cluster with contained.

https://github.com/alessandrolomanto/k8s-vanilla-containerd

Initialize the Kubernetes Cluster

make vagrant-start

After bringing machines up, verify if all components are up and running

vagrant ssh master

kubectl get nodes
NAME      STATUS   ROLES                  AGE     VERSION

master    Ready    control-plane,master   7m59s   v1.21.0

worker1   Ready    <span class="token tag"><span class="token punctuation"><</span>none<span class="token punctuation">></span></span>                 5m50s   v1.21.0

worker2   Ready    <span class="token tag"><span class="token punctuation"><</span>none<span class="token punctuation">></span></span>                 3m51s   v1.21.0

Install gVisor on worker1

ssh worker1 # Vagrant default password: vagrant

sudo su

Install latest gVisor release

(
  set -e
  ARCH=$(uname -m)
  URL=https://storage.googleapis.com/gvisor/releases/release/latest/${ARCH}
  wget ${URL}/runsc ${URL}/runsc.sha512 \\
    ${URL}/containerd-shim-runsc-v1 ${URL}/containerd-shim-runsc-v1.sha512
  sha512sum -c runsc.sha512 \\
    -c containerd-shim-runsc-v1.sha512
  rm -f *.sha512
  chmod a+rx runsc containerd-shim-runsc-v1
  sudo mv runsc containerd-shim-runsc-v1 /usr/local/bin
)
FINISHED --2022-04-28 07:24:44--

Total wall clock time: 5.2s

Downloaded: 4 files, 62M in 3.1s (20.2 MB/s)

runsc: OK

containerd-shim-runsc-v1: OK

Configure container runtime

cat <<EOF | sudo tee /etc/containerd/config.toml
version = 2
[plugins."io.containerd.runtime.v1.linux"]
  shim_debug = true
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
  runtime_type = "io.containerd.runc.v2"
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runsc]
  runtime_type = "io.containerd.runsc.v1"
EOF

Restart containerd service

sudo systemctl restart containerd

Install the RuntimeClass for gVisor

cat <<EOF | kubectl apply -f -
apiVersion: node.k8s.io/v1beta1
kind: RuntimeClass
metadata:
  name: gvisor
handler: runsc
EOF

Verify:

vagrant@master:~$ kubectl get runtimeclass

NAME     HANDLER   AGE

gvisor   runsc     17s

Create a Pod with the gVisor RuntimeClass:

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: nginx-gvisor
spec:
  runtimeClassName: gvisor
  containers:
  - name: nginx
    image: nginx
EOF

Verify that the Pod is running:

kubectl get pod nginx-gvisor -o wide
vagrant@master:~$ kubectl get pod nginx-gvisor -o wide

NAME           READY   STATUS    RESTARTS   AGE   IP                NODE      NOMINATED NODE   READINESS GATES

nginx-gvisor   1/1     Running   0          31s   192.168.235.129   worker1   <span class="token tag"><span class="token punctuation"><</span>none<span class="token punctuation">></span></span>           <span class="token tag"><span class="token punctuation"><</span>none<span class="token punctuation">></span></span>

For updated info, follow official docs.

Read More  gVisor Improves Performance With Root Filesystem Overlay

Conclusions

We have looked that the current container technology has weak isolation issue.
Common practices such as patching containers quickly and least security contexts privilege can effectively restrict the attack surface. We should start even implementing runtime security measures like the tutorial above because now it is possible to have  more than one container runtime.

Sure, it is not something that everyone needs, but it will certainly come in handy when you want to run untrusted containers without impacting the host in any way.

Let’s consider that you are a container hosting service and start containers of different customers on the same host machine. Could you compromise other customers because of the shared context? Start thinking about how to mitigate these issues.

 

 

 

Guest post originally published on the SIGHUP blog by Alessandro Lo Manto
Source CNCF


For enquiries, product placements, sponsorships, and collaborations, connect with us at [email protected]. We'd love to hear from you!

Our humans need coffee too! Your support is highly appreciated, thank you!

aster.cloud

Related Topics
  • CNCF
  • Container Runtime
  • Kubernetes
  • SIGHUP
You May Also Like
View Post
  • Software Engineering
  • Technology

Claude 3.7 Sonnet and Claude Code

  • February 25, 2025
View Post
  • Engineering
  • Software Engineering

This Month in Julia World

  • January 17, 2025
View Post
  • Engineering
  • Software Engineering

Google Summer of Code 2025 is here!

  • January 17, 2025
View Post
  • Software Engineering

5 Books Every Beginner Programmer Should Read

  • July 25, 2024
Ruby
View Post
  • Software Engineering

How To Get Started With A Ruby On Rails Project – A Developer’s Guide

  • January 27, 2024
View Post
  • Engineering
  • Software Engineering

5 Ways Platform Engineers Can Help Developers Create Winning APIs

  • January 25, 2024
Clouds
View Post
  • Cloud-Native
  • Platforms
  • Software Engineering

Microsoft Releases Azure Migrate Assessment Tool For .NET Application

  • January 14, 2024
View Post
  • Software Engineering
  • Technology

It’s Time For Developers And Enterprises To Build With Gemini Pro

  • December 21, 2023

Stay Connected!
LATEST
  • college-of-cardinals-2025 1
    The Definitive Who’s Who of the 2025 Papal Conclave
    • May 7, 2025
  • conclave-poster-black-smoke 2
    The World Is Revalidating Itself
    • May 6, 2025
  • oracle-ibm 3
    IBM and Oracle Expand Partnership to Advance Agentic AI and Hybrid Cloud
    • May 6, 2025
  • 4
    Conclave: How A New Pope Is Chosen
    • April 25, 2025
  • Getting things done makes her feel amazing 5
    Nurturing Minds in the Digital Revolution
    • April 25, 2025
  • 6
    AI is automating our jobs – but values need to change if we are to be liberated by it
    • April 17, 2025
  • 7
    Canonical Releases Ubuntu 25.04 Plucky Puffin
    • April 17, 2025
  • 8
    United States Army Enterprise Cloud Management Agency Expands its Oracle Defense Cloud Services
    • April 15, 2025
  • 9
    Tokyo Electron and IBM Renew Collaboration for Advanced Semiconductor Technology
    • April 2, 2025
  • 10
    IBM Accelerates Momentum in the as a Service Space with Growing Portfolio of Tools Simplifying Infrastructure Management
    • March 27, 2025
about
Hello World!

We are aster.cloud. We’re created by programmers for programmers.

Our site aims to provide guides, programming tips, reviews, and interesting materials for tech people and those who want to learn in general.

We would like to hear from you.

If you have any feedback, enquiries, or sponsorship request, kindly reach out to us at:

[email protected]
Most Popular
  • 1
    Tariffs, Trump, and Other Things That Start With T – They’re Not The Problem, It’s How We Use Them
    • March 25, 2025
  • 2
    IBM contributes key open-source projects to Linux Foundation to advance AI community participation
    • March 22, 2025
  • 3
    Co-op mode: New partners driving the future of gaming with AI
    • March 22, 2025
  • 4
    Mitsubishi Motors Canada Launches AI-Powered “Intelligent Companion” to Transform the 2025 Outlander Buying Experience
    • March 10, 2025
  • PiPiPi 5
    The Unexpected Pi-Fect Deals This March 14
    • March 13, 2025
  • /
  • Technology
  • Tools
  • About
  • Contact Us

Input your search keywords and press Enter.