Containd integration with Kubernetes, replacing Docker, is ready for production

Note trans. : We have already written about containerd and other executable environments for Kubernetes. A new publication is a translation of a recent announcement of an important milestone in the development of containerd, published in the official blog of the Kubernetes project. The text is written by employees of Google and IBM, who (of course, together with Docker Inc.) make a significant contribution to the improvement of containerd.

Earlier in the blog - in the Note Containerd Brings More Container Runtime Options for Kubernetes - we presented the alpha version of containerd integration with Kubernetes. The next 6 months of development led to the fact that integration has become publicly available! This means that you can now use containerd 1.1 as an executable environment for containers in Kubernetes clusters in production.

Containerd 1.1 works with Kubernetes version 1.10 and higher, supports all the features of Kubernetes. In the Kubernetes test infrastructure, the test coverage of integration with containerd on the Google Cloud Platform is the same as the integration with Docker (see test dashboard ).

“We are very pleased to see that containerd quickly reached this significant milestone. In Alibaba Cloud, we began to actively use containerd from its early days and, thanks to its emphasis on simplicity and reliability, made containerd the container engine by default in its Serverless Kubernetes product, which places high demands on performance and stability. Containerd will undoubtedly be the main engine of the container era and will lead to the development of innovation. ” - Xinwei, staff engineer from Alibaba Cloud

Architectural improvements

The containerd integration architecture with Kubernetes was changed twice. Each of its evolutionary steps stabilized and improved the efficiency of the stack.

Containerd 1.0 - CRI-Containerd (ceased to exist)

In containerd 1.0, the interaction between Kubelet and containerd required the demon cri-containerd (we wrote about it at the end of this article - approx. Transl. ) . This daemon served requests to the Kubelet Container Runtime Interface (CRI) and used containerd to manage containers and container images accordingly. This approach eliminated one additional link in the stack, when compared to the Docker CRI implementation ( dockershim ) - see the illustration above .

However, cri-containerd and containerd 1.0 were two separate daemons interacting on GRPC. An additional daemon in this bundle made it difficult for users to understand the device, and when deploying, and also generated unnecessary overhead for interaction.

Containerd 1.1 - CRI Plugin (current version)

In containerd 1.1, the cri-containerd daemon has been converted to the containerd CRI plugin. This plugin is built into containerd 1.1 and is enabled by default. Unlike cri-containerd, the plugin interacts with containerd by directly calling the necessary functions. The new architecture has made integration more stable and productive, removing another link (GRPC) from the stack. Now containerd 1.1 can be used directly in Kubernetes, and the cri-containerd daemon is no longer required.

Performance

One of the main challenges for the release of containerd 1.1 was performance improvement. Optimizations were carried out in the field of time for launching the flow and use of resources by the daemon.

The following results are a comparison of containerd 1.1 and Docker 18.03 CE. The integration of containerd 1.1 uses the built-in CRI plugin, and the integration for Docker 18.03 CE works with dockershim. The results were generated using the benchmark of the performance of the Kubernetes nodes, which is part of the e2e tests for K8s nodes . Most of the comparison data is publicly available in the node performance dashboard .

Delay at start

The results of the 105 pod batch startup benchmark show that containerd 1.1 integration has a lower delay when starting the pod than Docker 18.03 CE with dockershim (the smaller, the better).

CPU and memory

In idle state, containerd 1.1 integration with 105 pods consumes less CPU and memory than Docker 18.03 CE integration with dockershim. Results may vary depending on the number of pods launched on the node - the number of 105 pods is selected because the default is now the maximum value for custom sweeps on the node.

As can be seen from the graphs below, containerd 1.1 Kubelet integration consumes 30.89% less CPU and 11.30% less RSS (resident set size) memory, and also 12.78% less RSS memory consumes the executable environment for containers .

Addition from the translator

It is worth noting that continues its development and another alternative solution - CRI-O . In particular, at the Open Source Summit Japan 2018 being held these days, an NTT employee presented a report with a detailed comparison of the available executable media for containers. And this is how one of his slides, comparing their performance, looks like:

crictl

The console interface (CLI) of the executable environment for containers is a useful tool for identifying problems in the system and application. When using Docker as a container environment in Kubernetes, system administrators sometimes go to the Kubernetes node to execute Docker commands, gather information about the system and / or application. For example, they can use docker ps and docker inspect to check the status of the process, docker images to get a list of images on the site, docker info to get the configuration of the executable environment for containers, etc.

For containerd and all other CRI compliant environments, such as dockershim, we recommend using crictl as a CLI alternative to Docker console commands to analyze problems on subframes, containers and container images hosted on Kubernetes nodes.

crictl is a utility offering similar capabilities to the Docker CLI and works equally for all executable environments for CRI-compatible containers. It can be found in the kubernetes-incubator / cri-tools repository; The current version is cri-tools v1.11.0 (the version is corrected for the current 3-day old release instead of v1.0.0-beta.1 mentioned in the original article - approx. transl. ) . Although the crictl utility is designed to be similar to the Docker CLI, offering a simple transition for users, it is not a complete copy. Several important differences are discussed below.

Application limitations: crictl is a tool for troubleshooting

crictl is not a replacement for docker or kubectl - its use is limited to the area of problem detection and analysis. The Docker console interface offers a rich set of commands, which makes it a very useful development tool. However, this is not the best option for troubleshooting on Kubernetes nodes. Some Docker commands (for example, docker network and docker build ) are useless for Kubernetes, and some (for example, docker rename ) can break everything. The goal of crictl is to provide enough commands to identify problems on nodes that are safe to use in the production environment.

Kubernetes Orientation

crictl offers a more comprehensive view of containers in the world of Kubernetes. The Docker console interface does not operate with Kubernetes basic concepts, such as under (pod) and namespaces ( namespace ), which hinders the visual presentation of containers and sweeps translate. ) One such example, docker ps shows obscure, long Docker container names and a list of pause containers along with application containers:

However, pause containers are part of the hearth implementation, where one such container is used for each hearth; they should not be displayed when displaying containers included in the hearth.

crictl , on the other hand, was created for Kubernetes. The utility provides different sets of commands for podov and containers. For example, crictl pods displays information about the subframes, and crictl ps only information about the containers of the application. All data is formatted in a table view:

Another example - in crictl pods there is an argument --namespace , which allows you to filter pods by namespaces defined in Kubernetes:

For more information on how to use crictl with containerd, see here:

What about the Docker Engine?

We often hear the following question: “Does the switch to containerd mean that I can no longer use the Docker Engine?”, And the short answer to it is “NO”.

Docker Engine collected on the basis of containerd. The next release of Docker Community Edition (Docker CE) will use containerd version 1.1. Accordingly, it will have a built-in and enabled CRI plugin by default. This means that users will have the opportunity to continue working with the Docker Engine for other typical scenarios, as well as the ability to configure Kubernetes to use the underlying containerd that ships with the Docker Engine and is simultaneously used by Docker Engine itself on the same node. Take a look at the architectural diagram below, showing how the same containerd is used by the Docker Engine and Kubelet :

Since containerd is used by both Kubelet and Docker Engine, users who choose to integrate with containerd will not only get all the new features for Kubernetes, improvements in performance and stability, but also the option of using the Docker Engine, as before, for other needs.

The namespace mechanism in containerd ensures that Kubelet and Docker Engine will not have access to containers and images not created by them. This means that they will not interfere with each other, as well as:

Users entering the docker ps will not see containers created in Kubernetes. To do this, use crictl ps . Conversely, users will not see containers created in the Docker CLI in Kubernetes or on the command crictl ps . The crictl create and crictl run are intended only for troubleshooting. Manual launch of bogs or containers using crictl on production nodes is not recommended.
Users entering the docker images will not see images from Kubernetes. To do this, use the crictl images command. Conversely, Kubernetes will not see the images created by the docker pull , docker load and docker build commands. To do this, use the crictl pull command, as well as ctr cri load , if you need to load an image.

Summary

Containerd 1.1 has native CRI support. It can be directly used by Kubernetes.
Containerd 1.1 is ready for production.
Containd 1.1 has good performance in terms of start-up time and utilization of system resources.
crictl is a console (CLI) utility for communicating with containerd 1.1 and other executable container environments that comply with CRIs to identify problems on a node.
Containerd 1.1 will be included in the next stable release of Docker CE. Users will be allowed to continue working with Docker in non-Kubernetes cases, and configure Kubernetes to use the underlying containerd that is included with Docker.

We want to thank everyone from Google, IBM, Docker, ZTE, ZJU and the individual developers who have contributed and made it all possible!

For a detailed list of changes in the release of containerd 1.1, see the Release Notes .

How to try

Instructions for setting up a Kubernetes cluster to use containerd as the default executable environment:

for a cluster on a GCE raised with kube-up.sh , here ;
to install a cluster from many nodes using Ansible and kubeadm - here ;
to create a cluster from scratch on Google Cloud - see Kubernetes the Hard Way ;
for manual installation from a tarball archive - here ;
for installation using LinuxKit on a local virtual machine - here .

How to contribute

CRI plugin containerd - Open Source-project on GitHub, which is part of containerd: https://github.com/containerd/cri . All proposed changes are welcome in the form of ideas, tickets, corrections. This developer getting started guide is a good starting point for making changes.