Kubernetes (k8s) has been all the rage in recent years, but many developers find it too complicated. We spoke with Leonid Sandler, co-founder and CTO of ARMO, creators of the open-source Kubernetes security solution Kubescape, about the complex world of Kubernetes and the challenges it presents.

Why do people say Kubernetes is complicated?

Leonid Sandler: When people talk about “Kubernetes”, they don’t just mean the Kubernetes software itself. They’re talking about the world Kubernetes enables, a world of applications and microservices running in containers, connecting to databases, APIs, and endpoints, with Kubernetes orchestrating it all.

That’s where the complexity becomes clear. Working with Kubernetes means abstracting out at least four major technology layers: The Linux operating system that runs inside containers, the containers themselves, the networking infrastructure that connects containers to each other and to other services, and the orchestration solution that knows when to create, destroy, health-check, scale up, down and load-balance the containers onto the real hardware.

That’s pretty complicated!

Do you think that’s a fair accusation?

Leonid Sandler: Complexity doesn’t exist in a vacuum, so the real question is “complicated compared to what?”

Kubernetes needs to be compared with the alternatives. Instead of Kubernetes, a company might have to combine multiple platforms and tools, for example, Virtual Machines (requiring fine-tuning of Linux kernel configurations), cloud communications tools to provide the networking, load balancers to handle the scalability, and firewalls to protect them.

All of this is highly complex — much more complex than Kubernetes, and involves a much broader array of tools, skills, and required expertise.

Ultimately, Kubernetes is just a reflection of the microservices and container architecture it manages. Distributed applications, with their entire microservice and container architecture, are becoming more advanced and more complicated. The complexity leads to more complicated Kubernetes setups, but without Kubernetes complexity would still be rising.

The challenge, really, is to distribute the complexity of Kubernetes.

So why do people say Kubernetes is complicated and confusing?

Leonid Sandler: Because the complexity is distributed differently. In many cases, organisations have the expertise for traditional infrastructures, especially more established organisations with legacy systems. They have a Linux team that works on distros, kernels, and images; a networking team that worries about connectivity and firewalls; an IT team that ensures the on-premises hardware or datacenters can support the scaling, etc.

Kubernetes, then, seems complex, but not because it’s more complex than the alternatives. It isn’t. It’s because more of that complexity is falling on developers and DevOps rather than a broader array of specialists.

If you compare Kubernetes to an old monolithic application, it’s complicated for developers and DevOps teams, who are now exposed to a lot of abstracted layers they didn’t have to deal with before. And that can be overwhelming.

Can you give an example of how things can be simpler with Kubernetes?

Leonid Sandler: Sure, a great example is security, actually. An old infrastructure needed Linux security, network security, application security, and more. Kubernetes security isn't just another security discipline, it’s an abstraction layer to all of these existing disciplines that can sometimes allow a more integrated, consolidated approach.

You don't need to create access policies for every tool in your infrastructure separately when you can use Kubernetes to set global rules and policies. Instead of using a bunch of scripts and checking for root/priv/restricted, everything can be done through the k8s config. A good monitoring platform can more easily get a full view of the system to tell you that this vulnerable container is worse than another one because the platform can understand which containers are exposed to the Internet or run with elevated privileges.

Kubernetes is everywhere because it’s simpler, overall, than doing all the virtualization and orchestration yourself.

How can organisations handle the new complexity of Kubernetes?

Leonid Sandler: The challenge, really, is to distribute the complexity of Kubernetes, just like the complexity of the legacy systems was distributed within a software organisation. That means thinking about where responsibilities lie. Expecting all developers to handle networking, security, Linux, and more isn’t reasonable or sensible.

The old models of organisational structure can struggle to keep up with new infrastructures like Kubernetes, so organisations need to think about how to best structure their teams to ensure the right people with the right skills are working on all of the layers abstracted by Kubernetes — Linux, containers, networking, and orchestration. That means having network configuration, Linux, security experience, etc., inside developer and DevOps teams and not as external silos.

Organisation is one part of the puzzle, but it’s also important to use the right tools too. Some of the old tools simply won’t work on Kubernetes, while others are just inappropriate for the job. Kubernetes allows whole parts of your infrastructure to be managed centrally, though it sometimes requires using the right third-party tools for CI/CD, monitoring, and security. One of the benefits of Kubernetes is that it’s completely open source, and you can often find free, open-source tools to do what you need — for example, Kubescape, the free open-source security platform for Kubernetes.

Almost every new software startup uses Kubernetes to some degree, and not just cloud applications. Kubernetes is becoming standard for training machine learning models and it’s even being installed on IoT devices on the edge.

Ultimately, Kubernetes is everywhere because it’s simpler, overall, than doing all the virtualization and orchestration yourself. That’s why companies are choosing it. Successful companies will also understand that to get the most out of Kubernetes, they need to change their structures and tools so their engineering staff can get the most out of the platform.