As a computer scientist, I think the best thing about our job is clearly the need to keep up with the latest technological achievements. Depending on the specific area you work in, you can actually always be sure that the solutions to challenges are constantly changing. Kubernetes is no different in that regard. Let’s find out if attending the conference can fill the knowledge gaps.

Community in Bloom. A KubeCon in Amsterdam. A very special and beautiful city hosting a very special and beautiful conference. I’m proud to be part of this and to have the opportunity to write about my experience.

Before starting the trip to KubeCon, I planned to attend as many talks as possible. Several interesting talks took place at the same time, so I had to make a decision in each time slot. On the other hand, I underestimated how many interesting booths there were, so I decided spontaneously what to do. 

Each day begins with a keynote consisting of several presentations and interviews. 

First of all, two important pieces of information: KubeCon was sold out and the next KubeCon should therefore be bigger. It was also immediately announced when and where it will take place, from 19.-22. March 2024 in Paris, France.

Since the keynotes (and various presentations) have been recorded, I would like to dive deeper into two specific things here, the first is the (warning: pun) Cappucchi-Know.

The moderator was Taylor Dolezal, the Head of Ecosystem, Cloud Native Computing Foundation (CNCF), who is known for puns in his presentations. The talk was an interview with four interesting people leading the Kubernetes and cloud native efforts in their specific companies. Kasper Borg Nissen, the Lead Platform Architect from Lunar, Sabine Wolz, the Senior Product manager from Mercedes-Benz Tech Innovation, Sergiu Petean, the Head of DevOps at Allianz Direct, and Yoichi Nakamura, the Director of Hitachi.

Unlike expectation, the questions were quite deep and the answers interesting. I didn’t expect that at a keynote, they usually tend to be more high-level and not very interesting. The participants  shared their insights, experiences, and success stories with cloud native technologies. They also talked about overcoming challenges, navigating regulations, and fostering collaboration within the EU’s cloud native ecosystem.

Speaking of Mercedes Benz Tech Innovation, they also managed to win the Top End User Award. Congratulations! Well deserved!

The second keynote talk I want to mention had the name “Tales from the Cloud Native Community” by Nikhita Raghunath and Ricardo Rocha. Everybody who tries to set up a complete Kubernetes cluster with all the functionalities it needs, knows how big the Kubernetes landscape is. You will be overwhelmed by the sheer number of possible components. The presenters made one statement that stuck in my mind: “The lack of gatekeeping is what drives innovation.”

That got me thinking. My first thought was: I completely agree with that statement. But, what does this statement mean in reverse? Does it stifle or disrupt innovation if certain components are fixed such as in many company environments? I would rather say that gatekeeping focuses on innovation. Provided that certain components have been agreed upon within the team and set due to a certain degree of maturity, it is easier to focus on a distinct area together.

Each day, after the keynote sessions, I went to attend some talks and later to the booths. 

To give you a better view of the information and because of the popular divide and conquer pattern, I’ll be brief here and sort the presentations and conversations in the booths by topic:

Topic App Delivery, Build, Developer Experience

I know that Red Hat is in a strong position on this topic. With OpenShift Pipelines and OpenShift GitOps, we have developer experience friendly and hardened Kubernetes native solutions in our portfolio. In addition, we are also working on one of the trends, Backstage.

The State of Backstage in 2023 – Ben Lambert & Patrik Oldsberg, Spotify

The presenters did a project update, showed the roadmap and even did a short demo of how to create custom project templates. Backstage is quite popular, it has 1k+ contributors and 1k+ adopers, on GitHub the project has 21,5k stars. They also adapted the new project governance levels that are based upon the CNCF contributor ladder and are separated into the three project areas catalog, discoverability and TechDocs.

The current challenge is to refactor the project into using a new dependency injection model which would result in a reduction of the lines of code of the new backend from 115k to 26k (!). The result would also include a way more easy way to write project templates, which the presenters showed in a short demo.

> My opinion on this project is that it will be quite a big deal in the near future. Many companies are searching for a solution on how to implement blueprints of their software stack, so that their new projects may have an easier start and faster ROI.

Booth talk: Raftt

Another project that wants to simplify things is Raftt. They gave out the most beautiful socks of the whole conference. Pink socks showing multiple bees with elephant heads. I don’t really know why and what that has in common with app delivery, but who am I to judge?

The differentiator of this project is, it focuses on fast iterations with hot reload. I know the concept from the Quarkus dev mode, in which an instance of the java application runs locally and the new developments are almost immediately visible without having to rebuild the application. With Raftt, your application is running in a pod, your changes will be brought incrementally into the running container and no pod restart is needed. This results in very fast build times which provides, as well known for Quarkus users, a very good developer experience. 

Booth talk: Acorn

Do you know Rancher? Back then, 4 people started it and sold it to Suse. Now, these 4 people are back in town. They created Acorn, a holistic approach to application deployment. Acorn is able to package up all of an application’s Docker images, configuration, and deployment specifications into a single Acorn image artifact. This artifact is publishable to any OCI container registry allowing it to be deployed on any dev, test, or production Kubernetes. Developers create Acorn images by describing the application configuration in an Acornfile. The Acorn CLI is used to build, deploy, and operate Acorn images on any Kubernetes cluster. The container is built locally and pushed to a container registry. The deployment works via statements to the Acorn CLI to deploy the Acorn image onto an Acorn runtime, which can be installed on any Kubernetes cluster. 

> In my opinion the system can be described as a system similar to Jenkins with an helm-like application feature. The reason why I don’t like Jenkins and prefer Tekton or JenkinsX is, you don’t need/want an runtime environment on Kubernetes to build your application. Kubernetes with its Tekton functionality nowadays has a native build system built-in. You also don’t need to create your own pod and container creation runtime (just like Jenkins does with its workers) because in Kubernetes you can just use pods. With pods you are also able to use the same observability stack for your deployments as well for your built steps.

Booth talk: env0

Like many other projects, env0 tries to simplify the app delivery process. It uses webhooks to create whole automated processes and supports terraform. It provides several project templates that can be used. I asked if they provide ansible support as well, but that only works as a custom step. 

Navigating the Delivery Lifecycle with Keptn – Giovanni Liva, Dynatrace; Ana Margarita Medina, Lightstep; Brad McCoy, Basiq; Meha Bhalodiya, Red Hat

I must say, I never had the opportunity to try out keptn. It shines in showing a state of an application through their whole lifecycle, focussing on multi stage delivery. You enable the app via a label on the namespace and can use pre-deployments as well. The presenters showed a demo, where the deployment object stays in a pending state until the pre-deployment was successfully finished. 

It provides OpenTelementry tracing and common Kubernetes events, which can be used for debugging failed deployments.

> The feature to promote a successful deployment from a test stage to production looks interesting. I will try that out.

Topic Kubernetes distros (also for edge environments)

In regard to Kubernetes edge distributions, Red Hat has a good solution named Red Hat Device Edge. A colleague of mine is an expert and told me unofficially that we will announce a very big customer success story in the near future.

Booth talk: Canonical

From my computer science history at the university, I know canonical’s linux distribution ubuntu. Apart from that I switched later to Fedora, the name Canonical rang a bell. As it states, they also have a Kubernetes distribution, named Canonical Kubernetes. You can install their Kubernetes locally in a quite easy way and they also aim at edge workloads. 

Booth talk: AWS Rancher (Bastian Hoffmann)

In my previous job I worked in a public sector project where we created our own Kubernetes cluster build upon Rancher. We ran into some problems and contacted support, where I got to meet a gentleman named Bastian Hoffmann.

He helped us fix problems in our cluster and we stayed in touch. He is now in another position at Suse and showed me their AWS dedicated Rancher solution. On AWS, Red Hat provides a dedicated OpenShift and a managed offering named Red Hat OpenShift Service on AWS (ROSA), so I asked if they have a managed solution as well. It turns out, they have. You can’t buy it on the marketplace yet, currently on request only. They also have a solution which aims at edge workloads, which he showed me in detail.

Topic container scanning / policy management / security

At Red Hat we strongly trust our StackRox backed and hardened solution with the simple name Advanced Cluster Security. My personal opinion is that this solution is all you need, it scans your current security state at all times, notifies you when something went wrong, enforces policies and provides remediation recommendations. It is all the more important to think outside the box and to check what other companies have in their portfolio.

Booth talk: falco

Many projects are working on solutions leveraging eBPF (a mechanism that enables running sandboxed programs in a privileged context such as the operating system kernel, for more information check out https://ebpf.io/what-is-ebpf/). One of these is falco. It basically provides a cloud native policy management system with real time protection. The project completed a CNCF audit and is now at the incubating maturity level. Notifications can work via slack.

Booth talk: Sysdig Secure

Sysdig secure is the commercial product from the company that created falco. It provides a platform with remediation recommendations and can automatically create pull requests, if needed.

Booth talk: Mondoo

Another policy management system is mondoo. We had an extensive talk with Patrick Münch, who told us about the product. It aims to simplify the security strategy in all app lifecycle stages. You can use mondoo via VSCode extension and write RuleSets or Policies for each stage.

Booth talk: Incident.io

Incident.io is a project that wants to create visibility in the incident management process. Each incident can be created manually (e.g. via API), notifications can be pushed to many tools such as Slack or Jira. It provides a Dashboard where each incident can be managed. 

Booth talk: Veritas

Veritas is a data security company. It focuses on securing all kinds of data that is used in a cluster. I was especially interested in the backup and recovery of an OpenShift cluster and asked a lot of questions. As many of you probably know, Kubernetes doesn’t provide an automatic rollback or restore mechanism and the process itself is very complicated, where some steps must be done at a specific time (so that node starting process loads the right data and doesn’t just replicate the existing stuff). The people working at the booth took the time and we discussed the topic. I have my own OpenShift cluster that I’m managing by myself and was looking for a good backup and restore solution. Veritas now has got a high priority on my personal cluster to-do list.

Topic: Other interesting stuff

There were various other projects that were very interesting. They don’t fit into the previous topics, but they are worth mentioning nonetheless. 

I only knew Cilium as a Kubernetes CNI so far. But what was new to me is that Cilium can do much more. It also can provide multi cluster service connectivity, just like Skupper (Layer 7) or Submariner (Layer 3). It can provide enhanced observability (similar to netobserv) through prometheus compatible metrics and can enforce network policies (similar to Red Hat Advanced Cluster Security), so I think it can do way more than I thought and will try it out in the near future.

Also I learned about dapr, the distributed application runtime framework, which provides best practice building blocks via a sidecar into a container. Interesting approach with a developer focus, I will definitely try that out.

Another news was that ArgoCD and Flux got the graduated maturity level of the CNCF. Also, keycloak, the upstream project for Red Hat Single Sign On, got the incubation maturity level. Congrats to all these tools, in my previous projects I got the opportunity to try them out and we used the keycloak operator quite extensively.

Interviews at theCube

By the way, mentioning Red Hat. Our booth was always very crowded but many of us were interviewed in a central interview booth, named theCube. The interviews were recorded, here are the links:

• KubeCon + CloudNativeCon EU 2023 | Natale Vinto & Kevin Dubois | Red Hat
• KubeCon + CloudNativeCon EU 2023 | David Eads & Maciej Szulik | Red Hat 
• KubeCon + CloudNativeCon EU 2023 | Fabian Deutsch & Andrew Burden | Red Hat 

Also, a very special thanks to Sebastian Kister, who visited the KubeCon with his brand new Audi e-tron and was interviewed at theCube as well.

• KubeCon + CloudNativeCon EU 2023 | Sebastian Kister | Audi

PlayStation and Kubernetes: How to solve a problem like real-time: Joseph Irving, PlayStation

This talk was interesting because real-time is not really the most prioritized field of Kubernetes and OpenShift projects. Although OpenShift can be real-time, and be even better as a VM solution (due to removal of not needed OS components and reduced processing timeframes for the OS), most Kubernetes projects are focussing more into, what the presenter described as not real-time: Async request based, HTTPS/gRPC, non latency sensitive workload.

The central part of the talk was about Agones (https://github.com/googleforgames/agones), a framework which implements the concept of a game server which can be used as a Kubernetes CRD. Many game server instances create a fleet which makes it scalable.

> For a long time I thought that a kafka could be somewhat a real time broker solution. After getting insights about the sharding of the information, I understood that being real-time is not really the most important feature of a kafka cluster, so it was very interesting to see a system that can solve such challenges.

Booth talk: ING Bank

The ING bank had an exotic approach on how to gain interest of the people. They open sourced several products of their own Kubernetes portfolio and introduced them. I attended a talk about Skafos (https://github.com/ing-bank/skafos), a Kubernetes operator framework in python. You can scaffold all the Kubernetes operator lifecycle implementations and get an easier starting point.

They also already open sourced many other products. You can find them on their github page: https://github.com/ing-bank

Booth talk: kong

I always thought, kong is an alternative to 3scale, a system that helps you manage APIs through their development and maturity stages. I was only partially right about that.

A very patient booth employee showed me all the features that kong provides. He had multiple demos and showed me that kong is not only an API management system, it even is an API Gateway (similar to Istio) and an integration tool. He showed me the integrations part in a demo on how to add API authentication for keycloak via kong plugins. 

It was interesting to see how many things can be done via kong. You probably won’t get the functionality of the specific products with kong, but it’s nice to see a solution that simplifies things.

Booth talk: Microsoft (Matt Butcher)

One of the most daunting tasks in everyday life as a guy working with Kubernetes is, nobody in a non technical position really knows what Kubernetes is. Furthermore, even what a container really is. 

That’s a problem a gentleman named Matt Butcher also had. He was tired of explaining Kubernetes and its components everyday again. On top of that, he had children, who got into an age where they asked what their father was doing. That’s why he, Karen Chu and Bailey Beougher invented Phippy. Phippy is a giraffe, and protagonist of many childs books about the adventures of Kubernetes and their components. As I started with Kubernetes, I thought I knew and had a feeling of how and what each component of Kubernetes is. These books explained it way easier and better than a description with words. It’s really nice and heartwarming, so, if you have the time, try it out.

The thing about Matt Butcher and why I know this, is, this gentleman invented Helm. Me, as the helm fanboy that I am, use it in all my deployments. And that’s why I was the happiest helm user in the world when I got his autograph. He even let me do a selfie:

eBPF 201: Supercharging Your eBPF Dev Process for Cloud Native Apps – Sanjeev Rampal & Donald Hunter, Red Hat

This talk showed some deeper insights in eBPF, one of the technologies backing many projects at the KubeCon. The presentation starts explaining the functionality of the verifier, an application to show if your application can be run in the kernel. The presenter depicted some common use cases; observability and extending the linux networking stack. Implementing with eBPF can be done in 5 ways, the higher the level, the more unstable the API gets and the portability decreases. 

For new projects, the developers behind eBPF recommend using kfuncs. Unfortunately, kfuncs are unstable. The presenter also recommends using some further tooling and shows best practices (e.g. use a ring buffer for sending events to user space). Super interesting talk to an important technology, here are the slides.

Knative’s Road Ahead: A Project Update – Roland Huss & Naina Singh, Red Hat; Paul Schweigert, IBM; David Protasowski, VMware; Mauricio Salatino, Diagrid

Knative, the most widely-adopted serverless platform on Kubernetes, offers a streamlined developer experience for deploying and managing stateless and event-driven applications. The main benefit of knative and serverless applications is that they can save a lot of resources and costs at runtime.

Quarkus, the java framework, provides very neat knative functionality through the funqy extension (https://quarkus.io/guides/funqy-knative-events), which is on top of my todo-list of things that I want to have a look at. That’s why I was interested to know what’s on the Roadmap. The project maintainers provided a nice and short description of the three main functionalities of the knative framework; serving, eventing and functions. They talked about various improvements (e.g. longer timeouts to support websockets as well, AutoTLS) and some features they are currently working on. They also provided an overview of the knative CLI tool and its new scaffolding mechanism. 

Conclusion

I started this article with motivation and wrote that one of the nicest parts of being able to work in our profession is dealing with ever-changing technologies. The KubeCon is meant to be the Conference to solve all questions in regard to Kubernetes. 

For my part, I can say that participating in the conference brought me a lot. It allowed me to think outside the box and gave me an overview of solutions. That’s why I think that if you work in the field of Kubernetes, KubeCon is the place-to-be and I hope to be able to participate in Paris next year as well. 

And by participating, I mean not just going, but possibly giving a presentation or getting involved in one of the various CNCF projects. It remains to be seen whether I will look for a mentor or just start with the documentation of a project. But that’s another story. 

Who wants to have a look into some photos, there’s an official flickr channel.