How to pimp your Quarkus application to benefit from Kubernetes

In my last post I promised to introduce you to a selected set of Quarkus extensions. Today I will cover three of those. In my opinion they make total sense in the context of a container development or just container runtime platform (such as Red Hat OpenShift or any other Kubernetes distribution). The effort : benefit ratio is unbeatable.

Quarkus extensions are basically just dependencies of your application. But they are special. They are tailored to utilize the benefits of Quarkus to make applications faster and less resource hungry. In other words these dependencies are capable of running in native mode. You can find a complete list here.

What does change in a container world?

Your operating mode will change from an imperative to a declarative approach if using a container platform. This means the platform is in charge of operating your application. This includes monitoring and also identifying & fixing problems during runtime. This could be for example just scaling horizontally to respond to increasing demand of your service. Or it can mean restarting a failed instance because of a frozen process.

In this respect the platform needs to automatically decide on these incidents. We can help the platform making these decisions by adding some hints to our application Your business logic shouldn’t embed these non-functional requirements. It should rather be easily configurable for you. If not configurable then please just a few lines of code. And it should follow aspect oriented programming concepts: distinct code encapsulated in the corresponding concern!

Quarkus ships 2 extensions in this context:

• SmallRye Health
• SmallRye Metrics

What is your application capable of?

There is a quote which I really like: “Code is worth nothing until it is deployed!” And even if it’s deployed … what if no one knows how to use your API driven application?

I guess you already know where I am heading to: documentation. One of the things developers love to do – NOT! That’s why I think that automating this step can be a huge benefit with regards to the overall value.

Therefore I also want to quickly touch on this extension:

• SmallRye OpenAPI

Getting started with Quarkus extensions

Before we can utilize extensions we need to add them as dependencies in our Maven POM. You can do that manually. But the Quarkus Maven Plugin takes again care of that, too.

First of all you need to know about all the existing extensions. You can browse the above mentioned URL or use the command line. You can list all extensions with:

./mvnw quarkus:list-extensionsCode language: PHP (php)

Even if you don’t know the exact name of an extension, Quarkus makes an educated guess:

./mvnw quarkus:add-extension -Dextension=metric                                                                    
[INFO] Scanning for projects...
[INFO] 
[INFO] --------------------< com.redhat:greeting-quarkus >---------------------
[INFO] Building greeting-quarkus 1.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- quarkus-maven-plugin:1.9.2.Final:add-extension (default-cli) @ greeting-quarkus ---
❌ Multiple extensions matching 'metric'
     * io.quarkus:quarkus-micrometer
     * io.quarkus:quarkus-smallrye-metrics
     * org.apache.camel.quarkus:camel-quarkus-microprofile-metrics
     Be more specific e.g using the exact name or the full GAV.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  1.462 s
[INFO] Finished at: 2020-11-18T16:51:34+01:00
[INFO] ------------------------------------------------------------------------Code language: PHP (php)

You can either then add a single extension with quarkus:add-extension or add all at once:

./mvnw quarkus:add-extensions -Dextensions=quarkus-smallrye-health,quarkus-smallrye-metrics,quarkus-smallrye-openapi

[INFO] Scanning for projects...
[INFO] 
[INFO] --------------------< com.redhat:greeting-quarkus >---------------------
[INFO] Building greeting-quarkus 1.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- quarkus-maven-plugin:1.9.2.Final:add-extensions (default-cli) @ greeting-quarkus ---
✅ Extension io.quarkus:quarkus-smallrye-metrics has been installed
✅ Extension io.quarkus:quarkus-smallrye-health has been installed
✅ Extension io.quarkus:quarkus-smallrye-openapi has been installed
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  2.416 s
[INFO] Finished at: 2020-11-18T16:11:02+01:00
[INFO] ------------------------------------------------------------------------Code language: CSS (css)

Using probes to achieve zero downtime

Kubernetes provides different capabilities to check for the availability and readiness of an application:

• Startup Probe
• Readiness Probe
• Liveness Probe

Please find the explanation of each of them in the Kubernetes documentation.

Probes can be executed with different types of checks:

• HTTP Checks
• Container Execution Checks
• TCP Socket Checks

Especially the first one is highly interesting if we have a closer look at the Quarkus SmallRye Health extension.

Using the Smallrye Health Extension

Once added to your POM you will automatically have access to a /health endpoint.

$ curl localhost:8080/health

{
    "status": "UP",
    "checks": [
    ]
}Code language: JavaScript (javascript)

This endpoint can be easily added to your Kubernetes deployment as a “HTTP Check” kind of probe.

There’s even the option to differentiate between liveness and readiness probes. Each of them has a dedicated URL to check.

Adding Custom Health Checks

In this paragraph we’re gonna add some custom health check functionality before we call each of the specific check URLs.

First the readiness probe:

package com.redhat.probe;

import javax.enterprise.context.ApplicationScoped;

import org.eclipse.microprofile.health.HealthCheck;
import org.eclipse.microprofile.health.HealthCheckResponse;
import org.eclipse.microprofile.health.Readiness;

@Readiness
@ApplicationScoped
public class RProbe implements HealthCheck {

    @Override
    public HealthCheckResponse call() {
        return HealthCheckResponse.up("I'm ready!");
    }
    
}Code language: JavaScript (javascript)

Second the liveness probe:

package com.redhat.probe;

import javax.enterprise.context.ApplicationScoped;

import org.eclipse.microprofile.health.HealthCheck;
import org.eclipse.microprofile.health.HealthCheckResponse;
import org.eclipse.microprofile.health.Liveness;

@Liveness
@ApplicationScoped
public class LProbe implements HealthCheck {

    @Override
    public HealthCheckResponse call() {
        return HealthCheckResponse.up("I'm still alive!");
    }
    
}Code language: JavaScript (javascript)

Both checks will show up under the /health endpoint.

But we can also be specific now about what we want to query for:

$ curl localhost:8080/health/ready

{
    "status": "UP",
    "checks": [
        {
            "name": "I'm ready!",
            "status": "UP"
        }
    ]
}Code language: JavaScript (javascript)

Or in case we want to add a liveness probe:

$ curl localhost:8080/health/live 

{
    "status": "UP",
    "checks": [
        {
            "name": "I'm still alive!",
            "status": "UP"
        }
    ]
}Code language: JavaScript (javascript)

You can for sure add more sophisticated checks like

• check the availability of a database with a “select 1 from dual” kind of thing or
• check for the existence of a messaging queue or
• whatever your application requires to function properly
• etc etc etc

But be aware that these checks will add to the utilization of your application depending on what you’re checking, and also the entire platform.

As a best practice advice:

• test and monitor thoroughly before enabling it in production
• don’t make your checks utterly complex
• think about how often such a check needs to be executed
• and configure it appropriately!

Operations – check!

Smallrye Metrics Extension

Now that the platform is able to take care about the operations side of things, we will quickly touch on additional monitoring.

By adding the Smallrye Metrics extension we will gain access to another REST endpoint called /metrics. This one will give us a few JVM related statistics:

$ curl -s localhost:8080/metrics | grep -v -e ^#
base_classloader_loadedClasses_count 9609.0
base_classloader_loadedClasses_total 9627.0
base_classloader_unloadedClasses_total 18.0
base_cpu_availableProcessors 8.0
base_cpu_processCpuLoad_percent 8.835873647006848E-4
base_cpu_systemLoadAverage 0.41
base_gc_time_total_seconds{name="G1 Old Generation"} 0.0
base_gc_time_total_seconds{name="G1 Young Generation"} 0.074
base_gc_total{name="G1 Old Generation"} 0.0
base_gc_total{name="G1 Young Generation"} 7.0
base_jvm_uptime_seconds 3111.681
base_memory_committedHeap_bytes 3.86924544E8
base_memory_maxHeap_bytes 4.16284672E9
base_memory_usedHeap_bytes 2.01661408E8
base_thread_count 48.0
base_thread_daemon_count 7.0
base_thread_max_count 264.0
vendor_cpu_processCpuTime_seconds 10.63
vendor_cpu_systemCpuLoad_percent 0.0355565371024735
vendor_memory_committedNonHeap_bytes 7.2654848E7
vendor_memory_freePhysicalSize_bytes 3.293945856E9
vendor_memory_freeSwapSize_bytes 4.2949632E9
vendor_memory_maxNonHeap_bytes -1.0
vendor_memory_usedNonHeap_bytes 6.8187416E7
vendor_memoryPool_usage_bytes{name="CodeHeap 'non-nmethods'"} 1289216.0
vendor_memoryPool_usage_bytes{name="CodeHeap 'non-profiled nmethods'"} 1.015296E7
vendor_memoryPool_usage_bytes{name="Compressed Class Space"} 6165712.0
vendor_memoryPool_usage_bytes{name="G1 Eden Space"} 0.0
vendor_memoryPool_usage_bytes{name="G1 Old Gen"} 0.0
vendor_memoryPool_usage_bytes{name="G1 Survivor Space"} 1.6777216E7
vendor_memoryPool_usage_bytes{name="Metaspace"} 5.0579528E7
vendor_memoryPool_usage_max_bytes{name="CodeHeap 'non-nmethods'"} 1289216.0
vendor_memoryPool_usage_max_bytes{name="CodeHeap 'non-profiled nmethods'"} 1.0154112E7
vendor_memoryPool_usage_max_bytes{name="Compressed Class Space"} 6165712.0
vendor_memoryPool_usage_max_bytes{name="G1 Eden Space"} 1.44703488E8
vendor_memoryPool_usage_max_bytes{name="G1 Old Gen"} 4.8878592E7
vendor_memoryPool_usage_max_bytes{name="G1 Survivor Space"} 1.9922944E7
vendor_memoryPool_usage_max_bytes{name="Metaspace"} 5.0579528E7Code language: PHP (php)

These metrics can easily be scraped by Prometheus. Therefore Grafana can use this specific Prometheus datasource and graph all values of interest to you. Nice graphs, happy management!

Monitoring – check!

Smallrye OpenAPI Extension

There’s only one thing left before closing that we need to look at: documentation. Adding this capability to your application is the same as we’ve seen twice before. We’re adding the Smallrye OpenAPI extension to our project.

This dependency will provide us with multiple endpoints. The first one is to be used on command line and is reachable at /openapi.

$ curl -s localhost:8080/openapi                
---
openapi: 3.0.3
info:
  title: Generated API
  version: "1.0"
paths:
  /hello:
    get:
      responses:
        "200":
          description: OK
          content:
            text/plain:
              schema:
                type: stringCode language: PHP (php)

Now we can add additional documentation to our source code and have them generated into the output. You can use MicroProfile OpenAPI annotations to provide this kind of information:

package com.redhat;

import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;

import org.eclipse.microprofile.openapi.annotations.Operation;

@Path("/hello")
public class GreetingResource {

    @GET
    @Produces(MediaType.TEXT_PLAIN)
    @Operation(summary = "Greets people!", description = "This is a very friendly method and says hello to everyone!")
    
    public String hello() {
        return "hello";
    }
}Code language: CSS (css)

If you aren’t best friends with YAML you can also enjoy the JSON format. You just need to provide an Accept HTTP header:

$ curl -s -H "Accept: application/json" localhost:8080/openapi
{
  "openapi" : "3.0.3",
  "info" : {
    "title" : "Generated API",
    "version" : "1.0"
  },
  "paths" : {
    "/hello" : {
      "get" : {
        "summary" : "Greets people!",
        "description" : "This is a very friendly method and says hello to everyone!",
        "responses" : {
          "200" : {
            "description" : "OK",
            "content" : {
              "text/plain" : {
                "schema" : {
                  "type" : "string"
                }
...
}Code language: JavaScript (javascript)

This will give you an OpenAPI version 3 compliant representation of your REST API. You can utilise this information to e. g. feed this into some sort of API management like 3scale via REST and automate your entire application lifecycle.

That’s one side of the story. Another is Swagger. Swagger is the de facto standard if we’re talking about browsing API documentation.

And here are the good news: Quarkus already has it. Swagger UI is implicitly available at /swagger-ui.

There is much more to this topic. Please check the Quarkus guide on OpenAPI for more details.

Documentation – check!

What’s next?

In the next and last post of this series we will bring all that together which we’ve learned and seen so far. We’re gonna deploy this thing to OpenShift – in no time and backed by Git! We will also briefly touch on CI/CD in combination with Source-To-Image (S2I), chained builds using UBI base images, creating minimal images, using our rich OpenShift marketplace with software-defined operators and last but not least utilize all the great things of Quarkus with Serverless on OpenShift!