Understand API Deprecations

As an ongoing project, new features will move through various stages of maturity as Kubernetes continues to grow. Every feature is served by a new API that is registered to the Kubernetes API server. There are 3 stages of maturity for these APIs:

  • Alpha - These APIs serve features that are highly experimental and not recommended for any production use. Kubernetes includes these Alpha APIs but they must be enabled by setting the appropriate feature gate on the API server. Alpha APIs can either move to beta status or get removed depending on how well the feature is received and used.
  • Beta - These APIs serve features that are still in highly active development but considered ready for production use. Beta APIs and features are enabled by default. Depending on their development cycles, Beta APIs can move up through multiple versions between Kubernetes releases or regress back to Alpha for further development.
  • Stable - These APIs are highly mature and do not see too many changes between versions. Stable APIs are enabled by default. At this point, stable APIs can only be retired if their feature is no longer necessary or if a new version of the API supplants its use.

You can view all of the APIs registered with your Kubernetes API server using the kubectl api-versions command:

$ kubectl api-versions

admissionregistration.k8s.io/v1
apiextensions.k8s.io/v1
apiregistration.k8s.io/v1
apps/v1
authentication.k8s.io/v1
authorization.k8s.io/v1
autoscaling/v1
autoscaling/v2
autoscaling/v2beta1
autoscaling/v2beta2

...

$

API deprecations are a major event in the Kubernetes development cycle and come with many announcements. After a deprecation announcement, the affected features and/or APIs that get deprecated will:

  • Begin generating warnings in the CLI when invoked or submitted to the Kubernetes API server
  • Have their functionality disabled while continuing to generate warnings
  • Eventually get removed from the codebase all together, usually within 2-3 releases
$ kubectl run -o yaml --dry-run=client pod-with-reqs-limits --requests cpu=64m,memory=256Mi --limits cpu=256m,memory=512Mi --image nginx

Flag --requests has been deprecated, has no effect and will be removed in 1.24.
Flag --limits has been deprecated, has no effect and will be removed in 1.24.
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: pod-with-reqs-limits
  name: pod-with-reqs-limits
spec:
  containers:
  - image: nginx
    name: pod-with-reqs-limits
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}

$

For existing manifests, the apiVersion line must be changed to a non-deprecated API version. This can be done with kubectl-convert, an optional kubectl plugin that examines older manifests and updates them to reflect the latest API versions.

The kubectl-convert plugin must be installed separate, with instructions found here.

$ curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl-convert"

$ sudo install -o root -g root -m 0755 kubectl-convert /usr/local/bin/kubectl-convert

$

Given the following file, which describes a deployment from v1.16 (when deployments were under the old extensions API):

$ cat old-deploy.yaml

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: old-deploy
spec:
  replicas: 3
  selector:
    matchLabels:
      app: old-deploy
  template:
    metadata:
      labels:
        app: old-deploy
    spec:
      containers:
      - image: nginx
        name: nginx

$

kubectl-convert can read the file and automatically make the necessary changes to update it to the latest api (apps/v1 in 1.23):

$ kubectl-convert -f old-deploy.yaml --output-version apps/v1

apiVersion: apps/v1
kind: Deployment
metadata:
  creationTimestamp: null
  labels:
    app: old-deploy
  name: old-deploy
spec:
  progressDeadlineSeconds: 2147483647
  replicas: 3
  revisionHistoryLimit: 2147483647
  selector:
    matchLabels:
      app: old-deploy
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 1
    type: RollingUpdate
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: old-deploy
    spec:
      containers:
      - image: nginx
        imagePullPolicy: Always
        name: nginx
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30
status: {}

$

There are usually about 2-3 minor versions between a deprecation notice and the removal of a feature, so users typically have at least 6 months.

Learn more about and keep track of the API Deprecation process and how to use kubectl-convert.

Liveness Probes and Readiness Probes

A liveness probe is a health check that tells a kubelet when to restart a container. Liveness probes help catch locks where an application seems to be running but can not proceed. Implementing a liveness probe in a deployment is a start to making a self-healing application.

An application may not immediately be ready to accept traffic when its container first starts; a readiness probe informs Kubernetes when it is okay to start sending traffic to a container after it boots. For example, a container of a java application might take minutes to load and may not be ready to accept traffic until it’s fully running. In this scenario, the readiness probe mitigates long loading times.

There are three options for liveness and readiness probes:

  • exec - takes a command, an exit code of 0 is success
  • httpGet - performs an http get, a 200-399 status is good
  • tcpSocket - a successful connection to a specified port is success

Liveness and readiness probes are similarly configured for each container in a pod. For example the following pod manifest has an nginx. container with a liveness probe that runs an http get to the root path to port 80. If the nginx web server replies with a 200 - 399 code then the pod is alive. The liveness probe waits 10 seconds before the first check and periodically checks every 20 seconds.

apiVersion: v1
kind: Pod
metadata:
  name: ready-pod
  labels:
    app: ready-pod
spec:
  containers:
  - name: nginx
    image: nginx:latest
    livenessProbe:
      httpGet:
        path: /
        port: 80
      initialDelaySeconds: 10
      periodSeconds: 20

Learn more about Liveness and Readiness Probes.

Container Logging

Kubernetes retrieves container logs from a container’s standard output and standard error streams. The applications running in the container must output their logs to a place read by the container logging engine (commonly STDOUT).

You can retrieve container logs with the kubectl logs pod_name command. If the pod is a multi-container pod then the container must be specified with the -c option with the container name kubectl logs pod_name -c container_name

In this example, we create a pod called logging-pod that outputs the system date and time every second.

$ kubectl run logging-pod --image=busybox:latest --command -- /bin/sh -c 'while true; do date; sleep 1; done'

pod/logging-pod created

$

Then the container logs are retrieved with kubectl logs:

$ kubectl logs logging-pod

Wed Apr 20 16:31:45 UTC 2022
Wed Apr 20 16:31:46 UTC 2022

$

Here are other useful container logging options:

  • --previous - retrieves container logs from the previous instantiation of a container, this is helpful for containers that are crash looping
  • -f - streams container logs
  • --since - prints logs since a specific time period e.g. --since 15m
  • --timestamps - includes timestamps

Learn more about container logging.

Monitoring Applications

For the CKAD exam, the scope of monitoring applications is only within the scope of Kubernetes (including the Kubernetes metrics server). Key metrics to monitor are resources such as CPU and memory along with deployments and their running pods. Labels filter for a specific application or domain during monitoring.

The Kubernetes metrics server is not installed with the Kubernetes installation using kubeadm. Install the Kubernetes metrics-server and learn more about the metrics-server.

With the metrics server installed you can view the resources used reported by the kubelets on each pod with kubectl top pods:

$ kubectl top pods -n kube-system

NAME                                       CPU(cores)   MEMORY(bytes)   
coredns-64897985d-5gn7r                    1m           11Mi            
coredns-64897985d-vbl5w                    1m           12Mi            
etcd-ip-172-31-57-184                      15m          43Mi            
kube-apiserver-ip-172-31-57-184            52m          276Mi           
kube-controller-manager-ip-172-31-57-184   10m          45Mi            
kube-proxy-rw4nw                           2m           10Mi            
kube-scheduler-ip-172-31-57-184            3m           17Mi            
metrics-server-6f7946fdd7-dq6hd            4m           13Mi            
weave-net-tbhqd                            1m           45Mi  

Learn more about the tools for monitoring resources on Kubernetes, monitoring, and the metrics server.

Debugging

Debugging running applications in Kubernetes starts by retrieving simple status information about the pods.
Here are a few places to start looking:

  • Pod status - is the pod running, pending, or crash looping?
  • Pod restart count - does the pod have many recent restarts?
  • Pod resources - is the pod requesting more resources available in its namespace or on its node?

Pod details are retrieved by introspecting the pod with kubectl describe pod pod_name.

Take a look at how common pod issues are debugged using the pod’s description.

Here are key segments in a description of a pending pod:

$ kubectl describe pod busybox

Name:         busybox
Namespace:    default
...
Status:       Pending
IP:           10.32.0.5
IPs:
  IP:  10.32.0.5
Containers:
  myapp-container:
    Container ID:  
    Image:         busyBOX
  ...
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
...
Events:
  Type     Reason         Age              From               Message
  ----     ------         ----             ----               -------
  Normal   Scheduled      10s              default-scheduler  Successfully assigned default/busybox to ubuntu
  Warning  InspectFailed  9s (x2 over 9s)  kubelet, ubuntu     Failed to apply default image tag "busyBOX": couldn't parse image reference "busyBOX": invalid reference format: repository name must be lowercase
  Warning  Failed         9s (x2 over 9s)  kubelet, ubuntu     Error: InvalidImageName

$

The Events section of the description points out the image name is invalid. Correcting the image name will resolve this issue.

Here’s another pending pod:

$ kubectl get pods

NAME    READY   STATUS    RESTARTS   AGE
nginx   0/1     Pending   0          2m15s

$

This nginx pod has been pending for over 2 minutes.
Let’s look at the pod’s Events.

$ kubectl describe pod nginx | grep -A4 Events

Events:
  Type     Reason            Age                  From               Message
  ----     ------            ----                 ----               -------
  Warning  FailedScheduling  76s (x4 over 3m54s)  default-scheduler  0/1 nodes are available: 1 Insufficient cpu.

$

Not enough resources are available in the cluster. We can see if the pod is making a hard resource request by looking at the Containers section of the pod description.

$ kubectl describe pod nginx | grep -A10 Containers

Containers:
  nginx:
    Image:      nginx
    Port:       
    Host Port:  
    Limits:
      cpu:     2
      memory:  2Gi
    Requests:
      cpu:        1500m
      memory:     1536Mi

$

The pod is requesting 1500m of cpu. There are no nodes in the cluster with 1500m free of cpu so the pod stays pending. There are a few ways to resolve this issue: reduce the container’s cpu request, free up cpu on a cluster node, or add a new worker node to the cluster.

Learn more about application debugging.

Practice Drill

The pod defined in the problem.yaml below creates a pod meant to run a perpetual tail command in the busybox image. The pod, which runs an init container with the alpine:latest does not run as expected. Apply the manifest to your cluster, identify the problem and repair it so that the pod runs as expected.

Apply the problem.yaml manifest to your cluster using the following command:

kubectl apply -f https://raw.githubusercontent.com/RX-M/bust-a-kube/master/workload-1/problem.yaml

The pod must be in the running state with all containers ready for this problem to be considered resolved.

RX-M offers complete CKAD Boot Camps to make securing your CKAD certification manageable.

KTP-logo.svg
KCSP-logo.svg
cncf-member-silver.svg
lf-atp-logo.svg
apache-bronze-sponsor-logo.svg
devops-institute-partner-logo-2022-150.webp
scrumorg-ptn-logo-sq-150.webp
certnexus-silver-atp-logo-150.webp

Our team has been trusted to work alongside some of the world's leading companies

grey-client-logos-16-mar-2020.svg