CKAD Self-Study Mod 3
In this module of the RX-M online CKAD prep course, we will be covering the pod design and state persistence topics identified by the CNCF CKAD Exam Curriculum. If you are not already familiar with the curriculum, take a moment to familiarize yourself as you will be required to demonstrate knowledge of each topic in order to pass the exam.
Discover and Use Resources that Extend Kubernetes
A Kubernetes cluster's functionality is extended by registering additional APIs to the API Server. Custom APIs usually bring their own set of custom resources which can be specified. If your cluster has been expanded to include custom resource definitions, there are two primary ways to identify them.
First is to see the list of APIs that have been registered, which you can see with
$ kubectl api-versions admissionregistration.k8s.io/v1 apiextensions.k8s.io/v1 apiregistration.k8s.io/v1 apps/v1 authentication.k8s.io/v1 authorization.k8s.io/v1 autoscaling/v1 autoscaling/v2 autoscaling/v2beta1 autoscaling/v2beta2 batch/v1 batch/v1beta1 certificates.k8s.io/v1 coordination.k8s.io/v1 discovery.k8s.io/v1 discovery.k8s.io/v1beta1 events.k8s.io/v1 events.k8s.io/v1beta1 flowcontrol.apiserver.k8s.io/v1beta1 flowcontrol.apiserver.k8s.io/v1beta2 networking.k8s.io/v1 node.k8s.io/v1 node.k8s.io/v1beta1 policy/v1 policy/v1beta1 rbac.authorization.k8s.io/v1 scheduling.k8s.io/v1 storage.k8s.io/v1 storage.k8s.io/v1beta1 v1 $
Any additional APIs you have installed as part of various cluster extensions, like operators,
The resources available to your cluster are viewable with
kubectl api-resources, which shows the kinds of
resources you can create in a cluster:
$ kubectl api-resources NAME SHORTNAMES APIVERSION NAMESPACED KIND bindings v1 true Binding componentstatuses cs v1 false ComponentStatus configmaps cm v1 true ConfigMap endpoints ep v1 true Endpoints events ev v1 true Event limitranges limits v1 true LimitRange namespaces ns v1 false Namespace nodes no v1 false Node persistentvolumeclaims pvc v1 true PersistentVolumeClaim persistentvolumes pv v1 false PersistentVolume pods po v1 true Pod podtemplates v1 true PodTemplate replicationcontrollers rc v1 true ReplicationController resourcequotas quota v1 true ResourceQuota secrets v1 true Secret serviceaccounts sa v1 true ServiceAccount services svc v1 true Service ... $
Learn more about custom resource definitions.
Understanding Authentication, Authorization and Admission Control
Roles, ClusterRoles, RoleBinding and ClusterRoleBindings control user account permissions that control how they interact with resources deployed in the cluster. ClusterRoles and ClusterRoleBindings are non-namespaced resources. Roles and RoleBindings sets permissions and bind permissions in a specific namespace.
Kubernetes uses Role-based access control (RBAC) mechanisms to control the ability of users to perform a specific task on Kubernetes objects. Clusters bootstrapped with kubeadm have RBAC enabled by default.
Permissions to API resources are granted using Roles and ClusterRoles (the only difference being that clusterRoles apply to the entire cluster while regular roles apply to their namespace). Permissions are scoped to API resources and objects under the API resources. Verbs control what operations can be performed by each role.
Roles can be created imperatively using
kubectl create role. You can specify the API resources and verbs associated with the permissions the role will grant:
$ kubectl create role default-appmanager --resource pod,deploy,svc,ingresses --verb get,list,watch,create -o yaml apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: default-appmanager namespace: default rules: - apiGroups: - "" resources: - pods - services verbs: - get - list - watch - create - delete - apiGroups: - apps resources: - deployments verbs: - get - list - watch - create - delete $
Roles and clusterRoles are assigned to users and processes using roleBindings and clusterRoleBindings. Rolebindings associate a user, like a service account, with a role. Any permissions granted by a role are passed to the user through the rolebinding.
Rolebindings can also be created imperatively using
kubectl create rolebinding. Rolebindings bind roles to users using the
--user flag and serviceAccounts using the
--serviceaccount flag. The following example binds the default-appmanager role to the default namespace’s default service account:
$ kubectl create rolebinding default-appmanager-rb \ --serviceaccount default:default \ --role default-appmanager rolebinding.rbac.authorization.k8s.io/default-appmanager-rb created $
Learn more about configuring role-based access control.
Resource Requests and Limits
Resource requests and limits are set on a per-container basis within a pod. By specifying a resource request we tell the Kubernetes scheduler the minimum amount of each resource (CPU and memory) a container will need. By specifying limits, we set up cgroup constraints on the node where the process runs. An example of setting requests/limits looks like:
apiVersion: v1 kind: Pod metadata: name: ckad-resource-pod spec: containers: - name: ckad-resource-container image: my-app:v3.3 resources: limits: cpu: "1" memory: “1Gi” requests: cpu: "0.5" memory: “500Mi”
Learn more about pod resource requests/limits.
Users who have control over their namespaces can also define a LimitRange, which is an API object that ensures pods maintain a minimum and maximum value for certain resources. This is enforced using a validating webhook that either rejects pods whose containers violate the set resource limits for their containers or inserts a limit into all containers of a pod that do not define any resource limits.
LimitRanges must be defined in YAML, and can ensure that either CPU or memory constraints are enforced within the namespace:
apiVersion: v1 kind: LimitRange metadata: name: cpu-limit namespace: limited spec: limits: - max: cpu: "512m" min: cpu: "64m" type: Container
Once defined, the limitrange can be found within the description.
$ kubectl apply -f limitrange.yaml limitrange/cpu-limit created $ kubectl describe namespace limited Name: limited Labels: kubernetes.io/metadata.name=limited Annotations: Status: Active No resource quota. Resource Limits Type Resource Min Max Default Request Default Limit Max Limit/Request Ratio ---- -------- --- --- --------------- ------------- ----------------------- Container cpu 64m 512m 512m 512m -
Once a limitrange like the one described is in place, any pods you create will have that limit injected into their containers:
$ kubectl run -n limited --image nginx -o yaml webserver apiVersion: v1 kind: Pod metadata: annotations: kubernetes.io/limit-ranger: 'LimitRanger plugin set: cpu request for container webserver; cpu limit for container webserver' creationTimestamp: "2022-04-19T23:35:55Z" labels: run: webserver name: webserver namespace: limited resourceVersion: "105825" uid: f9132086-5b4d-4c05-a7e3-991cf8227360 spec: containers: - image: nginx imagePullPolicy: Always name: webserver resources: limits: cpu: 512m requests: cpu: 512m ... $
Learn more about LimitRanges and how they enforce resource constraints in your namespaces.
In addition to limiting resources for containers in pods, users also have options to control the resources on the Kubernetes namespace level.
Namespace quotas are API objects that place limits on:
- The number of certain resources, like pods or services, inside a namespace
- The total utilization of certain machine resources, like cpu or memory, by containers within pods of the namespace
Quotas are enforced in two different ways:
- Soft - where a warning is presented to the client if a request that violates the quota is made
- Hard - where a request that violates the quota is rejected
Quotas can be placed by defining a specification for the quota inside a given namespace, which can be done using
kubectl create quota:
$ kubectl create quota --hard pods=3 pod-limit resourcequota/pod-limit created $ kubectl describe namespace default Name: default Labels: kubernetes.io/metadata.name=default Annotations: Status: Active Resource Quotas Name: pod-limit Resource Used Hard -------- --- --- pods 5 3 No LimitRange resource. $
Once create, quotas are visible in the describe output for a given namespace.
As this quota has hard enforcement, any requests that would violate a quota in a namespace is rejected, generating an error:
$ kubectl get pods No resources found in default namespace. $ kubectl create deploy webserver --replicas=3 --image nginx deployment.apps/webserver created $ kubectl run webserver-new --image httpd Error from server (Forbidden): pods "webserver-new" is forbidden: exceeded quota: pod-limit, requested: pods=1, used: pods=3, limited: pods=3 $
Quotas are a great way of limiting the resource pools within namespaces.
Learn more about resource quotas for namespaces.
ConfigMaps are decoupled configuration artifacts keeping containerized applications portable.
The ConfigMap API resource provides mechanisms to inject containers with configuration data while
keeping containers agnostic of Kubernetes. A ConfigMap can be used to store fine-grained information like individual properties or coarse-grained information like entire config files or JSON blobs.
There are multiple ways to create a ConfigMap: from a directory upload, a file, or from literal values in command line as shown in the following example:
$ kubectl create configmap ckad-example-config --from-literal foo=bar -o yaml apiVersion: v1 data: foo: bar kind: ConfigMap metadata: name: ckad-example-config namespace: default $
Learn more about ConfigMaps.
Secrets hold sensitive information, such as passwords, OAuth tokens, and SSH keys. Putting this information in a secret is safer and more flexible than putting it verbatim in a pod definition or a Docker image!
There are three types of secrets, explained by the
$ kubectl create secret --help Create a secret using specified subcommand. Available Commands: docker-registry Create a secret for use with a Docker registry generic Create a secret from a local file, directory or literal value tls Create a TLS secret
Example of creating a secret imperatively:
$ kubectl create secret generic my-secret --from-literal=username=ckad-user --from-literal=password="Char1!3-K!10-Alpha-D31ta" -o yaml apiVersion: v1 data: password: Q2hhcjFLd2hpbGUgdHJ1ZSA7IGRvIHdoaWNoIHZpbSA7QWxwaGEtRDMxdGE= username: Y2thZC11c2Vy kind: Secret metadata: name: my-secret namespace: default type: Opaque $
Learn more about secrets.
Mounting ConfigMaps/Secrets as Volumes or Environment Variables
ConfigMaps and Secrets are mounted by Pods as either volumes or environment variables to be used by container in a Pod.
ConfigMaps and Secrets can be used with a pod in two ways:
- Files in a volume
- Environment variables
Secrets can also be used by the kubelet when pulling images for a pod, called an imagePullSecret
The following Pod manifest mounts the ConfigMap ckad-example-config as a volume to the
/etc/myapp directory in the container and uses a secret called "
ckad-training-docker-token" as an imagePullSecret:
apiVersion: v1 kind: Pod metadata: name: pod-config spec: containers: - name: nginx image: nginx:latest imagePullSecrets: - name: ckad-training-docker-token volumeMounts: - name: config mountPath: /etc/myapp volumes: - name: config configMap: name: ckad-example-config
Learn more about mounting:
Service Accounts are users managed by the Kubernetes API that provide processes in a pod with an identity in the cluster. Service Accounts are bound to a set of credentials stored as secrets in the same namespace in the cluster. Every container in a pod within a namespace inherits credentials from their designated service account.
Service Accounts are entirely managed by the API, and are created by making API calls to the Kubernetes API server.
kubectl automates the process of creating service accounts with the
create subcommand. The example below shows an imperative command that creates a serviceAccount called
ckadexample under the namespace called
$ kubectl create namespace ckadtraining $ kubectl create serviceaccount ckadexample --namespace ckadtraining
A service account has no permissions within the cluster by default. The service account must be bound to a role that defines its permissions using a rolebinding. The following example creates a role that allows our new service account to view pods within the ckadtraining namespace and a rolebinding that grants those permissions to the ckadexample SA:
$ kubectl create role ckadsarole\ --namespace ckadtraining \ --verb=get,list,watch \ --resource=pods $ kubectl create rolebinding ckadsarolebinding \ --namespace ckadtraining \ --role=mysarole \ --serviceaccount=ckadtraining:ckadexample $
Learn more about Service Accounts.
This is a setting in a PodSpec that enhances security for one or all of the containers in a pod and have the following settings:
- Discretionary Access Control - define user ID (UID) and group ID (GID) settings for processes inside containers
- Security Enhanced Linux (SELinux) - invoke predefined security labels
- Linux Capabilities - coarse-grained control of system calls to the Linux kernel in a whitelist or blacklist
- Marking a pod with privileged = true grants all capabilities
- AppArmor - invoke predefined program profiles to restrict the capabilities of individual programs
- Seccomp - Fine-grained control over a process’s system calls through the use of json policies
- AllowPrivilegeEscalation - Controls whether a process can gain more privileges than its parent
SecurityContext settings can be set for the pod and/or each container in the pod, for example:
apiVersion: v1 kind: Pod metadata: name: ckad-training-pod spec: securityContext: # pod securitycontext fsGroup: 2000 containers: - name: ckad-training-container image: nginx securityContext: # container securitycontext capabilities: add: ["NET_ADMIN"]
Learn more about SecurityContexts.
Create a pod that runs the
nginx image and uses a ServiceAccount called