55 KiB
CKA study guide
The 27 objectives of the Certified Kubernetes Administrator curriculum, in its own words and order: what to know, the fastest commands, the traps, the documentation pages the exam lets you open, and the tasks in this simulator that practise each one.
Generated by ./exam --track cka notes all --md from tracks/cka/curriculum.py - edit that file, not this one.
./exam objectives # the path and your progress
./exam study 3.2 # one objective: these notes, then its tasks, no clock, solutions open
./exam drill 3.2 # the same tasks against the clock
Storage (10%)
1.1 Implement storage classes and dynamic volume provisioning
./exam study 1.1 (or storageclasses)
Know
- A StorageClass names a provisioner and its settings. A claim that asks for the class gets a PersistentVolume made for it - that is dynamic provisioning.
- Fields: provisioner, reclaimPolicy (Delete by default, or Retain), volumeBindingMode (Immediate or WaitForFirstConsumer), allowVolumeExpansion, parameters.
- The default class carries the annotation storageclass.kubernetes.io/is-default-class: "true"; a claim with no storageClassName gets it.
storageClassName: ""asks for no class at all. - A StatefulSet's volumeClaimTemplates make one claim per replica (data-web-0, data-web-1 ...).
Fast
k get sc # (default) marks the default class
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata: {name: fast-local}
provisioner: rancher.io/local-path
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer
---
k patch sc standard -p '{"metadata":{"annotations":{"storageclass.kubernetes.io/is-default-class":"false"}}}'
k patch sc fast-local -p '{"metadata":{"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'
k get pvc,pv; k describe pvc data # the Events say why a claim is Pending
Traps
- With WaitForFirstConsumer a claim stays Pending until a Pod uses it. That is not an error.
- Leave exactly one class marked default: take the annotation off the old one as well as putting it on the new one.
- provisioner, reclaimPolicy and volumeBindingMode cannot be edited - delete the class and create it again.
Docs
- https://kubernetes.io/docs/concepts/storage/storage-classes/
- https://kubernetes.io/docs/concepts/storage/dynamic-provisioning/
- https://kubernetes.io/docs/tasks/administer-cluster/change-default-storage-class/
Tasks
- Q16 Default StorageClass and dynamic provisioning
- Q36 Data survives the pod
- Q59 Migrate data between claims
- Q99 Per-replica storage for a StatefulSet (Simulator Set D (killer.sh-style))
1.2 Configure volume types, access modes and reclaim policies
./exam study 1.2 (or volumes)
Know
- Volume types you meet: hostPath (a directory of whatever node the Pod lands on), local (a disk of one node, needs nodeAffinity), emptyDir, nfs, csi, and the configMap / secret / projected volumes.
- Access modes: ReadWriteOnce (one node), ReadOnlyMany, ReadWriteMany, ReadWriteOncePod (one Pod). A claim binds only to a volume that offers its mode.
- The reclaim policy is on the PersistentVolume: Retain keeps the volume and its data when the claim goes (it becomes Released), Delete removes both.
Fast
apiVersion: v1
kind: PersistentVolume
metadata: {name: pv-data}
spec:
capacity: {storage: 1Gi}
accessModes: [ReadWriteOnce]
persistentVolumeReclaimPolicy: Retain
storageClassName: manual
hostPath: {path: /mnt/data}
---
k patch pv pv-data -p '{"spec":{"persistentVolumeReclaimPolicy":"Retain"}}'
k patch pv pv-data --type=json -p '[{"op":"remove","path":"/spec/claimRef"}]' # Released -> Available
k get pv -o custom-columns=NAME:.metadata.name,MODES:.spec.accessModes,RECLAIM:.spec.persistentVolumeReclaimPolicy,STATUS:.status.phase,CLAIM:.spec.claimRef.name
Traps
- A Released volume never binds again by itself: remove its claimRef first.
- A
localvolume needs spec.nodeAffinity; a hostPath volume does not pin the Pod to the node that has the data. - The local-path provisioner of this cluster only does ReadWriteOnce and ReadWriteOncePod - a ReadWriteMany claim stays Pending.
Docs
- https://kubernetes.io/docs/concepts/storage/persistent-volumes/
- https://kubernetes.io/docs/concepts/storage/volumes/
- https://kubernetes.io/docs/tasks/administer-cluster/change-pv-reclaim-policy/
Tasks
- Q15 PersistentVolume, claim and pod
- Q35 Re-bind a Released PersistentVolume
- Q58 Bind the right local volume
- Q68 Provision geo-fence persistent storage (Simulator Set C)
- Q72 Migrate ride history to a Retain volume (Simulator Set C)
- Q102 A claim that can never bind (Simulator Set D (killer.sh-style))
1.3 Manage persistent volumes and persistent volume claims
./exam study 1.3 (or claims)
Know
- A claim binds to one volume, and only if class, access mode and size fit (the volume may be larger);
volumeNameor a selector narrows it to one volume. - Phases: a volume goes Available, Bound, Released; a claim Pending, Bound.
- A Pod mounts a claim of its own Namespace: volumes[].persistentVolumeClaim.claimName, then volumeMounts.
- A claim in use is protected: deleting it waits (Terminating) until no Pod references it, finished Pods included.
Fast
apiVersion: v1
kind: PersistentVolumeClaim
metadata: {name: pvc-data, namespace: storage}
spec:
accessModes: [ReadWriteOnce]
storageClassName: manual # omit for the default class
resources: {requests: {storage: 500Mi}}
--- # in the Pod:
volumes: [{name: data, persistentVolumeClaim: {claimName: pvc-data}}]
containers: [{..., volumeMounts: [{name: data, mountPath: /data}]}]
k get pvc,pv -A; k describe pvc pvc-data # Used By: and Events:
k describe pod POD | tail # unbound claim -> the Pod stays Pending
Traps
- There is no
kubectl create pvcorcreate pv- copy both from the docs. - For a hand-made volume the claim's storageClassName has to equal the volume's exactly; leaving it out means the default class, not none.
- Apart from growing the request (and only if the class allows expansion) a claim cannot be edited.
- To move data between claims, run a Pod or Job that mounts both.
Docs
- https://kubernetes.io/docs/concepts/storage/persistent-volumes/
- https://kubernetes.io/docs/tutorials/configuration/configure-persistent-volume-storage/
Tasks
- Q15 PersistentVolume, claim and pod
- Q36 Data survives the pod
- Q35 Re-bind a Released PersistentVolume
- Q58 Bind the right local volume
- Q59 Migrate data between claims
- Q68 Provision geo-fence persistent storage (Simulator Set C)
- Q72 Migrate ride history to a Retain volume (Simulator Set C)
- Q81 Fix the fare calculation pods (Simulator Set C)
- Q99 Per-replica storage for a StatefulSet (Simulator Set D (killer.sh-style))
- Q102 A claim that can never bind (Simulator Set D (killer.sh-style))
Troubleshooting (30%)
2.1 Troubleshoot clusters and nodes
./exam study 2.1 (or nodes)
Know
- Start from the cluster:
k get nodes, thenk describe node- Conditions, Taints, Unschedulable, Events. - A NotReady node is nearly always its kubelet: on the node,
systemctl status kubeletandjournalctl -u kubeletname the cause. - What the kubelet needs: its unit and drop-in (which binary, which flags), /var/lib/kubelet/config.yaml, /etc/kubernetes/kubelet.conf (its kubeconfig), its certificates in /var/lib/kubelet/pki, and a running container runtime.
- Ready but SchedulingDisabled is a cordon, not a fault:
k uncordon.
Fast
k get nodes -o wide; k describe node NODE | sed -n '/Conditions/,/Addresses/p'
ssh NODE
systemctl status kubelet; journalctl -u kubelet --no-pager | tail -30
systemctl cat kubelet # the unit and its drop-ins: ExecStart, EnvironmentFile
cat /var/lib/kubelet/config.yaml; cat /var/lib/kubelet/kubeadm-flags.env
systemctl daemon-reload; systemctl enable --now kubelet; systemctl restart kubelet
systemctl status containerd; crictl ps -a
k uncordon NODE; k describe node NODE | grep -i taint
Traps
- A kubelet that was stopped is often disabled too:
enable --now, or it is gone again after a reboot. - After changing a unit file nothing happens until
systemctl daemon-reload. - There may be more than one fault. After every fix read the journal again instead of assuming you are done.
Docs
- https://kubernetes.io/docs/tasks/debug/debug-cluster/
- https://kubernetes.io/docs/tasks/debug/debug-cluster/crictl/
- https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/
Tasks
- Q17 Node NotReady
- Q39 kubelet misconfiguration
- Q60 Worker node: three things wrong
- Q92 Kubelet unit points at nothing (Simulator Set D (killer.sh-style))
- Q90 Event forensics around kube-proxy (Simulator Set D (killer.sh-style))
- Q95 The kubelet's two certificates (Simulator Set D (killer.sh-style))
2.2 Troubleshoot cluster components
./exam study 2.2 (or components)
Know
- kube-apiserver, kube-scheduler, kube-controller-manager and etcd run as static Pods: the kubelet starts whatever is in /etc/kubernetes/manifests and restarts it when the file changes.
- Read the symptom: Pods stay Pending with no events - scheduler. Deployments make no ReplicaSet or Pods - controller-manager. kubectl is refused - API server (or etcd behind it).
- With the API server up:
k -n kube-system get pods,logs,describe. With it down:crictl ps -a,crictl logs, /var/log/pods and the kubelet's journal on the node.
Fast
k -n kube-system get pods; k -n kube-system logs kube-scheduler-NODE; k -n kube-system describe pod kube-scheduler-NODE
ssh CONTROL-PLANE
ls /etc/kubernetes/manifests; vi /etc/kubernetes/manifests/kube-scheduler.yaml
crictl ps -a | grep -E 'kube-|etcd'; crictl logs CONTAINER-ID
ls /var/log/pods/kube-system_kube-apiserver-*/kube-apiserver/ # its logs when kubectl cannot ask for them
journalctl -u kubelet --no-pager | grep -i -E 'manifest|apiserver' | tail
Traps
- The faults are small: a misspelt image, binary or flag, a wrong path in a volume or certificate flag, a wrong port in a probe.
- Do not leave a backup copy inside the manifests directory - the kubelet runs every file there.
- After saving a manifest give the kubelet up to a minute before deciding the fix did not work.
Docs
- https://kubernetes.io/docs/tasks/debug/debug-cluster/
- https://kubernetes.io/docs/concepts/overview/components/
- https://kubernetes.io/docs/tasks/configure-pod-container/static-pod/
Tasks
- Q20 Control plane component failure
- Q38 Deployments never create pods
- Q61 Two control-plane components down
- Q66 Nothing works: kube-apiserver itself is down
- Q97 Stop the scheduler, schedule by hand (Simulator Set D (killer.sh-style))
- Q86 How the control plane is run (Simulator Set D (killer.sh-style))
2.3 Monitor cluster and application resource usage
./exam study 2.3 (or usage)
Know
kubectl top node|podshows live CPU and memory from metrics-server through the metrics API (metrics.k8s.io);--containerssplits a Pod up,--sort-by=cpu|memoryorders it.k describe nodeshows what is allocated - the sum of requests and limits - which is not what is used.kubectl getsorts and shapes everything else: --sort-by, -o custom-columns, -o jsonpath, -A.
Fast
k top node; k top pod -A --sort-by=memory | head; k top pod -n NS --containers
k describe node NODE | grep -A8 'Allocated resources'
k get pods -A --sort-by='.status.containerStatuses[0].restartCount'
k -n kube-system get deploy metrics-server; k get apiservice v1beta1.metrics.k8s.io
k -n kube-system logs deploy/metrics-server
Traps
Metrics API not availablemeans metrics-server is missing or not Ready (on this cluster it needs --kubelet-insecure-tls).kubectl top --sort-byputs the largest first;kubectl get --sort-byputs it last.- A new Pod has no metrics for about a minute.
Docs
- https://kubernetes.io/docs/tasks/debug/debug-cluster/resource-metrics-pipeline/
- https://kubernetes.io/docs/tasks/debug/debug-cluster/resource-usage-monitoring/
- https://kubernetes.io/docs/reference/kubectl/quick-reference/
Tasks
- Q103 Find the heaviest Pods with kubectl top
- Q23 metrics-server repair and HPA
- Q53 Namespace audit to exact files
- Q85 Investigate vehicle tracker health (Simulator Set C)
- Q89 Sorting done by kubectl (Simulator Set D (killer.sh-style))
2.4 Manage and evaluate container output streams
./exam study 2.4 (or logs)
Know
kubectl logsshows what a container wrote to stdout and stderr. The runtime keeps it on the node under /var/log/pods/NAMESPACE_POD_UID/CONTAINER/, and it goes when the Pod is deleted.- Pick the container with -c (init containers too) or take --all-containers; --previous is the run before the last restart.
- Narrow with --tail=N, --since=10m, --timestamps; follow with -f; take every Pod of a label with -l and tell them apart with --prefix.
- On a node, without the API server:
crictl logs CONTAINER-ID.
Fast
k logs POD -c CONTAINER # --all-containers --prefix
k logs POD --previous # the crashed run
k logs POD --tail=50 --since=10m --timestamps
k logs -l app=web --prefix --tail=5 # every Pod with the label
k logs deploy/web; k logs job/render # one Pod of it
k logs POD | grep ERROR > errors.log
crictl ps -a; crictl logs --tail=20 CONTAINER-ID; ls /var/log/pods/
Traps
- With -l, only the last 10 lines of each Pod are shown unless you set --tail (--tail=-1 for everything).
kubectl logs deploy/xreads one Pod, not all replicas.- --previous fails when the container has never restarted.
Docs
- https://kubernetes.io/docs/concepts/cluster-administration/logging/
- https://kubernetes.io/docs/reference/kubectl/quick-reference/
- https://kubernetes.io/docs/tasks/debug/debug-cluster/crictl/
Tasks
- Q19 CrashLoopBackOff diagnosis
- Q104 One replica is logging errors
- Q8 Multi-container pod with shared volume
- Q85 Investigate vehicle tracker health (Simulator Set C)
2.5 Troubleshoot services and networking
./exam study 2.5 (or netdebug)
Know
- Work outwards from the Pod: is it Ready, does the Service have endpoints (selector against Pod labels), is targetPort the port the container listens on, does the name resolve, does a NetworkPolicy stand in the way.
- DNS is a Service like any other: the CoreDNS Pods, the kube-dns Service and its endpoints, the Corefile in ConfigMap coredns.
- Underneath: kube-proxy (a DaemonSet) turns Services into rules on every node, and the CNI plugin connects the Pods.
Fast
k get svc,endpointslices -o wide; k describe svc SVC # Endpoints: <none> = nothing selected
k get pods -l KEY=VALUE --show-labels -o wide
k run tmp --rm -it --restart=Never --image=busybox:1.36 -- sh # wget -qO- -T3 http://SVC:PORT ; nslookup SVC.NS.svc.cluster.local
k -n kube-system get pods -l k8s-app=kube-dns -o wide; k -n kube-system logs -l k8s-app=kube-dns
k -n kube-system get cm coredns -o yaml; k -n kube-system get svc,endpointslices -l k8s-app=kube-dns
k -n kube-system get ds kube-proxy; k -n kube-system logs ds/kube-proxy
k get netpol -A
Traps
- A Service with no endpoints still resolves - it is the connection that fails.
- Test names with the fully qualified form (SVC.NS.svc.cluster.local): busybox's nslookup misreports short ones.
- A NetworkPolicy in kube-system can cut DNS off for the whole cluster.
- Fix what is broken and leave the rest alone - the graders check what you did not have to touch.
Docs
- https://kubernetes.io/docs/tasks/debug/debug-application/debug-service/
- https://kubernetes.io/docs/tasks/administer-cluster/dns-debugging-resolution/
Tasks
- Q18 Broken deployment and service
- Q62 Application down for four reasons
- Q37 Cluster DNS is broken
- Q57 DNS down: three faults
- Q73 Fix ETA service connectivity (Simulator Set C)
- Q101 Resolves everything except the cluster (Simulator Set D (killer.sh-style))
- Q70 Fix the driver onboarding app (Simulator Set C)
- Q90 Event forensics around kube-proxy (Simulator Set D (killer.sh-style))
Workloads & Scheduling (15%)
3.1 Understand application deployments and how to perform rolling update and rollbacks
./exam study 3.1 (or rollouts)
Know
- Any change to
spec.templatestarts a rollout: a new ReplicaSet scales up while the old one scales down, within maxSurge and maxUnavailable (25% each by default). - Every rollout is a revision.
kubectl rollout: status, history, undo, pause, resume, restart. - minReadySeconds, revisionHistoryLimit and progressDeadlineSeconds tune how careful and how patient a rollout is.
Fast
k set image deploy/web nginx=nginx:1.27 # CONTAINER=IMAGE
k scale deploy/web --replicas=4
k rollout status deploy/web
k rollout history deploy/web; k rollout history deploy/web --revision=2 # that revision's template
k rollout undo deploy/web --to-revision=2
k rollout pause deploy/web; k rollout resume deploy/web
k patch deploy web -p '{"spec":{"strategy":{"rollingUpdate":{"maxSurge":1,"maxUnavailable":0}}}}'
Traps
- To go back to a particular image, look at each revision's template first - "the previous one" is not always the one asked for.
- An undo makes a new revision out of the old one, so the numbers keep climbing.
- A paused Deployment records changes but rolls nothing out until it is resumed.
- maxSurge and maxUnavailable cannot both be 0.
Docs
Tasks
- Q7 Rolling update & scaling
- Q51 Rollback to a specific revision, then pause
3.2 Use ConfigMaps and Secrets to configure applications
./exam study 3.2 (or config)
Know
- Both hold keys. A key becomes an environment variable (configMapKeyRef / secretKeyRef, or every key at once with envFrom) or a file (a configMap / secret volume;
itemspicks and renames keys). - Secret data is base64 - an encoding, not encryption. Types: Opaque, kubernetes.io/tls, kubernetes.io/dockerconfigjson.
- Mounted files follow a change after a short delay. Environment variables and subPath mounts never do: restart the Pods.
immutable: truefreezes one for good. To change the content, make a new object and point the workload at it.
Fast
k create cm app-config --from-literal=APP_MODE=production --from-file=app.properties
k create secret generic db-creds --from-literal=username=admin --from-literal=password='S3cr3t!'
k create secret tls web-tls --cert=tls.crt --key=tls.key
k get secret db-creds -o jsonpath='{.data.password}' | base64 -d
env:
- {name: APP_MODE, valueFrom: {configMapKeyRef: {name: app-config, key: APP_MODE}}}
- {name: DB_PASS, valueFrom: {secretKeyRef: {name: db-creds, key: password}}}
envFrom: [{configMapRef: {name: app-config}}, {secretRef: {name: db-creds}}]
volumes: [{name: creds, secret: {secretName: db-creds, defaultMode: 0400}}]
k set env deploy/web --from=configmap/app-config; k rollout restart deploy/web
Traps
- A secret volume names it with
secretName; a configMap volume usesname. - A Pod that references a missing ConfigMap, Secret or key waits in CreateContainerConfigError.
--from-filemakes one key named after the file;--from-env-filemakes one key per line.
Docs
- https://kubernetes.io/docs/concepts/configuration/configmap/
- https://kubernetes.io/docs/concepts/configuration/secret/
- https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/
Tasks
- Q10 ConfigMap and Secret consumption
- Q29 Immutable ConfigMap rotation
- Q79 Surge pricing configuration (Simulator Set C)
- Q93 Secrets as a read-only volume and renamed env vars (Simulator Set D (killer.sh-style))
- Q84 Enforce TLS 1.3 on the booking portal (Simulator Set C)
3.3 Configure workload autoscaling
./exam study 3.3 (or autoscaling)
Know
- A HorizontalPodAutoscaler (autoscaling/v2) keeps a workload between minReplicas and maxReplicas by a metric - usually average CPU or memory utilisation, which is usage divided by the container's request.
- It reads the metrics API, so it needs metrics-server, and the containers need a request for the resource it scales on.
behaviorshapes the change: scaleUp and scaleDown each take a stabilizationWindowSeconds and policies (so many Pods or Percent per periodSeconds).
Fast
k set resources deploy web --requests=cpu=100m
k autoscale deploy web --min=2 --max=6 --cpu=60% # kubectl before 1.34: --cpu-percent=60
k get hpa; k describe hpa web
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata: {name: web}
spec:
scaleTargetRef: {apiVersion: apps/v1, kind: Deployment, name: web}
minReplicas: 2
maxReplicas: 6
metrics:
- type: Resource
resource: {name: cpu, target: {type: Utilization, averageUtilization: 60}}
behavior:
scaleDown:
stabilizationWindowSeconds: 300
policies: [{type: Pods, value: 1, periodSeconds: 60}]
Traps
- TARGETS
<unknown>: no metrics-server, or no request on the containers. kubectl autoscalecannot writebehavior- edit the object or write the YAML.- minReplicas is enforced at once, even without metrics.
Docs
- https://kubernetes.io/docs/concepts/workloads/autoscaling/horizontal-pod-autoscale/
- https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/
- https://kubernetes.io/docs/concepts/workloads/autoscaling/
Tasks
- Q105 Autoscale on CPU, scale down calmly
- Q23 metrics-server repair and HPA
- Q82 Autoscale the route planner on memory (Simulator Set C)
3.4 Understand the primitives used to create robust, self-healing, application deployments
./exam study 3.4 (or selfhealing)
Know
- Controllers put back what is missing: a Deployment its replicas, a DaemonSet one Pod per node, a StatefulSet named replicas with their own storage, a Job its completions (backoffLimit retries), a CronJob its Jobs.
- Probes tell the kubelet about health: liveness restarts the container, readiness takes the Pod out of Service endpoints, startup holds both off until the app is up.
- Init containers run to completion, in order, before the app starts; a native sidecar (an init container with restartPolicy: Always) keeps running beside it.
- A PodDisruptionBudget (minAvailable or maxUnavailable) limits how many Pods a drain may take away at once.
Fast
containers:
- name: app
startupProbe: {httpGet: {path: /, port: 80}, periodSeconds: 5, failureThreshold: 12}
readinessProbe: {httpGet: {path: /ready, port: 8080}, periodSeconds: 5}
livenessProbe: {tcpSocket: {port: 80}, initialDelaySeconds: 15, periodSeconds: 10}
# exec: {exec: {command: [cat, /tmp/ready]}}
k create job render --image=busybox:1.36 -- sh -c 'echo done'
k create cronjob digest --image=busybox:1.36 --schedule='*/5 * * * *' -- date; k create job run-1 --from=cronjob/digest
k create pdb web --selector=app=web --min-available=2
k create deploy x --image=busybox:1.36 $do > ds.yaml # DaemonSet: change kind, drop replicas / strategy / status
k describe pod POD | grep -E 'Liveness|Readiness|Startup'; k get events --field-selector reason=Unhealthy
Traps
- Probes belong to a container. No kubectl flag adds one - edit the YAML.
- Restarts with Unhealthy and Killing events but a clean log: the liveness probe is wrong, not the app.
- A DaemonSet reaches control-plane nodes only with a toleration for node-role.kubernetes.io/control-plane.
- A Job's Pod template cannot be edited: delete the Job and create it again.
Docs
- https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/
- https://kubernetes.io/docs/concepts/workloads/controllers/deployment/
- https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/
- https://kubernetes.io/docs/tasks/run-application/configure-pdb/
Tasks
- Q106 Health checks that check the wrong thing
- Q8 Multi-container pod with shared volume
- Q28 Init container prepares content
- Q26 DaemonSet on every node
- Q27 CronJob with a manual run
- Q31 Parallel Job
- Q52 Native sidecar with probes and an exact report
- Q53 Namespace audit to exact files
- Q76 Add an init container to the trip logger (Simulator Set C)
- Q88 Ready only when its dependency answers (Simulator Set D (killer.sh-style))
- Q70 Fix the driver onboarding app (Simulator Set C)
3.5 Configure Pod admission and scheduling (limits, node affinity, etc.)
./exam study 3.5 (or scheduling)
Know
- Requests are what the scheduler reserves, limits what the node enforces. A LimitRange gives containers defaults and bounds; a ResourceQuota caps a Namespace's totals - both act at admission.
- Where a Pod may go: nodeSelector, node affinity (required or preferred), Pod affinity and anti-affinity with a topologyKey, topologySpreadConstraints, or nodeName, which skips the scheduler.
- Taints keep Pods off a node (NoSchedule, PreferNoSchedule, NoExecute); a toleration lets a Pod through.
- A PriorityClass orders Pods when room is short; the Pod names it in priorityClassName.
Fast
k label node NODE disktype=ssd
k taint node NODE dedicated=gpu:NoSchedule; k taint node NODE dedicated- # the trailing - removes it
tolerations: [{key: dedicated, operator: Equal, value: gpu, effect: NoSchedule}]
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms: [{matchExpressions: [{key: disktype, operator: In, values: [ssd]}]}]
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- {labelSelector: {matchLabels: {app: web}}, topologyKey: kubernetes.io/hostname}
k create priorityclass high --value=100000 --description='critical workloads'
k create quota team -n NS --hard=pods=5,requests.cpu=1,limits.memory=2Gi
k describe pod POD | tail # FailedScheduling: 0/4 nodes are available: ...
Traps
- A toleration lets a Pod onto a tainted node but does not send it there - pair it with a nodeSelector or affinity.
- Once a quota covers CPU or memory, every new Pod must state them or get them from a LimitRange.
- In a LimitRange,
defaultis the limit anddefaultRequestthe request. - Scheduling fields of a running Pod cannot be changed: change the Deployment, or delete and recreate the Pod.
Docs
- https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/
- https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/
- https://kubernetes.io/docs/concepts/scheduling-eviction/pod-priority-preemption/
- https://kubernetes.io/docs/concepts/policy/resource-quotas/
Tasks
- Q9 Scheduling with node affinity and resources
- Q25 Taints and tolerations
- Q24 PriorityClass
- Q32 ResourceQuota and LimitRange
- Q50 Spread, tolerate, prioritise — under a quota
- Q78 Priority for the rider support portal (Simulator Set C)
- Q96 A Pod for the control plane only (Simulator Set D (killer.sh-style))
- Q81 Fix the fare calculation pods (Simulator Set C)
- Q97 Stop the scheduler, schedule by hand (Simulator Set D (killer.sh-style))
Cluster Architecture, Installation & Configuration (25%)
4.1 Manage role based access control (RBAC)
./exam study 4.1 (or rbac)
Know
- A Role (one Namespace) or ClusterRole (everywhere, and cluster-scoped resources) lists verbs on resources; a RoleBinding or ClusterRoleBinding grants it to users, groups or ServiceAccounts.
- A RoleBinding may point at a ClusterRole: the rules then apply in that one Namespace only.
- A user is a certificate: CN is the name, O the groups. Create a key and CSR, submit a CertificateSigningRequest, approve it, put the certificate in a kubeconfig.
- An aggregated ClusterRole has no rules of its own: the controller fills them in from every ClusterRole carrying the label it selects.
Fast
k create role deploy-manager -n NS --verb=get,list,watch,create,update,patch,delete --resource=deployments.apps
k create rolebinding deploy-bot -n NS --role=deploy-manager --serviceaccount=NS:deploy-bot # --user=jane --group=devs
k create clusterrole node-reader --verb=get,list,watch --resource=nodes
k create clusterrolebinding node-reader --clusterrole=node-reader --serviceaccount=NS:SA
k auth can-i create deployments -n NS --as=system:serviceaccount:NS:deploy-bot; k auth can-i --list --as=jane -n NS
openssl genrsa -out jane.key 2048; openssl req -new -key jane.key -subj '/CN=jane/O=developers' -out jane.csr
# CertificateSigningRequest: spec.request = base64 -w0 of the .csr, signerName kubernetes.io/kube-apiserver-client, usages [client auth]
k certificate approve jane; k get csr jane -o jsonpath='{.status.certificate}' | base64 -d > jane.crt
k config set-credentials jane --client-certificate=jane.crt --client-key=jane.key --embed-certs
Traps
- "Exactly these permissions" also means finding what is already granted:
k get rolebindings,clusterrolebindings -A -o wide | grep NAME. - Deployments are in the group apps: --resource=deployments.apps; a subresource is deployments.apps/scale, pods/log.
- The same ServiceAccount is
--serviceaccount=NS:NAMEin a binding andsystem:serviceaccount:NS:NAMEafter--as. - Nodes, PersistentVolumes and Namespaces are cluster-scoped: they need a ClusterRole and a ClusterRoleBinding.
Docs
- https://kubernetes.io/docs/reference/access-authn-authz/rbac/
- https://kubernetes.io/docs/reference/access-authn-authz/certificate-signing-requests/
Tasks
- Q1 RBAC for a deployment bot
- Q33 Aggregated ClusterRole
- Q6 Certificates: expiry check and user CSR
- Q44 Exact RBAC with an existing over-grant
- Q47 User onboarding to a working kubeconfig
- Q75 Cluster-wide read access for the auth team (Simulator Set C)
- Q91 API resources and a Role census (Simulator Set D (killer.sh-style))
4.2 Prepare underlying infrastructure for installing a Kubernetes cluster
./exam study 4.2 (or infra)
Know
- Every machine needs, before kubeadm: a container runtime (containerd) using the same cgroup driver as the kubelet (systemd), the kernel modules overlay and br_netfilter, the sysctls net.bridge.bridge-nf-call-iptables and net.ipv4.ip_forward, and swap off or accounted for.
- kubeadm, kubelet and kubectl come from the package repository of one minor version (pkgs.k8s.io) and are held, so that a system update does not move them.
- Also: a unique hostname, MAC address and product_uuid per node, and the ports open - 6443 (API), 10250 (kubelet), 2379-2380 (etcd).
Fast
printf 'overlay\nbr_netfilter\n' > /etc/modules-load.d/k8s.conf; modprobe overlay; modprobe br_netfilter
cat > /etc/sysctl.d/k8s.conf <<EOF
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward = 1
EOF
sysctl --system
swapoff -a; sed -i '/ swap / s/^/#/' /etc/fstab; cat /proc/swaps
systemctl enable --now containerd; grep SystemdCgroup /etc/containerd/config.toml
apt-get install -y kubelet kubeadm kubectl; apt-mark hold kubelet kubeadm kubectl; dpkg -i ./some-package.deb
crictl version; cat /etc/crictl.yaml; grep -E 'cgroupDriver|containerRuntimeEndpoint' /var/lib/kubelet/config.yaml
Traps
sysctl -wandmodprobeare gone after a reboot - it is the files in /etc/sysctl.d and /etc/modules-load.d that last.- A kubelet on cgroupfs with a runtime on systemd (or the other way round) gives an unstable node.
- kubeadm's preflight stops at active swap unless the kubelet is configured for it (failSwapOn: false).
Docs
- https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/
- https://kubernetes.io/docs/setup/production-environment/container-runtimes/
Tasks
- Q107 Kernel prerequisites for a Kubernetes node
- Q108 The container runtime behind a Pod
- Q41 Component versions
4.3 Create and manage Kubernetes clusters using kubeadm
./exam study 4.3 (or kubeadm)
Know
kubeadm initmakes the control plane: certificates in /etc/kubernetes/pki, kubeconfigs in /etc/kubernetes, static Pod manifests in /etc/kubernetes/manifests, then CoreDNS and kube-proxy. A CNI plugin is yours to add.kubeadm joinadds a node with a bootstrap token and the hash of the cluster CA;kubeadm token create --print-join-commandwrites the whole line.- What the cluster was built with is in ConfigMap kubeadm-config (kube-system).
kubeadm certs check-expirationlists every certificate it manages. - A static Pod is a manifest in the kubelet's staticPodPath; it shows up in the API as NAME-NODE and can only be changed or removed through the file.
Fast
kubeadm init --pod-network-cidr=10.244.0.0/16 --control-plane-endpoint=HOST:6443
kubeadm token create --ttl 2h --print-join-command; kubeadm token list
openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex # the CA hash
kubeadm certs check-expiration; openssl x509 -in /etc/kubernetes/pki/apiserver.crt -noout -enddate
k -n kube-system get cm kubeadm-config -o yaml
grep staticPodPath /var/lib/kubelet/config.yaml; k run web --image=nginx:1.27 $do > /etc/kubernetes/manifests/web.yaml
Traps
- Tokens expire (24 hours unless you say otherwise) - make a new one rather than hunting for the old.
- Check staticPodPath before you write a static Pod: it is not always /etc/kubernetes/manifests.
- Deleting a static Pod with kubectl only deletes its mirror; the kubelet puts it straight back.
Docs
- https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/
- https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-token/
- https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-join/
- https://kubernetes.io/docs/tasks/configure-pod-container/static-pod/
Tasks
- Q40 kubeadm join token
- Q5 Static pod
- Q6 Certificates: expiry check and user CSR
- Q67 Prepare a node join without shell access to the control plane
- Q46 Static pod on a relocated manifest path
- Q94 Static Pod on the control plane behind a Service (Simulator Set D (killer.sh-style))
- Q95 The kubelet's two certificates (Simulator Set D (killer.sh-style))
- Q80 Plan the control-plane upgrade (Simulator Set C)
4.4 Manage the lifecycle of Kubernetes clusters
./exam study 4.4 (or lifecycle)
Know
- Upgrade one minor version at a time, control plane first: the kubeadm package,
kubeadm upgrade planandapply, drain, the kubelet and kubectl packages, restart the kubelet, uncordon. Workers:kubeadm upgrade nodein place ofapply. - Maintenance on a node:
kubectl draincordons it and evicts its Pods;kubectl uncordongives it back. - etcd:
etcdctl snapshot savebacks it up;etcdutl snapshot restore --data-dirunpacks a snapshot into a new directory that etcd's manifest must then point at. kubeadm certs renew allrenews the control-plane certificates; the static Pods have to be restarted to load them.
Fast
k drain NODE --ignore-daemonsets --delete-emptydir-data; k uncordon NODE
apt-get install -y kubeadm='1.36.*'; kubeadm upgrade plan; kubeadm upgrade apply v1.36.x # workers: kubeadm upgrade node
apt-get install -y kubelet='1.36.*' kubectl='1.36.*'; systemctl daemon-reload; systemctl restart kubelet
etcdctl --endpoints=https://127.0.0.1:2379 --cacert=/etc/kubernetes/pki/etcd/ca.crt \
--cert=/etc/kubernetes/pki/etcd/server.crt --key=/etc/kubernetes/pki/etcd/server.key snapshot save /opt/backup.db
etcdutl snapshot restore /opt/backup.db --data-dir /var/lib/etcd-restore # then etcd.yaml: the hostPath of volume etcd-data
kubeadm certs renew all; kubeadm certs check-expiration
Traps
- drain stops at DaemonSet Pods (--ignore-daemonsets), at emptyDir data (--delete-emptydir-data) and at bare Pods (--force, which deletes them for good).
- A drained node stays cordoned until you uncordon it.
- With etcd 3.6, as here, restore exists only in etcdutl -
etcdctl snapshot restoreis gone. - Restoring etcd takes the whole cluster back to the snapshot: everything created since is lost.
Docs
- https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/
- https://kubernetes.io/docs/tasks/administer-cluster/safely-drain-node/
- https://kubernetes.io/docs/tasks/administer-cluster/configure-upgrade-etcd/
- https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-certs/
Tasks
- Q109 Drain the control-plane node and bring it back
- Q2 etcd snapshot
- Q65 Renew control-plane certificates
- Q45 etcd restore
- Q80 Plan the control-plane upgrade (Simulator Set C)
4.5 Implement and configure a highly-available control plane
./exam study 4.5 (or ha)
Know
- Highly available means several control-plane nodes (three as a rule) behind one stable address - the controlPlaneEndpoint, a load balancer or virtual IP, fixed at
kubeadm init. - etcd is either stacked (a member on every control-plane node) or external (hosts of its own). It needs a majority to accept writes: 3 members survive the loss of 1, 5 of 2.
- More control-plane nodes join with
kubeadm join --control-plane --certificate-key KEY; the key comes fromkubeadm init phase upload-certs --upload-certsand is good for two hours. - Every kube-apiserver serves at once. Of the schedulers and controller-managers only the leader acts - they elect it through a Lease in kube-system.
Fast
k get nodes -l node-role.kubernetes.io/control-plane
k -n kube-system get cm kubeadm-config -o yaml | grep controlPlaneEndpoint
kubeadm init --control-plane-endpoint=LB:6443 --upload-certs
kubeadm init phase upload-certs --upload-certs # prints the certificate key
kubeadm token create --print-join-command --certificate-key KEY # ... --control-plane --certificate-key KEY
k -n kube-system get lease kube-scheduler kube-controller-manager # HOLDER = NODE_uuid
etcdctl member list -w table; etcdctl endpoint status -w table; etcdctl endpoint health # with the four etcd flags
k -n kube-system get pods -l component=etcd -o wide
Traps
- A cluster initialised without a controlPlaneEndpoint cannot simply be given a second control-plane node.
- An even number of etcd members tolerates no more failures than the odd number below it.
- The certificate key expires after two hours - upload the certificates again rather than reusing an old key.
Docs
- https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/high-availability/
- https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/ha-topology/
Tasks
- Q110 Prepare the join of a second control-plane node
- Q111 Who leads, and who is in etcd
- Q86 How the control plane is run (Simulator Set D (killer.sh-style))
- Q87 Cluster facts to an exact file (Simulator Set D (killer.sh-style))
4.6 Use Helm and Kustomize to install cluster components
./exam study 4.6 (or helmkustomize)
Know
- Helm: a chart is the package, a release one installation of it in a Namespace, and each install, upgrade or rollback a revision. Values come from the chart, then -f files, then --set.
helm templaterenders a chart without installing it;helm show valuesprints its defaults;helm get valueswhat a release was given.- Kustomize: a kustomization.yaml lists resources and changes them - namespace, namePrefix, labels, images, replicas, patches - and generates ConfigMaps and Secrets. An overlay lists a base and changes only what differs.
Fast
helm repo add NAME URL && helm repo update; helm search repo NAME --versions
helm install REL CHART -n NS --create-namespace --set replicaCount=2 -f values.yaml # --version X
helm upgrade REL CHART -n NS --reuse-values --set image.tag=1.27
helm template REL CHART -n NS --set replicaCount=3 > manifest.yaml
helm list -A; helm history REL -n NS; helm rollback REL 1 -n NS; helm uninstall REL -n NS
k kustomize DIR | less; k apply -k DIR
# kustomization.yaml: resources: [../../base] namespace: prod namePrefix: prod- images: [{name: nginx, newTag: "1.27"}]
# replicas: [{name: web, count: 3}] patches: [{path: patch.yaml}] configMapGenerator: [{name: cfg, literals: [k=v]}]
Traps
- Releases live in a Namespace: without -n Helm looks in
default. - An upgrade without --reuse-values (or the same -f and --set again) puts every value back to the chart's default.
- In a kustomization,
images,replicasand patches use the names of the base, before any prefix. namespace:in a kustomization does not create the Namespace.
Docs
- https://helm.sh/docs/intro/using_helm/
- https://helm.sh/docs/helm/helm_template/
- https://kubernetes.io/docs/tasks/manage-kubernetes-objects/kustomization/
Tasks
- Q21 Helm release install and upgrade
- Q3 Kustomize overlay
- Q48 Kustomize with generators and patches
- Q69 Render a chart, then install it (Simulator Set C)
- Q98 Helm release housekeeping (Simulator Set D (killer.sh-style))
4.7 Understand extension interfaces (CNI, CSI, CRI, etc.)
./exam study 4.7 (or interfaces)
Know
- CRI: the kubelet drives the container runtime (containerd, CRI-O) through a socket;
crictlspeaks the same protocol, so it works on a node when kubectl does not. - CNI: for every Pod the runtime calls the plugins named in /etc/cni/net.d (binaries in /opt/cni/bin) to give it an address and wire it up. The plugin - kindnet, Calico, Cilium, Flannel - usually runs as a DaemonSet.
- CSI: a storage driver runs as Pods, registers a CSIDriver object, and a StorageClass names it as provisioner.
Fast
k get nodes -o wide # CONTAINER-RUNTIME column
crictl version; crictl pods; crictl ps -a; crictl images; crictl inspect CONTAINER-ID
grep containerRuntimeEndpoint /var/lib/kubelet/config.yaml; cat /etc/crictl.yaml
ls /etc/cni/net.d; cat /etc/cni/net.d/*; ls /opt/cni/bin
k -n kube-system get ds # the CNI's DaemonSet beside kube-proxy
k get csidrivers; k get csinodes; k get sc # PROVISIONER
Traps
- Not every CNI plugin enforces NetworkPolicy (Flannel does not; kindnet in this cluster does).
- Without a CNI plugin the nodes stay NotReady:
cni plugin not initialized. - The runtime takes the first configuration file in /etc/cni/net.d, in alphabetical order.
Docs
- https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/
- https://kubernetes.io/docs/concepts/containers/cri/
- https://kubernetes.io/docs/tasks/debug/debug-cluster/crictl/
Tasks
- Q112 CNI, CSI and CRI: what this cluster plugs in
- Q108 The container runtime behind a Pod
- Q41 Component versions
- Q63 Cluster facts, exactly
- Q77 Audit the CNI, then lock a namespace down (Simulator Set C)
- Q87 Cluster facts to an exact file (Simulator Set D (killer.sh-style))
4.8 Understand CRDs, install and configure operators
./exam study 4.8 (or crds)
Know
- A CustomResourceDefinition adds a kind to the API: group, names (kind, plural, singular, shortNames), scope (Namespaced or Cluster) and versions, each with an OpenAPI v3 schema.
- The schema validates custom resources - types, required fields, enum, minimum, maximum, defaults - and feeds
kubectl explain. - An operator is a CRD plus a controller (a Deployment with RBAC) that acts on those objects. Installing one means applying its manifests or its Helm chart, then creating a custom resource.
Fast
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata: {name: backups.data.example.com} # PLURAL.GROUP
spec:
group: data.example.com
scope: Namespaced
names: {kind: Backup, plural: backups, singular: backup, shortNames: [bk]}
versions:
- name: v1
served: true
storage: true
schema:
openAPIV3Schema:
type: object
properties:
spec:
type: object
required: [schedule]
properties:
schedule: {type: string}
retention: {type: integer, minimum: 1, default: 7}
k get crd; k api-resources --api-group=data.example.com; k explain backup.spec --recursive
Traps
- The CRD's name must be PLURAL.GROUP, and exactly one version has
storage: true. - A custom resource's apiVersion is GROUP/VERSION of the CRD, not apiextensions.k8s.io/v1.
defaultand validation only work inside a structural schema: every level needs itstype.- Deleting a CRD deletes every object of that kind.
Docs
- https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/
- https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/
- https://kubernetes.io/docs/concepts/extend-kubernetes/operator/
Tasks
- Q4 CustomResourceDefinition
- Q34 Discover a CRD field with kubectl explain
- Q49 CRD with validation and defaults
- Q91 API resources and a Role census (Simulator Set D (killer.sh-style))
Services & Networking (20%)
5.1 Understand connectivity between Pods
./exam study 5.1 (or connectivity)
Know
- Every Pod has its own IP address, and any Pod reaches any other Pod on any node at that address without NAT. The CNI plugin makes that true.
- Each node hands out Pod addresses from its own range (spec.podCIDR), a slice of the cluster's Pod CIDR; Services get theirs from the separate service CIDR.
- A hostNetwork Pod lives in the node's network: it has the node's address and shares its ports.
- Containers of one Pod share a network namespace and talk over localhost.
Fast
k get pods -o wide -A; k get nodes -o custom-columns=NODE:.metadata.name,PODCIDR:.spec.podCIDR,IP:.status.addresses[0].address
k exec POD -- wget -qO- -T3 http://POD-IP:PORT
k -n kube-system get cm kubeadm-config -o yaml | grep -E 'podSubnet|serviceSubnet'
k -n kube-system get pods -o wide | grep -E 'kube-proxy|kindnet'; k -n kube-system get cm kube-proxy -o yaml | grep mode
Traps
- A Pod's address changes whenever the Pod is replaced - that is what Services are for.
- Reaching a Pod by IP proves the network; reaching it by Service name also needs endpoints and DNS.
- If only the Pods of one node are unreachable, look at that node's CNI and kube-proxy Pods.
Docs
- https://kubernetes.io/docs/concepts/services-networking/
- https://kubernetes.io/docs/concepts/cluster-administration/networking/
Tasks
- Q113 Pod to Pod, across nodes
- Q63 Cluster facts, exactly
- Q71 Expose and verify the dispatch API (Simulator Set C)
- Q77 Audit the CNI, then lock a namespace down (Simulator Set C)
- Q87 Cluster facts to an exact file (Simulator Set D (killer.sh-style))
5.2 Define and enforce Network Policies
./exam study 5.2 (or netpol)
Know
- Pods accept and send everything until a NetworkPolicy selects them. From then on, for the policyTypes it lists, only what some policy allows gets through.
- Policies only allow, and they add up: there is no deny rule and no ordering.
- Peers: podSelector (same Namespace), namespaceSelector, ipBlock. Two selectors in one list item mean AND; two list items mean OR.
- Default deny:
podSelector: {}with the policyTypes and no rules. An egress lock-down also has to allow DNS (UDP and TCP 53).
Fast
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata: {name: backend-policy, namespace: NS}
spec:
podSelector: {matchLabels: {app: backend}}
policyTypes: [Ingress, Egress]
ingress:
- from: [{podSelector: {matchLabels: {app: frontend}}}]
ports: [{protocol: TCP, port: 8080}]
egress:
- to: [{namespaceSelector: {matchLabels: {kubernetes.io/metadata.name: kube-system}}, podSelector: {matchLabels: {k8s-app: kube-dns}}}]
ports: [{protocol: UDP, port: 53}, {protocol: TCP, port: 53}]
k describe netpol backend-policy -n NS; k exec CLIENT -- wget -qO- -T2 http://backend:8080
Traps
- There is no
kubectl create networkpolicy- copy the skeleton from the docs. namespaceSelectorandpodSelectorin the same item: Pods with that label in those Namespaces. In two items: either.- A missing
ingressdenies everything;ingress: [{}]allows everything. - A Namespace is matched by its labels; kubernetes.io/metadata.name=NAME is always there.
Docs
Tasks
- Q11 NetworkPolicy
- Q42 Egress-only NetworkPolicy
- Q54 Zero-trust namespace
- Q83 NetworkPolicy for the safety monitor (Simulator Set C)
- Q100 Namespace AND pod selector, enforced (Simulator Set D (killer.sh-style))
- Q77 Audit the CNI, then lock a namespace down (Simulator Set C)
5.3 Use ClusterIP, NodePort, LoadBalancer service types and endpoints
./exam study 5.3 (or services)
Know
- A Service selects Pods by label and forwards
porttotargetPort- a number, or the name of a container port. - ClusterIP is reachable inside the cluster; NodePort adds a port (30000-32767) on every node; LoadBalancer adds an external address from the cloud; ExternalName is a DNS alias; headless (clusterIP: None) gives every Pod its own DNS record.
- The Pods behind a Service are its endpoints, kept in EndpointSlices; only Ready Pods are in them.
Fast
k expose deploy shop --name=shop-svc --port=80 --target-port=80 --type=NodePort
k create svc nodeport shop-svc --tcp=80:80 --node-port=30080 # then check the selector
k create svc externalname legacy --external-name=db.example.com
k get svc,endpointslices -o wide; k describe svc shop-svc # Endpoints:
k get pods -l app=shop --show-labels
k run tmp --rm -it --restart=Never --image=busybox:1.36 -- wget -qO- -T3 http://shop-svc
Traps
kubectl exposecopies the workload's selector;kubectl create svcselects app=, which may match nothing.exposecannot choose the nodePort - usecreate svc nodeport --node-portor edit the YAML.- A LoadBalancer stays where nothing provides one; it still works as a NodePort.
- A headless Service for a StatefulSet has to carry the name in the StatefulSet's serviceName.
Docs
- https://kubernetes.io/docs/concepts/services-networking/service/
- https://kubernetes.io/docs/concepts/services-networking/endpoint-slices/
Tasks
- Q12 NodePort Service
- Q30 Named ports
- Q43 Headless and ExternalName services
- Q64 The context's namespace
- Q71 Expose and verify the dispatch API (Simulator Set C)
- Q73 Fix ETA service connectivity (Simulator Set C)
- Q94 Static Pod on the control plane behind a Service (Simulator Set D (killer.sh-style))
5.4 Use the Gateway API to manage Ingress traffic
./exam study 5.4 (or gateway)
Know
- Three kinds, three owners: a GatewayClass names the controller, a Gateway opens listeners (port, protocol, hostname, which routes may attach), an HTTPRoute attaches to a Gateway (parentRefs) and sends requests to Services (backendRefs).
- An HTTPRoute rule matches on path, header, method or query, and may split traffic between backends by weight - that is a canary.
- The kinds are CRDs (gateway.networking.k8s.io): they have to be installed, and a controller has to implement them.
Fast
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata: {name: web-gw, namespace: NS}
spec:
gatewayClassName: example
listeners: [{name: http, protocol: HTTP, port: 80, hostname: shop.example.com}]
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata: {name: web, namespace: NS}
spec:
parentRefs: [{name: web-gw}]
hostnames: [shop.example.com]
rules:
- matches: [{path: {type: PathPrefix, value: /}}]
backendRefs: [{name: v1, port: 80, weight: 90}, {name: v2, port: 80, weight: 10}]
k get gatewayclass,gateway,httproute -A; k describe httproute web -n NS
Traps
- A route in another Namespace attaches only if the listener's allowedRoutes lets it.
- A header match belongs in
matches, beside or instead of the path - and several rules are tried most specific first. - No Gateway controller runs in this simulator, so these tasks are graded on the objects.
Docs
- https://gateway-api.sigs.k8s.io/guides/user-guides/http-routing/
- https://kubernetes.io/docs/concepts/services-networking/gateway/
Tasks
- Q22 Gateway API HTTPRoute
- Q56 Gateway API canary with header match
- Q74 Fleet dashboard canary with Gateway API (Simulator Set C)
5.5 Know how to use Ingress controllers and Ingress resources
./exam study 5.5 (or ingress)
Know
- An Ingress routes HTTP(S) by host and path to Services of its own Namespace. It does nothing without an ingress controller, chosen with ingressClassName (an IngressClass names the controller).
- Each path has a pathType (Prefix, Exact or ImplementationSpecific) and a backend: service.name plus service.port.number or .name.
- TLS: spec.tls lists hosts and the kubernetes.io/tls Secret that holds their certificate.
Fast
k create ingress app-ingress -n NS --class=nginx \
--rule='shop.example.com/api*=api:8080,tls=shop-tls' --rule='shop.example.com/*=web:80,tls=shop-tls'
openssl req -x509 -nodes -newkey rsa:2048 -days 365 -subj '/CN=shop.example.com' -keyout tls.key -out tls.crt
k create secret tls shop-tls -n NS --cert=tls.crt --key=tls.key
k get ingress,ingressclass -A; k describe ingress app-ingress -n NS
Traps
- In --rule a trailing * means pathType Prefix; without it the path is Exact.
- The backend port is the Service's port, not the container's.
- No ingress controller runs in this simulator, so Ingress tasks are graded on the object - on the exam, curl it.
Docs
- https://kubernetes.io/docs/concepts/services-networking/ingress/
- https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/
Tasks
- Q13 Ingress routing
- Q55 TLS Ingress with a generated certificate
5.6 Understand and use CoreDNS
./exam study 5.6 (or coredns)
Know
- CoreDNS is a Deployment in kube-system behind the Service kube-dns; its configuration, the Corefile, is in ConfigMap coredns: the
kubernetesplugin answers for cluster.local,forwardsends the rest upstream. - Names: SVC.NS.svc.cluster.local for a Service, POD.SVC.NS.svc.cluster.local for the Pods of a headless Service.
- A Pod's /etc/resolv.conf points at the kube-dns address and lists the search domains that make short names work. dnsPolicy decides: ClusterFirst (default), Default (the node's resolver), None with dnsConfig.
Fast
k -n kube-system get deploy coredns; k -n kube-system get pods -l k8s-app=kube-dns -o wide
k -n kube-system get svc kube-dns; k -n kube-system get endpointslices -l k8s-app=kube-dns
k -n kube-system get cm coredns -o yaml; k -n kube-system edit cm coredns
k -n kube-system logs -l k8s-app=kube-dns; k -n kube-system rollout restart deploy coredns
k run tmp --rm -it --restart=Never --image=busybox:1.36 -- nslookup kubernetes.default.svc.cluster.local
k exec POD -- cat /etc/resolv.conf
Traps
- A typo in the Corefile puts CoreDNS in CrashLoopBackOff - its log names the plugin and the line.
- DNS can be broken in several places at once: the Corefile, the Deployment (replicas, image), the kube-dns Service (selector, ports), a NetworkPolicy.
- A Pod with dnsPolicy: Default resolves the internet but not cluster names.
Docs
- https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/
- https://kubernetes.io/docs/tasks/administer-cluster/dns-custom-nameservers/
- https://kubernetes.io/docs/tasks/administer-cluster/dns-debugging-resolution/
Tasks
- Q14 Cluster DNS
- Q37 Cluster DNS is broken
- Q57 DNS down: three faults
- Q101 Resolves everything except the cluster (Simulator Set D (killer.sh-style))