"""Simulator Set C - 18 tasks, fixed set, modelled on a commercial CKA mock exam sitting. Original scenarios (not copies of any commercial simulator) exercising the same techniques as a typical 18-question mock: hostPath PV/PVC binding by volumeName, data migration between claims, Helm render-then-install, Gateway API header routing, RBAC to a file, probe/selector/targetPort repair, PSA-free securityContext work, HPA behavior blocks, and "write the answer to a file" data-extraction tasks. Two scenarios are deliberately adapted so they cannot wreck the practice cluster: - 77 audits the installed CNI instead of installing Calico over kindnet. - 80 plans a control-plane upgrade instead of running `kubeadm upgrade apply` for real. Run with: ./ksim sim c """ from sim.bank import C, NS, cmd, hand Q = [] def task(**kw): kw.setdefault("difficulty", "hard") kw["set"] = "c" Q.append(kw) # ------------------------------------------------------------------ 68 ---- task( id=68, domain="STOR", title="Provision geo-fence persistent storage", task=""" In Namespace `geo-fence`: 1. Create a PersistentVolume `geofence-pv`: capacity 250Mi, accessMode ReadWriteOnce, hostPath `/data/geofence`, storageClassName `manual`, reclaim policy `Retain`. 2. Create a PersistentVolumeClaim `geofence-pvc` in `geo-fence`: 250Mi, ReadWriteOnce, storageClassName `manual`, bound to **that specific PV by name**. 3. Create Pod `geofence-reader` in `geo-fence` using `busybox:1.36`, mounting the claim at `/geo-data`, running `sh -c "ls /geo-data && sleep 3600"`. The claim must end up `Bound` to `geofence-pv` and the Pod must reach Running. """, setup=NS + r""" mkns geo-fence $K -n geo-fence delete pod geofence-reader --ignore-not-found --wait=true >/dev/null 2>&1 $K -n geo-fence delete pvc geofence-pvc --ignore-not-found --wait=true >/dev/null 2>&1 $K delete pv geofence-pv --ignore-not-found --wait=true >/dev/null 2>&1 for n in "$CP" "$W1" "$W2" "$W3"; do nx "$n" 'mkdir -p /data/geofence' /dev/null)" 250Mi \ && eq "$($p -o jsonpath='{.spec.accessModes[0]}')" ReadWriteOnce \ && eq "$($p -o jsonpath='{.spec.hostPath.path}')" /data/geofence \ && eq "$($p -o jsonpath='{.spec.storageClassName}')" manual \ && eq "$($p -o jsonpath='{.spec.persistentVolumeReclaimPolicy}')" Retain"""), C("PVC geofence-pvc is Bound to geofence-pv", 3, r""" eq "$(j -n geo-fence pvc geofence-pvc '{.status.phase}')" Bound \ && eq "$(j -n geo-fence pvc geofence-pvc '{.spec.volumeName}')" geofence-pv"""), C("pod geofence-reader Running with the claim mounted at /geo-data", 2, r""" for i in $(seq 1 20); do [ "$(j -n geo-fence pod geofence-reader '{.status.phase}')" = Running ] && break; sleep 3; done eq "$(j -n geo-fence pod geofence-reader '{.status.phase}')" Running \ && eq "$(j -n geo-fence pod geofence-reader '{.spec.volumes[0].persistentVolumeClaim.claimName}')" geofence-pvc \ && j -n geo-fence pod geofence-reader '{.spec.containers[0].volumeMounts[*].mountPath}' | grep -q /geo-data"""), ], solution=[hand(r"""vim pv.yaml""", r"""The PersistentVolume and PersistentVolumeClaim from the docs page *Configure a Pod to Use a PersistentVolume for Storage*, `---` between them. `volumeName` pins the claim to this PV: without it the binder takes any PV whose size, access mode and class fit. ``` apiVersion: v1 kind: PersistentVolume metadata: name: geofence-pv spec: storageClassName: manual capacity: storage: 250Mi accessModes: - ReadWriteOnce persistentVolumeReclaimPolicy: Retain hostPath: path: /data/geofence --- apiVersion: v1 kind: PersistentVolumeClaim metadata: name: geofence-pvc namespace: geo-fence spec: storageClassName: manual volumeName: geofence-pv accessModes: - ReadWriteOnce resources: requests: storage: 250Mi ```""", run=r"""cat > pv.yaml <<'EOF' apiVersion: v1 kind: PersistentVolume metadata: name: geofence-pv spec: storageClassName: manual capacity: storage: 250Mi accessModes: - ReadWriteOnce persistentVolumeReclaimPolicy: Retain hostPath: path: /data/geofence --- apiVersion: v1 kind: PersistentVolumeClaim metadata: name: geofence-pvc namespace: geo-fence spec: storageClassName: manual volumeName: geofence-pv accessModes: - ReadWriteOnce resources: requests: storage: 250Mi EOF"""), cmd(r"""k apply -f pv.yaml k -n geo-fence get pvc k -n geo-fence run geofence-reader --image=busybox:1.36 --dry-run=client -o yaml --command -- sh -c "ls /geo-data && sleep 3600" > pod.yaml""", r"""`Bound` to `geofence-pv`."""), hand(r"""vim pod.yaml""", r"""Add to the container, below `name: geofence-reader`: ``` volumeMounts: - name: geo mountPath: /geo-data ``` and to the Pod's `spec:`, right below `spec:`: ``` volumes: - name: geo persistentVolumeClaim: claimName: geofence-pvc ```""", run=r"""yq -i '.spec.containers[0].volumeMounts = [{"name": "geo", "mountPath": "/geo-data"}] | .spec.volumes = [{"name": "geo", "persistentVolumeClaim": {"claimName": "geofence-pvc"}}]' pod.yaml"""), cmd(r"""k apply -f pod.yaml k -n geo-fence get pod geofence-reader""")], ) # ------------------------------------------------------------------ 69 ---- task( id=69, domain="ARCH", title="Render a chart, then install it", task=""" A chart is at `{WS}/q69/chart`. 1. Render the chart **without installing it** as release name `ride-argocd`, namespace `ride-gitops`, with `replicaCount=3`, saving the rendered manifests to `{WS}/q69/manifest.yaml`. 2. Then install that same chart into Namespace `ride-gitops` (create it) as release `ride-argocd` with `replicaCount=3`. 3. Save the output of listing the releases in that Namespace to `{WS}/q69/releases.txt`. """, setup=r""" rm -rf "$WS/q69"; mkdir -p "$WS/q69/chart/templates" cat > "$WS/q69/chart/Chart.yaml" <<'EOF' apiVersion: v2 name: ride version: 0.2.0 EOF cat > "$WS/q69/chart/values.yaml" <<'EOF' replicaCount: 1 image: nginx:1.27 EOF cat > "$WS/q69/chart/templates/deploy.yaml" <<'EOF' apiVersion: apps/v1 kind: Deployment metadata: {name: {{ .Release.Name }}-server, namespace: {{ .Release.Namespace }}} spec: replicas: {{ .Values.replicaCount }} selector: {matchLabels: {app: {{ .Release.Name }}-server}} template: metadata: {labels: {app: {{ .Release.Name }}-server}} spec: {containers: [{name: server, image: "{{ .Values.image }}"}]} EOF $K create ns ride-gitops --dry-run=client -o yaml | $K apply -f - >/dev/null helm -n ride-gitops uninstall ride-argocd >/dev/null 2>&1 || true command -v helm >/dev/null || echo "warn: helm is not installed on this machine" """, checks=[ C("manifest.yaml holds rendered manifests, not an installed release", 3, r""" f="$WS/q69/manifest.yaml"; [ -s "$f" ] || exit 1 grep -q 'kind: Deployment' "$f" && grep -q 'ride-argocd-server' "$f" && grep -Eq 'replicas: 3' "$f" """), C("release ride-argocd is deployed in ride-gitops with 3 replicas", 3, r""" helm -n ride-gitops status ride-argocd -o json 2>/dev/null \ | python3 -c 'import sys,json;sys.exit(0 if json.load(sys.stdin)["info"]["status"]=="deployed" else 1)' || exit 1 for i in $(seq 1 20); do [ "$(j -n ride-gitops deploy ride-argocd-server '{.status.availableReplicas}')" = 3 ] && exit 0; sleep 5; done; exit 1"""), C("releases.txt lists the ride-argocd release", 3, r""" grep -q 'ride-argocd' "$WS/q69/releases.txt" 2>/dev/null && grep -qi 'deployed' "$WS/q69/releases.txt" """), ], solution=""" helm template ride-argocd {WS}/q69/chart -n ride-gitops --set replicaCount=3 > {WS}/q69/manifest.yaml k create ns ride-gitops helm -n ride-gitops install ride-argocd {WS}/q69/chart --set replicaCount=3 helm -n ride-gitops list > {WS}/q69/releases.txt # `helm template` renders locally and talks to no cluster - it is how you review what a chart # would do, and how GitOps pipelines produce committed YAML. `helm install` is the one that # creates a release. --create-namespace on install saves the separate `k create ns`. """, ) # ------------------------------------------------------------------ 70 ---- task( id=70, domain="TSHOOT", title="Fix the driver onboarding app", task=""" Deployment `onboard-app` in Namespace `driver-onboard` never becomes Ready. It has **three** separate faults. Fix the existing Deployment in place — do not delete and recreate it, and do not change the container name. All replicas must end up Running and Ready. """, setup=NS + r""" mkns driver-onboard $K -n driver-onboard delete deploy onboard-app --ignore-not-found --wait=true >/dev/null 2>&1 $K apply -f - >/dev/null <<'EOF' apiVersion: apps/v1 kind: Deployment metadata: {name: onboard-app, namespace: driver-onboard} spec: replicas: 2 selector: {matchLabels: {app: onboard-app}} template: metadata: {labels: {app: onboard-app}} spec: containers: - name: onboard image: ngnix:1.25 ports: [{containerPort: 80}] livenessProbe: httpGet: {path: /, port: 9090} initialDelaySeconds: 5 readinessProbe: httpGet: {path: /healthz, port: 80} initialDelaySeconds: 3 EOF """, checks=[ C("container image is corrected to nginx (the ngnix typo is gone)", 3, r""" i=$(j -n driver-onboard deploy onboard-app '{.spec.template.spec.containers[0].image}') echo "$i" | grep -q '^nginx:' && ! echo "$i" | grep -q ngnix"""), C("liveness probe targets port 80", 2, r""" eq "$(j -n driver-onboard deploy onboard-app '{.spec.template.spec.containers[0].livenessProbe.httpGet.port}')" 80"""), # any readiness probe nginx can satisfy counts (/, /index.html, a tcpSocket on 80...); the # "all replicas Ready" check below proves it actually passes C("readiness probe no longer targets the missing /healthz", 2, r""" rp=$(j -n driver-onboard deploy onboard-app '{.spec.template.spec.containers[0].readinessProbe}') [ -n "$rp" ] && ! echo "$rp" | grep -q '/healthz'"""), C("all replicas Running and Ready", 2, r""" for i in $(seq 1 24); do [ "$(j -n driver-onboard deploy onboard-app '{.status.readyReplicas}')" = 2 ] && exit 0; sleep 5; done; exit 1"""), ], solution=""" k -n driver-onboard get pods # ImagePullBackOff k -n driver-onboard describe pod -l app=onboard-app | tail -20 k -n driver-onboard set image deploy/onboard-app onboard=nginx:1.25 # fault 1: image typo k -n driver-onboard edit deploy onboard-app # fault 2: livenessProbe.httpGet.port 9090 -> 80 (nothing listens on 9090, so the kubelet # kills the container in a loop once it finally starts) # fault 3: readinessProbe.httpGet.path /healthz -> / (nginx returns 404 there, so the pod # never joins the Service endpoints even when healthy) k -n driver-onboard rollout status deploy onboard-app # Fix them in one edit if you can: each change triggers a fresh rollout, and three sequential # rollouts on a 2-replica Deployment is a lot of wasted clock in an exam. """, ) # ------------------------------------------------------------------ 71 ---- task( id=71, domain="NET", title="Expose and verify the dispatch API", task=""" Deployment `dispatch-api` is running in Namespace `dispatch-hub` and serves on container port 80. 1. Expose it as a **ClusterIP** Service named `dispatch-service`, port 80 to target port 80. It must actually have endpoints. 2. From inside the cluster, fetch `http://dispatch-service.dispatch-hub.svc:80` and save the response body to `{WS}/q71/test-result.txt`. 3. Write the Service's ClusterIP — the address only, nothing else — to `{WS}/q71/service-ip.txt`. """, setup=NS + r""" mkns dispatch-hub rm -rf "$WS/q71"; mkdir -p "$WS/q71" $K -n dispatch-hub delete svc dispatch-service --ignore-not-found >/dev/null 2>&1 $K -n dispatch-hub delete deploy dispatch-api --ignore-not-found --wait=true >/dev/null 2>&1 $K -n dispatch-hub create deploy dispatch-api --image=nginx:1.27 --replicas=2 --port=80 >/dev/null $K -n dispatch-hub rollout status deploy dispatch-api --timeout=120s >/dev/null 2>&1 """, checks=[ C("dispatch-service is a ClusterIP Service on port 80 -> 80", 2, r""" s="$K -n dispatch-hub get svc dispatch-service" eq "$($s -o jsonpath='{.spec.type}' 2>/dev/null)" ClusterIP \ && eq "$($s -o jsonpath='{.spec.ports[0].port}')" 80 \ && eq "$($s -o jsonpath='{.spec.ports[0].targetPort}')" 80"""), C("the Service has active endpoints", 3, r""" [ -n "$(j -n dispatch-hub endpointslices -l kubernetes.io/service-name=dispatch-service '{.items[*].endpoints[*].addresses[*]}')" ]"""), C("test-result.txt holds the response from nginx", 2, r""" grep -qi 'nginx\|/dev/null"""), C("service-ip.txt contains exactly the Service ClusterIP", 2, r""" want=$(j -n dispatch-hub svc dispatch-service '{.spec.clusterIP}') got=$(tr -d '[:space:]' < "$WS/q71/service-ip.txt" 2>/dev/null) [ -n "$want" ] && eq "$got" "$want" """), ], solution=[cmd(r"""k -n dispatch-hub expose deploy dispatch-api --name=dispatch-service --port=80 --target-port=80 k -n dispatch-hub get svc,endpointslices""", r"""`expose` copies the Deployment's selector, which is what makes the endpoints appear. Note the Service's `CLUSTER-IP`."""), hand(r"""k -n dispatch-hub run probe --image=busybox:1.36 --restart=Never -i --rm -- wget -qO- http://dispatch-service.dispatch-hub.svc:80 > {WS}/q71/test-result.txt cat {WS}/q71/test-result.txt""", r"""nginx's welcome page, fetched from a Pod in the cluster.""", run=r"""kubectl -n dispatch-hub rollout status deploy dispatch-api --timeout=240s for i in 1 2 3 4 5; do kubectl -n dispatch-hub run probe --image=busybox:1.36 --restart=Never -i --rm -- wget -qO- http://dispatch-service.dispatch-hub.svc:80 > {WS}/q71/test-result.txt /dev/null grep -qi nginx {WS}/q71/test-result.txt && break; kubectl -n dispatch-hub delete pod probe --ignore-not-found >/dev/null 2>&1; sleep 3 done"""), hand(r"""echo 10.96.123.45 > {WS}/q71/service-ip.txt""", r"""With the `CLUSTER-IP` that `get svc` showed - the address alone.""", run=r"""kubectl -n dispatch-hub get svc dispatch-service -o jsonpath='{.spec.clusterIP}' > {WS}/q71/service-ip.txt""")], ) # ------------------------------------------------------------------ 72 ---- task( id=72, domain="STOR", title="Migrate ride history to a Retain volume", task=""" In Namespace `ride-history`, Deployment `ride-archive` stores data on PVC `ride-archive-pvc`, whose PersistentVolume has reclaim policy `Delete` — losing the claim would destroy the data. Migrate it: 1. Create PersistentVolume `ride-archive-pv-new`: 500Mi, ReadWriteOnce, hostPath `/data/ride-archive-new`, storageClassName `manual`, reclaim policy **Retain**. 2. Create PVC `ride-archive-pvc-new` in `ride-history` bound to that PV **by name**, 500Mi, ReadWriteOnce, storageClassName `manual`. 3. Copy the existing contents of the old volume to the new one. The file `trips.csv` must end up on the new volume. 4. Point Deployment `ride-archive` at `ride-archive-pvc-new` and get its pods Running again. The old claim is ReadWriteOnce, so something has to release it before a migration pod can mount it. """, setup=NS + r""" mkns ride-history $K -n ride-history delete deploy ride-archive --ignore-not-found --wait=true >/dev/null 2>&1 $K -n ride-history delete pod data-migrator --ignore-not-found --wait=false >/dev/null 2>&1 $K -n ride-history delete pvc ride-archive-pvc ride-archive-pvc-new --ignore-not-found --wait=true >/dev/null 2>&1 $K delete pv ride-archive-pv ride-archive-pv-new --ignore-not-found --wait=true >/dev/null 2>&1 # seed the source data on every node so the pod can land anywhere for n in "$CP" "$W1" "$W2" "$W3"; do nx "$n" 'mkdir -p /data/ride-archive && printf "trip,driver,fare\n1,alice,12.50\n2,bob,8.25\n" > /data/ride-archive/trips.csv; rm -rf /data/ride-archive-new' /dev/null <<'EOF' apiVersion: v1 kind: PersistentVolume metadata: {name: ride-archive-pv} spec: capacity: {storage: 500Mi} accessModes: [ReadWriteOnce] hostPath: {path: /data/ride-archive} storageClassName: manual persistentVolumeReclaimPolicy: Delete --- apiVersion: v1 kind: PersistentVolumeClaim metadata: {name: ride-archive-pvc, namespace: ride-history} spec: accessModes: [ReadWriteOnce] resources: {requests: {storage: 500Mi}} storageClassName: manual volumeName: ride-archive-pv --- apiVersion: apps/v1 kind: Deployment metadata: {name: ride-archive, namespace: ride-history} spec: replicas: 1 selector: {matchLabels: {app: ride-archive}} template: metadata: {labels: {app: ride-archive}} spec: volumes: [{name: data, persistentVolumeClaim: {claimName: ride-archive-pvc}}] containers: - name: archive image: busybox:1.36 command: [sh, -c, "sleep 86400"] volumeMounts: [{name: data, mountPath: /archive}] EOF $K -n ride-history rollout status deploy ride-archive --timeout=120s >/dev/null 2>&1 """, checks=[ C("PV ride-archive-pv-new exists with Retain and the right spec", 3, r""" p="$K get pv ride-archive-pv-new" eq "$($p -o jsonpath='{.spec.capacity.storage}' 2>/dev/null)" 500Mi \ && eq "$($p -o jsonpath='{.spec.hostPath.path}')" /data/ride-archive-new \ && eq "$($p -o jsonpath='{.spec.persistentVolumeReclaimPolicy}')" Retain \ && eq "$($p -o jsonpath='{.spec.storageClassName}')" manual"""), C("PVC ride-archive-pvc-new is Bound to it by volumeName", 2, r""" eq "$(j -n ride-history pvc ride-archive-pvc-new '{.status.phase}')" Bound \ && eq "$(j -n ride-history pvc ride-archive-pvc-new '{.spec.volumeName}')" ride-archive-pv-new"""), C("trips.csv was copied onto the new volume", 3, r""" for n in "$CP" "$W1" "$W2" "$W3"; do nx "$n" 'grep -q alice /data/ride-archive-new/trips.csv 2>/dev/null' /dev/null && exit 0; sleep 5; done; exit 1"""), ], solution=[cmd(r"""k -n ride-history get pvc k get pv ride-archive-pv""", r"""`RECLAIM POLICY Delete` on the old PV: losing the claim loses the data."""), hand(r"""vim new.yaml""", r"""The new PV and its claim, pinned to it by name (the docs page *Configure a Pod to Use a PersistentVolume for Storage* has both): ``` apiVersion: v1 kind: PersistentVolume metadata: name: ride-archive-pv-new spec: storageClassName: manual capacity: storage: 500Mi accessModes: - ReadWriteOnce persistentVolumeReclaimPolicy: Retain hostPath: path: /data/ride-archive-new --- apiVersion: v1 kind: PersistentVolumeClaim metadata: name: ride-archive-pvc-new namespace: ride-history spec: storageClassName: manual volumeName: ride-archive-pv-new accessModes: - ReadWriteOnce resources: requests: storage: 500Mi ```""", run=r"""cat > new.yaml <<'EOF' apiVersion: v1 kind: PersistentVolume metadata: name: ride-archive-pv-new spec: storageClassName: manual capacity: storage: 500Mi accessModes: - ReadWriteOnce persistentVolumeReclaimPolicy: Retain hostPath: path: /data/ride-archive-new --- apiVersion: v1 kind: PersistentVolumeClaim metadata: name: ride-archive-pvc-new namespace: ride-history spec: storageClassName: manual volumeName: ride-archive-pv-new accessModes: - ReadWriteOnce resources: requests: storage: 500Mi EOF"""), cmd(r"""k apply -f new.yaml k -n ride-history scale deploy ride-archive --replicas=0 k -n ride-history run data-migrator --image=busybox:1.36 --restart=Never --dry-run=client -o yaml -- sh -c "cp -a /old-data/. /new-data/ && echo migration-complete && sleep 3600" > m.yaml""", r"""Scale to 0 first: the running Pod holds the old claim. `cp -a /old-data/. /new-data/` copies the contents, dotfiles too, rather than the directory itself."""), hand(r"""vim m.yaml""", r"""Both claims: add to the container, below `name: data-migrator`, ``` volumeMounts: - name: old mountPath: /old-data - name: new mountPath: /new-data ``` and to the Pod's `spec:`, right below `spec:`: ``` volumes: - name: old persistentVolumeClaim: claimName: ride-archive-pvc - name: new persistentVolumeClaim: claimName: ride-archive-pvc-new ```""", run=r"""yq -i '.spec.containers[0].volumeMounts = [{"name": "old", "mountPath": "/old-data"}, {"name": "new", "mountPath": "/new-data"}] | .spec.volumes = [{"name": "old", "persistentVolumeClaim": {"claimName": "ride-archive-pvc"}}, {"name": "new", "persistentVolumeClaim": {"claimName": "ride-archive-pvc-new"}}]' m.yaml"""), cmd(r"""k apply -f m.yaml"""), hand(r"""k -n ride-history logs data-migrator""", r"""Again until it prints `migration-complete`.""", run=r"""for i in $(seq 1 60); do kubectl -n ride-history logs data-migrator 2>/dev/null | grep -q migration-complete && break; sleep 2; done"""), cmd(r"""k -n ride-history delete pod data-migrator --now"""), hand(r"""k -n ride-history edit deploy ride-archive""", r"""In the Pod template's `volumes:`, change `claimName: ride-archive-pvc` to `claimName: ride-archive-pvc-new`, then `:wq`.""", run=r"""kubectl -n ride-history patch deploy ride-archive --type=json -p '[{"op":"replace","path":"/spec/template/spec/volumes/0/persistentVolumeClaim/claimName","value":"ride-archive-pvc-new"}]'"""), cmd(r"""k -n ride-history scale deploy ride-archive --replicas=1 k -n ride-history get pods""")], ) # ------------------------------------------------------------------ 73 ---- task( id=73, domain="TSHOOT", title="Fix ETA service connectivity", task=""" In Namespace `eta-service`, Deployment `eta-calculator` is healthy but Service `eta-backend` returns nothing. There are **two** faults in the Service. Fix the Service so it has active endpoints pointing at the deployment's pods. Do not change the Deployment. """, setup=NS + r""" mkns eta-service $K -n eta-service delete svc eta-backend --ignore-not-found >/dev/null 2>&1 $K -n eta-service delete deploy eta-calculator --ignore-not-found --wait=true >/dev/null 2>&1 $K -n eta-service create deploy eta-calculator --image=nginx:1.27 --replicas=2 --port=80 >/dev/null $K apply -f - >/dev/null <<'EOF' apiVersion: v1 kind: Service metadata: {name: eta-backend, namespace: eta-service} spec: selector: {app: eta-service} ports: [{port: 80, targetPort: 9090, protocol: TCP}] EOF $K -n eta-service rollout status deploy eta-calculator --timeout=120s >/dev/null 2>&1 """, checks=[ C("Service selector matches the deployment's pod labels", 3, r""" want=$(j -n eta-service deploy eta-calculator '{.spec.template.metadata.labels.app}') [ -n "$want" ] && eq "$(j -n eta-service svc eta-backend '{.spec.selector.app}')" "$want" """), C("Service targetPort matches the container port 80", 2, r""" eq "$(j -n eta-service svc eta-backend '{.spec.ports[0].targetPort}')" 80"""), C("the Service has active endpoints", 2, r""" for i in $(seq 1 12); do [ -n "$(j -n eta-service endpointslices -l kubernetes.io/service-name=eta-backend '{.items[*].endpoints[*].addresses[*]}')" ] && exit 0 sleep 3 done; exit 1"""), ], solution=[cmd(r"""k -n eta-service get endpointslices -l kubernetes.io/service-name=eta-backend k -n eta-service get pods --show-labels k -n eta-service describe svc eta-backend""", r"""No endpoints. The Pods are `app=eta-calculator`, the Service selects `app=eta-service`; and it sends to `TargetPort 9090`, where nginx listens on 80. Empty endpoints is almost always the selector; endpoints that refuse connections, the target port."""), hand(r"""k -n eta-service edit svc eta-backend""", r"""Fix both, then `:wq`: ``` ports: - port: 80 protocol: TCP targetPort: 80 selector: app: eta-calculator ```""", run=r"""kubectl -n eta-service patch svc eta-backend -p '{"spec":{"selector":{"app":"eta-calculator"},"ports":[{"port":80,"targetPort":80,"protocol":"TCP"}]}}'"""), cmd(r"""k -n eta-service get endpointslices -l kubernetes.io/service-name=eta-backend""")], ) # ------------------------------------------------------------------ 74 ---- task( id=74, domain="NET", title="Fleet dashboard canary with Gateway API", task=""" Namespace `fleet-ops` has Services `fleet-dashboard-v1` and `fleet-dashboard-v2` (both port 80) and a GatewayClass `fleet-gateway-class` already exists. 1. Create Gateway `fleet-gateway` in `fleet-ops` using that GatewayClass, with a single HTTP listener named `http` on port 80 for hostname `fleet.example.com`. 2. Create HTTPRoute `fleet-route` in `fleet-ops`, attached to `fleet-gateway`, that: - sends requests carrying header `X-Fleet-Version: canary` to `fleet-dashboard-v2` port 80, - sends everything else (path prefix `/`) to `fleet-dashboard-v1` port 80. (No Gateway controller runs here, so you are graded on the objects.) """, setup=NS + r""" mkns fleet-ops $K -n fleet-ops delete httproute fleet-route --ignore-not-found >/dev/null 2>&1 $K -n fleet-ops delete gateway fleet-gateway --ignore-not-found >/dev/null 2>&1 for v in v1 v2; do $K -n fleet-ops create deploy fleet-dashboard-$v --image=nginx:1.27 --port=80 --dry-run=client -o yaml | $K apply -f - >/dev/null $K -n fleet-ops expose deploy fleet-dashboard-$v --port=80 --target-port=80 --dry-run=client -o yaml | $K apply -f - >/dev/null done ensure_gateway_crds || { echo "no Gateway API CRDs and could not download them (no network?) - install them, then re-seed"; exit 1; } $K apply -f - >/dev/null <<'EOF' apiVersion: gateway.networking.k8s.io/v1 kind: GatewayClass metadata: {name: fleet-gateway-class} spec: {controllerName: example.com/fleet-controller} EOF """, checks=[ C("Gateway fleet-gateway uses GatewayClass fleet-gateway-class", 2, r""" eq "$(j -n fleet-ops gateway fleet-gateway '{.spec.gatewayClassName}')" fleet-gateway-class"""), C("listener 'http' on port 80 for fleet.example.com", 2, r""" g="$K -n fleet-ops get gateway fleet-gateway" eq "$($g -o jsonpath='{.spec.listeners[0].name}' 2>/dev/null)" http \ && eq "$($g -o jsonpath='{.spec.listeners[0].port}')" 80 \ && eq "$($g -o jsonpath='{.spec.listeners[0].protocol}')" HTTP \ && eq "$($g -o jsonpath='{.spec.listeners[0].hostname}')" fleet.example.com"""), C("a header rule X-Fleet-Version: canary routes to fleet-dashboard-v2", 3, r""" $K -n fleet-ops get httproute fleet-route -o json 2>/dev/null | python3 -c ' import sys,json rules=json.load(sys.stdin)["spec"].get("rules",[]) ok=any( any(h.get("name","").lower()=="x-fleet-version" and h.get("value")=="canary" for m in r.get("matches",[]) for h in m.get("headers",[])) and any(b.get("name")=="fleet-dashboard-v2" and int(b.get("port",0))==80 for b in r.get("backendRefs",[])) for r in rules) sys.exit(0 if ok else 1)'"""), C("a default path-prefix rule routes to fleet-dashboard-v1", 2, r""" $K -n fleet-ops get httproute fleet-route -o json 2>/dev/null | python3 -c ' import sys,json rules=json.load(sys.stdin)["spec"].get("rules",[]) ok=any( any(m.get("path",{}).get("type")=="PathPrefix" and m.get("path",{}).get("value")=="/" and not m.get("headers") for m in r.get("matches",[])) and any(b.get("name")=="fleet-dashboard-v1" and int(b.get("port",0))==80 for b in r.get("backendRefs",[])) for r in rules) sys.exit(0 if ok else 1)'"""), C("the HTTPRoute is attached to fleet-gateway", 1, r""" j -n fleet-ops httproute fleet-route '{.spec.parentRefs[*].name}' | grep -qw fleet-gateway"""), ], solution=[hand(r"""vim gw.yaml""", r"""The docs page *Gateway API* has a Gateway and an HTTPRoute to start from: ``` apiVersion: gateway.networking.k8s.io/v1 kind: Gateway metadata: name: fleet-gateway namespace: fleet-ops spec: gatewayClassName: fleet-gateway-class listeners: - name: http protocol: HTTP port: 80 hostname: fleet.example.com --- apiVersion: gateway.networking.k8s.io/v1 kind: HTTPRoute metadata: name: fleet-route namespace: fleet-ops spec: parentRefs: - name: fleet-gateway hostnames: - fleet.example.com rules: - matches: - headers: - name: X-Fleet-Version value: canary backendRefs: - name: fleet-dashboard-v2 port: 80 - matches: - path: type: PathPrefix value: / backendRefs: - name: fleet-dashboard-v1 port: 80 ``` The more specific match wins, so the header rule takes canary requests though `/` matches everything.""", run=r"""cat > gw.yaml <<'EOF' apiVersion: gateway.networking.k8s.io/v1 kind: Gateway metadata: name: fleet-gateway namespace: fleet-ops spec: gatewayClassName: fleet-gateway-class listeners: - name: http protocol: HTTP port: 80 hostname: fleet.example.com --- apiVersion: gateway.networking.k8s.io/v1 kind: HTTPRoute metadata: name: fleet-route namespace: fleet-ops spec: parentRefs: - name: fleet-gateway hostnames: - fleet.example.com rules: - matches: - headers: - name: X-Fleet-Version value: canary backendRefs: - name: fleet-dashboard-v2 port: 80 - matches: - path: type: PathPrefix value: / backendRefs: - name: fleet-dashboard-v1 port: 80 EOF"""), cmd(r"""k apply -f gw.yaml k -n fleet-ops get gateway,httproute""")], ) # ------------------------------------------------------------------ 75 ---- task( id=75, domain="ARCH", title="Cluster-wide read access for the auth team", task=""" The rider auth team needs a read-only identity that works in **every** Namespace. 1. Create ServiceAccount `auth-auditor` in Namespace `rider-auth`. 2. Create ClusterRole `auth-cluster-viewer` granting only `get`, `list` and `watch` on `pods`, `deployments`, `services` and `events`. 3. Bind it cluster-wide to that ServiceAccount with ClusterRoleBinding `auth-viewer-binding`. 4. Verify the ServiceAccount can list pods in `kube-system` and write the single-word answer to `{WS}/q75/auth-check.txt`. """, setup=NS + r""" mkns rider-auth rm -rf "$WS/q75"; mkdir -p "$WS/q75" $K delete clusterrolebinding auth-viewer-binding --ignore-not-found >/dev/null 2>&1 $K delete clusterrole auth-cluster-viewer --ignore-not-found >/dev/null 2>&1 $K -n rider-auth delete sa auth-auditor --ignore-not-found >/dev/null 2>&1 """, checks=[ C("ServiceAccount rider-auth/auth-auditor exists", 2, r"""$K -n rider-auth get sa auth-auditor >/dev/null 2>&1"""), C("ClusterRole auth-cluster-viewer grants exactly read verbs on the four resources", 3, r""" $K get clusterrole auth-cluster-viewer -o json 2>/dev/null | python3 -c ' import sys,json rules=json.load(sys.stdin).get("rules",[]) verbs=set(); res=set() for r in rules: verbs|=set(r.get("verbs",[])); res|=set(r.get("resources",[])) sys.exit(0 if verbs=={"get","list","watch"} and {"pods","deployments","services","events"}<=res else 1)'"""), C("the binding grants it cluster-wide", 2, r""" b="$K get clusterrolebinding auth-viewer-binding" eq "$($b -o jsonpath='{.roleRef.name}' 2>/dev/null)" auth-cluster-viewer \ && $b -o jsonpath='{.subjects[*].name}' | grep -qw auth-auditor \ && eq "$($K auth can-i list pods -n kube-system --as=system:serviceaccount:rider-auth:auth-auditor 2>/dev/null)" yes"""), C("auth-check.txt records the verification result", 2, r""" grep -qx 'yes' "$WS/q75/auth-check.txt" 2>/dev/null"""), ], solution=""" k -n rider-auth create sa auth-auditor k create clusterrole auth-cluster-viewer --verb=get,list,watch --resource=pods,deployments,services,events k create clusterrolebinding auth-viewer-binding --clusterrole=auth-cluster-viewer \\ --serviceaccount=rider-auth:auth-auditor k auth can-i list pods -n kube-system --as=system:serviceaccount:rider-auth:auth-auditor > {WS}/q75/auth-check.txt # A ClusterRole bound with a RoleBinding only applies in that one Namespace - it is the # ClusterRoleBinding that makes it cluster-wide. The --serviceaccount flag takes ns:name. """, ) # ------------------------------------------------------------------ 76 ---- task( id=76, domain="WORK", title="Add an init container to the trip logger", task=""" Deployment `trip-logger` in Namespace `trip-tracker` must be given a startup step. Add to its Pod template: - an **init container** named `config-init` using `busybox:1.36` that writes the JSON `{"region":"us-east","mode":"production"}` to `/config/app-config.json`, - an `emptyDir` volume named `config-volume` mounted at `/config` in **both** the init container and the main container. Pods must end up Running with the init container Completed, and the file must be readable from the main container. """, setup=NS + r""" mkns trip-tracker $K -n trip-tracker delete deploy trip-logger --ignore-not-found --wait=true >/dev/null 2>&1 $K apply -f - >/dev/null <<'EOF' apiVersion: apps/v1 kind: Deployment metadata: {name: trip-logger, namespace: trip-tracker} spec: replicas: 1 selector: {matchLabels: {app: trip-logger}} template: metadata: {labels: {app: trip-logger}} spec: containers: - name: logger image: nginx:1.27 EOF $K -n trip-tracker rollout status deploy trip-logger --timeout=120s >/dev/null 2>&1 """, checks=[ C("init container config-init exists on the deployment", 2, r""" eq "$(j -n trip-tracker deploy trip-logger '{.spec.template.spec.initContainers[?(@.name=="config-init")].name}')" config-init"""), C("it uses busybox:1.36 and targets /config/app-config.json", 3, r""" c='{.spec.template.spec.initContainers[?(@.name=="config-init")]' eq "$(j -n trip-tracker deploy trip-logger "$c.image}")" busybox:1.36 \ && { j -n trip-tracker deploy trip-logger "$c.command[*]}"; j -n trip-tracker deploy trip-logger "$c.args[*]}"; } | grep -q '/config/app-config.json'"""), C("emptyDir config-volume mounted at /config in both containers", 2, r""" d="$K -n trip-tracker get deploy trip-logger" [ -n "$($d -o jsonpath='{.spec.template.spec.volumes[?(@.name=="config-volume")].emptyDir}')" ] \ && eq "$($d -o jsonpath='{.spec.template.spec.initContainers[?(@.name=="config-init")].volumeMounts[?(@.name=="config-volume")].mountPath}')" /config \ && eq "$($d -o jsonpath='{.spec.template.spec.containers[0].volumeMounts[?(@.name=="config-volume")].mountPath}')" /config"""), C("pods Running and the config file is present in the main container", 2, r""" for i in $(seq 1 20); do [ "$(j -n trip-tracker deploy trip-logger '{.status.readyReplicas}')" = 1 ] && break; sleep 5; done p=$(j -n trip-tracker pod -l app=trip-logger '{.items[0].metadata.name}'); [ -n "$p" ] || exit 1 $K -n trip-tracker exec "$p" -- cat /config/app-config.json 2>/dev/null | grep -q 'us-east'"""), ], solution=""" k -n trip-tracker edit deploy trip-logger # add to spec.template.spec: # volumes: # - {name: config-volume, emptyDir: {}} # initContainers: # - name: config-init # image: busybox:1.36 # command: ["sh","-c","echo '{\\"region\\":\\"us-east\\",\\"mode\\":\\"production\\"}' > /config/app-config.json"] # volumeMounts: [{name: config-volume, mountPath: /config}] # and the SAME volumeMount on the main container k -n trip-tracker rollout status deploy trip-logger # An init container runs to completion before the main containers start, and an emptyDir is the # handover: it lives as long as the Pod, so whatever init writes is there when the app boots. # Quoting that JSON inside a YAML command list is the fiddly part - single-quote the shell # string and escape the inner double quotes. """, ) # ------------------------------------------------------------------ 77 ---- task( id=77, domain="NET", title="Audit the CNI, then lock a namespace down", task=""" Before rolling out NetworkPolicies, confirm what the cluster's CNI can actually enforce. 1. Write the name of the DaemonSet in `kube-system` that provides pod networking to `{WS}/q77/cni.txt`. 2. Write `yes` or `no` to `{WS}/q77/enforced.txt`: does this CNI enforce NetworkPolicy? 3. In Namespace `zero-trust`, create NetworkPolicy `default-deny-all` that denies **all** ingress and egress for every Pod in the Namespace, with no allow rules of any kind. Answer step 2 from evidence, not memory — several popular CNIs accept NetworkPolicy objects happily and never enforce them, and a CNI's behaviour can change between releases. """, setup=NS + r""" mkns zero-trust rm -rf "$WS/q77"; mkdir -p "$WS/q77" $K -n zero-trust delete netpol default-deny-all --ignore-not-found >/dev/null 2>&1 """, checks=[ C("cni.txt names the CNI DaemonSet in kube-system", 2, r""" want=$($K -n kube-system get ds -o name 2>/dev/null | sed 's|.*/||' | grep -Ei 'kindnet|calico|cilium|flannel|weave' | head -1) [ -n "$want" ] || want=kindnet grep -qi "$want" "$WS/q77/cni.txt" 2>/dev/null"""), # The right answer depends on the build, so derive it: recent kindnet embeds a policy engine # (nftables table `kindnet-network-policies` on every node - verified enforcing on this # cluster), calico/cilium/weave enforce, flannel does not. C("enforced.txt gives the right answer for this cluster's CNI", 2, r""" ds=$($K -n kube-system get ds -o name 2>/dev/null | sed 's|.*/||' | tr '\n' ' ') case "$ds" in *calico*|*cilium*|*weave*|*antrea*) want=yes ;; *kindnet*) nx "$W1" 'nft list table inet kindnet-network-policies >/dev/null 2>&1' /dev/null"""), C("default-deny-all selects every pod in both directions", 4, r""" n="$K -n zero-trust get netpol default-deny-all" eq "$($n -o jsonpath='{.spec.podSelector}' 2>/dev/null)" '{}' \ && $n -o jsonpath='{.spec.policyTypes}' | grep -q Ingress \ && $n -o jsonpath='{.spec.policyTypes}' | grep -q Egress \ && [ -z "$($n -o jsonpath='{.spec.ingress}')" ] && [ -z "$($n -o jsonpath='{.spec.egress}')" ]"""), ], solution=[cmd(r"""k -n kube-system get ds echo kindnet > {WS}/q77/cni.txt""", r"""`kindnet` runs on every node: it is the CNI. `kube-proxy` is not."""), cmd(r"""k -n zero-trust run srv --image=nginx:1.27 --labels=app=srv k -n zero-trust expose pod srv --port 80 k -n zero-trust run cli --image=busybox:1.36 -- sleep 3600""", r"""Prove it rather than remember it: two Pods, a policy that denies everything, and a request between them."""), hand(r"""k -n zero-trust get pods""", r"""Again until both are `Running`.""", run=r"""kubectl -n zero-trust wait --for=condition=Ready pod/srv pod/cli --timeout=180s"""), cmd(r"""k -n zero-trust exec cli -- wget -qO- -T3 http://srv""", r"""nginx's page: no policy yet."""), hand(r"""vim deny.yaml""", r"""The default deny for both directions from the docs page *Network Policies*: every Pod, both policy types, no rules. ``` apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: default-deny-all namespace: zero-trust spec: podSelector: {} policyTypes: - Ingress - Egress ```""", run=r"""cat > deny.yaml <<'EOF' apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: default-deny-all namespace: zero-trust spec: podSelector: {} policyTypes: - Ingress - Egress EOF"""), cmd(r"""k apply -f deny.yaml k -n zero-trust exec cli -- wget -qO- -T3 http://srv""", r"""`download timed out`: the CNI enforces it. (Flannel would store the policy and enforce nothing; Calico and Cilium enforce. Recent kindnet has a policy engine of its own.)"""), hand(r"""echo yes > {WS}/q77/enforced.txt""", r"""What the test showed.""", run=r"""sleep 6 if kubectl -n zero-trust exec cli -- wget -qO- -T3 http://srv >/dev/null 2>&1; then echo no; else echo yes; fi > {WS}/q77/enforced.txt"""), cmd(r"""k -n zero-trust delete pod srv cli --now k -n zero-trust delete svc srv""", r"""Leave only the policy.""")], ) # ------------------------------------------------------------------ 78 ---- task( id=78, domain="WORK", title="Priority for the rider support portal", task=""" 1. Create PriorityClass `rider-support-priority` with value `1000000`, **not** the global default, and description `Priority for rider support services`. 2. Make Deployment `support-portal` in Namespace `rider-support` use it, and get the pods Running. """, setup=NS + r""" mkns rider-support $K delete priorityclass rider-support-priority --ignore-not-found >/dev/null 2>&1 $K -n rider-support delete deploy support-portal --ignore-not-found --wait=true >/dev/null 2>&1 $K -n rider-support create deploy support-portal --image=nginx:1.27 --replicas=2 >/dev/null $K -n rider-support rollout status deploy support-portal --timeout=120s >/dev/null 2>&1 """, checks=[ C("PriorityClass has value 1000000 and is not the global default", 3, r""" p="$K get priorityclass rider-support-priority" eq "$($p -o jsonpath='{.value}' 2>/dev/null)" 1000000 \ && [ "$($p -o jsonpath='{.globalDefault}')" != true ] \ && $p -o jsonpath='{.description}' | grep -q 'rider support'"""), C("the deployment's pod template requests that PriorityClass", 2, r""" eq "$(j -n rider-support deploy support-portal '{.spec.template.spec.priorityClassName}')" rider-support-priority"""), C("pods are Running and actually carry the priority value", 2, r""" for i in $(seq 1 20); do [ "$(j -n rider-support deploy support-portal '{.status.readyReplicas}')" = 2 ] && break; sleep 5; done eq "$(j -n rider-support deploy support-portal '{.status.readyReplicas}')" 2 \ && eq "$(j -n rider-support pod -l app=support-portal '{.items[0].spec.priority}')" 1000000"""), ], solution=[cmd(r"""k create priorityclass rider-support-priority --value=1000000 --description='Priority for rider support services'""", r"""Not the global default unless `--global-default`."""), hand(r"""k -n rider-support edit deploy support-portal""", r"""In the Pod template's `spec:`, beside `containers:` and indented like it, then `:wq`: ``` priorityClassName: rider-support-priority ``` The value is copied into each Pod when it is created, so only new Pods carry it - the edit rolls them.""", run=r"""kubectl -n rider-support patch deploy support-portal -p '{"spec":{"template":{"spec":{"priorityClassName":"rider-support-priority"}}}}'"""), cmd(r"""k -n rider-support rollout status deploy support-portal k -n rider-support get pods -o custom-columns=NAME:.metadata.name,PRIORITY:.spec.priority""")], ) # ------------------------------------------------------------------ 79 ---- task( id=79, domain="WORK", title="Surge pricing configuration", task=""" In Namespace `surge-pricing`: 1. Create ConfigMap `surge-config` with `PRICING_MODE=dynamic`, `MAX_MULTIPLIER=3.5`, `REGION=us-east-1`. 2. Create Secret `surge-db-secret` with keys `DB_HOST`, `DB_USER` and `DB_PASSWORD` (any values). 3. Deployment `surge-engine` must receive **every** key from both as environment variables, using `envFrom` — not one `env` entry per key. 4. The same ConfigMap must **also** be mounted as a volume at `/etc/surge-config`. Pods must be Running with the variables readable inside the container. """, setup=NS + r""" mkns surge-pricing $K -n surge-pricing delete cm surge-config --ignore-not-found >/dev/null 2>&1 $K -n surge-pricing delete secret surge-db-secret --ignore-not-found >/dev/null 2>&1 $K -n surge-pricing delete deploy surge-engine --ignore-not-found --wait=true >/dev/null 2>&1 $K -n surge-pricing create deploy surge-engine --image=nginx:1.27 >/dev/null $K -n surge-pricing rollout status deploy surge-engine --timeout=120s >/dev/null 2>&1 """, checks=[ C("ConfigMap surge-config holds all three keys", 2, r""" c="$K -n surge-pricing get cm surge-config" eq "$($c -o jsonpath='{.data.PRICING_MODE}' 2>/dev/null)" dynamic \ && eq "$($c -o jsonpath='{.data.MAX_MULTIPLIER}')" 3.5 \ && eq "$($c -o jsonpath='{.data.REGION}')" us-east-1"""), C("Secret surge-db-secret holds all three keys", 2, r""" s="$K -n surge-pricing get secret surge-db-secret" for k in DB_HOST DB_USER DB_PASSWORD; do [ -n "$($s -o jsonpath="{.data.$k}" 2>/dev/null)" ] || exit 1; done"""), C("the deployment uses envFrom for both the ConfigMap and the Secret", 3, r""" e=$(j -n surge-pricing deploy surge-engine '{.spec.template.spec.containers[0].envFrom}') echo "$e" | grep -q 'surge-config' && echo "$e" | grep -q 'surge-db-secret' \ && echo "$e" | grep -q configMapRef && echo "$e" | grep -q secretRef"""), C("the ConfigMap is also mounted as a volume at /etc/surge-config", 2, r""" d="$K -n surge-pricing get deploy surge-engine" $d -o jsonpath='{.spec.template.spec.volumes[*].configMap.name}' | grep -qw surge-config \ && $d -o jsonpath='{.spec.template.spec.containers[0].volumeMounts[*].mountPath}' | grep -q '/etc/surge-config'"""), C("pods Running with the variables visible in the container", 2, r""" for i in $(seq 1 20); do [ "$(j -n surge-pricing deploy surge-engine '{.status.readyReplicas}')" -ge 1 ] 2>/dev/null && break; sleep 5; done p=$(j -n surge-pricing pod -l app=surge-engine '{.items[0].metadata.name}'); [ -n "$p" ] || exit 1 $K -n surge-pricing exec "$p" -- env 2>/dev/null | grep -q 'PRICING_MODE=dynamic' \ && $K -n surge-pricing exec "$p" -- env 2>/dev/null | grep -q 'DB_HOST='"""), ], solution=""" k -n surge-pricing create cm surge-config \\ --from-literal=PRICING_MODE=dynamic --from-literal=MAX_MULTIPLIER=3.5 --from-literal=REGION=us-east-1 k -n surge-pricing create secret generic surge-db-secret \\ --from-literal=DB_HOST=pricing-db.surge-pricing.svc.cluster.local \\ --from-literal=DB_USER=surge_admin --from-literal=DB_PASSWORD='S3cur3Pr1c1ng!' k -n surge-pricing edit deploy surge-engine # in the container: # envFrom: # - configMapRef: {name: surge-config} # - secretRef: {name: surge-db-secret} # volumeMounts: [{name: config-vol, mountPath: /etc/surge-config}] # and at pod level: volumes: [{name: config-vol, configMap: {name: surge-config}}] k -n surge-pricing rollout status deploy surge-engine # envFrom pulls EVERY key in one line; `env` with valueFrom is for cherry-picking single keys or # renaming them. As a volume each key becomes a file named after the key - the same ConfigMap can # legitimately be consumed both ways at once, which is exactly what this task asks for. """, ) # ------------------------------------------------------------------ 80 ---- task( id=80, domain="ARCH", title="Plan the control-plane upgrade", task=""" The control-plane node is scheduled for a minor-version upgrade. Prepare the change — **do not execute an upgrade**. 1. Write the cluster's current server version, as `kubectl` reports it, to `{WS}/q80/current.txt`. 2. Write every node's name and kubelet version, one node per line, to `{WS}/q80/kubelets.txt`. 3. Write to `{WS}/q80/steps.sh` the ordered commands to upgrade the **control-plane** node — the kubeadm package install, the kubeadm upgrade step, draining, the kubelet/kubectl package install, restarting the kubelet, and uncordoning — one command per line, in execution order. 4. Write to `{WS}/q80/worker.txt` the single kubeadm subcommand that upgrades a **worker** node instead (it is not `upgrade apply`). """, setup=r"""rm -rf "$WS/q80"; mkdir -p "$WS/q80" """, checks=[ C("current.txt matches the live server version", 2, r""" v=$($K version -o json 2>/dev/null | python3 -c 'import sys,json;print(json.load(sys.stdin)["serverVersion"]["gitVersion"])') [ -n "$v" ] && grep -qF "$v" "$WS/q80/current.txt" 2>/dev/null"""), C("kubelets.txt lists every node with its kubelet version", 2, r""" f="$WS/q80/kubelets.txt"; [ -s "$f" ] || exit 1 for n in $($K get nodes -o jsonpath='{.items[*].metadata.name}'); do grep -q "$n" "$f" || exit 1; done v=$($K get nodes -o jsonpath='{.items[0].status.nodeInfo.kubeletVersion}') grep -qF "$v" "$f" """), C("steps.sh has the upgrade commands in a workable order", 4, r""" f="$WS/q80/steps.sh"; [ -f "$f" ] || exit 1 grep -q 'kubeadm upgrade apply' "$f" || exit 1 grep -Eq '(apt|yum|dnf).*install.*kubeadm' "$f" || exit 1 grep -Eq '(apt|yum|dnf).*install.*kubelet' "$f" || exit 1 grep -q 'drain' "$f" && grep -q 'uncordon' "$f" || exit 1 grep -Eq 'systemctl (restart|daemon-reload)' "$f" || exit 1 ap=$(grep -n 'kubeadm upgrade apply' "$f" | head -1 | cut -d: -f1) ka=$(grep -nE '(apt|yum|dnf).*install.*kubeadm' "$f" | head -1 | cut -d: -f1) kl=$(grep -nE '(apt|yum|dnf).*install.*kubelet' "$f" | head -1 | cut -d: -f1) un=$(grep -n 'uncordon' "$f" | head -1 | cut -d: -f1) [ "$ka" -lt "$ap" ] && [ "$ap" -lt "$kl" ] && [ "$kl" -lt "$un" ]"""), C("worker.txt names `kubeadm upgrade node`", 2, r""" grep -Eq 'kubeadm[[:space:]]+upgrade[[:space:]]+node' "$WS/q80/worker.txt" 2>/dev/null \ && ! grep -q 'upgrade apply' "$WS/q80/worker.txt" """), ], solution=[cmd(r"""k version k get nodes -o custom-columns=NAME:.metadata.name,KUBELET:.status.nodeInfo.kubeletVersion --no-headers > {WS}/q80/kubelets.txt cat {WS}/q80/kubelets.txt"""), hand(r"""echo v1.37.0 > {WS}/q80/current.txt""", r"""With the `Server Version` that `k version` printed.""", run=r"""kubectl version -o json | jq -r .serverVersion.gitVersion > {WS}/q80/current.txt"""), hand(r"""vim {WS}/q80/steps.sh""", r"""The order of the docs page *Upgrading kubeadm clusters*: kubeadm first, which upgrades the control plane; then the node's kubelet and kubectl, drained; uncordon last. The next minor is 1.38 - first point `/etc/apt/sources.list.d/kubernetes.list` at the v1.38 repository, and take its newest patch (`apt-cache madison kubeadm`): ``` apt-mark unhold kubeadm && apt-get update && apt-get install -y kubeadm=1.38.0-1.1 && apt-mark hold kubeadm kubeadm upgrade plan kubeadm upgrade apply v1.38.0 kubectl drain {CP} --ignore-daemonsets apt-mark unhold kubelet kubectl && apt-get update && apt-get install -y kubelet=1.38.0-1.1 kubectl=1.38.0-1.1 && apt-mark hold kubelet kubectl systemctl daemon-reload systemctl restart kubelet kubectl uncordon {CP} ```""", run=r"""cat > {WS}/q80/steps.sh <<'EOF' apt-mark unhold kubeadm && apt-get update && apt-get install -y kubeadm=1.38.0-1.1 && apt-mark hold kubeadm kubeadm upgrade plan kubeadm upgrade apply v1.38.0 kubectl drain {CP} --ignore-daemonsets apt-mark unhold kubelet kubectl && apt-get update && apt-get install -y kubelet=1.38.0-1.1 kubectl=1.38.0-1.1 && apt-mark hold kubelet kubectl systemctl daemon-reload systemctl restart kubelet kubectl uncordon {CP} EOF"""), cmd(r"""echo 'kubeadm upgrade node' > {WS}/q80/worker.txt""", r"""A worker hosts no control plane, so there is nothing for `upgrade apply` to do there; `upgrade node` updates its kubelet configuration.""")], ) # ------------------------------------------------------------------ 81 ---- task( id=81, domain="TSHOOT", title="Fix the fare calculation pods", task=""" Deployment `fare-calculator` in Namespace `fare-engine` has no running pods. There are **two** independent reasons, one storage and one scheduling. Diagnose both and get the pods Running and Ready. You may change the Deployment and the claim, but keep the Deployment's name. """, setup=NS + r""" mkns fare-engine $K -n fare-engine delete deploy fare-calculator --ignore-not-found --wait=true >/dev/null 2>&1 $K -n fare-engine delete pvc fare-data-pvc --ignore-not-found --wait=true >/dev/null 2>&1 for n in "$W1" "$W2" "$W3"; do $K label node "$n" disk- >/dev/null 2>&1 || true; done $K apply -f - >/dev/null <<'EOF' apiVersion: v1 kind: PersistentVolumeClaim metadata: {name: fare-data-pvc, namespace: fare-engine} spec: accessModes: [ReadWriteOnce] resources: {requests: {storage: 100Mi}} storageClassName: fast-storage --- apiVersion: apps/v1 kind: Deployment metadata: {name: fare-calculator, namespace: fare-engine} spec: replicas: 1 selector: {matchLabels: {app: fare-calculator}} template: metadata: {labels: {app: fare-calculator}} spec: affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: [{key: disk, operator: In, values: [nvme]}] volumes: [{name: data, persistentVolumeClaim: {claimName: fare-data-pvc}}] containers: - name: fare image: nginx:1.27 volumeMounts: [{name: data, mountPath: /data}] EOF """, checks=[ C("fare-data-pvc is Bound using a StorageClass that exists", 3, r""" eq "$(j -n fare-engine pvc fare-data-pvc '{.status.phase}')" Bound || exit 1 sc=$(j -n fare-engine pvc fare-data-pvc '{.spec.storageClassName}') [ -z "$sc" ] || $K get sc "$sc" >/dev/null 2>&1"""), C("the pods can actually be scheduled", 3, r""" for i in $(seq 1 24); do n=$(j -n fare-engine pod -l app=fare-calculator '{.items[0].spec.nodeName}') [ -n "$n" ] && exit 0; sleep 5 done; exit 1"""), C("fare-calculator pods are Running and Ready", 3, r""" for i in $(seq 1 24); do [ "$(j -n fare-engine deploy fare-calculator '{.status.readyReplicas}')" -ge 1 ] 2>/dev/null && exit 0; sleep 5; done; exit 1"""), ], solution=[cmd(r"""k -n fare-engine get pvc,pods k -n fare-engine describe pvc fare-data-pvc | tail -3 k get sc""", r"""`storageclass.storage.k8s.io "fast-storage" not found`: the class here is `standard`."""), cmd(r"""k -n fare-engine delete pvc fare-data-pvc""", r"""A claim's `storageClassName` cannot change: make it again."""), hand(r"""vim pvc.yaml""", r"""``` apiVersion: v1 kind: PersistentVolumeClaim metadata: name: fare-data-pvc namespace: fare-engine spec: storageClassName: standard accessModes: - ReadWriteOnce resources: requests: storage: 100Mi ```""", run=r"""cat > pvc.yaml <<'EOF' apiVersion: v1 kind: PersistentVolumeClaim metadata: name: fare-data-pvc namespace: fare-engine spec: storageClassName: standard accessModes: - ReadWriteOnce resources: requests: storage: 100Mi EOF"""), cmd(r"""k apply -f pvc.yaml k -n fare-engine describe pod -l app=fare-calculator | tail -3 k get nodes -L disk""", r"""`didn't match Pod's node affinity/selector`: it wants `disk=nvme`, which no node has. `describe pod` names only this one; the claim's fault you had to find yourself."""), cmd(r"""k label node {W1} disk=nvme k -n fare-engine rollout status deploy fare-calculator""", r"""Labelling a node is the smaller change than dropping the rule. `standard` binds only once the Pod is placed (`WaitForFirstConsumer`), so the claim goes `Bound` now too.""")], ) # ------------------------------------------------------------------ 82 ---- task( id=82, domain="WORK", title="Autoscale the route planner on memory", task=""" Create HorizontalPodAutoscaler `route-planner-hpa` in Namespace `route-planner` for Deployment `route-planner`: - minimum 1 replica, maximum 6, - scales on **memory** utilisation with a target average of 70%, - on the way up, may add at most **2 pods per 60 seconds**. Use the `autoscaling/v2` API. """, setup=NS + r""" mkns route-planner $K -n route-planner delete hpa route-planner-hpa --ignore-not-found >/dev/null 2>&1 $K -n route-planner delete deploy route-planner --ignore-not-found --wait=true >/dev/null 2>&1 $K -n route-planner create deploy route-planner --image=nginx:1.27 --replicas=1 >/dev/null $K -n route-planner set resources deploy route-planner --requests=cpu=50m,memory=64Mi >/dev/null $K -n route-planner rollout status deploy route-planner --timeout=120s >/dev/null 2>&1 """, checks=[ C("HPA targets the route-planner Deployment via autoscaling/v2", 2, r""" h="$K -n route-planner get hpa route-planner-hpa" eq "$($h -o jsonpath='{.spec.scaleTargetRef.kind}' 2>/dev/null)" Deployment \ && eq "$($h -o jsonpath='{.spec.scaleTargetRef.name}')" route-planner \ && $K -n route-planner get hpa.v2.autoscaling route-planner-hpa >/dev/null 2>&1"""), C("min 1, max 6 replicas", 2, r""" h="$K -n route-planner get hpa route-planner-hpa" eq "$($h -o jsonpath='{.spec.minReplicas}' 2>/dev/null)" 1 && eq "$($h -o jsonpath='{.spec.maxReplicas}')" 6"""), C("scales on memory utilisation at 70%", 3, r""" $K -n route-planner get hpa route-planner-hpa -o json 2>/dev/null | python3 -c ' import sys,json ms=json.load(sys.stdin)["spec"].get("metrics",[]) ok=any(m.get("type")=="Resource" and m["resource"]["name"]=="memory" and m["resource"]["target"].get("type")=="Utilization" and int(m["resource"]["target"].get("averageUtilization",0))==70 for m in ms) sys.exit(0 if ok else 1)'"""), C("scaleUp policy allows at most 2 pods per 60 seconds", 2, r""" $K -n route-planner get hpa route-planner-hpa -o json 2>/dev/null | python3 -c ' import sys,json b=json.load(sys.stdin)["spec"].get("behavior",{}).get("scaleUp",{}) ok=any(p.get("type")=="Pods" and int(p.get("value",0))==2 and int(p.get("periodSeconds",0))==60 for p in b.get("policies",[])) sys.exit(0 if ok else 1)'"""), ], solution=[cmd(r"""k -n route-planner autoscale deploy route-planner --name=route-planner-hpa --min=1 --max=6 --memory=70% --dry-run=client -o yaml > hpa.yaml""", r"""`autoscale` writes an `autoscaling/v2` HPA with the memory target; it has no flag for `behavior`."""), hand(r"""vim hpa.yaml""", r"""Add to its `spec:`, beside `maxReplicas:` (the docs page *Horizontal Pod Autoscaling* has the block under *Configurable scaling behavior*): ``` behavior: scaleUp: policies: - type: Pods value: 2 periodSeconds: 60 ```""", run=r"""yq -i '.spec.behavior.scaleUp.policies = [{"type": "Pods", "value": 2, "periodSeconds": 60}]' hpa.yaml"""), cmd(r"""k apply -f hpa.yaml k -n route-planner get hpa route-planner-hpa""", r"""Utilisation is a percentage of the request: the Deployment has a memory request, so the target reads a number, not ``.""")], ) # ------------------------------------------------------------------ 83 ---- task( id=83, domain="NET", title="NetworkPolicy for the safety monitor", task=""" In Namespace `safety-monitor`, create NetworkPolicy `safety-api-policy` for the Pods labelled `app=safety-api`: - **ingress**: only from Pods labelled `app=safety-dashboard`, on TCP 8443, - **egress**: to Pods labelled `app=safety-db` on TCP 5432, - **egress**: DNS on port 53, both UDP and TCP, to any destination, - everything else denied in both directions. (This cluster's CNI enforces policies, so a wrong policy really breaks traffic — but you are graded on the object.) """, setup=NS + r""" mkns safety-monitor $K -n safety-monitor delete netpol safety-api-policy --ignore-not-found >/dev/null 2>&1 for a in safety-api safety-dashboard safety-db; do $K -n safety-monitor create deploy $a --image=nginx:1.27 --dry-run=client -o yaml | $K apply -f - >/dev/null $K -n safety-monitor patch deploy $a -p "{\"spec\":{\"template\":{\"metadata\":{\"labels\":{\"app\":\"$a\"}}}}}" >/dev/null done """, checks=[ C("selects app=safety-api and declares both policyTypes", 2, r""" n="$K -n safety-monitor get netpol safety-api-policy" eq "$($n -o jsonpath='{.spec.podSelector.matchLabels.app}' 2>/dev/null)" safety-api \ && $n -o jsonpath='{.spec.policyTypes}' | grep -q Ingress && $n -o jsonpath='{.spec.policyTypes}' | grep -q Egress"""), C("ingress only from app=safety-dashboard on TCP 8443", 3, r""" $K -n safety-monitor get netpol safety-api-policy -o json 2>/dev/null | python3 -c ' import sys,json ing=json.load(sys.stdin)["spec"].get("ingress",[]) ok=len(ing)==1 and len(ing[0].get("from",[]))==1 \ and ing[0]["from"][0].get("podSelector",{}).get("matchLabels",{}).get("app")=="safety-dashboard" \ and any(str(p.get("port"))=="8443" and p.get("protocol","TCP")=="TCP" for p in ing[0].get("ports",[])) sys.exit(0 if ok else 1)'"""), C("egress to app=safety-db on TCP 5432", 3, r""" $K -n safety-monitor get netpol safety-api-policy -o json 2>/dev/null | python3 -c ' import sys,json eg=json.load(sys.stdin)["spec"].get("egress",[]) ok=any(any(t.get("podSelector",{}).get("matchLabels",{}).get("app")=="safety-db" for t in r.get("to",[])) and any(str(p.get("port"))=="5432" for p in r.get("ports",[])) for r in eg) sys.exit(0 if ok else 1)'"""), C("egress DNS on 53 over both UDP and TCP, unrestricted destination", 2, r""" $K -n safety-monitor get netpol safety-api-policy -o json 2>/dev/null | python3 -c ' import sys,json eg=json.load(sys.stdin)["spec"].get("egress",[]) ok=any(not r.get("to") and {p.get("protocol") for p in r.get("ports",[]) if str(p.get("port"))=="53"}>={"UDP","TCP"} for r in eg) sys.exit(0 if ok else 1)'"""), ], solution=[hand(r"""vim netpol.yaml""", r"""Paste the example from the docs page *Network Policies* and change it to: ``` apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: safety-api-policy namespace: safety-monitor spec: podSelector: matchLabels: app: safety-api policyTypes: - Ingress - Egress ingress: - from: - podSelector: matchLabels: app: safety-dashboard ports: - protocol: TCP port: 8443 egress: - to: - podSelector: matchLabels: app: safety-db ports: - protocol: TCP port: 5432 - ports: - protocol: UDP port: 53 - protocol: TCP port: 53 ``` The DNS rule has no `to:` - only ports, so port 53 anywhere. With `Egress` in `policyTypes` everything else outbound is denied, and forgetting DNS breaks every name lookup in the Pod.""", run=r"""cat > netpol.yaml <<'EOF' apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: safety-api-policy namespace: safety-monitor spec: podSelector: matchLabels: app: safety-api policyTypes: - Ingress - Egress ingress: - from: - podSelector: matchLabels: app: safety-dashboard ports: - protocol: TCP port: 8443 egress: - to: - podSelector: matchLabels: app: safety-db ports: - protocol: TCP port: 5432 - ports: - protocol: UDP port: 53 - protocol: TCP port: 53 EOF"""), cmd(r"""k apply -f netpol.yaml k -n safety-monitor describe netpol safety-api-policy""")], ) # ------------------------------------------------------------------ 84 ---- task( id=84, domain="ARCH", title="Enforce TLS 1.3 on the booking portal", task=""" Namespace `booking-tls` runs Deployment `booking-nginx`, which serves HTTPS using the nginx configuration in ConfigMap `booking-nginx-config`. It currently accepts TLS 1.2 **and** 1.3. 1. Change the ConfigMap so only TLS 1.3 is accepted. 2. Make the running pods pick up the change, without editing the Deployment's pod template by hand. 3. Write the nginx directive line you ended up with, as it appears in the ConfigMap, to `{WS}/q84/ssl-line.txt`. A TLS 1.2 handshake against the service must be refused afterwards. """, setup=NS + r""" mkns booking-tls rm -rf "$WS/q84"; mkdir -p "$WS/q84" $K -n booking-tls delete deploy booking-nginx --ignore-not-found --wait=true >/dev/null 2>&1 $K -n booking-tls delete cm booking-nginx-config --ignore-not-found >/dev/null 2>&1 $K -n booking-tls delete secret booking-tls-cert --ignore-not-found >/dev/null 2>&1 $K -n booking-tls delete svc booking-nginx-svc --ignore-not-found >/dev/null 2>&1 d=$(mktemp -d) openssl req -x509 -newkey rsa:2048 -nodes -days 30 -subj "/CN=booking.example.com" \ -keyout "$d/tls.key" -out "$d/tls.crt" >/dev/null 2>&1 $K -n booking-tls create secret tls booking-tls-cert --cert="$d/tls.crt" --key="$d/tls.key" >/dev/null rm -rf "$d" $K -n booking-tls create cm booking-nginx-config --from-literal=default.conf='server { listen 443 ssl; server_name booking.example.com; ssl_certificate /etc/nginx/tls/tls.crt; ssl_certificate_key /etc/nginx/tls/tls.key; ssl_protocols TLSv1.2 TLSv1.3; location / { return 200 "booking ok\n"; } }' >/dev/null $K apply -f - >/dev/null <<'EOF' apiVersion: apps/v1 kind: Deployment metadata: {name: booking-nginx, namespace: booking-tls} spec: replicas: 1 selector: {matchLabels: {app: booking-nginx}} template: metadata: {labels: {app: booking-nginx}} spec: volumes: - {name: conf, configMap: {name: booking-nginx-config}} - {name: certs, secret: {secretName: booking-tls-cert}} containers: - name: web image: nginx:1.27 ports: [{containerPort: 443}] volumeMounts: - {name: conf, mountPath: /etc/nginx/conf.d} - {name: certs, mountPath: /etc/nginx/tls} --- apiVersion: v1 kind: Service metadata: {name: booking-nginx-svc, namespace: booking-tls} spec: type: NodePort selector: {app: booking-nginx} ports: [{port: 443, targetPort: 443, nodePort: 30443}] EOF $K -n booking-tls rollout status deploy booking-nginx --timeout=120s >/dev/null 2>&1 """, checks=[ C("the ConfigMap allows TLSv1.3 only", 3, r""" c=$(j -n booking-tls cm booking-nginx-config '{.data.default\.conf}') echo "$c" | grep -q 'ssl_protocols.*TLSv1\.3' && ! echo "$c" | grep -q 'TLSv1\.2'"""), # rollout restart (restartedAt annotation) or deleting the pods - either way every running # pod must be younger than the last write to the ConfigMap C("the pods were restarted to pick the change up", 2, r""" j -n booking-tls deploy booking-nginx '{.spec.template.metadata.annotations}' | grep -q 'restartedAt' && exit 0 cm_t=$($K -n booking-tls get cm booking-nginx-config -o json --show-managed-fields 2>/dev/null | python3 -c 'import sys,json; print(max(m.get("time","") for m in json.load(sys.stdin)["metadata"].get("managedFields",[])))') ts=$(j -n booking-tls pod -l app=booking-nginx '{.items[*].metadata.creationTimestamp}'); [ -n "$cm_t" ] && [ -n "$ts" ] || exit 1 for t in $ts; do [[ "$t" > "$cm_t" ]] || exit 1; done"""), C("ssl-line.txt records the directive", 2, r""" f="$WS/q84/ssl-line.txt" grep -q 'ssl_protocols' "$f" 2>/dev/null && grep -q 'TLSv1\.3' "$f" && ! grep -q 'TLSv1\.2' "$f" """), C("TLS 1.3 is accepted and TLS 1.2 is refused", 3, r""" for i in $(seq 1 20); do [ "$(j -n booking-tls deploy booking-nginx '{.status.readyReplicas}')" = 1 ] && break; sleep 5; done np=$(j -n booking-tls svc booking-nginx-svc '{.spec.ports[0].nodePort}') [ -n "$np" ] || exit 1 # Probe from a node, not from this machine: under rootless podman the kind node IPs are not # routable from the host, so a host-side openssl just hangs. Every node answers its own NodePort. nx "$W1" "echo | timeout 10 openssl s_client -connect 127.0.0.1:$np -tls1_3 >/dev/null 2>&1" /dev/null 2>&1" ssl_protocols TLSv1.3; k -n booking-tls rollout restart deploy booking-nginx k -n booking-tls rollout status deploy booking-nginx echo 'ssl_protocols TLSv1.3;' > {WS}/q84/ssl-line.txt # verify on the host, which is a node itself (the Service is published as NodePort 30443): # echo | openssl s_client -connect 127.0.0.1:30443 -tls1_3 # connects # echo | openssl s_client -connect 127.0.0.1:30443 -tls1_2 # alert: protocol version # A ConfigMap mounted as a volume is updated in place by the kubelet, but nginx only reads its # config at startup - so the file changes and the running server does not. `rollout restart` # adds a kubectl.kubernetes.io/restartedAt annotation to the pod template, which is a real # template change and therefore a real rollout, without you editing the spec by hand. """, ) # ------------------------------------------------------------------ 85 ---- task( id=85, domain="TSHOOT", title="Investigate vehicle tracker health", task=""" Namespace `vehicle-track` has a misbehaving workload. Collect evidence: 1. List **all** pods in **all** namespaces sorted by restart count, and save the output to `{WS}/q85/most-restarts.txt`. 2. Save the last 50 log lines from the `app=vehicle-tracker` pods to `{WS}/q85/logs.txt`. 3. Save the Warning events from `vehicle-track`, sorted by time, to `{WS}/q85/warnings.txt`. """, setup=NS + r""" mkns vehicle-track rm -rf "$WS/q85"; mkdir -p "$WS/q85" $K -n vehicle-track delete deploy vehicle-tracker --ignore-not-found --wait=true >/dev/null 2>&1 $K apply -f - >/dev/null <<'EOF' apiVersion: apps/v1 kind: Deployment metadata: {name: vehicle-tracker, namespace: vehicle-track} spec: replicas: 1 selector: {matchLabels: {app: vehicle-tracker}} template: metadata: {labels: {app: vehicle-tracker}} spec: containers: - name: tracker image: busybox:1.36 command: [sh, -c, "echo tracker starting; echo gps fix lost; sleep 12; exit 1"] EOF """, checks=[ C("most-restarts.txt is a sorted all-namespace pod listing", 3, r""" f="$WS/q85/most-restarts.txt"; [ -s "$f" ] || exit 1 grep -qi 'NAMESPACE' "$f" && grep -q 'kube-system' "$f" && grep -q 'vehicle-track' "$f" \ && [ "$(wc -l < "$f")" -ge 5 ]"""), C("logs.txt holds the tracker's log output", 2, r""" grep -q 'tracker starting\|gps fix lost' "$WS/q85/logs.txt" 2>/dev/null"""), C("warnings.txt holds Warning events from the namespace", 3, r""" f="$WS/q85/warnings.txt"; [ -s "$f" ] || exit 1 grep -q 'Warning' "$f" && ! grep -q '^Normal' "$f" """), ], solution=""" k get pods -A --sort-by='.status.containerStatuses[0].restartCount' > {WS}/q85/most-restarts.txt k -n vehicle-track logs -l app=vehicle-tracker --tail=50 > {WS}/q85/logs.txt k -n vehicle-track get events --field-selector type=Warning --sort-by='.lastTimestamp' > {WS}/q85/warnings.txt # --sort-by takes a JSONPath into the object and sorts ascending, so the worst offenders land at # the BOTTOM of that first file. `logs -l` reads every pod matching the label, which is what you # want for a Deployment whose pod name keeps changing under a CrashLoopBackOff. """, ) QUESTIONS = Q