vumast3r
49fcb76166
Audit of the bank and harness after a 23% mock result. Two of these were
silently corrupting every practice session on this cluster.
restore_baseline:
- The w1-taint cleanup removed nothing at all. `kubectl taint node N a- b- c-`
aborts on the first key that is not present and applies none of the removals,
so with only `workload` set the command reported five "not found" lines,
exited 1, and left the taint in place - the precise failure the block's own
comment says it exists to prevent. W1 then rejected pods in every later
session. Now one call per key. Verified by re-seeding workload=gpu:NoSchedule
and confirming a single restore_baseline run clears it.
- metrics-server was never repaired. Q23 seeds its scenario by stripping
--kubelet-insecure-tls, but carried no "breaks" key, so nothing tracked it
and nothing undid it. On kind the kubelet serving certs have no IP SANs, so
without that flag metrics-server never goes Ready, v1beta1.metrics.k8s.io
has no endpoints, aggregated discovery goes stale, and EVERY namespace
deletion in the cluster hangs on NamespaceDeletionDiscoveryFailure - which
is where the ~40 stuck namespaces came from, plus a dead `kubectl top`.
Q23 now declares breaks: metrics-server and the baseline restores the flag.
Baseline runs before seeding, so a session containing Q23 re-breaks it a
moment later as intended. Verified by stripping the flag and confirming one
restore_baseline run puts it back.
Session composition:
- pick() dropped any domain whose tasks are all "hard" from a normal session.
CKS SUPPLY and RUNTIME hold one hard task each, so `./exam start` returned
6 of 9 tasks with 40% of the blueprint silently absent. It now tops up from
the hard pool when a domain's normal pool is short.
- compose_exam() iterated QUOTA in declaration order, so a one-task domain
lost every "breaks" tie to a bigger domain that claimed the shared slot
first: CKS RUNTIME (only Q9, apiserver-flag) was never drawn in ANY mode
because SETUP went first and took that slot with Q2. Now walks domains
scarcest-first, and warns on a shortfall instead of truncating in silence.
CKS now yields 8/9 covering all six domains, warning SETUP 1/2.
- session_questions() rebuilt the list by filtering QUESTIONS (sorted by id),
discarding the session's own order - so compose_exam's rng.shuffle never
reached the candidate and every exam arrived grouped by domain and
difficulty, base tasks then hard then sim sets. Fixed to preserve the stored
order, and cmd_drill now shuffles explicit-id drills after conflict
resolution, so which task gets dropped stays deterministic while the order
does not.
Bank correctness (verified by grading each reference solution):
- CKAD Q2 was unpassable, 5/8. Its init container waited on `until nslookup
backend`, but busybox nslookup exits 1 even on a successful lookup because
it walks the whole search list and the trailing NXDOMAINs set the status -
the loop never terminates. The premise was wrong too: a ClusterIP Service
resolves from creation, endpoints or not, so DNS could never have gated it.
Now waits on a real connection; 8/8, and the pod genuinely holds at Init:0/1
until the backend Deployment exists. Same bug class as e3f5ecd.
- CKS Q1 was unsatisfiable as written, 6/10. The text asked for three
NetworkPolicies but the grader required a fourth: one podSelector cannot
cover both the web egress half and the db ingress half. Task text now asks
for four and says why.
- CKS Q4 check 4 was a false positive. `... || r=0` read a failed exec as
"no token mounted", awarding 3 points when the candidate had done nothing
and the probe pod simply could not start - which happens often on this
cluster. Now requires the pod to reach Running and exec to work. Verified
it awards the points before the fix and fails after, with pod creation
blocked by a quota.
- CKAD Q8's solution asserted a Pod's image cannot be changed in place.
It can - spec.containers[].image is mutable; `kubectl set image pod/...`
works and grades 7/7. Rewritten, noting which fields really are immutable.
Also: the `sim` usage and help strings hardcoded "a|b" and a "17-task" count,
wrong for a third set and wrong for CKAD/CKS, which define no sets at all. Both
are now generated from SETS.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NziZdvBvy7CpEAAbZNgp8i
575 lines
28 KiB
Python
575 lines
28 KiB
Python
"""CKAD track. Starter bank - same format as the CKA banks, extend it the same way.
|
|
|
|
CKAD is a build-things exam: less cluster surgery, more "produce the right object, fast".
|
|
The clock is the adversary, so these are mostly single-object tasks graded exactingly.
|
|
"""
|
|
from tracks.cka.common import C, NS, ns_pick, name_pick
|
|
|
|
TRACK = {
|
|
"id": "ckad",
|
|
"name": "Certified Kubernetes Application Developer",
|
|
"minutes": 120,
|
|
"pass_pct": 66,
|
|
"domains": {
|
|
"DESIGN": "Application Design and Build (20%)",
|
|
"DEPLOY": "Application Deployment (20%)",
|
|
"OBS": "Application Observability and Maintenance (15%)",
|
|
"ENV": "Application Environment, Configuration and Security (25%)",
|
|
"NET": "Services and Networking (20%)",
|
|
},
|
|
"quota": {"DESIGN": 3, "DEPLOY": 3, "OBS": 2, "ENV": 2, "NET": 2},
|
|
"quota_hard": {"DESIGN": 3, "DEPLOY": 3, "OBS": 2, "ENV": 2, "NET": 2},
|
|
}
|
|
|
|
QUESTIONS = [
|
|
# ---------------------------------------------------------------- DESIGN ----
|
|
{
|
|
"id": 1, "domain": "DESIGN", "title": "Adapter sidecar",
|
|
"params": lambda r: {"ns": ns_pick(r, "app"), "pod": name_pick(r, "adapter")},
|
|
"task": """
|
|
Create Pod `%{pod}` in Namespace `%{ns}` with two containers sharing an `emptyDir` named `spool`
|
|
mounted at `/spool` in both:
|
|
|
|
- `writer` (busybox:1.36): every 2 seconds appends a line `metric 1` to `/spool/raw.txt`
|
|
- `adapter` (busybox:1.36): runs `tail -f /spool/raw.txt`
|
|
|
|
`kubectl logs %{pod} -c adapter` must show the writer's lines.
|
|
""",
|
|
"setup": NS + r"""mkns %{ns}; $K -n %{ns} delete pod %{pod} --ignore-not-found --wait=false >/dev/null""",
|
|
"checks": [
|
|
C("pod Running with containers writer and adapter", 2, r"""
|
|
p="$K -n %{ns} get pod %{pod}"
|
|
eq "$($p -o jsonpath='{.status.phase}' 2>/dev/null)" Running && $p -o jsonpath='{.spec.containers[*].name}' | grep -qw writer && $p -o jsonpath='{.spec.containers[*].name}' | grep -qw adapter"""),
|
|
C("emptyDir 'spool' mounted at /spool in both containers", 2, r"""
|
|
p="$K -n %{ns} get pod %{pod}"
|
|
[ -n "$($p -o jsonpath='{.spec.volumes[?(@.name=="spool")].emptyDir}')" ] \
|
|
&& eq "$($p -o jsonpath='{.spec.containers[?(@.name=="writer")].volumeMounts[?(@.name=="spool")].mountPath}')" /spool \
|
|
&& eq "$($p -o jsonpath='{.spec.containers[?(@.name=="adapter")].volumeMounts[?(@.name=="spool")].mountPath}')" /spool"""),
|
|
C("adapter logs show the writer's output", 2, r"""sleep 3; $K -n %{ns} logs %{pod} -c adapter --tail=3 2>/dev/null | grep -q 'metric 1'"""),
|
|
],
|
|
"solution": """
|
|
cat <<EOF | k apply -f -
|
|
apiVersion: v1
|
|
kind: Pod
|
|
metadata: {name: %{pod}, namespace: %{ns}}
|
|
spec:
|
|
volumes: [{name: spool, emptyDir: {}}]
|
|
containers:
|
|
- name: writer
|
|
image: busybox:1.36
|
|
command: [sh, -c, "while true; do echo 'metric 1' >> /spool/raw.txt; sleep 2; done"]
|
|
volumeMounts: [{name: spool, mountPath: /spool}]
|
|
- name: adapter
|
|
image: busybox:1.36
|
|
command: [sh, -c, "tail -f /spool/raw.txt"]
|
|
volumeMounts: [{name: spool, mountPath: /spool}]
|
|
EOF
|
|
k -n %{ns} logs %{pod} -c adapter
|
|
""",
|
|
},
|
|
{
|
|
"id": 2, "domain": "DESIGN", "title": "Init container waits for a dependency",
|
|
"params": lambda r: {"ns": ns_pick(r, "boot"), "pod": name_pick(r, "consumer")},
|
|
"task": """
|
|
Namespace `%{ns}` has a Service `backend` with no endpoints yet.
|
|
|
|
1. Create Pod `%{pod}` with an init container `wait-for-backend` (busybox:1.36) that blocks until
|
|
Service `backend` actually **answers on port 80**, and a main container `app` (nginx:1.27).
|
|
It will sit in `Init:0/1`.
|
|
2. Then create Deployment `backend` (nginx:1.27, 1 replica, Pod label `app=backend`) so the
|
|
Service gets an endpoint and your Pod starts.
|
|
|
|
Two traps here: a ClusterIP Service gets a DNS record the moment it is created, endpoints or not,
|
|
so waiting on name resolution alone never blocks. And busybox `nslookup` exits non-zero even when
|
|
it succeeds, because the search-domain attempts after the hit return NXDOMAIN. Wait on a real
|
|
connection instead.
|
|
""",
|
|
"setup": NS + r"""
|
|
mkns %{ns}
|
|
$K -n %{ns} delete pod %{pod} --ignore-not-found --wait=false >/dev/null
|
|
$K -n %{ns} delete deploy backend --ignore-not-found >/dev/null
|
|
$K apply -f - >/dev/null <<EOF
|
|
apiVersion: v1
|
|
kind: Service
|
|
metadata: {name: backend, namespace: %{ns}}
|
|
spec: {selector: {app: backend}, ports: [{port: 80}]}
|
|
EOF
|
|
""",
|
|
"checks": [
|
|
C("pod has an init container named wait-for-backend", 2, r"""[ -n "$(j -n %{ns} pod %{pod} '{.spec.initContainers[?(@.name=="wait-for-backend")].name}')" ]"""),
|
|
C("backend Deployment available with an endpoint on the Service", 3, r"""
|
|
eq "$(j -n %{ns} deploy backend '{.status.availableReplicas}')" 1 \
|
|
&& [ -n "$(j -n %{ns} endpointslices -l kubernetes.io/service-name=backend '{.items[*].endpoints[*].addresses[*]}')" ]"""),
|
|
C("pod is now Running", 3, r"""for i in $(seq 1 12); do [ "$(j -n %{ns} pod %{pod} '{.status.phase}')" = Running ] && exit 0; sleep 5; done; exit 1"""),
|
|
],
|
|
"solution": """
|
|
cat <<EOF | k apply -f -
|
|
apiVersion: v1
|
|
kind: Pod
|
|
metadata: {name: %{pod}, namespace: %{ns}}
|
|
spec:
|
|
initContainers:
|
|
- name: wait-for-backend
|
|
image: busybox:1.36
|
|
command: [sh, -c, "until wget -T 2 -q -O- http://backend:80 >/dev/null 2>&1; do echo waiting; sleep 2; done"]
|
|
containers: [{name: app, image: nginx:1.27}]
|
|
EOF
|
|
k -n %{ns} get pod %{pod} # Init:0/1
|
|
k -n %{ns} create deploy backend --image=nginx:1.27
|
|
k -n %{ns} patch deploy backend -p '{"spec":{"template":{"metadata":{"labels":{"app":"backend"}}}}}'
|
|
# A ClusterIP Service resolves in DNS from the moment it exists - endpoints change nothing about
|
|
# that - so `until nslookup backend` would not block even before you add the Deployment. Worse,
|
|
# busybox nslookup returns 1 even on a successful lookup, because it walks the whole search list
|
|
# and the trailing NXDOMAINs set the exit status, so that loop never terminates either way.
|
|
# With no endpoints, kube-proxy REJECTs the connection, so wget fails fast and the loop is real.
|
|
""",
|
|
},
|
|
{
|
|
"id": 3, "domain": "DESIGN", "title": "CronJob with deadlines",
|
|
"params": lambda r: {"ns": ns_pick(r, "batch"), "cj": name_pick(r, "digest"), "m": r.choice([3, 5, 7])},
|
|
"task": """
|
|
In `%{ns}` create CronJob `%{cj}`:
|
|
- schedule: every %{m} minutes
|
|
- image `busybox:1.36`, command prints `digest`
|
|
- a run that cannot start within 10 seconds of its schedule is skipped
|
|
- a run that takes longer than 20 seconds is killed
|
|
- keep 2 successful and 2 failed Jobs
|
|
- restartPolicy `Never`
|
|
""",
|
|
"setup": NS + r"""mkns %{ns}; $K -n %{ns} delete cj %{cj} --ignore-not-found >/dev/null""",
|
|
"checks": [
|
|
C("schedule and history limits", 2, r"""
|
|
c="$K -n %{ns} get cj %{cj}"
|
|
eq "$($c -o jsonpath='{.spec.schedule}' 2>/dev/null)" "*/%{m} * * * *" && eq "$($c -o jsonpath='{.spec.successfulJobsHistoryLimit}')" 2 && eq "$($c -o jsonpath='{.spec.failedJobsHistoryLimit}')" 2"""),
|
|
C("startingDeadlineSeconds 10 and activeDeadlineSeconds 20", 3, r"""
|
|
c="$K -n %{ns} get cj %{cj}"
|
|
eq "$($c -o jsonpath='{.spec.startingDeadlineSeconds}' 2>/dev/null)" 10 && eq "$($c -o jsonpath='{.spec.jobTemplate.spec.activeDeadlineSeconds}')" 20"""),
|
|
C("restartPolicy Never", 1, r"""eq "$(j -n %{ns} cj %{cj} '{.spec.jobTemplate.spec.template.spec.restartPolicy}')" Never"""),
|
|
],
|
|
"solution": """
|
|
k -n %{ns} create cj %{cj} --image=busybox:1.36 --schedule='*/%{m} * * * *' $do -- sh -c "echo digest" > cj.yaml
|
|
# spec.startingDeadlineSeconds: 10 (skip a late start)
|
|
# spec.successfulJobsHistoryLimit: 2 / failedJobsHistoryLimit: 2
|
|
# spec.jobTemplate.spec.activeDeadlineSeconds: 20 (kill an overrunning run)
|
|
# spec.jobTemplate.spec.template.spec.restartPolicy: Never
|
|
k apply -f cj.yaml
|
|
# startingDeadlineSeconds lives on the CronJob, activeDeadlineSeconds on the Job template.
|
|
""",
|
|
},
|
|
# ---------------------------------------------------------------- DEPLOY ----
|
|
{
|
|
"id": 4, "domain": "DEPLOY", "title": "Roll forward, then back",
|
|
"params": lambda r: {"ns": ns_pick(r, "rel"), "dep": name_pick(r, "frontend")},
|
|
"task": """
|
|
Deployment `%{dep}` in `%{ns}` runs `nginx:1.25` with 3 replicas.
|
|
|
|
1. Update the image to `nginx:1.27` and record a change-cause annotation of `pick 1.27`.
|
|
2. Then update it to `nginx:9.99-nope`, which will never become ready.
|
|
3. Roll back so all 3 replicas are Available on `nginx:1.27` again.
|
|
""",
|
|
"setup": NS + r"""
|
|
mkns %{ns}
|
|
$K -n %{ns} delete deploy %{dep} --ignore-not-found --wait=true >/dev/null
|
|
$K -n %{ns} create deploy %{dep} --image=nginx:1.25 --replicas=3 >/dev/null
|
|
$K -n %{ns} rollout status deploy %{dep} --timeout=120s >/dev/null 2>&1
|
|
""",
|
|
"checks": [
|
|
C("running image is nginx:1.27 with 3 available", 4, r"""
|
|
eq "$(j -n %{ns} deploy %{dep} '{.spec.template.spec.containers[0].image}')" nginx:1.27 && eq "$(j -n %{ns} deploy %{dep} '{.status.availableReplicas}')" 3"""),
|
|
C("rollout history has at least three revisions", 2, r"""[ "$($K -n %{ns} rollout history deploy %{dep} 2>/dev/null | grep -c '^[0-9]')" -ge 3 ]"""),
|
|
C("a change-cause annotation is recorded", 2, r"""j -n %{ns} deploy %{dep} '{.metadata.annotations.kubernetes\.io/change-cause}' | grep -q ."""),
|
|
],
|
|
"solution": """
|
|
k -n %{ns} set image deploy %{dep} nginx=nginx:1.27
|
|
k -n %{ns} annotate deploy %{dep} kubernetes.io/change-cause='pick 1.27' --overwrite
|
|
k -n %{ns} rollout status deploy %{dep}
|
|
k -n %{ns} set image deploy %{dep} nginx=nginx:9.99-nope
|
|
k -n %{ns} rollout status deploy %{dep} --timeout=30s # never completes
|
|
k -n %{ns} rollout history deploy %{dep}
|
|
k -n %{ns} rollout undo deploy %{dep}
|
|
k -n %{ns} rollout status deploy %{dep}
|
|
""",
|
|
},
|
|
{
|
|
"id": 5, "domain": "DEPLOY", "title": "Blue/green cutover",
|
|
"params": lambda r: {"ns": ns_pick(r, "bg"), "svc": name_pick(r, "shop")},
|
|
"task": """
|
|
Namespace `%{ns}` has Deployment `%{svc}-blue` (Pod labels `app=%{svc}`, `version=blue`) behind
|
|
Service `%{svc}`.
|
|
|
|
1. Create Deployment `%{svc}-green`: 2 replicas, image `nginx:1.27`, Pod labels `app=%{svc}` and
|
|
`version=green`.
|
|
2. When it is fully Available, switch Service `%{svc}` to serve green only.
|
|
3. Leave blue running untouched so the switch can be reversed.
|
|
""",
|
|
"setup": NS + r"""
|
|
mkns %{ns}
|
|
$K -n %{ns} delete deploy %{svc}-blue %{svc}-green --ignore-not-found >/dev/null
|
|
$K -n %{ns} delete svc %{svc} --ignore-not-found >/dev/null
|
|
$K apply -f - >/dev/null <<EOF
|
|
apiVersion: apps/v1
|
|
kind: Deployment
|
|
metadata: {name: %{svc}-blue, namespace: %{ns}}
|
|
spec:
|
|
replicas: 2
|
|
selector: {matchLabels: {app: %{svc}, version: blue}}
|
|
template:
|
|
metadata: {labels: {app: %{svc}, version: blue}}
|
|
spec: {containers: [{name: web, image: nginx:1.25, ports: [{containerPort: 80}]}]}
|
|
---
|
|
apiVersion: v1
|
|
kind: Service
|
|
metadata: {name: %{svc}, namespace: %{ns}}
|
|
spec: {selector: {app: %{svc}, version: blue}, ports: [{port: 80, targetPort: 80}]}
|
|
EOF
|
|
""",
|
|
"checks": [
|
|
C("green Deployment available with the right labels", 3, r"""
|
|
eq "$(j -n %{ns} deploy %{svc}-green '{.status.availableReplicas}')" 2 \
|
|
&& eq "$(j -n %{ns} deploy %{svc}-green '{.spec.template.metadata.labels.version}')" green \
|
|
&& eq "$(j -n %{ns} deploy %{svc}-green '{.spec.template.metadata.labels.app}')" %{svc}"""),
|
|
C("Service selector now targets version=green", 3, r"""eq "$(j -n %{ns} svc %{svc} '{.spec.selector.version}')" green"""),
|
|
C("every Service endpoint belongs to a green Pod", 2, r"""
|
|
ips=$(j -n %{ns} endpointslices -l kubernetes.io/service-name=%{svc} '{.items[*].endpoints[*].addresses[*]}')
|
|
[ -n "$ips" ] || exit 1
|
|
for ip in $ips; do j -n %{ns} pod -l version=green '{.items[*].status.podIP}' | grep -qw "$ip" || exit 1; done"""),
|
|
C("blue is still running", 2, r"""eq "$(j -n %{ns} deploy %{svc}-blue '{.status.availableReplicas}')" 2"""),
|
|
],
|
|
"solution": """
|
|
k -n %{ns} get deploy %{svc}-blue -o yaml > green.yaml
|
|
# rename to %{svc}-green, change every version: blue -> green, image -> nginx:1.27,
|
|
# strip status/uid/resourceVersion/creationTimestamp
|
|
k apply -f green.yaml
|
|
k -n %{ns} rollout status deploy %{svc}-green
|
|
k -n %{ns} patch svc %{svc} -p '{"spec":{"selector":{"app":"%{svc}","version":"green"}}}'
|
|
k -n %{ns} get endpointslices -l kubernetes.io/service-name=%{svc}
|
|
# Rolling back is the same patch with version: blue - which is why blue stays up.
|
|
""",
|
|
},
|
|
{
|
|
"id": 6, "domain": "DEPLOY", "title": "Helm release lifecycle",
|
|
"params": lambda r: {"ns": ns_pick(r, "helm"), "rel": name_pick(r, "site")},
|
|
"task": """
|
|
A chart is at `{WS}/q6/chart`.
|
|
|
|
1. Install it into Namespace `%{ns}` (create it) as release `%{rel}` with `replicaCount=2`.
|
|
2. Upgrade the release to `replicaCount=4`.
|
|
3. Roll the release back so the Deployment is at 2 replicas again — without uninstalling it.
|
|
4. Write the release's current revision number to `{WS}/q6/revision.txt`.
|
|
""",
|
|
"setup": r"""
|
|
rm -rf "$WS/q6"; mkdir -p "$WS/q6/chart/templates"
|
|
cat > "$WS/q6/chart/Chart.yaml" <<'EOF'
|
|
apiVersion: v2
|
|
name: site
|
|
version: 0.1.0
|
|
EOF
|
|
cat > "$WS/q6/chart/values.yaml" <<'EOF'
|
|
replicaCount: 1
|
|
image: nginx:1.27
|
|
EOF
|
|
cat > "$WS/q6/chart/templates/deploy.yaml" <<'EOF'
|
|
apiVersion: apps/v1
|
|
kind: Deployment
|
|
metadata: {name: {{ .Release.Name }}-site}
|
|
spec:
|
|
replicas: {{ .Values.replicaCount }}
|
|
selector: {matchLabels: {app: {{ .Release.Name }}-site}}
|
|
template:
|
|
metadata: {labels: {app: {{ .Release.Name }}-site}}
|
|
spec: {containers: [{name: web, image: "{{ .Values.image }}"}]}
|
|
EOF
|
|
$K create ns %{ns} --dry-run=client -o yaml | $K apply -f - >/dev/null
|
|
helm -n %{ns} uninstall %{rel} >/dev/null 2>&1 || true
|
|
command -v helm >/dev/null || echo "warn: helm not installed"
|
|
""",
|
|
"checks": [
|
|
C("release deployed with at least three revisions of history", 3, r"""
|
|
helm -n %{ns} status %{rel} -o json 2>/dev/null | python3 -c 'import sys,json;sys.exit(0 if json.load(sys.stdin)["info"]["status"]=="deployed" else 1)' || exit 1
|
|
[ "$(helm -n %{ns} history %{rel} -o json 2>/dev/null | python3 -c 'import sys,json;print(len(json.load(sys.stdin)))')" -ge 3 ]"""),
|
|
C("Deployment is back to 2 replicas", 3, r"""eq "$(j -n %{ns} deploy %{rel}-site '{.spec.replicas}')" 2"""),
|
|
C("revision.txt matches the release's current revision", 2, r"""
|
|
cur=$(helm -n %{ns} history %{rel} -o json 2>/dev/null | python3 -c 'import sys,json;print(max(r["revision"] for r in json.load(sys.stdin)))' 2>/dev/null)
|
|
[ -n "$cur" ] && eq "$(tr -dc '0-9' < "$WS/q6/revision.txt" 2>/dev/null)" "$cur" """),
|
|
],
|
|
"solution": """
|
|
k create ns %{ns}
|
|
helm -n %{ns} install %{rel} {WS}/q6/chart --set replicaCount=2
|
|
helm -n %{ns} upgrade %{rel} {WS}/q6/chart --set replicaCount=4
|
|
helm -n %{ns} history %{rel}
|
|
helm -n %{ns} rollback %{rel} 1 # creates a NEW revision identical to revision 1
|
|
helm -n %{ns} history %{rel} | grep '^[0-9]' | tail -1 | awk '{print $1}' > {WS}/q6/revision.txt
|
|
""",
|
|
},
|
|
# ------------------------------------------------------------------- OBS ----
|
|
{
|
|
"id": 7, "domain": "OBS", "title": "Three kinds of probe",
|
|
"params": lambda r: {"ns": ns_pick(r, "probe"), "dep": name_pick(r, "api")},
|
|
"task": """
|
|
Deployment `%{dep}` in `%{ns}` (nginx:1.27, 2 replicas) needs health checks:
|
|
|
|
- **startup**: HTTP GET `/` on port 80, `periodSeconds: 5`, allowed a total of 60 seconds to come up
|
|
- **readiness**: HTTP GET `/` on port 80, every 5 seconds, a single failure removes it from endpoints
|
|
- **liveness**: TCP on port 80, every 10 seconds, initial delay 15 seconds
|
|
|
|
All replicas must end up Ready.
|
|
""",
|
|
"setup": NS + r"""
|
|
mkns %{ns}
|
|
$K -n %{ns} delete deploy %{dep} --ignore-not-found >/dev/null
|
|
$K -n %{ns} create deploy %{dep} --image=nginx:1.27 --replicas=2 >/dev/null
|
|
""",
|
|
"checks": [
|
|
C("startupProbe: httpGet / :80, period 5, failureThreshold 12", 3, r"""
|
|
c='{.spec.template.spec.containers[0].startupProbe'
|
|
eq "$(j -n %{ns} deploy %{dep} "$c.httpGet.path}")" / && eq "$(j -n %{ns} deploy %{dep} "$c.periodSeconds}")" 5 && eq "$(j -n %{ns} deploy %{dep} "$c.failureThreshold}")" 12"""),
|
|
C("readinessProbe: httpGet, period 5, failureThreshold 1", 3, r"""
|
|
c='{.spec.template.spec.containers[0].readinessProbe'
|
|
eq "$(j -n %{ns} deploy %{dep} "$c.httpGet.path}")" / && eq "$(j -n %{ns} deploy %{dep} "$c.periodSeconds}")" 5 && eq "$(j -n %{ns} deploy %{dep} "$c.failureThreshold}")" 1"""),
|
|
C("livenessProbe: tcpSocket 80, period 10, initialDelay 15", 3, r"""
|
|
c='{.spec.template.spec.containers[0].livenessProbe'
|
|
eq "$(j -n %{ns} deploy %{dep} "$c.tcpSocket.port}")" 80 && eq "$(j -n %{ns} deploy %{dep} "$c.periodSeconds}")" 10 && eq "$(j -n %{ns} deploy %{dep} "$c.initialDelaySeconds}")" 15"""),
|
|
C("all replicas Ready", 2, r"""eq "$(j -n %{ns} deploy %{dep} '{.status.readyReplicas}')" 2"""),
|
|
],
|
|
"solution": """
|
|
k -n %{ns} edit deploy %{dep} # add to the container:
|
|
# startupProbe: {httpGet: {path: /, port: 80}, periodSeconds: 5, failureThreshold: 12}
|
|
# readinessProbe: {httpGet: {path: /, port: 80}, periodSeconds: 5, failureThreshold: 1}
|
|
# livenessProbe: {tcpSocket: {port: 80}, periodSeconds: 10, initialDelaySeconds: 15}
|
|
k -n %{ns} rollout status deploy %{dep}
|
|
# 12 x 5s = 60s. While a startupProbe runs, liveness and readiness are suspended, which is why
|
|
# it beats a long initialDelaySeconds on liveness for slow starters.
|
|
""",
|
|
},
|
|
{
|
|
"id": 8, "domain": "OBS", "title": "Diagnose and fix a broken Pod",
|
|
"params": lambda r: {"ns": ns_pick(r, "debug"), "pod": name_pick(r, "worker")},
|
|
"task": """
|
|
Pod `%{pod}` in `%{ns}` is not running.
|
|
|
|
1. Write the container's waiting **reason**, as reported in the Pod's status, to `{WS}/q8/reason.txt`.
|
|
2. Fix the Pod so it runs. Keep the name, Namespace and command — correct only what is wrong.
|
|
""",
|
|
"setup": NS + r"""
|
|
mkns %{ns}; rm -rf "$WS/q8"; mkdir -p "$WS/q8"
|
|
$K -n %{ns} delete pod %{pod} --ignore-not-found --wait=true >/dev/null
|
|
$K apply -f - >/dev/null <<EOF
|
|
apiVersion: v1
|
|
kind: Pod
|
|
metadata: {name: %{pod}, namespace: %{ns}}
|
|
spec:
|
|
containers:
|
|
- name: worker
|
|
image: busybox:1.99-doesnotexist
|
|
command: [sh, -c, "echo working; sleep 86400"]
|
|
EOF
|
|
""",
|
|
"checks": [
|
|
C("reason.txt records the image pull failure", 3, r"""grep -Eqi 'ImagePull|ErrImagePull|ImagePullBackOff' "$WS/q8/reason.txt" 2>/dev/null"""),
|
|
C("pod Running with its original command", 4, r"""
|
|
eq "$(j -n %{ns} pod %{pod} '{.status.phase}')" Running && j -n %{ns} pod %{pod} '{.spec.containers[0].command}' | grep -q sleep"""),
|
|
],
|
|
"solution": """
|
|
k -n %{ns} get pod %{pod}
|
|
k -n %{ns} get pod %{pod} -o jsonpath='{.status.containerStatuses[0].state.waiting.reason}'
|
|
# ImagePullBackOff -> {WS}/q8/reason.txt
|
|
k -n %{ns} describe pod %{pod} | tail -15 # "Failed to pull image ... not found"
|
|
k -n %{ns} set image pod/%{pod} worker=busybox:1.36
|
|
# `image` is one of the very few mutable fields on a running Pod, so no delete/recreate is needed
|
|
# here - the kubelet just pulls the new image and restarts the container. Most other spec fields
|
|
# (command, volumes, securityContext) really are immutable, and for those the fix is:
|
|
# k -n %{ns} get pod %{pod} -o yaml > fix.yaml # edit, strip status/uid/resourceVersion
|
|
# k -n %{ns} delete pod %{pod} --now && k apply -f fix.yaml
|
|
""",
|
|
},
|
|
# ------------------------------------------------------------------- ENV ----
|
|
{
|
|
"id": 9, "domain": "ENV", "title": "SecurityContext and capabilities",
|
|
"params": lambda r: {"ns": ns_pick(r, "sec"), "pod": name_pick(r, "hardened"), "uid": r.choice([1001, 2000, 10001])},
|
|
"task": """
|
|
Create Pod `%{pod}` in `%{ns}` (image `busybox:1.36`, command `sleep 86400`) that:
|
|
|
|
- runs as user %{uid} and group %{uid}, and must not run as root,
|
|
- has a read-only root filesystem,
|
|
- drops **all** capabilities and adds back only `NET_BIND_SERVICE`,
|
|
- cannot gain new privileges.
|
|
""",
|
|
"setup": NS + r"""mkns %{ns}; $K -n %{ns} delete pod %{pod} --ignore-not-found --wait=false >/dev/null""",
|
|
"checks": [
|
|
C("runs as UID %{uid} with runAsNonRoot", 3, r"""
|
|
p="$K -n %{ns} get pod %{pod}"
|
|
u=$($p -o jsonpath='{.spec.securityContext.runAsUser}' 2>/dev/null); [ -z "$u" ] && u=$($p -o jsonpath='{.spec.containers[0].securityContext.runAsUser}')
|
|
eq "$u" %{uid} && $p -o jsonpath='{.spec.securityContext.runAsNonRoot}{.spec.containers[0].securityContext.runAsNonRoot}' | grep -q true"""),
|
|
C("readOnlyRootFilesystem and allowPrivilegeEscalation=false", 3, r"""
|
|
c='{.spec.containers[0].securityContext'
|
|
eq "$(j -n %{ns} pod %{pod} "$c.readOnlyRootFilesystem}")" true && eq "$(j -n %{ns} pod %{pod} "$c.allowPrivilegeEscalation}")" false"""),
|
|
C("capabilities drop ALL, add NET_BIND_SERVICE", 3, r"""
|
|
c='{.spec.containers[0].securityContext.capabilities'
|
|
j -n %{ns} pod %{pod} "$c.drop[*]}" | grep -q ALL && j -n %{ns} pod %{pod} "$c.add[*]}" | grep -q NET_BIND_SERVICE"""),
|
|
C("pod Running as that UID", 2, r"""
|
|
eq "$(j -n %{ns} pod %{pod} '{.status.phase}')" Running && $K -n %{ns} exec %{pod} -- id 2>/dev/null | grep -q 'uid=%{uid}'"""),
|
|
],
|
|
"solution": """
|
|
cat <<EOF | k apply -f -
|
|
apiVersion: v1
|
|
kind: Pod
|
|
metadata: {name: %{pod}, namespace: %{ns}}
|
|
spec:
|
|
securityContext: {runAsUser: %{uid}, runAsGroup: %{uid}, runAsNonRoot: true}
|
|
containers:
|
|
- name: c
|
|
image: busybox:1.36
|
|
command: [sleep, "86400"]
|
|
securityContext:
|
|
readOnlyRootFilesystem: true
|
|
allowPrivilegeEscalation: false
|
|
capabilities: {drop: [ALL], add: [NET_BIND_SERVICE]}
|
|
EOF
|
|
k -n %{ns} exec %{pod} -- id
|
|
# runAsUser/runAsGroup/runAsNonRoot exist at pod OR container level;
|
|
# capabilities, readOnlyRootFilesystem and allowPrivilegeEscalation are container-level only.
|
|
""",
|
|
},
|
|
{
|
|
"id": 10, "domain": "ENV", "title": "Projected volume: config, secret and token",
|
|
"params": lambda r: {"ns": ns_pick(r, "cfg"), "pod": name_pick(r, "combined")},
|
|
"task": """
|
|
In `%{ns}`:
|
|
1. Create ConfigMap `app-cfg` with `mode=live` and Secret `app-sec` with `apikey=xyz789`.
|
|
2. Create Pod `%{pod}` (busybox:1.36, `sleep 86400`) with a **single projected volume** named
|
|
`bundle` mounted at `/etc/bundle`, exposing:
|
|
- ConfigMap key `mode` as file `mode`,
|
|
- Secret key `apikey` as file `apikey`,
|
|
- a ServiceAccount token with audience `vault` and expiry 3600 seconds as file `token`.
|
|
""",
|
|
"setup": NS + r"""
|
|
mkns %{ns}
|
|
$K -n %{ns} delete pod %{pod} --ignore-not-found --wait=false >/dev/null
|
|
$K -n %{ns} delete cm app-cfg --ignore-not-found >/dev/null; $K -n %{ns} delete secret app-sec --ignore-not-found >/dev/null
|
|
""",
|
|
"checks": [
|
|
C("ConfigMap and Secret hold the right values", 2, r"""
|
|
eq "$(j -n %{ns} cm app-cfg '{.data.mode}')" live && eq "$(j -n %{ns} secret app-sec '{.data.apikey}' | base64 -d)" xyz789"""),
|
|
C("one projected volume 'bundle' with all three sources", 3, r"""
|
|
s=$(j -n %{ns} pod %{pod} '{.spec.volumes[?(@.name=="bundle")].projected.sources}')
|
|
echo "$s" | grep -q configMap && echo "$s" | grep -q secret && echo "$s" | grep -q serviceAccountToken"""),
|
|
C("token source: audience vault, expiry 3600", 3, r"""
|
|
s=$(j -n %{ns} pod %{pod} '{.spec.volumes[?(@.name=="bundle")].projected.sources[*].serviceAccountToken}')
|
|
echo "$s" | grep -q '"audience":"vault"' && echo "$s" | grep -q '"expirationSeconds":3600'"""),
|
|
C("all three files readable at /etc/bundle", 3, r"""
|
|
eq "$($K -n %{ns} exec %{pod} -- cat /etc/bundle/mode 2>/dev/null)" live \
|
|
&& eq "$($K -n %{ns} exec %{pod} -- cat /etc/bundle/apikey 2>/dev/null)" xyz789 \
|
|
&& [ -n "$($K -n %{ns} exec %{pod} -- cat /etc/bundle/token 2>/dev/null)" ]"""),
|
|
],
|
|
"solution": """
|
|
k -n %{ns} create cm app-cfg --from-literal=mode=live
|
|
k -n %{ns} create secret generic app-sec --from-literal=apikey=xyz789
|
|
cat <<EOF | k apply -f -
|
|
apiVersion: v1
|
|
kind: Pod
|
|
metadata: {name: %{pod}, namespace: %{ns}}
|
|
spec:
|
|
volumes:
|
|
- name: bundle
|
|
projected:
|
|
sources:
|
|
- configMap: {name: app-cfg, items: [{key: mode, path: mode}]}
|
|
- secret: {name: app-sec, items: [{key: apikey, path: apikey}]}
|
|
- serviceAccountToken: {audience: vault, expirationSeconds: 3600, path: token}
|
|
containers:
|
|
- name: c
|
|
image: busybox:1.36
|
|
command: [sleep, "86400"]
|
|
volumeMounts: [{name: bundle, mountPath: /etc/bundle}]
|
|
EOF
|
|
k -n %{ns} exec %{pod} -- ls /etc/bundle
|
|
""",
|
|
},
|
|
# ------------------------------------------------------------------- NET ----
|
|
{
|
|
"id": 11, "domain": "NET", "title": "Expose a Deployment two ways",
|
|
"params": lambda r: {"ns": ns_pick(r, "expose"), "dep": name_pick(r, "portal"), "np": r.choice([30310, 30320, 30330])},
|
|
"task": """
|
|
In `%{ns}` Deployment `%{dep}` (nginx:1.27, 2 replicas) serves on container port 80.
|
|
|
|
1. Expose it internally as ClusterIP Service `%{dep}`, port 8080 → 80.
|
|
2. Also expose it as NodePort Service `%{dep}-ext`, port 80 → 80, nodePort %{np}.
|
|
3. Create Ingress `%{dep}` (class `nginx`, host `%{dep}.example.com`, path `/` Prefix) pointing at
|
|
the **ClusterIP** Service on port 8080.
|
|
""",
|
|
"setup": NS + r"""
|
|
mkns %{ns}
|
|
$K -n %{ns} delete deploy %{dep} --ignore-not-found >/dev/null
|
|
$K -n %{ns} delete svc %{dep} %{dep}-ext --ignore-not-found >/dev/null
|
|
$K -n %{ns} delete ingress %{dep} --ignore-not-found >/dev/null
|
|
$K -n %{ns} create deploy %{dep} --image=nginx:1.27 --replicas=2 --port=80 >/dev/null
|
|
""",
|
|
"checks": [
|
|
C("ClusterIP Service 8080 -> 80 with endpoints", 3, r"""
|
|
s="$K -n %{ns} get svc %{dep}"
|
|
eq "$($s -o jsonpath='{.spec.type}' 2>/dev/null)" ClusterIP && eq "$($s -o jsonpath='{.spec.ports[0].port}')" 8080 && eq "$($s -o jsonpath='{.spec.ports[0].targetPort}')" 80 \
|
|
&& [ -n "$(j -n %{ns} endpointslices -l kubernetes.io/service-name=%{dep} '{.items[*].endpoints[*].addresses[*]}')" ]"""),
|
|
C("NodePort Service on %{np}", 2, r"""
|
|
s="$K -n %{ns} get svc %{dep}-ext"
|
|
eq "$($s -o jsonpath='{.spec.type}' 2>/dev/null)" NodePort && eq "$($s -o jsonpath='{.spec.ports[0].nodePort}')" %{np}"""),
|
|
C("Ingress routes the host to the ClusterIP Service on 8080", 3, r"""
|
|
i="$K -n %{ns} get ingress %{dep}"
|
|
eq "$($i -o jsonpath='{.spec.ingressClassName}' 2>/dev/null)" nginx && eq "$($i -o jsonpath='{.spec.rules[0].host}')" %{dep}.example.com \
|
|
&& eq "$($i -o jsonpath='{.spec.rules[0].http.paths[0].backend.service.name}')" %{dep} \
|
|
&& eq "$($i -o jsonpath='{.spec.rules[0].http.paths[0].backend.service.port.number}')" 8080"""),
|
|
],
|
|
"solution": """
|
|
k -n %{ns} expose deploy %{dep} --port=8080 --target-port=80
|
|
k -n %{ns} expose deploy %{dep} --name=%{dep}-ext --type=NodePort --port=80 --target-port=80 $do > ext.yaml
|
|
# set spec.ports[0].nodePort: %{np}
|
|
k apply -f ext.yaml
|
|
k -n %{ns} create ingress %{dep} --class=nginx --rule="%{dep}.example.com/*=%{dep}:8080"
|
|
""",
|
|
},
|
|
{
|
|
"id": 12, "domain": "NET", "title": "Restrict a namespace with NetworkPolicy",
|
|
"params": lambda r: {"ns": ns_pick(r, "np"), "client": r.choice(["frontend", "gateway", "portal"])},
|
|
"task": """
|
|
Namespace `%{ns}` runs Pods labelled `app=db` and `app=%{client}`.
|
|
|
|
Create NetworkPolicy `db-allow` so `app=db` Pods accept ingress **only** from `app=%{client}`
|
|
Pods in the same Namespace on TCP 5432, and nothing else. Do not restrict egress.
|
|
|
|
(kindnet doesn't enforce policies here, so you are graded on the object.)
|
|
""",
|
|
"setup": NS + r"""
|
|
mkns %{ns}
|
|
for a in db %{client}; do
|
|
$K -n %{ns} create deploy $a --image=nginx:1.27 --dry-run=client -o yaml | $K apply -f - >/dev/null
|
|
$K -n %{ns} patch deploy $a -p "{\"spec\":{\"template\":{\"metadata\":{\"labels\":{\"app\":\"$a\"}}}}}" >/dev/null
|
|
done
|
|
$K -n %{ns} delete netpol db-allow --ignore-not-found >/dev/null
|
|
""",
|
|
"checks": [
|
|
C("selects app=db, Ingress only", 2, r"""
|
|
n="$K -n %{ns} get netpol db-allow"
|
|
eq "$($n -o jsonpath='{.spec.podSelector.matchLabels.app}' 2>/dev/null)" db && eq "$($n -o jsonpath='{.spec.policyTypes}')" '["Ingress"]'"""),
|
|
C("exactly one rule: from app=%{client} on TCP 5432", 4, r"""
|
|
$K -n %{ns} get netpol db-allow -o json 2>/dev/null | python3 -c '
|
|
import sys,json
|
|
i=json.load(sys.stdin)["spec"].get("ingress",[])
|
|
ok=len(i)==1 and len(i[0].get("from",[]))==1 and i[0]["from"][0].get("podSelector",{}).get("matchLabels",{}).get("app")=="%{client}" \
|
|
and len(i[0].get("ports",[]))==1 and str(i[0]["ports"][0]["port"])=="5432"
|
|
sys.exit(0 if ok else 1)'"""),
|
|
],
|
|
"solution": """
|
|
cat <<EOF | k apply -f -
|
|
apiVersion: networking.k8s.io/v1
|
|
kind: NetworkPolicy
|
|
metadata: {name: db-allow, namespace: %{ns}}
|
|
spec:
|
|
podSelector: {matchLabels: {app: db}}
|
|
policyTypes: [Ingress]
|
|
ingress:
|
|
- from: [{podSelector: {matchLabels: {app: %{client}}}}]
|
|
ports: [{protocol: TCP, port: 5432}]
|
|
EOF
|
|
""",
|
|
},
|
|
]
|