fix: repair the CKA bank and harness after a live audit; add Simulator Set D

Every changed task was seeded, graded untouched, solved and re-graded on a
separate kind cluster. The four sim sets were rehearsed as full sittings (real
baseline, seed everything, solve in order, grade at the end): A 150/150,
B 176/176, C 163/163, D 122/122. Every random-pool task was either passed in a
real session or cycled the same way.

Graders that could never pass, or passed for free:
  Q306 (local-path cannot resize; now graded as an in-place resize), Q314 (GNU
  grep prefixed file names), Q305 (plain `get events` has no SOURCE column and
  events expire; now managedFields), Q410 (kindnet DOES enforce NetworkPolicy;
  the expected answer is now derived from the CNI), Q35, Q212 (missing `]`),
  Q101 (`can-i create pods/exec` asks about a pod NAMED exec; multi-type get),
  Q109 (an emptyDir over nginx's /var/log crash-loops it), Q40 (TTL was
  measured at grading time), Q121, Q106, Q104, Q41, Q26, Q403, Q409, Q417,
  Q113, Q22, Q24/Q108 (flaky right after a rollout), and free points on
  untouched Q24/Q102/Q108/Q119/Q122.

Reference solutions: S[43], S[107], S[121], S[106] rewritten; S[116] (a
Completed Job pod still pins its PVC), S[119] (a pods quota equal to replicas
deadlocks the rollout); DNS checks use FQDNs; the explainer and exam guide
are corrected (etcd restore is etcdutl's job on 3.6+).

Cross-session leaks: the pre-session baseline now also repairs Q114 (the
kube-system NetworkPolicy and CoreDNS replicas), Q117/Q39 kubelet CA paths,
Q317, kubelet drop-in binary paths, a moved scheduler manifest, leftover
static pods and the default StorageClass. Q115 releases its fixed PVs from
claims left by earlier sessions.

Conflicts: `breaks` may be a list and `solo` keeps Q102's etcd restore out of
mixed sessions. Q118, Q5/Q103, Q26 vs the W1 taints, set A 203/204/208/212 and
set B 305/309 no longer sabotage each other. Q317 now disables only CSR
signing (the old break killed the whole controller-manager). Q110/Q311 no
longer compare against the whole cluster at submit time. Two delete/recreate
races in setups are fixed.

Harness: the weak list clears after two clean passes in a row, weak entries
show the latest attempt's failures, domain scores use the exam's own tasks,
history entries can be marked synthetic (entry 11 is), the web UI lists
every sim set, `./exam grade` refuses mid-exam (--force), preflight checks W1.

Set D (`./exam sim d`): 17 original killer.sh-style tasks, ids 501-517,
122 points - control-plane inventory, event forensics with crictl, a broken
kubelet unit, kubelet certificates, manual scheduling with the scheduler
stopped, Helm housekeeping with a stuck release, an enforced AND-vs-OR
NetworkPolicy, dnsPolicy and access-mode troubleshooting, and more.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
vumast3r 2026-09-22 21:59:35 -06:00
parent 7fae1f2c36
commit 55ab18f814
18 changed files with 1866 additions and 493 deletions

View File

@ -51,7 +51,7 @@ EOF
| **Troubleshoot Node (Kubelet)** | `systemctl status kubelet` <br> `journalctl -u kubelet -e --no-pager` | Check `/etc/kubernetes/kubelet.conf` & `/var/lib/kubelet/config.yaml` |
| **Control Plane Troubleshooting** | Inspect `/etc/kubernetes/manifests/` | Check `kube-apiserver.yaml`, `kube-scheduler.yaml`, `kube-controller-manager.yaml` |
| **Etcd Backup** | `etcdctl snapshot save <file>` | `--endpoints=https://127.0.0.1:2379 --cacert=... --cert=... --key=...` |
| **Etcd Restore** | `etcdctl snapshot restore <file>` | `--data-dir=/var/lib/etcd-backup` (update `etcd.yaml` volume hostPath) |
| **Etcd Restore** | `etcdutl snapshot restore <file>` | `--data-dir=/var/lib/etcd-backup` (update `etcd.yaml` volume hostPath). `etcdctl snapshot restore` is gone from etcd 3.6+ |
| **Upgrade Node (Control Plane / Worker)** | `apt-get update && apt-get install -y --allow-change-held-packages kubeadm=1.31.x-1.1` | `kubeadm upgrade plan` -> `kubeadm upgrade apply` -> upgrade `kubelet`/`kubectl` -> `systemctl restart kubelet` |
---
@ -98,12 +98,16 @@ ETCDCTL_API=3 etcdctl \
### Restore:
```bash
ETCDCTL_API=3 etcdctl \
--data-dir=/var/lib/etcd-from-backup \
snapshot restore /opt/snapshot-pre-boot.db
etcdutl snapshot restore /opt/snapshot-pre-boot.db \
--data-dir=/var/lib/etcd-from-backup
```
Then edit `/etc/kubernetes/manifests/etcd.yaml` to update the hostPath volume for `etcd-data` to `/var/lib/etcd-from-backup`.
Restore is an offline file operation, so it needs no endpoints or certificates - and it lives in
`etcdutl`, not `etcdctl`: `etcdctl snapshot restore` was deprecated in etcd 3.5 and is gone from
3.6+ (this cluster runs 3.7, where it fails with `unknown flag: --data-dir`). Check first with
`etcdctl version`; on a 3.5 node the old `ETCDCTL_API=3 etcdctl snapshot restore` still works.
---
## 5. Official Kubernetes Documentation URLs Allowed in CKA

View File

@ -830,7 +830,7 @@ The exam-day skill this table is really pointing at, regardless of which platfor
- labelSelector: {matchLabels: {app: %{dep}}}
topologyKey: kubernetes.io/hostname
tolerations:
- {key: %{key}, operator: {Equal, value: "%{val}", effect: NoSchedule}}
- {key: %{key}, operator: Equal, value: "%{val}", effect: NoSchedule}
containers:
- name: app
image: nginx:1.27
@ -838,6 +838,8 @@ The exam-day skill this table is really pointing at, regardless of which platfor
```
```bash
k create priorityclass %{dep}-prio --value=50000
# the seeded quota (pods: 2, requests.cpu: 250m) cannot hold 3 x 100m - raise BOTH, keep the quota
k -n %{ns} patch quota %{ns}-quota -p '{"spec":{"hard":{"pods":"3","requests.cpu":"300m"}}}'
```
- **Gotchas**: with exactly 3 worker nodes and `replicas: 3` under required anti-affinity, there is zero slack — if the quota's `pods` cap is lower than 3, or CPU requests don't fit the quota's `requests.cpu` budget, pods stay `Pending` with no scheduling error beyond an event; check `kubectl -n %{ns} get quota` and size requests accordingly rather than raising replicas blindly.
@ -874,12 +876,12 @@ The exam-day skill this table is really pointing at, regardless of which platfor
- name: agent
image: busybox:1.36
restartPolicy: Always
command: [sh, -c, "while true; do date >> /var/log/agent.log; sleep 1; done"]
volumeMounts: [{name: logs, mountPath: /var/log}]
command: [sh, -c, "while true; do date >> /var/log/agent/agent.log; sleep 1; done"]
volumeMounts: [{name: logs, mountPath: /var/log/agent}]
containers:
- name: app
image: nginx:1.27
volumeMounts: [{name: logs, mountPath: /var/log}]
volumeMounts: [{name: logs, mountPath: /var/log/agent}] # NOT /var/log - see Gotchas
readinessProbe: {httpGet: {path: /, port: 80}, periodSeconds: 5}
livenessProbe: {tcpSocket: {port: 80}}
volumes: [{name: logs, emptyDir: {}}]
@ -889,7 +891,7 @@ The exam-day skill this table is really pointing at, regardless of which platfor
| tr ' ' '\n' | grep . | sort > {WS}/q109/containers.txt
k -n %{ns} get pod %{pod} -o jsonpath='{.status.podIP}' | tr -d '\n' > {WS}/q109/ip.txt
```
- **Gotchas**: `restartPolicy: Always` goes **inside the initContainer entry itself**, not at the pod level — that per-container field is exactly what flips a normal init container into a native sidecar. The report file must list container names sorted, one per line, no header — matching the check's own `sort` byte-for-byte.
- **Gotchas**: never mount an emptyDir over `/var/log` in an nginx container — it hides nginx's own `/var/log/nginx` and nginx dies with `could not open error log file`, crash-looping forever. A volume mount **replaces** the directory's contents; mount a subdirectory instead. `restartPolicy: Always` goes **inside the initContainer entry itself**, not at the pod level — that per-container field is exactly what flips a normal init container into a native sidecar. The report file must list container names sorted, one per line, no header — matching the check's own `sort` byte-for-byte.
---
@ -1093,7 +1095,8 @@ The exam-day skill this table is really pointing at, regardless of which platfor
externalName: %{ext}
```
```bash
k -n %{ns} exec %{sts}-0 -- nslookup %{sts}-0.%{sts} > {WS}/q43/pod0.txt
# full name: busybox's nslookup applies no search domains, so the short form returns NXDOMAIN
k -n %{ns} exec %{sts}-0 -- nslookup %{sts}-0.%{sts}.%{ns}.svc.cluster.local > {WS}/q43/pod0.txt
```
- **Gotchas**: the headless Service's name **must match** `spec.serviceName` on the StatefulSet — that field is what wires the StatefulSet's per-pod DNS into that specific headless Service; a mismatched name means the stable pod hostnames never resolve, even though both objects individually look fine. `type: ExternalName` Services never get a `ClusterIP` or endpoints at all — that's expected, not a misconfiguration.
@ -1225,7 +1228,8 @@ The exam-day skill this table is really pointing at, regardless of which platfor
# forward . 10.255.255.254 -> forward . /etc/resolv.conf
k -n kube-system rollout restart deploy coredns
k run dns-check --image=busybox:1.36 --rm -it --restart=Never -- nslookup kubernetes.default
# FULL names: busybox's nslookup applies no search domains - `kubernetes.default` fails even when DNS works
k run dns-check --image=busybox:1.36 --rm -it --restart=Never -- nslookup kubernetes.default.svc.cluster.local
```
- **Gotchas**: fixing the Corefile alone looks successful (`kubectl edit` returns cleanly) but does nothing while replicas are still at 0 and/or a NetworkPolicy blocks ingress to the pods — always verify the actual symptom (`nslookup` from a real pod) rather than trusting that each individual fix "should" have worked. `rollout restart` is necessary after editing the ConfigMap because CoreDNS doesn't hot-reload `Corefile` changes on its own.
@ -1392,9 +1396,11 @@ The exam-day skill this table is really pointing at, regardless of which platfor
# re-point legacy: swap claimName old-data -> new-data, then recreate the pod
k -n %{ns} delete pod legacy --now
k -n %{ns} apply -f legacy.yaml
# the Job's Completed pod still references old-data and pins it - delete that pod first
k -n %{ns} delete pod -l job-name=migrate
k -n %{ns} delete pvc old-data
```
- **Gotchas**: `cp -a /src/. /dst/` (trailing `/.`) copies the **contents** of `/src` into `/dst`, preserving permissions/timestamps — dropping the trailing dot instead creates a nested `/dst/src/` directory, which silently fails any check expecting files directly under the new mount. Repointing `legacy`'s claim requires deleting and recreating the pod (`volumes[].persistentVolumeClaim.claimName` isn't mutable in place) — but the task requires the *Deployment/PVC* naming to stay the same, not the pod's UID, so this is safe here.
- **Gotchas**: a pod that references a claim holds it in `pvc-protection` even after it has **Completed** — `delete pvc old-data` hangs in Terminating until the Job's pod is gone (`k describe pvc old-data` names it under "Used By"). Deleting the pod leaves the Job object and its `succeeded: 1` intact. `cp -a /src/. /dst/` (trailing `/.`) copies the **contents** of `/src` into `/dst`, preserving permissions/timestamps — dropping the trailing dot instead creates a nested `/dst/src/` directory, which silently fails any check expecting files directly under the new mount. Repointing `legacy`'s claim requires deleting and recreating the pod (`volumes[].persistentVolumeClaim.claimName` isn't mutable in place) — but the task requires the *Deployment/PVC* naming to stay the same, not the pod's UID, so this is safe here.
---
@ -1484,7 +1490,8 @@ The exam-day skill this table is really pointing at, regardless of which platfor
# forward . 10.255.255.1 -> forward . /etc/resolv.conf
k -n kube-system rollout restart deploy coredns
k run test-dns --image=busybox:1.36 --rm -it --restart=Never -- nslookup kubernetes.default
# FULL names: busybox's nslookup applies no search domains - `kubernetes.default` fails even when DNS works
k run test-dns --image=busybox:1.36 --rm -it --restart=Never -- nslookup kubernetes.default.svc.cluster.local
```
- **Gotchas**: `forward . /etc/resolv.conf` tells CoreDNS to use the **node's own** resolver config for anything outside the cluster zone — that's the correct, portable default; pointing `forward` at a hardcoded IP address is exactly the kind of change that breaks the moment that IP stops being reachable, which is the fault being simulated here.
@ -1579,7 +1586,9 @@ The exam-day skill this table is really pointing at, regardless of which platfor
k -n %{ns} describe deploy %{dep}
k -n %{ns} describe rs -l app=%{dep} # "exceeded quota: pods, requested: pods=1"
k -n %{ns} patch quota tight -p '{"spec":{"hard":{"pods":"3"}}}'
# 4, not 3: the rolling update needs a surge pod, and with maxUnavailable 0 (25% of 3 rounds
# down) the controller will not remove an old pod first - pods=3 deadlocks the rollout
k -n %{ns} patch quota tight -p '{"spec":{"hard":{"pods":"4"}}}'
k -n %{ns} set image deploy %{dep} web=nginx:1.27
k -n %{ns} patch secret %{dep}-env -p '{"stringData":{"DATABASE_URL":"postgres://db"}}'

View File

@ -4,8 +4,9 @@ Entry point is `./exam` (renamed from `cka-sim` so it can't collide with the dir
Hands-on exam tasks graded **per check** against a real cluster, an exam clock, a browser
front end modelled on the real PSI exam, attempt history, and drills that rotate through the
domains before you sit a full exam. Tracks: **CKA** (65-task bank: 43 normal + 22 hard),
CKAD and CKS (scaffolded — same format, add questions when you get there).
domains before you sit a full exam. Tracks: **CKA** (136 tasks: a random pool of 43 normal +
24 hard, plus four fixed simulator sets A–D of 17–18 tasks each), CKAD and CKS (scaffolded — same
format, add questions when you get there).
## How a session is built
@ -65,8 +66,10 @@ The same state is available from the terminal if you prefer:
```
./exam drill # next least-drilled domain, 5 tasks (add --hard)
./exam drill weak | drill 2 17 20 | drill TSHOOT NET
./exam drill weak | drill 2 17 20 | drill TSHOOT NET | drill setd
./exam start [--hard] # full exam (gated until all domains drilled; --force)
./exam sim a|b|c|d # a fixed 17/18-task simulator set, exam rules, 120 min
./exam study [DOMAIN|weak|N..] # no clock, solutions visible, nothing recorded
./exam list | show N | timer | status # during
./exam end # submit
./exam weak | history # afterwards
@ -82,10 +85,12 @@ exam. Every submit appends to `history-<track>.jsonl` and regenerates `weak-spot
latest domain scores, a "drill next" section listing the *failing checks* per task, and a
✓ / n-of-m matrix across all attempts.
A task counts as weak if it failed most recently, has failed twice ever, or was never
attempted. Drills seed only those tasks, give 6 minutes per task, and — unlike exam mode —
let you check a task while you work. A task stops being weak only when you pass it again
under the clock.
A task counts as weak if it failed on its latest attempt, or has failed at least twice and has
not yet been passed twice in a row since, or was never attempted (fixed-set tasks excepted — note
that this makes an early `drill weak` very large; name tasks explicitly for a short drill). Drills
give 6 minutes per task and — unlike exam mode — let you check a task while you work. History
entries marked `"synthetic": true` (scripted verification runs, not a person) are kept in the file
but left out of every statistic.
## Using your own cluster instead of kind
@ -111,7 +116,7 @@ fails loudly if it cannot. `./etcd-tools.sh` does the same by hand.
## CKA bank
| Domain | Normal (43) | Hard (22) |
| Domain | Normal (43) | Hard (24) |
|---|---|---|
| Architecture | RBAC, etcd snapshot, Kustomize, CRD, static pod, certs/CSR, Helm, aggregated ClusterRole, `kubectl explain` on a CRD, kubeadm join token, component versions | exact RBAC with an over-grant to remove, **etcd restore**, static pod on a relocated path, user → self-contained kubeconfig, Kustomize generators + patches, CRD validation/defaults, renew all control-plane certs |
| Workloads | rolling update, sidecar, node affinity + resources, ConfigMap/Secret, metrics-server repair + HPA, PriorityClass, taints, DaemonSet, CronJob, init container, immutable ConfigMap rotation, parallel Job, quota + LimitRange | anti-affinity + toleration + priority under a quota, rollback to a specific revision then pause, native sidecar + probes + exact report, cluster audit to exact files, work in the context's namespace |
@ -120,12 +125,24 @@ fails loudly if it cannot. `./etcd-tools.sh` does the same by hand.
| Troubleshooting | node NotReady, broken deployment + service, CrashLoopBackOff, scheduler down, CoreDNS misconfig, controller-manager down, kubelet config drift | worker with three faults (+ uncordon check), two control-plane components down, app down for four reasons fixed in place (UID check), cluster facts exactly |
Disruptive tasks (anything that breaks a node or control-plane component) are seeded last and
can block other tasks until fixed — read everything first, as in the real exam. kindnet doesn't
enforce NetworkPolicy and no ingress/gateway controller runs, so those are graded on the objects.
can block other tasks until fixed — read everything first, as in the real exam. Tasks that damage
or depend on the same shared state declare it in `breaks` and never share a session, and the etcd
restore (Q102) is `solo`: restoring rolls the whole cluster back, so it only runs on its own.
Before every session a baseline pass repairs anything a previous, unfinished session left broken.
The kindnet CNI here **enforces** NetworkPolicy (it embeds a policy engine), so a wrong policy
really drops traffic; no ingress or Gateway controller runs, so those are graded on the objects.
**Simulator sets** (`./exam sim a|b|c|d`, never drawn into random sessions): A and B are
killer.sh-grade investigate-and-write-to-file sets, C is modelled on an 18-question commercial
mock sitting, and D is killer.sh-style — control-plane inventory, event forensics with crictl,
a broken kubelet unit, kubelet certificates, manual scheduling with the scheduler stopped, Helm
housekeeping with a stuck release, and an enforced AND-vs-OR NetworkPolicy. All are original
scenarios, not copies of any commercial simulator.
## Adding tasks or a new track
Banks live in `tracks/cka/{questions,bank_extra,bank_hard}.py`; a task is:
Banks live in `tracks/cka/{questions,bank_extra,bank_hard,bank_sim_*}.py`; a task is:
```python
{"id": 44, "domain": "NET", "difficulty": "normal", # or "hard"
@ -133,7 +150,9 @@ Banks live in `tracks/cka/{questions,bank_extra,bank_hard}.py`; a task is:
"title": "...", "task": "...markdown; %{ns} and %{port} are substituted; `code` copies on click...",
"setup": "bash that seeds the scenario (may use %{ns})",
"checks": [C("what this check proves", 2, "bash; exit 0 = pass"), ...],
"disruptive": False}
"disruptive": False, # True: seeded last, after everything else is scheduled
"breaks": ["w1-taint"], # shared state it damages or needs intact; never paired with a task sharing any
"solo": False} # True: only ever drilled on its own
```
Scripts get `$K` (pinned to the grading context), `$WS`, `$CP/$W1/$W2/$W3`, `nx NODE 'cmd'`,

204
exam
View File

@ -212,8 +212,8 @@ def q_by_id(n: int) -> dict[str, Any]:
for q in QUESTIONS:
if q["id"] == n:
return q
# ids are not contiguous (1-43 base, 101-124 hard, 201-217 / 301-317 sim sets), so print the
# real ranges rather than "1-<count>", which named ids that do not exist.
# ids are not contiguous (1-43 base, 101-124 hard, 201-217 / 301-317 / 401-418 / 501-517 sim
# sets), so print the real ranges rather than "1-<count>", which named ids that do not exist.
ids = sorted(q["id"] for q in QUESTIONS)
ranges, start, prev = [], ids[0], ids[0]
for i in ids[1:] + [None]:
@ -286,11 +286,15 @@ def score(st, results=None):
return earned, total, (round(100 * earned / total, 1) if total else 0)
# ----------------------------------------------------------- history ----
def load_history():
def load_history(include_synthetic=False):
"""Attempts from history-<track>.jsonl. Entries marked "synthetic" (a scripted verification
run, not a person sitting the tasks) stay in the file for the record but are left out of every
statistic, weak list and "latest exam" unless explicitly asked for."""
if not os.path.exists(HISTORY_FILE):
return []
with open(HISTORY_FILE) as f:
return [json.loads(l) for l in f if l.strip()]
entries = [json.loads(l) for l in f if l.strip()]
return entries if include_synthetic else [e for e in entries if not e.get("synthetic")]
def record_attempt(st):
earned, total, pct = score(st)
@ -313,6 +317,7 @@ class AggEntry:
self.last_fail_at: str = ""
self.failed: list[str] = []
self.last_earned: str = ""
self.streak: int = 0 # consecutive full passes at the end of the history
def __getitem__(self, item: str) -> Any:
return getattr(self, item)
@ -328,6 +333,7 @@ class AggEntry:
"last_fail_at": self.last_fail_at,
"failed": self.failed,
"last_earned": self.last_earned,
"streak": self.streak,
}
@ -341,6 +347,7 @@ def stats() -> dict[int, AggEntry]:
a.attempts += 1
a.last = r.get("pass")
a.last_earned = f"{r.get('earned', 0)}/{r.get('total', 0)}"
a.streak = a.streak + 1 if r.get("pass") else 0
if not r.get("pass"):
a.fails += 1
a.last_fail_at = str(e.get("at", ""))
@ -366,27 +373,44 @@ def drilled_domains():
out[q["domain"]] += 1
return out
def breaks_of(q):
"""A task's "breaks" as a set. It is a string or a list: Q118 damages the scheduler AND the
controller-manager, and declaring only one let it share a session with Q38/Q317."""
b = q.get("breaks")
return set() if not b else ({b} if isinstance(b, str) else set(b))
def no_conflicts(cands, n, taken=()):
"""Take up to n tasks, never two that break the same thing (or clash with tasks already taken)."""
used = {q.get("breaks") for q in taken if q.get("breaks")}
"""Take up to n tasks, never two that break the same thing (or clash with tasks already taken).
A "solo" task (Q102's etcd restore rolls the WHOLE cluster back to its snapshot, wiping every
other task's seed and answers) is only ever taken on its own."""
used = set().union(*(breaks_of(q) for q in taken)) if taken else set()
out = []
for q in cands:
b = q.get("breaks")
if b and b in used:
if q.get("solo") and (taken or out):
continue
if any(x.get("solo") for x in list(taken) + out):
break
b = breaks_of(q)
if b & used:
continue
out.append(q)
if b:
used.add(b)
used |= b
if len(out) == n:
break
return out
def split_solo(qs):
"""Pull solo tasks out of a multi-task selection; they have to be drilled alone."""
if len(qs) <= 1:
return list(qs), []
return [q for q in qs if not q.get("solo")], [q for q in qs if q.get("solo")]
def pick(domain, n, hard, rng, exclude=(), taken=(), unseen_only=False):
"""n tasks from a domain: unseen first, then least recently seen; hard sessions prefer hard tasks."""
seen = last_seen()
def build(allow_hard):
pool = [q for q in POOL if q["domain"] == domain and q["id"] not in exclude
pool = [q for q in POOL if q["domain"] == domain and q["id"] not in exclude and not q.get("solo")
and (allow_hard or q["difficulty"] == "normal") and not (unseen_only and q["id"] in seen)]
rng.shuffle(pool)
pool.sort(key=lambda q: (0 if (hard and q["difficulty"] == "hard") else 1, seen.get(q["id"], -1)))
@ -425,7 +449,7 @@ def campaign_batch(hard, rng):
"""'drill all': DRILL_SIZE tasks you have never attempted, round-robin across domains,
hard tasks included when --hard. When everything has been seen once, falls back to weak tasks."""
seen = last_seen()
pool = [q for q in POOL if q["id"] not in seen and (hard or q["difficulty"] == "normal")]
pool = [q for q in POOL if q["id"] not in seen and not q.get("solo") and (hard or q["difficulty"] == "normal")]
if not pool:
return None, []
qs, domains = [], list(DOMAINS)
@ -475,23 +499,46 @@ def cmd_progress(args):
print(dim(f"\n{len(QUESTIONS) - len(left)}/{len(QUESTIONS)} tasks attempted at least once." + (f" Never seen: Q{', Q'.join(map(str, left))}" if left and len(left) <= 25 else "")))
print(dim("passed = latest attempt was a full pass. ./exam drill all [--hard] works through what's left."))
def is_weak(q, a):
"""Weak = failed on the latest attempt, or failed at least twice and not yet passed twice in a
row since - a flaky task has to prove itself - or never attempted (outside the fixed sets).
The old rule ("failed twice, ever") kept a task weak forever: 13 of 29 listed were passing."""
if a.attempts == 0:
return not q.get("set")
return a.last is False or (a.fails >= 2 and a.streak < 2)
def weak_ids(agg=None):
agg = agg or stats()
return [q["id"] for q in QUESTIONS
if (agg[q["id"]]["last"] is False or agg[q["id"]]["fails"] >= 2)
or (agg[q["id"]]["attempts"] == 0 and not q.get("set"))]
return [q["id"] for q in QUESTIONS if is_weak(q, agg[q["id"]])]
def weak_lines(a):
"""What to show under a weak task: the failing checks if it failed last time, otherwise how
close it is to dropping off the list - not the ✗ list of some older, since-fixed attempt."""
if a.last is False:
return [f"✗ {n}" for n in (a.failed or [])]
return [f"passed last time ({a.last_earned}) - one more clean pass takes it off this list"]
def exam_domain_scores(e):
"""domain -> (earned, total) over the tasks that were actually IN that attempt. Summing the
weight of every task in the bank instead made a 100% run read "ARCH 38/266"."""
out = {}
for k, r in e.get("results", {}).items():
q = QBYID.get(int(k))
if q:
de, dt = out.get(q["domain"], (0, 0))
out[q["domain"]] = (de + r.get("earned", 0), dt + r.get("total", 0))
return out
def write_weak_spots():
agg = stats(); hist = load_history()
L = [f"# {TRACK_ID.upper()} weak spots", "", f"_Generated by exam from {len(hist)} attempt(s); regenerated on every `end`._", ""]
exams = [e for e in hist if e["mode"] == "exam"]
if exams:
e = exams[-1]
e = exams[-1]; ds = exam_domain_scores(e)
L += [f"## Domain scores (latest full exam, {e['at']}, {e['score_pct']}%)", ""]
for d, name in DOMAINS.items():
dq = [q for q in QUESTIONS if q["domain"] == d]
dt = sum(q["weight"] for q in dq); de = sum(e["results"].get(str(q["id"]), {}).get("earned", 0) for q in dq)
L.append(f"- **{d}** {de}/{dt} — {name}")
if d in ds:
L.append(f"- **{d}** {ds[d][0]}/{ds[d][1]} — {name}")
L.append("")
L += ["## Drill next (`./exam drill weak`)", ""]
weak = [q for q in QUESTIONS if q["id"] in weak_ids(agg) and agg[q["id"]]["attempts"] > 0]
@ -499,9 +546,8 @@ def write_weak_spots():
L.append("Nothing failing right now.")
for q in weak:
a = agg[q["id"]]
L.append(f"### Q{q['id']:02d} · {q['domain']} · {q['title']} ({a['fails']}/{a['attempts']} failed, last {a['last_earned']} pts, {a['last_fail_at']})")
failed_list = list(a["failed"]) if isinstance(a["failed"], (list, tuple)) else []
L += [f"- ✗ {n}" for n in failed_list]
L.append(f"### Q{q['id']:02d} · {q['domain']} · {q['title']} ({a['fails']}/{a['attempts']} failed, last {a['last_earned']} pts, last failed {a['last_fail_at']})")
L += [f"- {n}" for n in weak_lines(a)]
L.append("")
L += ["## Attempt matrix", "", "| Q | Domain | Title | " + " | ".join(f"{e['mode'][0].upper()}{i + 1}" for i, e in enumerate(hist)) + " |",
"|---|---|---|" + "---|" * len(hist)]
@ -517,13 +563,13 @@ def write_weak_spots():
# ------------------------------------------------------------ session ----
def preflight():
out = subprocess.run(["kubectl", "get", "nodes", "-o", "jsonpath={.items[*].metadata.name}"],
out = subprocess.run(["kubectl", "--context", CONTEXT, "get", "nodes", "-o", "jsonpath={.items[*].metadata.name}"],
capture_output=True, text=True).stdout.split()
missing = [v for v in NODES.values() if v not in out]
if missing:
die(f"Nodes not found in cluster: {missing}\n cluster has: {out}\n"
" Set CKA_CP, CKA_W1, CKA_W2, CKA_W3 to your node names (needs 1 control-plane + 3 workers).")
rc, _ = run('nx "$CP" "true" </dev/null && nx "$W2" "true" </dev/null && nx "$W3" "true" </dev/null', timeout=60)
rc, _ = run('nx "$CP" "true" </dev/null && nx "$W1" "true" </dev/null && nx "$W2" "true" </dev/null && nx "$W3" "true" </dev/null', timeout=60)
if rc != 0:
die(f"Cannot run commands on nodes with CKA_NODE_ACCESS={NODE_ACCESS}.\n"
" podman/docker -> node names must be kind container names (kind get nodes --name cka-sim)\n"
@ -598,17 +644,26 @@ if nx "$CP" 'grep -q "^ *- kube-apiserverr$" /etc/kubernetes/manifests/kube-apis
for _ in $(seq 1 30); do kubectl --context "${CKA_CONTEXT:-kind-cka-sim}" get --raw=/readyz >/dev/null 2>&1 && break; sleep 2; done
fi
# scheduler: undo the "kube-scheduler" -> "kube-schedulerr" command typo (Q20/Q118)
# scheduler: undo the "kube-scheduler" -> "kube-schedulerr" command typo (Q20/Q118), and put the
# manifest back if a task had the candidate stop the scheduler by moving it out (Q512) and they
# never moved it back - that setup keeps a pristine copy outside the static pod directory.
nx "$CP" '
f=/etc/kubernetes/manifests/kube-scheduler.yaml
[ -f $f ] || { [ -f /etc/kubernetes/.cka-sim/kube-scheduler.yaml ] && cp /etc/kubernetes/.cka-sim/kube-scheduler.yaml $f; }
sed -i "s|^\( *\)- kube-schedulerr$|\1- kube-scheduler|" $f 2>/dev/null
' </dev/null
# controller-manager: undo Q38's injected flag and Q118's broken image tag
# controller-manager: undo Q38's injected flag, Q118's broken image tag and Q317's sabotage - both
# its current form (the CSR signing controller switched off in --controllers) and the older one
# (--cluster-signing-cert-file pointed at a missing file). That older form crash-looped the WHOLE
# controller-manager, because a controller that fails to start is fatal there, so every later
# session silently lost deployments, jobs, endpoints and PV phases, not just CSR signing.
nx "$CP" '
f=/etc/kubernetes/manifests/kube-controller-manager.yaml
sed -i "/--leader-elect-lease-duration=1s/d" $f 2>/dev/null
sed -i "s|\(image: registry.k8s.io/kube-controller-manager:v[0-9.]*\)-broken|\1|" $f 2>/dev/null
sed -i "s|--cluster-signing-cert-file=/etc/kubernetes/pki/ca-signing.crt|--cluster-signing-cert-file=/etc/kubernetes/pki/ca.crt|" $f 2>/dev/null
sed -i "s|,-certificatesigningrequest-signing-controller||; s|,-csrsigning||" $f 2>/dev/null
' </dev/null
# coredns: restore the default upstream forward (Q37/Q114 point it somewhere bogus)
@ -618,25 +673,61 @@ if [ -n "$cf" ] && ! printf '%s' "$cf" | grep -Eq '^[[:space:]]*forward \. /etc/
$K -n kube-system create cm coredns --from-file=Corefile=/tmp/Corefile.restore --dry-run=client -o yaml | $K apply -f - >/dev/null 2>&1
$K -n kube-system rollout restart deploy coredns >/dev/null 2>&1
fi
# ...and Q114's other two faults. kindnet ENFORCES NetworkPolicy, so a leftover lockdown policy on
# the CoreDNS pods kills name resolution for the whole cluster in every later session, and so does
# a Deployment left at 0 replicas.
$K -n kube-system delete netpol maintenance-lockdown --ignore-not-found >/dev/null 2>&1
r=$($K -n kube-system get deploy coredns -o jsonpath='{.spec.replicas}' 2>/dev/null)
if [ -n "$r" ] && [ "$r" -lt 2 ]; then $K -n kube-system scale deploy coredns --replicas=2 >/dev/null 2>&1; fi
# w1/w2/w3 kubelet: undo Q17/Q39-style tampering, then make sure it's enabled and running.
# staticPodPath is normalised unconditionally: Q103 relocates it to /etc/kubernetes/static, and
# the config.yaml.orig copy below only helps if something made that backup first - nothing does,
# so a relocated path used to survive every later session and silently break Q05 (whose check
# looks in /etc/kubernetes/manifests). Only restart the kubelet if we actually changed the file.
# clientCAFile gets the same treatment: Q39 points it at ca-bundle.crt and Q117 at ca-wrong.crt, and
# only W1 ever gets a config.yaml.orig (Q39 makes it), so an unfinished Q117 left W2's kubelet
# crash-looping in every later session - enable/start below cannot help a kubelet that exits.
# The systemd drop-in is checked too: Q507 points ExecStart at a kubelet binary that does not exist.
for n in "$W1" "$W2" "$W3"; do
nx "$n" '
[ -f /var/lib/kubelet/config.yaml.orig ] && cp -f /var/lib/kubelet/config.yaml.orig /var/lib/kubelet/config.yaml
if grep -q "^staticPodPath:" /var/lib/kubelet/config.yaml 2>/dev/null \
&& ! grep -q "^staticPodPath: /etc/kubernetes/manifests$" /var/lib/kubelet/config.yaml; then
sed -i "s|^staticPodPath: .*|staticPodPath: /etc/kubernetes/manifests|" /var/lib/kubelet/config.yaml
systemctl restart kubelet >/dev/null 2>&1
c=/var/lib/kubelet/config.yaml; d=/etc/systemd/system/kubelet.service.d/10-kubeadm.conf
[ -f $c.orig ] && cp -f $c.orig $c
restart=0
if grep -q "^staticPodPath:" $c 2>/dev/null && ! grep -q "^staticPodPath: /etc/kubernetes/manifests$" $c; then
sed -i "s|^staticPodPath: .*|staticPodPath: /etc/kubernetes/manifests|" $c; restart=1
fi
ca=$(awk "/clientCAFile:/{print \$2}" $c 2>/dev/null)
if [ -n "$ca" ] && [ ! -f "$ca" ]; then
sed -i "s|clientCAFile: .*|clientCAFile: /etc/kubernetes/pki/ca.crt|" $c; restart=1
fi
bin=$(sed -n "s|^ExecStart=\([^ ]*kubelet\) .*|\1|p" $d 2>/dev/null | tail -1)
if [ -n "$bin" ] && [ ! -x "$bin" ]; then
sed -i "s|^ExecStart=$bin |ExecStart=/usr/bin/kubelet |" $d; systemctl daemon-reload; restart=1
fi
[ $restart = 1 ] && systemctl restart kubelet >/dev/null 2>&1
systemctl enable kubelet >/dev/null 2>&1
systemctl start kubelet >/dev/null 2>&1
' </dev/null
done
# static pods the bank drops on nodes: Q05's static-web and Q314's edge-cache on W3 (edge-cache
# sat in ContainerCreating for days, its hostPath deliberately missing), Q103's relocated copies,
# Q509's ops-beacon on the control plane. Each setup recreates what its own session needs.
nx "$W3" 'rm -f /etc/kubernetes/manifests/static-web*.yaml /etc/kubernetes/manifests/edge-cache.yaml /etc/kubernetes/static/*.yaml' </dev/null
nx "$CP" 'rm -f /etc/kubernetes/manifests/ops-beacon*.yaml' </dev/null
# default StorageClass: Q16 makes fast-local the default and strips `standard`. Leave exactly one
# default - `standard` - so "the default class" means the same thing in every session.
if $K get sc standard >/dev/null 2>&1; then
for sc in $($K get sc -o jsonpath='{.items[*].metadata.name}'); do
[ "$sc" = standard ] && continue
[ "$($K get sc "$sc" -o jsonpath='{.metadata.annotations.storageclass\.kubernetes\.io/is-default-class}')" = true ] \
&& $K annotate sc "$sc" storageclass.kubernetes.io/is-default-class- >/dev/null 2>&1
done
$K annotate sc standard storageclass.kubernetes.io/is-default-class=true --overwrite >/dev/null 2>&1
fi
# w1-taint: clear every randomized demo taint key the bank can apply. Q25 draws from
# dedicated/team/workload, Q107 from pool/lane, and Q307 pins maintenance=inprogress - miss one
# and that node silently stops accepting pods for every later session.
@ -794,7 +885,7 @@ def cmd_drill(args):
label, qs = campaign_batch(hard, random.Random())
if not qs:
print(bold("Every task in the bank has been attempted." ) + " Switching to weak tasks.")
qs = [q for q in QUESTIONS if q["id"] in weak_ids()]
qs, _ = split_solo([q for q in QUESTIONS if q["id"] in weak_ids()])
qs = no_conflicts(qs, DRILL_SIZE)
if not qs:
die("Nothing weak either - you have passed everything on its latest attempt. Time for ./exam start --hard.")
@ -816,7 +907,10 @@ def cmd_drill(args):
else:
ids.append(q_by_id(int(a))["id"])
ids = sorted(set(ids))
qs = [q for q in QUESTIONS if q["id"] in ids]
qs, solo = split_solo([q for q in QUESTIONS if q["id"] in ids])
if solo:
print(yellow(f"Dropped Q{', Q'.join(str(q['id']) for q in solo)}: restoring it rolls the whole cluster back and would "
f"wipe the other tasks. Drill it on its own: ./exam drill {solo[0]['id']}"))
kept = no_conflicts(qs, len(qs))
if len(kept) < len(qs):
dropped = [q["id"] for q in qs if q not in kept]
@ -840,10 +934,10 @@ def cmd_study(args):
qs = pick(domain, n, hard, rng)
print(bold(f"Study: {DOMAINS[domain]}"))
elif rest and rest[0] == "weak":
qs = no_conflicts([q for q in QUESTIONS if q["id"] in weak_ids()], n)
qs = no_conflicts(split_solo([q for q in QUESTIONS if q["id"] in weak_ids()])[0], n)
print(bold("Study: tasks you have been failing"))
elif rest:
qs = no_conflicts([q_by_id(int(a)) for a in rest], len(rest))
qs = no_conflicts(split_solo([q_by_id(int(a)) for a in rest])[0], len(rest))
print(bold("Study: " + ", ".join(f"Q{q['id']}" for q in qs)))
else:
domain, qs = next_drill(hard, rng)
@ -851,7 +945,7 @@ def cmd_study(args):
begin_session(qs, "drill", EXAM_SECONDS, force, hard=hard, untimed_session=True)
def cmd_sim(args):
"""Fixed 17-task simulator set, full exam rules and clock."""
"""A fixed simulator set, full exam rules and clock."""
force = "--force" in args
names = [a.lower() for a in args if not a.startswith("--")]
if not names or names[0] not in SETS:
@ -895,8 +989,13 @@ def cmd_show(args):
def cmd_grade(args):
st = require_started()
force = "--force" in args; args = [a for a in args if a != "--force"]
if not args:
die("usage: ./exam grade N | all")
# The web UI already refused this; the CLI handed out per-check feedback mid-exam, which a sim
# meant to rehearse the real thing (no feedback until you submit) must not do by default.
if st.get("mode") == "exam" and not st.get("ended_at") and not force:
die("No feedback during an exam - the real one gives none. Submit with ./exam end (--force to peek anyway).")
ids = [q["id"] for q in session_questions(st)] if args[0] == "all" else [int(a) for a in args]
for n in ids:
q = q_by_id(n); r = grade_one(q)
@ -980,7 +1079,7 @@ def cmd_end(args):
print_score(st)
if entry is None:
print(dim("Study session - not added to history or weak-spots.")); return
print(dim(f"Time used: {fmt(st['ended_at'] - st['started_at'])}" + (" (over time)" if st.get("over_time") else "")))
print(dim(f"Time used: {fmt(st['ended_at'] - st['started_at'] - paused_total(st))}" + (" (over time)" if st.get("over_time") else "")))
failed = sorted(int(k) for k, v in entry["results"].items() if not v["pass"])
print(dim(f"Attempt #{len(load_history())} logged to {os.path.basename(HISTORY_FILE)}; {os.path.basename(WEAK_FILE)} updated."))
if failed:
@ -1012,31 +1111,34 @@ def cmd_weak(args):
print(green(" none - everything attempted has passed on its latest try"))
for i in ids:
q = q_by_id(i); a = agg[i]
print(f"\n{red('Q%02d' % i)} {q['domain']:<7} {q['title']} " + dim(f"{a['fails']}/{a['attempts']} failed, last {a['last_earned']} pts, {a['last_fail_at']}"))
for n in a["failed"]:
print(dim(" ✗ " + n))
print(f"\n{red('Q%02d' % i)} {q['domain']:<7} {q['title']} " + dim(f"{a['fails']}/{a['attempts']} failed, last {a['last_earned']} pts, last failed {a['last_fail_at']}"))
for n in weak_lines(a):
print(dim(" " + n))
never = [q["id"] for q in QUESTIONS if agg[q["id"]]["attempts"] == 0]
if never:
print(dim(f"\nNever attempted: Q{', Q'.join(map(str, never))}"))
print(dim(f"\nFull log: {WEAK_FILE} Drill them: ./exam drill weak"))
def cmd_history(args):
hist = load_history()
hist = load_history(include_synthetic=True)
if not hist:
die("No history yet.")
print(bold(f"{'#':>3} {'when':<17}{'mode':<7}{'score':>7} {'time':<9} tasks"))
for i, e in enumerate(hist, 1):
col = green if e["score_pct"] >= PASS_PCT else red
tasks = f"all {len(QUESTIONS)}" if len(e["ids"]) == len(QUESTIONS) else "Q" + ",".join(map(str, e["ids"]))
print(f"{i:>3} {e['at']:<17}{(e['mode'] + ('*' if e.get('hard') else '')):<7}{col('%5.1f%%' % e['score_pct']):>16} {fmt(e['time_used']):<9} {tasks}")
exams = [e for e in hist if e["mode"] == "exam"]
tag = dim(" [scripted verification - not counted]") if e.get("synthetic") else ""
print(f"{i:>3} {e['at']:<17}{(e['mode'] + ('*' if e.get('hard') else '')):<7}{col('%5.1f%%' % e['score_pct']):>16} {fmt(e['time_used']):<9} {tasks}{tag}")
exams = [e for e in hist if e["mode"] == "exam" and not e.get("synthetic")]
print(dim("* = hard"))
if len(exams) >= 2:
print(bold("\nDomain trend (full exams, % of points):"))
print(bold("\nDomain trend (full exams, % of that exam's points in the domain):"))
for d in DOMAINS:
dq = [q for q in QUESTIONS if q["domain"] == d]; dt = sum(q["weight"] for q in dq)
pts = [round(100 * sum(e["results"].get(str(q["id"]), {}).get("earned", 0) for q in dq) / dt) for e in exams]
print(f" {d:<7} " + " → ".join(f"{p:>3}%" for p in pts))
cells = []
for e in exams:
de, dt = exam_domain_scores(e).get(d, (0, 0))
cells.append(f"{round(100 * de / dt):>3}%" if dt else " -")
print(f" {d:<7} " + " → ".join(cells))
def cmd_reset(args):
if not args:
@ -1114,12 +1216,12 @@ class Api(http.server.BaseHTTPRequestHandler):
elif body.get("campaign"):
_, qs = campaign_batch(hard, rng)
if not qs:
qs = no_conflicts([q for q in QUESTIONS if q["id"] in weak_ids()], DRILL_SIZE)
qs = no_conflicts(split_solo([q for q in QUESTIONS if q["id"] in weak_ids()])[0], DRILL_SIZE)
elif body.get("next"):
_, qs = next_drill(hard, rng)
else:
ids = body.get("ids") or weak_ids()
qs = no_conflicts([q for q in QUESTIONS if q["id"] in ids], len(ids))
qs = no_conflicts(split_solo([q for q in QUESTIONS if q["id"] in ids])[0], len(ids))
per = DRILL_SECONDS_PER_TASK_HARD if hard else DRILL_SECONDS_PER_TASK
begin_session(qs, "drill", per * len(qs), True, log=ulog, hard=hard,
untimed_session=bool(body.get("study")))
@ -1269,9 +1371,9 @@ def cmd_help(args):
exact-output tasks, compound failures, collateral checks and exam-style contexts.
list tasks in this session, with current + historical results
show N display task N
grade N|all grade one task (or all) per check - drills only, in the UI
grade N|all grade one task (or all) per check - drills only (--force to peek in an exam)
status / timer time remaining (timer = live countdown)
pause / resume stop the clock (recorded on the attempt - the real exam has no pause)
pause / resume stop the clock (not recorded anywhere - but the real exam has no pause)
end submit: grade per check, score, log attempt, update weak-spots
weak what you keep getting wrong, with the failing checks
history every attempt, scores, and per-domain trend

View File

@ -8,4 +8,4 @@
{"at": "2026-09-15T20:55", "mode": "drill", "hard": false, "ids": [13, 15, 16, 19, 25, 28, 29, 30, 33, 37, 39, 40, 42, 106, 112, 115, 120, 121], "score_pct": 41.7, "earned": 40, "total": 96, "time_used": 608, "results": {"13": {"pass": true, "earned": 5, "total": 5, "failed": []}, "19": {"pass": true, "earned": 6, "total": 6, "failed": []}, "33": {"pass": true, "earned": 5, "total": 5, "failed": []}, "30": {"pass": true, "earned": 4, "total": 4, "failed": []}, "37": {"pass": true, "earned": 6, "total": 6, "failed": []}, "39": {"pass": true, "earned": 7, "total": 7, "failed": []}, "40": {"pass": false, "earned": 0, "total": 5, "failed": ["join.sh contains a kubeadm join with token and CA hash", "the token in the file exists in the cluster with TTL ~6h"]}, "42": {"pass": true, "earned": 4, "total": 4, "failed": []}, "15": {"pass": false, "earned": 0, "total": 6, "failed": ["PV pv-data: 1Gi hostPath /mnt/data, RWO, Retain, class manual", "PVC pvc-data Bound to pv-data", "data-pod Running with claim mounted at /usr/share/data"]}, "16": {"pass": false, "earned": 0, "total": 5, "failed": ["StorageClass fast-local: local-path, WaitForFirstConsumer, Delete", "fast-local is default and standard is not", "fast-claim picked up fast-local and is Bound", "fast-pod Running with claim at /data"]}, "25": {"pass": false, "earned": 0, "total": 4, "failed": ["cka-sim-worker carries team=finance:NoSchedule", "deployment tolerates the taint", "both pods Running on cka-sim-worker"]}, "28": {"pass": false, "earned": 0, "total": 4, "failed": ["pod Running, init container named setup completed", "emptyDir 'content' shared by both containers", "nginx serves the init-written content"]}, "29": {"pass": false, "earned": 1, "total": 4, "failed": ["deployment now references a different ConfigMap", "all pods available and see mode=new"]}, "106": {"pass": false, "earned": 0, "total": 7, "failed": ["CRD exists, Namespaced, replicas has min/max and is required", "good created with environment prod", "replicas 0 is rejected and the message was saved", "defaulting works: defaulted.spec.environment == dev"]}, "112": {"pass": false, "earned": 0, "total": 7, "failed": ["Secret portal.lab.local-tls is kubernetes.io/tls with CN portal.lab.local", "Ingress web: class nginx, host, TLS secret, ssl-redirect annotation", "fp.txt matches the secret's certificate fingerprint"]}, "115": {"pass": false, "earned": 1, "total": 6, "failed": ["PVC uses a selector and is Bound to lv-nvme", "consumer Running on the nvme node with /data mounted"]}, "120": {"pass": false, "earned": 0, "total": 5, "failed": ["pod-cidr and service-cidr", "dns-ip and proxy-mode", "cka-sim-worker3 pod CIDR"]}, "121": {"pass": false, "earned": 1, "total": 6, "failed": ["Deployment ledger-21 in ac-echo (the context's namespace), 2 available, owner=ac", "Service ledger-21 port 9000 in the same namespace", "ns.txt has the namespace"]}}}
{"at": "2026-09-16T19:40", "mode": "drill", "hard": false, "ids": [15, 16, 25, 28, 29, 40], "score_pct": 50.0, "earned": 14, "total": 28, "time_used": 1258, "results": {"15": {"pass": true, "earned": 6, "total": 6, "failed": []}, "25": {"pass": true, "earned": 4, "total": 4, "failed": []}, "29": {"pass": true, "earned": 4, "total": 4, "failed": []}, "16": {"pass": false, "earned": 0, "total": 5, "failed": ["StorageClass fast-local: local-path, WaitForFirstConsumer, Delete", "fast-local is default and standard is not", "fast-claim picked up fast-local and is Bound", "fast-pod Running with claim at /data"]}, "28": {"pass": false, "earned": 0, "total": 4, "failed": ["pod Running, init container named setup completed", "emptyDir 'content' shared by both containers", "nginx serves the init-written content"]}, "40": {"pass": false, "earned": 0, "total": 5, "failed": ["join.sh contains a kubeadm join with token and CA hash", "the token in the file exists in the cluster with TTL ~24h"]}}}
{"at": "2026-09-19T12:27", "mode": "drill", "hard": false, "ids": [106, 217, 34, 202, 102, 301, 39, 13, 32, 204, 25, 2, 8, 112, 4, 109, 313, 9], "score_pct": 39.7, "earned": 48, "total": 121, "time_used": 4197, "results": {"8": {"pass": true, "earned": 4, "total": 4, "failed": []}, "34": {"pass": true, "earned": 4, "total": 4, "failed": []}, "39": {"pass": true, "earned": 7, "total": 7, "failed": []}, "13": {"pass": true, "earned": 5, "total": 5, "failed": []}, "25": {"pass": true, "earned": 4, "total": 4, "failed": []}, "301": {"pass": true, "earned": 9, "total": 9, "failed": []}, "4": {"pass": true, "earned": 4, "total": 4, "failed": []}, "9": {"pass": true, "earned": 4, "total": 4, "failed": []}, "106": {"pass": false, "earned": 0, "total": 7, "failed": ["CRD exists, Namespaced, replicas has min/max and is required", "good created with environment prod", "replicas 0 is rejected and the message was saved", "defaulting works: defaulted.spec.environment == dev"]}, "217": {"pass": false, "earned": 0, "total": 10, "failed": ["Pod tiger-probe Running with both labels", "container.txt holds a real container ID for that Pod", "container.txt records the runtimeType", "container.log holds the httpd startup output"]}, "202": {"pass": false, "earned": 3, "total": 8, "failed": ["watchAllNamespaces=true reached the pod env", "Ledger main-ledger has currency USD and retentionDays 30"]}, "102": {"pass": false, "earned": 2, "total": 10, "failed": ["ConfigMap restore-proof-67 exists again with its original data", "etcd uses a restored data directory (manifest changed)"]}, "32": {"pass": false, "earned": 0, "total": 4, "failed": ["ResourceQuota limits pods/6, requests.cpu 1, limits.memory 2Gi", "LimitRange defaults set", "worker pods Running with defaulted requests"]}, "204": {"pass": false, "earned": 0, "total": 8, "failed": ["evicted-first.txt lists exactly the BestEffort pods", "api-qos.txt says Burstable", "guaranteed.txt names orion-ledger"]}, "2": {"pass": false, "earned": 0, "total": 7, "failed": ["/opt/cka/etcd-snapshot.db exists on the control-plane node", "snapshot is a real etcd backup (> 100 KB)"]}, "112": {"pass": false, "earned": 0, "total": 7, "failed": ["Secret secure.lab.local-tls is kubernetes.io/tls with CN secure.lab.local", "Ingress web: class nginx, host, TLS secret, ssl-redirect annotation", "fp.txt matches the secret's certificate fingerprint"]}, "109": {"pass": false, "earned": 2, "total": 8, "failed": ["native sidecar: initContainer agent with restartPolicy Always", "probes on app as specified", "Running, shared log written by agent, grace period 5"]}, "313": {"pass": false, "earned": 0, "total": 11, "failed": ["Secret classic-tls is a TLS secret with CN classic.r700.io", "both host rules route to the right Services on /", "TLS covers both hosts with classic-tls", "defaultBackend points at web-classic:80, class is nginx"]}}}
{"at": "2026-09-20T17:32", "mode": "exam", "hard": true, "ids": [401, 402, 403, 404, 405, 406, 407, 408, 409, 410, 411, 412, 413, 414, 415, 416, 417, 418], "score_pct": 100.0, "earned": 163, "total": 163, "time_used": 2909, "results": {"402": {"pass": true, "earned": 9, "total": 9, "failed": []}, "404": {"pass": true, "earned": 9, "total": 9, "failed": []}, "405": {"pass": true, "earned": 11, "total": 11, "failed": []}, "401": {"pass": true, "earned": 8, "total": 8, "failed": []}, "403": {"pass": true, "earned": 9, "total": 9, "failed": []}, "406": {"pass": true, "earned": 7, "total": 7, "failed": []}, "407": {"pass": true, "earned": 10, "total": 10, "failed": []}, "408": {"pass": true, "earned": 9, "total": 9, "failed": []}, "409": {"pass": true, "earned": 9, "total": 9, "failed": []}, "410": {"pass": true, "earned": 8, "total": 8, "failed": []}, "411": {"pass": true, "earned": 7, "total": 7, "failed": []}, "412": {"pass": true, "earned": 11, "total": 11, "failed": []}, "413": {"pass": true, "earned": 10, "total": 10, "failed": []}, "414": {"pass": true, "earned": 9, "total": 9, "failed": []}, "415": {"pass": true, "earned": 9, "total": 9, "failed": []}, "416": {"pass": true, "earned": 10, "total": 10, "failed": []}, "417": {"pass": true, "earned": 10, "total": 10, "failed": []}, "418": {"pass": true, "earned": 8, "total": 8, "failed": []}}}
{"at": "2026-09-20T17:32", "mode": "exam", "hard": true, "ids": [401, 402, 403, 404, 405, 406, 407, 408, 409, 410, 411, 412, 413, 414, 415, 416, 417, 418], "score_pct": 100.0, "earned": 163, "total": 163, "time_used": 2909, "results": {"402": {"pass": true, "earned": 9, "total": 9, "failed": []}, "404": {"pass": true, "earned": 9, "total": 9, "failed": []}, "405": {"pass": true, "earned": 11, "total": 11, "failed": []}, "401": {"pass": true, "earned": 8, "total": 8, "failed": []}, "403": {"pass": true, "earned": 9, "total": 9, "failed": []}, "406": {"pass": true, "earned": 7, "total": 7, "failed": []}, "407": {"pass": true, "earned": 10, "total": 10, "failed": []}, "408": {"pass": true, "earned": 9, "total": 9, "failed": []}, "409": {"pass": true, "earned": 9, "total": 9, "failed": []}, "410": {"pass": true, "earned": 8, "total": 8, "failed": []}, "411": {"pass": true, "earned": 7, "total": 7, "failed": []}, "412": {"pass": true, "earned": 11, "total": 11, "failed": []}, "413": {"pass": true, "earned": 10, "total": 10, "failed": []}, "414": {"pass": true, "earned": 9, "total": 9, "failed": []}, "415": {"pass": true, "earned": 9, "total": 9, "failed": []}, "416": {"pass": true, "earned": 10, "total": 10, "failed": []}, "417": {"pass": true, "earned": 10, "total": 10, "failed": []}, "418": {"pass": true, "earned": 8, "total": 8, "failed": []}}, "synthetic": true, "note": "Scripted end-to-end verification of Set C (tasks solved programmatically), not a person sitting it"}

View File

@ -98,4 +98,4 @@ echo
echo "Cluster ready. Nodes:"
kubectl get nodes -o wide
echo
echo "Next: ./exam start (seeds all 20 tasks and starts the 120-minute clock)"
echo "Next: ./exam web (browser UI) or ./exam drill (5 tasks from your least-drilled domain)"

View File

@ -169,8 +169,9 @@ for t in $($K -n kube-system get secrets -o jsonpath='{.items[?(@.type=="bootstr
C("the token in the file exists in the cluster with TTL ~%{ttl}", 3, r"""
tok=$(grep -o -- '--token [a-z0-9]\{6\}\.[a-z0-9]\{16\}' "$WS/q40/join.sh" 2>/dev/null | awk '{print $2}'); [ -n "$tok" ] || exit 1
id=${tok%%.*}; exp=$(j -n kube-system secret bootstrap-token-$id '{.data.expiration}' | base64 -d 2>/dev/null); [ -n "$exp" ] || exit 1
secs=$(( $(date -d "$exp" +%s) - $(date +%s) )); want=$(( $(echo %{ttl} | tr -d h) * 3600 ))
[ $secs -gt $((want - 900)) ] && [ $secs -le $((want + 60)) ]"""),
ct=$(j -n kube-system secret bootstrap-token-$id '{.metadata.creationTimestamp}'); [ -n "$ct" ] || exit 1
secs=$(( $(date -d "$exp" +%s) - $(date -d "$ct" +%s) )); want=$(( $(echo %{ttl} | tr -d h) * 3600 ))
[ $secs -ge $((want - 120)) ] && [ $secs -le $((want + 120)) ] && [ "$(date -d "$exp" +%s)" -gt "$(date +%s)" ]"""),
],
},
{
@ -197,9 +198,9 @@ kub=$(j node %{node} '{.status.nodeInfo.kubeletVersion}')
grep -qx "apiserver=$api" "$f" && grep -qx "kubelet=$kub" "$f" """),
C("etcd and runtime versions correct", 2, r"""
f="$WS/q41/versions.txt"
etcd=$(j -n kube-system pod -l component=etcd '{.items[0].spec.containers[0].image}' | sed 's/.*://')
etcd=$(j -n kube-system pod -l component=etcd '{.items[0].spec.containers[0].image}' | sed 's/.*://'); ev=${etcd%-*}
rt=$(j node %{node} '{.status.nodeInfo.containerRuntimeVersion}')
grep -qx "etcd=$etcd" "$f" && grep -qx "runtime=$rt" "$f" """),
grep -qxE "etcd=v?($etcd|$ev)" "$f" && grep -qx "runtime=$rt" "$f" """),
],
},
# ---------------------------------------------------------------- WORK ----
@ -244,11 +245,17 @@ c["args"]=[a for a in c["args"] if a!="--kubelet-insecure-tls"]; print(json.dump
"setup": NS + r"""
mkns %{ns}
$K delete priorityclass %{pc} --ignore-not-found >/dev/null
$K -n %{ns} delete deploy payments --ignore-not-found --wait=true >/dev/null 2>&1
$K -n %{ns} create deploy payments --image=nginx:1.25 --replicas=2 --dry-run=client -o yaml | $K apply -f - >/dev/null
""",
"checks": [
C("PriorityClass %{pc} value %{val}, not default", 2, r"""eq "$(j priorityclass %{pc} '{.value}')" %{val} && [ "$(j priorityclass %{pc} '{.globalDefault}')" != true ]"""),
C("payments pods running with priority %{val}", 2, r"""eq "$(j -n %{ns} deploy payments '{.status.availableReplicas}')" 2 && for p in $(j -n %{ns} pod -l app=payments '{.items[*].spec.priority}'); do eq "$p" %{val} || exit 1; done"""),
C("payments pods running with priority %{val}", 2, r"""
for t in $(seq 1 15); do # the old ReplicaSet's pods linger for a moment while they terminate
ok=1; eq "$(j -n %{ns} deploy payments '{.status.availableReplicas}')" 2 || ok=0
for p in $(j -n %{ns} pod -l app=payments '{.items[*].spec.priority}'); do eq "$p" %{val} || ok=0; done
[ $ok = 1 ] && exit 0; sleep 2
done; exit 1"""),
],
},
{
@ -275,6 +282,8 @@ $K -n %{ns} delete deploy %{dep} --ignore-not-found >/dev/null
},
{
"id": 26, "domain": "WORK", "title": "DaemonSet on every node",
# needs every node schedulable: Q25/Q107 taint W1 and must keep the taint, Q307 blocks it
"breaks": ["w1-taint", "w1-block"],
"params": lambda r: {"ns": ns_pick(r, "agents"), "ds": name_pick(r, "node-agent")},
"task": """
Create a DaemonSet `%{ds}` in namespace `%{ns}` (image `busybox:1.36`, command `sleep 3600`)
@ -284,7 +293,9 @@ Requests: cpu 10m, memory 16Mi.
"setup": NS + r"""mkns %{ns}; $K -n %{ns} delete ds %{ds} --ignore-not-found >/dev/null""",
"checks": [
C("DaemonSet exists with requests set", 1, r"""eq "$(j -n %{ns} ds %{ds} '{.spec.template.spec.containers[0].resources.requests.cpu}')" 10m"""),
C("tolerates the control-plane taint", 1, r"""j -n %{ns} ds %{ds} '{.spec.template.spec.tolerations[*].key}' | grep -q control-plane"""),
C("tolerates the control-plane taint", 1, r"""$K -n %{ns} get ds %{ds} -o json 2>/dev/null | python3 -c '
import sys,json; t=json.load(sys.stdin)["spec"]["template"]["spec"].get("tolerations") or []
sys.exit(0 if any("control-plane" in (x.get("key") or "") or (x.get("operator")=="Exists" and not x.get("key")) for x in t) else 1)'"""),
C("one ready pod per node (including control-plane)", 2, r"""n=$($K get nodes --no-headers 2>/dev/null | wc -l); eq "$(j -n %{ns} ds %{ds} '{.status.numberReady}')" "$n" && eq "$(j -n %{ns} ds %{ds} '{.status.desiredNumberScheduled}')" "$n" """),
],
},
@ -418,7 +429,8 @@ $K -n %{ns} create deploy web --image=nginx:1.25 --port=80 --dry-run=client -o y
$K -n %{ns} create deploy api --image=nginx:1.25 --port=80 --dry-run=client -o yaml | $K apply -f - >/dev/null
$K -n %{ns} expose deploy web --port=80 --dry-run=client -o yaml | $K apply -f - >/dev/null
$K -n %{ns} expose deploy api --port=8080 --target-port=80 --dry-run=client -o yaml | $K apply -f - >/dev/null
$K delete gatewayclass %{gc} --ignore-not-found >/dev/null 2>&1; $K -n %{ns} delete gateway %{gw} --ignore-not-found >/dev/null 2>&1; $K -n %{ns} delete httproute %{host}-route --ignore-not-found >/dev/null 2>&1
# these fail harmlessly when the Gateway API CRDs are not installed yet - installing them is step 1
$K delete gatewayclass %{gc} --ignore-not-found >/dev/null 2>&1; $K -n %{ns} delete gateway %{gw} --ignore-not-found >/dev/null 2>&1; $K -n %{ns} delete httproute %{host}-route --ignore-not-found >/dev/null 2>&1 || true
""",
"checks": [
C("Gateway API CRDs installed (gateways, httproutes)", 1, r"""$K get crd gateways.gateway.networking.k8s.io httproutes.gateway.networking.k8s.io >/dev/null 2>&1"""),
@ -430,7 +442,7 @@ eq "$($h -o jsonpath='{.spec.parentRefs[0].name}')" %{gw} || exit 1
$h -o json | python3 -c '
import sys,json; r=json.load(sys.stdin)["spec"]["rules"]
api=[x for x in r if any(m.get("path",{}).get("value")=="/api" for m in x.get("matches",[]))]
web=[x for x in r if x is not api and x.get("backendRefs",[{}])[0].get("name")=="web"]
web=[x for x in r if x not in api and x.get("backendRefs",[{}])[0].get("name")=="web"]
ok=api and api[0]["backendRefs"][0]["name"]=="api" and api[0]["backendRefs"][0]["port"]==8080 and web and web[0]["backendRefs"][0]["port"]==80
sys.exit(0 if ok else 1)'"""),
],
@ -492,8 +504,10 @@ sys.exit(0 if ok else 1)'"""),
1. In `%{ns}`, StatefulSet `%{sts}` (2 replicas) exists but has no Service. Create the **headless**
Service `%{sts}` it needs (port 5432) so that `%{sts}-0.%{sts}.%{ns}.svc.cluster.local` resolves.
2. Create Service `legacy` of type ExternalName pointing at `%{ext}`.
3. From a pod in the namespace, resolve `%{sts}-0.%{sts}` and write the full nslookup output
to `{WS}/q43/pod0.txt`.
3. From a pod in the namespace, resolve pod 0's stable name
`%{sts}-0.%{sts}.%{ns}.svc.cluster.local` and write the full nslookup output to
`{WS}/q43/pod0.txt`. (busybox's nslookup applies no search domains, so a short name such as
`%{sts}-0.%{sts}` fails even when DNS is healthy.)
""",
"setup": NS + r"""
mkns %{ns}; rm -rf "$WS/q43"; mkdir -p "$WS/q43"
@ -550,11 +564,18 @@ spec: {accessModes: [ReadWriteOnce], storageClassName: manual, resources: {reque
EOF
for i in $(seq 1 20); do [ "$(j -n %{ns} pvc old-claim '{.status.phase}')" = Bound ] && break; sleep 1; done
$K -n %{ns} delete pvc old-claim >/dev/null
# every PV that exists now; the "no extra PV" check compares against this, so Retain leftovers from
# earlier sessions (this task's own, Q15's pv-data, Set C's) can no longer fail it
$K get pv -o jsonpath='{.items[*].metadata.name}' > "$WS/.q35-pvs"
""",
"checks": [
C("PV %{pv} still exists and was not recreated", 2, r"""[ -n "$(j pv %{pv} '{.metadata.uid}')" ] && eq "$(j pv %{pv} '{.spec.persistentVolumeReclaimPolicy}')" Retain"""),
C("PVC %{pvc} Bound to %{pv}", 3, r"""eq "$(j -n %{ns} pvc %{pvc} '{.status.phase}')" Bound && eq "$(j -n %{ns} pvc %{pvc} '{.spec.volumeName}')" %{pv}"""),
C("no extra PVs were created", 1, r"""[ "$($K get pv --no-headers 2>/dev/null | grep -c manual)" -le 2 ]"""),
C("no extra PVs were created", 1, r"""
before=" $(cat "$WS/.q35-pvs" 2>/dev/null) "; [ "$before" != " " ] || exit 1
for pv in $($K get pv -o jsonpath='{range .items[?(@.spec.storageClassName=="manual")]}{.metadata.name}{" "}{end}' 2>/dev/null); do
case "$before" in *" $pv "*) ;; *) exit 1 ;; esac
done"""),
],
},
{

View File

@ -32,14 +32,18 @@ $K delete clusterrole %{sa}-nodes --ignore-not-found >/dev/null; $K delete clust
$K create clusterrolebinding %{sa}-legacy-admin --clusterrole=cluster-admin --serviceaccount=%{ns}:%{sa} >/dev/null
""",
"checks": [
C("can get/list/watch/patch deployments and replicasets in %{ns}", 2, r"""for res in deployments.apps replicasets.apps; do for v in get list watch patch; do eq "$($K auth can-i $v $res -n %{ns} --as=system:serviceaccount:%{ns}:%{sa} 2>/dev/null)" yes || exit 1; done; done"""),
C("can create pods/exec in %{ns}; can list nodes", 2, r"""eq "$($K auth can-i create pods/exec -n %{ns} --as=system:serviceaccount:%{ns}:%{sa} 2>/dev/null)" yes && eq "$($K auth can-i list nodes --as=system:serviceaccount:%{ns}:%{sa} 2>/dev/null)" yes"""),
C("can get/list/watch/patch deployments and replicasets in %{ns}", 2, r"""! $K get clusterrolebinding %{sa}-legacy-admin >/dev/null 2>&1 || exit 1 # not via cluster-admin
for res in deployments.apps replicasets.apps; do for v in get list watch patch; do eq "$($K auth can-i $v $res -n %{ns} --as=system:serviceaccount:%{ns}:%{sa} 2>/dev/null)" yes || exit 1; done; done"""),
C("can create pods/exec in %{ns}; can list nodes", 2, r"""! $K get clusterrolebinding %{sa}-legacy-admin >/dev/null 2>&1 || exit 1
eq "$($K auth can-i create pods --subresource=exec -n %{ns} --as=system:serviceaccount:%{ns}:%{sa} 2>/dev/null)" yes && eq "$($K auth can-i list nodes --as=system:serviceaccount:%{ns}:%{sa} 2>/dev/null)" yes"""),
C("no over-grant: cannot delete deployments, cannot get secrets, cannot get nodes", 3, r"""for t in "delete deployments -n %{ns}" "get secrets -n %{ns}" "get nodes" "create pods -n default"; do eq "$($K auth can-i $t --as=system:serviceaccount:%{ns}:%{sa} 2>/dev/null || true)" no || exit 1; done"""),
C("the cluster-admin binding is gone and named objects exist", 2, r"""! $K get clusterrolebinding %{sa}-legacy-admin >/dev/null 2>&1 && $K -n %{ns} get role %{sa}-role rolebinding %{sa}-rb >/dev/null 2>&1 && $K get clusterrole %{sa}-nodes clusterrolebinding %{sa}-crb >/dev/null 2>&1"""),
C("the cluster-admin binding is gone and named objects exist", 2, r"""! $K get clusterrolebinding %{sa}-legacy-admin >/dev/null 2>&1 && $K -n %{ns} get role/%{sa}-role rolebinding/%{sa}-rb >/dev/null 2>&1 && $K get clusterrole/%{sa}-nodes clusterrolebinding/%{sa}-crb >/dev/null 2>&1"""),
],
},
{
"id": 102, "domain": "ARCH", "breaks": "etcd", "difficulty": "hard", "title": "etcd restore",
# solo: restoring rolls EVERY API object back to the snapshot - other tasks' seeds and answers too
"solo": True,
"params": lambda r: {"ctx": "k8s-c1-H", "cm": name_pick(r, "restore-proof")},
"task": CTX + """
An etcd snapshot taken earlier is at `/opt/cka/backup.db` on the control-plane node.
@ -49,6 +53,9 @@ been deleted.
**Restore the cluster's etcd from that snapshot** so the ConfigMap exists again. The cluster
must be healthy afterwards (API server up, all kube-system pods Running).
Do not recreate the ConfigMap by hand — its data must come from the snapshot.
A restore rolls every API object back to the moment of the snapshot — on a real exam cluster that
would include the work you did on other tasks, so a restore is the last thing you do.
""",
"setup": r"""
nx "$CP" 'mkdir -p /opt/cka; rm -f /opt/cka/backup.db'
@ -58,15 +65,24 @@ $K -n default create cm %{cm} --from-literal=proof=$(date +%s)-%{cm} >/dev/null
out=$(nx "$CP" 'etcdctl --endpoints=https://127.0.0.1:2379 --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/server.crt --key=/etc/kubernetes/pki/etcd/server.key snapshot save /opt/cka/backup.db' 2>&1) \
|| { echo "snapshot save on $CP failed: $out"; exit 1; }
sleep 2; $K -n default delete cm %{cm} >/dev/null
nx "$CP" 'awk "/hostPath:/{getline; p=\$2} /name: etcd-data/{if (p) print p}" /etc/kubernetes/manifests/etcd.yaml' </dev/null > "$WS/.q102-datadir"
date +%s > "$WS/.q102-t0"
""",
"checks": [
C("ConfigMap %{cm} exists again with its original data", 6, r"""j -n default cm %{cm} '{.data.proof}' | grep -q -- '-%{cm}$'"""),
C("etcd uses a restored data directory (manifest changed)", 2, r"""! nx "$CP" 'grep -q "path: /var/lib/etcd$" /etc/kubernetes/manifests/etcd.yaml' """),
C("control plane healthy: etcd, apiserver, scheduler, controller-manager Running", 2, r"""for c in etcd kube-apiserver kube-scheduler kube-controller-manager; do eq "$(j -n kube-system pod -l component=$c '{.items[0].status.phase}')" Running || exit 1; done"""),
C("etcd runs on a data directory restored from the snapshot", 2, r"""
was=$(cat "$WS/.q102-datadir" 2>/dev/null); t0=$(cat "$WS/.q102-t0" 2>/dev/null); [ -n "$was" ] && [ -n "$t0" ] || exit 1
now=$(nx "$CP" 'awk "/hostPath:/{getline; p=\$2} /name: etcd-data/{if (p) print p}" /etc/kubernetes/manifests/etcd.yaml' </dev/null)
[ -n "$now" ] || exit 1
[ "$now" != "$was" ] && exit 0
# the same directory name as before is fine if it was rebuilt from the snapshot after seeding
m=$(nx "$CP" "stat -c %Y $now/member 2>/dev/null" </dev/null); [ -n "$m" ] && [ "$m" -gt "$t0" ]"""),
C("control plane healthy: etcd, apiserver, scheduler, controller-manager Running", 2, r"""j -n default cm %{cm} '{.data.proof}' | grep -q -- '-%{cm}$' || exit 1 # only once restored
for c in etcd kube-apiserver kube-scheduler kube-controller-manager; do eq "$(j -n kube-system pod -l component=$c '{.items[0].status.phase}')" Running || exit 1; done"""),
],
},
{
"id": 103, "domain": "ARCH", "breaks": "w3-kubelet", "difficulty": "hard", "title": "Static pod on a relocated manifest path",
"id": 103, "domain": "ARCH", "breaks": ["w3-kubelet", "w3-staticpath"], "difficulty": "hard", "title": "Static pod on a relocated manifest path",
"params": lambda r: {"ctx": "k8s-c1-H", "ns": ns_pick(r, "edge"), "pod": name_pick(r, "edge-proxy")},
"task": CTX + """
Exactly one worker node is labelled `tier=edge`. On that node create a **static pod**:
@ -108,7 +124,7 @@ Onboard user `%{user}` (group `developers`) end to end:
C("CSR %{user} approved with certificate, O=developers", 2, r"""[ -n "$(j csr %{user} '{.status.certificate}')" ] && j csr %{user} '{.status.certificate}' | base64 -d | openssl x509 -noout -subject 2>/dev/null | grep -q 'O *= *developers'"""),
C("kubeconfig is self-contained and works without flags", 3, r"""f="$WS/q104/%{user}.kubeconfig"; [ -f "$f" ] || exit 1; grep -q 'client-certificate-data' "$f" && grep -q 'certificate-authority-data' "$f" && ! grep -q 'client-certificate:' "$f" && kubectl --kubeconfig "$f" get pods >/dev/null 2>&1"""),
C("context %{user}@lab defaults to %{ns}", 1, r"""eq "$(kubectl --kubeconfig "$WS/q104/%{user}.kubeconfig" config view -o jsonpath='{.contexts[?(@.name=="%{user}@lab")].context.namespace}')" %{ns}"""),
C("edit rights in %{ns}, none elsewhere", 2, r"""eq "$($K auth can-i create deployments -n %{ns} --as=%{user} 2>/dev/null)" yes && eq "$($K auth can-i delete pods -n %{ns} --as=%{user} 2>/dev/null)" yes && eq "$($K auth can-i get pods -n default --as=%{user} 2>/dev/null || true)" no && eq "$($K auth can-i get secrets -n kube-system --as=%{user} 2>/dev/null || true)" no"""),
C("edit rights in %{ns}, none elsewhere", 2, r"""eq "$($K auth can-i create deployments -n %{ns} --as=%{user} --as-group=developers 2>/dev/null)" yes && eq "$($K auth can-i delete pods -n %{ns} --as=%{user} --as-group=developers 2>/dev/null)" yes && eq "$($K auth can-i get pods -n default --as=%{user} --as-group=developers 2>/dev/null || true)" no && eq "$($K auth can-i get secrets -n kube-system --as=%{user} --as-group=developers 2>/dev/null || true)" no"""),
],
},
{
@ -159,18 +175,22 @@ $K delete ns %{ns} --ignore-not-found --wait=false >/dev/null
"id": 106, "domain": "ARCH", "difficulty": "hard", "title": "CRD with validation and defaults",
"params": lambda r: {"ctx": ctx_pick(r), "kind": r.choice(["Rollout", "Canary", "Release"]), "env": r.choice(["prod", "stage"])},
"task": CTX + """
Create CRD `%{kind}` (group `deploy.example.com`, v1, Namespaced, plural lowercase+s, shortName `%{kind}`-initials) with schema:
Create CRD `%{kind}` (group `deploy.example.com`, v1, Namespaced, plural = the kind in lowercase plus
`s`, shortName = the first two letters of the kind in lowercase) with schema:
- `spec.replicas` integer, **required**, minimum 1, maximum 10,
- `spec.environment` string, enum `prod|stage|dev`, **default** `dev`,
- `spec.image` string, required.
Then:
1. Create `%{kind}` `good` in `default` with replicas 3, image `nginx:1.27`, environment `%{env}`.
2. Save the API server's rejection message for a `%{kind}` with `replicas: 0` to `{WS}/q106/rejected.txt`.
3. Create `%{kind}` `defaulted` with only replicas 2 and image set; its environment must be `dev` after creation.
3. Create `%{kind}` `defaulted` in `default` with only replicas 2 and image set; its environment
must be `dev` after creation.
This context's default namespace is not necessarily `default` — say `-n default` explicitly.
""",
"setup": r"""rm -rf "$WS/q106"; mkdir -p "$WS/q106"; k=$(echo %{kind} | tr A-Z a-z); $K delete crd ${k}s.deploy.example.com --ignore-not-found >/dev/null""",
"checks": [
C("CRD exists, Namespaced, replicas has min/max and is required", 2, r"""k=$(echo %{kind} | tr A-Z a-z); c="$K get crd ${k}s.deploy.example.com"; eq "$($c -o jsonpath='{.spec.scope}' 2>/dev/null)" Namespaced && s=$($c -o jsonpath='{.spec.versions[0].schema.openAPIV3Schema.properties.spec}'); echo "$s" | grep -q '"minimum":1' && echo "$s" | grep -q '"maximum":10' && echo "$s" | grep -q '"required":\[' """),
C("CRD exists, Namespaced, replicas has min/max and is required", 2, r"""s=""; k=$(echo %{kind} | tr A-Z a-z); c="$K get crd ${k}s.deploy.example.com"; eq "$($c -o jsonpath='{.spec.scope}' 2>/dev/null)" Namespaced && s=$($c -o jsonpath='{.spec.versions[0].schema.openAPIV3Schema.properties.spec}'); echo "$s" | grep -q '"minimum":1' && echo "$s" | grep -q '"maximum":10' && echo "$s" | grep -q '"required":\[' """),
C("good created with environment %{env}", 1, r"""k=$(echo %{kind} | tr A-Z a-z); eq "$(j -n default ${k}s.deploy.example.com good '{.spec.environment}')" %{env}"""),
C("replicas 0 is rejected and the message was saved", 2, r"""grep -qiE 'minimum|greater than|>= 1|invalid' "$WS/q106/rejected.txt" 2>/dev/null"""),
C("defaulting works: defaulted.spec.environment == dev", 2, r"""k=$(echo %{kind} | tr A-Z a-z); eq "$(j -n default ${k}s.deploy.example.com defaulted '{.spec.environment}')" dev"""),
@ -191,7 +211,9 @@ Write the new `apiserver` certificate's expiry (as printed by `kubeadm certs che
C("apiserver certificate was renewed (newer notBefore)", 3, r"""b=$(sed 's/notBefore=//' "$WS/.q122-before"); a=$(nx "$CP" 'openssl x509 -in /etc/kubernetes/pki/apiserver.crt -noout -startdate' | sed 's/notBefore=//'); [ "$(date -d "$a" +%s)" -gt "$(date -d "$b" +%s)" ]"""),
C("all components restarted and Running with the new certs (apiserver serving cert matches file)", 3, r"""
for c in kube-apiserver kube-scheduler kube-controller-manager etcd; do eq "$(j -n kube-system pod -l component=$c '{.items[0].status.phase}')" Running || exit 1; done
srv=$(nx "$CP" 'echo | openssl s_client -connect 127.0.0.1:6443 2>/dev/null | openssl x509 -noout -serial'); f=$(nx "$CP" 'openssl x509 -in /etc/kubernetes/pki/apiserver.crt -noout -serial'); [ -n "$srv" ] && eq "$srv" "$f" """),
srv=$(nx "$CP" 'echo | openssl s_client -connect 127.0.0.1:6443 2>/dev/null | openssl x509 -noout -serial -startdate'); f=$(nx "$CP" 'openssl x509 -in /etc/kubernetes/pki/apiserver.crt -noout -serial -startdate'); [ -n "$srv" ] && eq "$srv" "$f" || exit 1
# ...and it is the RENEWED certificate: an untouched cluster also serves the file it has on disk
b=$(sed 's/notBefore=//' "$WS/.q122-before"); a=$(echo "$srv" | sed -n 's/notBefore=//p'); [ "$(date -d "$a" +%s)" -gt "$(date -d "$b" +%s)" ]"""),
C("admin.conf uses the renewed client cert", 2, r"""nx "$CP" 'grep client-certificate-data /etc/kubernetes/admin.conf | awk "{print \$2}" | base64 -d | openssl x509 -noout -startdate' | sed 's/notBefore=//' > /tmp/q122a; b=$(sed 's/notBefore=//' "$WS/.q122-before"); [ "$(date -d "$(cat /tmp/q122a)" +%s)" -gt "$(date -d "$b" +%s)" ]"""),
C("expiry.txt matches kubeadm's report for apiserver", 1, r"""exp=$(nx "$CP" 'kubeadm certs check-expiration 2>/dev/null' | awk '$1=="apiserver"{print $2,$3,$4,$5,$6}'); [ -n "$exp" ] && grep -qF "$exp" "$WS/q122/expiry.txt" 2>/dev/null"""),
],
@ -291,9 +313,15 @@ $K -n %{ns} set image deploy %{dep} nginx=nginx:1.27 >/dev/null; $K -n %{ns} rol
$K -n %{ns} set image deploy %{dep} nginx=nginx:9.99-broken >/dev/null
""",
"checks": [
C("running pods use nginx:1.25 and 3 are available", 3, r"""eq "$(j -n %{ns} deploy %{dep} '{.status.availableReplicas}')" 3 && for i in $(j -n %{ns} pod -l app=%{dep} '{.items[*].spec.containers[0].image}'); do eq "$i" nginx:1.25 || exit 1; done"""),
C("running pods use nginx:1.25 and 3 are available", 3, r"""
for t in $(seq 1 15); do # pods of the replaced ReplicaSets linger for a moment while they terminate
ok=1; eq "$(j -n %{ns} deploy %{dep} '{.status.availableReplicas}')" 3 || ok=0
for i in $(j -n %{ns} pod -l app=%{dep} '{.items[*].spec.containers[0].image}'); do eq "$i" nginx:1.25 || ok=0; done
[ $ok = 1 ] && exit 0; sleep 2
done; exit 1"""),
C("deployment is paused with template image nginx:1.27 and RELEASE=next", 2, r"""d="$K -n %{ns} get deploy %{dep}"; eq "$($d -o jsonpath='{.spec.paused}' 2>/dev/null)" true && eq "$($d -o jsonpath='{.spec.template.spec.containers[0].image}')" nginx:1.27 && eq "$($d -o jsonpath='{.spec.template.spec.containers[0].env[?(@.name=="RELEASE")].value}')" next"""),
C("no ReplicaSet exists for the paused template (1.27 + RELEASE)", 2, r"""! $K -n %{ns} get rs -o json 2>/dev/null | python3 -c '
C("no ReplicaSet exists for the paused template (1.27 + RELEASE)", 2, r"""eq "$(j -n %{ns} deploy %{dep} '{.spec.template.spec.containers[0].image}')" nginx:1.27 || exit 1
! $K -n %{ns} get rs -o json 2>/dev/null | python3 -c '
import sys,json
for rs in json.load(sys.stdin)["items"]:
c=rs["spec"]["template"]["spec"]["containers"][0]
@ -307,44 +335,99 @@ sys.exit(1)'"""),
"task": CTX + """
Create pod `%{pod}` in `%{ns}`:
- a **native sidecar** (an init container with `restartPolicy: Always`) named `agent` (busybox:1.36)
that appends `$(date)` to `/var/log/agent.log` every second,
that appends `$(date)` to `/var/log/agent/agent.log` every second,
- main container `app` (nginx:1.27) with a readiness probe (HTTP GET / on 80, period 5s)
and a liveness probe (TCP 80),
- shared `emptyDir` `logs` mounted at `/var/log` in both,
- shared `emptyDir` `logs` mounted at `/var/log/agent` in both (not at `/var/log`: nginx keeps its
own logs under `/var/log/nginx`, and an emptyDir mounted over `/var/log` hides that directory),
- `terminationGracePeriodSeconds: 5`.
When it is Running, write the names of **all** containers of the pod (init and regular), sorted,
one per line, to `{WS}/q109/containers.txt`, and the pod's `.status.podIP` to `{WS}/q109/ip.txt`.
""",
"setup": NS + r"""mkns %{ns}; rm -rf "$WS/q109"; mkdir -p "$WS/q109"; $K -n %{ns} delete pod %{pod} --ignore-not-found --wait=false >/dev/null""",
"setup": NS + r"""mkns %{ns}; rm -rf "$WS/q109"; mkdir -p "$WS/q109"; $K -n %{ns} delete pod %{pod} --ignore-not-found --force --grace-period=0 --wait=true >/dev/null 2>&1""",
"checks": [
C("native sidecar: initContainer agent with restartPolicy Always", 2, r"""eq "$(j -n %{ns} pod %{pod} '{.spec.initContainers[?(@.name=="agent")].restartPolicy}')" Always"""),
C("probes on app as specified", 2, r"""c='{.spec.containers[?(@.name=="app")]'; eq "$(j -n %{ns} pod %{pod} "$c.readinessProbe.httpGet.path}")" / && eq "$(j -n %{ns} pod %{pod} "$c.readinessProbe.periodSeconds}")" 5 && eq "$(j -n %{ns} pod %{pod} "$c.livenessProbe.tcpSocket.port}")" 80"""),
C("Running, shared log written by agent, grace period 5", 2, r"""eq "$(j -n %{ns} pod %{pod} '{.status.phase}')" Running && eq "$(j -n %{ns} pod %{pod} '{.spec.terminationGracePeriodSeconds}')" 5 && [ -n "$($K -n %{ns} exec %{pod} -c app -- sh -c 'tail -1 /var/log/agent.log' 2>/dev/null)" ]"""),
C("report files exact", 2, r"""want=$( (j -n %{ns} pod %{pod} '{.spec.initContainers[*].name}'; echo; j -n %{ns} pod %{pod} '{.spec.containers[*].name}') | tr ' ' '\n' | grep . | sort); [ "$(grep . "$WS/q109/containers.txt" 2>/dev/null | sort)" = "$want" ] && eq "$(tr -d '[:space:]' < "$WS/q109/ip.txt" 2>/dev/null)" "$(j -n %{ns} pod %{pod} '{.status.podIP}')" """),
C("Running, shared log written by agent, grace period 5", 2, r"""eq "$(j -n %{ns} pod %{pod} '{.status.phase}')" Running && eq "$(j -n %{ns} pod %{pod} '{.spec.terminationGracePeriodSeconds}')" 5 && [ -n "$($K -n %{ns} exec %{pod} -c app -- sh -c 'tail -1 /var/log/agent/agent.log' 2>/dev/null)" ]"""),
C("report files exact", 2, r"""want=$( (j -n %{ns} pod %{pod} '{.spec.initContainers[*].name}'; echo; j -n %{ns} pod %{pod} '{.spec.containers[*].name}') | tr ' ' '\n' | grep . | sort); [ -n "$want" ] && [ "$(grep . "$WS/q109/containers.txt" 2>/dev/null | sort)" = "$want" ] && eq "$(tr -d '[:space:]' < "$WS/q109/ip.txt" 2>/dev/null)" "$(j -n %{ns} pod %{pod} '{.status.podIP}')" """),
],
},
{
"id": 110, "domain": "WORK", "difficulty": "hard", "title": "Cluster audit to exact files",
"params": lambda r: {"ctx": "k8s-c1-H"},
# Scoped to its own namespace: the cluster-wide version compared a file written mid-exam with the
# whole cluster at submit time, so every pod another task (or a grader probe) created or deleted
# in between failed it. killer.sh-style exact-output tasks work on data the task owns.
"id": 110, "domain": "WORK", "difficulty": "hard", "title": "Namespace audit to exact files",
"params": lambda r: {"ctx": "k8s-c1-H", "ns": ns_pick(r, "audit")},
"task": CTX + """
Produce three files, contents **exactly** as specified (no headers, sorted with `sort`, one per line):
1. `{WS}/q110/no-readiness.txt` — `<namespace>/<pod>` for every pod in the cluster whose first
container has **no** readiness probe. Exclude namespace `kube-system`.
2. `{WS}/q110/per-node.txt` — `<node> <count>` of Running pods per node, all namespaces.
3. `{WS}/q110/images.txt` — every distinct container image running in namespace `kube-system`.
Namespace `%{ns}` runs a mix of workloads. Produce three files with contents **exactly** as
specified — no headers, one entry per line, sorted with `sort`:
1. `{WS}/q110/no-readiness.txt` — the name of every Pod in `%{ns}` whose **first** container has
no readiness probe (whatever the Pod's phase).
2. `{WS}/q110/per-node.txt` — `<node> <count>`: how many Pods in `%{ns}` are **Running** on each
node, listing only nodes that have at least one.
3. `{WS}/q110/images.txt` — every distinct container image used by Pods in `kube-system`.
""",
"setup": NS + r"""
mkns %{ns}; rm -rf "$WS/q110"; mkdir -p "$WS/q110"
$K -n %{ns} delete pod --all --force --grace-period=0 --wait=true >/dev/null 2>&1
$K apply -f - >/dev/null <<EOF
apiVersion: v1
kind: Pod
metadata: {name: web-a, namespace: %{ns}}
spec:
nodeSelector: {kubernetes.io/hostname: $W1}
containers: [{name: web, image: nginx:1.27, readinessProbe: {httpGet: {path: /, port: 80}}}]
---
apiVersion: v1
kind: Pod
metadata: {name: web-b, namespace: %{ns}}
spec:
nodeSelector: {kubernetes.io/hostname: $W2}
containers: [{name: web, image: nginx:1.27, readinessProbe: {tcpSocket: {port: 80}}}]
---
apiVersion: v1
kind: Pod
metadata: {name: batch-a, namespace: %{ns}}
spec:
nodeSelector: {kubernetes.io/hostname: $W1}
containers: [{name: batch, image: busybox:1.36, command: [sleep, "86400"]}]
---
apiVersion: v1
kind: Pod
metadata: {name: cache-a, namespace: %{ns}}
spec:
nodeSelector: {kubernetes.io/hostname: $W3}
containers:
- {name: cache, image: redis:7-alpine}
- {name: exporter, image: nginx:1.27, readinessProbe: {httpGet: {path: /, port: 80}}}
---
apiVersion: v1
kind: Pod
metadata: {name: once-a, namespace: %{ns}}
spec:
restartPolicy: Never
nodeSelector: {kubernetes.io/hostname: $W3}
containers: [{name: job, image: busybox:1.36, command: ["true"]}]
---
apiVersion: v1
kind: Pod
metadata: {name: idle-a, namespace: %{ns}}
spec:
nodeSelector: {kubernetes.io/hostname: $W2}
containers: [{name: idle, image: busybox:1.36, command: [sleep, "86400"]}]
EOF
for i in $(seq 1 60); do [ "$($K -n %{ns} get pod once-a -o jsonpath='{.status.phase}' 2>/dev/null)" = Succeeded ] && break; sleep 2; done
""",
"setup": r"""rm -rf "$WS/q110"; mkdir -p "$WS/q110" """,
"checks": [
C("no-readiness.txt exact", 3, r"""want=$($K get pods -A -o json 2>/dev/null | python3 -c '
C("no-readiness.txt exact (first container only, every phase)", 3, r"""want=$($K -n %{ns} get pods -o json 2>/dev/null | python3 -c '
import sys,json
for p in json.load(sys.stdin)["items"]:
if p["metadata"]["namespace"]=="kube-system": continue
if not p["spec"]["containers"][0].get("readinessProbe"): print(p["metadata"]["namespace"]+"/"+p["metadata"]["name"])' | sort); [ "$(grep . "$WS/q110/no-readiness.txt" 2>/dev/null | sort)" = "$want" ]"""),
C("per-node.txt exact", 2, r"""want=$($K get pods -A -o json 2>/dev/null | python3 -c '
if not p["spec"]["containers"][0].get("readinessProbe"): print(p["metadata"]["name"])' | sort); [ -n "$want" ] && [ "$(grep . "$WS/q110/no-readiness.txt" 2>/dev/null)" = "$want" ]"""),
C("per-node.txt exact (Running pods only)", 2, r"""want=$($K -n %{ns} get pods -o json 2>/dev/null | python3 -c '
import sys,json,collections
c=collections.Counter(p["spec"].get("nodeName") for p in json.load(sys.stdin)["items"] if p["status"]["phase"]=="Running" and p["spec"].get("nodeName"))
for n in sorted(c): print(n, c[n])'); [ "$(grep . "$WS/q110/per-node.txt" 2>/dev/null | sort)" = "$(echo "$want" | sort)" ]"""),
C("images.txt exact", 2, r"""want=$(j -n kube-system pods '{.items[*].spec.containers[*].image}' | tr ' ' '\n' | sort -u); [ "$(grep . "$WS/q110/images.txt" 2>/dev/null | sort -u)" = "$want" ]"""),
for n in c: print(n, c[n])' | sort); [ -n "$want" ] && [ "$(grep . "$WS/q110/per-node.txt" 2>/dev/null)" = "$want" ]"""),
C("images.txt exact", 2, r"""want=$(j -n kube-system pods '{.items[*].spec.containers[*].image}' | tr ' ' '\n' | sort -u); [ -n "$want" ] && [ "$(grep . "$WS/q110/images.txt" 2>/dev/null)" = "$want" ]"""),
],
},
{
@ -361,7 +444,7 @@ by `kubectl config view --minify`, to `{WS}/q121/ns.txt`.
""",
"setup": NS + r"""mkns %{ns}; rm -rf "$WS/q121"; mkdir -p "$WS/q121"; $K -n %{ns} delete deploy,svc %{dep} --ignore-not-found >/dev/null; $K -n default delete deploy,svc %{dep} --ignore-not-found >/dev/null""",
"checks": [
C("Deployment %{dep} in %{ns} (the context's namespace), 2 available, owner=ac", 3, r"""eq "$(j -n %{ns} deploy %{dep} '{.status.availableReplicas}')" 2 && eq "$(j -n %{ns} deploy %{dep} '{.spec.template.metadata.labels.owner}')" ac"""),
C("Deployment %{dep} in %{ns} (the context's namespace), 2 available, owner=ac", 3, r"""eq "$(j -n %{ns} deploy %{dep} '{.status.availableReplicas}')" 2 && { eq "$(j -n %{ns} deploy %{dep} '{.metadata.labels.owner}')" ac || eq "$(j -n %{ns} deploy %{dep} '{.spec.template.metadata.labels.owner}')" ac; }"""),
C("Service %{dep} port 9000 in the same namespace", 1, r"""eq "$(j -n %{ns} svc %{dep} '{.spec.ports[0].port}')" 9000"""),
C("nothing was created in namespace default by mistake", 1, r"""! $K -n default get deploy %{dep} >/dev/null 2>&1"""),
C("ns.txt has the namespace", 1, r"""eq "$(tr -d '[:space:]' < "$WS/q121/ns.txt" 2>/dev/null)" %{ns}"""),
@ -461,7 +544,7 @@ ok=any(any(any(h.get("name","").lower()=="x-canary" and h.get("value")=="always"
sys.exit(0 if ok else 1)'"""),
C("default rule weights v1 %{w2} / v2 %{w}", 2, r"""$K -n %{ns} get httproute %{host}-route -o json 2>/dev/null | python3 -c '
import sys,json; r=json.load(sys.stdin)["spec"]["rules"]
ok=any(not x.get("matches") or all(not m.get("headers") for m in x["matches"]) and {b["name"]:b.get("weight") for b in x["backendRefs"]}=={"v1":%{w2},"v2":%{w}} for x in r if len(x.get("backendRefs",[]))==2)
ok=any((not x.get("matches") or all(not m.get("headers") for m in x["matches"])) and {b["name"]:b.get("weight") for b in x["backendRefs"]}=={"v1":%{w2},"v2":%{w}} for x in r if len(x.get("backendRefs",[]))==2)
sys.exit(0 if ok else 1)'"""),
],
},
@ -513,7 +596,18 @@ mkns %{ns}
for n in "$W1" "$W2" "$W3"; do $K label node $n disk- >/dev/null 2>&1 || true; done
$K label node "$W2" disk=nvme --overwrite >/dev/null
$K -n %{ns} delete pod consumer --ignore-not-found --wait=false >/dev/null; $K -n %{ns} delete pvc %{pvc} --ignore-not-found >/dev/null
$K delete pv lv-hdd lv-nvme --ignore-not-found >/dev/null
# The PV names are fixed but every session draws a new namespace, so a claim from an earlier run
# (another namespace, often with its pod still running) keeps lv-hdd/lv-nvme Bound. pv-protection
# then holds `kubectl delete pv` forever and the seed timed out. Release them from whoever has them.
for pv in lv-hdd lv-nvme; do
cns=$(j pv $pv '{.spec.claimRef.namespace}'); cn=$(j pv $pv '{.spec.claimRef.name}')
[ -n "$cns" ] && [ -n "$cn" ] || continue
for p in $($K -n "$cns" get pod -o jsonpath='{range .items[*]}{.metadata.name}{" "}{.spec.volumes[*].persistentVolumeClaim.claimName}{"\n"}{end}' 2>/dev/null | awk -v c="$cn" '{for(i=2;i<=NF;i++) if($i==c) print $1}'); do
$K -n "$cns" delete pod "$p" --grace-period=0 --force >/dev/null 2>&1
done
$K -n "$cns" delete pvc "$cn" --ignore-not-found --wait=false >/dev/null 2>&1
done
$K delete pv lv-hdd lv-nvme --ignore-not-found --timeout=90s >/dev/null 2>&1 || true
for pair in "lv-hdd:$W1:hdd:2Gi" "lv-nvme:$W2:nvme:1Gi"; do IFS=: read name node kind size <<<"$pair"
nx "$node" "mkdir -p /mnt/$name"
$K apply -f - >/dev/null <<EOF
@ -576,7 +670,7 @@ echo "$pv" > "$WS/.q116-pv"
},
# ============================================================== TSHOOT ====
{
"id": 117, "domain": "TSHOOT", "breaks": "w2-kubelet", "difficulty": "hard", "title": "Worker node: three things wrong", "disruptive": True,
"id": 117, "domain": "TSHOOT", "breaks": ["w2-kubelet", "w2-drain"], "difficulty": "hard", "title": "Worker node: three things wrong", "disruptive": True,
"params": lambda r: {"ctx": "k8s-c1-H"},
"task": CTX + """
Node `{W2}` is unusable. Make it fully usable again: `Ready`, schedulable, kubelet running and
@ -600,7 +694,7 @@ $K -n default delete pod node-probe --wait=false >/dev/null 2>&1; exit $ok"""),
],
},
{
"id": 118, "domain": "TSHOOT", "breaks": "scheduler", "difficulty": "hard", "title": "Two control-plane components down", "disruptive": True,
"id": 118, "domain": "TSHOOT", "breaks": ["scheduler", "controller-manager"], "difficulty": "hard", "title": "Two control-plane components down", "disruptive": True,
"params": lambda r: {"ctx": "k8s-c1-H"},
"task": CTX + """
New Deployments create no pods, and pods that do get created stay `Pending`. Two different
@ -673,7 +767,8 @@ j -n %{ns} deploy %{dep} '{.metadata.uid}' > "$WS/.q119-uid"; j -n %{ns} svc %{d
"checks": [
C("3 replicas Available", 3, r"""for i in $(seq 1 6); do [ "$(j -n %{ns} deploy %{dep} '{.status.availableReplicas}')" = 3 ] && exit 0; sleep 5; done; exit 1"""),
C("Service routes: endpoints ready, targetPort 80", 2, r"""eq "$(j -n %{ns} svc %{dep} '{.spec.ports[0].targetPort}')" 80 && [ -n "$(j -n %{ns} endpointslices -l kubernetes.io/service-name=%{dep} '{.items[*].endpoints[?(@.conditions.ready==true)].addresses[*]}')" ]"""),
C("Deployment UID and Service ClusterIP unchanged (fixed in place)", 2, r"""eq "$(j -n %{ns} deploy %{dep} '{.metadata.uid}')" "$(cat "$WS/.q119-uid")" && eq "$(j -n %{ns} svc %{dep} '{.spec.clusterIP}')" "$(cat "$WS/.q119-ip")" """),
C("Deployment UID and Service ClusterIP unchanged (fixed in place)", 2, r"""eq "$(j -n %{ns} deploy %{dep} '{.status.availableReplicas}')" 3 || exit 1 # only once it is actually fixed
eq "$(j -n %{ns} deploy %{dep} '{.metadata.uid}')" "$(cat "$WS/.q119-uid")" && eq "$(j -n %{ns} svc %{dep} '{.spec.clusterIP}')" "$(cat "$WS/.q119-ip")" """),
C("quota still present and permits the 3 pods", 1, r"""p=$(j -n %{ns} quota tight '{.spec.hard.pods}'); [ -n "$p" ] && [ "$p" -ge 3 ]"""),
],
},

View File

@ -243,12 +243,15 @@ Namespace `project-orion` is close to filling its nodes.
setup=NS + r"""
mkns project-orion
rm -rf "$WS/q204"; mkdir -p "$WS/q204"
$K -n project-orion delete deploy --all >/dev/null 2>&1; $K -n project-orion delete pod orion-api-0 --ignore-not-found --wait=false >/dev/null
$K apply -f - >/dev/null <<'EOF'
# the pod is re-created below under the same name: a delete that does not wait lets the apply land on
# the terminating old pod, which then vanishes and leaves the task without orion-api-0
$K -n project-orion delete deploy --all >/dev/null 2>&1; $K -n project-orion delete pod orion-api-0 --ignore-not-found --force --grace-period=0 --wait=true >/dev/null 2>&1
$K apply -f - >/dev/null <<EOF
apiVersion: v1
kind: Pod
metadata: {name: orion-api-0, namespace: project-orion, labels: {app: orion-api}}
spec:
affinity: {nodeAffinity: {requiredDuringSchedulingIgnoredDuringExecution: {nodeSelectorTerms: [{matchExpressions: [{key: kubernetes.io/hostname, operator: NotIn, values: [$W2]}]}]}}}
containers:
- name: api
image: busybox:1.36
@ -264,6 +267,7 @@ spec:
template:
metadata: {labels: {app: orion-batch}}
spec:
affinity: {nodeAffinity: {requiredDuringSchedulingIgnoredDuringExecution: {nodeSelectorTerms: [{matchExpressions: [{key: kubernetes.io/hostname, operator: NotIn, values: [$W2]}]}]}}}
containers: [{name: batch, image: busybox:1.36, command: [sleep, "86400"]}]
---
apiVersion: apps/v1
@ -275,6 +279,7 @@ spec:
template:
metadata: {labels: {app: orion-ledger}}
spec:
affinity: {nodeAffinity: {requiredDuringSchedulingIgnoredDuringExecution: {nodeSelectorTerms: [{matchExpressions: [{key: kubernetes.io/hostname, operator: NotIn, values: [$W2]}]}]}}}
containers:
- name: ledger
image: busybox:1.36
@ -283,11 +288,18 @@ spec:
EOF
""",
checks=[
C("evicted-first.txt lists exactly the BestEffort pods", 4, r"""
want=$($K -n project-orion get pod -o json 2>/dev/null | python3 -c '
# Every BestEffort pod this task seeded must be listed and no Burstable/Guaranteed one may be.
# BestEffort pods other tasks add to the namespace later (212's spread-check has no
# resources at all) are tolerated either way - they did not exist when the file was written.
C("evicted-first.txt lists the BestEffort pods and nothing else", 4, r"""
f="$WS/q204/evicted-first.txt"; [ -s "$f" ] || exit 1
$K -n project-orion get pod -o json 2>/dev/null | python3 -c '
import sys,json
print("\n".join(sorted(p["metadata"]["name"] for p in json.load(sys.stdin)["items"] if p["status"].get("qosClass")=="BestEffort")))')
[ -n "$want" ] && [ "$(grep . "$WS/q204/evicted-first.txt" 2>/dev/null | sort)" = "$want" ]"""),
items=json.load(sys.stdin)["items"]; got={l.strip() for l in open(sys.argv[1]) if l.strip()}
qos={p["metadata"]["name"]:p["status"].get("qosClass") for p in items}
seeded={p["metadata"]["name"] for p in items if p["metadata"].get("labels",{}).get("app")=="orion-batch"}
ok=seeded and seeded<=got and not any(qos.get(n) in ("Burstable","Guaranteed") for n in got)
sys.exit(0 if ok else 1)' "$f" """),
C("api-qos.txt says Burstable", 2, r"""grep -qi burstable "$WS/q204/api-qos.txt" 2>/dev/null"""),
C("guaranteed.txt names orion-ledger", 2, r"""grep -q 'orion-ledger' "$WS/q204/guaranteed.txt" 2>/dev/null"""),
],
@ -504,6 +516,11 @@ chmod +x {WS}/q207/usage-nodes.sh {WS}/q207/usage-pods.sh
)
# ------------------------------------------------------------------ 208 ----
# Own namespace (project-vega): it used to share project-atlas with 203, whose "running pods"
# count then dropped by one when this drain deleted the bare pod. And the grader checks the pods
# that were on the node when it was seeded, not "nothing on the node at grading time" - after the
# required uncordon the scheduler is free to put new pods there (212's one-per-node spread always
# does), which failed this task no matter what the candidate did.
task(
id=208, domain="ARCH", title="Drain a node for maintenance", breaks="w2-drain",
task="""
@ -512,23 +529,28 @@ Node `{W2}` needs kernel maintenance.
1. Evict every non-DaemonSet Pod from it. Pods with `emptyDir` volumes must not stop you,
and neither must Pods that aren't managed by a controller.
2. Write the exact command you used to `{WS}/q208/drain.sh`.
3. Deployment `atlas-web` in Namespace `project-atlas` must still have all 3 replicas Available afterwards.
3. Deployment `vega-web` in Namespace `project-vega` must still have all 3 replicas Available afterwards.
4. When you're done, make the node schedulable again.
A Pod without a controller is **deleted** by a forced drain, not rescheduled. Before you drain,
look at what else runs on the node (`k get pods -A -o wide --field-selector spec.nodeName={W2}`) —
anything you created there for another task has to be recreated afterwards.
""",
setup=NS + r"""
mkns project-atlas
mkns project-vega
rm -rf "$WS/q208"; mkdir -p "$WS/q208"
$K -n project-atlas delete deploy atlas-web --ignore-not-found >/dev/null
$K -n project-vega delete deploy vega-web --ignore-not-found >/dev/null
$K -n project-vega delete pod vega-orphan --ignore-not-found --force --grace-period=0 --wait=true >/dev/null 2>&1
$K uncordon "$W2" >/dev/null 2>&1 || true
$K apply -f - >/dev/null <<EOF
apiVersion: apps/v1
kind: Deployment
metadata: {name: atlas-web, namespace: project-atlas}
metadata: {name: vega-web, namespace: project-vega}
spec:
replicas: 3
selector: {matchLabels: {app: atlas-web}}
selector: {matchLabels: {app: vega-web}}
template:
metadata: {labels: {app: atlas-web}}
metadata: {labels: {app: vega-web}}
spec:
containers:
- name: web
@ -536,33 +558,41 @@ spec:
resources: {requests: {cpu: 10m, memory: 16Mi}}
volumeMounts: [{name: scratch, mountPath: /scratch}]
volumes: [{name: scratch, emptyDir: {}}]
# at least one replica on the node being drained, so the emptyDir flag really matters
topologySpreadConstraints: [{maxSkew: 1, topologyKey: kubernetes.io/hostname, whenUnsatisfiable: ScheduleAnyway, labelSelector: {matchLabels: {app: vega-web}}}]
EOF
$K -n project-atlas rollout status deploy atlas-web --timeout=120s >/dev/null 2>&1
$K -n project-atlas run atlas-orphan --image=busybox:1.36 --restart=Never --overrides="{\"spec\":{\"nodeName\":\"$W2\"}}" -- sleep 86400 >/dev/null 2>&1
$K -n project-vega rollout status deploy vega-web --timeout=120s >/dev/null 2>&1
$K -n project-vega run vega-orphan --image=busybox:1.36 --restart=Never --overrides="{\"spec\":{\"nodeName\":\"$W2\"}}" -- sleep 86400 >/dev/null 2>&1
$K -n project-vega wait --for=condition=Ready pod/vega-orphan --timeout=90s >/dev/null 2>&1
# what the drain has to clear: every non-DaemonSet, non-mirror, non-kube-system pod on the node now
$K get pods -A -o json | python3 -c '
import sys,json
n=sys.argv[1]
for p in json.load(sys.stdin)["items"]:
m=p["metadata"]; o=(m.get("ownerReferences") or [{}])[0].get("kind")
if p["spec"].get("nodeName")==n and m["namespace"]!="kube-system" and o not in ("DaemonSet","Node"):
print(m["uid"])' "$W2" > "$WS/.q208-uids"
""",
checks=[
C("no non-DaemonSet workload pods remain on the node", 4, r"""
left=$($K get pods -A -o json 2>/dev/null | python3 -c '
import sys,json,os
n=os.environ["CKA_W2"]
print(sum(1 for p in json.load(sys.stdin)["items"]
if p["spec"].get("nodeName")==n and p["status"]["phase"]=="Running"
and (p["metadata"].get("ownerReferences") or [{}])[0].get("kind") not in ("DaemonSet",)
and p["metadata"]["namespace"] not in ("kube-system",)))')
eq "$left" 0"""),
C("every pod that was on the node when it was seeded has been evicted", 4, r"""
[ -s "$WS/.q208-uids" ] || exit 1
now=$($K get pods -A -o jsonpath='{range .items[*]}{.metadata.uid}{" "}{.spec.nodeName}{"\n"}{end}' 2>/dev/null)
while read -r u; do echo "$now" | grep -q "^$u $W2\$" && exit 1; done < "$WS/.q208-uids"
! $K -n project-vega get pod vega-orphan >/dev/null 2>&1"""),
C("drain.sh records a drain with --ignore-daemonsets and --delete-emptydir-data", 2, r"""
f="$WS/q208/drain.sh"; grep -q 'drain' "$f" 2>/dev/null && grep -q -- '--ignore-daemonsets' "$f" && grep -q -- '--delete-emptydir-data' "$f" """),
C("atlas-web still has 3 replicas Available", 2, r"""eq "$(j -n project-atlas deploy atlas-web '{.status.availableReplicas}')" 3"""),
C("vega-web still has 3 replicas Available", 2, r"""eq "$(j -n project-vega deploy vega-web '{.status.availableReplicas}')" 3"""),
C("node is schedulable again", 2, r"""[ "$(j node "$W2" '{.spec.unschedulable}')" != true ]"""),
],
solution="""
k get pods -A -o wide --field-selector spec.nodeName={W2} # what is about to be evicted or deleted
k drain {W2} --ignore-daemonsets --delete-emptydir-data --force
# --ignore-daemonsets DaemonSet pods can't be evicted, skip them
# --delete-emptydir-data required because atlas-web mounts an emptyDir
# --force required for the bare pod atlas-orphan (no controller)
# --delete-emptydir-data required because vega-web mounts an emptyDir (its data is lost)
# --force required for the bare pod vega-orphan - which is DELETED, not moved
echo 'kubectl drain {W2} --ignore-daemonsets --delete-emptydir-data --force' > {WS}/q208/drain.sh
k get pods -A -o wide --field-selector spec.nodeName={W2}
k -n project-atlas rollout status deploy atlas-web
k get pods -A -o wide --field-selector spec.nodeName={W2} # only DaemonSet pods left
k -n project-vega rollout status deploy vega-web
k uncordon {W2}
""",
)
@ -657,7 +687,7 @@ k -n project-atlas auth can-i get secret --as=system:serviceaccount:project-a
# ------------------------------------------------------------------ 211 ----
task(
id=211, domain="WORK", title="DaemonSet on every node",
id=211, domain="WORK", title="DaemonSet on every node", breaks=["w1-taint", "w1-block"],
task="""
In Namespace `project-orion` create a DaemonSet `node-collector`:
@ -697,7 +727,7 @@ k -n project-orion get ds node-collector -o wide
# ------------------------------------------------------------------ 212 ----
task(
id=212, domain="WORK", title="One Pod per node, on purpose",
id=212, domain="WORK", title="One Pod per node, on purpose", breaks=["w1-taint", "w1-block"],
task="""
In Namespace `project-orion` create Deployment `spread-check`:
@ -722,7 +752,7 @@ echo "$t" | grep -q '"topologyKey":"kubernetes.io/hostname"' && { echo "$t" | gr
C("exactly 3 Pods Running, one per node, 1 Pending", 3, r"""
r=$($K -n project-orion get pod -l id=spread-check --no-headers 2>/dev/null | grep -c Running)
p=$($K -n project-orion get pod -l id=spread-check --no-headers 2>/dev/null | grep -c Pending)
eq "$r" 3 && eq "$p" 1 && [ "$(j -n project-orion pod -l id=spread-check '{.items[*].spec.nodeName}' | tr ' ' '\n' | grep . | sort -u | wc -l)" = 3"""),
eq "$r" 3 && eq "$p" 1 && [ "$(j -n project-orion pod -l id=spread-check '{.items[*].spec.nodeName}' | tr ' ' '\n' | grep . | sort -u | wc -l)" = 3 ]"""),
C("pending.txt names the Pending pod", 2, r"""
want=$($K -n project-orion get pod -l id=spread-check --no-headers 2>/dev/null | awk '$3=="Pending"{print $1}')
[ -n "$want" ] && grep -qF "$want" "$WS/q212/pending.txt" 2>/dev/null"""),

View File

@ -233,14 +233,16 @@ k -n project-token get secret reporter-legacy -o yaml # the controller fills
)
# ------------------------------------------------------------------ 305 ----
# {W2}, not {W3}: 309 drains {W3} in this same set, and a drain without --force refuses to touch
# bare Pods - while one with --force deletes them, taking this task's answer with it.
task(
id=305, domain="WORK", title="Schedule without the scheduler",
task="""
In Namespace `project-orion`:
1. Create Pod `direct-assign` (image `busybox:1.36`, command `sleep 86400`) that runs on node
`{W3}` **without the scheduler being involved at all**.
2. Create Pod `scheduled` (same image and command) that lands on `{W3}` **through** the scheduler.
`{W2}` **without the scheduler being involved at all**.
2. Create Pod `scheduled` (same image and command) that lands on `{W2}` **through** the scheduler.
3. Write to `{WS}/q305/proof.txt` the value of the field you used for Pod 1, in the form
`field=<name>`, and on a second line `scheduler=<the schedulerName of the second Pod>`.
""",
@ -248,16 +250,26 @@ In Namespace `project-orion`:
mkns project-orion
rm -rf "$WS/q305"; mkdir -p "$WS/q305"
$K -n project-orion delete pod direct-assign scheduled --ignore-not-found --wait=false >/dev/null
$K label node "$W3" tier- >/dev/null 2>&1 || true
""",
# Who set spec.nodeName is recorded permanently in managedFields: a client that submits it
# (kubectl create/apply/run) owns f:nodeName, while a scheduler binding goes through the
# pods/binding subresource and leaves no managedFields entry for it (verified on 1.37). Events
# were the old signal, but plain `get events` has no SOURCE column and they expire after 1h.
checks=[
C("direct-assign Running on the right node with spec.nodeName set", 3, r"""
eq "$(j -n project-orion pod direct-assign '{.status.phase}')" Running && eq "$(j -n project-orion pod direct-assign '{.spec.nodeName}')" "$W3" """),
C("direct-assign was never scheduled (no Scheduled event from default-scheduler)", 3, r"""
! $K -n project-orion get events --field-selector involvedObject.name=direct-assign 2>/dev/null | grep -q 'default-scheduler'"""),
C("scheduled Running on the same node via the default scheduler", 3, r"""
eq "$(j -n project-orion pod scheduled '{.status.phase}')" Running && eq "$(j -n project-orion pod scheduled '{.spec.nodeName}')" "$W3" \
&& $K -n project-orion get events --field-selector involvedObject.name=scheduled 2>/dev/null | grep -q 'default-scheduler'"""),
C("direct-assign Running on {W2}", 3, r"""
eq "$(j -n project-orion pod direct-assign '{.status.phase}')" Running && eq "$(j -n project-orion pod direct-assign '{.spec.nodeName}')" "$W2" """),
C("direct-assign bypassed the scheduler (spec.nodeName came from your manifest)", 3, r"""
$K -n project-orion get pod direct-assign -o json --show-managed-fields 2>/dev/null | python3 -c '
import sys,json
mf=json.load(sys.stdin)["metadata"].get("managedFields",[])
sys.exit(0 if any("f:nodeName" in json.dumps(m.get("fieldsV1",{})) for m in mf if m.get("subresource","")!="status") else 1)'"""),
C("scheduled Running on {W2}, placed there by default-scheduler (no nodeName in its manifest)", 3, r"""
eq "$(j -n project-orion pod scheduled '{.status.phase}')" Running && eq "$(j -n project-orion pod scheduled '{.spec.nodeName}')" "$W2" \
&& eq "$(j -n project-orion pod scheduled '{.spec.schedulerName}')" default-scheduler || exit 1
$K -n project-orion get pod scheduled -o json --show-managed-fields 2>/dev/null | python3 -c '
import sys,json
mf=json.load(sys.stdin)["metadata"].get("managedFields",[])
sys.exit(1 if any("f:nodeName" in json.dumps(m.get("fieldsV1",{})) for m in mf) else 0)'"""),
C("proof.txt names nodeName and default-scheduler", 2, r"""
grep -q 'field= *nodeName' "$WS/q305/proof.txt" 2>/dev/null && grep -q 'scheduler= *default-scheduler' "$WS/q305/proof.txt" """),
],
@ -267,7 +279,7 @@ apiVersion: v1
kind: Pod
metadata: {name: direct-assign, namespace: project-orion}
spec:
nodeName: {W3} # bypasses the scheduler entirely - the kubelet just picks it up
nodeName: {W2} # bypasses the scheduler entirely - the kubelet just picks it up
containers: [{name: c, image: busybox:1.36, command: [sleep, "86400"]}]
EOF
cat <<EOF | k apply -f -
@ -275,12 +287,15 @@ apiVersion: v1
kind: Pod
metadata: {name: scheduled, namespace: project-orion}
spec:
nodeSelector: {kubernetes.io/hostname: {W3}} # the scheduler still makes the decision
nodeSelector: {kubernetes.io/hostname: {W2}} # the scheduler still makes the decision
containers: [{name: c, image: busybox:1.36, command: [sleep, "86400"]}]
EOF
k -n project-orion get events --field-selector involvedObject.name=direct-assign # no Scheduled event
k -n project-orion get events --field-selector involvedObject.name=scheduled # Scheduled by default-scheduler
k -n project-orion describe pod scheduled | grep -A3 Events # Scheduled default-scheduler Successfully assigned...
k -n project-orion describe pod direct-assign | grep -A3 Events # no Scheduled event at all
printf 'field=nodeName\\nscheduler=default-scheduler\\n' > {WS}/q305/proof.txt
# `k get events` hides who reported an event; -o wide (or describe) shows the SOURCE column.
# Events expire after an hour, so for lasting proof look at who owns spec.nodeName:
# k -n project-orion get pod direct-assign -o yaml --show-managed-fields | grep -B2 -A2 nodeName
""",
)
@ -294,8 +309,13 @@ In Namespace `project-orbit`:
`volumeBindingMode: WaitForFirstConsumer`, and allowing volume expansion.
2. Create PVC `growing` (1Gi, RWO) using that class and Pod `grower` (busybox:1.36, `sleep 86400`)
mounting it at `/data`, so the claim binds.
3. Then grow the claim to **2Gi** without deleting it, and write the claim's
`.status.capacity.storage` after the resize to `{WS}/q306/capacity.txt`.
3. Then grow the claim to **2Gi** without deleting it, and write both sizes the claim reports
afterwards to `{WS}/q306/capacity.txt`, one per line:
`requested=<.spec.resources.requests.storage>` and `status=<.status.capacity.storage>`.
(The local-path provisioner has no volume resizer, so on this cluster the status will not catch up
with the request — that gap is what step 3 asks you to observe. With a CSI driver that supports
expansion, the status would follow within seconds.)
""",
setup=NS + r"""
mkns project-orbit
@ -312,10 +332,17 @@ eq "$($s -o jsonpath='{.provisioner}' 2>/dev/null)" rancher.io/local-path && eq
C("PVC growing Bound on that class with grower Running", 3, r"""
eq "$(j -n project-orbit pvc growing '{.spec.storageClassName}')" expandable && eq "$(j -n project-orbit pvc growing '{.status.phase}')" Bound \
&& eq "$(j -n project-orbit pod grower '{.status.phase}')" Running"""),
C("claim requests 2Gi and reports at least 2Gi capacity", 4, r"""
# local-path cannot resize (verified: ExternalExpanding, capacity never moves), so a bound PV
# that is still 1Gi under a claim requesting 2Gi is exactly what "grown in place" looks like
# here. A claim deleted and recreated at 2Gi would have been provisioned a 2Gi volume instead.
C("claim grown in place: requests 2Gi on the original 1Gi volume", 4, r"""
eq "$(j -n project-orbit pvc growing '{.spec.resources.requests.storage}')" 2Gi || exit 1
for i in $(seq 1 12); do c=$(j -n project-orbit pvc growing '{.status.capacity.storage}'); [ "$c" = 2Gi ] && exit 0; sleep 5; done; exit 1"""),
C("capacity.txt records the new capacity", 1, r"""grep -q '2Gi' "$WS/q306/capacity.txt" 2>/dev/null"""),
pv=$(j -n project-orbit pvc growing '{.spec.volumeName}'); [ -n "$pv" ] || exit 1
eq "$(j pv "$pv" '{.spec.capacity.storage}')" 1Gi"""),
C("capacity.txt records requested and status sizes as the cluster reports them", 1, r"""
f="$WS/q306/capacity.txt"
eq "$(sed -n 's/^ *requested *= *//p' "$f" 2>/dev/null | tr -d '[:space:]')" 2Gi \
&& eq "$(sed -n 's/^ *status *= *//p' "$f" | tr -d '[:space:]')" "$(j -n project-orbit pvc growing '{.status.capacity.storage}')" """),
],
solution="""
cat <<EOF | k apply -f -
@ -342,15 +369,19 @@ spec:
containers: [{name: c, image: busybox:1.36, command: [sleep, "86400"], volumeMounts: [{name: d, mountPath: /data}]}]
EOF
k -n project-orbit patch pvc growing -p '{"spec":{"resources":{"requests":{"storage":"2Gi"}}}}'
k -n project-orbit get pvc growing -w # spec 2Gi first, then status.capacity follows
k -n project-orbit get pvc growing -o jsonpath='{.status.capacity.storage}' > {WS}/q306/capacity.txt
# Shrinking is never allowed. Without allowVolumeExpansion the patch is rejected outright.
k -n project-orbit describe pvc growing # event ExternalExpanding: waiting for an external controller
k -n project-orbit get pvc growing -o jsonpath='requested={.spec.resources.requests.storage}{"\\n"}status={.status.capacity.storage}{"\\n"}' > {WS}/q306/capacity.txt
cat {WS}/q306/capacity.txt # requested=2Gi status=1Gi
# allowVolumeExpansion is the gate: without it the API rejects the patch outright, and shrinking is
# never allowed. Accepting the request is all the API does - a resizer (the CSI driver's
# external-resizer sidecar) then grows the volume and the kubelet the filesystem, and only then does
# .status.capacity change. local-path has no resizer, so here the status stays at 1Gi forever.
""",
)
# ------------------------------------------------------------------ 307 ----
task(
id=307, domain="TSHOOT", title="Node accepts nothing new", breaks="w1-block",
id=307, domain="TSHOOT", title="Node accepts nothing new", breaks=["w1-block", "w1-taint"],
task="""
Node `{W1}` is `Ready` and its kubelet is healthy, but no new Pod will schedule onto it.
@ -451,6 +482,10 @@ at all times.
2. Drain `{W3}` (DaemonSet Pods aside), then make it schedulable again.
3. Write the value of the PDB's `.status.disruptionsAllowed` after your change to
`{WS}/q309/allowed.txt`.
Look before you drain: `k get pods -A -o wide --field-selector spec.nodeName={W3}`. A Pod without a
controller blocks a plain drain, and `--force` deletes it for good — if it belongs to another
task, recreate it once the node is back.
""",
setup=NS + r"""
mkns project-atlas
@ -498,6 +533,7 @@ k -n project-atlas get pdb atlas-api
# a drain would hang forever waiting for the eviction API.
k -n project-atlas patch pdb atlas-api -p '{"spec":{"minAvailable":1}}'
k -n project-atlas get pdb atlas-api # ALLOWED DISRUPTIONS: 1
k get pods -A -o wide --field-selector spec.nodeName={W3} # bare pods here? they need --force and are then gone
k drain {W3} --ignore-daemonsets --delete-emptydir-data
k -n project-atlas rollout status deploy atlas-api
k uncordon {W3}
@ -583,24 +619,33 @@ Then write the number of **distinct** images in use across those Deployments to
""",
setup=r"""rm -rf "$WS/q311"; mkdir -p "$WS/q311" """,
checks=[
C("images.txt matches the cluster exactly", 5, r"""
want=$($K get deploy -A -o json 2>/dev/null | python3 -c '
import sys,json
out=[]
# Graded against the Deployments that existed when the file was written (its mtime): other
# set-B tasks create Deployments later (303's cidr-demo), and an exact cluster-wide
# comparison at submit time failed a correct answer whenever one of them came after this.
C("images.txt matches the cluster as it was when you wrote it", 5, r"""
f="$WS/q311/images.txt"; [ -s "$f" ] || exit 1
$K get deploy -A -o json 2>/dev/null | python3 -c '
import sys,json,os,datetime
f=sys.argv[1]; T=os.path.getmtime(f)
def ts(d): return datetime.datetime.strptime(d["metadata"]["creationTimestamp"],"%Y-%m-%dT%H:%M:%SZ").replace(tzinfo=datetime.timezone.utc).timestamp()
want=set(); later=set()
for d in json.load(sys.stdin)["items"]:
s=d["spec"]["template"]["spec"]
imgs=[c["image"] for c in s.get("initContainers",[])]+[c["image"] for c in s["containers"]]
out.append("%s/%s %s"%(d["metadata"]["namespace"],d["metadata"]["name"],",".join(imgs)))
print("\n".join(sorted(out)))')
[ "$(grep . "$WS/q311/images.txt" 2>/dev/null | sort)" = "$want" ]"""),
key="%s/%s"%(d["metadata"]["namespace"],d["metadata"]["name"]); s=d["spec"]["template"]["spec"]
if ts(d) > T: later.add(key); continue
want.add("%s %s"%(key,",".join([c["image"] for c in s.get("initContainers",[])]+[c["image"] for c in s["containers"]])))
got={l.strip() for l in open(f) if l.strip()}
extra={g for g in got-want if g.split(" ")[0] in {w.split(" ")[0] for w in want}} # a real deployment, wrong images
sys.exit(0 if want<=got and not extra else 1)' "$f" """),
C("distinct.txt has the right count", 3, r"""
f="$WS/q311/images.txt"; [ -s "$f" ] || exit 1
want=$($K get deploy -A -o json 2>/dev/null | python3 -c '
import sys,json
s=set()
import sys,json,os,datetime
T=os.path.getmtime(sys.argv[1]); s=set()
for d in json.load(sys.stdin)["items"]:
if datetime.datetime.strptime(d["metadata"]["creationTimestamp"],"%Y-%m-%dT%H:%M:%SZ").replace(tzinfo=datetime.timezone.utc).timestamp() > T: continue
sp=d["spec"]["template"]["spec"]
for c in sp.get("initContainers",[])+sp["containers"]: s.add(c["image"])
print(len(s))')
print(len(s))' "$f")
eq "$(tr -dc '0-9' < "$WS/q311/distinct.txt" 2>/dev/null)" "$want" """),
],
solution="""
@ -784,8 +829,10 @@ for i in $(seq 1 12); do
[ "$(j -n default pod "edge-cache-$W3" '{.status.phase}')" = Running ] && [ "$(j -n default pod "edge-cache-$W3" '{.spec.containers[0].image}')" = nginx:1.27 ] && exit 0
sleep 5
done; exit 1"""),
# -h: with two file operands GNU grep prefixes every line with the file name, which made
# awk's $2 the literal word "path:" and this check unpassable.
C("the hostPath the manifest references now exists on the node", 3, r"""
p=$(nx "$W3" 'grep -A2 hostPath /etc/kubernetes/manifests/edge-cache.yaml /etc/kubernetes/static/edge-cache.yaml 2>/dev/null | grep "path:" | head -1' | awk '{print $2}')
p=$(nx "$W3" 'grep -h -A2 hostPath /etc/kubernetes/manifests/edge-cache.yaml /etc/kubernetes/static/edge-cache.yaml 2>/dev/null | grep "path:" | head -1' | awk '{print $2}')
[ -n "$p" ] && nx "$W3" "test -d $p" """),
C("reason.txt mentions the missing host directory", 2, r"""grep -Eqi 'director|hostpath|/opt/edge-cache|not exist|missing' "$WS/q314/reason.txt" 2>/dev/null"""),
],
@ -916,11 +963,16 @@ EOF
)
# ------------------------------------------------------------------ 317 ----
# The sabotage switches off ONE controller. It used to point --cluster-signing-cert-file at a
# missing file instead, but a controller that fails to start is fatal to kube-controller-manager
# (verified on 1.37: "Error building controllers" -> exit), so the whole KCM crash-looped and every
# other set-B task that needs a controller (Deployments, Jobs, EndpointSlices, PV phases, PDB
# status, token Secrets) silently stopped working until this task - the last one - was fixed.
task(
id=317, domain="TSHOOT", title="Certificates stop being issued", breaks="controller-manager",
task="""
Certificate signing requests are approved but never get a certificate: `kubectl get csr` shows
them `Approved` with an empty `CONDITION`-issued certificate forever.
them `Approved` and never `Approved,Issued`. Everything else in the cluster works.
Find the responsible control-plane component, repair it on the control-plane node, and prove it
works: a CertificateSigningRequest named `issue-probe` (CN `probe`, signer
@ -931,14 +983,14 @@ receive a certificate.
$K delete csr issue-probe --ignore-not-found >/dev/null 2>&1
nx "$CP" '
f=/etc/kubernetes/manifests/kube-controller-manager.yaml
grep -q "cluster-signing-cert-file=/etc/kubernetes/pki/ca.crt" $f && sed -i "s|--cluster-signing-cert-file=/etc/kubernetes/pki/ca.crt|--cluster-signing-cert-file=/etc/kubernetes/pki/ca-signing.crt|" $f
grep -q -- "-certificatesigningrequest-signing-controller" $f || sed -i "s|^\( *- --controllers=.*\)$|\1,-certificatesigningrequest-signing-controller|" $f
'
""",
checks=[
C("kube-controller-manager points at a signing certificate that exists", 4, r"""
p=$(nx "$CP" 'grep -o "\-\-cluster-signing-cert-file=[^ ]*" /etc/kubernetes/manifests/kube-controller-manager.yaml' | cut -d= -f2)
[ -n "$p" ] && nx "$CP" "test -f $p" """),
C("kube-controller-manager pod is Running", 3, r"""
C("kube-controller-manager runs the CSR signing controller again", 4, r"""
c=$(nx "$CP" 'grep -o -- "--controllers=[^ ]*" /etc/kubernetes/manifests/kube-controller-manager.yaml')
[ -n "$c" ] && ! echo "$c" | grep -Eq -- '-(certificatesigningrequest-signing-controller|csrsigning)(,|$)'"""),
C("kube-controller-manager still Running after your edit", 3, r"""
for i in $(seq 1 12); do [ "$(j -n kube-system pod -l component=kube-controller-manager '{.items[0].status.phase}')" = Running ] && exit 0; sleep 5; done; exit 1"""),
C("an approved CSR now receives a certificate", 5, r"""
$K get csr issue-probe >/dev/null 2>&1 || exit 1
@ -946,15 +998,18 @@ $K get csr issue-probe -o jsonpath='{.status.conditions[*].type}' | grep -q Appr
for i in $(seq 1 15); do [ -n "$(j csr issue-probe '{.status.certificate}')" ] && exit 0; sleep 4; done; exit 1"""),
],
solution="""
k get csr # Approved but never Issued
k -n kube-system logs kube-controller-manager-{CP} | tail -20
# ... unable to load client CA file .../ca-signing.crt: no such file or directory
k get csr # Approved - but never Approved,Issued
# Approving is the API's job; ISSUING is a controller's job, and controllers live in
# kube-controller-manager. So: is it running, and is the right controller in it running?
k -n kube-system get pod -l component=kube-controller-manager # Running - so not a crash
k -n kube-system logs kube-controller-manager-{CP} | grep -i disabled
# "Warning: controller is disabled" controller="certificatesigningrequest-signing-controller"
{SHELL_CP}
grep signing /etc/kubernetes/manifests/kube-controller-manager.yaml
ls /etc/kubernetes/pki/ca.crt # the real file
vi /etc/kubernetes/manifests/kube-controller-manager.yaml
# --cluster-signing-cert-file=/etc/kubernetes/pki/ca.crt
grep -- --controllers /etc/kubernetes/manifests/kube-controller-manager.yaml
# --controllers=*,bootstrapsigner,tokencleaner,-certificatesigningrequest-signing-controller
vi /etc/kubernetes/manifests/kube-controller-manager.yaml # remove ,-certificatesigningrequest-signing-controller
exit
k -n kube-system get pod -l component=kube-controller-manager -w # the kubelet restarts the static pod
openssl genrsa -out probe.key 2048
openssl req -new -key probe.key -subj "/CN=probe" -out probe.csr
cat <<EOF | k apply -f -
@ -964,9 +1019,9 @@ metadata: {name: issue-probe}
spec: {request: $(base64 -w0 probe.csr), signerName: kubernetes.io/kube-apiserver-client, usages: [client auth]}
EOF
k certificate approve issue-probe
k get csr issue-probe # certificate now populated
# The csrsigning controller lives in kube-controller-manager: approval is the API's job,
# but issuing the certificate is the controller's.
k get csr issue-probe # Approved,Issued
# `--controllers`: '*' = every default-on controller, 'name' adds one, '-name' removes one.
# kubeadm itself adds bootstrapsigner and tokencleaner, which are off by default.
""",
)

View File

@ -202,8 +202,11 @@ i=$(j -n driver-onboard deploy onboard-app '{.spec.template.spec.containers[0].i
echo "$i" | grep -q '^nginx:' && ! echo "$i" | grep -q ngnix"""),
C("liveness probe targets port 80", 2, r"""
eq "$(j -n driver-onboard deploy onboard-app '{.spec.template.spec.containers[0].livenessProbe.httpGet.port}')" 80"""),
C("readiness probe uses a path nginx actually serves", 2, r"""
eq "$(j -n driver-onboard deploy onboard-app '{.spec.template.spec.containers[0].readinessProbe.httpGet.path}')" /"""),
# any readiness probe nginx can satisfy counts (/, /index.html, a tcpSocket on 80...); the
# "all replicas Ready" check below proves it actually passes
C("readiness probe no longer targets the missing /healthz", 2, r"""
rp=$(j -n driver-onboard deploy onboard-app '{.spec.template.spec.containers[0].readinessProbe}')
[ -n "$rp" ] && ! echo "$rp" | grep -q '/healthz'"""),
C("all replicas Running and Ready", 2, r"""
for i in $(seq 1 24); do [ "$(j -n driver-onboard deploy onboard-app '{.status.readyReplicas}')" = 2 ] && exit 0; sleep 5; done; exit 1"""),
],
@ -637,7 +640,7 @@ eq "$(j -n trip-tracker deploy trip-logger '{.spec.template.spec.initContainers[
C("it uses busybox:1.36 and targets /config/app-config.json", 3, r"""
c='{.spec.template.spec.initContainers[?(@.name=="config-init")]'
eq "$(j -n trip-tracker deploy trip-logger "$c.image}")" busybox:1.36 \
&& j -n trip-tracker deploy trip-logger "$c.command[*]}" | grep -q '/config/app-config.json'"""),
&& { j -n trip-tracker deploy trip-logger "$c.command[*]}"; j -n trip-tracker deploy trip-logger "$c.args[*]}"; } | grep -q '/config/app-config.json'"""),
C("emptyDir config-volume mounted at /config in both containers", 2, r"""
d="$K -n trip-tracker get deploy trip-logger"
[ -n "$($d -o jsonpath='{.spec.template.spec.volumes[?(@.name=="config-volume")].emptyDir}')" ] \
@ -678,8 +681,8 @@ Before rolling out NetworkPolicies, confirm what the cluster's CNI can actually
3. In Namespace `zero-trust`, create NetworkPolicy `default-deny-all` that denies **all** ingress
and egress for every Pod in the Namespace, with no allow rules of any kind.
Note which CNI is installed here before answering step 2 — several popular CNIs accept
NetworkPolicy objects happily and never enforce them.
Answer step 2 from evidence, not memory — several popular CNIs accept NetworkPolicy objects
happily and never enforce them, and a CNI's behaviour can change between releases.
""",
setup=NS + r"""
mkns zero-trust
@ -691,8 +694,17 @@ $K -n zero-trust delete netpol default-deny-all --ignore-not-found >/dev/null 2>
want=$($K -n kube-system get ds -o name 2>/dev/null | sed 's|.*/||' | grep -Ei 'kindnet|calico|cilium|flannel|weave' | head -1)
[ -n "$want" ] || want=kindnet
grep -qi "$want" "$WS/q410/cni.txt" 2>/dev/null"""),
C("enforced.txt correctly says this CNI does not enforce NetworkPolicy", 2, r"""
grep -qix 'no' "$WS/q410/enforced.txt" 2>/dev/null"""),
# The right answer depends on the build, so derive it: recent kindnet embeds a policy engine
# (nftables table `kindnet-network-policies` on every node - verified enforcing on this
# cluster), calico/cilium/weave enforce, flannel does not.
C("enforced.txt gives the right answer for this cluster's CNI", 2, r"""
ds=$($K -n kube-system get ds -o name 2>/dev/null | sed 's|.*/||' | tr '\n' ' ')
case "$ds" in
*calico*|*cilium*|*weave*|*antrea*) want=yes ;;
*kindnet*) nx "$W1" 'nft list table inet kindnet-network-policies >/dev/null 2>&1' </dev/null && want=yes || want=no ;;
*) want=no ;;
esac
grep -qix "$want" "$WS/q410/enforced.txt" 2>/dev/null"""),
C("default-deny-all selects every pod in both directions", 4, r"""
n="$K -n zero-trust get netpol default-deny-all"
eq "$($n -o jsonpath='{.spec.podSelector}' 2>/dev/null)" '{}' \
@ -702,7 +714,10 @@ eq "$($n -o jsonpath='{.spec.podSelector}' 2>/dev/null)" '{}' \
],
solution="""
k -n kube-system get ds # kindnet -> {WS}/q410/cni.txt
echo no > {WS}/q410/enforced.txt # kindnet has no policy engine
# Don't answer from memory - prove it. Two pods, one deny policy, one wget:
k -n zero-trust run srv --image=nginx:1.27 --labels=app=srv; k -n zero-trust expose pod srv --port 80
k -n zero-trust run cli --image=busybox:1.36 -- sleep 3600
k -n zero-trust exec cli -- wget -qO- -T3 http://srv >/dev/null && echo reachable
cat <<EOF | k apply -f -
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
@ -711,10 +726,14 @@ spec:
podSelector: {}
policyTypes: [Ingress, Egress]
EOF
k -n zero-trust exec cli -- wget -qO- -T3 http://srv >/dev/null || echo BLOCKED # -> BLOCKED here
echo yes > {WS}/q410/enforced.txt
k -n zero-trust delete pod srv cli; k -n zero-trust delete svc srv # leave only the policy
# podSelector {} means every pod in the namespace; naming both policyTypes with no ingress/egress
# blocks listed means "deny everything". kindnet (like flannel) has no policy engine, so the API
# server stores these objects and nothing enforces them - a real trap in a hardening review.
# Calico and Cilium do enforce; the exam expects you to know which is which.
# blocks listed means "deny everything". Recent kindnet builds (this cluster's included) embed a
# policy engine - look for `nft list table inet kindnet-network-policies` on a node. Flannel stores
# NetworkPolicies and enforces nothing; Calico and Cilium enforce. A policy that "works" on a
# non-enforcing CNI is the classic hardening-review trap.
""",
)
@ -1066,7 +1085,8 @@ In Namespace `safety-monitor`, create NetworkPolicy `safety-api-policy` for the
- **egress**: DNS on port 53, both UDP and TCP, to any destination,
- everything else denied in both directions.
(kindnet does not enforce policies; you are graded on the object.)
(This cluster's CNI enforces policies, so a wrong policy really breaks traffic — but you are graded on
the object.)
""",
setup=NS + r"""
mkns safety-monitor
@ -1195,8 +1215,13 @@ $K -n booking-tls rollout status deploy booking-nginx --timeout=120s >/dev/null
C("the ConfigMap allows TLSv1.3 only", 3, r"""
c=$(j -n booking-tls cm booking-nginx-config '{.data.default\.conf}')
echo "$c" | grep -q 'ssl_protocols.*TLSv1\.3' && ! echo "$c" | grep -q 'TLSv1\.2'"""),
# rollout restart (restartedAt annotation) or deleting the pods - either way every running
# pod must be younger than the last write to the ConfigMap
C("the pods were restarted to pick the change up", 2, r"""
j -n booking-tls deploy booking-nginx '{.spec.template.metadata.annotations}' | grep -q 'restartedAt'"""),
j -n booking-tls deploy booking-nginx '{.spec.template.metadata.annotations}' | grep -q 'restartedAt' && exit 0
cm_t=$($K -n booking-tls get cm booking-nginx-config -o json --show-managed-fields 2>/dev/null | python3 -c 'import sys,json; print(max(m.get("time","") for m in json.load(sys.stdin)["metadata"].get("managedFields",[])))')
ts=$(j -n booking-tls pod -l app=booking-nginx '{.items[*].metadata.creationTimestamp}'); [ -n "$cm_t" ] && [ -n "$ts" ] || exit 1
for t in $ts; do [[ "$t" > "$cm_t" ]] || exit 1; done"""),
C("ssl-line.txt records the directive", 2, r"""
f="$WS/q417/ssl-line.txt"
grep -q 'ssl_protocols' "$f" 2>/dev/null && grep -q 'TLSv1\.3' "$f" && ! grep -q 'TLSv1\.2' "$f" """),

988
tracks/cka/bank_sim_d.py Normal file
View File

@ -0,0 +1,988 @@
"""Simulator Set D - 17 tasks, fixed set, killer.sh-style.
Original scenarios, not copies of any commercial simulator. They exercise the techniques the
killer.sh CKA environment is known for that sets A-C do not already cover: saying how every
control-plane piece is run, cluster facts to an exact file, readiness gated on a Service,
`--sort-by` scripts, event forensics around kube-proxy with crictl, API-resource and RBAC census,
a kubelet whose systemd unit points at the wrong binary, Secrets mounted read-only and renamed into
env vars, a static Pod on the control plane behind a Service, kubelet client/serving certificate
forensics, control-plane-only scheduling, stopping the scheduler and scheduling by hand, Helm
release management (including a release stuck in pending-install), StatefulSet storage, the
namespaceSelector-AND-podSelector NetworkPolicy trap (enforced here - kindnet enforces), a
dnsPolicy that silently breaks name resolution, and a claim Pending on an access mode.
Every value a grader compares against is derived from the live cluster, never hardcoded, and every
task was seeded, solved and graded on an isolated kind cluster before it went in.
Run with: ./exam sim d
"""
from tracks.cka.common import C, NS
Q = []
CTX = "Use context: `kubectl config use-context %{ctx}`\n\n"
def task(**kw):
kw.setdefault("difficulty", "hard")
kw.setdefault("params", lambda r: {"ctx": "k8s-c1-H"})
kw["task"] = CTX + kw["task"].lstrip("\n")
kw["set"] = "d"
Q.append(kw)
# ------------------------------------------------------------------ 501 ----
task(
id=501, domain="ARCH", title="How the control plane is run",
task="""
The operations handbook has to say how each part of this cluster's control plane is run. Write
`{WS}/q501/controlplane.txt` with exactly these six lines, each value one of `process`,
`static-pod`, `pod`, `deployment` or `daemonset`:
```
kubelet: <value>
kube-apiserver: <value>
kube-scheduler: <value>
kube-controller-manager: <value>
etcd: <value>
dns: <value> <name of that object>
```
`dns` is whatever serves cluster DNS, for example `dns: daemonset some-name`.
""",
setup=r"""rm -rf "$WS/q501"; mkdir -p "$WS/q501" """,
checks=[
C("kubelet is a process", 1, r"""
f="$WS/q501/controlplane.txt"; [ -s "$f" ] || exit 1
want=process; [ "$(nx "$CP" 'systemctl is-active kubelet' </dev/null)" = active ] || want=unknown
eq "$(sed -n 's/^ *kubelet *: *//p' "$f" | head -1 | tr -d '[:space:]' | tr A-Z a-z)" "$want" """),
C("apiserver, scheduler, controller-manager and etcd are static pods", 3, r"""
f="$WS/q501/controlplane.txt"
# a kubeadm control plane: the manifests live in the kubelet's staticPodPath on the node
nx "$CP" 'ls /etc/kubernetes/manifests/kube-apiserver.yaml' </dev/null >/dev/null 2>&1 || exit 1
for c in kube-apiserver kube-scheduler kube-controller-manager etcd; do
eq "$(sed -n "s/^ *$c *: *//p" "$f" | head -1 | tr -d '[:space:]' | tr A-Z a-z)" static-pod || exit 1
done"""),
C("dns names the right kind of object and its name", 2, r"""
f="$WS/q501/controlplane.txt"
sel=$(j -n kube-system svc kube-dns '{.spec.selector.k8s-app}')
own=$(j -n kube-system pod -l k8s-app=$sel '{.items[0].metadata.ownerReferences[0].kind}')
name=$(j -n kube-system pod -l k8s-app=$sel '{.items[0].metadata.ownerReferences[0].name}')
if [ "$own" = ReplicaSet ]; then want="deployment $(j -n kube-system rs "$name" '{.metadata.ownerReferences[0].name}')"
else want="$(echo "$own" | tr A-Z a-z) $name"; fi
eq "$(sed -n 's/^ *dns *: *//p' "$f" | head -1 | tr -s ' ' | tr A-Z a-z | sed 's/ *$//')" "$want" """),
],
solution="""
{SHELL_CP}
systemctl status kubelet | head -3 # a systemd service: a plain process on the node
ls /etc/kubernetes/manifests # etcd, kube-apiserver, kube-controller-manager, kube-scheduler
exit
k -n kube-system get pod -o wide # static pods carry the node name as a suffix...
k -n kube-system get pod kube-apiserver-{CP} -o jsonpath='{.metadata.ownerReferences[0].kind}{"\\n"}' # ...and are owned by the Node
k -n kube-system get deploy,ds # coredns is a Deployment (kube-proxy, kindnet are DaemonSets)
cat > {WS}/q501/controlplane.txt <<'EOF'
kubelet: process
kube-apiserver: static-pod
kube-scheduler: static-pod
kube-controller-manager: static-pod
etcd: static-pod
dns: deployment coredns
EOF
# The Service is called kube-dns for historical reasons; what actually answers is the coredns
# Deployment. Follow the chain svc selector -> pod -> ownerReferences when unsure.
""",
)
# ------------------------------------------------------------------ 502 ----
task(
id=502, domain="TSHOOT", title="Cluster facts to an exact file",
task="""
Write `{WS}/q502/cluster-info.txt` — answers only, exactly this format:
```
1: <number of control-plane nodes>
2: <number of worker nodes>
3: <the Service CIDR>
4: <name of the CNI plugin>, <full path of its configuration file on the nodes>
5: <the suffix the names of static Pods running on node {W3} end with>
```
""",
setup=r"""rm -rf "$WS/q502"; mkdir -p "$WS/q502" """,
checks=[
C("node counts (lines 1 and 2)", 2, r"""
f="$WS/q502/cluster-info.txt"; [ -s "$f" ] || exit 1
cp=$($K get nodes -l node-role.kubernetes.io/control-plane --no-headers 2>/dev/null | wc -l); all=$($K get nodes --no-headers | wc -l)
eq "$(sed -n 's/^ *1 *: *//p' "$f" | tr -d '[:space:]')" "$cp" && eq "$(sed -n 's/^ *2 *: *//p' "$f" | tr -d '[:space:]')" "$((all - cp))" """),
C("Service CIDR (line 3)", 2, r"""
sc=$(j -n kube-system pod -l component=kube-apiserver '{.items[0].spec.containers[0].command}' | tr ',' '\n' | grep -o 'service-cluster-ip-range=[^"]*' | cut -d= -f2)
[ -n "$sc" ] && eq "$(sed -n 's/^ *3 *: *//p' "$WS/q502/cluster-info.txt" | tr -d '[:space:]')" "$sc" """),
C("CNI plugin and its config file (line 4)", 2, r"""
l=$(sed -n 's/^ *4 *: *//p' "$WS/q502/cluster-info.txt" | tr -d '[:space:]')
cni=$($K -n kube-system get ds -o name | sed 's|.*/||' | grep -Ei 'kindnet|calico|cilium|flannel|weave' | head -1)
cfg=$(nx "$W1" 'ls /etc/cni/net.d/*.conflist /etc/cni/net.d/*.conf 2>/dev/null | head -1' </dev/null)
[ -n "$cni" ] && [ -n "$cfg" ] && echo "$l" | grep -qi "$cni" && echo "$l" | grep -qF "$cfg" """),
C("static Pod name suffix on {W3} (line 5)", 1, r"""
v=$(sed -n 's/^ *5 *: *//p' "$WS/q502/cluster-info.txt" | tr -d '[:space:]'); [ "$v" = "-$W3" ] || [ "$v" = "$W3" ]"""),
],
solution="""
k get nodes # ROLES shows control-plane; count the rest
k -n kube-system get pod -l component=kube-apiserver -o yaml | grep service-cluster-ip-range
k -n kube-system get ds # kindnet
{SHELL_W1}
ls /etc/cni/net.d/ # 10-kindnet.conflist
exit
cat > {WS}/q502/cluster-info.txt <<'EOF'
1: 1
2: 3
3: 10.96.0.0/16
4: kindnet, /etc/cni/net.d/10-kindnet.conflist
5: -{W3}
EOF
# A static Pod's mirror is named <pod>-<node name>, e.g. kube-apiserver-{CP}. Values come from THIS
# cluster - check each one instead of trusting the ones shown here.
""",
)
# ------------------------------------------------------------------ 503 ----
task(
id=503, domain="NET", title="Ready only when its dependency answers",
task="""
In Namespace `oslo-web`:
1. Create Pod `gatekeeper` (image `busybox:1.36`, command `sleep 86400`) with an **exec readiness
probe** that runs `wget -T2 -qO- http://ledger-api:8080`, initial delay 5 seconds, period 5
seconds. For now it must stay **not ready** — confirm that it does.
2. Deployment `ledger` serves HTTP on port 80, but nothing exposes it. Create the Service
`ledger-api` (ClusterIP, port **8080**) for it, so that `gatekeeper` turns Ready without you
touching the Pod again.
""",
setup=NS + r"""
mkns oslo-web
$K -n oslo-web delete pod gatekeeper --ignore-not-found --force --grace-period=0 >/dev/null 2>&1
$K -n oslo-web delete svc ledger-api --ignore-not-found >/dev/null
$K -n oslo-web create deploy ledger --image=nginx:1.27 --port=80 --dry-run=client -o yaml | $K apply -f - >/dev/null
$K -n oslo-web rollout status deploy ledger --timeout=120s >/dev/null 2>&1
""",
checks=[
C("gatekeeper has the exec readiness probe (delay 5, period 5)", 2, r"""
p='{.spec.containers[0].readinessProbe'
j -n oslo-web pod gatekeeper "$p.exec.command}" | grep -q 'ledger-api:8080' \
&& eq "$(j -n oslo-web pod gatekeeper "$p.initialDelaySeconds}")" 5 && eq "$(j -n oslo-web pod gatekeeper "$p.periodSeconds}")" 5"""),
C("Service ledger-api: 8080 -> 80 with ready endpoints", 2, r"""
eq "$(j -n oslo-web svc ledger-api '{.spec.ports[0].port}')" 8080 && eq "$(j -n oslo-web svc ledger-api '{.spec.ports[0].targetPort}')" 80 \
&& [ -n "$(j -n oslo-web endpointslices -l kubernetes.io/service-name=ledger-api '{.items[*].endpoints[?(@.conditions.ready==true)].addresses[*]}')" ]"""),
C("gatekeeper is Ready", 3, r"""
for i in $(seq 1 20); do [ "$(j -n oslo-web pod gatekeeper '{.status.conditions[?(@.type=="Ready")].status}')" = True ] && exit 0; sleep 3; done; exit 1"""),
],
solution="""
k -n oslo-web run gatekeeper --image=busybox:1.36 $do -- sleep 86400 > gk.yaml
# add to the container:
# readinessProbe:
# exec:
# command: ["sh", "-c", "wget -T2 -qO- http://ledger-api:8080"]
# initialDelaySeconds: 5
# periodSeconds: 5
k apply -f gk.yaml
k -n oslo-web get pod gatekeeper # 0/1 Running - the probe cannot resolve ledger-api
k -n oslo-web describe pod gatekeeper | tail -3 # Readiness probe failed: wget: bad address 'ledger-api:8080'
k -n oslo-web expose deploy ledger --name=ledger-api --port=8080 --target-port=80
k -n oslo-web get pod gatekeeper -w # 1/1 after the next probe
# Readiness removes a Pod from Service endpoints and blocks rollouts; liveness restarts it.
# A dependency check belongs in readiness - failing liveness here would just restart-loop the Pod.
""",
)
# ------------------------------------------------------------------ 504 ----
task(
id=504, domain="WORK", title="Sorting done by kubectl",
task="""
Write two executable scripts that call `kubectl` (spelled out — aliases do not exist in a fresh
shell) and let **kubectl itself** do the sorting with `--sort-by`:
1. `{WS}/q504/pods-by-age.sh` — every Pod in every Namespace, oldest first (by creation time).
2. `{WS}/q504/pods-by-uid.sh` — the Pods in Namespace `kube-system`, sorted by `metadata.uid`.
""",
setup=r"""rm -rf "$WS/q504"; mkdir -p "$WS/q504" """,
checks=[
C("pods-by-age.sh: all namespaces, --sort-by creationTimestamp, runs", 2, r"""
f="$WS/q504/pods-by-age.sh"; [ -x "$f" ] || exit 1
grep -Eq 'kubectl\b.*\bget\b.*\b(po|pods?)\b' "$f" && grep -Eq -- '(-A|--all-namespaces)' "$f" \
&& grep -Eq -- "--sort-by[= ]*['\"]?\{?\.?metadata\.creationTimestamp" "$f" || exit 1
out=$(timeout 60 bash "$f" 2>/dev/null); [ "$(echo "$out" | wc -l)" -ge 5 ] && echo "$out" | grep -q kube-system"""),
C("pods-by-uid.sh: kube-system, --sort-by metadata.uid, runs", 2, r"""
f="$WS/q504/pods-by-uid.sh"; [ -x "$f" ] || exit 1
grep -Eq 'kubectl\b.*\bget\b.*\b(po|pods?)\b' "$f" && grep -Eq -- '(-n|--namespace)[= ]*kube-system' "$f" \
&& grep -Eq -- "--sort-by[= ]*['\"]?\{?\.?metadata\.uid" "$f" || exit 1
timeout 60 bash "$f" 2>/dev/null | grep -q etcd"""),
C("no aliases in either script", 1, r"""[ -s "$WS/q504/pods-by-age.sh" ] && [ -s "$WS/q504/pods-by-uid.sh" ] && ! grep -Eq '(^|[;|&(] *| )k +' "$WS/q504/pods-by-age.sh" "$WS/q504/pods-by-uid.sh" """),
],
solution="""
cat > {WS}/q504/pods-by-age.sh <<'EOF'
kubectl get pods -A --sort-by=.metadata.creationTimestamp
EOF
cat > {WS}/q504/pods-by-uid.sh <<'EOF'
kubectl -n kube-system get pods --sort-by=.metadata.uid
EOF
chmod +x {WS}/q504/*.sh
bash {WS}/q504/pods-by-age.sh | head -3 # the control-plane pods come first - they are oldest
# --sort-by takes a JSONPath into each item; ascending only, so "newest first" is `| tac` or
# sorting by a field that grows the other way.
""",
)
# ------------------------------------------------------------------ 505 ----
task(
id=505, domain="TSHOOT", title="Event forensics around kube-proxy", breaks="w2-kube-proxy",
task="""
1. Write `{WS}/q505/cluster-events.sh`: a script that prints the latest events of the **whole**
cluster (all Namespaces) sorted by creation time, using `kubectl`. Make it executable.
2. Delete the kube-proxy Pod that runs on node `{W2}`. Write the events this caused to
`{WS}/q505/pod-deleted.log`.
3. On node `{W2}`, stop **and remove** the kube-proxy **container** with `crictl` — leave the Pod
object alone. Write the events this caused to `{WS}/q505/container-killed.log`.
4. Which of the two actions made the Pod's `RESTARTS` counter go up? Write `pod` or `container` to
`{WS}/q505/restarts.txt`.
Do step 2 before step 3.
""",
setup=r"""
rm -rf "$WS/q505"; mkdir -p "$WS/q505"
$K -n kube-system get pod -l k8s-app=kube-proxy --field-selector spec.nodeName=$W2 -o jsonpath='{.items[0].metadata.uid}' > "$WS/.q505-uid"
""",
checks=[
C("cluster-events.sh: all namespaces, sorted by time, runs", 2, r"""
f="$WS/q505/cluster-events.sh"; [ -x "$f" ] || exit 1
grep -Eq 'kubectl\b.*\bget\b.*\b(ev|events?)\b' "$f" && grep -Eq -- '(-A|--all-namespaces)' "$f" \
&& grep -Eq -- "--sort-by[= ]*['\"]?\{?\.?(metadata\.creationTimestamp|lastTimestamp|eventTime|firstTimestamp)" "$f" || exit 1
timeout 60 bash "$f" >/dev/null 2>&1"""),
C("the kube-proxy Pod on {W2} was deleted and replaced", 2, r"""
old=$(cat "$WS/.q505-uid" 2>/dev/null); [ -n "$old" ] || exit 1
now=$(j -n kube-system pod -l k8s-app=kube-proxy --field-selector spec.nodeName=$W2 '{.items[0].metadata.uid}')
[ -n "$now" ] && [ "$now" != "$old" ] && eq "$(j -n kube-system pod -l k8s-app=kube-proxy --field-selector spec.nodeName=$W2 '{.items[0].status.phase}')" Running"""),
# counted on the REPLACEMENT pod, which starts at 0 - after a node reboot the original pod
# often already shows a restart, which would otherwise hand these points out for free
C("then its container was killed (restart counter >= 1)", 2, r"""
old=$(cat "$WS/.q505-uid" 2>/dev/null); now=$(j -n kube-system pod -l k8s-app=kube-proxy --field-selector spec.nodeName=$W2 '{.items[0].metadata.uid}')
[ -n "$old" ] && [ -n "$now" ] && [ "$now" != "$old" ] || exit 1
[ "$(j -n kube-system pod -l k8s-app=kube-proxy --field-selector spec.nodeName=$W2 '{.items[0].status.containerStatuses[0].restartCount}')" -ge 1 ] 2>/dev/null"""),
C("pod-deleted.log and container-killed.log hold the events", 2, r"""
for f in pod-deleted.log container-killed.log; do
grep -q 'kube-proxy' "$WS/q505/$f" 2>/dev/null && grep -Eq 'Killing|SuccessfulCreate|SuccessfulDelete|Scheduled|Pulled|Created|Started|BackOff' "$WS/q505/$f" || exit 1
done"""),
C("restarts.txt says container", 2, r"""eq "$(tr -d '[:space:]' < "$WS/q505/restarts.txt" 2>/dev/null | tr A-Z a-z)" container"""),
],
solution="""
cat > {WS}/q505/cluster-events.sh <<'EOF'
kubectl get events -A --sort-by=.metadata.creationTimestamp
EOF
chmod +x {WS}/q505/cluster-events.sh
k -n kube-system get pod -l k8s-app=kube-proxy -o wide # the one on {W2}
k -n kube-system delete pod <kube-proxy-on-{W2}>
k -n kube-system get events --sort-by=.metadata.creationTimestamp | grep kube-proxy | tail -8 > {WS}/q505/pod-deleted.log
# Killing (old pod), SuccessfulCreate (DaemonSet), Scheduled, Pulled, Created, Started (new pod)
{SHELL_W2}
crictl ps --name kube-proxy # container id
crictl stop <id> && crictl rm <id>
exit
k -n kube-system get events --sort-by=.metadata.creationTimestamp | grep kube-proxy | tail -4 > {WS}/q505/container-killed.log
# Pulled, Created, Started - same Pod, new container
k -n kube-system get pod -l k8s-app=kube-proxy -o wide # RESTARTS 1 on {W2}
echo container > {WS}/q505/restarts.txt
# Deleting the Pod makes the DaemonSet create a brand-new Pod (RESTARTS 0). Killing only the
# container makes the kubelet start a new container inside the SAME Pod - that is a restart.
""",
)
# ------------------------------------------------------------------ 506 ----
def _census(r):
n = r.sample(range(2, 10), 4)
return {"ctx": "k8s-c1-H", "north": n[0], "south": n[1], "east": n[2], "west": n[3]}
task(
id=506, domain="ARCH", title="API resources and a Role census", params=_census,
task="""
1. Write the names of every **namespaced** resource type this API server offers to
`{WS}/q506/namespaced.txt`, one per line — plural names, as `kubectl api-resources` prints them.
2. Of the Namespaces `census-north`, `census-south`, `census-east` and `census-west`, which one holds
the most Roles? Write `<namespace> <number of Roles>` to `{WS}/q506/most-roles.txt`.
""",
setup=NS + r"""
rm -rf "$WS/q506"; mkdir -p "$WS/q506"
for pair in north:%{north} south:%{south} east:%{east} west:%{west}; do
ns=census-${pair%%:*}; n=${pair##*:}
mkns $ns
$K -n $ns delete role --all >/dev/null 2>&1
for i in $(seq 1 $n); do $K -n $ns create role reader-$i --verb=get,list --resource=configmaps >/dev/null; done
done
""",
checks=[
C("namespaced.txt lists exactly the namespaced resource types", 3, r"""
f="$WS/q506/namespaced.txt"; [ -s "$f" ] || exit 1
want=$($K api-resources --namespaced=true -o name 2>/dev/null | sed 's/\..*//' | sort -u)
got=$(awk '{print $1}' "$f" | sed 's/\..*//' | grep . | sort -u)
[ -n "$want" ] && [ "$got" = "$want" ]"""),
C("most-roles.txt names the namespace and its Role count", 3, r"""
best=$(for ns in census-north census-south census-east census-west; do echo "$ns $($K -n $ns get role --no-headers 2>/dev/null | wc -l)"; done | sort -k2 -n | tail -1)
eq "$(tr -s ' ' < "$WS/q506/most-roles.txt" 2>/dev/null | sed 's/^ *//; s/ *$//')" "$best" """),
],
solution="""
k api-resources --namespaced=true -o name > /tmp/r.txt # deployments.apps, ... (group suffix)
k api-resources --namespaced=true --no-headers | awk '{print $1}' > {WS}/q506/namespaced.txt
for ns in census-north census-south census-east census-west; do echo "$ns $(k -n $ns get role --no-headers | wc -l)"; done
# pick the largest, e.g. echo "census-east 8" > {WS}/q506/most-roles.txt
# --namespaced=false gives the cluster-scoped half (nodes, pv, storageclasses, clusterroles...).
# `k get role -A --no-headers | awk '{print $1}' | sort | uniq -c | sort -n` counts every
# namespace at once.
""",
)
# ------------------------------------------------------------------ 507 ----
task(
id=507, domain="TSHOOT", title="Kubelet unit points at nothing", breaks="w1-kubelet", disruptive=True,
task="""
Node `{W1}` went `NotReady` after someone "tidied up" its kubelet installation. Bring it back to
`Ready`; the kubelet has to run as the systemd service it always was. Then write the full path of the
file you had to fix to `{WS}/q507/fixed-file.txt`.
Reach the node with `{SHELL_W1}`. Pods on a node that stays NotReady are evicted after five minutes.
""",
setup=r"""
rm -rf "$WS/q507"; mkdir -p "$WS/q507"
nx "$W1" '
d=/etc/systemd/system/kubelet.service.d/10-kubeadm.conf
grep -q "^ExecStart=/usr/bin/kubelet " $d && sed -i "s|^ExecStart=/usr/bin/kubelet |ExecStart=/usr/local/bin/kubelet |" $d
systemctl daemon-reload; systemctl restart kubelet' </dev/null >/dev/null 2>&1
exit 0
""",
checks=[
C("{W1} is Ready", 4, r"""for i in $(seq 1 15); do [ "$(j node "$W1" '{.status.conditions[?(@.type=="Ready")].status}')" = True ] && exit 0; sleep 4; done; exit 1"""),
C("kubelet active, and its unit runs a kubelet binary that exists", 2, r"""
nx "$W1" 'systemctl is-active --quiet kubelet && b=$(sed -n "s|^ExecStart=\([^ ]*kubelet\) .*|\1|p" /etc/systemd/system/kubelet.service.d/10-kubeadm.conf | tail -1) && [ -x "$b" ]' </dev/null"""),
C("fixed-file.txt names the drop-in", 2, r"""grep -q '/etc/systemd/system/kubelet.service.d/10-kubeadm.conf' "$WS/q507/fixed-file.txt" 2>/dev/null"""),
],
solution="""
k get nodes # {W1} NotReady
{SHELL_W1}
systemctl status kubelet
# Process: ... ExecStart=/usr/local/bin/kubelet ... (code=exited, status=203/EXEC)
journalctl -u kubelet -n 20 --no-pager # Unable to locate executable '/usr/local/bin/kubelet'
which kubelet # /usr/bin/kubelet - the binary was never moved
systemctl cat kubelet # shows the unit AND every drop-in, with file names
vi /etc/systemd/system/kubelet.service.d/10-kubeadm.conf # ExecStart=/usr/bin/kubelet ...
systemctl daemon-reload # systemd caches unit files - without this nothing changes
systemctl restart kubelet
systemctl status kubelet # active (running)
exit
echo /etc/systemd/system/kubelet.service.d/10-kubeadm.conf > {WS}/q507/fixed-file.txt
k get nodes -w
# status=203/EXEC means systemd could not exec the binary at all - a path problem, not a kubelet
# configuration problem. `ExecStart=` (empty) in a drop-in clears the unit's own ExecStart first.
""",
)
# ------------------------------------------------------------------ 508 ----
task(
id=508, domain="WORK", title="Secrets as a read-only volume and renamed env vars",
params=lambda r: {"ctx": "k8s-c1-H", "pw": "ledger-" + "".join(r.choice("abcdefghjkmnpqrstuvwxyz23456789") for _ in range(10))},
task="""
In Namespace `oslo-secure`:
1. Create Secret `bundle` from the file `{WS}/q508/tls-bundle.pem`; the key must be the file's name.
2. Create Secret `db-login` with `user=svc-ledger` and `pass=%{pw}`.
3. Create Pod `vault-reader` (image `busybox:1.36`, command `sleep 86400`) that
- mounts Secret `bundle` **read-only** at `/etc/bundle`,
- receives `db-login`'s keys as environment variables named **`APP_USER`** and **`APP_PASS`**.
""",
setup=NS + r"""
mkns oslo-secure
rm -rf "$WS/q508"; mkdir -p "$WS/q508"
openssl req -x509 -newkey rsa:2048 -nodes -days 30 -subj "/CN=ledger.oslo.internal" -keyout /dev/null -out "$WS/q508/tls-bundle.pem" >/dev/null 2>&1
$K -n oslo-secure delete pod vault-reader --ignore-not-found --force --grace-period=0 >/dev/null 2>&1
$K -n oslo-secure delete secret bundle db-login --ignore-not-found >/dev/null
""",
checks=[
C("Secrets bundle (key tls-bundle.pem, the file's content) and db-login", 2, r"""
[ "$(j -n oslo-secure secret bundle '{.data.tls-bundle\.pem}' | base64 -d 2>/dev/null)" = "$(cat "$WS/q508/tls-bundle.pem")" ] \
&& eq "$(j -n oslo-secure secret db-login '{.data.user}' | base64 -d)" svc-ledger && eq "$(j -n oslo-secure secret db-login '{.data.pass}' | base64 -d)" '%{pw}'"""),
C("vault-reader Running with bundle mounted read-only at /etc/bundle", 3, r"""
p="$K -n oslo-secure get pod vault-reader"
eq "$($p -o jsonpath='{.status.phase}' 2>/dev/null)" Running || exit 1
v=$($p -o jsonpath='{.spec.volumes[?(@.secret.secretName=="bundle")].name}'); [ -n "$v" ] || exit 1
eq "$($p -o jsonpath="{.spec.containers[0].volumeMounts[?(@.name==\"$v\")].mountPath}")" /etc/bundle \
&& eq "$($p -o jsonpath="{.spec.containers[0].volumeMounts[?(@.name==\"$v\")].readOnly}")" true \
&& $K -n oslo-secure exec vault-reader -- cat /etc/bundle/tls-bundle.pem 2>/dev/null | grep -q 'BEGIN CERTIFICATE'"""),
C("APP_USER / APP_PASS come from db-login", 3, r"""
e=$(j -n oslo-secure pod vault-reader '{.spec.containers[0].env}')
echo "$e" | grep -q '"name":"APP_USER"' && echo "$e" | grep -q '"name":"APP_PASS"' && echo "$e" | grep -q 'db-login' || exit 1
env=$($K -n oslo-secure exec vault-reader -- env 2>/dev/null)
echo "$env" | grep -qx 'APP_USER=svc-ledger' && echo "$env" | grep -qx 'APP_PASS=%{pw}'"""),
],
solution="""
k -n oslo-secure create secret generic bundle --from-file={WS}/q508/tls-bundle.pem # key = file name
k -n oslo-secure create secret generic db-login --from-literal=user=svc-ledger --from-literal=pass='%{pw}'
k -n oslo-secure run vault-reader --image=busybox:1.36 $do -- sleep 86400 > vr.yaml
# in vr.yaml, pod level:
# volumes: [{name: bundle, secret: {secretName: bundle}}]
# in the container:
# volumeMounts: [{name: bundle, mountPath: /etc/bundle, readOnly: true}]
# env:
# - name: APP_USER
# valueFrom: {secretKeyRef: {name: db-login, key: user}}
# - name: APP_PASS
# valueFrom: {secretKeyRef: {name: db-login, key: pass}}
k apply -f vr.yaml
k -n oslo-secure exec vault-reader -- env | grep APP_
k -n oslo-secure exec vault-reader -- ls /etc/bundle
# envFrom would inject the keys under their own names (user, pass); renaming needs env + secretKeyRef.
# A Secret volume updates in place when the Secret changes; env vars never do.
""",
)
# ------------------------------------------------------------------ 509 ----
task(
id=509, domain="ARCH", title="Static Pod on the control plane behind a Service",
task="""
1. On the control-plane node (`{SHELL_CP}`) create a **static Pod** `ops-beacon` in Namespace
`default`: image `nginx:1.27`, requests `cpu: 10m` and `memory: 20Mi`, label `app=ops-beacon`.
2. Expose it with a Service `ops-beacon-svc` of type **NodePort** (port 80 → 80) whose endpoints
contain that static Pod.
3. Write the node port the Service was given to `{WS}/q509/nodeport.txt`.
""",
setup=r"""
rm -rf "$WS/q509"; mkdir -p "$WS/q509"
nx "$CP" 'rm -f /etc/kubernetes/manifests/ops-beacon*.yaml' </dev/null
$K -n default delete svc ops-beacon-svc --ignore-not-found >/dev/null
# the kubelet drops the mirror pod on its next sync of the manifest directory
for i in $(seq 1 30); do $K -n default get pod "ops-beacon-$CP" >/dev/null 2>&1 || break; sleep 2; done
""",
checks=[
C("manifest in the control-plane node's staticPodPath", 1, r"""
nx "$CP" 'd=$(awk "/^staticPodPath:/{print \$2}" /var/lib/kubelet/config.yaml); grep -ls "ops-beacon" "$d"/*.yaml' </dev/null >/dev/null 2>&1"""),
C("mirror Pod ops-beacon-{CP} Running with the requests and label", 3, r"""
p="$K -n default get pod ops-beacon-$CP"
eq "$($p -o jsonpath='{.status.phase}' 2>/dev/null)" Running && eq "$($p -o jsonpath='{.metadata.labels.app}')" ops-beacon \
&& eq "$($p -o jsonpath='{.spec.containers[0].resources.requests.cpu}')" 10m && eq "$($p -o jsonpath='{.spec.containers[0].resources.requests.memory}')" 20Mi"""),
C("NodePort Service ops-beacon-svc 80 -> 80 with the static Pod as endpoint", 3, r"""
s="$K -n default get svc ops-beacon-svc"
eq "$($s -o jsonpath='{.spec.type}' 2>/dev/null)" NodePort && eq "$($s -o jsonpath='{.spec.ports[0].port}')" 80 && eq "$($s -o jsonpath='{.spec.ports[0].targetPort}')" 80 || exit 1
ip=$(j -n default pod "ops-beacon-$CP" '{.status.podIP}'); [ -n "$ip" ] || exit 1
j -n default endpointslices -l kubernetes.io/service-name=ops-beacon-svc '{.items[*].endpoints[*].addresses[*]}' | tr ' ' '\n' | grep -qx "$ip" """),
C("nodeport.txt holds the Service's node port", 1, r"""np=$(j -n default svc ops-beacon-svc '{.spec.ports[0].nodePort}'); [ -n "$np" ] && eq "$(tr -d '[:space:]' < "$WS/q509/nodeport.txt" 2>/dev/null)" "$np" """),
],
solution="""
k run ops-beacon -n default --image=nginx:1.27 --labels=app=ops-beacon $do > beacon.yaml
# add to the container: resources: {requests: {cpu: 10m, memory: 20Mi}}
{SHELL_CP}
grep staticPodPath /var/lib/kubelet/config.yaml # /etc/kubernetes/manifests
vi /etc/kubernetes/manifests/ops-beacon.yaml # paste beacon.yaml
exit
k -n default get pod ops-beacon-{CP} -o wide # mirror pod, named <pod>-<node>
k -n default expose pod ops-beacon-{CP} --name=ops-beacon-svc --type=NodePort --port=80 --target-port=80
k -n default get endpointslices -l kubernetes.io/service-name=ops-beacon-svc # the mirror pod's IP
k -n default get svc ops-beacon-svc -o jsonpath='{.spec.ports[0].nodePort}' > {WS}/q509/nodeport.txt
# The control-plane taint only keeps the SCHEDULER from placing pods there; the kubelet runs its
# static pods regardless. `expose pod` copies the pod's labels into the selector.
""",
)
# ------------------------------------------------------------------ 510 ----
task(
id=510, domain="ARCH", title="The kubelet's two certificates",
task="""
On node `{W2}` find the certificate the kubelet uses as a **client** (to talk to the API server)
and the one it **serves** its own API with (port 10250). Write exactly this to
`{WS}/q510/kubelet-certs.txt`:
```
client-issuer: <issuer, as openssl prints it>
client-eku: <the extended key usage>
server-issuer: <issuer>
server-eku: <the extended key usage>
```
Reach the node with `{SHELL_W2}`.
""",
setup=r"""rm -rf "$WS/q510"; mkdir -p "$WS/q510" """,
checks=[
C("client certificate: issuer and extended key usage", 3, r"""
f="$WS/q510/kubelet-certs.txt"; [ -s "$f" ] || exit 1
iss=$(nx "$W2" 'openssl x509 -in /var/lib/kubelet/pki/kubelet-client-current.pem -noout -issuer' </dev/null | sed 's/^issuer= *//' | tr -d ' ' | tr A-Z a-z)
[ -n "$iss" ] && sed -n 's/^ *client-issuer *: *//p' "$f" | tr -d ' ' | tr A-Z a-z | grep -qF "$iss" \
&& sed -n 's/^ *client-eku *: *//p' "$f" | grep -qi 'client auth'"""),
C("serving certificate: issuer and extended key usage", 3, r"""
f="$WS/q510/kubelet-certs.txt"
iss=$(nx "$W2" 'openssl x509 -in /var/lib/kubelet/pki/kubelet.crt -noout -issuer' </dev/null | sed 's/^issuer= *//' | tr -d ' ' | tr A-Z a-z)
[ -n "$iss" ] && sed -n 's/^ *server-issuer *: *//p' "$f" | tr -d ' ' | tr A-Z a-z | grep -qF "$iss" \
&& sed -n 's/^ *server-eku *: *//p' "$f" | grep -qi 'server auth'"""),
],
solution="""
{SHELL_W2}
grep -A3 users: /etc/kubernetes/kubelet.conf # client-certificate: /var/lib/kubelet/pki/kubelet-client-current.pem
ls /var/lib/kubelet/pki/ # kubelet.crt / kubelet.key = the serving pair
openssl x509 -in /var/lib/kubelet/pki/kubelet-client-current.pem -noout -text | grep -E 'Issuer|Extended Key Usage' -A1
# Issuer: CN = kubernetes Extended Key Usage: TLS Web Client Authentication
openssl x509 -in /var/lib/kubelet/pki/kubelet.crt -noout -text | grep -E 'Issuer|Extended Key Usage' -A1
# Issuer: CN = {W2}-ca@<timestamp> Extended Key Usage: TLS Web Server Authentication
exit
# write the four lines with the values YOUR openssl printed:
# client-issuer: CN=kubernetes
# client-eku: TLS Web Client Authentication
# server-issuer: CN={W2}-ca@<timestamp>
# server-eku: TLS Web Server Authentication
# The client cert is signed by the cluster CA and rotated by the kubelet through CSRs. The serving
# cert here is self-signed by the kubelet at startup (no serverTLSBootstrap) - which is exactly why
# metrics-server needs --kubelet-insecure-tls on this cluster.
""",
)
# ------------------------------------------------------------------ 511 ----
task(
id=511, domain="WORK", title="A Pod for the control plane only",
task="""
Create Pod `cp-watch` in Namespace `default`: image `httpd:2-alpine`, container name `watch`.
It must be able to run **only** on control-plane nodes. Select them by the node label kubeadm puts
on control-plane nodes — not by host name, and not with `nodeName` — and tolerate whatever keeps
ordinary Pods off them. It must be Running.
""",
setup=r"""$K -n default delete pod cp-watch --ignore-not-found --force --grace-period=0 >/dev/null 2>&1""",
checks=[
C("cp-watch Running on the control-plane node, container named watch", 2, r"""
eq "$(j -n default pod cp-watch '{.status.phase}')" Running && eq "$(j -n default pod cp-watch '{.spec.nodeName}')" "$CP" && eq "$(j -n default pod cp-watch '{.spec.containers[0].name}')" watch"""),
C("selected by the node-role.kubernetes.io/control-plane label", 2, r"""
s=$(j -n default pod cp-watch '{.spec.nodeSelector}{.spec.affinity.nodeAffinity}')
echo "$s" | grep -q 'node-role.kubernetes.io/control-plane' && ! echo "$s" | grep -q 'kubernetes.io/hostname'"""),
C("tolerates the control-plane taint, no nodeName in the manifest", 2, r"""
$K -n default get pod cp-watch -o json 2>/dev/null | python3 -c '
import sys,json; t=json.load(sys.stdin)["spec"].get("tolerations") or []
sys.exit(0 if any("control-plane" in (x.get("key") or "") or (x.get("operator")=="Exists" and not x.get("key")) for x in t) else 1)' || exit 1
$K -n default get pod cp-watch -o json --show-managed-fields | python3 -c '
import sys,json
mf=json.load(sys.stdin)["metadata"].get("managedFields",[])
sys.exit(1 if any("f:nodeName" in json.dumps(m.get("fieldsV1",{})) for m in mf) else 0)'"""),
],
solution="""
k get node {CP} --show-labels | tr ',' '\\n' | grep node-role # node-role.kubernetes.io/control-plane=
k describe node {CP} | grep Taints # node-role.kubernetes.io/control-plane:NoSchedule
k -n default run cp-watch --image=httpd:2-alpine $do > cp.yaml
# rename the container to watch, then add at pod level:
# nodeSelector:
# node-role.kubernetes.io/control-plane: ""
# tolerations:
# - {key: node-role.kubernetes.io/control-plane, operator: Exists, effect: NoSchedule}
k apply -f cp.yaml
k -n default get pod cp-watch -o wide
# The toleration only ALLOWS the control plane; the nodeSelector is what REQUIRES it. You need both.
""",
)
# ------------------------------------------------------------------ 512 ----
task(
id=512, domain="TSHOOT", title="Stop the scheduler, schedule by hand", breaks="scheduler",
task="""
1. Stop the kube-scheduler **temporarily** — it runs as a static Pod on the control-plane node — and
confirm it is gone.
2. Create Pod `manual-1` (image `busybox:1.36`, command `sleep 86400`) in Namespace `default` and
confirm it stays `Pending`.
3. Schedule `manual-1` **by hand** onto the control-plane node and get it Running.
4. Start the kube-scheduler again, then create Pod `manual-2` (same image and command) in `default`
and confirm that the scheduler places it.
""",
setup=r"""
# a pristine copy outside the static pod directory, so the pre-session baseline can put the
# scheduler back if this task is abandoned half-done
nx "$CP" 'mkdir -p /etc/kubernetes/.cka-sim && [ -f /etc/kubernetes/manifests/kube-scheduler.yaml ] && cp /etc/kubernetes/manifests/kube-scheduler.yaml /etc/kubernetes/.cka-sim/kube-scheduler.yaml' </dev/null
$K -n default delete pod manual-1 manual-2 --ignore-not-found --force --grace-period=0 >/dev/null 2>&1
exit 0
""",
checks=[
C("kube-scheduler is back and Running", 2, r"""
$K -n default get pod manual-1 >/dev/null 2>&1 || exit 1 # only once the task was actually done
nx "$CP" 'test -f /etc/kubernetes/manifests/kube-scheduler.yaml' </dev/null || exit 1
for i in $(seq 1 15); do [ "$(j -n kube-system pod -l component=kube-scheduler '{.items[0].status.containerStatuses[0].ready}')" = true ] && exit 0; sleep 4; done; exit 1"""),
C("manual-1 Running on the control-plane node", 3, r"""
eq "$(j -n default pod manual-1 '{.status.phase}')" Running && eq "$(j -n default pod manual-1 '{.spec.nodeName}')" "$CP" """),
C("manual-2 Running, placed by the scheduler (no nodeName in its manifest)", 3, r"""
eq "$(j -n default pod manual-2 '{.status.phase}')" Running && [ -n "$(j -n default pod manual-2 '{.spec.nodeName}')" ] || exit 1
$K -n default get pod manual-2 -o json --show-managed-fields | python3 -c '
import sys,json
mf=json.load(sys.stdin)["metadata"].get("managedFields",[])
sys.exit(1 if any("f:nodeName" in json.dumps(m.get("fieldsV1",{})) for m in mf) else 0)'"""),
],
solution="""
{SHELL_CP}
mv /etc/kubernetes/manifests/kube-scheduler.yaml /etc/kubernetes/ # the kubelet stops what it no longer sees
exit
k -n kube-system get pod -l component=kube-scheduler # gone (give it ~20s)
k -n default run manual-1 --image=busybox:1.36 -- sleep 86400
k -n default get pod manual-1 # Pending, no node
# spec.nodeName is only settable on creation, so recreate it with the node filled in:
k -n default get pod manual-1 -o yaml > m1.yaml
# add under spec: nodeName: {CP}
k replace --force -f m1.yaml
k -n default get pod manual-1 -o wide # Running on {CP}
{SHELL_CP}
mv /etc/kubernetes/kube-scheduler.yaml /etc/kubernetes/manifests/
exit
k -n kube-system get pod -l component=kube-scheduler -w # Running again
k -n default run manual-2 --image=busybox:1.36 -- sleep 86400
k -n default get pod manual-2 -o wide # a worker, chosen by the scheduler
# nodeName skips the scheduler entirely - including its taint checks, which is why manual-1 can
# sit on the tainted control plane. (The Binding API does the same without recreating the pod.)
""",
)
# ------------------------------------------------------------------ 513 ----
_CHART = r'''
mkchart() { # dir name version image replicas
mkdir -p "$1/templates"
printf 'apiVersion: v2\nname: %s\nversion: %s\nappVersion: "%s"\n' "$2" "$3" "${4##*:}" > "$1/Chart.yaml"
printf 'replicaCount: %s\nimage: %s\n' "$5" "$4" > "$1/values.yaml"
cat > "$1/templates/deployment.yaml" <<'YAML'
apiVersion: apps/v1
kind: Deployment
metadata: {name: {{ .Release.Name }}, labels: {app: {{ .Release.Name }}}}
spec:
replicas: {{ .Values.replicaCount }}
selector: {matchLabels: {app: {{ .Release.Name }}}}
template:
metadata: {labels: {app: {{ .Release.Name }}}}
spec:
containers: [{name: main, image: "{{ .Values.image }}"}]
YAML
}
'''
task(
id=513, domain="ARCH", title="Helm release housekeeping",
task="""
Namespace `oslo-helm` holds several Helm releases; local charts are under `{WS}/q513/charts`.
1. Delete the release `web-legacy`.
2. Upgrade the release `web-main` to chart version **1.1.0** (`{WS}/q513/charts/nginx-lite-1.1.0`).
3. Install a new release `cache` from `{WS}/q513/charts/redis-lite` with **2** replicas (the chart's
`replicaCount` value).
4. One release in the Namespace never finished installing. Find it and remove it.
""",
setup=NS + _CHART + r"""
mkns oslo-helm
C="$WS/q513/charts"; rm -rf "$WS/q513"; mkdir -p "$C"
mkchart "$C/nginx-lite-1.0.0" nginx-lite 1.0.0 nginx:1.25 1
mkchart "$C/nginx-lite-1.1.0" nginx-lite 1.1.0 nginx:1.27 1
mkchart "$C/redis-lite" redis-lite 0.3.0 redis:7-alpine 1
mkchart /tmp/q513-stuck cache-edge 0.1.0 busybox:0.0-does-not-exist 1
for r in $(helm -n oslo-helm list -q 2>/dev/null); do helm -n oslo-helm uninstall "$r" >/dev/null 2>&1; done
helm -n oslo-helm install web-legacy "$C/nginx-lite-1.0.0" >/dev/null
helm -n oslo-helm install web-main "$C/nginx-lite-1.0.0" >/dev/null
# helm records the release as pending-install BEFORE it starts waiting; kill it mid-wait and it
# stays that way - the classic result of an interrupted install or a CI job that timed out
timeout -s KILL 6 helm -n oslo-helm install cache-broken /tmp/q513-stuck --wait --timeout 120s >/dev/null 2>&1
rm -rf /tmp/q513-stuck
helm -n oslo-helm status cache-broken -o json 2>/dev/null | grep -q '"status":"pending-install"' || echo "warn: cache-broken did not end up pending-install"
""",
checks=[
C("web-legacy is gone", 1, r"""! helm -n oslo-helm status web-legacy >/dev/null 2>&1 && helm -n oslo-helm status web-main >/dev/null 2>&1"""),
C("web-main deployed at chart version 1.1.0", 2, r"""
helm -n oslo-helm list -o json 2>/dev/null | python3 -c '
import sys,json
r={x["name"]:x for x in json.load(sys.stdin)}.get("web-main") or {}
sys.exit(0 if r.get("chart")=="nginx-lite-1.1.0" and r.get("status")=="deployed" and int(r.get("revision",0))>=2 else 1)'"""),
C("cache deployed from redis-lite with 2 available replicas", 2, r"""
helm -n oslo-helm list -o json 2>/dev/null | python3 -c '
import sys,json
r={x["name"]:x for x in json.load(sys.stdin)}.get("cache") or {}
sys.exit(0 if r.get("chart","").startswith("redis-lite-") and r.get("status")=="deployed" else 1)' || exit 1
for i in $(seq 1 20); do [ "$(j -n oslo-helm deploy cache '{.status.availableReplicas}')" = 2 ] && exit 0; sleep 3; done; exit 1"""),
C("the stuck release is removed; nothing pending remains", 2, r"""
! helm -n oslo-helm status cache-broken >/dev/null 2>&1 && [ -z "$(helm -n oslo-helm list --pending -q 2>/dev/null)" ] && ! $K -n oslo-helm get deploy cache-broken >/dev/null 2>&1"""),
],
solution="""
helm -n oslo-helm list # Helm 4 shows every status; Helm 3 hides pending ones
helm -n oslo-helm list --pending # works in both: cache-broken pending-install
helm -n oslo-helm uninstall web-legacy
helm -n oslo-helm upgrade web-main {WS}/q513/charts/nginx-lite-1.1.0
helm -n oslo-helm install cache {WS}/q513/charts/redis-lite --set replicaCount=2
helm -n oslo-helm uninstall cache-broken # also removes the Deployment it half-created
helm -n oslo-helm list # web-main (1.1.0, rev 2) and cache, both deployed
# A pending-install release blocks a plain re-install under the same name ("cannot re-use a name
# that is still in use"). `helm history <rel>` shows when it got stuck; uninstall clears it.
""",
)
# ------------------------------------------------------------------ 514 ----
task(
id=514, domain="STOR", title="Per-replica storage for a StatefulSet",
task="""
In Namespace `lima-data`:
1. Create a headless Service `ledger-db` (port 5432) and a StatefulSet `ledger-db` with **3**
replicas: image `busybox:1.36`, command `sh -c 'hostname > /data/id.txt && sleep 86400'`, and a
`volumeClaimTemplates` entry `data` — 100Mi, ReadWriteOnce, StorageClass `standard` — mounted at
`/data`.
2. Once all 3 are Ready, scale the StatefulSet down to **2**.
3. Write the names of the PersistentVolumeClaims that belong to the StatefulSet and still exist
after the scale-down to `{WS}/q514/pvcs.txt`, one per line, sorted.
""",
setup=NS + r"""
mkns lima-data
rm -rf "$WS/q514"; mkdir -p "$WS/q514"
$K -n lima-data delete sts ledger-db --ignore-not-found --wait=true >/dev/null 2>&1
$K -n lima-data delete svc ledger-db --ignore-not-found >/dev/null
$K -n lima-data delete pvc -l app=ledger-db --ignore-not-found >/dev/null 2>&1
for i in 0 1 2 3 4; do $K -n lima-data delete pvc data-ledger-db-$i --ignore-not-found --wait=false >/dev/null 2>&1; done
""",
checks=[
C("headless Service and a StatefulSet with the right claim template", 2, r"""
eq "$(j -n lima-data svc ledger-db '{.spec.clusterIP}')" None && eq "$(j -n lima-data sts ledger-db '{.spec.serviceName}')" ledger-db || exit 1
t='{.spec.volumeClaimTemplates[?(@.metadata.name=="data")]'
eq "$(j -n lima-data sts ledger-db "$t.spec.resources.requests.storage}")" 100Mi && eq "$(j -n lima-data sts ledger-db "$t.spec.storageClassName}")" standard \
&& j -n lima-data sts ledger-db "$t.spec.accessModes}" | grep -q ReadWriteOnce"""),
C("scaled to 2, both Ready", 2, r"""eq "$(j -n lima-data sts ledger-db '{.spec.replicas}')" 2 && eq "$(j -n lima-data sts ledger-db '{.status.readyReplicas}')" 2"""),
C("each replica wrote its own identity to its own volume", 2, r"""
for i in 0 1; do eq "$($K -n lima-data exec ledger-db-$i -- cat /data/id.txt 2>/dev/null | tr -d '[:space:]')" "ledger-db-$i" || exit 1; done
[ "$(j -n lima-data pod ledger-db-0 '{.spec.volumes[?(@.name=="data")].persistentVolumeClaim.claimName}')" = data-ledger-db-0 ]"""),
C("pvcs.txt lists every claim that survived the scale-down", 2, r"""
want=$($K -n lima-data get pvc --no-headers -o custom-columns=N:.metadata.name 2>/dev/null | grep '^data-ledger-db-' | sort)
[ -n "$want" ] && [ "$(echo "$want" | wc -l)" -ge 3 ] && [ "$(grep . "$WS/q514/pvcs.txt" 2>/dev/null)" = "$want" ]"""),
],
solution="""
cat <<EOF | k apply -f -
apiVersion: v1
kind: Service
metadata: {name: ledger-db, namespace: lima-data}
spec: {clusterIP: None, selector: {app: ledger-db}, ports: [{port: 5432}]}
---
apiVersion: apps/v1
kind: StatefulSet
metadata: {name: ledger-db, namespace: lima-data}
spec:
serviceName: ledger-db
replicas: 3
selector: {matchLabels: {app: ledger-db}}
template:
metadata: {labels: {app: ledger-db}}
spec:
containers:
- name: db
image: busybox:1.36
command: [sh, -c, "hostname > /data/id.txt && sleep 86400"]
volumeMounts: [{name: data, mountPath: /data}]
volumeClaimTemplates:
- metadata: {name: data}
spec: {accessModes: [ReadWriteOnce], storageClassName: standard, resources: {requests: {storage: 100Mi}}}
EOF
k -n lima-data rollout status sts ledger-db
k -n lima-data scale sts ledger-db --replicas=2
k -n lima-data get pvc -o custom-columns=N:.metadata.name --no-headers | grep '^data-ledger-db-' | sort > {WS}/q514/pvcs.txt
cat {WS}/q514/pvcs.txt # all THREE: data-ledger-db-0, -1, -2
# Claims are named <template>-<statefulset>-<ordinal> and outlive their pods: scaling down (or
# deleting the StatefulSet) keeps them unless spec.persistentVolumeClaimRetentionPolicy says
# otherwise, so scaling back up re-attaches ledger-db-2 to its old data.
""",
)
# ------------------------------------------------------------------ 515 ----
task(
id=515, domain="NET", title="Namespace AND pod selector, enforced",
task="""
Pods labelled `app=db` in Namespace `oslo-db` must accept connections on TCP port 80 **only** from
Pods labelled `role=backend` that run in Namespaces labelled `env=prod`. Every other source must be
refused. Do not restrict egress.
Create the NetworkPolicy `db-ingress` in `oslo-db`. This cluster's CNI enforces NetworkPolicy, so
test what you built: the `api` Pods in `oslo-api` must get through to Service `db`, the `debug`
Pods in `oslo-api` must not, and neither must the `api` Pods in `lima-api`.
""",
setup=NS + r"""
mkns oslo-db; mkns oslo-api; mkns lima-api
$K label ns oslo-api env=prod --overwrite >/dev/null; $K label ns lima-api env=dev --overwrite >/dev/null
$K -n oslo-db delete netpol --all >/dev/null 2>&1
$K -n oslo-db create deploy db --image=nginx:1.27 --port=80 --dry-run=client -o yaml | $K apply -f - >/dev/null
$K -n oslo-db expose deploy db --port=80 --dry-run=client -o yaml | $K apply -f - >/dev/null
mk() { # ns name role
cat <<EOF | $K apply -f - >/dev/null
apiVersion: apps/v1
kind: Deployment
metadata: {name: $2, namespace: $1}
spec:
replicas: 1
selector: {matchLabels: {app: $2}}
template:
metadata: {labels: {app: $2, role: $3}}
spec: {containers: [{name: c, image: busybox:1.36, command: [sleep, "86400"]}]}
EOF
}
mk oslo-api api backend; mk oslo-api debug debug; mk lima-api api backend
for d in oslo-db/db oslo-api/api oslo-api/debug lima-api/api; do $K -n ${d%%/*} rollout status deploy ${d##*/} --timeout=120s >/dev/null 2>&1; done
""",
checks=[
C("selects app=db, Ingress only", 1, r"""
n="$K -n oslo-db get netpol db-ingress"
eq "$($n -o jsonpath='{.spec.podSelector.matchLabels.app}' 2>/dev/null)" db && eq "$($n -o jsonpath='{.spec.policyTypes}')" '["Ingress"]'"""),
C("one peer holding BOTH namespaceSelector env=prod AND podSelector role=backend, on TCP 80", 3, r"""
$K -n oslo-db get netpol db-ingress -o json 2>/dev/null | python3 -c '
import sys,json
ing=json.load(sys.stdin)["spec"].get("ingress",[])
ok=len(ing)==1 and len(ing[0].get("from",[]))==1
if ok:
f=ing[0]["from"][0]
ok=f.get("namespaceSelector",{}).get("matchLabels",{}).get("env")=="prod" and f.get("podSelector",{}).get("matchLabels",{}).get("role")=="backend"
ok=ok and [(str(p.get("port")),p.get("protocol","TCP")) for p in ing[0].get("ports",[])]==[("80","TCP")]
sys.exit(0 if ok else 1)'"""),
C("oslo-api/api reaches db", 2, r"""
$K -n oslo-db get netpol db-ingress >/dev/null 2>&1 || exit 1 # no policy yet: nothing earned
for i in 1 2 3 4 5; do $K -n oslo-api exec deploy/api -- wget -qO- -T3 http://db.oslo-db >/dev/null 2>&1 && exit 0; sleep 2; done; exit 1"""),
C("oslo-api/debug and lima-api/api are refused", 2, r"""
$K -n oslo-db get netpol db-ingress >/dev/null 2>&1 || exit 1
! $K -n oslo-api exec deploy/debug -- wget -qO- -T3 http://db.oslo-db >/dev/null 2>&1 \
&& ! $K -n lima-api exec deploy/api -- wget -qO- -T3 http://db.oslo-db >/dev/null 2>&1"""),
],
solution="""
k get ns --show-labels | grep -E 'oslo-api|lima-api' # env=prod / env=dev
cat <<EOF | k apply -f -
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata: {name: db-ingress, namespace: oslo-db}
spec:
podSelector: {matchLabels: {app: db}}
policyTypes: [Ingress]
ingress:
- from:
- namespaceSelector: {matchLabels: {env: prod}}
podSelector: {matchLabels: {role: backend}} # SAME list item = AND
ports: [{protocol: TCP, port: 80}]
EOF
k -n oslo-api exec deploy/api -- wget -qO- -T3 http://db.oslo-db | head -1 # <!DOCTYPE html>
k -n oslo-api exec deploy/debug -- wget -qO- -T3 http://db.oslo-db # download timed out
k -n lima-api exec deploy/api -- wget -qO- -T3 http://db.oslo-db # download timed out
# The trap is one dash: "- namespaceSelector: ...\\n - podSelector: ..." (two list items) means OR -
# any pod in a prod namespace, OR a backend pod in oslo-db's own namespace. One item with both
# selectors means AND.
""",
)
# ------------------------------------------------------------------ 516 ----
task(
id=516, domain="NET", title="Resolves everything except the cluster",
task="""
Deployment `reporter` in Namespace `lima-client` cannot reach Service `ledger` — its log says
`bad address`. The Service, its endpoints and cluster DNS are all fine: other Pods in the Namespace
resolve `ledger` without trouble.
Fix the `reporter` Deployment (not the Service), without changing its image or command, so that its
requests succeed. Write the field you changed and its new value, as `<field>=<value>`, to
`{WS}/q516/fix.txt`.
""",
setup=NS + r"""
mkns lima-client
rm -rf "$WS/q516"; mkdir -p "$WS/q516"
$K -n lima-client create deploy ledger --image=nginx:1.27 --port=80 --dry-run=client -o yaml | $K apply -f - >/dev/null
$K -n lima-client expose deploy ledger --port=8080 --target-port=80 --dry-run=client -o yaml | $K apply -f - >/dev/null
$K -n lima-client delete deploy reporter --ignore-not-found --wait=true >/dev/null 2>&1
$K apply -f - >/dev/null <<'EOF'
apiVersion: apps/v1
kind: Deployment
metadata: {name: reporter, namespace: lima-client}
spec:
replicas: 1
selector: {matchLabels: {app: reporter}}
template:
metadata: {labels: {app: reporter}}
spec:
dnsPolicy: Default
containers:
- name: reporter
image: busybox:1.36
command: [sh, -c, 'while true; do if wget -qO- -T2 http://ledger:8080/ >/dev/null 2>&1; then echo "$(date +%T) ledger ok"; else wget -qO- -T2 http://ledger:8080/ 2>&1 | head -1; fi; sleep 3; done']
EOF
$K -n lima-client rollout status deploy ledger --timeout=120s >/dev/null 2>&1
""",
checks=[
C("reporter uses the cluster DNS policy again", 3, r"""
p=$(j -n lima-client deploy reporter '{.spec.template.spec.dnsPolicy}'); [ -z "$p" ] || [ "$p" = ClusterFirst ] || [ "$p" = ClusterFirstWithHostNet ]"""),
C("reporter's requests now succeed", 3, r"""
for i in $(seq 1 15); do
p=$($K -n lima-client get pod -l app=reporter --field-selector status.phase=Running -o jsonpath='{.items[0].metadata.name}' 2>/dev/null)
[ -n "$p" ] && $K -n lima-client logs "$p" --tail=3 2>/dev/null | grep -q 'ledger ok' && exit 0; sleep 3
done; exit 1"""),
C("fix.txt names dnsPolicy and its new value", 1, r"""grep -Eqi '^ *dnsPolicy *= *ClusterFirst *$' "$WS/q516/fix.txt" 2>/dev/null"""),
],
solution="""
k -n lima-client logs deploy/reporter --tail=3 # wget: bad address 'ledger:8080'
k -n lima-client get endpointslices # the Service has endpoints - not the problem
k -n lima-client get deploy reporter -o yaml | grep -n dns
# dnsPolicy: Default <- "Default" means: use the NODE's resolver, NOT cluster DNS
k -n lima-client exec deploy/reporter -- cat /etc/resolv.conf # the podman/host resolver, no cluster search domains
k -n lima-client patch deploy reporter -p '{"spec":{"template":{"spec":{"dnsPolicy":"ClusterFirst"}}}}'
k -n lima-client logs deploy/reporter -f # "ledger ok"
echo dnsPolicy=ClusterFirst > {WS}/q516/fix.txt
# The trap is the name: the DEFAULT dnsPolicy is ClusterFirst, while the policy literally called
# "Default" inherits the node's /etc/resolv.conf. ClusterFirstWithHostNet is for hostNetwork pods.
""",
)
# ------------------------------------------------------------------ 517 ----
task(
id=517, domain="STOR", title="A claim that can never bind",
task="""
Deployment `uploader` in Namespace `oslo-files` has no running Pod. Find out why and fix it, so its
Pod runs with the claim mounted at `/files`. Keep the claim's name (`shared-files`), its size and its
StorageClass. Write the root cause, in one line, to `{WS}/q517/reason.txt`.
""",
setup=NS + r"""
mkns oslo-files
rm -rf "$WS/q517"; mkdir -p "$WS/q517"
$K -n oslo-files delete deploy uploader --ignore-not-found --wait=true >/dev/null 2>&1
$K -n oslo-files delete pvc shared-files --ignore-not-found --wait=true >/dev/null 2>&1
$K apply -f - >/dev/null <<'EOF'
apiVersion: v1
kind: PersistentVolumeClaim
metadata: {name: shared-files, namespace: oslo-files}
spec: {accessModes: [ReadWriteMany], storageClassName: standard, resources: {requests: {storage: 1Gi}}}
---
apiVersion: apps/v1
kind: Deployment
metadata: {name: uploader, namespace: oslo-files}
spec:
replicas: 1
selector: {matchLabels: {app: uploader}}
template:
metadata: {labels: {app: uploader}}
spec:
volumes: [{name: files, persistentVolumeClaim: {claimName: shared-files}}]
containers: [{name: up, image: busybox:1.36, command: [sleep, "86400"], volumeMounts: [{name: files, mountPath: /files}]}]
EOF
""",
checks=[
C("shared-files Bound: 1Gi, class standard, an access mode local-path supports", 3, r"""
eq "$(j -n oslo-files pvc shared-files '{.status.phase}')" Bound && eq "$(j -n oslo-files pvc shared-files '{.spec.resources.requests.storage}')" 1Gi \
&& eq "$(j -n oslo-files pvc shared-files '{.spec.storageClassName}')" standard \
&& ! j -n oslo-files pvc shared-files '{.spec.accessModes}' | grep -q ReadWriteMany"""),
C("uploader available with the claim at /files", 2, r"""
for i in $(seq 1 20); do [ "$(j -n oslo-files deploy uploader '{.status.availableReplicas}')" = 1 ] && break; sleep 3; done
eq "$(j -n oslo-files deploy uploader '{.status.availableReplicas}')" 1 \
&& eq "$(j -n oslo-files deploy uploader '{.spec.template.spec.volumes[?(@.persistentVolumeClaim.claimName=="shared-files")].name}')" files \
&& j -n oslo-files deploy uploader '{.spec.template.spec.containers[0].volumeMounts[*].mountPath}' | grep -qw /files"""),
C("reason.txt names the unsupported access mode", 2, r"""grep -Eqi 'ReadWriteMany|RWX|access ?mode' "$WS/q517/reason.txt" 2>/dev/null"""),
],
solution="""
k -n oslo-files get pod # Pending
k -n oslo-files describe pod -l app=uploader | tail -4 # unbound immediate PersistentVolumeClaims / waiting
k -n oslo-files describe pvc shared-files | tail -4
# ProvisioningFailed ... NodePath only supports ReadWriteOnce and ReadWriteOncePod (1.22+) access modes
# accessModes is immutable on a PVC, so recreate it. The Pending pod was never scheduled, so it
# does not hold the claim (pvc-protection only counts scheduled pods) and the delete goes through.
k -n oslo-files get pvc shared-files -o yaml > pvc.yaml # change ReadWriteMany -> ReadWriteOnce, strip status/uid
k -n oslo-files delete pvc shared-files
k apply -f pvc.yaml
k -n oslo-files rollout restart deploy uploader # or just wait - the pending pod retries
k -n oslo-files get pvc,pod
echo 'PVC requested ReadWriteMany, which the local-path provisioner cannot provide (RWO/RWOP only)' > {WS}/q517/reason.txt
# RWX needs a filesystem many nodes can mount at once (NFS, CephFS, a cloud file service).
# ReadWriteOnce is per NODE, not per pod - ReadWriteOncePod is the per-pod one.
""",
)
QUESTIONS = Q

View File

@ -23,8 +23,8 @@ TRACK = {
# tasks per domain drawn into a session (weights follow the exam blueprint)
"quota": {"ARCH": 5, "WORK": 4, "NET": 4, "STOR": 2, "TSHOOT": 5}, # 20 tasks
"quota_hard": {"ARCH": 6, "WORK": 5, "NET": 5, "STOR": 3, "TSHOOT": 6}, # 25 tasks, hard ones preferred
# fixed simulator sets (./exam sim a|b|c) - excluded from random sampling
"sets": {"a": "Simulator Set A", "b": "Simulator Set B", "c": "Simulator Set C"},
# fixed simulator sets (./exam sim a|b|c|d) - excluded from random sampling
"sets": {"a": "Simulator Set A", "b": "Simulator Set B", "c": "Simulator Set C", "d": "Simulator Set D (killer.sh-style)"},
"domains": {
"ARCH": "Cluster Architecture, Installation & Configuration (25%)",
"WORK": "Workloads & Scheduling (15%)",
@ -177,7 +177,7 @@ $K delete ns store-prod --ignore-not-found --wait=false >/dev/null
],
},
{
"id": 5, "domain": "ARCH", "title": "Static pod",
"id": 5, "domain": "ARCH", "title": "Static pod", "breaks": ["w3-staticpath"], # Q103 relocates W3's staticPodPath
"task": """
Create a **static pod** named `static-web` running image `nginx:1.27`
on node `{W3}` (namespace `default`).
@ -362,8 +362,8 @@ Create a NetworkPolicy `backend-policy` in `netpol` so that pods with `app=backe
on TCP port **8080**,
- reject all other ingress.
Do not restrict egress. (The default kind CNI does not enforce policies;
you are graded on the policy spec.)
Do not restrict egress. (This cluster's kindnet enforces NetworkPolicy, so a wrong policy really
blocks traffic; you are graded on the policy spec.)
""",
"setup": NS + r"""
mkns netpol
@ -631,10 +631,10 @@ $K -n default delete pod sched-probe --wait=false >/dev/null 2>&1
]
# ---- aggregate the banks: 1-20 here, 21-43 in bank_extra, 101-124 (hard) in bank_hard,
# ---- 201-217 / 301-317 / 401-418 in the three fixed simulator sets
from tracks.cka import bank_extra, bank_hard, bank_sim_a, bank_sim_b, bank_sim_c
# ---- 201-217 / 301-317 / 401-418 / 501-517 in the four fixed simulator sets
from tracks.cka import bank_extra, bank_hard, bank_sim_a, bank_sim_b, bank_sim_c, bank_sim_d
QUESTIONS = (QUESTIONS + bank_extra.QUESTIONS + bank_hard.QUESTIONS
+ bank_sim_a.QUESTIONS + bank_sim_b.QUESTIONS + bank_sim_c.QUESTIONS)
+ bank_sim_a.QUESTIONS + bank_sim_b.QUESTIONS + bank_sim_c.QUESTIONS + bank_sim_d.QUESTIONS)
QUESTIONS.sort(key=lambda q: q["id"])
_ids = [q["id"] for q in QUESTIONS]
assert len(_ids) == len(set(_ids)), "duplicate question ids"

View File

@ -262,8 +262,10 @@ k -n %{ns} create rolebinding %{sa}-rb --role=%{sa}-role --serviceaccount=%{ns}:
k create clusterrole %{sa}-nodes --verb=list --resource=nodes
k create clusterrolebinding %{sa}-crb --clusterrole=%{sa}-nodes --serviceaccount=%{ns}:%{sa}
# Step 6: Verify effective permissions
# Step 6: Verify effective permissions. For a SUBRESOURCE use --subresource: `can-i create
# pods/exec` means "create the pod named exec" and says no even when the Role is right.
k auth can-i --list -n %{ns} --as=system:serviceaccount:%{ns}:%{sa}
k auth can-i create pods --subresource=exec -n %{ns} --as=system:serviceaccount:%{ns}:%{sa} # yes
"""
S[102] = """# Step 1: Access the control plane node
@ -381,7 +383,10 @@ k create ns %{ns}
k apply -k .
"""
S[106] = """# Step 1: Create CRD with validation rules (required fields, enum, integer bounds)
S[106] = """# Step 0: This context does not default to the `default` namespace - check, then pass -n default
k config view --minify -o jsonpath='{..namespace}{"\\n"}'
# Step 1: CRD with validation (required fields, integer bounds, enum) and a server-side default
cat <<EOF | k apply -f -
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
@ -389,7 +394,11 @@ metadata: {name: $(echo %{kind} | tr A-Z a-z)s.deploy.example.com}
spec:
group: deploy.example.com
scope: Namespaced
names: {kind: %{kind}, plural: $(echo %{kind} | tr A-Z a-z)s, singular: $(echo %{kind} | tr A-Z a-z)}
names:
kind: %{kind}
plural: $(echo %{kind} | tr A-Z a-z)s
singular: $(echo %{kind} | tr A-Z a-z)
shortNames: [$(echo %{kind} | tr A-Z a-z | cut -c1-2)]
versions:
- name: v1
served: true
@ -400,26 +409,28 @@ spec:
properties:
spec:
type: object
required: [replicas, image]
required: [replicas, image] # sits NEXT TO properties, not inside a field
properties:
replicas: {type: integer, minimum: 1, maximum: 10}
environment: {type: string, enum: [prod, stage, dev], default: dev}
image: {type: string}
EOF
# Step 2: Apply valid custom resource
k apply -f - <<EOF
# Step 2: A valid object
k -n default apply -f - <<EOF
apiVersion: deploy.example.com/v1
kind: %{kind}
metadata: {name: good}
spec: {replicas: 3, image: nginx:1.27, environment: %{env}}
EOF
# Step 3: Test schema rejection of invalid resource (0 replicas)
printf 'apiVersion: deploy.example.com/v1\\nkind: %{kind}\\nmetadata: {name: bad}\\nspec: {replicas: 0, image: x}\\n' | k apply -f - 2> {WS}/q106/rejected.txt
# Step 3: The rejection goes to STDERR - redirect 2>, not >, or the file stays empty
printf 'apiVersion: deploy.example.com/v1\\nkind: %{kind}\\nmetadata: {name: bad}\\nspec: {replicas: 0, image: x}\\n' | k -n default apply -f - 2> {WS}/q106/rejected.txt
cat {WS}/q106/rejected.txt # ...spec.replicas in body should be greater than or equal to 1
# Step 4: Verify default values on omitted fields
printf 'apiVersion: deploy.example.com/v1\\nkind: %{kind}\\nmetadata: {name: defaulted}\\nspec: {replicas: 2, image: nginx:1.27}\\n' | k apply -f -
# Step 4: Omit environment and let the API server fill in the default
printf 'apiVersion: deploy.example.com/v1\\nkind: %{kind}\\nmetadata: {name: defaulted}\\nspec: {replicas: 2, image: nginx:1.27}\\n' | k -n default apply -f -
k -n default get $(echo %{kind} | tr A-Z a-z) defaulted -o jsonpath='{.spec.environment}{"\\n"}' # dev
"""
S[122] = """# Step 1: Access control plane node
@ -782,40 +793,47 @@ k -n %{ns} get pod -l app=worker -o jsonpath='{.items[0].spec.containers[0].reso
"""
# Hard Workload Tasks
S[107] = """# Step 1: Inspect namespace quota constraints
k -n %{ns} get quota
S[107] = """# Step 1: Read the constraints before writing anything
k get node {W1} -o jsonpath='{.spec.taints}{"\\n"}' # %{key}=%{val}:NoSchedule
k -n %{ns} describe quota # pods: 2, requests.cpu: 250m
# Step 2: Deploy pods with podAntiAffinity, tolerations, PriorityClass, and fitted resources
# Step 2: The quota is the trap twice over: 3 pods do not fit "pods: 2", and 3 x 100m = 300m does
# not fit 250m either. Raise both - the quota has to stay and still cap pods.
k -n %{ns} patch quota %{ns}-quota -p '{"spec":{"hard":{"pods":"3","requests.cpu":"300m"}}}'
# Step 3: The PriorityClass
k create priorityclass %{dep}-prio --value=50000
# Step 4: One pod per worker (required anti-affinity on hostname) that tolerates {W1}'s taint
cat <<EOF | k apply -f -
apiVersion: apps/v1
kind: Deployment
metadata: {name: anti-workload, namespace: %{ns}}
metadata: {name: %{dep}, namespace: %{ns}}
spec:
replicas: 3
selector: {matchLabels: {app: anti-workload}}
selector: {matchLabels: {app: %{dep}}}
template:
metadata: {labels: {app: anti-workload}}
metadata: {labels: {app: %{dep}}}
spec:
priorityClassName: high-priority
priorityClassName: %{dep}-prio
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector: {matchLabels: {app: anti-workload}}
- labelSelector: {matchLabels: {app: %{dep}}}
topologyKey: kubernetes.io/hostname
tolerations:
- key: dedicated
operator: Equal
value: special
effect: NoSchedule
- {key: %{key}, operator: Equal, value: "%{val}", effect: NoSchedule}
containers:
- name: app
image: busybox:1.36
command: [sleep, "3600"]
resources: {requests: {cpu: 50m, memory: 32Mi}, limits: {cpu: 100m, memory: 64Mi}}
- name: nginx
image: nginx:1.27
resources: {requests: {cpu: 100m, memory: 64Mi}}
EOF
# Step 3: Confirm each replica landed on a distinct node
k -n %{ns} get pods -o wide -l app=anti-workload
# Step 5: Verify: 3 Running on 3 different nodes. The control-plane is tainted too, so with three
# workers there is zero slack - a missed quota field or toleration shows up as a Pending pod, and
# `k -n %{ns} describe rs` tells you which (FailedCreate: exceeded quota).
k -n %{ns} get pod -l app=%{dep} -o wide
k -n %{ns} describe quota %{ns}-quota # pods 3/3, requests.cpu 300m/300m
"""
S[108] = """# Step 1: Find WHICH revision used nginx:1.25 - the task asks for that one specifically, which
@ -846,7 +864,9 @@ k -n %{ns} get rs -o custom-columns=NAME:.metadata.name,IMAGE:.spec.template.spe
"""
S[109] = """# Step 1: Deploy Pod with a native sidecar (restartPolicy: Always inside initContainers),
# a shared emptyDir the sidecar writes into and the main container reads from
# a shared emptyDir the sidecar writes into and the main container reads from. Mount it at
# /var/log/agent, NOT /var/log: an emptyDir over /var/log hides nginx's /var/log/nginx and
# nginx crash-loops with "could not open error log file"
cat <<EOF | k apply -f -
apiVersion: v1
kind: Pod
@ -860,9 +880,9 @@ spec:
- name: agent
image: busybox:1.36
restartPolicy: Always
command: [sh, -c, "while true; do date >> /var/log/agent.log; sleep 1; done"]
command: [sh, -c, "while true; do date >> /var/log/agent/agent.log; sleep 1; done"]
volumeMounts:
- {name: logs, mountPath: /var/log}
- {name: logs, mountPath: /var/log/agent}
containers:
- name: app
image: nginx:1.27
@ -872,7 +892,7 @@ spec:
livenessProbe:
tcpSocket: {port: 80}
volumeMounts:
- {name: logs, mountPath: /var/log}
- {name: logs, mountPath: /var/log/agent}
EOF
# Step 2: Once Running, write the exact report files
@ -881,23 +901,40 @@ k -n %{ns} get pod %{pod} -o jsonpath='{.spec.initContainers[*].name} {.spec.con
k -n %{ns} get pod %{pod} -o jsonpath='{.status.podIP}' > {WS}/q109/ip.txt
"""
S[110] = r"""# Step 1: pods outside kube-system whose FIRST container has no readiness probe
k get pods -A -o json | jq -r '.items[] | select(.metadata.namespace!="kube-system") | select(.spec.containers[0].readinessProbe == null) | "\(.metadata.namespace)/\(.metadata.name)"' | sort > {WS}/q110/no-readiness.txt
S[110] = r"""# Step 0: look before you script - which pods, which phases, which nodes
k -n %{ns} get pods -o wide
# Step 2: count of Running pods per node ("<node> <count>", node first)
k get pods -A --field-selector=status.phase=Running -o jsonpath='{range .items[*]}{.spec.nodeName}{"\n"}{end}' | sort | uniq -c | awk '{print $2, $1}' | sort > {WS}/q110/per-node.txt
# Step 1: FIRST container without a readinessProbe, any phase (the Completed pod counts too).
# jsonpath prints nothing for a missing field, so emit "<name> <probe-or-empty>" and keep the bare names.
k -n %{ns} get pods -o jsonpath='{range .items[*]}{.metadata.name} {.spec.containers[0].readinessProbe}{"\n"}{end}' \
| awk 'NF==1{print $1}' | sort > {WS}/q110/no-readiness.txt
# Step 3: sorted unique container images used across kube-system
# Step 2: Running pods per node, "<node> <count>". uniq -c prints "count node", so swap the columns.
k -n %{ns} get pods --field-selector=status.phase=Running -o jsonpath='{range .items[*]}{.spec.nodeName}{"\n"}{end}' \
| sort | uniq -c | awk '{print $2, $1}' | sort > {WS}/q110/per-node.txt
# Step 3: distinct images in kube-system
k -n kube-system get pods -o jsonpath='{.items[*].spec.containers[*].image}' | tr ' ' '\n' | sort -u > {WS}/q110/images.txt
# Traps: cache-a has a readiness probe - on its SECOND container, so it belongs in file 1;
# once-a is Completed, so it is in file 1 but not in file 2. `cat -A` shows stray spaces or \r.
"""
S[121] = """# Step 1: Confirm current context namespace
k config view --minify -o jsonpath='{.contexts[0].context.namespace}'
S[121] = """# Step 1: Switch context and find out which namespace it defaults to
k config use-context %{ctx}
k config view --minify -o jsonpath='{..namespace}{"\\n"}' # -> %{ns}
# Step 2: Perform all required operations without specifying -n, relying on context namespace
k create deploy %{dep} --image=nginx:1.27 --replicas=2
k set env deploy/%{dep} APP_ENV=production
k expose deploy %{dep} --port=80 --target-port=80
# Step 2: Everything WITHOUT -n, so it lands in the context's namespace
k create deploy %{dep} --image=busybox:1.36 --replicas=2 -- sleep 3600
k label deploy %{dep} owner=ac
k expose deploy %{dep} --port=9000 --target-port=9000 # ClusterIP is the default type
# Step 3: Record the namespace exactly as kubectl reports it
k config view --minify -o jsonpath='{..namespace}' > {WS}/q121/ns.txt
# Step 4: Verify it all landed there - and nothing in default
k get deploy,svc %{dep}
k -n default get deploy %{dep} # NotFound is the right answer
"""
@ -1090,27 +1127,32 @@ EOF
k -n %{ns} describe netpol %{app}-egress
"""
S[43] = """# Step 1: Create Headless Service (clusterIP: None)
S[43] = """# Step 1: The headless Service must be named EXACTLY like the StatefulSet's spec.serviceName -
# that name is what makes CoreDNS publish one record per pod: <pod>.<service>.<ns>.svc.cluster.local
k -n %{ns} get sts %{sts} -o jsonpath='{.spec.serviceName}{"\\n"}' # -> %{sts}
cat <<EOF | k apply -f -
apiVersion: v1
kind: Service
metadata: {name: db-headless, namespace: %{ns}}
metadata: {name: %{sts}, namespace: %{ns}}
spec:
clusterIP: None
selector: {app: db}
selector: {app: %{sts}}
ports: [{port: 5432, targetPort: 5432}]
---
# Step 2: Create ExternalName Service pointing to external domain
apiVersion: v1
kind: Service
metadata: {name: external-api, namespace: %{ns}}
metadata: {name: legacy, namespace: %{ns}}
spec:
type: ExternalName
externalName: api.partner.com
externalName: %{ext}
EOF
# Step 3: Verify DNS resolution for headless endpoints and external CNAME
k -n %{ns} get svc
# Step 2: Resolve pod 0 by its FULL name. busybox's nslookup applies no search domains, so the
# short form %{sts}-0.%{sts} returns NXDOMAIN even though DNS is perfectly healthy.
k -n %{ns} exec %{sts}-0 -- nslookup %{sts}-0.%{sts}.%{ns}.svc.cluster.local > {WS}/q43/pod0.txt
cat {WS}/q43/pod0.txt # Address: <the pod's IP>
k -n %{ns} get pod %{sts}-0 -o wide # same IP
k -n %{ns} get svc legacy # EXTERNAL-IP column shows %{ext} - it is only a CNAME
"""
# Hard Networking Tasks
@ -1223,7 +1265,9 @@ k -n kube-system edit cm coredns
# Step 4: Restart CoreDNS deployment and test DNS resolution
k -n kube-system rollout restart deploy coredns
k run dns-check --image=busybox:1.36 --rm -it --restart=Never -- nslookup kubernetes.default
# busybox's nslookup applies no search domains, so use FULL names (a short one fails even when DNS works)
k run dns-check --image=busybox:1.36 --rm -it --restart=Never -- nslookup kubernetes.default.svc.cluster.local
k run dns-ext --image=busybox:1.36 --rm -it --restart=Never -- nslookup kubernetes.io
"""
@ -1419,8 +1463,13 @@ k -n %{ns} get pod legacy -o yaml > legacy.yaml
# Edit legacy.yaml: replace claimName: old-data with claimName: new-data
k -n %{ns} delete pod legacy --now && k apply -f legacy.yaml
# Step 5: Clean up old PVC (PV remains Retained)
# Step 5: Delete the old claim. The Job's finished pod still references old-data, and a pod that
# references a claim - even a Completed one - holds it in pvc-protection: `delete pvc` just sits
# there (describe pvc -> "Used By: migrate-xxxxx"). Delete that pod first; the Job itself stays,
# still reporting succeeded: 1.
k -n %{ns} delete pod -l job-name=migrate
k -n %{ns} delete pvc old-data
k get pv | grep old-data # STATUS Released - Retain kept the data
"""
@ -1518,7 +1567,9 @@ k -n kube-system edit cm coredns
k -n kube-system rollout restart deploy coredns
# Step 5: Verify DNS resolution from a test pod
k run test-dns --image=busybox:1.36 --rm -it --restart=Never -- nslookup kubernetes.default
# busybox's nslookup applies no search domains, so use FULL names (a short one fails even when DNS works)
k run test-dns --image=busybox:1.36 --rm -it --restart=Never -- nslookup kubernetes.default.svc.cluster.local
k run test-ext --image=busybox:1.36 --rm -it --restart=Never -- nslookup kubernetes.io
"""
S[38] = """# Step 1: Check control-plane pod status
@ -1609,8 +1660,11 @@ k -n %{ns} get all
k -n %{ns} describe deploy %{dep}
k -n %{ns} describe rs -l app=%{dep} # Quota exceeded: pods limit is 1
# Step 2: Fix quota to accommodate required replicas
k -n %{ns} patch quota tight -p '{"spec":{"hard":{"pods":"3"}}}'
# Step 2: Fix the quota - with room for the rolling update, not just the end state. The default
# strategy (maxSurge 25% -> 1, maxUnavailable 25% of 3 -> 0) needs a 4th pod mid-rollout, and with
# maxUnavailable 0 the controller will not remove an old pod first either: at pods=3 the rollout
# deadlocks at "0 out of 3 new replicas have been updated".
k -n %{ns} patch quota tight -p '{"spec":{"hard":{"pods":"4"}}}'
# Step 3: Fix deployment image tag
k -n %{ns} set image deploy %{dep} web=nginx:1.27

View File

@ -535,7 +535,8 @@ Namespace `%{ns}` runs Pods labelled `app=db` and `app=%{client}`.
Create NetworkPolicy `db-allow` so `app=db` Pods accept ingress **only** from `app=%{client}`
Pods in the same Namespace on TCP 5432, and nothing else. Do not restrict egress.
(kindnet doesn't enforce policies here, so you are graded on the object.)
(kindnet enforces NetworkPolicy here, so a wrong policy really blocks traffic; you are graded
on the object.)
""",
"setup": NS + r"""
mkns %{ns}

View File

@ -1,8 +1,8 @@
"""CKS track. Starter bank.
Caveat worth knowing before you rely on this: several CKS objectives need infrastructure the
default kind cluster does not have - kindnet does not enforce NetworkPolicy, and there is no
Falco, AppArmor profile loader, gVisor runtime or image scanner installed. Tasks here either
default kind cluster does not have - there is no Falco, AppArmor profile loader, gVisor runtime
or image scanner installed. (NetworkPolicy IS enforced: recent kindnet embeds a policy engine.) Tasks here either
stick to what a stock kubeadm/kind cluster can really do (PSA, RBAC, admission policies, audit
logging, seccomp, token hygiene) or are graded on the object rather than on enforcement, and say so.
"""
@ -43,7 +43,8 @@ A NetworkPolicy has exactly one `podSelector`, so the egress half (selecting `we
ingress half (selecting `db`) cannot live in the same object. Once a default deny is in place
both halves are required — forgetting one is the classic CKS mistake.
(kindnet does not enforce policies; you are graded on the objects.)
(This cluster's kindnet enforces NetworkPolicy, so a gap really drops traffic; you are graded
on the objects.)
""",
"setup": NS + r"""
mkns %{ns}

View File

@ -1,149 +1,121 @@
# CKA weak spots
_Generated by exam from 11 attempt(s); regenerated on every `end`._
_Generated by exam from 10 attempt(s); regenerated on every `end`._
## Domain scores (latest full exam, 2026-09-20T17:32, 100.0%)
## Domain scores (latest full exam, 2026-09-08T19:59, 14.3%)
- **ARCH** 38/266 — Cluster Architecture, Installation & Configuration (25%)
- **WORK** 36/187 — Workloads & Scheduling (15%)
- **NET** 37/179 — Services & Networking (20%)
- **STOR** 19/85 — Storage (10%)
- **TSHOOT** 33/171 — Troubleshooting (30%)
- **ARCH** 12/26 — Cluster Architecture, Installation & Configuration (25%)
- **WORK** 1/16 — Workloads & Scheduling (15%)
- **NET** 0/17 — Services & Networking (20%)
- **STOR** 0/11 — Storage (10%)
- **TSHOOT** 2/35 — Troubleshooting (30%)
## Drill next (`./exam drill weak`)
### Q02 · ARCH · etcd snapshot (1/2 failed, last 0/7 pts, 2026-09-19T12:27)
### Q02 · ARCH · etcd snapshot (1/2 failed, last 0/7 pts, last failed 2026-09-19T12:27)
- ✗ /opt/cka/etcd-snapshot.db exists on the control-plane node
- ✗ snapshot is a real etcd backup (> 100 KB)
### Q06 · ARCH · Certificates: expiry check and user CSR (2/3 failed, last 6/6 pts, 2026-09-12T22:17)
- ✗ CSR jane exists with signer kubernetes.io/kube-apiserver-client
- ✗ CSR jane approved and certificate issued
### Q06 · ARCH · Certificates: expiry check and user CSR (2/3 failed, last 6/6 pts, last failed 2026-09-12T22:17)
- passed last time (6/6) - one more clean pass takes it off this list
### Q13 · NET · Ingress routing (2/4 failed, last 5/5 pts, 2026-09-14T16:49)
- ✗ ingressClassName nginx and host shop.example.com
- ✗ /api (Prefix) -> api:8080
- ✗ / (Prefix) -> web:80
### Q15 · STOR · PersistentVolume, claim and pod (3/4 failed, last 6/6 pts, last failed 2026-09-15T20:55)
- passed last time (6/6) - one more clean pass takes it off this list
### Q15 · STOR · PersistentVolume, claim and pod (3/4 failed, last 6/6 pts, 2026-09-15T20:55)
- ✗ PV pv-data: 1Gi hostPath /mnt/data, RWO, Retain, class manual
- ✗ PVC pvc-data Bound to pv-data
- ✗ data-pod Running with claim mounted at /usr/share/data
### Q16 · STOR · Default StorageClass and dynamic provisioning (4/4 failed, last 0/5 pts, 2026-09-16T19:40)
### Q16 · STOR · Default StorageClass and dynamic provisioning (4/4 failed, last 0/5 pts, last failed 2026-09-16T19:40)
- ✗ StorageClass fast-local: local-path, WaitForFirstConsumer, Delete
- ✗ fast-local is default and standard is not
- ✗ fast-claim picked up fast-local and is Bound
- ✗ fast-pod Running with claim at /data
### Q17 · TSHOOT · Node NotReady (3/4 failed, last 8/8 pts, 2026-09-13T01:06)
- ✗ kubelet is active
- ✗ kubelet is enabled (survives reboot)
### Q17 · TSHOOT · Node NotReady (3/4 failed, last 8/8 pts, last failed 2026-09-13T01:06)
- passed last time (8/8) - one more clean pass takes it off this list
### Q19 · TSHOOT · CrashLoopBackOff diagnosis (4/5 failed, last 6/6 pts, 2026-09-14T16:49)
- ✗ crasher.log captured and contains the DB_HOST failure
### Q19 · TSHOOT · CrashLoopBackOff diagnosis (4/5 failed, last 6/6 pts, last failed 2026-09-14T16:49)
- passed last time (6/6) - one more clean pass takes it off this list
### Q25 · WORK · Taints and tolerations (3/5 failed, last 4/4 pts, 2026-09-15T20:55)
- ✗ cka-sim-worker carries team=finance:NoSchedule
- ✗ deployment tolerates the taint
- ✗ both pods Running on cka-sim-worker
### Q28 · WORK · Init container prepares content (4/4 failed, last 0/4 pts, 2026-09-16T19:40)
### Q28 · WORK · Init container prepares content (4/4 failed, last 0/4 pts, last failed 2026-09-16T19:40)
- ✗ pod Running, init container named setup completed
- ✗ emptyDir 'content' shared by both containers
- ✗ nginx serves the init-written content
### Q29 · WORK · Immutable ConfigMap rotation (3/4 failed, last 4/4 pts, 2026-09-15T20:55)
- ✗ deployment now references a different ConfigMap
- ✗ all pods available and see mode=new
### Q29 · WORK · Immutable ConfigMap rotation (3/4 failed, last 4/4 pts, last failed 2026-09-15T20:55)
- passed last time (4/4) - one more clean pass takes it off this list
### Q30 · NET · Named ports (2/3 failed, last 4/4 pts, 2026-09-14T16:49)
- ✗ container port 80 named svc-http
- ✗ Service port 8080 -> targetPort svc-http (by name)
- ✗ readiness probe uses the named port; endpoints ready
### Q30 · NET · Named ports (2/3 failed, last 4/4 pts, last failed 2026-09-14T16:49)
- passed last time (4/4) - one more clean pass takes it off this list
### Q32 · WORK · ResourceQuota and LimitRange (1/1 failed, last 0/4 pts, 2026-09-19T12:27)
### Q32 · WORK · ResourceQuota and LimitRange (1/1 failed, last 0/4 pts, last failed 2026-09-19T12:27)
- ✗ ResourceQuota limits pods/6, requests.cpu 1, limits.memory 2Gi
- ✗ LimitRange defaults set
- ✗ worker pods Running with defaulted requests
### Q33 · ARCH · Aggregated ClusterRole (2/3 failed, last 5/5 pts, 2026-09-14T16:49)
- ✗ ops-19 has an aggregationRule with the right label
- ✗ aggregation populated rules from the two labelled roles
- ✗ sam can list pods and services cluster-wide
### Q33 · ARCH · Aggregated ClusterRole (2/3 failed, last 5/5 pts, last failed 2026-09-14T16:49)
- passed last time (5/5) - one more clean pass takes it off this list
### Q37 · TSHOOT · Cluster DNS is broken (4/5 failed, last 6/6 pts, 2026-09-14T16:49)
- ✗ CoreDNS Corefile forwards to the node resolver again
- ✗ in-cluster and external names resolve from a pod
### Q37 · TSHOOT · Cluster DNS is broken (4/5 failed, last 6/6 pts, last failed 2026-09-14T16:49)
- passed last time (6/6) - one more clean pass takes it off this list
### Q38 · TSHOOT · Deployments never create pods (2/3 failed, last 7/7 pts, 2026-09-13T01:06)
- ✗ kube-controller-manager manifest repaired and pod Running
- ✗ a new Deployment gets a ReplicaSet and running pods
### Q38 · TSHOOT · Deployments never create pods (2/3 failed, last 7/7 pts, last failed 2026-09-13T01:06)
- passed last time (7/7) - one more clean pass takes it off this list
### Q39 · TSHOOT · kubelet misconfiguration (3/5 failed, last 7/7 pts, 2026-09-14T16:49)
- ✗ kubelet config points at a CA file that exists
- ✗ node Ready
- ✗ kubelet active and enabled
### Q40 · ARCH · kubeadm join token (4/4 failed, last 0/5 pts, 2026-09-16T19:40)
### Q40 · ARCH · kubeadm join token (4/4 failed, last 0/5 pts, last failed 2026-09-16T19:40)
- ✗ join.sh contains a kubeadm join with token and CA hash
- ✗ the token in the file exists in the cluster with TTL ~24h
### Q42 · NET · Egress-only NetworkPolicy (2/3 failed, last 4/4 pts, 2026-09-14T16:49)
- ✗ applies to app=client-64, Egress only
- ✗ DNS rule: kube-system namespace + k8s-app=kube-dns, ports 53 UDP and TCP
- ✗ TCP 443 to 172.16.0.0/16 only
### Q42 · NET · Egress-only NetworkPolicy (2/3 failed, last 4/4 pts, last failed 2026-09-14T16:49)
- passed last time (4/4) - one more clean pass takes it off this list
### Q102 · ARCH · etcd restore (1/1 failed, last 2/10 pts, 2026-09-19T12:27)
### Q102 · ARCH · etcd restore (1/1 failed, last 2/10 pts, last failed 2026-09-19T12:27)
- ✗ ConfigMap restore-proof-67 exists again with its original data
- ✗ etcd uses a restored data directory (manifest changed)
### Q106 · ARCH · CRD with validation and defaults (4/4 failed, last 0/7 pts, 2026-09-19T12:27)
### Q106 · ARCH · CRD with validation and defaults (4/4 failed, last 0/7 pts, last failed 2026-09-19T12:27)
- ✗ CRD exists, Namespaced, replicas has min/max and is required
- ✗ good created with environment prod
- ✗ replicas 0 is rejected and the message was saved
- ✗ defaulting works: defaulted.spec.environment == dev
### Q109 · WORK · Native sidecar with probes and an exact report (1/1 failed, last 2/8 pts, 2026-09-19T12:27)
### Q109 · WORK · Native sidecar with probes and an exact report (1/1 failed, last 2/8 pts, last failed 2026-09-19T12:27)
- ✗ native sidecar: initContainer agent with restartPolicy Always
- ✗ probes on app as specified
- ✗ Running, shared log written by agent, grace period 5
### Q112 · NET · TLS Ingress with a generated certificate (4/4 failed, last 0/7 pts, 2026-09-19T12:27)
### Q112 · NET · TLS Ingress with a generated certificate (4/4 failed, last 0/7 pts, last failed 2026-09-19T12:27)
- ✗ Secret secure.lab.local-tls is kubernetes.io/tls with CN secure.lab.local
- ✗ Ingress web: class nginx, host, TLS secret, ssl-redirect annotation
- ✗ fp.txt matches the secret's certificate fingerprint
### Q115 · STOR · Bind the right local volume (3/3 failed, last 1/6 pts, 2026-09-15T20:55)
### Q115 · STOR · Bind the right local volume (3/3 failed, last 1/6 pts, last failed 2026-09-15T20:55)
- ✗ PVC uses a selector and is Bound to lv-nvme
- ✗ consumer Running on the nvme node with /data mounted
### Q120 · TSHOOT · Cluster facts, exactly (3/3 failed, last 0/5 pts, 2026-09-15T20:55)
### Q120 · TSHOOT · Cluster facts, exactly (3/3 failed, last 0/5 pts, last failed 2026-09-15T20:55)
- ✗ pod-cidr and service-cidr
- ✗ dns-ip and proxy-mode
- ✗ cka-sim-worker3 pod CIDR
### Q121 · WORK · The context's namespace (3/3 failed, last 1/6 pts, 2026-09-15T20:55)
### Q121 · WORK · The context's namespace (3/3 failed, last 1/6 pts, last failed 2026-09-15T20:55)
- ✗ Deployment ledger-21 in ac-echo (the context's namespace), 2 available, owner=ac
- ✗ Service ledger-21 port 9000 in the same namespace
- ✗ ns.txt has the namespace
### Q202 · ARCH · Helm chart with CRDs, then a custom resource (1/1 failed, last 3/8 pts, 2026-09-19T12:27)
### Q202 · ARCH · Helm chart with CRDs, then a custom resource (1/1 failed, last 3/8 pts, last failed 2026-09-19T12:27)
- ✗ watchAllNamespaces=true reached the pod env
- ✗ Ledger main-ledger has currency USD and retentionDays 30
### Q204 · WORK · Which Pods get evicted first (1/1 failed, last 0/8 pts, 2026-09-19T12:27)
### Q204 · WORK · Which Pods get evicted first (1/1 failed, last 0/8 pts, last failed 2026-09-19T12:27)
- ✗ evicted-first.txt lists exactly the BestEffort pods
- ✗ api-qos.txt says Burstable
- ✗ guaranteed.txt names orion-ledger
### Q217 · TSHOOT · Inspect a container with crictl (1/1 failed, last 0/10 pts, 2026-09-19T12:27)
### Q217 · TSHOOT · Inspect a container with crictl (1/1 failed, last 0/10 pts, last failed 2026-09-19T12:27)
- ✗ Pod tiger-probe Running with both labels
- ✗ container.txt holds a real container ID for that Pod
- ✗ container.txt records the runtimeType
- ✗ container.log holds the httpd startup output
### Q313 · NET · Two hosts, TLS, and a default backend (1/1 failed, last 0/11 pts, 2026-09-19T12:27)
### Q313 · NET · Two hosts, TLS, and a default backend (1/1 failed, last 0/11 pts, last failed 2026-09-19T12:27)
- ✗ Secret classic-tls is a TLS secret with CN classic.r700.io
- ✗ both host rules route to the right Services on /
- ✗ TLS covers both hosts with classic-tls
@ -151,126 +123,126 @@ _Generated by exam from 11 attempt(s); regenerated on every `end`._
## Attempt matrix
| Q | Domain | Title | D1 | E2 | D3 | D4 | D5 | D6 | D7 | D8 | D9 | D10 | E11 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | ARCH | RBAC for a deployment bot | 2/6 | · | · | ✓ | · | · | · | · | · | · | · |
| 2 | ARCH | etcd snapshot | · | ✓ | · | · | · | · | · | · | · | 0/7 | · |
| 3 | ARCH | Kustomize overlay | · | 0/5 | · | · | · | · | ✓ | · | · | · | · |
| 4 | ARCH | CustomResourceDefinition | · | ✓ | · | · | · | · | · | · | · | ✓ | · |
| 5 | ARCH | Static pod | 0/5 | · | · | · | · | · | ✓ | · | · | · | · |
| 6 | ARCH | Certificates: expiry check and user CSR | 0/6 | · | · | 3/6 | · | · | ✓ | · | · | · | · |
| 7 | WORK | Rolling update & scaling | · | · | · | · | · | · | · | · | · | · | · |
| 8 | WORK | Multi-container pod with shared volume | · | 0/4 | · | · | · | · | ✓ | · | · | ✓ | · |
| 9 | WORK | Scheduling with node affinity and resources | · | · | · | · | · | · | · | · | · | ✓ | · |
| 10 | WORK | ConfigMap and Secret consumption | · | · | · | · | · | · | · | · | · | · | · |
| 11 | NET | NetworkPolicy | · | · | · | · | · | · | · | · | · | · | · |
| 12 | NET | NodePort Service | · | 0/4 | · | · | · | · | ✓ | · | · | · | · |
| 13 | NET | Ingress routing | · | 0/5 | · | · | · | · | 0/5 | ✓ | · | ✓ | · |
| 14 | NET | Cluster DNS | · | · | · | · | · | · | · | · | · | · | · |
| 15 | STOR | PersistentVolume, claim and pod | · | 0/6 | · | · | · | · | 0/6 | 0/6 | ✓ | · | · |
| 16 | STOR | Default StorageClass and dynamic provisioning | · | 0/5 | · | · | · | · | 0/5 | 0/5 | 0/5 | · | · |
| 17 | TSHOOT | Node NotReady | · | 0/8 | · | · | 4/8 | 4/8 | ✓ | · | · | · | · |
| 18 | TSHOOT | Broken deployment and service | · | 0/7 | · | · | · | · | ✓ | · | · | · | · |
| 19 | TSHOOT | CrashLoopBackOff diagnosis | · | 1/6 | · | · | 1/6 | 1/6 | 4/6 | ✓ | · | · | · |
| 20 | TSHOOT | Control plane component failure | · | 0/8 | · | · | · | · | ✓ | · | · | · | · |
| 21 | ARCH | Helm release install and upgrade | · | · | · | ✓ | · | · | · | · | · | · | · |
| 22 | NET | Gateway API HTTPRoute | · | · | · | · | · | · | · | · | · | · | · |
| 23 | WORK | metrics-server repair and HPA | · | · | · | · | · | · | · | · | · | · | · |
| 24 | WORK | PriorityClass | · | · | · | · | · | · | · | · | · | · | · |
| 25 | WORK | Taints and tolerations | · | 0/4 | · | · | · | · | 0/4 | 0/4 | ✓ | ✓ | · |
| 26 | WORK | DaemonSet on every node | · | · | · | · | · | · | · | · | · | · | · |
| 27 | WORK | CronJob with a manual run | · | · | · | · | · | · | · | · | · | · | · |
| 28 | WORK | Init container prepares content | · | 0/4 | · | · | · | · | 0/4 | 0/4 | 0/4 | · | · |
| 29 | WORK | Immutable ConfigMap rotation | · | 1/4 | · | · | · | · | 1/4 | 1/4 | ✓ | · | · |
| 30 | NET | Named ports | · | 0/4 | · | · | · | · | 0/4 | ✓ | · | · | · |
| 31 | WORK | Parallel Job | · | · | · | · | · | · | · | · | · | · | · |
| 32 | WORK | ResourceQuota and LimitRange | · | · | · | · | · | · | · | · | · | 0/4 | · |
| 33 | ARCH | Aggregated ClusterRole | · | 1/5 | · | · | · | · | 1/5 | ✓ | · | · | · |
| 34 | ARCH | Discover a CRD field with kubectl explain | 0/4 | · | · | ✓ | · | · | · | · | · | ✓ | · |
| 35 | STOR | Re-bind a Released PersistentVolume | · | · | · | · | · | · | · | · | · | · | · |
| 36 | STOR | Data survives the pod | · | · | · | · | · | · | · | · | · | · | · |
| 37 | TSHOOT | Cluster DNS is broken | · | 1/6 | · | · | 1/6 | 1/6 | 1/6 | ✓ | · | · | · |
| 38 | TSHOOT | Deployments never create pods | · | · | · | · | 0/7 | 0/7 | ✓ | · | · | · | · |
| 39 | TSHOOT | kubelet misconfiguration | · | · | · | · | 4/7 | 4/7 | 0/7 | ✓ | · | ✓ | · |
| 40 | ARCH | kubeadm join token | · | 0/5 | · | · | · | · | 0/5 | 0/5 | 0/5 | · | · |
| 41 | ARCH | Component versions | 0/4 | · | · | ✓ | · | · | · | · | · | · | · |
| 42 | NET | Egress-only NetworkPolicy | · | 0/4 | · | · | · | · | 0/4 | ✓ | · | · | · |
| 43 | NET | Headless and ExternalName services | · | · | · | · | · | · | · | · | · | · | · |
| 101 | ARCH | Exact RBAC with an existing over-grant | · | · | · | · | · | · | · | · | · | · | · |
| 102 | ARCH | etcd restore | · | · | · | · | · | · | · | · | · | 2/10 | · |
| 103 | ARCH | Static pod on a relocated manifest path | · | · | · | · | · | · | · | · | · | · | · |
| 104 | ARCH | User onboarding to a working kubeconfig | · | · | · | · | · | · | · | · | · | · | · |
| 105 | ARCH | Kustomize with generators and patches | · | · | · | · | · | · | · | · | · | · | · |
| 106 | ARCH | CRD with validation and defaults | · | · | 0/7 | · | · | · | 0/7 | 0/7 | · | 0/7 | · |
| 107 | WORK | Spread, tolerate, prioritise — under a quota | · | · | · | · | · | · | · | · | · | · | · |
| 108 | WORK | Rollback to a specific revision, then pause | · | · | · | · | · | · | · | · | · | · | · |
| 109 | WORK | Native sidecar with probes and an exact report | · | · | · | · | · | · | · | · | · | 2/8 | · |
| 110 | WORK | Cluster audit to exact files | · | · | · | · | · | · | · | · | · | · | · |
| 111 | NET | Zero-trust namespace | · | · | · | · | · | · | · | · | · | · | · |
| 112 | NET | TLS Ingress with a generated certificate | · | · | 0/7 | · | · | · | 0/7 | 0/7 | · | 0/7 | · |
| 113 | NET | Gateway API canary with header match | · | · | · | · | · | · | · | · | · | · | · |
| 114 | NET | DNS down: three faults | · | · | · | · | · | · | · | · | · | · | · |
| 115 | STOR | Bind the right local volume | · | · | 1/6 | · | · | · | 1/6 | 1/6 | · | · | · |
| 116 | STOR | Migrate data between claims | · | · | · | · | · | · | · | · | · | · | · |
| 117 | TSHOOT | Worker node: three things wrong | · | · | · | · | · | · | · | · | · | · | · |
| 118 | TSHOOT | Two control-plane components down | · | · | · | · | · | · | · | · | · | · | · |
| 119 | TSHOOT | Application down for four reasons | · | · | · | · | · | · | · | · | · | · | · |
| 120 | TSHOOT | Cluster facts, exactly | · | · | 0/5 | · | · | · | 0/5 | 0/5 | · | · | · |
| 121 | WORK | The context's namespace | · | · | 1/6 | · | · | · | 1/6 | 1/6 | · | · | · |
| 122 | ARCH | Renew control-plane certificates | · | · | · | · | · | · | · | · | · | · | · |
| 123 | TSHOOT | Nothing works: kube-apiserver itself is down | · | · | · | · | · | · | · | · | · | · | · |
| 124 | ARCH | Prepare a node join without shell access to the control plane | · | · | · | · | · | · | · | · | · | · | · |
| 201 | ARCH | kubeconfig forensics | · | · | · | · | · | · | · | · | · | · | · |
| 202 | ARCH | Helm chart with CRDs, then a custom resource | · | · | · | · | · | · | · | · | · | 3/8 | · |
| 203 | WORK | Find the controller, then scale it | · | · | · | · | · | · | · | · | · | · | · |
| 204 | WORK | Which Pods get evicted first | · | · | · | · | · | · | · | · | · | 0/8 | · |
| 205 | ARCH | Kustomize: drop a ConfigMap, add an HPA, override in prod | · | · | · | · | · | · | · | · | · | · | · |
| 206 | STOR | PV, PVC and a Deployment that mounts it | · | · | · | · | · | · | · | · | · | · | · |
| 207 | TSHOOT | Resource usage scripts | · | · | · | · | · | · | · | · | · | · | · |
| 208 | ARCH | Drain a node for maintenance | · | · | · | · | · | · | · | · | · | · | · |
| 209 | ARCH | Query the API from inside a Pod | · | · | · | · | · | · | · | · | · | · | · |
| 210 | ARCH | ServiceAccount limited to two resources | · | · | · | · | · | · | · | · | · | · | · |
| 211 | WORK | DaemonSet on every node | · | · | · | · | · | · | · | · | · | · | · |
| 212 | WORK | One Pod per node, on purpose | · | · | · | · | · | · | · | · | · | · | · |
| 213 | NET | Convert an Ingress to Gateway API, then split on a header | · | · | · | · | · | · | · | · | · | · | · |
| 214 | ARCH | Certificate expiry, two ways | · | · | · | · | · | · | · | · | · | · | · |
| 215 | NET | Egress policy with paired destinations | · | · | · | · | · | · | · | · | · | · | · |
| 216 | NET | CoreDNS: back up, extend, verify | · | · | · | · | · | · | · | · | · | · | · |
| 217 | TSHOOT | Inspect a container with crictl | · | · | · | · | · | · | · | · | · | 0/10 | · |
| 301 | ARCH | etcd configuration facts | · | · | · | · | · | · | · | · | · | ✓ | · |
| 302 | NET | Service rules on the node | · | · | · | · | · | · | · | · | · | · | · |
| 303 | NET | A second Service CIDR | · | · | · | · | · | · | · | · | · | · | · |
| 304 | ARCH | Bound tokens versus legacy tokens | · | · | · | · | · | · | · | · | · | · | · |
| 305 | WORK | Schedule without the scheduler | · | · | · | · | · | · | · | · | · | · | · |
| 306 | STOR | Grow a volume in place | · | · | · | · | · | · | · | · | · | · | · |
| 307 | TSHOOT | Node accepts nothing new | · | · | · | · | · | · | · | · | · | · | · |
| 308 | ARCH | API resource reference card | · | · | · | · | · | · | · | · | · | · | · |
| 309 | WORK | A PodDisruptionBudget that blocks maintenance | · | · | · | · | · | · | · | · | · | · | · |
| 310 | NET | Route to something outside the cluster | · | · | · | · | · | · | · | · | · | · | · |
| 311 | ARCH | Image inventory across the cluster | · | · | · | · | · | · | · | · | · | · | · |
| 312 | WORK | A Job that fails, then one that doesn't | · | · | · | · | · | · | · | · | · | · | · |
| 313 | NET | Two hosts, TLS, and a default backend | · | · | · | · | · | · | · | · | · | 0/11 | · |
| 314 | TSHOOT | A static pod that never starts | · | · | · | · | · | · | · | · | · | · | · |
| 315 | STOR | Keep the data when the claim goes | · | · | · | · | · | · | · | · | · | · | · |
| 316 | ARCH | Plan a control-plane upgrade | · | · | · | · | · | · | · | · | · | · | · |
| 317 | TSHOOT | Certificates stop being issued | · | · | · | · | · | · | · | · | · | · | · |
| 401 | STOR | Provision geo-fence persistent storage | · | · | · | · | · | · | · | · | · | · | ✓ |
| 402 | ARCH | Render a chart, then install it | · | · | · | · | · | · | · | · | · | · | ✓ |
| 403 | TSHOOT | Fix the driver onboarding app | · | · | · | · | · | · | · | · | · | · | ✓ |
| 404 | NET | Expose and verify the dispatch API | · | · | · | · | · | · | · | · | · | · | ✓ |
| 405 | STOR | Migrate ride history to a Retain volume | · | · | · | · | · | · | · | · | · | · | ✓ |
| 406 | TSHOOT | Fix ETA service connectivity | · | · | · | · | · | · | · | · | · | · | ✓ |
| 407 | NET | Fleet dashboard canary with Gateway API | · | · | · | · | · | · | · | · | · | · | ✓ |
| 408 | ARCH | Cluster-wide read access for the auth team | · | · | · | · | · | · | · | · | · | · | ✓ |
| 409 | WORK | Add an init container to the trip logger | · | · | · | · | · | · | · | · | · | · | ✓ |
| 410 | NET | Audit the CNI, then lock a namespace down | · | · | · | · | · | · | · | · | · | · | ✓ |
| 411 | WORK | Priority for the rider support portal | · | · | · | · | · | · | · | · | · | · | ✓ |
| 412 | WORK | Surge pricing configuration | · | · | · | · | · | · | · | · | · | · | ✓ |
| 413 | ARCH | Plan the control-plane upgrade | · | · | · | · | · | · | · | · | · | · | ✓ |
| 414 | TSHOOT | Fix the fare calculation pods | · | · | · | · | · | · | · | · | · | · | ✓ |
| 415 | WORK | Autoscale the route planner on memory | · | · | · | · | · | · | · | · | · | · | ✓ |
| 416 | NET | NetworkPolicy for the safety monitor | · | · | · | · | · | · | · | · | · | · | ✓ |
| 417 | ARCH | Enforce TLS 1.3 on the booking portal | · | · | · | · | · | · | · | · | · | · | ✓ |
| 418 | TSHOOT | Investigate vehicle tracker health | · | · | · | · | · | · | · | · | · | · | ✓ |
| Q | Domain | Title | D1 | E2 | D3 | D4 | D5 | D6 | D7 | D8 | D9 | D10 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | ARCH | RBAC for a deployment bot | 2/6 | · | · | ✓ | · | · | · | · | · | · |
| 2 | ARCH | etcd snapshot | · | ✓ | · | · | · | · | · | · | · | 0/7 |
| 3 | ARCH | Kustomize overlay | · | 0/5 | · | · | · | · | ✓ | · | · | · |
| 4 | ARCH | CustomResourceDefinition | · | ✓ | · | · | · | · | · | · | · | ✓ |
| 5 | ARCH | Static pod | 0/5 | · | · | · | · | · | ✓ | · | · | · |
| 6 | ARCH | Certificates: expiry check and user CSR | 0/6 | · | · | 3/6 | · | · | ✓ | · | · | · |
| 7 | WORK | Rolling update & scaling | · | · | · | · | · | · | · | · | · | · |
| 8 | WORK | Multi-container pod with shared volume | · | 0/4 | · | · | · | · | ✓ | · | · | ✓ |
| 9 | WORK | Scheduling with node affinity and resources | · | · | · | · | · | · | · | · | · | ✓ |
| 10 | WORK | ConfigMap and Secret consumption | · | · | · | · | · | · | · | · | · | · |
| 11 | NET | NetworkPolicy | · | · | · | · | · | · | · | · | · | · |
| 12 | NET | NodePort Service | · | 0/4 | · | · | · | · | ✓ | · | · | · |
| 13 | NET | Ingress routing | · | 0/5 | · | · | · | · | 0/5 | ✓ | · | ✓ |
| 14 | NET | Cluster DNS | · | · | · | · | · | · | · | · | · | · |
| 15 | STOR | PersistentVolume, claim and pod | · | 0/6 | · | · | · | · | 0/6 | 0/6 | ✓ | · |
| 16 | STOR | Default StorageClass and dynamic provisioning | · | 0/5 | · | · | · | · | 0/5 | 0/5 | 0/5 | · |
| 17 | TSHOOT | Node NotReady | · | 0/8 | · | · | 4/8 | 4/8 | ✓ | · | · | · |
| 18 | TSHOOT | Broken deployment and service | · | 0/7 | · | · | · | · | ✓ | · | · | · |
| 19 | TSHOOT | CrashLoopBackOff diagnosis | · | 1/6 | · | · | 1/6 | 1/6 | 4/6 | ✓ | · | · |
| 20 | TSHOOT | Control plane component failure | · | 0/8 | · | · | · | · | ✓ | · | · | · |
| 21 | ARCH | Helm release install and upgrade | · | · | · | ✓ | · | · | · | · | · | · |
| 22 | NET | Gateway API HTTPRoute | · | · | · | · | · | · | · | · | · | · |
| 23 | WORK | metrics-server repair and HPA | · | · | · | · | · | · | · | · | · | · |
| 24 | WORK | PriorityClass | · | · | · | · | · | · | · | · | · | · |
| 25 | WORK | Taints and tolerations | · | 0/4 | · | · | · | · | 0/4 | 0/4 | ✓ | ✓ |
| 26 | WORK | DaemonSet on every node | · | · | · | · | · | · | · | · | · | · |
| 27 | WORK | CronJob with a manual run | · | · | · | · | · | · | · | · | · | · |
| 28 | WORK | Init container prepares content | · | 0/4 | · | · | · | · | 0/4 | 0/4 | 0/4 | · |
| 29 | WORK | Immutable ConfigMap rotation | · | 1/4 | · | · | · | · | 1/4 | 1/4 | ✓ | · |
| 30 | NET | Named ports | · | 0/4 | · | · | · | · | 0/4 | ✓ | · | · |
| 31 | WORK | Parallel Job | · | · | · | · | · | · | · | · | · | · |
| 32 | WORK | ResourceQuota and LimitRange | · | · | · | · | · | · | · | · | · | 0/4 |
| 33 | ARCH | Aggregated ClusterRole | · | 1/5 | · | · | · | · | 1/5 | ✓ | · | · |
| 34 | ARCH | Discover a CRD field with kubectl explain | 0/4 | · | · | ✓ | · | · | · | · | · | ✓ |
| 35 | STOR | Re-bind a Released PersistentVolume | · | · | · | · | · | · | · | · | · | · |
| 36 | STOR | Data survives the pod | · | · | · | · | · | · | · | · | · | · |
| 37 | TSHOOT | Cluster DNS is broken | · | 1/6 | · | · | 1/6 | 1/6 | 1/6 | ✓ | · | · |
| 38 | TSHOOT | Deployments never create pods | · | · | · | · | 0/7 | 0/7 | ✓ | · | · | · |
| 39 | TSHOOT | kubelet misconfiguration | · | · | · | · | 4/7 | 4/7 | 0/7 | ✓ | · | ✓ |
| 40 | ARCH | kubeadm join token | · | 0/5 | · | · | · | · | 0/5 | 0/5 | 0/5 | · |
| 41 | ARCH | Component versions | 0/4 | · | · | ✓ | · | · | · | · | · | · |
| 42 | NET | Egress-only NetworkPolicy | · | 0/4 | · | · | · | · | 0/4 | ✓ | · | · |
| 43 | NET | Headless and ExternalName services | · | · | · | · | · | · | · | · | · | · |
| 101 | ARCH | Exact RBAC with an existing over-grant | · | · | · | · | · | · | · | · | · | · |
| 102 | ARCH | etcd restore | · | · | · | · | · | · | · | · | · | 2/10 |
| 103 | ARCH | Static pod on a relocated manifest path | · | · | · | · | · | · | · | · | · | · |
| 104 | ARCH | User onboarding to a working kubeconfig | · | · | · | · | · | · | · | · | · | · |
| 105 | ARCH | Kustomize with generators and patches | · | · | · | · | · | · | · | · | · | · |
| 106 | ARCH | CRD with validation and defaults | · | · | 0/7 | · | · | · | 0/7 | 0/7 | · | 0/7 |
| 107 | WORK | Spread, tolerate, prioritise — under a quota | · | · | · | · | · | · | · | · | · | · |
| 108 | WORK | Rollback to a specific revision, then pause | · | · | · | · | · | · | · | · | · | · |
| 109 | WORK | Native sidecar with probes and an exact report | · | · | · | · | · | · | · | · | · | 2/8 |
| 110 | WORK | Namespace audit to exact files | · | · | · | · | · | · | · | · | · | · |
| 111 | NET | Zero-trust namespace | · | · | · | · | · | · | · | · | · | · |
| 112 | NET | TLS Ingress with a generated certificate | · | · | 0/7 | · | · | · | 0/7 | 0/7 | · | 0/7 |
| 113 | NET | Gateway API canary with header match | · | · | · | · | · | · | · | · | · | · |
| 114 | NET | DNS down: three faults | · | · | · | · | · | · | · | · | · | · |
| 115 | STOR | Bind the right local volume | · | · | 1/6 | · | · | · | 1/6 | 1/6 | · | · |
| 116 | STOR | Migrate data between claims | · | · | · | · | · | · | · | · | · | · |
| 117 | TSHOOT | Worker node: three things wrong | · | · | · | · | · | · | · | · | · | · |
| 118 | TSHOOT | Two control-plane components down | · | · | · | · | · | · | · | · | · | · |
| 119 | TSHOOT | Application down for four reasons | · | · | · | · | · | · | · | · | · | · |
| 120 | TSHOOT | Cluster facts, exactly | · | · | 0/5 | · | · | · | 0/5 | 0/5 | · | · |
| 121 | WORK | The context's namespace | · | · | 1/6 | · | · | · | 1/6 | 1/6 | · | · |
| 122 | ARCH | Renew control-plane certificates | · | · | · | · | · | · | · | · | · | · |
| 123 | TSHOOT | Nothing works: kube-apiserver itself is down | · | · | · | · | · | · | · | · | · | · |
| 124 | ARCH | Prepare a node join without shell access to the control plane | · | · | · | · | · | · | · | · | · | · |
| 201 | ARCH | kubeconfig forensics | · | · | · | · | · | · | · | · | · | · |
| 202 | ARCH | Helm chart with CRDs, then a custom resource | · | · | · | · | · | · | · | · | · | 3/8 |
| 203 | WORK | Find the controller, then scale it | · | · | · | · | · | · | · | · | · | · |
| 204 | WORK | Which Pods get evicted first | · | · | · | · | · | · | · | · | · | 0/8 |
| 205 | ARCH | Kustomize: drop a ConfigMap, add an HPA, override in prod | · | · | · | · | · | · | · | · | · | · |
| 206 | STOR | PV, PVC and a Deployment that mounts it | · | · | · | · | · | · | · | · | · | · |
| 207 | TSHOOT | Resource usage scripts | · | · | · | · | · | · | · | · | · | · |
| 208 | ARCH | Drain a node for maintenance | · | · | · | · | · | · | · | · | · | · |
| 209 | ARCH | Query the API from inside a Pod | · | · | · | · | · | · | · | · | · | · |
| 210 | ARCH | ServiceAccount limited to two resources | · | · | · | · | · | · | · | · | · | · |
| 211 | WORK | DaemonSet on every node | · | · | · | · | · | · | · | · | · | · |
| 212 | WORK | One Pod per node, on purpose | · | · | · | · | · | · | · | · | · | · |
| 213 | NET | Convert an Ingress to Gateway API, then split on a header | · | · | · | · | · | · | · | · | · | · |
| 214 | ARCH | Certificate expiry, two ways | · | · | · | · | · | · | · | · | · | · |
| 215 | NET | Egress policy with paired destinations | · | · | · | · | · | · | · | · | · | · |
| 216 | NET | CoreDNS: back up, extend, verify | · | · | · | · | · | · | · | · | · | · |
| 217 | TSHOOT | Inspect a container with crictl | · | · | · | · | · | · | · | · | · | 0/10 |
| 301 | ARCH | etcd configuration facts | · | · | · | · | · | · | · | · | · | ✓ |
| 302 | NET | Service rules on the node | · | · | · | · | · | · | · | · | · | · |
| 303 | NET | A second Service CIDR | · | · | · | · | · | · | · | · | · | · |
| 304 | ARCH | Bound tokens versus legacy tokens | · | · | · | · | · | · | · | · | · | · |
| 305 | WORK | Schedule without the scheduler | · | · | · | · | · | · | · | · | · | · |
| 306 | STOR | Grow a volume in place | · | · | · | · | · | · | · | · | · | · |
| 307 | TSHOOT | Node accepts nothing new | · | · | · | · | · | · | · | · | · | · |
| 308 | ARCH | API resource reference card | · | · | · | · | · | · | · | · | · | · |
| 309 | WORK | A PodDisruptionBudget that blocks maintenance | · | · | · | · | · | · | · | · | · | · |
| 310 | NET | Route to something outside the cluster | · | · | · | · | · | · | · | · | · | · |
| 311 | ARCH | Image inventory across the cluster | · | · | · | · | · | · | · | · | · | · |
| 312 | WORK | A Job that fails, then one that doesn't | · | · | · | · | · | · | · | · | · | · |
| 313 | NET | Two hosts, TLS, and a default backend | · | · | · | · | · | · | · | · | · | 0/11 |
| 314 | TSHOOT | A static pod that never starts | · | · | · | · | · | · | · | · | · | · |
| 315 | STOR | Keep the data when the claim goes | · | · | · | · | · | · | · | · | · | · |
| 316 | ARCH | Plan a control-plane upgrade | · | · | · | · | · | · | · | · | · | · |
| 317 | TSHOOT | Certificates stop being issued | · | · | · | · | · | · | · | · | · | · |
| 401 | STOR | Provision geo-fence persistent storage | · | · | · | · | · | · | · | · | · | · |
| 402 | ARCH | Render a chart, then install it | · | · | · | · | · | · | · | · | · | · |
| 403 | TSHOOT | Fix the driver onboarding app | · | · | · | · | · | · | · | · | · | · |
| 404 | NET | Expose and verify the dispatch API | · | · | · | · | · | · | · | · | · | · |
| 405 | STOR | Migrate ride history to a Retain volume | · | · | · | · | · | · | · | · | · | · |
| 406 | TSHOOT | Fix ETA service connectivity | · | · | · | · | · | · | · | · | · | · |
| 407 | NET | Fleet dashboard canary with Gateway API | · | · | · | · | · | · | · | · | · | · |
| 408 | ARCH | Cluster-wide read access for the auth team | · | · | · | · | · | · | · | · | · | · |
| 409 | WORK | Add an init container to the trip logger | · | · | · | · | · | · | · | · | · | · |
| 410 | NET | Audit the CNI, then lock a namespace down | · | · | · | · | · | · | · | · | · | · |
| 411 | WORK | Priority for the rider support portal | · | · | · | · | · | · | · | · | · | · |
| 412 | WORK | Surge pricing configuration | · | · | · | · | · | · | · | · | · | · |
| 413 | ARCH | Plan the control-plane upgrade | · | · | · | · | · | · | · | · | · | · |
| 414 | TSHOOT | Fix the fare calculation pods | · | · | · | · | · | · | · | · | · | · |
| 415 | WORK | Autoscale the route planner on memory | · | · | · | · | · | · | · | · | · | · |
| 416 | NET | NetworkPolicy for the safety monitor | · | · | · | · | · | · | · | · | · | · |
| 417 | ARCH | Enforce TLS 1.3 on the booking portal | · | · | · | · | · | · | · | · | · | · |
| 418 | TSHOOT | Investigate vehicle tracker health | · | · | · | · | · | · | · | · | · | · |
_E = full exam, D = drill, · = not in that session, n/m = partial credit._

View File

@ -186,8 +186,7 @@
<button class="opt" id="optNext"><b>Drill next domain</b><span id="nextDesc"></span></button>
<button class="opt" id="optAll"><b>Drill the whole bank</b><span id="allDesc"></span></button>
<button class="opt" id="optDrill"><b>Drill: weak / pick</b><span>Tasks you're weak on, or choose your own. Feedback per task while you work.</span></button>
<button class="opt" id="optSimA"><b>Simulator Set A</b><span id="simADesc"></span></button>
<button class="opt" id="optSimB"><b>Simulator Set B</b><span id="simBDesc"></span></button>
<span id="simOpts" style="display:contents"></span>
<button class="opt" id="optExam" style="grid-column:1/3"><b>Full exam</b><span id="examDesc"></span></button>
</div>
<div id="gate" class="hidden" style="color:var(--warn);margin-top:10px;font-size:13px"></div>
@ -348,11 +347,11 @@
const eligible = S.bank.filter(q => hard || q.difficulty !== 'hard'), seenN = eligible.filter(q => (S.seen || []).includes(q.id)).length;
$('#allDesc').textContent = `${seenN}/${eligible.length} tasks attempted. 5 unseen tasks per session, round-robin across domains${hard ? ', hard first' : ''}; then whatever you failed.`;
$('#nextDesc').textContent = `${S.track.domains[nxt]} — 5 tasks, ${hard ? 5 : 6} min each. Drills rotate through all domains before the exam.`;
for (const s of ['a', 'b']) {
const info = (S.sets || {})[s];
const el = $('#sim' + s.toUpperCase() + 'Desc');
if (el) el.textContent = info ? `${info.count} fixed tasks, ${S.track.minutes} min, exam rules. Hardest set in the bank — closest to a real sitting.` : '';
}
// one button per fixed set the track defines (Set C and later were unreachable from here when
// the buttons were hard-coded to A and B)
$('#simOpts').innerHTML = Object.entries(S.sets || {}).map(([k, info]) =>
`<button class="opt" data-sim="${k}"><b>${esc(info.name)}</b><span>${info.count} fixed tasks, ${S.track.minutes} min, exam rules, no feedback until you submit.</span></button>`).join('');
$('#simOpts').querySelectorAll('[data-sim]').forEach(el => el.onclick = () => startSim(el.dataset.sim));
$('#gate').classList.toggle('hidden', !undone.length);
$('#gate').innerHTML = undone.length ? `Drill every domain once before the full exam. Not yet drilled: ${undone.join(', ')}. <a href="#" id="forceExam">Start the exam anyway</a>` : '';
$('#optExam').disabled = !!undone.length; $('#optExam').style.opacity = undone.length ? .5 : 1;
@ -451,8 +450,6 @@
$('#optNext').onclick = () => start('drill', null, true);
$('#optStudy').onclick = () => start('drill', null, true, false, null, true);
$('#optAll').onclick = () => start('drill', null, false, true);
$('#optSimA').onclick = () => startSim('a');
$('#optSimB').onclick = () => startSim('b');
$('#hardMode').onchange = () => load();
$('#optDrill').onclick = async () => {
const h = await api('/api/history');