Go to file
vumast3r 49fcb76166 fix: taint cleanup removed nothing, Q23 leaked a broken metrics-server, CKS lost 40% of its blueprint
Audit of the bank and harness after a 23% mock result. Two of these were
silently corrupting every practice session on this cluster.

restore_baseline:

  - The w1-taint cleanup removed nothing at all. `kubectl taint node N a- b- c-`
    aborts on the first key that is not present and applies none of the removals,
    so with only `workload` set the command reported five "not found" lines,
    exited 1, and left the taint in place - the precise failure the block's own
    comment says it exists to prevent. W1 then rejected pods in every later
    session. Now one call per key. Verified by re-seeding workload=gpu:NoSchedule
    and confirming a single restore_baseline run clears it.
  - metrics-server was never repaired. Q23 seeds its scenario by stripping
    --kubelet-insecure-tls, but carried no "breaks" key, so nothing tracked it
    and nothing undid it. On kind the kubelet serving certs have no IP SANs, so
    without that flag metrics-server never goes Ready, v1beta1.metrics.k8s.io
    has no endpoints, aggregated discovery goes stale, and EVERY namespace
    deletion in the cluster hangs on NamespaceDeletionDiscoveryFailure - which
    is where the ~40 stuck namespaces came from, plus a dead `kubectl top`.
    Q23 now declares breaks: metrics-server and the baseline restores the flag.
    Baseline runs before seeding, so a session containing Q23 re-breaks it a
    moment later as intended. Verified by stripping the flag and confirming one
    restore_baseline run puts it back.

Session composition:

  - pick() dropped any domain whose tasks are all "hard" from a normal session.
    CKS SUPPLY and RUNTIME hold one hard task each, so `./exam start` returned
    6 of 9 tasks with 40% of the blueprint silently absent. It now tops up from
    the hard pool when a domain's normal pool is short.
  - compose_exam() iterated QUOTA in declaration order, so a one-task domain
    lost every "breaks" tie to a bigger domain that claimed the shared slot
    first: CKS RUNTIME (only Q9, apiserver-flag) was never drawn in ANY mode
    because SETUP went first and took that slot with Q2. Now walks domains
    scarcest-first, and warns on a shortfall instead of truncating in silence.
    CKS now yields 8/9 covering all six domains, warning SETUP 1/2.
  - session_questions() rebuilt the list by filtering QUESTIONS (sorted by id),
    discarding the session's own order - so compose_exam's rng.shuffle never
    reached the candidate and every exam arrived grouped by domain and
    difficulty, base tasks then hard then sim sets. Fixed to preserve the stored
    order, and cmd_drill now shuffles explicit-id drills after conflict
    resolution, so which task gets dropped stays deterministic while the order
    does not.

Bank correctness (verified by grading each reference solution):

  - CKAD Q2 was unpassable, 5/8. Its init container waited on `until nslookup
    backend`, but busybox nslookup exits 1 even on a successful lookup because
    it walks the whole search list and the trailing NXDOMAINs set the status -
    the loop never terminates. The premise was wrong too: a ClusterIP Service
    resolves from creation, endpoints or not, so DNS could never have gated it.
    Now waits on a real connection; 8/8, and the pod genuinely holds at Init:0/1
    until the backend Deployment exists. Same bug class as e3f5ecd.
  - CKS Q1 was unsatisfiable as written, 6/10. The text asked for three
    NetworkPolicies but the grader required a fourth: one podSelector cannot
    cover both the web egress half and the db ingress half. Task text now asks
    for four and says why.
  - CKS Q4 check 4 was a false positive. `... || r=0` read a failed exec as
    "no token mounted", awarding 3 points when the candidate had done nothing
    and the probe pod simply could not start - which happens often on this
    cluster. Now requires the pod to reach Running and exec to work. Verified
    it awards the points before the fix and fails after, with pod creation
    blocked by a quota.
  - CKAD Q8's solution asserted a Pod's image cannot be changed in place.
    It can - spec.containers[].image is mutable; `kubectl set image pod/...`
    works and grades 7/7. Rewritten, noting which fields really are immutable.

Also: the `sim` usage and help strings hardcoded "a|b" and a "17-task" count,
wrong for a third set and wrong for CKAD/CKS, which define no sets at all. Both
are now generated from SETS.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NziZdvBvy7CpEAAbZNgp8i
2026-09-18 09:05:03 -06:00
tracks fix: taint cleanup removed nothing, Q23 leaked a broken metrics-server, CKS lost 40% of its blueprint 2026-09-18 09:05:03 -06:00
web style: redesign exam web UI with a dark developer theme 2026-09-12 10:24:38 -06:00
.gitignore fix: repair apiserver/staticPodPath leaks, 3 drifted solutions, workspace tracking 2026-09-14 22:09:26 -06:00
CKA_EXAM_GUIDE.md docs: add CKA exam preparation guide and cheat sheet 2026-09-07 08:44:55 -06:00
CKA_SOLUTIONS_EXPLAINED.md feat: add kube-apiserver failure and node-join-verification tasks 2026-09-10 21:25:18 -06:00
etcd-tools.sh fix: install etcdctl/etcdutl on the control-plane node for the etcd tasks 2026-09-07 13:51:37 -06:00
exam fix: taint cleanup removed nothing, Q23 leaked a broken metrics-server, CKS lost 40% of its blueprint 2026-09-18 09:05:03 -06:00
examrc feat: add CKA/CKAD/CKS exam simulator and question banks 2026-09-06 19:06:10 -06:00
examrc.fish feat: add CKA/CKAD/CKS exam simulator and question banks 2026-09-06 19:06:10 -06:00
history-cka.jsonl docs: updated docs for current cka track. exam on Sat. 2026-09-16 19:48:09 -06:00
kind-config.yaml feat: add CKA/CKAD/CKS exam simulator and question banks 2026-09-06 19:06:10 -06:00
LICENSE Initial commit 2026-09-07 00:45:37 +00:00
node-tools.sh feat: add CKA/CKAD/CKS exam simulator and question banks 2026-09-06 19:06:10 -06:00
README.md fix: install etcdctl/etcdutl on the control-plane node for the etcd tasks 2026-09-07 13:51:37 -06:00
setup.sh fix: use kind load image-archive instead of docker-image for podman 2026-09-12 10:17:36 -06:00
teardown.sh fix: repair apiserver/staticPodPath leaks, 3 drifted solutions, workspace tracking 2026-09-14 22:09:26 -06:00
weak-spots-cka.md docs: updated docs for current cka track. exam on Sat. 2026-09-16 19:48:09 -06:00

cka-sim — Kubernetes certification exam simulator

Entry point is ./exam (renamed from cka-sim so it can't collide with the directory name).

Hands-on exam tasks graded per check against a real cluster, an exam clock, a browser front end modelled on the real PSI exam, attempt history, and drills that rotate through the domains before you sit a full exam. Tracks: CKA (65-task bank: 43 normal + 22 hard), CKAD and CKS (scaffolded — same format, add questions when you get there).

How a session is built

Every session draws fresh tasks: start samples per domain in the exam's proportions (ARCH 5, WORK 4, NET 4, STOR 2, TSHOOT 5 = 20), preferring tasks you have never seen, then the least-recently seen. Most tasks are parameterized — namespaces, names, replica counts, ports, which node is broken — so even a repeat isn't the same task. drill with no arguments picks the domain you've drilled least and gives you 5 tasks from it; a full exam is gated until every domain has been drilled at least once (--force skips that).

Hard mode (--hard, or the toggle in the UI lobby) is modelled on what people report about killer.sh, not on its content: 25 tasks in the same 120 minutes, hard tasks preferred, no hints about paths or nodes, traps (an over-granting binding you must find, an allow-all NetworkPolicy left behind, a relocated staticPodPath, two PVs where only one is right), exact-output tasks graded byte-for-byte, compound failures (three faults on one node, two control-plane components down for different reasons, DNS broken three ways), collateral-damage checks (still cordoned, recreated instead of fixed in place, extra PVs), and exam-style contexts k8s-c1-H / k8s-c2-AC / k8s-c3-CCC with different default namespaces — each task says which to use. Grading always runs against the real cluster context regardless of which one you're on.

Why kind and not k3s / hosted labs

kind is kubeadm-based: real etcd, static-pod control plane, systemd kubelet on every node, kubeadm certs on the control plane. That is what lets the troubleshooting tasks break real things (kubelet, scheduler manifest) the way the exam does. Hosted labs and CK-X give you no node access, which rules out exactly those tasks. podman exec -it <node> bash plays the role of ssh <node> here; use the Vagrant/ssh mode below if you want a real ssh session.

Requirements

podman, kind, kubectl, python3, ttyd (for the terminal pane in the browser UI). ~4 GB RAM for the 4-node cluster. Rootless podman needs cgroup v2; otherwise sudo -E ./setup.sh. Docker still works: CONTAINER_RUNTIME=docker ./setup.sh.

Run

./setup.sh          # create the kind cluster, preload images (idempotent)
./exam web       # open http://127.0.0.1:8080 — start an exam or drill from there

Both the built-in terminal and your own shell can have the exam aliases:

source ./examrc          # bash: k=kubectl, $do, $now, kubectl completion (also for k), vim defaults
source ./examrc.fish     # fish equivalent

./exam web sources the right one automatically for its terminal pane.

That's the whole loop. The UI seeds the scenarios, runs the clock, lets you flag questions and keep notes, submits (or auto-submits at 00:00), shows the score with every check, and offers "Drill what I missed". The terminal pane is a real bash via ttyd with kubectl already pointed at the cluster (--shell fish if you'd rather practise in fish; the real exam is bash).

The same state is available from the terminal if you prefer:

./exam drill                      # next least-drilled domain, 5 tasks (add --hard)
./exam drill weak | drill 2 17 20 | drill TSHOOT NET
./exam start [--hard]             # full exam (gated until all domains drilled; --force)
./exam list | show N | timer | status                       # during
./exam end                                                  # submit
./exam weak | history                                       # afterwards
./exam --track ckad web                                     # other tracks
./teardown.sh                                                  # delete cluster + state (history kept)

Scoring, history, drills

Each task is a list of named, weighted checks (tracks/cka/questions.py). You get the points for every check that passes, so a task can score 4/6 — the same partial-credit model as the exam. Every submit appends to history-<track>.jsonl and regenerates weak-spots-<track>.md: latest domain scores, a "drill next" section listing the failing checks per task, and a ✓ / n-of-m matrix across all attempts.

A task counts as weak if it failed most recently, has failed twice ever, or was never attempted. Drills seed only those tasks, give 6 minutes per task, and — unlike exam mode — let you check a task while you work. A task stops being weak only when you pass it again under the clock.

Using your own cluster instead of kind

Any kubeadm-style cluster with 1 control-plane + 3 workers works. Skip setup.sh, set (Fish):

set -x CKA_CONTEXT     kubernetes-admin@kubernetes
set -x CKA_NODE_ACCESS ssh        # podman | docker | ssh | vagrant
set -x CKA_CP cp1;  set -x CKA_W1 node1;  set -x CKA_W2 node2;  set -x CKA_W3 node3
./exam web

ssh needs passwordless ssh <node-name> (for Vagrant: vagrant ssh-config >> ~/.ssh/config); vagrant runs vagrant ssh from the Vagrantfile directory. Tasks 2 and 20 write to /opt/cka and /etc/kubernetes/manifests, so the login must be able to. Managed clusters (EKS/AKS/GKE) won't work — no node or control-plane access. Startup preflights all of this.

The etcd tasks (2 and 102) need etcdctl/etcdutl on the control-plane node, as the real exam has them — they cannot be copied out of the running pod, since the etcd image is distroless. setup.sh installs them on kind; on any cluster, seeding those tasks installs them if they are missing (from the etcd image in the node's containerd, else the upstream release, else apt) and fails loudly if it cannot. ./etcd-tools.sh does the same by hand.

CKA bank

Domain Normal (43) Hard (22)
Architecture RBAC, etcd snapshot, Kustomize, CRD, static pod, certs/CSR, Helm, aggregated ClusterRole, kubectl explain on a CRD, kubeadm join token, component versions exact RBAC with an over-grant to remove, etcd restore, static pod on a relocated path, user → self-contained kubeconfig, Kustomize generators + patches, CRD validation/defaults, renew all control-plane certs
Workloads rolling update, sidecar, node affinity + resources, ConfigMap/Secret, metrics-server repair + HPA, PriorityClass, taints, DaemonSet, CronJob, init container, immutable ConfigMap rotation, parallel Job, quota + LimitRange anti-affinity + toleration + priority under a quota, rollback to a specific revision then pause, native sidecar + probes + exact report, cluster audit to exact files, work in the context's namespace
Networking NetworkPolicy, NodePort, Ingress, cluster DNS, Gateway API, named ports, egress-only policy, headless + ExternalName zero-trust namespace (4 policies + trap), TLS Ingress with generated cert + fingerprint, Gateway canary with header match and weights, DNS down three ways
Storage PV/PVC/pod, default StorageClass, re-bind a Released PV, data survives the pod bind the right local volume by selector, migrate data between claims via a Job
Troubleshooting node NotReady, broken deployment + service, CrashLoopBackOff, scheduler down, CoreDNS misconfig, controller-manager down, kubelet config drift worker with three faults (+ uncordon check), two control-plane components down, app down for four reasons fixed in place (UID check), cluster facts exactly

Disruptive tasks (anything that breaks a node or control-plane component) are seeded last and can block other tasks until fixed — read everything first, as in the real exam. kindnet doesn't enforce NetworkPolicy and no ingress/gateway controller runs, so those are graded on the objects.

Adding tasks or a new track

Banks live in tracks/cka/{questions,bank_extra,bank_hard}.py; a task is:

{"id": 44, "domain": "NET", "difficulty": "normal",   # or "hard"
 "params": lambda r: {"ns": ns_pick(r, "svc"), "port": r.choice([8080, 9090])},
 "title": "...", "task": "...markdown; %{ns} and %{port} are substituted; `code` copies on click...",
 "setup": "bash that seeds the scenario (may use %{ns})",
 "checks": [C("what this check proves", 2, "bash; exit 0 = pass"), ...],
 "disruptive": False}

Scripts get $K (pinned to the grading context), $WS, $CP/$W1/$W2/$W3, nx NODE 'cmd', and helpers j <get args> '{jsonpath}' and eq a b. Task text may use {WS}, {CP}, {W2}, {W3}, {SHELL_CP} etc. TRACK["quota"] / ["quota_hard"] set how many tasks per domain a session draws. For CKAD/CKS, fill in QUESTIONS in the existing placeholder track; TRACK already carries the domain table and pass mark.