diff --git a/CKAD_STUDY_GUIDE.md b/CKAD_STUDY_GUIDE.md
new file mode 100644
index 0000000..664deac
--- /dev/null
+++ b/CKAD_STUDY_GUIDE.md
@@ -0,0 +1,1033 @@
+# CKAD study guide
+
+The 24 objectives of the Certified Kubernetes Application Developer curriculum, in its own words and order: what to know, the fastest commands, the traps, the documentation pages the exam lets you open, and the tasks in this simulator that practise each one.
+
+_Generated by `./exam --track ckad notes all --md` from `tracks/ckad/curriculum.py` - edit that file, not this one._
+
+```
+./exam objectives # the path and your progress
+./exam study 3.2 # one objective: these notes, then its tasks, no clock, solutions open
+./exam drill 3.2 # the same tasks against the clock
+```
+
+## Application Design and Build (20%)
+
+### 1.1 Define, build and modify container images
+
+`./exam study 1.1` (or `images`)
+
+**Know**
+
+- Read and change a Dockerfile: FROM, COPY, RUN, ENV, ARG (build time only), WORKDIR, USER, LABEL, EXPOSE, ENTRYPOINT, CMD.
+- ENTRYPOINT is the program, CMD its default arguments. In a Pod, `command` replaces ENTRYPOINT and `args` replaces CMD.
+- Build, tag, list, inspect, run, save and remove images. The exam terminal has docker or podman - the verbs are the same.
+- A tag is only a pointer: one image can carry several tags, and rebuilding under an old tag moves it.
+
+**Fast**
+
+```
+podman build -t localhost/app:v1 . # -f path/Dockerfile --build-arg KEY=val
+podman tag localhost/app:v1 localhost/app:latest
+podman images | grep app
+podman inspect localhost/app:v1 --format '{{.Config.User}} {{.Config.Entrypoint}} {{.Config.Cmd}} {{.Config.Env}}'
+podman run --rm localhost/app:v1 [args replacing CMD] # -d --name x keeps it running; podman logs x
+podman save --format oci-archive -o app.tar localhost/app:v1 # docker save -o app.tar ...
+podman rmi localhost/app:v0
+```
+
+**Traps**
+
+- An ARG is gone at run time unless you copy it into an ENV (`ARG V` then `ENV APP_VERSION=$V`).
+- `podman save` writes a docker-archive unless you pass `--format oci-archive`.
+- Build before you edit when a task wants the old version kept under its own tag.
+- imagePullPolicy defaults to Always for `:latest` (or no tag) and IfNotPresent for any other tag.
+
+**Docs**
+
+-
+-
+
+**Tasks**
+
+- Q13 Build, change and export a container image
+- Q60 Entrypoint, default argument and a non-root user
+- Q41 Build with a build argument, run it detached _(Simulator Set A (killer.sh-style))_
+
+### 1.2 Choose and use the right workload resource (Deployment, DaemonSet, CronJob, etc.)
+
+`./exam study 1.2` (or `workloads`)
+
+**Know**
+
+- Choose by behaviour: Deployment (interchangeable replicas, rolling updates), StatefulSet (stable name, DNS and storage per replica), DaemonSet (one Pod per node), Job (run to completion), CronJob (Jobs on a schedule), bare Pod (one-off).
+- Job: completions, parallelism, backoffLimit, activeDeadlineSeconds, ttlSecondsAfterFinished; restartPolicy must be Never or OnFailure.
+- CronJob: schedule, startingDeadlineSeconds, concurrencyPolicy (Allow, Forbid, Replace), suspend, successfulJobsHistoryLimit, failedJobsHistoryLimit.
+- DaemonSet: no replicas; a toleration for node-role.kubernetes.io/control-plane:NoSchedule puts it on control-plane nodes too.
+
+**Fast**
+
+```
+k create deploy web --image=nginx:1.27 --replicas=3 $do > deploy.yaml
+k create job render --image=busybox:1.36 $do -- sh -c 'echo done' > job.yaml
+k create cronjob digest --image=busybox:1.36 --schedule='*/5 * * * *' $do -- sh -c 'echo digest' > cj.yaml
+k create job digest-manual --from=cronjob/digest # run a CronJob once, now
+k run tmp --image=busybox:1.36 --restart=Never --rm -it -- sh
+# DaemonSet has no generator: create a Deployment with $do, change kind, delete replicas/strategy/status
+k explain cronjob.spec.jobTemplate.spec --recursive | less
+```
+
+**Traps**
+
+- Three nested specs in a CronJob: spec (schedule, startingDeadlineSeconds) > jobTemplate.spec (completions, activeDeadlineSeconds) > template.spec (containers, restartPolicy).
+- A Job's Pod template is immutable - delete the Job and create it again instead of editing it.
+- A StatefulSet needs `serviceName` naming a headless Service (clusterIP: None) before its Pods get their own DNS names.
+
+**Docs**
+
+-
+-
+-
+
+**Tasks**
+
+- Q3 CronJob with deadlines
+- Q14 Parallel batch Job
+- Q15 DaemonSet on every node
+- Q39 A suspended CronJob and a manual run _(Simulator Set A (killer.sh-style))_
+- Q38 A Pod and a status one-liner _(Simulator Set A (killer.sh-style))_
+- Q44 From a single Pod to a hardened Deployment _(Simulator Set A (killer.sh-style))_
+- Q56 Stable DNS names for a StatefulSet _(Simulator Set A (killer.sh-style))_
+
+### 1.3 Understand multi-container Pod design patterns (e.g. sidecar, init and others)
+
+`./exam study 1.3` (or `multicontainer`)
+
+**Know**
+
+- Containers of a Pod share its network (they reach each other on localhost) and any volume both mount - that is all a sidecar, adapter or ambassador is.
+- Init containers run one at a time, in order, each to completion, before any app container starts.
+- A native sidecar is an init container with `restartPolicy: Always`: it starts before the app containers, keeps running beside them, and does not keep a Job from completing.
+- Address one container with `-c`: `k logs POD -c NAME`, `k exec POD -c NAME -- ...`. Status `Init:0/1` means an init container is still running.
+
+**Fast**
+
+```
+k run app --image=nginx:1.27 $do > pod.yaml # then add the second container by hand
+spec:
+ initContainers:
+ - {name: wait, image: busybox:1.36, command: [sh, -c, "until wget -qO- -T2 http://backend; do sleep 2; done"]}
+ - {name: shipper, image: busybox:1.36, restartPolicy: Always, command: [sh, -c, "tail -F /logs/app.log"],
+ volumeMounts: [{name: logs, mountPath: /logs}]} # native sidecar
+ containers:
+ - {name: app, image: nginx:1.27, volumeMounts: [{name: logs, mountPath: /var/log/app}]}
+ volumes: [{name: logs, emptyDir: {}}]
+k logs app -c shipper; k logs app --all-containers --prefix
+```
+
+**Traps**
+
+- `kubectl run` creates one container only - generate the Pod and add the others in the YAML.
+- A Service with no endpoints still resolves in DNS, so an init container waiting on `nslookup` is not waiting for the app. Wait on a real request.
+- A classic sidecar that never exits keeps a Job running for ever; make it a native sidecar.
+
+**Docs**
+
+-
+-
+
+**Tasks**
+
+- Q1 Adapter sidecar
+- Q2 Init container waits for a dependency
+- Q16 Native sidecar lets a Job finish
+- Q40 An init container writes the page _(Simulator Set A (killer.sh-style))_
+
+### 1.4 Utilize persistent and ephemeral volumes
+
+`./exam study 1.4` (or `volumes`)
+
+**Know**
+
+- Ephemeral: emptyDir (lives as long as the Pod; `medium: Memory` is tmpfs, `sizeLimit` caps it), and the configMap, secret, downwardAPI and projected volumes.
+- Persistent: a PersistentVolumeClaim asks for storage (accessModes, resources.requests.storage, storageClassName); a StorageClass provisions the PersistentVolume, or a pre-made one binds.
+- Access modes: ReadWriteOnce, ReadOnlyMany, ReadWriteMany, ReadWriteOncePod.
+- volumeMounts: mountPath, readOnly, and subPath to mount one file or directory without hiding the rest of the target directory.
+
+**Fast**
+
+```
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata: {name: data}
+spec:
+ accessModes: [ReadWriteOnce]
+ storageClassName: standard
+ resources: {requests: {storage: 100Mi}}
+--- # in the Pod spec:
+volumes:
+ - {name: data, persistentVolumeClaim: {claimName: data}}
+ - {name: scratch, emptyDir: {medium: Memory, sizeLimit: 32Mi}}
+containers:
+ - volumeMounts: [{name: data, mountPath: /data}, {name: scratch, mountPath: /scratch}]
+k get pvc,pv; k describe pvc data # Pending? the Events say why
+```
+
+**Traps**
+
+- There is no `kubectl create pvc` - copy the claim from the docs.
+- With a WaitForFirstConsumer StorageClass (`standard` here) a claim stays Pending until a Pod uses it. That is not an error.
+- emptyDir survives a container restart but not a new Pod. To prove data persists, delete the Pod and read the file again.
+- A non-root container cannot write to a fresh volume unless the Pod sets `securityContext.fsGroup`.
+
+**Docs**
+
+-
+-
+-
+
+**Tasks**
+
+- Q17 Persistent and ephemeral volumes
+- Q10 Projected volume: config, secret and token
+- Q52 Two users, one volume: fsGroup _(Simulator Set A (killer.sh-style))_
+- Q40 An init container writes the page _(Simulator Set A (killer.sh-style))_
+- Q50 An immutable Secret, one file into a busy directory _(Simulator Set A (killer.sh-style))_
+
+## Application Deployment (20%)
+
+### 2.1 Use Kubernetes primitives to implement common deployment strategies (e.g. blue/green or canary)
+
+`./exam study 2.1` (or `strategies`)
+
+**Know**
+
+- Blue/green: two Deployments, and the Service selector decides which one gets all the traffic. Cut over by changing the selector; roll back by changing it back.
+- Canary: two Deployments whose Pods share the label the Service selects. Traffic splits by the number of ready Pods: 1 canary beside 4 stable is 20%.
+- A Deployment's own strategy is RollingUpdate (default) or Recreate (every old Pod stops first - downtime).
+
+**Fast**
+
+```
+k get svc web -o wide; k get pods --show-labels
+k set selector svc web 'app=web,version=green' # blue/green cut-over
+k scale deploy web-stable --replicas=4 # canary ratio: 4 stable + 1 canary = 20%
+k get endpointslices -l kubernetes.io/service-name=web -o wide
+k get pods -l app=web -L version,track
+```
+
+**Traps**
+
+- Wait until the new Deployment is fully available before you switch the selector.
+- `kubectl set selector` replaces the whole selector - name every label you want in it.
+- A canary only shares traffic if the Service selector leaves out the label that tells the two versions apart.
+
+**Docs**
+
+-
+-
+
+**Tasks**
+
+- Q5 Blue/green cutover
+- Q18 Canary by replica ratio
+
+### 2.2 Understand Deployments and how to perform rolling updates
+
+`./exam study 2.2` (or `rollouts`)
+
+**Know**
+
+- Any change to `spec.template` starts a rollout: a new ReplicaSet scales up while the old one scales down, within maxSurge and maxUnavailable (25% each by default).
+- minReadySeconds (how long a Pod must be Ready to count as available), revisionHistoryLimit (old ReplicaSets kept), progressDeadlineSeconds.
+- `kubectl rollout`: status, history, undo, pause, resume, restart.
+
+**Fast**
+
+```
+k set image deploy/web nginx=nginx:1.27 # CONTAINER=IMAGE
+k rollout status deploy/web
+k rollout history deploy/web # --revision=3 shows that revision's template
+k rollout undo deploy/web # --to-revision=2
+k rollout restart deploy/web # same spec, new Pods
+k annotate deploy/web kubernetes.io/change-cause='pick 1.27'
+k patch deploy web -p '{"spec":{"strategy":{"rollingUpdate":{"maxSurge":1,"maxUnavailable":0}}}}'
+```
+
+**Traps**
+
+- `--record` is deprecated: write the kubernetes.io/change-cause annotation yourself.
+- An undo makes a new revision out of the old one, so the active revision number is not the one you rolled back to.
+- A rollout stuck on a bad image does not fail by itself - `rollout status` just waits, and the old Pods keep serving.
+- maxSurge and maxUnavailable cannot both be 0.
+
+**Docs**
+
+-
+
+**Tasks**
+
+- Q4 Roll forward, then back
+- Q19 Rolling update without losing capacity
+- Q42 Back to the last good revision _(Simulator Set A (killer.sh-style))_
+- Q44 From a single Pod to a hardened Deployment _(Simulator Set A (killer.sh-style))_
+
+### 2.3 Use the Helm package manager to deploy existing packages
+
+`./exam study 2.3` (or `helm`)
+
+**Know**
+
+- A chart is the package, a release is one installation of it in a Namespace, and every install, upgrade or rollback adds a revision.
+- Values: the chart's defaults (`helm show values`), overridden with `-f file.yaml` and `--set key=value`; `helm get values` shows what a release was given.
+- Life cycle: repo add/update, search, install, upgrade, rollback, uninstall, list, history, status.
+
+**Fast**
+
+```
+helm repo add NAME URL && helm repo update; helm search repo nginx --versions
+helm show values CHART > values.yaml
+helm install REL CHART -n NS --create-namespace -f values.yaml --set replicaCount=2 # --version X
+helm upgrade REL CHART -n NS --reuse-values --set replicaCount=4
+helm list -n NS # -A for every Namespace; --failed, --pending filter by status
+helm history REL -n NS; helm rollback REL 1 -n NS; helm uninstall REL -n NS
+helm get values REL -n NS # --all adds the defaults; helm get manifest REL -n NS
+```
+
+**Traps**
+
+- Releases live in a Namespace: without `-n` Helm looks in `default` and finds nothing.
+- An upgrade without `--reuse-values` (or the same `-f`/`--set` again) puts every value back to the chart's default.
+- A rollback is a new revision, not a step back in the numbering.
+- Helm 3's `helm list` leaves out pending releases unless you add `-a`; Helm 4 (on this machine) lists every status and has no `-a`. `--pending` and `--failed` work in both.
+
+**Docs**
+
+-
+-
+
+**Tasks**
+
+- Q6 Helm release lifecycle
+- Q21 Helm: values file and an upgrade that keeps values
+- Q43 Helm housekeeping _(Simulator Set A (killer.sh-style))_
+
+### 2.4 Kustomize
+
+`./exam study 2.4` (or `kustomize`)
+
+**Know**
+
+- A kustomization.yaml lists `resources` (files or other kustomizations) and transforms them: namespace, namePrefix, nameSuffix, labels, images, replicas, patches.
+- Generators build ConfigMaps and Secrets from files, env files or literals, add a content hash to the name and rewrite every reference to it.
+- Base and overlays: an overlay lists `../../base` under resources and changes only what differs.
+- `kubectl kustomize DIR` prints the result; `kubectl apply -k DIR` applies it.
+
+**Fast**
+
+```
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+resources: [../../base] # or files: [deployment.yaml, service.yaml]
+namespace: prod
+namePrefix: prod-
+labels: [{pairs: {env: prod}, includeSelectors: false}]
+images: [{name: nginx, newTag: "1.27"}]
+replicas: [{name: web, count: 3}]
+patches: [{path: patch.yaml}] # a partial Deployment with the same name
+configMapGenerator: [{name: settings, envs: [settings.env]}]
+secretGenerator: [{name: token, literals: [token=abc]}]
+generatorOptions: {disableNameSuffixHash: true}
+k kustomize DIR | less; k apply -k DIR; k diff -k DIR
+```
+
+**Traps**
+
+- Always read the render before you apply it.
+- `images`, `replicas` and patches name things as the base does, before any prefix or suffix.
+- Refer to a generated ConfigMap by its plain name - Kustomize appends the hash everywhere it is used.
+- `namespace:` does not create the Namespace.
+
+**Docs**
+
+-
+
+**Tasks**
+
+- Q20 Kustomize overlay
+- Q61 A kustomization that does not build
+- Q45 Kustomize: a patch and a generated ConfigMap _(Simulator Set A (killer.sh-style))_
+
+## Application Observability and Maintenance (15%)
+
+### 3.1 Understand API deprecations
+
+`./exam study 3.1` (or `deprecations`)
+
+**Know**
+
+- Every object is a kind in an API group at a version (apps/v1 Deployment). Versions are deprecated, keep being served for some releases, then are removed.
+- What this cluster serves: `kubectl api-resources` (the APIVERSION column), `kubectl api-versions`, `kubectl explain KIND`.
+- A removed version fails with `no matches for kind "X" in version "g/v"`; a deprecated one still works and prints `Warning: ... is deprecated`.
+- Migrating is the new apiVersion plus whatever changed shape: an Ingress backend becomes `service: {name, port: {number}}` with a required pathType, and an apps/v1 Deployment needs `spec.selector`.
+
+**Fast**
+
+```
+k api-resources | grep -i -E 'cronjob|ingress' # NAME SHORTNAMES APIVERSION NAMESPACED KIND
+k api-versions | grep batch
+k explain hpa | head -4 # GROUP, KIND, VERSION
+k apply -f old.yaml --dry-run=server # the error or warning, without changing anything
+# removed, and what replaced them:
+# extensions/v1beta1, networking.k8s.io/v1beta1 Ingress -> networking.k8s.io/v1 (1.22)
+# extensions/v1beta1, apps/v1beta1, apps/v1beta2 Deployment -> apps/v1 (1.16)
+# batch/v1beta1 CronJob -> batch/v1 (1.25)
+# policy/v1beta1 PodDisruptionBudget -> policy/v1 (1.25)
+# autoscaling/v2beta1, v2beta2 HorizontalPodAutoscaler -> autoscaling/v2 (1.25, 1.26)
+```
+
+**Traps**
+
+- The `kubectl convert` plugin is usually not installed - know the Ingress and Deployment rewrites by hand.
+- `kubectl get` shows an object in the API's preferred version whatever version it was written with.
+- A warning is not an error: the object was created. Read stderr.
+
+**Docs**
+
+-
+-
+
+**Tasks**
+
+- Q22 Manifests written for removed APIs
+- Q58 Removed, and deprecated but still served
+
+### 3.2 Implement probes and health checks
+
+`./exam study 3.2` (or `probes`)
+
+**Know**
+
+- Liveness: failing restarts the container. Readiness: failing takes the Pod out of Service endpoints, no restart. Startup: holds the other two off until it has passed once.
+- Handlers: httpGet (path, port), tcpSocket (port), exec (command), grpc (port).
+- Timing: initialDelaySeconds (0), periodSeconds (10), timeoutSeconds (1), failureThreshold (3), successThreshold (1). A startup probe allows failureThreshold x periodSeconds to come up.
+
+**Fast**
+
+```
+containers:
+ - name: app
+ startupProbe: {httpGet: {path: /, port: 80}, periodSeconds: 5, failureThreshold: 12} # 60 s to start
+ readinessProbe: {httpGet: {path: /ready, port: 8080}, periodSeconds: 5, failureThreshold: 1}
+ livenessProbe: {tcpSocket: {port: 80}, initialDelaySeconds: 15, periodSeconds: 10}
+ # exec: readinessProbe: {exec: {command: [cat, /tmp/ready]}, periodSeconds: 5}
+k describe pod POD | grep -E 'Liveness|Readiness|Startup'
+k get events --field-selector reason=Unhealthy
+k explain pod.spec.containers.livenessProbe
+```
+
+**Traps**
+
+- No kubectl flag adds a probe: generate the YAML and edit it, or `kubectl edit`.
+- Probes belong to a container, not to the Pod spec.
+- Restarts climbing with `Unhealthy` and `Killing` events but nothing wrong in the logs: the liveness probe is wrong or too impatient, not the app.
+- Running, 0/1 Ready and no restarts: the readiness probe is failing.
+
+**Docs**
+
+-
+-
+
+**Tasks**
+
+- Q7 Three kinds of probe
+- Q62 Health checks that check the wrong thing
+- Q46 Probes that match the app's warm-up _(Simulator Set A (killer.sh-style))_
+
+### 3.3 Use built-in CLI tools to monitor Kubernetes applications
+
+`./exam study 3.3` (or `monitoring`)
+
+**Know**
+
+- `kubectl get` output: -o wide, yaml, json, name, jsonpath=, custom-columns=; narrowed with -l (labels), --field-selector, -n / -A; ordered with --sort-by.
+- `kubectl top pod|node` shows live CPU and memory from metrics-server; `--containers` splits a Pod up.
+- `kubectl describe` for the story of one object, `kubectl get events` for the Namespace.
+
+**Fast**
+
+```
+k top pod -A --sort-by=memory; k top pod --containers; k top node
+k get pods -o wide --sort-by=.spec.nodeName
+k get pods -o custom-columns=POD:.metadata.name,NODE:.spec.nodeName,IMAGE:.spec.containers[0].image
+k get pod POD -o jsonpath='{.status.phase}'
+k get pods -l tier=web --field-selector=status.phase!=Running -o name
+k get pods -A --sort-by='.status.containerStatuses[0].restartCount'
+k get events --sort-by=.lastTimestamp; k events --types=Warning --for pod/POD
+```
+
+**Traps**
+
+- `kubectl get --sort-by` sorts ascending (the largest is last); `kubectl top --sort-by` sorts descending (the largest is first).
+- `kubectl top` shows usage, not requests or limits, and needs about a minute after a Pod starts.
+- Quote jsonpath for the shell: '{.items[*].metadata.name}'.
+
+**Docs**
+
+-
+-
+
+**Tasks**
+
+- Q23 Find the heaviest Pods with kubectl top
+- Q63 An inventory with kubectl alone
+- Q48 Restart and event forensics _(Simulator Set A (killer.sh-style))_
+- Q38 A Pod and a status one-liner _(Simulator Set A (killer.sh-style))_
+
+### 3.4 Utilize container logs
+
+`./exam study 3.4` (or `logs`)
+
+**Know**
+
+- `kubectl logs` shows what a container wrote to stdout and stderr. The kubelet keeps it on the node, and it goes when the Pod is deleted.
+- Pick the container with -c (init containers too), or take --all-containers; --previous is the run before the last restart.
+- Narrow with --tail=N, --since=5m, --timestamps; follow with -f; by label with -l.
+
+**Fast**
+
+```
+k logs POD -c CONTAINER # --all-containers --prefix
+k logs POD --previous # the crashed run
+k logs POD --tail=20 --since=10m --timestamps
+k logs -l app=web --prefix --tail=5 # every Pod with the label
+k logs deploy/web; k logs job/render # one Pod of it
+k logs POD | grep ERROR > errors.log
+```
+
+**Traps**
+
+- With -l, only the last 10 lines of each Pod are shown unless you set --tail (--tail=-1 for everything).
+- `kubectl logs deploy/x` reads one Pod, not all replicas.
+- --previous fails when the container has never restarted.
+
+**Docs**
+
+-
+-
+
+**Tasks**
+
+- Q24 Logs of a container that keeps crashing
+- Q64 One replica is logging errors
+- Q49 RBAC, proven by the app's own log _(Simulator Set A (killer.sh-style))_
+- Q54 Expose a Pod and prove it from inside _(Simulator Set A (killer.sh-style))_
+- Q39 A suspended CronJob and a manual run _(Simulator Set A (killer.sh-style))_
+
+### 3.5 Debugging in Kubernetes
+
+`./exam study 3.5` (or `debugging`)
+
+**Know**
+
+- Always the same order: get (status), describe (Events at the bottom), logs (--previous), then exec or debug.
+- Pending: nothing can schedule it (resources, taints, nodeSelector, an unbound claim). ImagePullBackOff / ErrImagePull: wrong image or tag. CreateContainerConfigError: a ConfigMap, Secret or key it references is missing.
+- CrashLoopBackOff: the process exits - read the previous log and lastState.terminated (reason, exitCode). OOMKilled (137): the memory limit is too low.
+- A Deployment with no Pods at all: describe its ReplicaSet - a quota, a LimitRange or Pod Security admission is rejecting them.
+
+**Fast**
+
+```
+k get pods -o wide; k describe pod POD | tail -20
+k get pod POD -o jsonpath='{.status.containerStatuses[0].state.waiting.reason}'
+k get pod POD -o jsonpath='{.status.containerStatuses[0].lastState.terminated.reason}' # .exitCode
+k logs POD --previous; k exec -it POD -- sh
+k debug -it POD --image=busybox:1.36 --target=CONTAINER # ephemeral container, sees its processes
+k run tmp --rm -it --restart=Never --image=busybox:1.36 -- wget -qO- -T3 http://SVC:PORT
+k describe rs -l app=web --show-events | tail; k get events --sort-by=.lastTimestamp
+k get pod POD -o yaml > pod.yaml && vi pod.yaml && k replace --force -f pod.yaml
+```
+
+**Traps**
+
+- Most of a running Pod is immutable (the image is not): fix a bare Pod with `replace --force`, fix a Deployment's Pods in the Deployment.
+- Fix what the task names and nothing else - graders check that the ConfigMap, Service or Namespace you were told to leave is unchanged.
+- After fixing a Deployment wait for the rollout; old Pods linger in Terminating.
+- `kubectl describe` with a selector that matches several objects leaves their Events out - add `--show-events`, or describe one by name.
+
+**Docs**
+
+-
+-
+-
+
+**Tasks**
+
+- Q8 Diagnose and fix a broken Pod
+- Q25 Two faults behind a stuck Deployment
+- Q26 OOMKilled
+- Q36 A Service that routes nowhere
+- Q47 Debugging a container with no shell _(Simulator Set A (killer.sh-style))_
+- Q48 Restart and event forensics _(Simulator Set A (killer.sh-style))_
+
+## Application Environment, Configuration and Security (25%)
+
+### 4.1 Discover and use resources that extend Kubernetes (CRD, Operators)
+
+`./exam study 4.1` (or `crds`)
+
+**Know**
+
+- A CustomResourceDefinition adds a kind to the API; its objects (custom resources) work with every kubectl verb. An operator is a CRD plus a controller acting on those objects.
+- Discover: `k get crd`, `k api-resources --api-group=GROUP` (kind, short names, namespaced or not), `k explain KIND.spec`.
+- The CRD's schema validates custom resources: required fields, enum values, minimum and maximum. The controller reports back in `.status`.
+
+**Fast**
+
+```
+k get crd | grep -i backup # the CRD's name is PLURAL.GROUP
+k api-resources --api-group=data.example.com
+k explain backup.spec # --recursive for the whole tree
+k get backups -A; k get bkp NAME -o yaml; k describe backup NAME
+k get crd backups.data.example.com -o yaml | less # served versions, schema, enum, maximum
+k apply -f cr.yaml --dry-run=server # every validation error, nothing created
+```
+
+**Traps**
+
+- A custom resource's apiVersion is the CRD's GROUP/VERSION, never apiextensions.k8s.io/v1.
+- Field names come from the CRD's schema - check with `kubectl explain` instead of guessing.
+- Deleting a CRD deletes every object of that kind.
+
+**Docs**
+
+-
+-
+-
+
+**Tasks**
+
+- Q27 Discover and use a custom resource
+- Q59 A custom resource the API server rejects
+- Q58 Removed, and deprecated but still served
+
+### 4.2 Understand authentication, authorization and admission control
+
+`./exam study 4.2` (or `access`)
+
+**Know**
+
+- Every request passes authentication (who is it: a user certificate or token, a ServiceAccount token), authorization (may they: RBAC), then admission control (change or reject it: LimitRanger, ResourceQuota, PodSecurity).
+- RBAC: a Role or ClusterRole lists verbs on resources (and subresources such as pods/log, deployments/scale); a RoleBinding or ClusterRoleBinding grants it to users, groups or ServiceAccounts.
+- Pod Security admission: Namespace labels pod-security.kubernetes.io/enforce|warn|audit = privileged | baseline | restricted.
+- `Forbidden: User ... cannot list` is authorization. `violates PodSecurity`, `exceeded quota` and `maximum memory usage` are admission.
+
+**Fast**
+
+```
+k create role ci --verb=get,list,watch --resource=pods,pods/log -n NS
+k create role ci --verb=get,update,patch --resource=deployments.apps,deployments.apps/scale -n NS
+k create rolebinding ci --role=ci --serviceaccount=NS:SA -n NS # --user=jane --clusterrole=view
+k auth can-i list pods -n NS --as=system:serviceaccount:NS:SA
+k auth can-i --list -n NS --as=system:serviceaccount:NS:SA
+k get ns NS --show-labels
+# what `restricted` wants from every container:
+securityContext: {allowPrivilegeEscalation: false, runAsNonRoot: true, capabilities: {drop: [ALL]},
+ seccompProfile: {type: RuntimeDefault}}
+```
+
+**Traps**
+
+- The same ServiceAccount is `--serviceaccount=NS:NAME` in a binding and `system:serviceaccount:NS:NAME` after `--as`.
+- `can-i create pods/exec` asks about a Pod named exec; use `can-i create pods --subresource=exec`.
+- Admission rejects Pods, not Deployments: the Deployment exists and shows 0 Pods, and the reason is in its ReplicaSet's events.
+
+**Docs**
+
+-
+-
+-
+
+**Tasks**
+
+- Q28 RBAC for a CI ServiceAccount
+- Q29 Pod Security admission blocks a Deployment
+- Q49 RBAC, proven by the app's own log _(Simulator Set A (killer.sh-style))_
+- Q53 Pods rejected by the Namespace's limits _(Simulator Set A (killer.sh-style))_
+
+### 4.3 Understand requests, limits, quotas
+
+`./exam study 4.3` (or `quotas`)
+
+**Know**
+
+- Requests are what the scheduler reserves on a node. Limits are the ceiling: CPU above it is throttled, memory above it gets the container OOMKilled.
+- A ResourceQuota caps a Namespace's totals (requests.cpu, limits.memory, pods, count/...). Once it covers CPU or memory, every new Pod must state them.
+- A LimitRange gives containers a default request and limit and enforces a min and max per container, at admission.
+
+**Fast**
+
+```
+k create quota compute-quota -n NS --hard=requests.cpu=1,requests.memory=1Gi,limits.cpu=2,limits.memory=2Gi,pods=6
+k describe quota -n NS # Used against Hard
+k describe limitrange -n NS
+apiVersion: v1
+kind: LimitRange
+metadata: {name: container-defaults}
+spec:
+ limits:
+ - type: Container
+ defaultRequest: {cpu: 100m, memory: 128Mi} # the request a container gets
+ default: {cpu: 250m, memory: 256Mi} # the LIMIT a container gets
+ max: {cpu: 500m, memory: 512Mi}
+```
+
+**Traps**
+
+- `default` is the default limit and `defaultRequest` the default request - they are easy to swap.
+- Quotas and LimitRanges only act on Pods created afterwards.
+- The rejection is not on the Deployment: `k describe rs` shows `exceeded quota` or `maximum memory usage per Container`.
+
+**Docs**
+
+-
+-
+
+**Tasks**
+
+- Q30 Quota with LimitRange defaults
+- Q26 OOMKilled
+- Q53 Pods rejected by the Namespace's limits _(Simulator Set A (killer.sh-style))_
+
+### 4.4 Understand ConfigMaps
+
+`./exam study 4.4` (or `configmaps`)
+
+**Know**
+
+- Create from literals, files, directories or env files. Each key becomes an environment variable (configMapKeyRef, envFrom) or a file (a configMap volume; `items` picks keys).
+- Mounted files follow a changed ConfigMap after a short delay. Environment variables and subPath mounts never do - restart the Pods.
+- A Pod that references a missing ConfigMap or key waits in CreateContainerConfigError unless the reference is `optional: true`.
+
+**Fast**
+
+```
+k create cm app --from-literal=TIER=backend --from-file=app.properties # --from-file=KEY=path --from-env-file=x.env
+env:
+ - {name: APP_TIER, valueFrom: {configMapKeyRef: {name: app, key: TIER}}}
+envFrom: [{configMapRef: {name: app}}]
+volumes: [{name: cfg, configMap: {name: app, items: [{key: app.properties, path: app.properties}]}}]
+volumeMounts: [{name: cfg, mountPath: /etc/app}] # one file into a full directory: add subPath: app.properties
+k set env deploy/web --from=configmap/app; k exec POD -- env | grep TIER
+k rollout restart deploy/web # pick up a change
+```
+
+**Traps**
+
+- `--from-file` makes one key named after the file; `--from-env-file` makes one key per KEY=VALUE line.
+- A volume mounted on a directory hides what the image had there.
+- `immutable: true` cannot be edited or undone - delete the ConfigMap and create it again.
+
+**Docs**
+
+-
+-
+
+**Tasks**
+
+- Q32 ConfigMap as an env var and as a file
+- Q10 Projected volume: config, secret and token
+- Q25 Two faults behind a stuck Deployment
+- Q51 A ConfigMap-driven nginx that needs a restart _(Simulator Set A (killer.sh-style))_
+- Q45 Kustomize: a patch and a generated ConfigMap _(Simulator Set A (killer.sh-style))_
+
+### 4.5 Define resource requirements
+
+`./exam study 4.5` (or `resources`)
+
+**Know**
+
+- Per container: resources.requests and resources.limits for cpu (cores or millicores: 250m) and memory (Mi, Gi).
+- QoS class: Guaranteed (every container has requests equal to limits for both CPU and memory), Burstable (some are set), BestEffort (none). BestEffort is evicted first.
+- A container with a limit but no request gets a request equal to the limit.
+
+**Fast**
+
+```
+k set resources deploy/web -c app --requests=cpu=250m,memory=128Mi --limits=cpu=250m,memory=128Mi
+resources:
+ requests: {cpu: 100m, memory: 64Mi}
+ limits: {cpu: 250m, memory: 128Mi}
+k get pod POD -o jsonpath='{.status.qosClass}'
+k describe node NODE | grep -A8 'Allocated resources'
+```
+
+**Traps**
+
+- `kubectl run` has no --requests or --limits any more: generate the YAML, or use `kubectl set resources` on a workload.
+- Guaranteed needs every container of the Pod, sidecars included.
+- Memory is Mi, not m: `128m` is a fraction of a byte.
+- A request no node can fit leaves the Pod Pending with `Insufficient cpu` or `Insufficient memory`.
+
+**Docs**
+
+-
+-
+
+**Tasks**
+
+- Q31 Resource requirements for Guaranteed QoS
+- Q26 OOMKilled
+- Q15 DaemonSet on every node
+- Q53 Pods rejected by the Namespace's limits _(Simulator Set A (killer.sh-style))_
+
+### 4.6 Create & consume Secrets
+
+`./exam study 4.6` (or `secrets`)
+
+**Know**
+
+- A Secret's `data` is base64 - an encoding, not encryption. `stringData` takes plain text when you write one.
+- Types: Opaque (generic), kubernetes.io/tls, kubernetes.io/dockerconfigjson, kubernetes.io/service-account-token.
+- Consumed like a ConfigMap: secretKeyRef, envFrom secretRef, or a secret volume (`secretName`, `defaultMode`, `items`).
+
+**Fast**
+
+```
+k create secret generic api --from-literal=username=svc --from-literal=password='S3cr3t!'
+k create secret tls web-tls --cert=tls.crt --key=tls.key
+k create secret docker-registry regcred --docker-server=REG --docker-username=U --docker-password=P
+k get secret api -o jsonpath='{.data.password}' | base64 -d
+env: [{name: API_USER, valueFrom: {secretKeyRef: {name: api, key: username}}}]
+volumes: [{name: creds, secret: {secretName: api, defaultMode: 0400}}]
+volumeMounts: [{name: creds, mountPath: /etc/creds, readOnly: true}]
+```
+
+**Traps**
+
+- A secret volume names it with `secretName`; a configMap volume uses `name`.
+- Encoding by hand: `echo -n value | base64`, or the newline becomes part of the secret.
+- defaultMode is octal in YAML (0400) and shows as decimal in JSON (256).
+
+**Docs**
+
+-
+-
+
+**Tasks**
+
+- Q33 Secrets: decode one, consume another
+- Q10 Projected volume: config, secret and token
+- Q37 Ingress with two paths and TLS
+- Q50 An immutable Secret, one file into a busy directory _(Simulator Set A (killer.sh-style))_
+
+### 4.7 Understand ServiceAccounts
+
+`./exam study 4.7` (or `serviceaccounts`)
+
+**Know**
+
+- Every Pod runs as a ServiceAccount (`default` if you name none); its short-lived token is mounted at /var/run/secrets/kubernetes.io/serviceaccount.
+- Choose it with spec.serviceAccountName. Stop the mount with `automountServiceAccountToken: false` on the ServiceAccount or on the Pod (the Pod wins).
+- Tokens: `kubectl create token` for a short-lived one; a Secret of type kubernetes.io/service-account-token annotated with the account's name for a long-lived one.
+- A ServiceAccount can do nothing until RBAC binds a Role to it.
+
+**Fast**
+
+```
+k create sa ci -n NS
+k set serviceaccount deploy/web ci -n NS # = spec.template.spec.serviceAccountName
+k patch sa ci -n NS -p '{"automountServiceAccountToken": false}'
+k create token ci -n NS --duration=1h
+k auth can-i list pods -n NS --as=system:serviceaccount:NS:ci
+apiVersion: v1
+kind: Secret
+metadata: {name: ci-token, annotations: {kubernetes.io/service-account.name: ci}}
+type: kubernetes.io/service-account-token
+```
+
+**Traps**
+
+- No token Secret is made for a new ServiceAccount (since 1.24) - create it yourself when a task wants one.
+- serviceAccountName cannot be changed on a running Pod; on a Deployment it starts a rollout.
+- A Pod naming a ServiceAccount that does not exist is never created - see the ReplicaSet's events.
+
+**Docs**
+
+-
+-
+
+**Tasks**
+
+- Q34 ServiceAccount without an automounted token
+- Q28 RBAC for a CI ServiceAccount
+- Q10 Projected volume: config, secret and token
+- Q49 RBAC, proven by the app's own log _(Simulator Set A (killer.sh-style))_
+
+### 4.8 Understand Application Security (SecurityContexts, Capabilities, etc.)
+
+`./exam study 4.8` (or `security`)
+
+**Know**
+
+- Pod level (spec.securityContext): runAsUser, runAsGroup, runAsNonRoot, fsGroup, supplementalGroups, seccompProfile - for every container.
+- Container level adds allowPrivilegeEscalation, privileged, readOnlyRootFilesystem and capabilities, and overrides the Pod level.
+- Capabilities are written without the CAP_ prefix: NET_BIND_SERVICE, NET_ADMIN, SYS_TIME. Drop ALL, then add back what the app needs.
+
+**Fast**
+
+```
+spec:
+ securityContext: {runAsUser: 1000, runAsGroup: 1000, runAsNonRoot: true, fsGroup: 2000}
+ containers:
+ - name: app
+ securityContext:
+ allowPrivilegeEscalation: false
+ readOnlyRootFilesystem: true
+ capabilities: {drop: [ALL], add: [NET_BIND_SERVICE]}
+k exec POD -- id
+k explain pod.spec.securityContext; k explain pod.spec.containers.securityContext
+```
+
+**Traps**
+
+- capabilities exist only on a container, fsGroup only on the Pod.
+- runAsNonRoot with an image that runs as root and no runAsUser: CreateContainerConfigError.
+- readOnlyRootFilesystem breaks an app that writes to /tmp or /var/cache - mount an emptyDir there.
+
+**Docs**
+
+-
+
+**Tasks**
+
+- Q9 SecurityContext and capabilities
+- Q29 Pod Security admission blocks a Deployment
+- Q44 From a single Pod to a hardened Deployment _(Simulator Set A (killer.sh-style))_
+- Q52 Two users, one volume: fsGroup _(Simulator Set A (killer.sh-style))_
+
+## Services and Networking (20%)
+
+### 5.1 Demonstrate basic understanding of NetworkPolicies
+
+`./exam study 5.1` (or `netpol`)
+
+**Know**
+
+- Pods accept and send everything until a NetworkPolicy selects them. From then on, for the policyTypes it lists, only what some policy allows gets through.
+- Policies only allow, and they add up: there is no deny rule and no ordering.
+- Peers: podSelector (same Namespace), namespaceSelector, ipBlock. Two selectors in one list item mean AND; two list items mean OR.
+- Default deny: `podSelector: {}` with a policyType and no rules. Locking down egress also needs DNS allowed (UDP and TCP 53).
+
+**Fast**
+
+```
+apiVersion: networking.k8s.io/v1
+kind: NetworkPolicy
+metadata: {name: db-allow, namespace: NS}
+spec:
+ podSelector: {matchLabels: {app: db}}
+ policyTypes: [Ingress]
+ ingress:
+ - from:
+ - podSelector: {matchLabels: {app: api}} # OR
+ - namespaceSelector: {matchLabels: {purpose: monitoring}}
+ ports: [{protocol: TCP, port: 5432}]
+# egress that keeps DNS: egress: [{ports: [{protocol: UDP, port: 53}, {protocol: TCP, port: 53}]}, ...]
+k describe netpol db-allow -n NS
+k exec CLIENT -- wget -qO- -T2 http://db:5432 # test it with real traffic
+```
+
+**Traps**
+
+- There is no `kubectl create networkpolicy` - copy the skeleton from the docs.
+- `- podSelector: ...` followed by `namespaceSelector:` in the same item is AND; a second `- namespaceSelector:` is OR.
+- A missing or empty `ingress` denies everything; `ingress: [{}]` allows everything.
+- The policy lives in the Namespace of the Pods it selects, and a Namespace is matched by its labels (kubernetes.io/metadata.name=NAME always exists).
+
+**Docs**
+
+-
+
+**Tasks**
+
+- Q12 Restrict a namespace with NetworkPolicy
+- Q35 Egress lockdown that keeps DNS working
+- Q55 Two sources, one port _(Simulator Set A (killer.sh-style))_
+
+### 5.2 Provide and troubleshoot access to applications via services
+
+`./exam study 5.2` (or `services`)
+
+**Know**
+
+- A Service selects Pods by label and forwards `port` to `targetPort` (a number, or the name of a container port).
+- Types: ClusterIP (inside the cluster), NodePort (30000-32767 on every node), LoadBalancer, ExternalName; headless (`clusterIP: None`) gives each Pod its own DNS record.
+- DNS names: SVC, SVC.NS, SVC.NS.svc.cluster.local.
+- Troubleshoot in this order: are there endpoints, does the selector match the Pod labels, are the Pods Ready, is targetPort the port the container listens on.
+
+**Fast**
+
+```
+k expose deploy web --port=8080 --target-port=80 # --name=web-svc --type=NodePort
+k create svc nodeport web-ext --tcp=80:80 --node-port=30080
+k get svc,endpointslices -o wide; k describe svc web # Endpoints: = nothing selected
+k get pods -l app=web --show-labels
+k run tmp --rm -it --restart=Never --image=busybox:1.36 -- wget -qO- -T3 http://web:8080
+k port-forward svc/web 8080:80
+```
+
+**Traps**
+
+- `kubectl expose` copies the workload's selector; `kubectl create svc` selects app=, which may match nothing.
+- `expose` cannot choose the nodePort - use `create svc nodeport --node-port` or edit the YAML.
+- A Service without endpoints still resolves in DNS; the connection is what fails.
+
+**Docs**
+
+-
+-
+-
+
+**Tasks**
+
+- Q11 Expose a Deployment two ways
+- Q36 A Service that routes nowhere
+- Q54 Expose a Pod and prove it from inside _(Simulator Set A (killer.sh-style))_
+- Q56 Stable DNS names for a StatefulSet _(Simulator Set A (killer.sh-style))_
+
+### 5.3 Use Ingress rules to expose applications
+
+`./exam study 5.3` (or `ingress`)
+
+**Know**
+
+- An Ingress routes HTTP(S) by host and path to Services of its own Namespace; an ingress controller, chosen with ingressClassName, does the work.
+- Each path has a pathType (Prefix, Exact or ImplementationSpecific) and a backend: service.name plus service.port.number or .name.
+- TLS: spec.tls lists hosts and the kubernetes.io/tls Secret that holds their certificate.
+
+**Fast**
+
+```
+k create ingress shop --class=nginx \
+ --rule="shop.example.com/api*=api:8080,tls=shop-tls" --rule="shop.example.com/*=web:80,tls=shop-tls"
+k create secret tls shop-tls --cert=tls.crt --key=tls.key
+k get ingress,ingressclass; k describe ingress shop # a missing backend shows as
+k explain ingress.spec.rules.http.paths
+```
+
+**Traps**
+
+- In --rule a trailing * means pathType Prefix; without it the path is Exact.
+- The backend port is the Service's port, not the container's.
+- No ingress controller runs in this simulator, so Ingress tasks are graded on the object - on the exam, curl it.
+
+**Docs**
+
+-
+
+**Tasks**
+
+- Q37 Ingress with two paths and TLS
+- Q11 Expose a Deployment two ways
+- Q57 An Ingress pointing at the wrong backends _(Simulator Set A (killer.sh-style))_
+
diff --git a/CKA_SOLUTIONS_EXPLAINED.md b/CKA_SOLUTIONS_EXPLAINED.md
index 9306696..d14eac6 100644
--- a/CKA_SOLUTIONS_EXPLAINED.md
+++ b/CKA_SOLUTIONS_EXPLAINED.md
@@ -40,7 +40,7 @@ This bank mirrors the real CKA curriculum weights (25/15/20/10/30% across the fi
| Operator Namespace Creator | Cluster Architecture | *no equivalent here* — this bank doesn't cover writing controller logic (arguably CKAD/dev territory more than CKA) |
| Static Pod Move | Cluster Architecture | Q05, Q46 |
| Cluster Setup and Node Join | Cluster Architecture | Q40 (token) + Q67 (independently verifying the CA discovery hash and real in-cluster API endpoint) — the token/crypto half only, see gap note |
-| Cluster Upgrade | Cluster Architecture | *no equivalent here* — see gap note below |
+| Cluster Upgrade | Cluster Architecture | Q109 (the drain and uncordon around it) — the upgrade itself has no equivalent here, see gap note below |
| Cluster Certificate Management | Cluster Architecture | Q06, Q65 |
### Real gaps worth knowing about
@@ -524,6 +524,100 @@ The exam-day skill this table is really pointing at, regardless of which platfor
---
+### Q107 · Kernel Prerequisites for a Kubernetes Node
+- **Concept**: before `kubeadm` runs, every machine needs the `overlay` and `br_netfilter` kernel modules and three sysctls (`net.bridge.bridge-nf-call-iptables`, `net.bridge.bridge-nf-call-ip6tables`, `net.ipv4.ip_forward`) so that bridged Pod traffic passes through iptables and the node routes between Pods. `modprobe` and `sysctl -w` change the running kernel only; the files under `/etc/modules-load.d` and `/etc/sysctl.d` are what make the settings survive a reboot — which is what this task grades. Swap and the cgroup driver are the other two preflight classics: kubeadm refuses a node with active swap unless the kubelet says `failSwapOn: false`, and the kubelet's `cgroupDriver` must match the runtime's.
+- **Solution**:
+ ```bash
+ {SHELL_W3}
+ printf 'overlay\nbr_netfilter\n' > /etc/modules-load.d/k8s.conf
+ cat > /etc/sysctl.d/k8s.conf < -q)
+ crictl inspect | grep -E '"pid"|logPath'
+ ```
+- **Gotchas**: `crictl ps` truncates IDs to 13 characters — `--no-trunc` (or `kubectl get pod -o jsonpath='{.status.containerStatuses[0].containerID}'`, minus the `containerd://` prefix) gives the full one. The PID is the process on the *node*; inside the container that process is PID 1. The log path under `/var/log/pods/__/` is the file `kubectl logs` reads.
+
+---
+
+### Q109 · Drain the Control-Plane Node and Bring It Back
+- **Concept**: `kubectl drain` is cordon plus eviction — the first half of every node upgrade or maintenance window. It leaves DaemonSet Pods (they would be recreated at once) and static Pods (the kubelet owns them), and it refuses to go on when it meets emptyDir data or Pods without a controller until you say so explicitly. The node then stays `SchedulingDisabled` until `kubectl uncordon`. A Pod that can only run on the drained node (here: a nodeSelector for the control plane) sits in `Pending` for the duration.
+- **Solution**:
+ ```bash
+ kubectl drain {CP} --ignore-daemonsets --delete-emptydir-data
+ kubectl get nodes > {WS}/q109/drained.txt
+ kubectl -n get pod -l app= # Pending
+ echo Pending > {WS}/q109/phase.txt
+ kubectl uncordon {CP}
+ kubectl -n rollout status deploy
+ ```
+- **Gotchas**: `--force` is only for bare Pods and deletes them for good — do not add it by reflex. CoreDNS usually sits on the control-plane node in a fresh kubeadm cluster, so the drain moves it to the workers; it does not move back after the uncordon, and that is fine. Save `kubectl get nodes` *before* you uncordon — the file is the proof the node was drained.
+
+---
+
+### Q110 · Prepare the Join of a Second Control-Plane Node
+- **Concept**: a joining control-plane node needs the cluster's CA, front-proxy CA, etcd CA and service-account keys. `kubeadm init phase upload-certs --upload-certs` encrypts them with a one-off *certificate key* and stores them for two hours in the Secret `kubeadm-certs`; `kubeadm join --control-plane --certificate-key ` downloads and decrypts them. All of this presupposes a `controlPlaneEndpoint` — the one stable address (a load balancer or VIP in production) that every node and kubeconfig uses instead of one particular API server.
+- **Solution**:
+ ```bash
+ kubectl -n kube-system get cm kubeadm-config -o yaml | grep controlPlaneEndpoint
+ {SHELL_CP}
+ kubeadm init phase upload-certs --upload-certs # last line: the certificate key
+ kubeadm token create --print-join-command --certificate-key
+ ```
+- **Gotchas**: without `--certificate-key` (or `--control-plane`) `--print-join-command` prints a *worker's* join line. A key made with `kubeadm certs certificate-key` alone is just a random key — it only works if the certificates were uploaded *with* it (`upload-certs --certificate-key `), so the simplest route is to let `upload-certs` generate it and copy what it prints.
+
+---
+
+### Q111 · Who Leads, and Who Is in etcd
+- **Concept**: in a highly-available control plane every kube-apiserver serves at once, but only one kube-scheduler and one kube-controller-manager act — they elect a leader through a `Lease` in `kube-system`, and the holder identity is `_`. etcd elects its own leader and needs a majority of members to accept writes (3 members survive the loss of 1, 5 of 2). *Stacked* means an etcd member runs on each control-plane node, *external* that etcd has hosts of its own.
+- **Solution**:
+ ```bash
+ kubectl get nodes -l node-role.kubernetes.io/control-plane
+ kubectl -n kube-system get pods -l component=etcd -o wide
+ kubectl -n kube-system get lease kube-scheduler kube-controller-manager
+ {SHELL_CP}
+ export ETCDCTL_ENDPOINTS=https://127.0.0.1:2379 ETCDCTL_CACERT=/etc/kubernetes/pki/etcd/ca.crt \
+ ETCDCTL_CERT=/etc/kubernetes/pki/etcd/server.crt ETCDCTL_KEY=/etc/kubernetes/pki/etcd/server.key
+ etcdctl member list -w table
+ etcdctl endpoint status -w table # IS LEADER
+ ```
+- **Gotchas**: write the *node* name for the two leases, not the whole holder identity — the uuid changes with every restart of the component. `ETCDCTL_API=3` is not needed any more; etcdctl 3.6 only warns about it. `endpoint status --cluster` asks every member through its advertised client URL, which fails when a node's address has changed; without `--cluster` you ask the local member, whose answer already names the leader.
+
+---
+
+### Q112 · CNI, CSI and CRI: What This Cluster Plugs In
+- **Concept**: three interfaces keep networking, storage and container execution out of Kubernetes itself. **CRI**: the kubelet's socket to the runtime. **CNI**: for every Pod sandbox the runtime runs the plugin chain of the first configuration file in `/etc/cni/net.d` (binaries in `/opt/cni/bin`) — here `ptp` wires the veth pair, its `host-local` IPAM plugin hands out an address from the node's Pod CIDR, and `portmap` serves hostPorts. **CSI**: a storage driver registers a `CSIDriver` object and a StorageClass names it as `provisioner`.
+- **Solution**:
+ ```bash
+ {SHELL_W1}
+ ls /etc/cni/net.d/ && cat /etc/cni/net.d/*.conflist
+ grep containerRuntimeEndpoint /var/lib/kubelet/config.yaml
+ exit
+ kubectl get csidrivers
+ kubectl get sc
+ ```
+- **Gotchas**: `cni-plugins` asks for the `type` of each plugin in the chain, in order — not the network's `name` (`kindnet`) and not the IPAM plugin, which has its own line. "No resources found" for CSIDrivers is the answer `0`: the local-path provisioner of this cluster is a plain external provisioner, not a CSI driver.
+
+---
+
## Domain 2: Workloads & Scheduling (15%)
### Q07 · Rolling Update Strategy & Scaling
@@ -939,6 +1033,38 @@ The exam-day skill this table is really pointing at, regardless of which platfor
---
+### Q105 · Autoscale on CPU, Scale Down Calmly
+- **Concept**: a HorizontalPodAutoscaler compares a metric with a target and sets the replica count between `minReplicas` and `maxReplicas`. CPU *utilisation* is usage divided by the container's CPU **request**, so a container without one can never be scaled on it. `behavior` shapes the change: `scaleDown.stabilizationWindowSeconds` makes the HPA act on the highest recommendation of that window (no flapping), and `policies` cap how fast replicas go — here one Pod per minute.
+- **Solution**:
+ ```bash
+ kubectl -n set resources deploy -c web --requests=cpu=100m
+ kubectl -n autoscale deploy --min= --max= --cpu=% # older kubectl: --cpu-percent=
+ kubectl -n edit hpa
+ # spec:
+ # behavior:
+ # scaleDown:
+ # stabilizationWindowSeconds:
+ # policies: [{type: Pods, value: 1, periodSeconds: 60}]
+ kubectl -n get hpa
+ ```
+- **Gotchas**: `kubectl autoscale` cannot write `behavior` — edit the object afterwards or write the `autoscaling/v2` YAML from the start. `TARGETS` shows `` until metrics-server delivers data (it may not be installed here), but `minReplicas` is enforced immediately either way — which is what the last check looks at. `set resources` without `-c` changes every container; with it, only `web`.
+
+---
+
+### Q106 · Health Checks That Check the Wrong Thing
+- **Concept**: the kubelet acts on probes, not on the application: a failing **liveness** probe gets the container killed and restarted, a failing **readiness** probe keeps the Pod out of every Service's endpoints without restarting it. So an application that is perfectly healthy looks broken when its probes point at the wrong port or path. The event text tells the two apart: `connection refused` is a port nobody listens on, `HTTP probe failed with statuscode: 404` is a path the server does not have.
+- **Solution**:
+ ```bash
+ kubectl -n describe pod -l app= --show-events | grep -E 'Liveness|Readiness|Unhealthy|Killing'
+ echo Unhealthy > {WS}/q106/reason.txt
+ kubectl -n get cm -conf -o yaml # listen ; the liveness and readiness locations
+ kubectl -n edit deploy # livenessProbe port, readinessProbe path
+ kubectl -n rollout status deploy
+ ```
+- **Gotchas**: `--show-events` costs nothing on the CKA's kubectl 1.35, which shows Events anyway, and is required from kubectl 1.36 on, where `describe` with a selector that matches several Pods leaves them out. Keep the probes HTTP and keep their timings — the graders compare them. Fixing only the readiness path produces Pods that turn Ready and are killed ten seconds later, so wait and look at `RESTARTS` before you move on.
+
+---
+
## Domain 3: Services & Networking (20%)
### Q11 · NetworkPolicy (Ingress Restriction)
@@ -1237,6 +1363,20 @@ The exam-day skill this table is really pointing at, regardless of which platfor
---
+### Q113 · Pod to Pod, Across Nodes
+- **Concept**: the Kubernetes network model in one task — every Pod has its own address, taken from its node's Pod CIDR (`.spec.podCIDR`), and any Pod reaches any other Pod on any node at that address without NAT; the CNI plugin is what makes it true. A Service is not involved at all. The exception is a `hostNetwork: true` Pod, which lives in the node's own network namespace: its address is the node's, outside every Pod CIDR, and it shares the node's ports.
+- **Solution**:
+ ```bash
+ kubectl -n get pods -o wide
+ kubectl -n get pod -o jsonpath='{.status.podIP} {.spec.nodeName}' > {WS}/q113/web.txt
+ kubectl -n exec -- wget -qO- -T3 http:// > {WS}/q113/direct.html
+ kubectl get nodes -o jsonpath='{range .items[*]}{.metadata.name} {.spec.podCIDR}{"\n"}{end}' > {WS}/q113/pod-cidrs.txt
+ echo > {WS}/q113/outside.txt; echo hostNetwork > {WS}/q113/why.txt
+ ```
+- **Gotchas**: `pod-cidrs.txt` wants *every* node, the control plane included. Compare the odd Pod's IP with `kubectl get nodes -o wide` and you see it is its node's INTERNAL-IP. Pod addresses change whenever a Pod is replaced, which is the whole reason Services exist — never hard-code one outside an exercise like this.
+
+---
+
## Domain 4: Storage (10%)
### Q15 · PersistentVolume, PVC & Pod Mounting
@@ -1645,3 +1785,30 @@ The exam-day skill this table is really pointing at, regardless of which platfor
kubectl -n kube-system get pods -l component=kube-apiserver
```
- **Gotchas**: don't reach for `kubectl describe`/`logs` first here — they need the very component that's down, so they'll just hang or refuse to connect. `crictl` talks to containerd directly on the node and works regardless of API server state, making it (along with `journalctl -u kubelet`) your only real diagnostic tool until the fix lands. Once fixed, double-check the *other* static pods (etcd, scheduler, controller-manager) are still healthy — it's easy to nick something else while editing manifests under pressure with no `kubectl` to sanity-check against.
+
+---
+
+### Q103 · Find the Heaviest Pods with `kubectl top`
+- **Concept**: `kubectl top` reads the metrics API (`metrics.k8s.io`), which metrics-server serves from what each kubelet reports. It shows *usage* — not requests or limits, which are in `kubectl describe node` under "Allocated resources". `--sort-by=cpu|memory` puts the largest first; `--containers` splits a Pod into its containers; `-A` covers every Namespace.
+- **Solution**:
+ ```bash
+ kubectl top pod -n --sort-by=cpu --no-headers | head -1 | awk '{print $1}' > {WS}/q103/cpu.txt
+ kubectl top pod -n --sort-by=memory --no-headers | head -1 | awk '{print $1}' > {WS}/q103/memory.txt
+ echo 'kubectl top node' > {WS}/q103/nodes.sh
+ echo 'kubectl top pod -A --sort-by=memory' > {WS}/q103/pods.sh
+ ```
+- **Gotchas**: give metrics-server about a minute after the Pods start — until then `top` says `metrics not available yet` for them. `Metrics API not available` means metrics-server itself is missing or not Ready (that is Q23's scenario, which is why the two never share a session). The scripts run in a plain shell: spell out `kubectl`, an alias `k` does not exist there.
+
+---
+
+### Q104 · One Replica Is Logging Errors
+- **Concept**: `kubectl logs` reads one container's stdout/stderr. With `-l` it reads every Pod of a label at once — but then shows only the last 10 lines per Pod unless `--tail` says otherwise, and you cannot tell the Pods apart without `--prefix`. An init container's log is read like any other container's, with `-c`, long after it has finished.
+- **Solution**:
+ ```bash
+ kubectl -n logs -l app= --prefix --tail=-1 | grep ERROR | head -3 # [pod/-N/router] ERROR ...
+ echo -N > {WS}/q104/bad-pod.txt
+ kubectl -n logs -N | grep ERROR > {WS}/q104/errors.log
+ kubectl -n logs -0 -c migrate # schema version ... is current
+ echo "kubectl -n logs -l app= --prefix --tail=2" > {WS}/q104/tail.sh
+ ```
+- **Gotchas**: `errors.log` must hold the ERROR lines and nothing else — redirecting the whole log fails the check. `kubectl logs statefulset/` (or `deploy/...`) picks *one* Pod; only a selector covers them all. `--tail=2` with `--prefix` gives two lines per Pod, each starting `[pod//]`.
diff --git a/CKA_STUDY_GUIDE.md b/CKA_STUDY_GUIDE.md
new file mode 100644
index 0000000..bd989b6
--- /dev/null
+++ b/CKA_STUDY_GUIDE.md
@@ -0,0 +1,1250 @@
+# CKA study guide
+
+The 27 objectives of the Certified Kubernetes Administrator curriculum, in its own words and order: what to know, the fastest commands, the traps, the documentation pages the exam lets you open, and the tasks in this simulator that practise each one.
+
+_Generated by `./exam --track cka notes all --md` from `tracks/cka/curriculum.py` - edit that file, not this one._
+
+```
+./exam objectives # the path and your progress
+./exam study 3.2 # one objective: these notes, then its tasks, no clock, solutions open
+./exam drill 3.2 # the same tasks against the clock
+```
+
+## Storage (10%)
+
+### 1.1 Implement storage classes and dynamic volume provisioning
+
+`./exam study 1.1` (or `storageclasses`)
+
+**Know**
+
+- A StorageClass names a provisioner and its settings. A claim that asks for the class gets a PersistentVolume made for it - that is dynamic provisioning.
+- Fields: provisioner, reclaimPolicy (Delete by default, or Retain), volumeBindingMode (Immediate or WaitForFirstConsumer), allowVolumeExpansion, parameters.
+- The default class carries the annotation storageclass.kubernetes.io/is-default-class: "true"; a claim with no storageClassName gets it. `storageClassName: ""` asks for no class at all.
+- A StatefulSet's volumeClaimTemplates make one claim per replica (data-web-0, data-web-1 ...).
+
+**Fast**
+
+```
+k get sc # (default) marks the default class
+apiVersion: storage.k8s.io/v1
+kind: StorageClass
+metadata: {name: fast-local}
+provisioner: rancher.io/local-path
+reclaimPolicy: Delete
+volumeBindingMode: WaitForFirstConsumer
+---
+k patch sc standard -p '{"metadata":{"annotations":{"storageclass.kubernetes.io/is-default-class":"false"}}}'
+k patch sc fast-local -p '{"metadata":{"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'
+k get pvc,pv; k describe pvc data # the Events say why a claim is Pending
+```
+
+**Traps**
+
+- With WaitForFirstConsumer a claim stays Pending until a Pod uses it. That is not an error.
+- Leave exactly one class marked default: take the annotation off the old one as well as putting it on the new one.
+- provisioner, reclaimPolicy and volumeBindingMode cannot be edited - delete the class and create it again.
+
+**Docs**
+
+-
+-
+-
+
+**Tasks**
+
+- Q16 Default StorageClass and dynamic provisioning
+- Q36 Data survives the pod
+- Q59 Migrate data between claims
+- Q99 Per-replica storage for a StatefulSet _(Simulator Set D (killer.sh-style))_
+
+### 1.2 Configure volume types, access modes and reclaim policies
+
+`./exam study 1.2` (or `volumes`)
+
+**Know**
+
+- Volume types you meet: hostPath (a directory of whatever node the Pod lands on), local (a disk of one node, needs nodeAffinity), emptyDir, nfs, csi, and the configMap / secret / projected volumes.
+- Access modes: ReadWriteOnce (one node), ReadOnlyMany, ReadWriteMany, ReadWriteOncePod (one Pod). A claim binds only to a volume that offers its mode.
+- The reclaim policy is on the PersistentVolume: Retain keeps the volume and its data when the claim goes (it becomes Released), Delete removes both.
+
+**Fast**
+
+```
+apiVersion: v1
+kind: PersistentVolume
+metadata: {name: pv-data}
+spec:
+ capacity: {storage: 1Gi}
+ accessModes: [ReadWriteOnce]
+ persistentVolumeReclaimPolicy: Retain
+ storageClassName: manual
+ hostPath: {path: /mnt/data}
+---
+k patch pv pv-data -p '{"spec":{"persistentVolumeReclaimPolicy":"Retain"}}'
+k patch pv pv-data --type=json -p '[{"op":"remove","path":"/spec/claimRef"}]' # Released -> Available
+k get pv -o custom-columns=NAME:.metadata.name,MODES:.spec.accessModes,RECLAIM:.spec.persistentVolumeReclaimPolicy,STATUS:.status.phase,CLAIM:.spec.claimRef.name
+```
+
+**Traps**
+
+- A Released volume never binds again by itself: remove its claimRef first.
+- A `local` volume needs spec.nodeAffinity; a hostPath volume does not pin the Pod to the node that has the data.
+- The local-path provisioner of this cluster only does ReadWriteOnce and ReadWriteOncePod - a ReadWriteMany claim stays Pending.
+
+**Docs**
+
+-
+-
+-
+
+**Tasks**
+
+- Q15 PersistentVolume, claim and pod
+- Q35 Re-bind a Released PersistentVolume
+- Q58 Bind the right local volume
+- Q68 Provision geo-fence persistent storage _(Simulator Set C)_
+- Q72 Migrate ride history to a Retain volume _(Simulator Set C)_
+- Q102 A claim that can never bind _(Simulator Set D (killer.sh-style))_
+
+### 1.3 Manage persistent volumes and persistent volume claims
+
+`./exam study 1.3` (or `claims`)
+
+**Know**
+
+- A claim binds to one volume, and only if class, access mode and size fit (the volume may be larger); `volumeName` or a selector narrows it to one volume.
+- Phases: a volume goes Available, Bound, Released; a claim Pending, Bound.
+- A Pod mounts a claim of its own Namespace: volumes[].persistentVolumeClaim.claimName, then volumeMounts.
+- A claim in use is protected: deleting it waits (Terminating) until no Pod references it, finished Pods included.
+
+**Fast**
+
+```
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata: {name: pvc-data, namespace: storage}
+spec:
+ accessModes: [ReadWriteOnce]
+ storageClassName: manual # omit for the default class
+ resources: {requests: {storage: 500Mi}}
+--- # in the Pod:
+volumes: [{name: data, persistentVolumeClaim: {claimName: pvc-data}}]
+containers: [{..., volumeMounts: [{name: data, mountPath: /data}]}]
+k get pvc,pv -A; k describe pvc pvc-data # Used By: and Events:
+k describe pod POD | tail # unbound claim -> the Pod stays Pending
+```
+
+**Traps**
+
+- There is no `kubectl create pvc` or `create pv` - copy both from the docs.
+- For a hand-made volume the claim's storageClassName has to equal the volume's exactly; leaving it out means the default class, not none.
+- Apart from growing the request (and only if the class allows expansion) a claim cannot be edited.
+- To move data between claims, run a Pod or Job that mounts both.
+
+**Docs**
+
+-
+-
+
+**Tasks**
+
+- Q15 PersistentVolume, claim and pod
+- Q36 Data survives the pod
+- Q35 Re-bind a Released PersistentVolume
+- Q58 Bind the right local volume
+- Q59 Migrate data between claims
+- Q68 Provision geo-fence persistent storage _(Simulator Set C)_
+- Q72 Migrate ride history to a Retain volume _(Simulator Set C)_
+- Q81 Fix the fare calculation pods _(Simulator Set C)_
+- Q99 Per-replica storage for a StatefulSet _(Simulator Set D (killer.sh-style))_
+- Q102 A claim that can never bind _(Simulator Set D (killer.sh-style))_
+
+## Troubleshooting (30%)
+
+### 2.1 Troubleshoot clusters and nodes
+
+`./exam study 2.1` (or `nodes`)
+
+**Know**
+
+- Start from the cluster: `k get nodes`, then `k describe node` - Conditions, Taints, Unschedulable, Events.
+- A NotReady node is nearly always its kubelet: on the node, `systemctl status kubelet` and `journalctl -u kubelet` name the cause.
+- What the kubelet needs: its unit and drop-in (which binary, which flags), /var/lib/kubelet/config.yaml, /etc/kubernetes/kubelet.conf (its kubeconfig), its certificates in /var/lib/kubelet/pki, and a running container runtime.
+- Ready but SchedulingDisabled is a cordon, not a fault: `k uncordon`.
+
+**Fast**
+
+```
+k get nodes -o wide; k describe node NODE | sed -n '/Conditions/,/Addresses/p'
+ssh NODE
+systemctl status kubelet; journalctl -u kubelet --no-pager | tail -30
+systemctl cat kubelet # the unit and its drop-ins: ExecStart, EnvironmentFile
+cat /var/lib/kubelet/config.yaml; cat /var/lib/kubelet/kubeadm-flags.env
+systemctl daemon-reload; systemctl enable --now kubelet; systemctl restart kubelet
+systemctl status containerd; crictl ps -a
+k uncordon NODE; k describe node NODE | grep -i taint
+```
+
+**Traps**
+
+- A kubelet that was stopped is often disabled too: `enable --now`, or it is gone again after a reboot.
+- After changing a unit file nothing happens until `systemctl daemon-reload`.
+- There may be more than one fault. After every fix read the journal again instead of assuming you are done.
+
+**Docs**
+
+-
+-
+-
+
+**Tasks**
+
+- Q17 Node NotReady
+- Q39 kubelet misconfiguration
+- Q60 Worker node: three things wrong
+- Q92 Kubelet unit points at nothing _(Simulator Set D (killer.sh-style))_
+- Q90 Event forensics around kube-proxy _(Simulator Set D (killer.sh-style))_
+- Q95 The kubelet's two certificates _(Simulator Set D (killer.sh-style))_
+
+### 2.2 Troubleshoot cluster components
+
+`./exam study 2.2` (or `components`)
+
+**Know**
+
+- kube-apiserver, kube-scheduler, kube-controller-manager and etcd run as static Pods: the kubelet starts whatever is in /etc/kubernetes/manifests and restarts it when the file changes.
+- Read the symptom: Pods stay Pending with no events - scheduler. Deployments make no ReplicaSet or Pods - controller-manager. kubectl is refused - API server (or etcd behind it).
+- With the API server up: `k -n kube-system get pods`, `logs`, `describe`. With it down: `crictl ps -a`, `crictl logs`, /var/log/pods and the kubelet's journal on the node.
+
+**Fast**
+
+```
+k -n kube-system get pods; k -n kube-system logs kube-scheduler-NODE; k -n kube-system describe pod kube-scheduler-NODE
+ssh CONTROL-PLANE
+ls /etc/kubernetes/manifests; vi /etc/kubernetes/manifests/kube-scheduler.yaml
+crictl ps -a | grep -E 'kube-|etcd'; crictl logs CONTAINER-ID
+ls /var/log/pods/kube-system_kube-apiserver-*/kube-apiserver/ # its logs when kubectl cannot ask for them
+journalctl -u kubelet --no-pager | grep -i -E 'manifest|apiserver' | tail
+```
+
+**Traps**
+
+- The faults are small: a misspelt image, binary or flag, a wrong path in a volume or certificate flag, a wrong port in a probe.
+- Do not leave a backup copy inside the manifests directory - the kubelet runs every file there.
+- After saving a manifest give the kubelet up to a minute before deciding the fix did not work.
+
+**Docs**
+
+-
+-
+-
+
+**Tasks**
+
+- Q20 Control plane component failure
+- Q38 Deployments never create pods
+- Q61 Two control-plane components down
+- Q66 Nothing works: kube-apiserver itself is down
+- Q97 Stop the scheduler, schedule by hand _(Simulator Set D (killer.sh-style))_
+- Q86 How the control plane is run _(Simulator Set D (killer.sh-style))_
+
+### 2.3 Monitor cluster and application resource usage
+
+`./exam study 2.3` (or `usage`)
+
+**Know**
+
+- `kubectl top node|pod` shows live CPU and memory from metrics-server through the metrics API (metrics.k8s.io); `--containers` splits a Pod up, `--sort-by=cpu|memory` orders it.
+- `k describe node` shows what is allocated - the sum of requests and limits - which is not what is used.
+- `kubectl get` sorts and shapes everything else: --sort-by, -o custom-columns, -o jsonpath, -A.
+
+**Fast**
+
+```
+k top node; k top pod -A --sort-by=memory | head; k top pod -n NS --containers
+k describe node NODE | grep -A8 'Allocated resources'
+k get pods -A --sort-by='.status.containerStatuses[0].restartCount'
+k -n kube-system get deploy metrics-server; k get apiservice v1beta1.metrics.k8s.io
+k -n kube-system logs deploy/metrics-server
+```
+
+**Traps**
+
+- `Metrics API not available` means metrics-server is missing or not Ready (on this cluster it needs --kubelet-insecure-tls).
+- `kubectl top --sort-by` puts the largest first; `kubectl get --sort-by` puts it last.
+- A new Pod has no metrics for about a minute.
+
+**Docs**
+
+-
+-
+-
+
+**Tasks**
+
+- Q103 Find the heaviest Pods with kubectl top
+- Q23 metrics-server repair and HPA
+- Q53 Namespace audit to exact files
+- Q85 Investigate vehicle tracker health _(Simulator Set C)_
+- Q89 Sorting done by kubectl _(Simulator Set D (killer.sh-style))_
+
+### 2.4 Manage and evaluate container output streams
+
+`./exam study 2.4` (or `logs`)
+
+**Know**
+
+- `kubectl logs` shows what a container wrote to stdout and stderr. The runtime keeps it on the node under /var/log/pods/NAMESPACE_POD_UID/CONTAINER/, and it goes when the Pod is deleted.
+- Pick the container with -c (init containers too) or take --all-containers; --previous is the run before the last restart.
+- Narrow with --tail=N, --since=10m, --timestamps; follow with -f; take every Pod of a label with -l and tell them apart with --prefix.
+- On a node, without the API server: `crictl logs CONTAINER-ID`.
+
+**Fast**
+
+```
+k logs POD -c CONTAINER # --all-containers --prefix
+k logs POD --previous # the crashed run
+k logs POD --tail=50 --since=10m --timestamps
+k logs -l app=web --prefix --tail=5 # every Pod with the label
+k logs deploy/web; k logs job/render # one Pod of it
+k logs POD | grep ERROR > errors.log
+crictl ps -a; crictl logs --tail=20 CONTAINER-ID; ls /var/log/pods/
+```
+
+**Traps**
+
+- With -l, only the last 10 lines of each Pod are shown unless you set --tail (--tail=-1 for everything).
+- `kubectl logs deploy/x` reads one Pod, not all replicas.
+- --previous fails when the container has never restarted.
+
+**Docs**
+
+-
+-
+-
+
+**Tasks**
+
+- Q19 CrashLoopBackOff diagnosis
+- Q104 One replica is logging errors
+- Q8 Multi-container pod with shared volume
+- Q85 Investigate vehicle tracker health _(Simulator Set C)_
+
+### 2.5 Troubleshoot services and networking
+
+`./exam study 2.5` (or `netdebug`)
+
+**Know**
+
+- Work outwards from the Pod: is it Ready, does the Service have endpoints (selector against Pod labels), is targetPort the port the container listens on, does the name resolve, does a NetworkPolicy stand in the way.
+- DNS is a Service like any other: the CoreDNS Pods, the kube-dns Service and its endpoints, the Corefile in ConfigMap coredns.
+- Underneath: kube-proxy (a DaemonSet) turns Services into rules on every node, and the CNI plugin connects the Pods.
+
+**Fast**
+
+```
+k get svc,endpointslices -o wide; k describe svc SVC # Endpoints: = nothing selected
+k get pods -l KEY=VALUE --show-labels -o wide
+k run tmp --rm -it --restart=Never --image=busybox:1.36 -- sh # wget -qO- -T3 http://SVC:PORT ; nslookup SVC.NS.svc.cluster.local
+k -n kube-system get pods -l k8s-app=kube-dns -o wide; k -n kube-system logs -l k8s-app=kube-dns
+k -n kube-system get cm coredns -o yaml; k -n kube-system get svc,endpointslices -l k8s-app=kube-dns
+k -n kube-system get ds kube-proxy; k -n kube-system logs ds/kube-proxy
+k get netpol -A
+```
+
+**Traps**
+
+- A Service with no endpoints still resolves - it is the connection that fails.
+- Test names with the fully qualified form (SVC.NS.svc.cluster.local): busybox's nslookup misreports short ones.
+- A NetworkPolicy in kube-system can cut DNS off for the whole cluster.
+- Fix what is broken and leave the rest alone - the graders check what you did not have to touch.
+
+**Docs**
+
+-
+-
+
+**Tasks**
+
+- Q18 Broken deployment and service
+- Q62 Application down for four reasons
+- Q37 Cluster DNS is broken
+- Q57 DNS down: three faults
+- Q73 Fix ETA service connectivity _(Simulator Set C)_
+- Q101 Resolves everything except the cluster _(Simulator Set D (killer.sh-style))_
+- Q70 Fix the driver onboarding app _(Simulator Set C)_
+- Q90 Event forensics around kube-proxy _(Simulator Set D (killer.sh-style))_
+
+## Workloads & Scheduling (15%)
+
+### 3.1 Understand application deployments and how to perform rolling update and rollbacks
+
+`./exam study 3.1` (or `rollouts`)
+
+**Know**
+
+- Any change to `spec.template` starts a rollout: a new ReplicaSet scales up while the old one scales down, within maxSurge and maxUnavailable (25% each by default).
+- Every rollout is a revision. `kubectl rollout`: status, history, undo, pause, resume, restart.
+- minReadySeconds, revisionHistoryLimit and progressDeadlineSeconds tune how careful and how patient a rollout is.
+
+**Fast**
+
+```
+k set image deploy/web nginx=nginx:1.27 # CONTAINER=IMAGE
+k scale deploy/web --replicas=4
+k rollout status deploy/web
+k rollout history deploy/web; k rollout history deploy/web --revision=2 # that revision's template
+k rollout undo deploy/web --to-revision=2
+k rollout pause deploy/web; k rollout resume deploy/web
+k patch deploy web -p '{"spec":{"strategy":{"rollingUpdate":{"maxSurge":1,"maxUnavailable":0}}}}'
+```
+
+**Traps**
+
+- To go back to a particular image, look at each revision's template first - "the previous one" is not always the one asked for.
+- An undo makes a new revision out of the old one, so the numbers keep climbing.
+- A paused Deployment records changes but rolls nothing out until it is resumed.
+- maxSurge and maxUnavailable cannot both be 0.
+
+**Docs**
+
+-
+
+**Tasks**
+
+- Q7 Rolling update & scaling
+- Q51 Rollback to a specific revision, then pause
+
+### 3.2 Use ConfigMaps and Secrets to configure applications
+
+`./exam study 3.2` (or `config`)
+
+**Know**
+
+- Both hold keys. A key becomes an environment variable (configMapKeyRef / secretKeyRef, or every key at once with envFrom) or a file (a configMap / secret volume; `items` picks and renames keys).
+- Secret data is base64 - an encoding, not encryption. Types: Opaque, kubernetes.io/tls, kubernetes.io/dockerconfigjson.
+- Mounted files follow a change after a short delay. Environment variables and subPath mounts never do: restart the Pods.
+- `immutable: true` freezes one for good. To change the content, make a new object and point the workload at it.
+
+**Fast**
+
+```
+k create cm app-config --from-literal=APP_MODE=production --from-file=app.properties
+k create secret generic db-creds --from-literal=username=admin --from-literal=password='S3cr3t!'
+k create secret tls web-tls --cert=tls.crt --key=tls.key
+k get secret db-creds -o jsonpath='{.data.password}' | base64 -d
+env:
+ - {name: APP_MODE, valueFrom: {configMapKeyRef: {name: app-config, key: APP_MODE}}}
+ - {name: DB_PASS, valueFrom: {secretKeyRef: {name: db-creds, key: password}}}
+envFrom: [{configMapRef: {name: app-config}}, {secretRef: {name: db-creds}}]
+volumes: [{name: creds, secret: {secretName: db-creds, defaultMode: 0400}}]
+k set env deploy/web --from=configmap/app-config; k rollout restart deploy/web
+```
+
+**Traps**
+
+- A secret volume names it with `secretName`; a configMap volume uses `name`.
+- A Pod that references a missing ConfigMap, Secret or key waits in CreateContainerConfigError.
+- `--from-file` makes one key named after the file; `--from-env-file` makes one key per line.
+
+**Docs**
+
+-
+-
+-
+
+**Tasks**
+
+- Q10 ConfigMap and Secret consumption
+- Q29 Immutable ConfigMap rotation
+- Q79 Surge pricing configuration _(Simulator Set C)_
+- Q93 Secrets as a read-only volume and renamed env vars _(Simulator Set D (killer.sh-style))_
+- Q84 Enforce TLS 1.3 on the booking portal _(Simulator Set C)_
+
+### 3.3 Configure workload autoscaling
+
+`./exam study 3.3` (or `autoscaling`)
+
+**Know**
+
+- A HorizontalPodAutoscaler (autoscaling/v2) keeps a workload between minReplicas and maxReplicas by a metric - usually average CPU or memory utilisation, which is usage divided by the container's request.
+- It reads the metrics API, so it needs metrics-server, and the containers need a request for the resource it scales on.
+- `behavior` shapes the change: scaleUp and scaleDown each take a stabilizationWindowSeconds and policies (so many Pods or Percent per periodSeconds).
+
+**Fast**
+
+```
+k set resources deploy web --requests=cpu=100m
+k autoscale deploy web --min=2 --max=6 --cpu=60% # kubectl before 1.34: --cpu-percent=60
+k get hpa; k describe hpa web
+apiVersion: autoscaling/v2
+kind: HorizontalPodAutoscaler
+metadata: {name: web}
+spec:
+ scaleTargetRef: {apiVersion: apps/v1, kind: Deployment, name: web}
+ minReplicas: 2
+ maxReplicas: 6
+ metrics:
+ - type: Resource
+ resource: {name: cpu, target: {type: Utilization, averageUtilization: 60}}
+ behavior:
+ scaleDown:
+ stabilizationWindowSeconds: 300
+ policies: [{type: Pods, value: 1, periodSeconds: 60}]
+```
+
+**Traps**
+
+- TARGETS ``: no metrics-server, or no request on the containers.
+- `kubectl autoscale` cannot write `behavior` - edit the object or write the YAML.
+- minReplicas is enforced at once, even without metrics.
+
+**Docs**
+
+-
+-
+-
+
+**Tasks**
+
+- Q105 Autoscale on CPU, scale down calmly
+- Q23 metrics-server repair and HPA
+- Q82 Autoscale the route planner on memory _(Simulator Set C)_
+
+### 3.4 Understand the primitives used to create robust, self-healing, application deployments
+
+`./exam study 3.4` (or `selfhealing`)
+
+**Know**
+
+- Controllers put back what is missing: a Deployment its replicas, a DaemonSet one Pod per node, a StatefulSet named replicas with their own storage, a Job its completions (backoffLimit retries), a CronJob its Jobs.
+- Probes tell the kubelet about health: liveness restarts the container, readiness takes the Pod out of Service endpoints, startup holds both off until the app is up.
+- Init containers run to completion, in order, before the app starts; a native sidecar (an init container with restartPolicy: Always) keeps running beside it.
+- A PodDisruptionBudget (minAvailable or maxUnavailable) limits how many Pods a drain may take away at once.
+
+**Fast**
+
+```
+containers:
+ - name: app
+ startupProbe: {httpGet: {path: /, port: 80}, periodSeconds: 5, failureThreshold: 12}
+ readinessProbe: {httpGet: {path: /ready, port: 8080}, periodSeconds: 5}
+ livenessProbe: {tcpSocket: {port: 80}, initialDelaySeconds: 15, periodSeconds: 10}
+ # exec: {exec: {command: [cat, /tmp/ready]}}
+k create job render --image=busybox:1.36 -- sh -c 'echo done'
+k create cronjob digest --image=busybox:1.36 --schedule='*/5 * * * *' -- date; k create job run-1 --from=cronjob/digest
+k create pdb web --selector=app=web --min-available=2
+k create deploy x --image=busybox:1.36 $do > ds.yaml # DaemonSet: change kind, drop replicas / strategy / status
+k describe pod POD | grep -E 'Liveness|Readiness|Startup'; k get events --field-selector reason=Unhealthy
+```
+
+**Traps**
+
+- Probes belong to a container. No kubectl flag adds one - edit the YAML.
+- Restarts with Unhealthy and Killing events but a clean log: the liveness probe is wrong, not the app.
+- A DaemonSet reaches control-plane nodes only with a toleration for node-role.kubernetes.io/control-plane.
+- A Job's Pod template cannot be edited: delete the Job and create it again.
+
+**Docs**
+
+-
+-
+-
+-
+
+**Tasks**
+
+- Q106 Health checks that check the wrong thing
+- Q8 Multi-container pod with shared volume
+- Q28 Init container prepares content
+- Q26 DaemonSet on every node
+- Q27 CronJob with a manual run
+- Q31 Parallel Job
+- Q52 Native sidecar with probes and an exact report
+- Q53 Namespace audit to exact files
+- Q76 Add an init container to the trip logger _(Simulator Set C)_
+- Q88 Ready only when its dependency answers _(Simulator Set D (killer.sh-style))_
+- Q70 Fix the driver onboarding app _(Simulator Set C)_
+
+### 3.5 Configure Pod admission and scheduling (limits, node affinity, etc.)
+
+`./exam study 3.5` (or `scheduling`)
+
+**Know**
+
+- Requests are what the scheduler reserves, limits what the node enforces. A LimitRange gives containers defaults and bounds; a ResourceQuota caps a Namespace's totals - both act at admission.
+- Where a Pod may go: nodeSelector, node affinity (required or preferred), Pod affinity and anti-affinity with a topologyKey, topologySpreadConstraints, or nodeName, which skips the scheduler.
+- Taints keep Pods off a node (NoSchedule, PreferNoSchedule, NoExecute); a toleration lets a Pod through.
+- A PriorityClass orders Pods when room is short; the Pod names it in priorityClassName.
+
+**Fast**
+
+```
+k label node NODE disktype=ssd
+k taint node NODE dedicated=gpu:NoSchedule; k taint node NODE dedicated- # the trailing - removes it
+tolerations: [{key: dedicated, operator: Equal, value: gpu, effect: NoSchedule}]
+affinity:
+ nodeAffinity:
+ requiredDuringSchedulingIgnoredDuringExecution:
+ nodeSelectorTerms: [{matchExpressions: [{key: disktype, operator: In, values: [ssd]}]}]
+ podAntiAffinity:
+ requiredDuringSchedulingIgnoredDuringExecution:
+ - {labelSelector: {matchLabels: {app: web}}, topologyKey: kubernetes.io/hostname}
+k create priorityclass high --value=100000 --description='critical workloads'
+k create quota team -n NS --hard=pods=5,requests.cpu=1,limits.memory=2Gi
+k describe pod POD | tail # FailedScheduling: 0/4 nodes are available: ...
+```
+
+**Traps**
+
+- A toleration lets a Pod onto a tainted node but does not send it there - pair it with a nodeSelector or affinity.
+- Once a quota covers CPU or memory, every new Pod must state them or get them from a LimitRange.
+- In a LimitRange, `default` is the limit and `defaultRequest` the request.
+- Scheduling fields of a running Pod cannot be changed: change the Deployment, or delete and recreate the Pod.
+
+**Docs**
+
+-
+-
+-
+-
+
+**Tasks**
+
+- Q9 Scheduling with node affinity and resources
+- Q25 Taints and tolerations
+- Q24 PriorityClass
+- Q32 ResourceQuota and LimitRange
+- Q50 Spread, tolerate, prioritise — under a quota
+- Q78 Priority for the rider support portal _(Simulator Set C)_
+- Q96 A Pod for the control plane only _(Simulator Set D (killer.sh-style))_
+- Q81 Fix the fare calculation pods _(Simulator Set C)_
+- Q97 Stop the scheduler, schedule by hand _(Simulator Set D (killer.sh-style))_
+
+## Cluster Architecture, Installation & Configuration (25%)
+
+### 4.1 Manage role based access control (RBAC)
+
+`./exam study 4.1` (or `rbac`)
+
+**Know**
+
+- A Role (one Namespace) or ClusterRole (everywhere, and cluster-scoped resources) lists verbs on resources; a RoleBinding or ClusterRoleBinding grants it to users, groups or ServiceAccounts.
+- A RoleBinding may point at a ClusterRole: the rules then apply in that one Namespace only.
+- A user is a certificate: CN is the name, O the groups. Create a key and CSR, submit a CertificateSigningRequest, approve it, put the certificate in a kubeconfig.
+- An aggregated ClusterRole has no rules of its own: the controller fills them in from every ClusterRole carrying the label it selects.
+
+**Fast**
+
+```
+k create role deploy-manager -n NS --verb=get,list,watch,create,update,patch,delete --resource=deployments.apps
+k create rolebinding deploy-bot -n NS --role=deploy-manager --serviceaccount=NS:deploy-bot # --user=jane --group=devs
+k create clusterrole node-reader --verb=get,list,watch --resource=nodes
+k create clusterrolebinding node-reader --clusterrole=node-reader --serviceaccount=NS:SA
+k auth can-i create deployments -n NS --as=system:serviceaccount:NS:deploy-bot; k auth can-i --list --as=jane -n NS
+openssl genrsa -out jane.key 2048; openssl req -new -key jane.key -subj '/CN=jane/O=developers' -out jane.csr
+# CertificateSigningRequest: spec.request = base64 -w0 of the .csr, signerName kubernetes.io/kube-apiserver-client, usages [client auth]
+k certificate approve jane; k get csr jane -o jsonpath='{.status.certificate}' | base64 -d > jane.crt
+k config set-credentials jane --client-certificate=jane.crt --client-key=jane.key --embed-certs
+```
+
+**Traps**
+
+- "Exactly these permissions" also means finding what is already granted: `k get rolebindings,clusterrolebindings -A -o wide | grep NAME`.
+- Deployments are in the group apps: --resource=deployments.apps; a subresource is deployments.apps/scale, pods/log.
+- The same ServiceAccount is `--serviceaccount=NS:NAME` in a binding and `system:serviceaccount:NS:NAME` after `--as`.
+- Nodes, PersistentVolumes and Namespaces are cluster-scoped: they need a ClusterRole and a ClusterRoleBinding.
+
+**Docs**
+
+-
+-
+
+**Tasks**
+
+- Q1 RBAC for a deployment bot
+- Q33 Aggregated ClusterRole
+- Q6 Certificates: expiry check and user CSR
+- Q44 Exact RBAC with an existing over-grant
+- Q47 User onboarding to a working kubeconfig
+- Q75 Cluster-wide read access for the auth team _(Simulator Set C)_
+- Q91 API resources and a Role census _(Simulator Set D (killer.sh-style))_
+
+### 4.2 Prepare underlying infrastructure for installing a Kubernetes cluster
+
+`./exam study 4.2` (or `infra`)
+
+**Know**
+
+- Every machine needs, before kubeadm: a container runtime (containerd) using the same cgroup driver as the kubelet (systemd), the kernel modules overlay and br_netfilter, the sysctls net.bridge.bridge-nf-call-iptables and net.ipv4.ip_forward, and swap off or accounted for.
+- kubeadm, kubelet and kubectl come from the package repository of one minor version (pkgs.k8s.io) and are held, so that a system update does not move them.
+- Also: a unique hostname, MAC address and product_uuid per node, and the ports open - 6443 (API), 10250 (kubelet), 2379-2380 (etcd).
+
+**Fast**
+
+```
+printf 'overlay\nbr_netfilter\n' > /etc/modules-load.d/k8s.conf; modprobe overlay; modprobe br_netfilter
+cat > /etc/sysctl.d/k8s.conf <
+-
+
+**Tasks**
+
+- Q107 Kernel prerequisites for a Kubernetes node
+- Q108 The container runtime behind a Pod
+- Q41 Component versions
+
+### 4.3 Create and manage Kubernetes clusters using kubeadm
+
+`./exam study 4.3` (or `kubeadm`)
+
+**Know**
+
+- `kubeadm init` makes the control plane: certificates in /etc/kubernetes/pki, kubeconfigs in /etc/kubernetes, static Pod manifests in /etc/kubernetes/manifests, then CoreDNS and kube-proxy. A CNI plugin is yours to add.
+- `kubeadm join` adds a node with a bootstrap token and the hash of the cluster CA; `kubeadm token create --print-join-command` writes the whole line.
+- What the cluster was built with is in ConfigMap kubeadm-config (kube-system). `kubeadm certs check-expiration` lists every certificate it manages.
+- A static Pod is a manifest in the kubelet's staticPodPath; it shows up in the API as NAME-NODE and can only be changed or removed through the file.
+
+**Fast**
+
+```
+kubeadm init --pod-network-cidr=10.244.0.0/16 --control-plane-endpoint=HOST:6443
+kubeadm token create --ttl 2h --print-join-command; kubeadm token list
+openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex # the CA hash
+kubeadm certs check-expiration; openssl x509 -in /etc/kubernetes/pki/apiserver.crt -noout -enddate
+k -n kube-system get cm kubeadm-config -o yaml
+grep staticPodPath /var/lib/kubelet/config.yaml; k run web --image=nginx:1.27 $do > /etc/kubernetes/manifests/web.yaml
+```
+
+**Traps**
+
+- Tokens expire (24 hours unless you say otherwise) - make a new one rather than hunting for the old.
+- Check staticPodPath before you write a static Pod: it is not always /etc/kubernetes/manifests.
+- Deleting a static Pod with kubectl only deletes its mirror; the kubelet puts it straight back.
+
+**Docs**
+
+-
+-
+-
+-
+
+**Tasks**
+
+- Q40 kubeadm join token
+- Q5 Static pod
+- Q6 Certificates: expiry check and user CSR
+- Q67 Prepare a node join without shell access to the control plane
+- Q46 Static pod on a relocated manifest path
+- Q94 Static Pod on the control plane behind a Service _(Simulator Set D (killer.sh-style))_
+- Q95 The kubelet's two certificates _(Simulator Set D (killer.sh-style))_
+- Q80 Plan the control-plane upgrade _(Simulator Set C)_
+
+### 4.4 Manage the lifecycle of Kubernetes clusters
+
+`./exam study 4.4` (or `lifecycle`)
+
+**Know**
+
+- Upgrade one minor version at a time, control plane first: the kubeadm package, `kubeadm upgrade plan` and `apply`, drain, the kubelet and kubectl packages, restart the kubelet, uncordon. Workers: `kubeadm upgrade node` in place of `apply`.
+- Maintenance on a node: `kubectl drain` cordons it and evicts its Pods; `kubectl uncordon` gives it back.
+- etcd: `etcdctl snapshot save` backs it up; `etcdutl snapshot restore --data-dir` unpacks a snapshot into a new directory that etcd's manifest must then point at.
+- `kubeadm certs renew all` renews the control-plane certificates; the static Pods have to be restarted to load them.
+
+**Fast**
+
+```
+k drain NODE --ignore-daemonsets --delete-emptydir-data; k uncordon NODE
+apt-get install -y kubeadm='1.36.*'; kubeadm upgrade plan; kubeadm upgrade apply v1.36.x # workers: kubeadm upgrade node
+apt-get install -y kubelet='1.36.*' kubectl='1.36.*'; systemctl daemon-reload; systemctl restart kubelet
+etcdctl --endpoints=https://127.0.0.1:2379 --cacert=/etc/kubernetes/pki/etcd/ca.crt \
+ --cert=/etc/kubernetes/pki/etcd/server.crt --key=/etc/kubernetes/pki/etcd/server.key snapshot save /opt/backup.db
+etcdutl snapshot restore /opt/backup.db --data-dir /var/lib/etcd-restore # then etcd.yaml: the hostPath of volume etcd-data
+kubeadm certs renew all; kubeadm certs check-expiration
+```
+
+**Traps**
+
+- drain stops at DaemonSet Pods (--ignore-daemonsets), at emptyDir data (--delete-emptydir-data) and at bare Pods (--force, which deletes them for good).
+- A drained node stays cordoned until you uncordon it.
+- With etcd 3.6, as here, restore exists only in etcdutl - `etcdctl snapshot restore` is gone.
+- Restoring etcd takes the whole cluster back to the snapshot: everything created since is lost.
+
+**Docs**
+
+-
+-
+-
+-
+
+**Tasks**
+
+- Q109 Drain the control-plane node and bring it back
+- Q2 etcd snapshot
+- Q65 Renew control-plane certificates
+- Q45 etcd restore
+- Q80 Plan the control-plane upgrade _(Simulator Set C)_
+
+### 4.5 Implement and configure a highly-available control plane
+
+`./exam study 4.5` (or `ha`)
+
+**Know**
+
+- Highly available means several control-plane nodes (three as a rule) behind one stable address - the controlPlaneEndpoint, a load balancer or virtual IP, fixed at `kubeadm init`.
+- etcd is either stacked (a member on every control-plane node) or external (hosts of its own). It needs a majority to accept writes: 3 members survive the loss of 1, 5 of 2.
+- More control-plane nodes join with `kubeadm join --control-plane --certificate-key KEY`; the key comes from `kubeadm init phase upload-certs --upload-certs` and is good for two hours.
+- Every kube-apiserver serves at once. Of the schedulers and controller-managers only the leader acts - they elect it through a Lease in kube-system.
+
+**Fast**
+
+```
+k get nodes -l node-role.kubernetes.io/control-plane
+k -n kube-system get cm kubeadm-config -o yaml | grep controlPlaneEndpoint
+kubeadm init --control-plane-endpoint=LB:6443 --upload-certs
+kubeadm init phase upload-certs --upload-certs # prints the certificate key
+kubeadm token create --print-join-command --certificate-key KEY # ... --control-plane --certificate-key KEY
+k -n kube-system get lease kube-scheduler kube-controller-manager # HOLDER = NODE_uuid
+etcdctl member list -w table; etcdctl endpoint status -w table; etcdctl endpoint health # with the four etcd flags
+k -n kube-system get pods -l component=etcd -o wide
+```
+
+**Traps**
+
+- A cluster initialised without a controlPlaneEndpoint cannot simply be given a second control-plane node.
+- An even number of etcd members tolerates no more failures than the odd number below it.
+- The certificate key expires after two hours - upload the certificates again rather than reusing an old key.
+
+**Docs**
+
+-
+-
+
+**Tasks**
+
+- Q110 Prepare the join of a second control-plane node
+- Q111 Who leads, and who is in etcd
+- Q86 How the control plane is run _(Simulator Set D (killer.sh-style))_
+- Q87 Cluster facts to an exact file _(Simulator Set D (killer.sh-style))_
+
+### 4.6 Use Helm and Kustomize to install cluster components
+
+`./exam study 4.6` (or `helmkustomize`)
+
+**Know**
+
+- Helm: a chart is the package, a release one installation of it in a Namespace, and each install, upgrade or rollback a revision. Values come from the chart, then -f files, then --set.
+- `helm template` renders a chart without installing it; `helm show values` prints its defaults; `helm get values` what a release was given.
+- Kustomize: a kustomization.yaml lists resources and changes them - namespace, namePrefix, labels, images, replicas, patches - and generates ConfigMaps and Secrets. An overlay lists a base and changes only what differs.
+
+**Fast**
+
+```
+helm repo add NAME URL && helm repo update; helm search repo NAME --versions
+helm install REL CHART -n NS --create-namespace --set replicaCount=2 -f values.yaml # --version X
+helm upgrade REL CHART -n NS --reuse-values --set image.tag=1.27
+helm template REL CHART -n NS --set replicaCount=3 > manifest.yaml
+helm list -A; helm history REL -n NS; helm rollback REL 1 -n NS; helm uninstall REL -n NS
+k kustomize DIR | less; k apply -k DIR
+# kustomization.yaml: resources: [../../base] namespace: prod namePrefix: prod- images: [{name: nginx, newTag: "1.27"}]
+# replicas: [{name: web, count: 3}] patches: [{path: patch.yaml}] configMapGenerator: [{name: cfg, literals: [k=v]}]
+```
+
+**Traps**
+
+- Releases live in a Namespace: without -n Helm looks in `default`.
+- An upgrade without --reuse-values (or the same -f and --set again) puts every value back to the chart's default.
+- In a kustomization, `images`, `replicas` and patches use the names of the base, before any prefix.
+- `namespace:` in a kustomization does not create the Namespace.
+
+**Docs**
+
+-
+-
+-
+
+**Tasks**
+
+- Q21 Helm release install and upgrade
+- Q3 Kustomize overlay
+- Q48 Kustomize with generators and patches
+- Q69 Render a chart, then install it _(Simulator Set C)_
+- Q98 Helm release housekeeping _(Simulator Set D (killer.sh-style))_
+
+### 4.7 Understand extension interfaces (CNI, CSI, CRI, etc.)
+
+`./exam study 4.7` (or `interfaces`)
+
+**Know**
+
+- CRI: the kubelet drives the container runtime (containerd, CRI-O) through a socket; `crictl` speaks the same protocol, so it works on a node when kubectl does not.
+- CNI: for every Pod the runtime calls the plugins named in /etc/cni/net.d (binaries in /opt/cni/bin) to give it an address and wire it up. The plugin - kindnet, Calico, Cilium, Flannel - usually runs as a DaemonSet.
+- CSI: a storage driver runs as Pods, registers a CSIDriver object, and a StorageClass names it as provisioner.
+
+**Fast**
+
+```
+k get nodes -o wide # CONTAINER-RUNTIME column
+crictl version; crictl pods; crictl ps -a; crictl images; crictl inspect CONTAINER-ID
+grep containerRuntimeEndpoint /var/lib/kubelet/config.yaml; cat /etc/crictl.yaml
+ls /etc/cni/net.d; cat /etc/cni/net.d/*; ls /opt/cni/bin
+k -n kube-system get ds # the CNI's DaemonSet beside kube-proxy
+k get csidrivers; k get csinodes; k get sc # PROVISIONER
+```
+
+**Traps**
+
+- Not every CNI plugin enforces NetworkPolicy (Flannel does not; kindnet in this cluster does).
+- Without a CNI plugin the nodes stay NotReady: `cni plugin not initialized`.
+- The runtime takes the first configuration file in /etc/cni/net.d, in alphabetical order.
+
+**Docs**
+
+-
+-
+-
+
+**Tasks**
+
+- Q112 CNI, CSI and CRI: what this cluster plugs in
+- Q108 The container runtime behind a Pod
+- Q41 Component versions
+- Q63 Cluster facts, exactly
+- Q77 Audit the CNI, then lock a namespace down _(Simulator Set C)_
+- Q87 Cluster facts to an exact file _(Simulator Set D (killer.sh-style))_
+
+### 4.8 Understand CRDs, install and configure operators
+
+`./exam study 4.8` (or `crds`)
+
+**Know**
+
+- A CustomResourceDefinition adds a kind to the API: group, names (kind, plural, singular, shortNames), scope (Namespaced or Cluster) and versions, each with an OpenAPI v3 schema.
+- The schema validates custom resources - types, required fields, enum, minimum, maximum, defaults - and feeds `kubectl explain`.
+- An operator is a CRD plus a controller (a Deployment with RBAC) that acts on those objects. Installing one means applying its manifests or its Helm chart, then creating a custom resource.
+
+**Fast**
+
+```
+apiVersion: apiextensions.k8s.io/v1
+kind: CustomResourceDefinition
+metadata: {name: backups.data.example.com} # PLURAL.GROUP
+spec:
+ group: data.example.com
+ scope: Namespaced
+ names: {kind: Backup, plural: backups, singular: backup, shortNames: [bk]}
+ versions:
+ - name: v1
+ served: true
+ storage: true
+ schema:
+ openAPIV3Schema:
+ type: object
+ properties:
+ spec:
+ type: object
+ required: [schedule]
+ properties:
+ schedule: {type: string}
+ retention: {type: integer, minimum: 1, default: 7}
+k get crd; k api-resources --api-group=data.example.com; k explain backup.spec --recursive
+```
+
+**Traps**
+
+- The CRD's name must be PLURAL.GROUP, and exactly one version has `storage: true`.
+- A custom resource's apiVersion is GROUP/VERSION of the CRD, not apiextensions.k8s.io/v1.
+- `default` and validation only work inside a structural schema: every level needs its `type`.
+- Deleting a CRD deletes every object of that kind.
+
+**Docs**
+
+-
+-
+-
+
+**Tasks**
+
+- Q4 CustomResourceDefinition
+- Q34 Discover a CRD field with kubectl explain
+- Q49 CRD with validation and defaults
+- Q91 API resources and a Role census _(Simulator Set D (killer.sh-style))_
+
+## Services & Networking (20%)
+
+### 5.1 Understand connectivity between Pods
+
+`./exam study 5.1` (or `connectivity`)
+
+**Know**
+
+- Every Pod has its own IP address, and any Pod reaches any other Pod on any node at that address without NAT. The CNI plugin makes that true.
+- Each node hands out Pod addresses from its own range (spec.podCIDR), a slice of the cluster's Pod CIDR; Services get theirs from the separate service CIDR.
+- A hostNetwork Pod lives in the node's network: it has the node's address and shares its ports.
+- Containers of one Pod share a network namespace and talk over localhost.
+
+**Fast**
+
+```
+k get pods -o wide -A; k get nodes -o custom-columns=NODE:.metadata.name,PODCIDR:.spec.podCIDR,IP:.status.addresses[0].address
+k exec POD -- wget -qO- -T3 http://POD-IP:PORT
+k -n kube-system get cm kubeadm-config -o yaml | grep -E 'podSubnet|serviceSubnet'
+k -n kube-system get pods -o wide | grep -E 'kube-proxy|kindnet'; k -n kube-system get cm kube-proxy -o yaml | grep mode
+```
+
+**Traps**
+
+- A Pod's address changes whenever the Pod is replaced - that is what Services are for.
+- Reaching a Pod by IP proves the network; reaching it by Service name also needs endpoints and DNS.
+- If only the Pods of one node are unreachable, look at that node's CNI and kube-proxy Pods.
+
+**Docs**
+
+-
+-
+
+**Tasks**
+
+- Q113 Pod to Pod, across nodes
+- Q63 Cluster facts, exactly
+- Q71 Expose and verify the dispatch API _(Simulator Set C)_
+- Q77 Audit the CNI, then lock a namespace down _(Simulator Set C)_
+- Q87 Cluster facts to an exact file _(Simulator Set D (killer.sh-style))_
+
+### 5.2 Define and enforce Network Policies
+
+`./exam study 5.2` (or `netpol`)
+
+**Know**
+
+- Pods accept and send everything until a NetworkPolicy selects them. From then on, for the policyTypes it lists, only what some policy allows gets through.
+- Policies only allow, and they add up: there is no deny rule and no ordering.
+- Peers: podSelector (same Namespace), namespaceSelector, ipBlock. Two selectors in one list item mean AND; two list items mean OR.
+- Default deny: `podSelector: {}` with the policyTypes and no rules. An egress lock-down also has to allow DNS (UDP and TCP 53).
+
+**Fast**
+
+```
+apiVersion: networking.k8s.io/v1
+kind: NetworkPolicy
+metadata: {name: backend-policy, namespace: NS}
+spec:
+ podSelector: {matchLabels: {app: backend}}
+ policyTypes: [Ingress, Egress]
+ ingress:
+ - from: [{podSelector: {matchLabels: {app: frontend}}}]
+ ports: [{protocol: TCP, port: 8080}]
+ egress:
+ - to: [{namespaceSelector: {matchLabels: {kubernetes.io/metadata.name: kube-system}}, podSelector: {matchLabels: {k8s-app: kube-dns}}}]
+ ports: [{protocol: UDP, port: 53}, {protocol: TCP, port: 53}]
+k describe netpol backend-policy -n NS; k exec CLIENT -- wget -qO- -T2 http://backend:8080
+```
+
+**Traps**
+
+- There is no `kubectl create networkpolicy` - copy the skeleton from the docs.
+- `namespaceSelector` and `podSelector` in the same item: Pods with that label in those Namespaces. In two items: either.
+- A missing `ingress` denies everything; `ingress: [{}]` allows everything.
+- A Namespace is matched by its labels; kubernetes.io/metadata.name=NAME is always there.
+
+**Docs**
+
+-
+
+**Tasks**
+
+- Q11 NetworkPolicy
+- Q42 Egress-only NetworkPolicy
+- Q54 Zero-trust namespace
+- Q83 NetworkPolicy for the safety monitor _(Simulator Set C)_
+- Q100 Namespace AND pod selector, enforced _(Simulator Set D (killer.sh-style))_
+- Q77 Audit the CNI, then lock a namespace down _(Simulator Set C)_
+
+### 5.3 Use ClusterIP, NodePort, LoadBalancer service types and endpoints
+
+`./exam study 5.3` (or `services`)
+
+**Know**
+
+- A Service selects Pods by label and forwards `port` to `targetPort` - a number, or the name of a container port.
+- ClusterIP is reachable inside the cluster; NodePort adds a port (30000-32767) on every node; LoadBalancer adds an external address from the cloud; ExternalName is a DNS alias; headless (clusterIP: None) gives every Pod its own DNS record.
+- The Pods behind a Service are its endpoints, kept in EndpointSlices; only Ready Pods are in them.
+
+**Fast**
+
+```
+k expose deploy shop --name=shop-svc --port=80 --target-port=80 --type=NodePort
+k create svc nodeport shop-svc --tcp=80:80 --node-port=30080 # then check the selector
+k create svc externalname legacy --external-name=db.example.com
+k get svc,endpointslices -o wide; k describe svc shop-svc # Endpoints:
+k get pods -l app=shop --show-labels
+k run tmp --rm -it --restart=Never --image=busybox:1.36 -- wget -qO- -T3 http://shop-svc
+```
+
+**Traps**
+
+- `kubectl expose` copies the workload's selector; `kubectl create svc` selects app=, which may match nothing.
+- `expose` cannot choose the nodePort - use `create svc nodeport --node-port` or edit the YAML.
+- A LoadBalancer stays where nothing provides one; it still works as a NodePort.
+- A headless Service for a StatefulSet has to carry the name in the StatefulSet's serviceName.
+
+**Docs**
+
+-
+-
+
+**Tasks**
+
+- Q12 NodePort Service
+- Q30 Named ports
+- Q43 Headless and ExternalName services
+- Q64 The context's namespace
+- Q71 Expose and verify the dispatch API _(Simulator Set C)_
+- Q73 Fix ETA service connectivity _(Simulator Set C)_
+- Q94 Static Pod on the control plane behind a Service _(Simulator Set D (killer.sh-style))_
+
+### 5.4 Use the Gateway API to manage Ingress traffic
+
+`./exam study 5.4` (or `gateway`)
+
+**Know**
+
+- Three kinds, three owners: a GatewayClass names the controller, a Gateway opens listeners (port, protocol, hostname, which routes may attach), an HTTPRoute attaches to a Gateway (parentRefs) and sends requests to Services (backendRefs).
+- An HTTPRoute rule matches on path, header, method or query, and may split traffic between backends by weight - that is a canary.
+- The kinds are CRDs (gateway.networking.k8s.io): they have to be installed, and a controller has to implement them.
+
+**Fast**
+
+```
+apiVersion: gateway.networking.k8s.io/v1
+kind: Gateway
+metadata: {name: web-gw, namespace: NS}
+spec:
+ gatewayClassName: example
+ listeners: [{name: http, protocol: HTTP, port: 80, hostname: shop.example.com}]
+---
+apiVersion: gateway.networking.k8s.io/v1
+kind: HTTPRoute
+metadata: {name: web, namespace: NS}
+spec:
+ parentRefs: [{name: web-gw}]
+ hostnames: [shop.example.com]
+ rules:
+ - matches: [{path: {type: PathPrefix, value: /}}]
+ backendRefs: [{name: v1, port: 80, weight: 90}, {name: v2, port: 80, weight: 10}]
+k get gatewayclass,gateway,httproute -A; k describe httproute web -n NS
+```
+
+**Traps**
+
+- A route in another Namespace attaches only if the listener's allowedRoutes lets it.
+- A header match belongs in `matches`, beside or instead of the path - and several rules are tried most specific first.
+- No Gateway controller runs in this simulator, so these tasks are graded on the objects.
+
+**Docs**
+
+-
+-
+
+**Tasks**
+
+- Q22 Gateway API HTTPRoute
+- Q56 Gateway API canary with header match
+- Q74 Fleet dashboard canary with Gateway API _(Simulator Set C)_
+
+### 5.5 Know how to use Ingress controllers and Ingress resources
+
+`./exam study 5.5` (or `ingress`)
+
+**Know**
+
+- An Ingress routes HTTP(S) by host and path to Services of its own Namespace. It does nothing without an ingress controller, chosen with ingressClassName (an IngressClass names the controller).
+- Each path has a pathType (Prefix, Exact or ImplementationSpecific) and a backend: service.name plus service.port.number or .name.
+- TLS: spec.tls lists hosts and the kubernetes.io/tls Secret that holds their certificate.
+
+**Fast**
+
+```
+k create ingress app-ingress -n NS --class=nginx \
+ --rule='shop.example.com/api*=api:8080,tls=shop-tls' --rule='shop.example.com/*=web:80,tls=shop-tls'
+openssl req -x509 -nodes -newkey rsa:2048 -days 365 -subj '/CN=shop.example.com' -keyout tls.key -out tls.crt
+k create secret tls shop-tls -n NS --cert=tls.crt --key=tls.key
+k get ingress,ingressclass -A; k describe ingress app-ingress -n NS
+```
+
+**Traps**
+
+- In --rule a trailing * means pathType Prefix; without it the path is Exact.
+- The backend port is the Service's port, not the container's.
+- No ingress controller runs in this simulator, so Ingress tasks are graded on the object - on the exam, curl it.
+
+**Docs**
+
+-
+-
+
+**Tasks**
+
+- Q13 Ingress routing
+- Q55 TLS Ingress with a generated certificate
+
+### 5.6 Understand and use CoreDNS
+
+`./exam study 5.6` (or `coredns`)
+
+**Know**
+
+- CoreDNS is a Deployment in kube-system behind the Service kube-dns; its configuration, the Corefile, is in ConfigMap coredns: the `kubernetes` plugin answers for cluster.local, `forward` sends the rest upstream.
+- Names: SVC.NS.svc.cluster.local for a Service, POD.SVC.NS.svc.cluster.local for the Pods of a headless Service.
+- A Pod's /etc/resolv.conf points at the kube-dns address and lists the search domains that make short names work. dnsPolicy decides: ClusterFirst (default), Default (the node's resolver), None with dnsConfig.
+
+**Fast**
+
+```
+k -n kube-system get deploy coredns; k -n kube-system get pods -l k8s-app=kube-dns -o wide
+k -n kube-system get svc kube-dns; k -n kube-system get endpointslices -l k8s-app=kube-dns
+k -n kube-system get cm coredns -o yaml; k -n kube-system edit cm coredns
+k -n kube-system logs -l k8s-app=kube-dns; k -n kube-system rollout restart deploy coredns
+k run tmp --rm -it --restart=Never --image=busybox:1.36 -- nslookup kubernetes.default.svc.cluster.local
+k exec POD -- cat /etc/resolv.conf
+```
+
+**Traps**
+
+- A typo in the Corefile puts CoreDNS in CrashLoopBackOff - its log names the plugin and the line.
+- DNS can be broken in several places at once: the Corefile, the Deployment (replicas, image), the kube-dns Service (selector, ports), a NetworkPolicy.
+- A Pod with dnsPolicy: Default resolves the internet but not cluster names.
+
+**Docs**
+
+-
+-
+-
+
+**Tasks**
+
+- Q14 Cluster DNS
+- Q37 Cluster DNS is broken
+- Q57 DNS down: three faults
+- Q101 Resolves everything except the cluster _(Simulator Set D (killer.sh-style))_
+
diff --git a/README.md b/README.md
index 23dbb01..680d2ed 100644
--- a/README.md
+++ b/README.md
@@ -4,11 +4,15 @@ Entry point is `./exam` (renamed from `cka-sim` so it can't collide with the dir
Hands-on exam tasks graded **per check** against a real cluster, an exam clock, a browser
front end modelled on the real PSI exam, attempt history, and drills that rotate through the
-domains before you sit a full exam. Tracks: **CKA** (102 tasks: a random pool of 43 normal +
-24 hard, plus two fixed simulator sets, C and D, of 18 and 17 tasks), **CKAD** (37 pool tasks, at
-least one for every competency in the curriculum, plus a 20-task killer.sh-style Set A) and CKS (a
-9-task starter bank in the same format). Each track runs on the Kubernetes version its real exam
-runs — CKA and CKS on **v1.35**, CKAD on **v1.37** (see [Clusters and versions](#clusters-and-versions)).
+domains before you sit a full exam. Tracks: **CKA** (113 tasks: a random pool of 54 normal +
+24 hard, plus two fixed simulator sets, C and D, of 18 and 17 tasks), **CKAD** (44 pool tasks, plus
+a 20-task killer.sh-style Set A) and CKS (a 9-task starter bank in the same format). Each track
+runs on the Kubernetes version its real exam runs — CKA and CKS on **v1.35**, CKAD on **v1.37**
+(see [Clusters and versions](#clusters-and-versions)).
+
+CKA and CKAD are also laid out as a **study path**: the curriculum objective by objective, each
+with notes, at least two pool tasks, and tracked progress from first reading to exam conditions
+(see [Study path](#study-path-the-curriculum-objective-by-objective)).
## How a session is built
@@ -104,7 +108,10 @@ The same state is available from the terminal if you prefer:
./exam drill weak | drill 2 17 20 | drill TSHOOT NET | drill setd
./exam start [--hard] # full exam (gated until all domains drilled; --force)
./exam sim c|d # a fixed 18/17-task simulator set, exam rules, 120 min
-./exam study [DOMAIN|weak|N..] # no clock, solutions visible, nothing recorded
+./exam objectives # CKA/CKAD: the curriculum as a path, with your progress
+./exam next # start whatever is next on that path
+./exam study [OBJ|DOMAIN|weak|N..] # no clock, solutions visible, kept out of the history
+./exam notes [OBJ|DOMAIN|all] # study notes of an objective (3.2 or probes)
./exam list | show N | timer | status # during
./exam end # submit
./exam weak | history # afterwards
@@ -113,21 +120,69 @@ The same state is available from the terminal if you prefer:
./teardown.sh [cka|ckad|cks|legacy] # delete clusters + state (history kept; default all)
```
+## Study path: the curriculum, objective by objective
+
+`tracks/cka/curriculum.py` and `tracks/ckad/curriculum.py` hold the exam curricula in their own
+words and order - 27 objectives for CKA, 24 for CKAD. Every objective has study notes (what to
+know, the fastest commands, the traps, the documentation pages the exam lets you open) and lists
+the tasks that practise it; every task of the bank is listed under at least one objective, and
+every objective has at least two pool tasks.
+
+```
+./exam objectives # every objective, its tasks, where you are, and the next step
+./exam next # start that next step
+./exam notes 3.2 # the notes of one objective, by id or key (3.2, probes); `all`, or a domain
+./exam study 3.2 # those notes, then its tasks: no clock, solutions open (--all adds simulator-set tasks)
+./exam drill 3.2 # the same tasks against the clock, no solution until a task is graded
+```
+
+Each objective goes through two stages, and the path ends in the mocks:
+
+1. **Study** - solve every task of the objective once, with the notes and the reference solutions
+ open. The objective is then *studied* (`~`).
+2. **Clock** - pass the same tasks as a timed drill (6 minutes per task, new parameters). When the
+ latest attempt at every task is a full pass, the objective is *complete* (`✓`).
+3. **Mock exam** (`./exam start`) once every objective is complete, then the fixed
+ **simulator sets** (`./exam sim a` for CKAD, `c` and `d` for CKA) - exam rules, no feedback
+ until you submit.
+
+`./exam next` always starts the first open step in that order; `./exam objectives` shows the whole
+ladder. An objective with many tasks takes several sittings: each brings up to five tasks you have
+not solved yet. The CKA path walks the domains Workloads, Networking, Storage, Architecture and
+then Troubleshooting - it is 30% of the exam and builds on all the others (`STUDY_ORDER` in its
+curriculum.py); CKAD follows the curriculum's order. Nothing is locked: study or drill any
+objective at any time.
+
+Progress is tracked in two places. Timed attempts are in `history-