0e5f7686ee
Add lessons: untimed guided labs, one per curriculum objective, on a host of their own, in the browser and in the terminal. Each step has a check, a hint and its commands; a lesson writes no history or scores, only a done mark on its objective. CKAD 2.1-2.4 (24 steps) and all 27 CKA objectives (135 steps) have one, and `./ksim lesson audit` proves that every step fails, is done by its own commands, then passes. Add CKA tasks 132-147 (cka/bank_course.py) on what a full CKA course teaches beyond the bank: a real upgrade from v1.36, a drain under a PodDisruptionBudget, QoS and eviction, in-place resize, kube-proxy's chains, encryption at rest, an operator, and CSI expansion, snapshots and clones (ensure_csi_hostpath). Move CKA and CKS to Kubernetes v1.37 and rebuild the hosts on Ubuntu 24.04, with kubelet, kubeadm and kubectl as held packages from pkgs.k8s.io; the desktop runs Ubuntu too. Hosts gain man pages, hexdump and xxd. Fix CKS Q4, which gave 3 of 12 points untouched, and CKA Q4 and Q49, whose solutions raced their new CRD. Audited on v1.37: CKA 147/147, CKAD 105/105, CKS 109/110 (Q103, a Falco task, failed to seed once and awaits a re-audit); lessons: 135 of 135 CKA steps and 24 of 24 CKAD steps. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1034 lines
55 KiB
Python
1034 lines
55 KiB
Python
"""CKA pool tasks 132-147: what a full CKA course teaches beyond the bank as it was - checked on
|
|
2026-10-19 against the topics of KubeDojo's CKA course, and written here from scratch. Each has a
|
|
lesson that teaches it (cka/lessons_*.py); these are the timed tasks that test it.
|
|
|
|
132 upgrades its cluster from v1.36 to v1.37 for real ("k8s": "v1.36"). 133 drains a node under a
|
|
PodDisruptionBudget, 134 debugs a container without a shell, 135 issues and withholds
|
|
ServiceAccount tokens, 136 builds a projected volume, 137 reads QoS classes, 138 resizes a Pod in
|
|
place, 139 sets the kubelet's eviction thresholds, 140 follows a Service into kube-proxy's rules,
|
|
141 backs a Service with a hand-made EndpointSlice, 142 encrypts Secrets at rest, 143 spreads
|
|
replicas with anti-affinity, 144 rolls a StatefulSet out behind a partition, 145 makes an
|
|
operator with broken RBAC do its job, and 146-147 grow, snapshot, restore and clone volumes on
|
|
the CSI hostpath driver's class `csi-hostpath-sc`, as the real exam's clusters have it
|
|
(ensure_csi_hostpath).
|
|
"""
|
|
from cka.common import C, CAN, NET, NS, ONE, backup_operator, ns_pick, name_pick
|
|
|
|
_ETCD = ("ETCDCTL_API=3 etcdctl --endpoints=https://127.0.0.1:2379 --cacert=/etc/kubernetes/pki/etcd/ca.crt "
|
|
"--cert=/etc/kubernetes/pki/etcd/server.crt --key=/etc/kubernetes/pki/etcd/server.key")
|
|
|
|
# packages of a node, and whether they are held
|
|
APT = r"""
|
|
pkgs() { nx "$1" "dpkg-query -W -f='\${Version} ' kubelet kubeadm kubectl" </dev/null; }
|
|
held() { [ "$(nx "$1" 'apt-mark showhold | grep -c -x -e kubelet -e kubeadm -e kubectl' </dev/null)" = 3 ]; }
|
|
"""
|
|
|
|
QUESTIONS = [
|
|
# ------------------------------------------------------------------ 132 ----
|
|
{
|
|
"id": 132, "domain": "ARCH", "difficulty": "hard", "workers": 1, "k8s": "v1.36", "title": "Upgrade the cluster to v1.37",
|
|
"params": lambda r: {"ns": ns_pick(r, "shop")},
|
|
"task": """
|
|
The cluster runs Kubernetes **v1.36**. Upgrade it to **v1.37.1**: first the control-plane node
|
|
`{CP}` (this host), then worker `{W1}`, the way kubeadm documents it.
|
|
|
|
- `{CP}`: the kubeadm package, `kubeadm upgrade apply`, then the kubelet and kubectl packages.
|
|
- `{W1}`: the kubeadm package, `kubeadm upgrade node`, then the kubelet and kubectl packages.
|
|
|
|
Drain each node before its kubelet is upgraded and make it schedulable again afterwards. The three
|
|
packages are held (`apt-mark`) and must be held again when you are done. Deployment `web` in
|
|
`%{ns}` must be available at the end.
|
|
|
|
Reach the worker with `{SHELL_W1}`.
|
|
""",
|
|
"setup": NS + r"""
|
|
mkns %{ns}
|
|
$K -n %{ns} create deploy web --image=nginx:1.27 --replicas=2 >/dev/null
|
|
$K -n %{ns} rollout status deploy web --timeout=180s >/dev/null
|
|
""",
|
|
"checks": [
|
|
C("the API server runs v1.37.1", 3, r"""
|
|
eq "$($K version -o json 2>/dev/null | python3 -c 'import json,sys; print(json.load(sys.stdin)["serverVersion"]["gitVersion"])')" v1.37.1"""),
|
|
C("{CP}: kubelet v1.37.1; kubelet, kubeadm, kubectl packages 1.37.1 and held", 2, APT + r"""
|
|
eq "$(j node "$CP" '{.status.nodeInfo.kubeletVersion}')" v1.37.1 && eq "$(pkgs "$CP")" "1.37.1-1.1 1.37.1-1.1 1.37.1-1.1 " && held "$CP" """),
|
|
C("{W1}: kubelet v1.37.1; kubelet, kubeadm, kubectl packages 1.37.1 and held", 3, APT + r"""
|
|
eq "$(j node "$W1" '{.status.nodeInfo.kubeletVersion}')" v1.37.1 && eq "$(pkgs "$W1")" "1.37.1-1.1 1.37.1-1.1 1.37.1-1.1 " && held "$W1" """),
|
|
C("both nodes Ready and schedulable, Deployment web available", 2, r"""
|
|
for n in "$CP" "$W1"; do
|
|
eq "$(j node "$n" '{.status.conditions[?(@.type=="Ready")].status}')" True && [ "$(j node "$n" '{.spec.unschedulable}')" != true ] || exit 1
|
|
done
|
|
eq "$(j node "$W1" '{.status.nodeInfo.kubeletVersion}')" v1.37.1 && eq "$(j -n %{ns} deploy web '{.status.availableReplicas}')" 2"""),
|
|
],
|
|
"solution": """
|
|
# the control plane - this host
|
|
sudo sed -i 's|/v1.36/|/v1.37/|' /etc/apt/sources.list.d/kubernetes.list # one repository per minor
|
|
sudo apt-get update
|
|
apt-cache madison kubeadm | head -3
|
|
sudo apt-mark unhold kubeadm && sudo apt-get install -y kubeadm=1.37.1-1.1 && sudo apt-mark hold kubeadm
|
|
sudo kubeadm upgrade plan
|
|
sudo kubeadm upgrade apply v1.37.1 -y # a few minutes
|
|
k drain {CP} --ignore-daemonsets --delete-emptydir-data
|
|
sudo apt-mark unhold kubelet kubectl && sudo apt-get install -y kubelet=1.37.1-1.1 kubectl=1.37.1-1.1 && sudo apt-mark hold kubelet kubectl
|
|
sudo systemctl daemon-reload && sudo systemctl restart kubelet
|
|
k uncordon {CP}
|
|
# the worker
|
|
k drain {W1} --ignore-daemonsets --delete-emptydir-data
|
|
{SHELL_W1}
|
|
sudo -i
|
|
sed -i 's|/v1.36/|/v1.37/|' /etc/apt/sources.list.d/kubernetes.list
|
|
apt-get update
|
|
apt-mark unhold kubeadm kubelet kubectl
|
|
apt-get install -y kubeadm=1.37.1-1.1
|
|
kubeadm upgrade node # a worker: node, not apply
|
|
apt-get install -y kubelet=1.37.1-1.1 kubectl=1.37.1-1.1
|
|
apt-mark hold kubeadm kubelet kubectl
|
|
systemctl daemon-reload && systemctl restart kubelet
|
|
exit; exit
|
|
k uncordon {W1}
|
|
k get nodes
|
|
# docs: "Upgrading kubeadm clusters". pkgs.k8s.io has one repository per minor, so 1.37 packages
|
|
# appear only once the line names v1.37. kubeadm first - it drives the upgrade and rewrites the
|
|
# kubelet's configuration; kubelet and kubectl after `upgrade apply`, then the restart: `k get
|
|
# nodes` shows each kubelet's running version. Workers run `kubeadm upgrade node`.
|
|
""",
|
|
},
|
|
# ------------------------------------------------------------------ 133 ----
|
|
{
|
|
"id": 133, "domain": "WORK", "workers": 2, "title": "Maintenance under a disruption budget",
|
|
"params": lambda r: {"ns": ns_pick(r, "orders"), "dep": name_pick(r, "checkout")},
|
|
"task": """
|
|
Node `{W1}` goes down for maintenance tonight. Deployment `%{dep}` in `%{ns}` (3 replicas) must
|
|
never have fewer than **2** Pods available while that happens - not for a drain, and not for any
|
|
other voluntary disruption.
|
|
|
|
1. Create a PodDisruptionBudget `%{dep}` in `%{ns}` that guarantees it.
|
|
2. Drain `{W1}`. DaemonSet Pods stay where they are; Pod `scratch` in `%{ns}` is disposable.
|
|
3. Leave `{W1}` unschedulable. All three Pods of `%{dep}` must be running, on other nodes.
|
|
""",
|
|
"setup": NS + r"""
|
|
mkns %{ns}
|
|
$K uncordon "$W1" >/dev/null 2>&1
|
|
$K -n %{ns} apply -f - >/dev/null <<EOF
|
|
apiVersion: apps/v1
|
|
kind: Deployment
|
|
metadata: {name: %{dep}}
|
|
spec:
|
|
replicas: 3
|
|
selector: {matchLabels: {app: %{dep}}}
|
|
template:
|
|
metadata: {labels: {app: %{dep}}}
|
|
spec:
|
|
topologySpreadConstraints:
|
|
- {maxSkew: 1, topologyKey: kubernetes.io/hostname, whenUnsatisfiable: ScheduleAnyway, labelSelector: {matchLabels: {app: %{dep}}}}
|
|
containers: [{name: web, image: nginx:1.27}]
|
|
---
|
|
apiVersion: v1
|
|
kind: Pod
|
|
metadata: {name: scratch}
|
|
spec:
|
|
nodeName: $W1
|
|
terminationGracePeriodSeconds: 1
|
|
containers: [{name: scratch, image: busybox:1.36, command: [sleep, '86400']}]
|
|
EOF
|
|
$K -n %{ns} rollout status deploy %{dep} --timeout=180s >/dev/null && $K -n %{ns} wait --for=condition=Ready pod/scratch --timeout=120s >/dev/null
|
|
""",
|
|
"checks": [
|
|
C("PDB %{dep} keeps 2 of the 3 Pods available", 2, r"""
|
|
eq "$(j -n %{ns} pdb %{dep} '{.spec.selector.matchLabels.app}')" %{dep} || exit 1
|
|
m=$(j -n %{ns} pdb %{dep} '{.spec.minAvailable}'); u=$(j -n %{ns} pdb %{dep} '{.spec.maxUnavailable}')
|
|
[ "$m" = 2 ] || [ "$u" = 1 ]"""),
|
|
C("{W1} is drained and stays unschedulable", 2, r"""
|
|
eq "$(j node "$W1" '{.spec.unschedulable}')" true || exit 1
|
|
[ -z "$($K -n %{ns} get pods -o jsonpath="{range .items[?(@.spec.nodeName==\"$W1\")]}{.metadata.name}{end}" 2>/dev/null)" ]"""),
|
|
C("all 3 Pods of %{dep} are available on other nodes", 3, r"""
|
|
eq "$(j -n %{ns} deploy %{dep} '{.status.availableReplicas}')" 3 || exit 1
|
|
for n in $($K -n %{ns} get pods -l app=%{dep} -o jsonpath='{.items[*].spec.nodeName}'); do [ "$n" != "$W1" ] || exit 1; done
|
|
eq "$(j node "$W1" '{.spec.unschedulable}')" true"""),
|
|
],
|
|
"solution": """
|
|
k -n %{ns} create pdb %{dep} --selector=app=%{dep} --min-available=2
|
|
k -n %{ns} get pods -o wide # which of them run on {W1}
|
|
k drain {W1} --ignore-daemonsets --delete-emptydir-data --force
|
|
k -n %{ns} get pods -o wide
|
|
# drain = cordon + evict. Evictions respect the PDB: with 3 available and minAvailable 2, one Pod
|
|
# may go at a time, and the drain retries the others until their replacements are ready. --force
|
|
# is needed for `scratch`, which no controller would re-create - it is deleted for good. Leave the
|
|
# node cordoned: `k uncordon {W1}` is for after the maintenance.
|
|
""",
|
|
},
|
|
# ------------------------------------------------------------------ 134 ----
|
|
{
|
|
"id": 134, "domain": "TSHOOT", "workers": 1, "title": "A container you cannot exec into",
|
|
"params": lambda r: {"ns": ns_pick(r, "vault"), "pod": name_pick(r, "sealer"), "mode": r.choice(["sealed-3", "sealed-7", "sealed-9"])},
|
|
"task": """
|
|
Pod `%{pod}` in `%{ns}` runs an image that holds nothing but its own program: `kubectl exec` finds
|
|
no shell. Its container `app` takes its environment from ConfigMap `%{pod}-env` - which someone has
|
|
edited since the container started.
|
|
|
|
Without restarting or replacing the Pod:
|
|
|
|
1. Attach an ephemeral container named `debugger` (image `busybox:1.36`) that sees the processes
|
|
of container `app`.
|
|
2. Find out which value of `MODE` the running process of `app` actually has, and write it to
|
|
`{WS}/q134/mode.txt`.
|
|
""",
|
|
"setup": NS + r"""
|
|
mkns %{ns}; rm -rf "$WS/q134"; mkdir -p "$WS/q134"
|
|
$K -n %{ns} delete pod %{pod} --ignore-not-found --wait=true --grace-period=1 >/dev/null 2>&1
|
|
$K -n %{ns} create configmap %{pod}-env --from-literal=MODE=%{mode} --dry-run=client -o yaml | $K apply -f - >/dev/null
|
|
$K -n %{ns} apply -f - >/dev/null <<'EOF'
|
|
apiVersion: v1
|
|
kind: Pod
|
|
metadata: {name: %{pod}}
|
|
spec:
|
|
containers:
|
|
- name: app
|
|
image: registry.k8s.io/pause:3.10.2
|
|
envFrom: [{configMapRef: {name: %{pod}-env}}]
|
|
EOF
|
|
$K -n %{ns} wait --for=condition=Ready pod/%{pod} --timeout=120s >/dev/null || exit 1
|
|
$K -n %{ns} create configmap %{pod}-env --from-literal=MODE=open --dry-run=client -o yaml | $K apply -f - >/dev/null
|
|
j -n %{ns} pod %{pod} '{.metadata.uid}' > "$PRIV/q134.uid"
|
|
""",
|
|
"checks": [
|
|
C("ephemeral container debugger (busybox:1.36) targets app - in the same Pod, never restarted", 3, r"""
|
|
eq "$(j -n %{ns} pod %{pod} '{.spec.ephemeralContainers[?(@.name=="debugger")].targetContainerName} {.spec.ephemeralContainers[?(@.name=="debugger")].image}')" "app busybox:1.36" \
|
|
&& eq "$(j -n %{ns} pod %{pod} '{.metadata.uid} {.status.containerStatuses[0].restartCount}')" "$(cat "$PRIV/q134.uid") 0" """),
|
|
C("mode.txt holds the value the process runs with", 3, ONE + r"""eq "$(one "$WS/q134/mode.txt")" %{mode}"""),
|
|
],
|
|
"solution": """
|
|
k -n %{ns} exec %{pod} -- sh # fails: no shell in the image
|
|
k -n %{ns} get cm %{pod}-env -o yaml # MODE=open - but that is the ConfigMap NOW
|
|
k -n %{ns} debug %{pod} --image=busybox:1.36 --target=app -c debugger --profile=general -- sleep 3600
|
|
for i in $(seq 1 30); do k -n %{ns} exec %{pod} -c debugger -- true 2>/dev/null && break; sleep 2; done
|
|
k -n %{ns} exec %{pod} -c debugger -- ps # PID 1 is /pause, the app's process
|
|
k -n %{ns} exec %{pod} -c debugger -- sh -c 'tr "\\0" "\\n" < /proc/1/environ' | grep '^MODE='
|
|
k -n %{ns} exec %{pod} -c debugger -- sh -c 'tr "\\0" "\\n" < /proc/1/environ' | sed -n 's/^MODE=//p' > {WS}/q134/mode.txt
|
|
# Environment variables from a ConfigMap are read once, when the container starts: editing the
|
|
# ConfigMap changes nothing for a running process. --target puts the debug container into app's
|
|
# process namespace, so /proc/1 is app's process; the `general` profile gives it SYS_PTRACE,
|
|
# which reading another user's /proc/PID/environ needs. Interactively: `k debug -it ... -- sh`.
|
|
""",
|
|
},
|
|
# ------------------------------------------------------------------ 135 ----
|
|
{
|
|
"id": 135, "domain": "ARCH", "title": "A token for the pipeline, none for the app",
|
|
"params": lambda r: {"ns": ns_pick(r, "ci"), "h": r.choice([2, 3, 4])},
|
|
"task": """
|
|
In Namespace `%{ns}`:
|
|
|
|
1. Create ServiceAccount `pipeline`, and allow it - through a Role and a RoleBinding, both named
|
|
`pipeline` - to `get`, `list`, `create`, `update` and `patch` Deployments, and nothing else.
|
|
2. Issue a token for `pipeline` that is valid for **%{h} hours**, and save just the token in
|
|
`{WS}/q135/pipeline.token`.
|
|
3. Deployment `frontend` never talks to the API server. Change it so that its Pods get **no**
|
|
ServiceAccount token at all. It keeps running as the `default` ServiceAccount.
|
|
""",
|
|
"setup": NS + r"""
|
|
mkns %{ns}; rm -rf "$WS/q135"; mkdir -p "$WS/q135"
|
|
$K -n %{ns} create deploy frontend --image=nginx:1.27 --replicas=2 >/dev/null
|
|
$K -n %{ns} rollout status deploy frontend --timeout=180s >/dev/null
|
|
""",
|
|
"checks": [
|
|
C("pipeline may get, list, create, update, patch Deployments - nothing more", 2, CAN + r"""
|
|
S=system:serviceaccount:%{ns}:pipeline
|
|
for v in get list create update patch; do can --as=$S -- $v deployments.apps -n %{ns} || exit 1; done
|
|
cannot --as=$S -- delete deployments.apps -n %{ns} && cannot --as=$S -- get secrets -n %{ns} && cannot --as=$S -- list pods -n %{ns} \
|
|
&& eq "$(j -n %{ns} rolebinding pipeline '{.roleRef.kind}/{.roleRef.name}')" Role/pipeline"""),
|
|
C("pipeline.token authenticates as pipeline and lasts %{h} hours", 3, r"""
|
|
t=$(tr -d ' \r\n' < "$WS/q135/pipeline.token" 2>/dev/null); [ -n "$t" ] || exit 1
|
|
srv=$($K config view --minify -o jsonpath='{.clusters[0].cluster.server}')
|
|
eq "$($K --kubeconfig=/dev/null --server="$srv" --insecure-skip-tls-verify --token="$t" auth whoami -o jsonpath='{.status.userInfo.username}' 2>/dev/null)" system:serviceaccount:%{ns}:pipeline || exit 1
|
|
echo "$t" | cut -d. -f2 | tr '_-' '/+' | python3 -c '
|
|
import base64, json, sys
|
|
s = sys.stdin.read().strip(); c = json.loads(base64.b64decode(s + "=" * (-len(s) % 4)))
|
|
sys.exit(0 if c["exp"] - c["iat"] == %{h} * 3600 else 1)'"""),
|
|
C("frontend's Pods run with no ServiceAccount token, as default", 3, r"""
|
|
eq "$(j -n %{ns} deploy frontend '{.status.availableReplicas} {.status.updatedReplicas}')" "2 2" || exit 1
|
|
sa=$(j -n %{ns} deploy frontend '{.spec.template.spec.serviceAccountName}'); [ -z "$sa" ] || [ "$sa" = default ] || exit 1
|
|
for p in $($K -n %{ns} get pods -l app=frontend -o name); do
|
|
$K -n %{ns} get "$p" -o json | python3 -c '
|
|
import json, sys; s = json.load(sys.stdin)["spec"]
|
|
tok = [v for v in s.get("volumes", []) if any("serviceAccountToken" in src for src in v.get("projected", {}).get("sources", []))]
|
|
sys.exit(1 if tok else 0)' || exit 1
|
|
done"""),
|
|
],
|
|
"solution": """
|
|
k -n %{ns} create serviceaccount pipeline
|
|
k -n %{ns} create role pipeline --verb=get,list,create,update,patch --resource=deployments.apps
|
|
k -n %{ns} create rolebinding pipeline --role=pipeline --serviceaccount=%{ns}:pipeline
|
|
k auth can-i delete deployments -n %{ns} --as=system:serviceaccount:%{ns}:pipeline # no
|
|
k -n %{ns} create token pipeline --duration=%{h}h > {WS}/q135/pipeline.token
|
|
k -n %{ns} patch deploy frontend -p '{"spec":{"template":{"spec":{"automountServiceAccountToken":false}}}}'
|
|
k -n %{ns} rollout status deploy frontend
|
|
# `create token` asks the TokenRequest API for a short-lived, bound token - nothing is stored in a
|
|
# Secret. automountServiceAccountToken: false on the Pod template (or on the ServiceAccount; the
|
|
# Pod's setting wins) leaves out the projected volume kube-api-access-*. To try a token alone, use
|
|
# no kubeconfig: a client certificate in yours would be checked first.
|
|
""",
|
|
},
|
|
# ------------------------------------------------------------------ 136 ----
|
|
{
|
|
"id": 136, "domain": "STOR", "title": "One mount, four sources",
|
|
"params": lambda r: {"ns": ns_pick(r, "portal"), "pod": name_pick(r, "portal"), "mem": r.choice([96, 128, 160])},
|
|
"task": """
|
|
Create Pod `%{pod}` in `%{ns}`: image `busybox:1.36`, command `sleep 86400`, label `app=%{pod}`,
|
|
memory limit `%{mem}Mi`. Mount a **single** volume, read-only, at `/etc/portal`, that holds:
|
|
|
|
- every key of ConfigMap `portal-config`, each as a file of its name;
|
|
- key `password` of Secret `portal-db`, as the file `db/password`;
|
|
- the Pod's labels, as the file `labels`;
|
|
- the container's memory limit in MiB, as the file `mem-limit` (just the number).
|
|
""",
|
|
"setup": NS + r"""
|
|
mkns %{ns}
|
|
$K -n %{ns} delete pod %{pod} --ignore-not-found --wait=true --grace-period=1 >/dev/null 2>&1
|
|
$K -n %{ns} create configmap portal-config --from-literal=theme=dark --from-literal=locale=en_GB --dry-run=client -o yaml | $K apply -f - >/dev/null
|
|
$K -n %{ns} create secret generic portal-db --from-literal=password=pw-%{mem}-x --from-literal=user=portal --dry-run=client -o yaml | $K apply -f - >/dev/null
|
|
""",
|
|
"checks": [
|
|
C("one projected volume, read-only at /etc/portal", 2, r"""
|
|
eq "$(j -n %{ns} pod %{pod} '{.spec.containers[0].volumeMounts[?(@.mountPath=="/etc/portal")].readOnly}')" true || exit 1
|
|
v=$(j -n %{ns} pod %{pod} '{.spec.containers[0].volumeMounts[?(@.mountPath=="/etc/portal")].name}')
|
|
[ -n "$(j -n %{ns} pod %{pod} "{.spec.volumes[?(@.name==\"$v\")].projected.sources}")" ]"""),
|
|
C("the ConfigMap's keys and db/password are files there", 2, r"""
|
|
x() { $K -n %{ns} exec %{pod} -- cat "/etc/portal/$1" 2>/dev/null; }
|
|
eq "$(x theme)" dark && eq "$(x locale)" en_GB && eq "$(x db/password)" pw-%{mem}-x && ! $K -n %{ns} exec %{pod} -- test -e /etc/portal/user"""),
|
|
C("labels and mem-limit come from the downward API", 3, r"""
|
|
x() { $K -n %{ns} exec %{pod} -- cat "/etc/portal/$1" 2>/dev/null; }
|
|
x labels | grep -qx 'app="%{pod}"' && eq "$(x mem-limit)" %{mem}"""),
|
|
],
|
|
"solution": """
|
|
k -n %{ns} apply -f - <<'EOF'
|
|
apiVersion: v1
|
|
kind: Pod
|
|
metadata:
|
|
name: %{pod}
|
|
labels: {app: %{pod}}
|
|
spec:
|
|
containers:
|
|
- name: portal
|
|
image: busybox:1.36
|
|
command: [sleep, '86400']
|
|
resources:
|
|
limits: {memory: %{mem}Mi}
|
|
volumeMounts:
|
|
- {name: portal, mountPath: /etc/portal, readOnly: true}
|
|
volumes:
|
|
- name: portal
|
|
projected:
|
|
sources:
|
|
- configMap:
|
|
name: portal-config
|
|
- secret:
|
|
name: portal-db
|
|
items:
|
|
- {key: password, path: db/password}
|
|
- downwardAPI:
|
|
items:
|
|
- path: labels
|
|
fieldRef: {fieldPath: metadata.labels}
|
|
- path: mem-limit
|
|
resourceFieldRef: {containerName: portal, resource: limits.memory, divisor: 1Mi}
|
|
EOF
|
|
k -n %{ns} wait --for=condition=Ready pod/%{pod}
|
|
k -n %{ns} exec %{pod} -- sh -c 'ls -R /etc/portal; cat /etc/portal/labels /etc/portal/mem-limit'
|
|
# `items` picks keys and renames them: only `password` from the Secret, as db/password. A
|
|
# resourceFieldRef needs the container's name in a volume (in an env var it is implied), and the
|
|
# divisor turns bytes into MiB.
|
|
""",
|
|
},
|
|
# ------------------------------------------------------------------ 137 ----
|
|
{
|
|
"id": 137, "domain": "WORK", "title": "Who is evicted first",
|
|
"params": lambda r: {"ns": ns_pick(r, "batch")},
|
|
"task": """
|
|
Namespace `%{ns}` runs Pods `alpha`, `beta`, `gamma` and `delta`, and Deployment `critical`.
|
|
|
|
1. Write the QoS class of each of the four Pods to `{WS}/q137/qos.txt`, one per line, as
|
|
`POD CLASS`.
|
|
2. When the node runs short of memory, which of the four does the kubelet evict **first**? Write
|
|
its name to `{WS}/q137/first.txt`.
|
|
3. Change Deployment `critical` so that its Pods are of class `Guaranteed`, with `200m` CPU and
|
|
`128Mi` memory.
|
|
""",
|
|
"setup": NS + r"""
|
|
mkns %{ns}; rm -rf "$WS/q137"; mkdir -p "$WS/q137"
|
|
$K -n %{ns} delete pod alpha beta gamma delta --ignore-not-found --wait=true --grace-period=1 >/dev/null 2>&1
|
|
$K -n %{ns} apply -f - >/dev/null <<'EOF'
|
|
apiVersion: v1
|
|
kind: Pod
|
|
metadata: {name: alpha}
|
|
spec:
|
|
containers: [{name: c, image: busybox:1.36, command: [sleep, '86400'], resources: {requests: {cpu: 50m, memory: 32Mi}, limits: {cpu: 50m, memory: 32Mi}}}]
|
|
---
|
|
apiVersion: v1
|
|
kind: Pod
|
|
metadata: {name: beta}
|
|
spec:
|
|
containers: [{name: c, image: busybox:1.36, command: [sleep, '86400'], resources: {requests: {memory: 32Mi}}}]
|
|
---
|
|
apiVersion: v1
|
|
kind: Pod
|
|
metadata: {name: gamma}
|
|
spec:
|
|
containers: [{name: c, image: busybox:1.36, command: [sleep, '86400']}]
|
|
---
|
|
apiVersion: v1
|
|
kind: Pod
|
|
metadata: {name: delta}
|
|
spec:
|
|
containers:
|
|
- {name: main, image: busybox:1.36, command: [sleep, '86400'], resources: {requests: {cpu: 50m, memory: 32Mi}, limits: {cpu: 50m, memory: 32Mi}}}
|
|
- {name: helper, image: busybox:1.36, command: [sleep, '86400']}
|
|
---
|
|
apiVersion: apps/v1
|
|
kind: Deployment
|
|
metadata: {name: critical}
|
|
spec:
|
|
replicas: 1
|
|
selector: {matchLabels: {app: critical}}
|
|
template:
|
|
metadata: {labels: {app: critical}}
|
|
spec:
|
|
containers: [{name: app, image: nginx:1.27, resources: {requests: {cpu: 100m, memory: 64Mi}}}]
|
|
EOF
|
|
$K -n %{ns} wait --for=condition=Ready pod --all --timeout=180s >/dev/null
|
|
""",
|
|
"checks": [
|
|
C("qos.txt names each Pod's class", 3, r"""
|
|
eq "$(tr -s ' \t' ' ' < "$WS/q137/qos.txt" 2>/dev/null | tr -d '\r' | sed 's/ *$//' | grep . | sort)" "$(printf 'alpha Guaranteed\nbeta Burstable\ndelta Burstable\ngamma BestEffort')" """),
|
|
C("first.txt names the Pod evicted first", 2, ONE + r"""eq "$(one "$WS/q137/first.txt")" gamma"""),
|
|
C("critical's Pods are Guaranteed with 200m and 128Mi", 3, r"""
|
|
eq "$(j -n %{ns} deploy critical '{.status.availableReplicas} {.status.updatedReplicas}')" "1 1" || exit 1
|
|
eq "$(j -n %{ns} deploy critical '{.spec.template.spec.containers[0].resources.requests.cpu} {.spec.template.spec.containers[0].resources.limits.cpu} {.spec.template.spec.containers[0].resources.requests.memory} {.spec.template.spec.containers[0].resources.limits.memory}')" "200m 200m 128Mi 128Mi" || exit 1
|
|
for q in $($K -n %{ns} get pods -l app=critical -o jsonpath='{.items[*].status.qosClass}'); do [ "$q" = Guaranteed ] || exit 1; done"""),
|
|
],
|
|
"solution": """
|
|
k -n %{ns} get pods -o custom-columns=NAME:.metadata.name,QOS:.status.qosClass
|
|
k -n %{ns} get pods alpha beta delta gamma -o jsonpath='{range .items[*]}{.metadata.name} {.status.qosClass}{"\\n"}{end}' > {WS}/q137/qos.txt
|
|
echo gamma > {WS}/q137/first.txt
|
|
k -n %{ns} set resources deploy critical --requests=cpu=200m,memory=128Mi --limits=cpu=200m,memory=128Mi
|
|
k -n %{ns} rollout status deploy critical
|
|
k -n %{ns} get pods -l app=critical -o jsonpath='{.items[*].status.qosClass}'; echo
|
|
# Guaranteed: every container has requests equal to limits for both CPU and memory. BestEffort: no
|
|
# container has any. Everything else is Burstable - delta too, because one container sets nothing.
|
|
# Under memory pressure the kubelet evicts BestEffort first, then Burstable Pods using more than
|
|
# they requested; Guaranteed Pods last.
|
|
""",
|
|
},
|
|
# ------------------------------------------------------------------ 138 ----
|
|
{
|
|
"id": 138, "domain": "WORK", "title": "More CPU, same Pod",
|
|
"params": lambda r: {"ns": ns_pick(r, "render"), "pod": name_pick(r, "encoder"), "req": r.choice(["250m", "300m"]), "lim": r.choice(["500m", "600m"])},
|
|
"task": """
|
|
Pod `%{pod}` in `%{ns}` is throttled at its CPU limit. Give its container `encoder` a CPU request
|
|
of `%{req}` and a CPU limit of `%{lim}` - **without** replacing the Pod and without restarting the
|
|
container. Its memory request and limit stay as they are.
|
|
""",
|
|
"setup": NS + r"""
|
|
mkns %{ns}
|
|
$K -n %{ns} delete pod %{pod} --ignore-not-found --wait=true --grace-period=1 >/dev/null 2>&1
|
|
$K -n %{ns} apply -f - >/dev/null <<'EOF'
|
|
apiVersion: v1
|
|
kind: Pod
|
|
metadata: {name: %{pod}}
|
|
spec:
|
|
containers:
|
|
- name: encoder
|
|
image: busybox:1.36
|
|
command: [sh, -c, 'while true; do :; done']
|
|
resources: {requests: {cpu: 100m, memory: 48Mi}, limits: {cpu: 200m, memory: 48Mi}}
|
|
EOF
|
|
$K -n %{ns} wait --for=condition=Ready pod/%{pod} --timeout=120s >/dev/null || exit 1
|
|
j -n %{ns} pod %{pod} '{.metadata.uid}' > "$PRIV/q138.uid"
|
|
""",
|
|
"checks": [
|
|
C("its spec asks for %{req} / %{lim} CPU, memory unchanged", 2, r"""
|
|
eq "$(j -n %{ns} pod %{pod} '{.spec.containers[0].resources.requests.cpu} {.spec.containers[0].resources.limits.cpu} {.spec.containers[0].resources.requests.memory} {.spec.containers[0].resources.limits.memory}')" "%{req} %{lim} 48Mi 48Mi" """),
|
|
C("the running container has the new CPU limit", 2, r"""
|
|
eq "$(j -n %{ns} pod %{pod} '{.status.containerStatuses[0].resources.requests.cpu} {.status.containerStatuses[0].resources.limits.cpu}')" "%{req} %{lim}" """),
|
|
C("in the same Pod, and its container never restarted", 2, r"""
|
|
eq "$(j -n %{ns} pod %{pod} '{.spec.containers[0].resources.limits.cpu}')" %{lim} \
|
|
&& eq "$(j -n %{ns} pod %{pod} '{.metadata.uid} {.status.containerStatuses[0].restartCount}')" "$(cat "$PRIV/q138.uid") 0" """),
|
|
],
|
|
"solution": """
|
|
k -n %{ns} get pod %{pod} -o jsonpath='{.spec.containers[0].resources}'; echo
|
|
k -n %{ns} patch pod %{pod} --subresource resize --patch '{"spec":{"containers":[{"name":"encoder","resources":{"requests":{"cpu":"%{req}"},"limits":{"cpu":"%{lim}"}}}]}}'
|
|
k -n %{ns} get pod %{pod} -o jsonpath='{.status.containerStatuses[0].resources}{"\\n"}{.status.containerStatuses[0].restartCount}{"\\n"}'
|
|
# A running Pod's CPU and memory can be changed in place, through the `resize` subresource - a
|
|
# plain `k edit` or `k apply` of a Pod's resources is refused. The spec
|
|
# says what you asked for; status.containerStatuses[].resources what the kubelet has applied.
|
|
# A container's resizePolicy decides whether a resource's change needs a restart (CPU: no).
|
|
""",
|
|
},
|
|
# ------------------------------------------------------------------ 139 ----
|
|
{
|
|
"id": 139, "domain": "TSHOOT", "workers": 1, "title": "Evict before the node runs dry",
|
|
"params": lambda r: {"mem": r.choice([200, 250, 300]), "fs": r.choice([10, 12, 15]), "soft": r.choice([400, 500])},
|
|
"task": """
|
|
Node `{W1}` ran out of memory last week and froze before its kubelet did anything about it.
|
|
Configure the kubelet on `{W1}` - in its configuration file, so that it lasts - to:
|
|
|
|
- evict Pods at once when less than `%{mem}Mi` of memory, or less than `%{fs}%` of the node
|
|
filesystem, is available;
|
|
- evict Pods when less than `%{soft}Mi` of memory has been available for **90 seconds**.
|
|
|
|
Leave its other settings as they are. The node must be `Ready` afterwards. Reach it with
|
|
`{SHELL_W1}`.
|
|
""",
|
|
"setup": r"""
|
|
nx "$W1" 'cp -n /var/lib/kubelet/config.yaml /var/lib/kubelet/config.yaml.ksim 2>/dev/null; true' </dev/null
|
|
""",
|
|
"checks": [
|
|
C("hard thresholds: memory.available %{mem}Mi, nodefs.available %{fs}%", 3, r"""
|
|
$K get --raw "/api/v1/nodes/$W1/proxy/configz" 2>/dev/null | python3 -c '
|
|
import json, sys; c = json.load(sys.stdin)["kubeletconfig"]; e = c.get("evictionHard") or {}
|
|
sys.exit(0 if e.get("memory.available") == "%{mem}Mi" and e.get("nodefs.available") == "%{fs}%" else 1)'"""),
|
|
C("soft threshold memory.available %{soft}Mi with a 90 s grace period", 3, r"""
|
|
$K get --raw "/api/v1/nodes/$W1/proxy/configz" 2>/dev/null | python3 -c '
|
|
import json, re, sys; c = json.load(sys.stdin)["kubeletconfig"]
|
|
s = (c.get("evictionSoft") or {}).get("memory.available"); g = (c.get("evictionSoftGracePeriod") or {}).get("memory.available", "")
|
|
m = re.fullmatch(r"(?:(\d+)m)?(?:(\d+)s)?", g); secs = int(m.group(1) or 0) * 60 + int(m.group(2) or 0) if m and g else -1
|
|
sys.exit(0 if s == "%{soft}Mi" and secs == 90 else 1)'"""),
|
|
C("set in the file, the other settings kept, the node Ready", 2, r"""
|
|
nx "$W1" 'grep -q "%{soft}Mi" /var/lib/kubelet/config.yaml && grep -q "%{mem}Mi" /var/lib/kubelet/config.yaml && grep -q "^staticPodPath:" /var/lib/kubelet/config.yaml && systemctl is-active -q kubelet' </dev/null || exit 1
|
|
eq "$(j node "$W1" '{.status.conditions[?(@.type=="Ready")].status}')" True"""),
|
|
],
|
|
"solution": """
|
|
{SHELL_W1}
|
|
sudo -i
|
|
grep -n -A4 '^eviction' /var/lib/kubelet/config.yaml # kind sets the hard thresholds to 0%
|
|
yq -i '.evictionHard."memory.available" = "%{mem}Mi" | .evictionHard."nodefs.available" = "%{fs}%"
|
|
| .evictionSoft."memory.available" = "%{soft}Mi" | .evictionSoftGracePeriod."memory.available" = "1m30s"' /var/lib/kubelet/config.yaml
|
|
systemctl restart kubelet
|
|
journalctl -u kubelet -n 5 --no-pager
|
|
exit; exit
|
|
k get --raw /api/v1/nodes/{W1}/proxy/configz | jq '.kubeletconfig | {evictionHard, evictionSoft, evictionSoftGracePeriod}'
|
|
# A soft threshold needs a grace period for the same signal, or the kubelet refuses to start. By
|
|
# hand: under evictionHard add `memory.available: "%{mem}Mi"` and change nodefs.available; then
|
|
# evictionSoft:
|
|
# memory.available: "%{soft}Mi"
|
|
# evictionSoftGracePeriod:
|
|
# memory.available: "1m30s"
|
|
# configz shows what the running kubelet uses.
|
|
""",
|
|
},
|
|
# ------------------------------------------------------------------ 140 ----
|
|
{
|
|
"id": 140, "domain": "TSHOOT", "title": "Follow a Service into kube-proxy's rules",
|
|
"params": lambda r: {"ns": ns_pick(r, "pay"), "svc": name_pick(r, "ledger"), "n": r.choice([2, 3, 4])},
|
|
"task": """
|
|
Service `%{svc}` in `%{ns}` works through the rules kube-proxy writes on every node. On `{CP}`
|
|
(this host):
|
|
|
|
1. Write the mode kube-proxy runs in to `{WS}/q140/mode.txt`.
|
|
2. Write the name of the chain kube-proxy made for Service `%{svc}` - the one traffic to its
|
|
ClusterIP is sent to - to `{WS}/q140/chain.txt`.
|
|
3. Write the number of endpoint chains that chain spreads the traffic over to
|
|
`{WS}/q140/endpoints.txt`.
|
|
""",
|
|
"setup": NS + r"""
|
|
mkns %{ns}; rm -rf "$WS/q140"; mkdir -p "$WS/q140"
|
|
$K -n %{ns} create deploy %{svc} --image=nginx:1.27 --replicas=%{n} --port=80 --dry-run=client -o yaml | $K apply -f - >/dev/null
|
|
$K -n %{ns} expose deploy %{svc} --port=80 --dry-run=client -o yaml | $K apply -f - >/dev/null
|
|
$K -n %{ns} rollout status deploy %{svc} --timeout=180s >/dev/null
|
|
""",
|
|
"checks": [
|
|
C("mode.txt: kube-proxy's mode", 2, ONE + r"""
|
|
m=$($K -n kube-system get cm kube-proxy -o jsonpath='{.data.config\.conf}' | awk '/^mode:/ {print $2}' | tr -d '"'); m=${m:-iptables}
|
|
eq "$(one "$WS/q140/mode.txt" | tr -d '"')" "$m" """),
|
|
C("chain.txt: the Service's KUBE-SVC chain", 3, ONE + r"""
|
|
ip=$(j -n %{ns} svc %{svc} '{.spec.clusterIP}')
|
|
want=$(nx "$CP" "iptables-save -t nat | grep -- '-d $ip/32' | grep -o 'KUBE-SVC-[A-Z0-9]*' | head -1" </dev/null)
|
|
[ -n "$want" ] && eq "$(one "$WS/q140/chain.txt")" "$want" """),
|
|
C("endpoints.txt: the number of KUBE-SEP chains behind it", 2, ONE + r"""
|
|
ip=$(j -n %{ns} svc %{svc} '{.spec.clusterIP}')
|
|
ch=$(nx "$CP" "iptables-save -t nat | grep -- '-d $ip/32' | grep -o 'KUBE-SVC-[A-Z0-9]*' | head -1" </dev/null)
|
|
n=$(nx "$CP" "iptables-save -t nat | grep -- '^-A $ch ' | grep -c -- '-j KUBE-SEP-'" </dev/null)
|
|
eq "$n" %{n} && eq "$(one "$WS/q140/endpoints.txt")" "$n" """),
|
|
],
|
|
"solution": """
|
|
k -n kube-system get cm kube-proxy -o jsonpath='{.data.config\\.conf}' | grep '^mode' # iptables
|
|
echo iptables > {WS}/q140/mode.txt
|
|
ip=$(k -n %{ns} get svc %{svc} -o jsonpath='{.spec.clusterIP}')
|
|
sudo iptables-save -t nat | grep -- "-d $ip/32" # KUBE-SERVICES ... -j KUBE-SVC-XXXX
|
|
chain=$(sudo iptables-save -t nat | grep -- "-d $ip/32" | grep -o 'KUBE-SVC-[A-Z0-9]*' | head -1)
|
|
echo $chain > {WS}/q140/chain.txt
|
|
sudo iptables-save -t nat | grep -- "^-A $chain " # one -j KUBE-SEP-... per endpoint
|
|
sudo iptables-save -t nat | grep -- "^-A $chain " | grep -c -- '-j KUBE-SEP-' > {WS}/q140/endpoints.txt
|
|
# KUBE-SERVICES matches ClusterIP:port and jumps to the Service's KUBE-SVC chain; that picks an
|
|
# endpoint chain at random (statistic mode random), and each KUBE-SEP chain DNATs to one Pod.
|
|
# One KUBE-SEP per ready endpoint. In nftables mode: `sudo nft list ruleset | grep -A5 %{svc}`.
|
|
""",
|
|
},
|
|
# ------------------------------------------------------------------ 141 ----
|
|
{
|
|
"id": 141, "domain": "NET", "workers": 1, "title": "A Service for an API outside the cluster",
|
|
"params": lambda r: {"ns": ns_pick(r, "apps"), "port": r.choice([8080, 9090])},
|
|
"task": """
|
|
The billing API is not part of the cluster: it answers at the address and port written in
|
|
`{WS}/q141/billing.txt`. Give the applications in `%{ns}` a stable name for it:
|
|
|
|
1. Create Service `billing` in `%{ns}`, port `80`, **without** a selector.
|
|
2. Its endpoints are yours to maintain: create EndpointSlice `billing-1` that sends its traffic
|
|
to that address and port.
|
|
|
|
`wget -qO- billing` from Pod `client` in `%{ns}` must return the API's answer.
|
|
""",
|
|
"setup": NS + r"""
|
|
mkns %{ns}; mkns billing-host; rm -rf "$WS/q141"; mkdir -p "$WS/q141"
|
|
$K -n %{ns} delete svc billing --ignore-not-found >/dev/null 2>&1
|
|
$K -n %{ns} run client --image=busybox:1.36 -- sleep 86400 >/dev/null 2>&1
|
|
$K -n billing-host run api --image=busybox:1.36 --overrides="{\"spec\":{\"hostNetwork\":true,\"nodeName\":\"$W1\"}}" --command -- \
|
|
sh -c 'mkdir -p /www && echo billing-api-%{port} > /www/index.html && exec httpd -f -p %{port} -h /www' >/dev/null 2>&1
|
|
$K -n %{ns} wait --for=condition=Ready pod/client --timeout=120s >/dev/null && $K -n billing-host wait --for=condition=Ready pod/api --timeout=120s >/dev/null || exit 1
|
|
echo "$(j node "$W1" '{.status.addresses[?(@.type=="InternalIP")].address}'):%{port}" > "$WS/q141/billing.txt"
|
|
""",
|
|
"checks": [
|
|
C("Service billing, port 80, has no selector", 1, r"""eq "$(j -n %{ns} svc billing '{.spec.ports[0].port}')" 80 && [ -z "$(j -n %{ns} svc billing '{.spec.selector}')" ]"""),
|
|
C("EndpointSlice billing-1 belongs to it and holds the API's address and port", 3, r"""
|
|
a=$(cut -d: -f1 "$WS/q141/billing.txt")
|
|
eq "$(j -n %{ns} endpointslice billing-1 '{.metadata.labels.kubernetes\.io/service-name} {.addressType} {.endpoints[0].addresses[0]} {.ports[0].port}')" "billing IPv4 $a %{port}" """),
|
|
C("wget billing from client returns the API's answer", 3, r"""
|
|
for i in 1 2 3 4 5; do [ "$($K -n %{ns} exec client -- wget -qO- -T 3 billing 2>/dev/null)" = billing-api-%{port} ] && exit 0; sleep 3; done; exit 1"""),
|
|
],
|
|
"solution": """
|
|
cat {WS}/q141/billing.txt # ADDRESS:PORT
|
|
addr=$(cut -d: -f1 {WS}/q141/billing.txt); port=$(cut -d: -f2 {WS}/q141/billing.txt)
|
|
k -n %{ns} apply -f - <<EOF
|
|
apiVersion: v1
|
|
kind: Service
|
|
metadata:
|
|
name: billing
|
|
spec:
|
|
ports:
|
|
- {port: 80, protocol: TCP}
|
|
---
|
|
apiVersion: discovery.k8s.io/v1
|
|
kind: EndpointSlice
|
|
metadata:
|
|
name: billing-1
|
|
labels:
|
|
kubernetes.io/service-name: billing
|
|
addressType: IPv4
|
|
endpoints:
|
|
- addresses: ["$addr"]
|
|
ports:
|
|
- {port: $port, protocol: TCP}
|
|
EOF
|
|
k -n %{ns} exec client -- wget -qO- -T3 billing
|
|
# Without a selector, no controller writes endpoints for the Service: the label
|
|
# kubernetes.io/service-name ties a slice to it. The slice's port is where traffic goes (the
|
|
# Service's targetPort plays no part), and its name must match the Service port's - here both
|
|
# unnamed. ExternalName would need a DNS name; an address needs this.
|
|
""",
|
|
},
|
|
# ------------------------------------------------------------------ 142 ----
|
|
{
|
|
"id": 142, "domain": "WORK", "difficulty": "hard", "title": "Secrets encrypted at rest",
|
|
"params": lambda r: {"ns": ns_pick(r, "bank"), "sec": name_pick(r, "vault-key")},
|
|
"task": """
|
|
This cluster stores Secrets in etcd in plain text. Change that:
|
|
|
|
1. Write an `EncryptionConfiguration` to `/etc/kubernetes/enc/enc.yaml` on `{CP}` (this host) that
|
|
encrypts `secrets` with the `aescbc` provider, with one key named `key1`, and can still read
|
|
Secrets stored unencrypted.
|
|
2. Make the API server use it.
|
|
3. Secret `%{sec}` in `%{ns}` exists already: make sure it, too, is stored encrypted.
|
|
""",
|
|
"setup": NS + r"""
|
|
mkns %{ns}
|
|
$K -n %{ns} create secret generic %{sec} --from-literal=key=k-%{sec} --dry-run=client -o yaml | $K apply -f - >/dev/null
|
|
""",
|
|
"checks": [
|
|
C("enc.yaml encrypts secrets with aescbc key1 first, identity after", 1, r"""
|
|
nx "$CP" 'cat /etc/kubernetes/enc/enc.yaml' </dev/null 2>/dev/null | python3 -c '
|
|
import sys, yaml
|
|
c = yaml.safe_load(sys.stdin); r = [x for x in c["resources"] if "secrets" in x["resources"]][0]["providers"]
|
|
sys.exit(0 if "aescbc" in r[0] and r[0]["aescbc"]["keys"][0]["name"] == "key1" and any("identity" in p for p in r[1:]) else 1)' 2>/dev/null \
|
|
|| nx "$CP" 'grep -q aescbc /etc/kubernetes/enc/enc.yaml && grep -q "name: key1" /etc/kubernetes/enc/enc.yaml && grep -q identity /etc/kubernetes/enc/enc.yaml' </dev/null"""),
|
|
C("the API server runs with --encryption-provider-config", 2, r"""
|
|
nx "$CP" 'grep -q -- "--encryption-provider-config=/etc/kubernetes/enc/enc.yaml" /etc/kubernetes/manifests/kube-apiserver.yaml' </dev/null || exit 1
|
|
$K -n kube-system get pod -l component=kube-apiserver -o jsonpath='{.items[0].spec.containers[0].command}' | grep -q encryption-provider-config"""),
|
|
C("Secret %{sec} is stored encrypted, and still reads back", 3, r"""
|
|
nx "$CP" '""" + _ETCD + r""" get /registry/secrets/%{ns}/%{sec} --print-value-only 2>/dev/null | head -c 64' </dev/null | grep -aq '^k8s:enc:aescbc:v1:key1:' \
|
|
&& eq "$(j -n %{ns} secret %{sec} '{.data.key}' | base64 -d)" k-%{sec}"""),
|
|
C("a new Secret is stored encrypted too", 2, r"""
|
|
$K -n %{ns} delete secret ksim-probe --ignore-not-found >/dev/null 2>&1
|
|
$K -n %{ns} create secret generic ksim-probe --from-literal=x=y >/dev/null 2>&1 || exit 1
|
|
nx "$CP" '""" + _ETCD + r""" get /registry/secrets/%{ns}/ksim-probe --print-value-only 2>/dev/null | head -c 64' </dev/null | grep -aq '^k8s:enc:aescbc:v1:'; r=$?
|
|
$K -n %{ns} delete secret ksim-probe --ignore-not-found >/dev/null 2>&1; exit $r"""),
|
|
],
|
|
"solution": """
|
|
sudo mkdir -p /etc/kubernetes/enc
|
|
sudo tee /etc/kubernetes/enc/enc.yaml >/dev/null <<EOF
|
|
apiVersion: apiserver.config.k8s.io/v1
|
|
kind: EncryptionConfiguration
|
|
resources:
|
|
- resources: [secrets]
|
|
providers:
|
|
- aescbc:
|
|
keys:
|
|
- name: key1
|
|
secret: $(head -c 32 /dev/urandom | base64)
|
|
- identity: {}
|
|
EOF
|
|
sudo cp /etc/kubernetes/manifests/kube-apiserver.yaml /root/kube-apiserver.yaml.bak # a backup OUTSIDE manifests/
|
|
sudo cp /etc/kubernetes/manifests/kube-apiserver.yaml /tmp/kube-apiserver.yaml
|
|
sudo yq -i '.spec.containers[0].command += ["--encryption-provider-config=/etc/kubernetes/enc/enc.yaml"]
|
|
| .spec.containers[0].volumeMounts += [{"name": "enc", "mountPath": "/etc/kubernetes/enc", "readOnly": true}]
|
|
| .spec.volumes += [{"name": "enc", "hostPath": {"path": "/etc/kubernetes/enc", "type": "DirectoryOrCreate"}}]' /tmp/kube-apiserver.yaml
|
|
sudo cp /tmp/kube-apiserver.yaml /etc/kubernetes/manifests/kube-apiserver.yaml
|
|
sleep 20; for i in $(seq 1 40); do k get --raw /readyz >/dev/null 2>&1 && break; sleep 3; done
|
|
k -n %{ns} get secret %{sec} -o json | k replace -f - # stored again: encrypted now
|
|
sudo ETCDCTL_API=3 etcdctl --endpoints=https://127.0.0.1:2379 --cacert=/etc/kubernetes/pki/etcd/ca.crt \\
|
|
--cert=/etc/kubernetes/pki/etcd/server.crt --key=/etc/kubernetes/pki/etcd/server.key \\
|
|
get /registry/secrets/%{ns}/%{sec} | od -c | head -3 # k 8 s : e n c : a e s c b c : v 1 : k e y 1
|
|
# By hand: in kube-apiserver.yaml add the flag to the command, a volumeMount for /etc/kubernetes/enc
|
|
# and a hostPath volume of the same path - the API server runs in a container and sees only what is
|
|
# mounted. The FIRST provider encrypts; `identity` after it reads what was stored before. Every
|
|
# Secret: `k get secrets -A -o json | k replace -f -`.
|
|
""",
|
|
},
|
|
# ------------------------------------------------------------------ 143 ----
|
|
{
|
|
"id": 143, "domain": "WORK", "workers": 3, "title": "Never two on one node",
|
|
"params": lambda r: {"ns": ns_pick(r, "edge"), "dep": name_pick(r, "gateway")},
|
|
"task": """
|
|
Deployment `%{dep}` in `%{ns}` runs 3 replicas, and nothing stops two of them from landing on the
|
|
same node. Change it so that:
|
|
|
|
- no two of its Pods ever run on the same node - a hard rule, not a preference;
|
|
- among the nodes allowed, those labelled `tier=edge` are **preferred**.
|
|
|
|
Label `{W2}` and `{W3}` with `tier=edge`. All three replicas must be running afterwards.
|
|
""",
|
|
"setup": NS + r"""
|
|
mkns %{ns}
|
|
for n in "$W1" "$W2" "$W3"; do $K label node "$n" tier- >/dev/null 2>&1; done
|
|
$K -n %{ns} create deploy %{dep} --image=nginx:1.27 --replicas=3 --dry-run=client -o yaml | $K apply -f - >/dev/null
|
|
$K -n %{ns} rollout status deploy %{dep} --timeout=180s >/dev/null
|
|
""",
|
|
"checks": [
|
|
C("required pod anti-affinity on kubernetes.io/hostname for its own Pods", 3, r"""
|
|
$K -n %{ns} get deploy %{dep} -o json 2>/dev/null | python3 -c '
|
|
import json, sys; d = json.load(sys.stdin); t = d["spec"]["template"]
|
|
a = t["spec"].get("affinity", {}).get("podAntiAffinity", {}).get("requiredDuringSchedulingIgnoredDuringExecution", [])
|
|
lab = t["metadata"]["labels"]
|
|
ok = any(x.get("topologyKey") == "kubernetes.io/hostname" and all(lab.get(k) == v for k, v in x.get("labelSelector", {}).get("matchLabels", {}).items()) and x.get("labelSelector", {}).get("matchLabels") for x in a)
|
|
sys.exit(0 if ok else 1)'"""),
|
|
C("preferred node affinity for tier=edge; {W2} and {W3} labelled", 2, r"""
|
|
eq "$(j node "$W2" '{.metadata.labels.tier}') $(j node "$W3" '{.metadata.labels.tier}')" "edge edge" || exit 1
|
|
$K -n %{ns} get deploy %{dep} -o json 2>/dev/null | python3 -c '
|
|
import json, sys; s = json.load(sys.stdin)["spec"]["template"]["spec"]
|
|
p = s.get("affinity", {}).get("nodeAffinity", {}).get("preferredDuringSchedulingIgnoredDuringExecution", [])
|
|
ok = any(e.get("key") == "tier" and e.get("operator") == "In" and "edge" in e.get("values", []) for x in p for e in x["preference"].get("matchExpressions", []))
|
|
sys.exit(0 if ok else 1)'"""),
|
|
C("3 Pods of the new template running on 3 different nodes", 2, r"""
|
|
[ -n "$(j -n %{ns} deploy %{dep} '{.spec.template.spec.affinity.podAntiAffinity.requiredDuringSchedulingIgnoredDuringExecution}')" ] || exit 1
|
|
eq "$(j -n %{ns} deploy %{dep} '{.status.availableReplicas} {.status.updatedReplicas} {.status.replicas}')" "3 3 3" || exit 1
|
|
[ "$($K -n %{ns} get pods -l app=%{dep} --field-selector=status.phase=Running -o jsonpath='{range .items[*]}{.spec.nodeName}{"\n"}{end}' | sort -u | grep -c .)" = 3 ]"""),
|
|
],
|
|
"solution": """
|
|
k label node {W2} {W3} tier=edge
|
|
k -n %{ns} patch deploy %{dep} --type merge -p '{"spec":{"strategy":{"rollingUpdate":{"maxSurge":0,"maxUnavailable":1}},"template":{"spec":{"affinity":{
|
|
"podAntiAffinity":{"requiredDuringSchedulingIgnoredDuringExecution":[{"labelSelector":{"matchLabels":{"app":"%{dep}"}},"topologyKey":"kubernetes.io/hostname"}]},
|
|
"nodeAffinity":{"preferredDuringSchedulingIgnoredDuringExecution":[{"weight":100,"preference":{"matchExpressions":[{"key":"tier","operator":"In","values":["edge"]}]}}]}}}}}}'
|
|
k -n %{ns} rollout status deploy %{dep}
|
|
k -n %{ns} get pods -o wide
|
|
# The anti-affinity's labelSelector picks the Pods to stay away from - its own, by its template's
|
|
# labels - and topologyKey says per what: one per hostname = one per node. With 3 workers a 4th
|
|
# replica would stay Pending - and so does the rolling update's surge Pod, since every node already
|
|
# has one of the old Pods (same labels). The default maxUnavailable of 25% is 0 Pods for 3
|
|
# replicas, so the rollout would wait for ever: let it replace one Pod at a time instead
|
|
# (maxSurge 0, maxUnavailable 1), in the same change.
|
|
""",
|
|
},
|
|
# ------------------------------------------------------------------ 144 ----
|
|
{
|
|
"id": 144, "domain": "WORK", "title": "A canary for a StatefulSet",
|
|
"params": lambda r: {"ns": ns_pick(r, "store"), "sts": name_pick(r, "cache")},
|
|
"task": """
|
|
StatefulSet `%{sts}` in `%{ns}` runs `nginx:1.26` on 3 replicas. Roll out `nginx:1.27` as a canary:
|
|
only the replica with the **highest ordinal** gets the new image now; the other two keep running
|
|
`nginx:1.26`, untouched, until someone lowers the partition.
|
|
""",
|
|
"setup": NS + r"""
|
|
mkns %{ns}
|
|
$K -n %{ns} apply -f - >/dev/null <<'EOF'
|
|
apiVersion: v1
|
|
kind: Service
|
|
metadata: {name: %{sts}}
|
|
spec: {clusterIP: None, selector: {app: %{sts}}, ports: [{port: 80}]}
|
|
---
|
|
apiVersion: apps/v1
|
|
kind: StatefulSet
|
|
metadata: {name: %{sts}}
|
|
spec:
|
|
serviceName: %{sts}
|
|
replicas: 3
|
|
selector: {matchLabels: {app: %{sts}}}
|
|
template:
|
|
metadata: {labels: {app: %{sts}}}
|
|
spec:
|
|
containers: [{name: nginx, image: nginx:1.26}]
|
|
EOF
|
|
$K -n %{ns} rollout status sts %{sts} --timeout=240s >/dev/null || exit 1
|
|
for i in 0 1; do j -n %{ns} pod %{sts}-$i '{.metadata.uid}'; echo; done > "$PRIV/q144.uids"
|
|
""",
|
|
"checks": [
|
|
C("updateStrategy RollingUpdate with partition 2", 2, r"""eq "$(j -n %{ns} sts %{sts} '{.spec.updateStrategy.type} {.spec.updateStrategy.rollingUpdate.partition}')" "RollingUpdate 2" """),
|
|
C("%{sts}-2 runs nginx:1.27 and is ready", 3, r"""
|
|
eq "$(j -n %{ns} sts %{sts} '{.spec.template.spec.containers[0].image}')" nginx:1.27 || exit 1
|
|
eq "$(j -n %{ns} pod %{sts}-2 '{.spec.containers[0].image} {.status.containerStatuses[0].ready}')" "nginx:1.27 true" """),
|
|
C("%{sts}-0 and %{sts}-1 still run nginx:1.26, never replaced", 2, r"""
|
|
eq "$(j -n %{ns} pod %{sts}-0 '{.spec.containers[0].image}') $(j -n %{ns} pod %{sts}-1 '{.spec.containers[0].image}')" "nginx:1.26 nginx:1.26" || exit 1
|
|
eq "$(for i in 0 1; do j -n %{ns} pod %{sts}-$i '{.metadata.uid}'; echo; done)" "$(cat "$PRIV/q144.uids")" \
|
|
&& eq "$(j -n %{ns} sts %{sts} '{.spec.template.spec.containers[0].image}')" nginx:1.27"""),
|
|
],
|
|
"solution": """
|
|
k -n %{ns} patch sts %{sts} -p '{"spec":{"updateStrategy":{"type":"RollingUpdate","rollingUpdate":{"partition":2}}}}'
|
|
k -n %{ns} set image sts/%{sts} nginx=nginx:1.27
|
|
k -n %{ns} rollout status sts %{sts}
|
|
k -n %{ns} get pods -l app=%{sts} -o custom-columns=NAME:.metadata.name,IMAGE:.spec.containers[0].image
|
|
# The partition FIRST: a StatefulSet updates from the highest ordinal down, and with no partition
|
|
# the new image would reach -1 and -0 within moments. Pods with an ordinal >= partition get the new
|
|
# template; the others keep theirs. Next step of the canary: partition 1, then 0.
|
|
""",
|
|
},
|
|
# ------------------------------------------------------------------ 145 ----
|
|
{
|
|
"id": 145, "domain": "ARCH", "workers": 1, "title": "An operator that cannot finish its job",
|
|
"params": lambda r: {"ns": ns_pick(r, "reports"), "name": name_pick(r, "daily")},
|
|
"task": """
|
|
The manifests of a backup operator are in `{WS}/q145/`: its CustomResourceDefinition, its RBAC and
|
|
its controller.
|
|
|
|
1. Install the operator.
|
|
2. Create Namespace `%{ns}` and, in it, Backup `%{name}` with the schedule `30 1 * * *`.
|
|
3. The operator should create ConfigMap `backup-%{name}` for it and set its `status.phase` to
|
|
`Scheduled`. Make sure it does - its manifests are not quite right. Change only what is wrong.
|
|
""",
|
|
"setup": r"""
|
|
rm -rf "$WS/q145"; mkdir -p "$WS/q145"
|
|
cat > "$WS/q145/operator.yaml" <<'EOF'
|
|
""" + backup_operator(crd=True, status_rbac=False) + r"""EOF
|
|
$K delete crd backups.data.example.com --ignore-not-found >/dev/null 2>&1
|
|
$K delete ns backup-operator --ignore-not-found >/dev/null 2>&1
|
|
""",
|
|
"checks": [
|
|
C("the CRD is established and the controller runs", 2, r"""
|
|
eq "$(j crd backups.data.example.com '{.status.conditions[?(@.type=="Established")].status}')" True && eq "$(j -n backup-operator deploy controller '{.status.readyReplicas}')" 1"""),
|
|
C("Backup %{name} in %{ns} has the schedule", 1, r"""eq "$(j -n %{ns} backup %{name} '{.spec.schedule}')" "30 1 * * *" """),
|
|
C("the operator made ConfigMap backup-%{name}", 1, r"""eq "$(j -n %{ns} cm backup-%{name} '{.data.schedule}')" "30 1 * * *" """),
|
|
C("Backup %{name} is Scheduled: the controller may update its status", 3, r"""
|
|
eq "$(j -n %{ns} backup %{name} '{.status.phase}')" Scheduled && eq "$(j clusterrolebinding backup-operator '{.subjects[0].name}')" controller"""),
|
|
],
|
|
"solution": """
|
|
grep -E '^kind:|^ name:' {WS}/q145/operator.yaml
|
|
k apply -f {WS}/q145/operator.yaml
|
|
k wait --for condition=established crd/backups.data.example.com
|
|
k -n backup-operator rollout status deploy controller
|
|
k create ns %{ns}
|
|
k -n %{ns} apply -f - <<'EOF'
|
|
apiVersion: data.example.com/v1
|
|
kind: Backup
|
|
metadata:
|
|
name: %{name}
|
|
spec:
|
|
schedule: "30 1 * * *"
|
|
EOF
|
|
sleep 10
|
|
k -n %{ns} get cm backup-%{name} # made - but:
|
|
k -n %{ns} get backup %{name} -o jsonpath='{.status}'; echo # no phase
|
|
k -n backup-operator logs deploy/controller | tail -3 # ... cannot patch resource "backups/status" ...
|
|
k auth can-i patch backups.data.example.com --subresource=status --as=system:serviceaccount:backup-operator:controller -A
|
|
k patch clusterrole backup-operator --type json -p '[{"op":"add","path":"/rules/0/resources/-","value":"backups/status"}]'
|
|
sleep 10
|
|
k -n %{ns} get backup %{name} -o jsonpath='{.status.phase}'; echo
|
|
# The CRD has a status subresource, and a subresource is a resource of its own for RBAC:
|
|
# `backups/status`. The controller could read Backups and write ConfigMaps, but not set a phase.
|
|
# The log of an operator's controller is where such a gap shows - here as `forbidden`.
|
|
""",
|
|
},
|
|
# ------------------------------------------------------------------ 146 ----
|
|
{
|
|
"id": 146, "domain": "STOR", "workers": 1, "title": "Grow a claim that is in use",
|
|
"params": lambda r: {"ns": ns_pick(r, "media"), "size": r.choice(["70Mi", "80Mi", "100Mi"])},
|
|
"task": """
|
|
In Namespace `%{ns}`:
|
|
|
|
1. Create PersistentVolumeClaim `pv-volume`: class `csi-hostpath-sc`, access mode `ReadWriteOnce`,
|
|
capacity `10Mi`.
|
|
2. Create Pod `web-server` (image `nginx:1.27`) that mounts the claim at `/usr/share/nginx/html`.
|
|
3. The site needs more room: expand the claim to `%{size}` while `web-server` keeps running.
|
|
""",
|
|
"setup": NS + r"""
|
|
ensure_csi_hostpath || { echo "could not install the CSI hostpath driver (no network?) - the task needs it" >&2; exit 1; }
|
|
mkns %{ns}
|
|
$K -n %{ns} delete pod web-server --ignore-not-found --wait=true --grace-period=1 >/dev/null 2>&1
|
|
$K -n %{ns} delete pvc pv-volume --ignore-not-found --wait=true >/dev/null 2>&1
|
|
""",
|
|
"checks": [
|
|
C("pv-volume uses csi-hostpath-sc, ReadWriteOnce, and is Bound", 1, r"""
|
|
eq "$(j -n %{ns} pvc pv-volume '{.spec.storageClassName} {.spec.accessModes[0]} {.status.phase}')" "csi-hostpath-sc ReadWriteOnce Bound" """),
|
|
C("web-server runs nginx with the claim at /usr/share/nginx/html", 2, r"""
|
|
eq "$(j -n %{ns} pod web-server '{.status.phase} {.spec.containers[0].image}')" "Running nginx:1.27" || exit 1
|
|
v=$(j -n %{ns} pod web-server '{.spec.containers[0].volumeMounts[?(@.mountPath=="/usr/share/nginx/html")].name}'); [ -n "$v" ] || exit 1
|
|
eq "$(j -n %{ns} pod web-server "{.spec.volumes[?(@.name==\"$v\")].persistentVolumeClaim.claimName}")" pv-volume"""),
|
|
C("the claim has grown to %{size}, and the Pod kept running", 3, r"""
|
|
eq "$(j -n %{ns} pvc pv-volume '{.spec.resources.requests.storage} {.status.capacity.storage}')" "%{size} %{size}" || exit 1
|
|
eq "$(j -n %{ns} pod web-server '{.status.phase} {.status.containerStatuses[0].restartCount}')" "Running 0" """),
|
|
],
|
|
"solution": """
|
|
k get sc csi-hostpath-sc # ALLOWVOLUMEEXPANSION: true - a claim of this class may grow
|
|
k -n %{ns} apply -f - <<'EOF'
|
|
apiVersion: v1
|
|
kind: PersistentVolumeClaim
|
|
metadata:
|
|
name: pv-volume
|
|
spec:
|
|
storageClassName: csi-hostpath-sc
|
|
accessModes: [ReadWriteOnce]
|
|
resources:
|
|
requests: {storage: 10Mi}
|
|
---
|
|
apiVersion: v1
|
|
kind: Pod
|
|
metadata:
|
|
name: web-server
|
|
spec:
|
|
containers:
|
|
- name: web-server
|
|
image: nginx:1.27
|
|
volumeMounts:
|
|
- {name: html, mountPath: /usr/share/nginx/html}
|
|
volumes:
|
|
- name: html
|
|
persistentVolumeClaim: {claimName: pv-volume}
|
|
EOF
|
|
k -n %{ns} wait --for=condition=Ready pod/web-server
|
|
k -n %{ns} patch pvc pv-volume -p '{"spec":{"resources":{"requests":{"storage":"%{size}"}}}}'
|
|
k -n %{ns} get pvc pv-volume # CAPACITY follows once the driver has resized it
|
|
# `k edit pvc pv-volume` works just as well. A claim only grows, and only when its class has
|
|
# allowVolumeExpansion: true and its driver can expand; spec.resources.requests is what you ask
|
|
# for, status.capacity what you have. Filesystem volumes of this driver grow online - the Pod
|
|
# keeps running.
|
|
""",
|
|
},
|
|
# ------------------------------------------------------------------ 147 ----
|
|
{
|
|
"id": 147, "domain": "STOR", "workers": 1, "title": "Snapshot, restore, clone",
|
|
"params": lambda r: {"ns": ns_pick(r, "ledger"), "n": r.randint(100, 999)},
|
|
"task": """
|
|
Claim `ledger-data` in `%{ns}` (class `csi-hostpath-sc`, `50Mi`) holds `ledger.txt`, written by Pod
|
|
`writer`.
|
|
|
|
1. Take a VolumeSnapshot `ledger-snap` of the claim, with snapshot class `csi-hostpath-snapclass`,
|
|
and wait until it is ready to use.
|
|
2. Create claim `ledger-restore` (`50Mi`, same class) **from that snapshot**.
|
|
3. Create claim `ledger-copy` (`50Mi`, same class) as a **clone of `ledger-data`** itself.
|
|
4. Run Pod `auditor` (image `busybox:1.36`, command `sleep 86400`) that mounts `ledger-restore` at
|
|
`/restore` and `ledger-copy` at `/copy`.
|
|
""",
|
|
"setup": NS + r"""
|
|
ensure_csi_hostpath || { echo "could not install the CSI hostpath driver (no network?) - the task needs it" >&2; exit 1; }
|
|
mkns %{ns}
|
|
$K -n %{ns} apply -f - >/dev/null <<'EOF'
|
|
apiVersion: v1
|
|
kind: PersistentVolumeClaim
|
|
metadata: {name: ledger-data}
|
|
spec: {storageClassName: csi-hostpath-sc, accessModes: [ReadWriteOnce], resources: {requests: {storage: 50Mi}}}
|
|
---
|
|
apiVersion: v1
|
|
kind: Pod
|
|
metadata: {name: writer}
|
|
spec:
|
|
containers:
|
|
- {name: writer, image: busybox:1.36, command: [sh, -c, 'echo entries-%{n} > /data/ledger.txt; sync; sleep 86400'], volumeMounts: [{name: d, mountPath: /data}]}
|
|
volumes: [{name: d, persistentVolumeClaim: {claimName: ledger-data}}]
|
|
EOF
|
|
$K -n %{ns} wait --for=condition=Ready pod/writer --timeout=180s >/dev/null
|
|
""",
|
|
"checks": [
|
|
C("VolumeSnapshot ledger-snap of ledger-data is ready to use", 2, r"""
|
|
eq "$(j -n %{ns} volumesnapshot ledger-snap '{.spec.source.persistentVolumeClaimName} {.spec.volumeSnapshotClassName} {.status.readyToUse}')" "ledger-data csi-hostpath-snapclass true" """),
|
|
C("ledger-restore is Bound, made from the snapshot", 2, r"""
|
|
eq "$(j -n %{ns} pvc ledger-restore '{.spec.dataSource.kind}/{.spec.dataSource.name} {.status.phase}')" "VolumeSnapshot/ledger-snap Bound" """),
|
|
C("ledger-copy is Bound, a clone of ledger-data", 2, r"""
|
|
eq "$(j -n %{ns} pvc ledger-copy '{.spec.dataSource.kind}/{.spec.dataSource.name} {.status.phase}')" "PersistentVolumeClaim/ledger-data Bound" """),
|
|
C("auditor reads the ledger from both", 2, r"""
|
|
x() { $K -n %{ns} exec auditor -- cat "$1/ledger.txt" 2>/dev/null; }
|
|
eq "$(x /restore)" entries-%{n} && eq "$(x /copy)" entries-%{n} \
|
|
&& eq "$(j -n %{ns} pod auditor '{.spec.volumes[*].persistentVolumeClaim.claimName}' | tr ' ' '\n' | sort | tr '\n' ' ')" "ledger-copy ledger-restore " """),
|
|
],
|
|
"solution": """
|
|
k get volumesnapshotclass # csi-hostpath-snapclass, driver hostpath.csi.k8s.io
|
|
k -n %{ns} apply -f - <<'EOF'
|
|
apiVersion: snapshot.storage.k8s.io/v1
|
|
kind: VolumeSnapshot
|
|
metadata:
|
|
name: ledger-snap
|
|
spec:
|
|
volumeSnapshotClassName: csi-hostpath-snapclass
|
|
source:
|
|
persistentVolumeClaimName: ledger-data
|
|
EOF
|
|
k -n %{ns} wait --for=jsonpath='{.status.readyToUse}'=true volumesnapshot/ledger-snap --timeout=120s
|
|
k -n %{ns} apply -f - <<'EOF'
|
|
apiVersion: v1
|
|
kind: PersistentVolumeClaim
|
|
metadata:
|
|
name: ledger-restore
|
|
spec:
|
|
storageClassName: csi-hostpath-sc
|
|
accessModes: [ReadWriteOnce]
|
|
resources:
|
|
requests: {storage: 50Mi}
|
|
dataSource:
|
|
apiGroup: snapshot.storage.k8s.io
|
|
kind: VolumeSnapshot
|
|
name: ledger-snap
|
|
---
|
|
apiVersion: v1
|
|
kind: PersistentVolumeClaim
|
|
metadata:
|
|
name: ledger-copy
|
|
spec:
|
|
storageClassName: csi-hostpath-sc
|
|
accessModes: [ReadWriteOnce]
|
|
resources:
|
|
requests: {storage: 50Mi}
|
|
dataSource:
|
|
kind: PersistentVolumeClaim
|
|
name: ledger-data
|
|
---
|
|
apiVersion: v1
|
|
kind: Pod
|
|
metadata:
|
|
name: auditor
|
|
spec:
|
|
containers:
|
|
- name: auditor
|
|
image: busybox:1.36
|
|
command: [sleep, '86400']
|
|
volumeMounts:
|
|
- {name: restore, mountPath: /restore}
|
|
- {name: copy, mountPath: /copy}
|
|
volumes:
|
|
- name: restore
|
|
persistentVolumeClaim: {claimName: ledger-restore}
|
|
- name: copy
|
|
persistentVolumeClaim: {claimName: ledger-copy}
|
|
EOF
|
|
k -n %{ns} wait --for=condition=Ready pod/auditor --timeout=180s
|
|
k -n %{ns} exec auditor -- cat /restore/ledger.txt /copy/ledger.txt
|
|
# A snapshot is taken by the CSI driver (VolumeSnapshotClass -> driver); `dataSource` makes a new
|
|
# claim from it - apiGroup snapshot.storage.k8s.io - or from another claim of the same class and
|
|
# Namespace, a clone (no apiGroup: claims are core). Both need a CSI driver that can do it, and
|
|
# the new claim must be at least as big as the source.
|
|
""",
|
|
},
|
|
]
|