c266eeb4b5
Study mode is gone: no more untimed sessions with the reference solutions open from the start. The path is now objective drills against the clock, then a mock, then the fixed sets, and an objective is done when every task passed on its latest timed attempt. In the lobby "Study mode" becomes "Drill an objective" (the objective list, with one "Against the clock" button per objective); ./ksim study is removed and ./ksim next always drills. study-<track>.json keeps focus, known and the done stamps; results of old study sessions stay in it, read by nothing. Also in this commit, from the work before it: - ./ksim skip / unskip, and links beside each domain in the lobby: mark a domain or an objective as known, so the path, the no-argument drill and drill all leave it out; a mock still draws from every domain - DOMAIN:N picks N tasks of a domain, spread over its objectives - a command with neither --track nor KSIM_TRACK is for the track whose session is under way; ./ksim show says where the task's host resolves - CKAD Q60: the check names which file holds which run's output Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JZPd3GGWyxTgAkL6a4gAJz
724 lines
49 KiB
Python
724 lines
49 KiB
Python
"""CKS curriculum as a path to the exam: the 26 objectives of the six domains, in the curriculum's own
|
|
words and order, each with what to know, the fastest commands, the traps, the documentation pages
|
|
the exam lets you open, and the tasks that practise it.
|
|
|
|
The pages under "docs" are all on the exam's allowed list (Linux Foundation, "Resources allowed",
|
|
read 2026-10-12): the Kubernetes documentation and blog, and the documentation of Falco, bom, etcd,
|
|
the NGINX Ingress Controller, Cilium and Istio. Trivy's, kubesec's, KubeLinter's and AppArmor's own
|
|
documentation are not on it: for those tools the manual on the day is `--help`.
|
|
|
|
./ksim --track cks objectives the path, with your progress per objective
|
|
./ksim --track cks notes 4.2 the notes for one objective (id or key: 4.2, secrets)
|
|
./ksim --track cks drill 4.2 its tasks against the clock
|
|
|
|
"tasks" lists every task that really exercises the objective, most direct first; a task may sit
|
|
under several objectives. Every task must be listed at least once and every objective needs two
|
|
pool tasks - `./ksim objectives` says so when that stops being true.
|
|
"""
|
|
|
|
OBJECTIVES = [
|
|
# ------------------------------------------------------------- 1. Cluster Setup (15%) ----
|
|
{
|
|
"id": "1.1", "key": "netpol", "domain": "SETUP",
|
|
"text": "Use Network security policies to restrict cluster level access",
|
|
"tasks": [1, 10, 11, 12],
|
|
"know": [
|
|
"A Pod is open in a direction until some NetworkPolicy selects it for that direction; from then on only what a policy allows gets through. Policies add up - there is no deny rule and no order.",
|
|
"Default deny is `podSelector: {}` with `policyTypes` and no rules. Deny egress and DNS stops too: allow UDP and TCP 53 to kube-system.",
|
|
"Inside one `from`/`to` entry, namespaceSelector and podSelector are ANDed; as two list entries they are ORed. That one dash is the difference between 'these Pods in that Namespace' and 'that whole Namespace, plus these Pods here'.",
|
|
"A connection needs both ends: egress allowed on the client and ingress allowed on the server. Replies are allowed automatically.",
|
|
"Select a Namespace by its built-in label `kubernetes.io/metadata.name`; `ipBlock` with `except` is for addresses outside the cluster.",
|
|
],
|
|
"fast": """
|
|
k -n NS get netpol; k -n NS describe netpol NAME # describe spells out who may talk to whom
|
|
k -n NS get pod --show-labels; k get ns --show-labels
|
|
k -n NS exec CLIENT -- wget -T 2 -qO- http://SVC.NS2 # prove it: allowed answers, blocked times out
|
|
k explain networkpolicy.spec.ingress.from
|
|
""",
|
|
"traps": [
|
|
"`policyTypes: [Egress]` with no `egress:` key denies all egress; `egress: [{}]` allows all of it.",
|
|
"A podSelector in `from` only matches Pods of the policy's own Namespace unless a namespaceSelector sits in the same entry.",
|
|
"Ports in a policy are the Pod's ports (targetPort), not the Service's.",
|
|
"Policies are namespaced: a 'cluster-wide' default deny means one per Namespace.",
|
|
],
|
|
"docs": ["https://kubernetes.io/docs/concepts/services-networking/network-policies/",
|
|
"https://kubernetes.io/docs/tasks/administer-cluster/declare-network-policy/"],
|
|
},
|
|
{
|
|
"id": "1.2", "key": "cis", "domain": "SETUP",
|
|
"text": "Use CIS benchmark to review the security configuration of Kubernetes components (etcd, kubelet, kubedns, kubeapi)",
|
|
"tasks": [2, 13, 14, 15],
|
|
"know": [
|
|
"kube-bench runs the CIS Kubernetes Benchmark on a node and prints PASS, FAIL or WARN per numbered check, with the remediation underneath.",
|
|
"Control-plane components are static Pods: fix a flag in /etc/kubernetes/manifests/<component>.yaml and the kubelet restarts the Pod by itself.",
|
|
"The kubelet is a systemd service configured in /var/lib/kubelet/config.yaml: edit, then `systemctl restart kubelet`. Do it on every node the task names.",
|
|
"Usual findings - apiserver: `--profiling=false`, `--anonymous-auth=false`, `--authorization-mode=Node,RBAC`, NodeRestriction, audit flags. kubelet: `authentication.anonymous.enabled: false`, `authorization.mode: Webhook`, `readOnlyPort: 0`. etcd: `--client-cert-auth=true`, `--peer-client-cert-auth=true`, no `--auto-tls`. Files: mode 600, owner root:root; etcd data directory 700, owner etcd:etcd.",
|
|
],
|
|
"fast": """
|
|
kube-bench run --targets master # also: node, etcd, controlplane, policies
|
|
kube-bench run --targets master --check 1.2.15 # one check, with its remediation
|
|
kube-bench run --targets node | grep -A3 FAIL
|
|
vi /etc/kubernetes/manifests/kube-apiserver.yaml # then: watch crictl ps until it is back
|
|
vi /var/lib/kubelet/config.yaml && systemctl restart kubelet
|
|
stat -c '%a %U:%G' /etc/kubernetes/manifests/*.yaml /var/lib/etcd
|
|
""",
|
|
"traps": [
|
|
"Never leave a backup copy inside /etc/kubernetes/manifests - the kubelet runs it as a second static Pod. Copy to /root first.",
|
|
"A typo in a manifest makes the component vanish instead of failing loudly: `crictl ps -a`, `crictl logs`, and `journalctl -u kubelet` say why.",
|
|
"Check numbers move between benchmark versions - go by the check's text, not by memory of its number.",
|
|
"A flag that points at a file needs that file mounted into the static Pod (volume + volumeMount).",
|
|
],
|
|
"docs": ["https://kubernetes.io/docs/reference/command-line-tools-reference/kube-apiserver/",
|
|
"https://kubernetes.io/docs/reference/config-api/kubelet-config.v1beta1/",
|
|
"https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/"],
|
|
},
|
|
{
|
|
"id": "1.3", "key": "ingress-tls", "domain": "SETUP",
|
|
"text": "Properly set up Ingress with TLS",
|
|
"tasks": [16, 17, 18],
|
|
"know": [
|
|
"TLS for an Ingress is a Secret of type kubernetes.io/tls (keys tls.crt and tls.key) in the Ingress's own Namespace, named under `spec.tls[].secretName` with the hosts it serves.",
|
|
"The host in `spec.tls[].hosts` must match a host in `spec.rules` and a name in the certificate (its subjectAltName; CN alone is no longer enough for clients).",
|
|
"The controller terminates TLS; traffic from the controller to the Pods is plain HTTP unless the backend is set up otherwise.",
|
|
"With ingress-nginx, a host that has TLS redirects HTTP to HTTPS by default (annotation `nginx.ingress.kubernetes.io/ssl-redirect`).",
|
|
],
|
|
"fast": """
|
|
openssl req -x509 -newkey rsa:2048 -nodes -days 365 -keyout tls.key -out tls.crt \\
|
|
-subj "/CN=shop.example.com" -addext "subjectAltName=DNS:shop.example.com"
|
|
k -n NS create secret tls shop-tls --cert=tls.crt --key=tls.key
|
|
k -n NS create ingress shop --class=nginx --rule="shop.example.com/*=shop:80,tls=shop-tls"
|
|
openssl x509 -in tls.crt -noout -subject -ext subjectAltName -enddate
|
|
curl -kv --resolve shop.example.com:443:NODE_IP https://shop.example.com/ 2>&1 | grep -E 'subject|HTTP'
|
|
""",
|
|
"traps": [
|
|
"A Secret in another Namespace is silently ignored and the controller serves its self-signed default certificate.",
|
|
"`kubectl create secret generic` gives type Opaque - an Ingress wants `create secret tls`.",
|
|
"Certificate and key that do not belong together are rejected: compare `openssl x509 -noout -pubkey` with `openssl pkey -pubout`.",
|
|
],
|
|
"docs": ["https://kubernetes.io/docs/concepts/services-networking/ingress/#tls",
|
|
"https://kubernetes.io/docs/concepts/configuration/secret/#tls-secrets",
|
|
"https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/"],
|
|
},
|
|
{
|
|
"id": "1.4", "key": "metadata", "domain": "SETUP",
|
|
"text": "Protect node metadata and endpoints",
|
|
"tasks": [19, 20, 21],
|
|
"know": [
|
|
"Cloud instance metadata (169.254.169.254) hands node credentials to anything that can reach it. Block it for Pods with an egress NetworkPolicy: `ipBlock` 0.0.0.0/0 `except` 169.254.169.254/32, and allow it only for Pods that need it.",
|
|
"The kubelet API on 10250 must authenticate and authorise (anonymous off, Webhook); its read-only port 10255 must be off (`readOnlyPort: 0`).",
|
|
"etcd on 2379 holds every Secret: client certificates required, reachable only by the API server.",
|
|
"Policies add up: a second policy that selects a few Pods and allows the metadata address is how you make the exception.",
|
|
],
|
|
"fast": """
|
|
k -n NS exec POD -- wget -T 2 -qO- http://169.254.169.254/latest/meta-data/ # what a Pod can see
|
|
curl -sk https://NODE:10250/pods | head -c 200 # Unauthorized is the answer you want
|
|
curl -s http://NODE:10255/pods | head -c 200 # connection refused is the answer you want
|
|
grep -n -A3 'anonymous\\|readOnlyPort\\|authorization' /var/lib/kubelet/config.yaml
|
|
""",
|
|
"traps": [
|
|
"An egress policy that only names the metadata exception also cuts DNS unless the allowed block covers the DNS Service - 0.0.0.0/0 with the one exception keeps everything else open.",
|
|
"`except` must lie inside the `cidr` it belongs to.",
|
|
"hostNetwork Pods are not covered by NetworkPolicy at all.",
|
|
],
|
|
"docs": ["https://kubernetes.io/docs/concepts/services-networking/network-policies/#behavior-of-to-and-from-selectors",
|
|
"https://kubernetes.io/docs/reference/access-authn-authz/kubelet-authn-authz/",
|
|
"https://kubernetes.io/docs/concepts/security/security-checklist/"],
|
|
},
|
|
{
|
|
"id": "1.5", "key": "binaries", "domain": "SETUP",
|
|
"text": "Verify platform binaries before deploying",
|
|
"tasks": [22, 23, 24],
|
|
"know": [
|
|
"Every Kubernetes release publishes a checksum per binary (dl.k8s.io/release/vX.Y.Z/bin/linux/amd64/<binary>.sha256 and .sha512). A binary is genuine only if its hash equals the published one for that exact version and architecture.",
|
|
"sha512sum/sha256sum print '<hash> <file>'; `--check` reads that format back and says OK or FAILED.",
|
|
"The binary a running process really uses is /proc/<pid>/exe - for a container, crictl gives the pid.",
|
|
],
|
|
"fast": """
|
|
sha512sum kubelet | cut -d' ' -f1
|
|
echo "$(cat kubelet.sha512) kubelet" | sha512sum --check # two spaces between hash and name
|
|
sha512sum * | sort # identical files line up
|
|
kubelet --version; kubectl version --client; kubeadm version -o short
|
|
sha512sum /proc/$(pgrep -o kube-apiserver)/exe
|
|
""",
|
|
"traps": [
|
|
"Comparing two long hashes by eye fails - use `--check`, `diff` or `[ \"$a\" = \"$b\" ]`.",
|
|
"A correct hash of the wrong version is still the wrong binary: check the version the task names.",
|
|
"sha256 and sha512 files are not interchangeable; the length tells you which one you hold (64 or 128 hex digits).",
|
|
],
|
|
"docs": ["https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/",
|
|
"https://kubernetes.io/releases/download/"],
|
|
},
|
|
# ---------------------------------------------------------- 2. Cluster Hardening (15%) ----
|
|
{
|
|
"id": "2.1", "key": "rbac", "domain": "HARDEN",
|
|
"text": "Use Role Based Access Controls to minimize exposure",
|
|
"tasks": [3, 25, 26, 27, 28],
|
|
"know": [
|
|
"Role and RoleBinding live in a Namespace; ClusterRole and ClusterRoleBinding are cluster-wide. A RoleBinding may point at a ClusterRole - the rules then apply in that Namespace only.",
|
|
"RBAC only adds. To take a permission away you find and remove the rule or the binding that grants it.",
|
|
"Dangerous grants: `*`, secrets (get, list or watch - list shows the contents too), pods/exec, `escalate` and `bind` on roles, `impersonate`, create on pods (mount any Secret, use any ServiceAccount), nodes/proxy.",
|
|
"`resourceNames` narrows a rule to named objects, but cannot restrict `list`, `watch` or `create`.",
|
|
"Subjects: User and Group come from the client certificate (CN and O); a ServiceAccount is `system:serviceaccount:<ns>:<name>`. The group system:masters bypasses RBAC entirely.",
|
|
],
|
|
"fast": """
|
|
k auth can-i --list --as=system:serviceaccount:NS:SA -n NS
|
|
k auth can-i get secrets -n NS --as=jane --as-group=dev
|
|
k get clusterrolebinding,rolebinding -A -o wide | grep -w SUBJECT
|
|
k -n NS create role R --verb=get,list --resource=pods,pods/log [--resource-name=x]
|
|
k -n NS create rolebinding RB --role=R --serviceaccount=NS:SA # --clusterrole= --user= --group=
|
|
k get clusterrole R -o yaml
|
|
""",
|
|
"traps": [
|
|
"roleRef cannot be changed: delete the binding and create it again.",
|
|
"`-n` matters with `auth can-i`: without it you test the default Namespace.",
|
|
"Subresources are separate resources in a Role: pods/log, pods/exec, deployments/scale. To `auth can-i`, though, `pods/exec` is a Pod named exec - ask with `can-i create pods --subresource=exec`.",
|
|
"A binding to system:authenticated or system:unauthenticated grants to everybody.",
|
|
],
|
|
"docs": ["https://kubernetes.io/docs/reference/access-authn-authz/rbac/",
|
|
"https://kubernetes.io/docs/concepts/security/rbac-good-practices/"],
|
|
},
|
|
{
|
|
"id": "2.2", "key": "serviceaccounts", "domain": "HARDEN",
|
|
"text": "Exercise caution in using service accounts e.g. disable defaults, minimize permissions on newly created ones",
|
|
"tasks": [4, 29, 30, 31, 7],
|
|
"know": [
|
|
"Every Pod gets the Namespace's `default` ServiceAccount and its token unless told otherwise. Turn the mount off with `automountServiceAccountToken: false` on the ServiceAccount or on the Pod; the Pod's setting wins.",
|
|
"One ServiceAccount per workload, bound to a Role with exactly what it needs. The default ServiceAccount should have no bindings.",
|
|
"Tokens are short-lived and bound to the Pod (projected volume). `kubectl create token` issues one by hand with `--duration`; a projected `serviceAccountToken` source sets `audience` and `expirationSeconds`.",
|
|
"A Secret of type kubernetes.io/service-account-token is a token that never expires - find and delete them.",
|
|
],
|
|
"fast": """
|
|
k -n NS patch sa default -p '{"automountServiceAccountToken": false}'
|
|
k -n NS create sa app-sa; k -n NS set serviceaccount deploy app app-sa
|
|
k -n NS create token app-sa --duration=10m
|
|
k -n NS get pod -o custom-columns=POD:.metadata.name,SA:.spec.serviceAccountName,AUTO:.spec.automountServiceAccountToken
|
|
k get secret -A --field-selector type=kubernetes.io/service-account-token
|
|
k -n NS exec POD -- ls /var/run/secrets/kubernetes.io/serviceaccount
|
|
""",
|
|
"traps": [
|
|
"Changing a ServiceAccount does nothing to Pods that already run - they must be re-created.",
|
|
"A Pod's serviceAccountName cannot be edited in place; change the workload's template.",
|
|
"`automountServiceAccountToken: false` on the ServiceAccount is overridden by `true` on a Pod.",
|
|
],
|
|
"docs": ["https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/",
|
|
"https://kubernetes.io/docs/concepts/security/service-accounts/"],
|
|
},
|
|
{
|
|
"id": "2.3", "key": "apiaccess", "domain": "HARDEN",
|
|
"text": "Restrict access to Kubernetes API",
|
|
"tasks": [32, 33, 34, 35, 36],
|
|
"know": [
|
|
"A request passes authentication, then authorisation, then admission. Harden each: no anonymous requests, `--authorization-mode=Node,RBAC` (never AlwaysAllow), and the NodeRestriction admission plugin so a kubelet can only touch its own Node and Pods.",
|
|
"Anonymous requests arrive as system:anonymous in group system:unauthenticated. Turn them off with `--anonymous-auth=false`, or keep them for the health endpoints only with an AuthenticationConfiguration file.",
|
|
"People get certificates through the CertificateSigningRequest API: key and CSR with openssl, a CSR object with signerName kubernetes.io/kube-apiserver-client, approve, read the certificate from .status.certificate.",
|
|
"Do not expose the API further than needed: the `kubernetes` Service stays ClusterIP, no `--token-auth-file`, kubeconfigs and /etc/kubernetes/pki readable by root only.",
|
|
],
|
|
"fast": """
|
|
grep -n 'anonymous\\|authorization-mode\\|admission-plugins\\|token-auth' /etc/kubernetes/manifests/kube-apiserver.yaml
|
|
curl -sk https://localhost:6443/api/v1/namespaces | head -5 # anonymous: 403 Forbidden, or 401 when off
|
|
k get clusterrolebinding -o wide | grep -E 'anonymous|unauthenticated'
|
|
openssl genrsa -out jane.key 2048; openssl req -new -key jane.key -subj "/CN=jane/O=dev" -out jane.csr
|
|
k certificate approve jane; sleep 2 # the certificate is issued a moment after the approval
|
|
k get csr jane -o jsonpath='{.status.certificate}' | base64 -d > jane.crt
|
|
k config set-credentials jane --client-certificate=jane.crt --client-key=jane.key --embed-certs
|
|
""",
|
|
"traps": [
|
|
"kubeadm's own probes call /livez and /readyz anonymously: after turning anonymous access off, check that the API server stays up.",
|
|
"NodeRestriction goes in `--enable-admission-plugins`; listing it under authorisation modes does nothing.",
|
|
"The request in a CSR object is the base64 of the whole PEM file on one line: `base64 -w0`.",
|
|
"Certificates cannot be revoked - remove the bindings of the user or group instead.",
|
|
],
|
|
"docs": ["https://kubernetes.io/docs/concepts/security/controlling-access/",
|
|
"https://kubernetes.io/docs/reference/access-authn-authz/authentication/",
|
|
"https://kubernetes.io/docs/reference/access-authn-authz/certificate-signing-requests/",
|
|
"https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#noderestriction"],
|
|
},
|
|
{
|
|
"id": "2.4", "key": "upgrade", "domain": "HARDEN",
|
|
"text": "Upgrade Kubernetes to avoid vulnerabilities",
|
|
"tasks": [37, 38, 39],
|
|
"know": [
|
|
"Order: control plane first, then workers; one minor version at a time. kubeadm first, then `kubeadm upgrade`, then kubelet and kubectl.",
|
|
"Control plane: `kubeadm upgrade plan`, `kubeadm upgrade apply vX.Y.Z`. Other nodes: `kubeadm upgrade node`.",
|
|
"Drain a node before its kubelet is upgraded and uncordon it afterwards; restart the kubelet after installing the package.",
|
|
"Version skew: kubelets may be up to three minors older than the API server, never newer.",
|
|
],
|
|
"fast": """
|
|
kubeadm upgrade plan
|
|
apt-mark unhold kubeadm && apt-get update && apt-get install -y kubeadm='1.35.x-*' && apt-mark hold kubeadm
|
|
kubeadm upgrade apply v1.35.x # workers: kubeadm upgrade node
|
|
k drain NODE --ignore-daemonsets --delete-emptydir-data
|
|
apt-mark unhold kubelet kubectl && apt-get install -y kubelet='1.35.x-*' kubectl='1.35.x-*' && apt-mark hold kubelet kubectl
|
|
systemctl daemon-reload && systemctl restart kubelet; k uncordon NODE
|
|
k get nodes; kubelet --version
|
|
""",
|
|
"traps": [
|
|
"The package repository is per minor version (pkgs.k8s.io/core:/stable:/v1.35): a new minor needs the repository line changed first.",
|
|
"`kubectl get nodes` shows the kubelet's version, so a node looks old until its kubelet is restarted.",
|
|
"Forgetting to uncordon leaves the node unschedulable.",
|
|
],
|
|
"docs": ["https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/",
|
|
"https://kubernetes.io/releases/version-skew-policy/"],
|
|
},
|
|
# ----------------------------------------------------------- 3. System Hardening (10%) ----
|
|
{
|
|
"id": "3.1", "key": "footprint", "domain": "SYS",
|
|
"text": "Minimize host OS footprint (reduce attack surface)",
|
|
"tasks": [40, 41, 42, 43, 44],
|
|
"know": [
|
|
"Less on the node means less to attack: remove packages nobody needs, stop and disable services, close listening ports, delete stray users and binaries.",
|
|
"Find it first: `ss -tlnp` (who listens, and which process), `systemctl list-units --type=service --state=running`, `ps aux`, `dpkg -l` or `apt list --installed`.",
|
|
"From a port to its owner: `ss -tlnp` gives the pid, `/proc/<pid>/exe` the binary, `systemctl status <pid>` the unit, `dpkg -S <file>` the package.",
|
|
"Kernel modules: `lsmod`; keep one from loading with a `blacklist` line in /etc/modprobe.d/.",
|
|
],
|
|
"fast": """
|
|
ss -tlnp # -u for UDP
|
|
systemctl status PID; systemctl disable --now UNIT
|
|
ls -l /proc/PID/exe; dpkg -S /usr/sbin/BINARY; apt-get purge -y PACKAGE
|
|
systemctl list-unit-files --state=enabled --type=service
|
|
find / -xdev -perm -4000 -type f 2>/dev/null # setuid binaries
|
|
echo 'blacklist sctp' > /etc/modprobe.d/sctp.conf
|
|
""",
|
|
"traps": [
|
|
"`systemctl stop` alone does not survive a reboot: `disable --now` does both.",
|
|
"Killing a process a unit owns just gets it restarted - stop the unit.",
|
|
"`apt remove` keeps configuration files; `purge` removes them too.",
|
|
],
|
|
"docs": ["https://kubernetes.io/docs/concepts/security/security-checklist/",
|
|
"https://kubernetes.io/docs/concepts/security/linux-kernel-security-constraints/"],
|
|
},
|
|
{
|
|
"id": "3.2", "key": "iam", "domain": "SYS",
|
|
"text": "Using least-privilege identity and access management",
|
|
"tasks": [45, 46, 47],
|
|
"know": [
|
|
"Give every identity - person, service, node - only what it needs, for as long as it needs it. On a node that means few accounts, few sudo rules, and no shared keys.",
|
|
"Root-equivalent in practice: membership of sudo, docker or any group that can reach a container runtime socket; write access to /etc/kubernetes or a kubeconfig with cluster-admin.",
|
|
"sshd: `PermitRootLogin no`, `PasswordAuthentication no`, `AllowUsers`; check with `sshd -t`, then reload.",
|
|
"In a cloud, a node's instance role should not be able to do more than a node must - Pods inherit it through the metadata service.",
|
|
],
|
|
"fast": """
|
|
getent group sudo docker; id USER; gpasswd -d USER GROUP
|
|
usermod -L USER; usermod -s /usr/sbin/nologin USER; userdel -r USER
|
|
visudo -c; ls -l /etc/sudoers.d/; sudo -l -U USER
|
|
sshd -T | grep -Ei 'permitrootlogin|passwordauthentication|allowusers'
|
|
sshd -t && systemctl reload ssh
|
|
find /etc/kubernetes /home -name '*.conf' -o -name config | xargs ls -l
|
|
""",
|
|
"traps": [
|
|
"A broken sudoers file locks everyone out of sudo: edit with visudo or check with `visudo -c -f FILE`.",
|
|
"Files in /etc/ssh/sshd_config.d/ are read first, and for most options the first value wins.",
|
|
"Group changes apply at the next login, not to sessions already open.",
|
|
],
|
|
"docs": ["https://kubernetes.io/docs/concepts/security/security-checklist/",
|
|
"https://kubernetes.io/docs/concepts/security/rbac-good-practices/"],
|
|
},
|
|
{
|
|
"id": "3.3", "key": "network", "domain": "SYS",
|
|
"text": "Minimize external access to the network",
|
|
"tasks": [48, 49, 50, 51],
|
|
"know": [
|
|
"Whatever listens on a node address is reachable from outside unless a firewall says otherwise: bind to 127.0.0.1 where possible, filter the rest.",
|
|
"Services: NodePort and LoadBalancer open a port on every node; ClusterIP does not. `externalIPs` on a Service is another way in.",
|
|
"Host firewall with ufw, iptables or nft - allow ssh, the API server and node-to-node traffic before you set a default deny.",
|
|
"Egress from Pods to the outside is limited with NetworkPolicy `ipBlock`.",
|
|
],
|
|
"fast": """
|
|
ss -tlnp | grep -v 127.0.0.1
|
|
k get svc -A | grep -E 'NodePort|LoadBalancer'; k -n NS patch svc S -p '{"spec":{"type":"ClusterIP"}}'
|
|
nft list ruleset | less; iptables -S INPUT
|
|
iptables -A INPUT -p tcp --dport 9090 ! -s 10.0.0.0/8 -j DROP
|
|
ufw status verbose; ufw allow from 10.0.0.0/8 to any port 10250 proto tcp; ufw deny 9090/tcp
|
|
""",
|
|
"traps": [
|
|
"A default-deny firewall applied before allowing ssh ends your session.",
|
|
"Changing a Service from NodePort to ClusterIP with `kubectl edit` needs the nodePort fields removed too; `patch` with the new type does it for you.",
|
|
"kube-proxy's rules handle NodePorts before the INPUT chain sees them - close a NodePort by changing the Service.",
|
|
],
|
|
"docs": ["https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types",
|
|
"https://kubernetes.io/docs/reference/networking/ports-and-protocols/"],
|
|
},
|
|
{
|
|
"id": "3.4", "key": "kernel", "domain": "SYS",
|
|
"text": "Appropriately use kernel hardening tools such as AppArmor, seccomp",
|
|
"tasks": [5, 52, 53, 54, 55, 108, 109, 110],
|
|
"know": [
|
|
"seccomp filters system calls. `RuntimeDefault` is the runtime's profile; `Localhost` names a JSON file relative to the kubelet's seccomp directory (/var/lib/kubelet/seccomp) on the node where the Pod runs.",
|
|
"AppArmor confines a program to files, capabilities and network access listed in a profile. The profile must be loaded on every node the Pod can land on (`apparmor_parser`), then named in `securityContext.appArmorProfile` (type Localhost, localhostProfile: the profile's name).",
|
|
"Both can be set for the whole Pod or per container; the container's setting wins.",
|
|
"A profile name is the name inside the file (`profile NAME ...`), not the file name.",
|
|
],
|
|
"fast": """
|
|
apparmor_parser -q /etc/apparmor.d/PROFILE; aa-status | grep PROFILE # on the node
|
|
ssh NODE 'cat /sys/kernel/security/apparmor/profiles'
|
|
ls /var/lib/kubelet/seccomp/profiles/
|
|
k explain pod.spec.securityContext.seccompProfile; k explain pod.spec.containers.securityContext.appArmorProfile
|
|
k exec POD -- grep Seccomp /proc/1/status # 0 off, 2 filtering
|
|
k exec POD -- cat /proc/1/attr/current # the AppArmor profile in force
|
|
""",
|
|
"traps": [
|
|
"`localhostProfile: profiles/audit.json` is relative to /var/lib/kubelet/seccomp - an absolute path is rejected.",
|
|
"A Pod naming a profile that is not on its node stays in CreateContainerError (seccomp) or Blocked (AppArmor).",
|
|
"The old annotation `container.apparmor.security.beta.kubernetes.io/<container>` still works but the field is what current documentation shows.",
|
|
],
|
|
"docs": ["https://kubernetes.io/docs/tutorials/security/seccomp/",
|
|
"https://kubernetes.io/docs/tutorials/security/apparmor/",
|
|
"https://kubernetes.io/docs/concepts/security/linux-kernel-security-constraints/"],
|
|
},
|
|
# ------------------------------------------ 4. Minimize Microservice Vulnerabilities (20%) ----
|
|
{
|
|
"id": "4.1", "key": "pss", "domain": "MICRO",
|
|
"text": "Use appropriate pod security standards",
|
|
"tasks": [6, 56, 57, 58, 59],
|
|
"know": [
|
|
"Three standards: privileged (anything), baseline (no known privilege escalations: no privileged, host namespaces, hostPath, hostPorts, added capabilities beyond a short list) and restricted (also non-root, no privilege escalation, drop ALL capabilities, seccomp RuntimeDefault or Localhost, only safe volume types).",
|
|
"Pod Security Admission applies them per Namespace through labels `pod-security.kubernetes.io/<mode>: <level>` with modes enforce (reject), audit (annotate the audit log) and warn (tell the client), each with an optional `-version`.",
|
|
"It checks Pods, at creation. A Deployment that violates the standard is accepted and its ReplicaSet then fails to create Pods; Pods already running are left alone.",
|
|
"Cluster-wide defaults and exemptions (users, RuntimeClasses, Namespaces) come from an AdmissionConfiguration file passed with `--admission-control-config-file`.",
|
|
],
|
|
"fast": """
|
|
k label ns NS pod-security.kubernetes.io/enforce=restricted pod-security.kubernetes.io/enforce-version=latest --overwrite
|
|
k label --dry-run=server --overwrite ns --all pod-security.kubernetes.io/enforce=baseline # who would break?
|
|
k -n NS get events --field-selector reason=FailedCreate
|
|
k -n NS describe rs | grep -A3 Warning
|
|
# restricted, per container: allowPrivilegeEscalation: false, capabilities: {drop: [ALL]},
|
|
# runAsNonRoot: true, seccompProfile: {type: RuntimeDefault}
|
|
""",
|
|
"traps": [
|
|
"After labelling, existing Pods keep running: delete them or restart the rollout to see what is really admitted.",
|
|
"`runAsNonRoot: true` with an image that runs as root fails at start, not at admission - set `runAsUser` as well.",
|
|
"restricted allows adding one capability only: NET_BIND_SERVICE.",
|
|
],
|
|
"docs": ["https://kubernetes.io/docs/concepts/security/pod-security-standards/",
|
|
"https://kubernetes.io/docs/concepts/security/pod-security-admission/",
|
|
"https://kubernetes.io/docs/tasks/configure-pod-container/enforce-standards-namespace-labels/",
|
|
"https://kubernetes.io/docs/tasks/configure-pod-container/enforce-standards-admission-controller/"],
|
|
},
|
|
{
|
|
"id": "4.2", "key": "secrets", "domain": "MICRO",
|
|
"text": "Manage Kubernetes secrets",
|
|
"tasks": [60, 61, 62, 63, 64],
|
|
"know": [
|
|
"A Secret is base64, not encrypted: anyone who can read it, read etcd, or exec into a Pod that uses it has the value.",
|
|
"Encryption at rest: an EncryptionConfiguration file named by `--encryption-provider-config`, mounted into the API server. The first provider in the list encrypts; every listed provider can decrypt. `identity: {}` is plain text.",
|
|
"Only new writes are encrypted. Rewrite what exists: `kubectl get secrets -A -o json | kubectl replace -f -`.",
|
|
"Proof is in etcd: the stored value starts with `k8s:enc:aescbc:v1:<key name>` when encrypted and shows the plain data when not.",
|
|
"Prefer files over environment variables (env leaks into logs and child processes), mount read-only, and keep RBAC on secrets tight - `list` reveals the values too.",
|
|
],
|
|
"fast": """
|
|
head -c 32 /dev/urandom | base64 # a key for aescbc or secretbox
|
|
ETCDCTL_API=3 etcdctl --endpoints=https://127.0.0.1:2379 --cacert=/etc/kubernetes/pki/etcd/ca.crt \\
|
|
--cert=/etc/kubernetes/pki/etcd/server.crt --key=/etc/kubernetes/pki/etcd/server.key \\
|
|
get /registry/secrets/NS/NAME | hexdump -C | head
|
|
k get secrets -A -o json | k replace -f -
|
|
k -n NS get secret S -o jsonpath='{.data.password}' | base64 -d
|
|
k -n NS create secret generic S --from-literal=password=x --from-file=key.pem
|
|
""",
|
|
"traps": [
|
|
"The configuration file must be mounted into the kube-apiserver Pod (hostPath volume) or the server does not come back.",
|
|
"`identity` listed first means nothing is encrypted, whatever follows it.",
|
|
"The key in the file is base64 of 32 random bytes for aescbc and secretbox - encoding it twice or using 16 bytes fails.",
|
|
"Removing a key that still encrypts stored Secrets makes them unreadable: decrypt (rewrite) first, remove after.",
|
|
],
|
|
"docs": ["https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/",
|
|
"https://kubernetes.io/docs/concepts/configuration/secret/",
|
|
"https://kubernetes.io/docs/concepts/security/secrets-good-practices/"],
|
|
},
|
|
{
|
|
"id": "4.3", "key": "isolation", "domain": "MICRO",
|
|
"text": "Understand and implement isolation techniques (multi-tenancy, sandboxed containers, etc.)",
|
|
"tasks": [7, 65, 66, 67],
|
|
"know": [
|
|
"Tenants share a cluster through a Namespace each, fenced by RBAC, NetworkPolicy, ResourceQuota and LimitRange, and Pod Security.",
|
|
"A container shares the node's kernel. A sandbox puts a boundary in between: gVisor (runsc) answers system calls in user space, Kata Containers runs each Pod in a small VM.",
|
|
"A RuntimeClass names a handler configured in the node's container runtime; a Pod asks for it with `runtimeClassName`. The handler must exist on the node the Pod lands on (`scheduling.nodeSelector` in the RuntimeClass steers it).",
|
|
"What breaks isolation: privileged, hostPID, hostNetwork, hostIPC, hostPath volumes, extra capabilities, a mounted runtime socket.",
|
|
],
|
|
"fast": """
|
|
cat <<EOF | k apply -f -
|
|
apiVersion: node.k8s.io/v1
|
|
kind: RuntimeClass
|
|
metadata: {name: gvisor}
|
|
handler: runsc
|
|
EOF
|
|
k -n NS patch deploy D -p '{"spec":{"template":{"spec":{"runtimeClassName":"gvisor"}}}}'
|
|
k -n NS exec POD -- dmesg | head -3 # 'Starting gVisor...' proves the sandbox
|
|
k -n NS create quota team --hard=pods=10,requests.cpu=2,requests.memory=2Gi
|
|
k get pod -A -o json | jq -r '.items[]|select(.spec.hostPID or .spec.hostNetwork)|.metadata.namespace+"/"+.metadata.name'
|
|
""",
|
|
"traps": [
|
|
"RuntimeClass is cluster-scoped and `handler` is a top-level field, not under spec.",
|
|
"runtimeClassName is immutable on a Pod - change the workload's template.",
|
|
"A ResourceQuota on cpu or memory rejects Pods without requests or limits unless a LimitRange supplies defaults.",
|
|
],
|
|
"docs": ["https://kubernetes.io/docs/concepts/containers/runtime-class/",
|
|
"https://kubernetes.io/docs/concepts/security/multi-tenancy/",
|
|
"https://kubernetes.io/docs/concepts/policy/resource-quotas/"],
|
|
},
|
|
{
|
|
"id": "4.4", "key": "mtls", "domain": "MICRO",
|
|
"text": "Implement Pod-to-Pod encryption (Cilium, Istio)",
|
|
"tasks": [68, 69, 70],
|
|
"know": [
|
|
"Istio: a sidecar (or the ambient ztunnel) per workload does mutual TLS for it. Label the Namespace `istio-injection=enabled` and re-create the Pods to get sidecars.",
|
|
"A PeerAuthentication sets what a workload accepts: PERMISSIVE (mTLS or plain) or STRICT (mTLS only). Named `default` with no selector it covers its Namespace; in the root Namespace (istio-system) it covers the mesh.",
|
|
"Cilium: transparent encryption between nodes with WireGuard or IPsec (a Helm value, `cilium encrypt status` to verify), and mutual authentication per policy with `authentication.mode: required` in a CiliumNetworkPolicy rule.",
|
|
"Encryption in transit does not replace NetworkPolicy: it says the peer is who it claims to be, not that it may connect.",
|
|
],
|
|
"fast": """
|
|
k label ns NS istio-injection=enabled && k -n NS rollout restart deploy
|
|
k -n NS get pod # 2/2 = sidecar present
|
|
cat <<EOF | k apply -f -
|
|
apiVersion: security.istio.io/v1
|
|
kind: PeerAuthentication
|
|
metadata: {name: default, namespace: NS}
|
|
spec: {mtls: {mode: STRICT}}
|
|
EOF
|
|
cilium encrypt status; cilium config view | grep -i encrypt
|
|
k explain ciliumnetworkpolicy.spec.ingress.authentication
|
|
""",
|
|
"traps": [
|
|
"Pods created before the Namespace was labelled have no sidecar; with STRICT they stop being reachable from the mesh until restarted.",
|
|
"A workload-level PeerAuthentication (with a selector) overrides the Namespace-wide one; port-level settings need the selector.",
|
|
"STRICT cuts off clients outside the mesh - that is the point, and also the usual surprise.",
|
|
],
|
|
"docs": ["https://istio.io/latest/docs/tasks/security/authentication/mtls-migration/",
|
|
"https://istio.io/latest/docs/reference/config/security/peer_authentication/",
|
|
"https://docs.cilium.io/en/stable/security/network/encryption/",
|
|
"https://docs.cilium.io/en/stable/network/servicemesh/mutual-authentication/mutual-authentication/"],
|
|
},
|
|
# ----------------------------------------------------- 5. Supply Chain Security (20%) ----
|
|
{
|
|
"id": "5.1", "key": "baseimage", "domain": "SUPPLY",
|
|
"text": "Minimize base image footprint",
|
|
"tasks": [71, 72, 73],
|
|
"know": [
|
|
"Small base, pinned: alpine, distroless or scratch at an exact version (or digest), never `latest`.",
|
|
"Multi-stage builds: compile in a full image, `COPY --from=build` only the result into the small one.",
|
|
"Run as a non-root USER, by number. No secrets in ENV, ARG or any layer - a later `RUN rm` does not remove them from the earlier layer.",
|
|
"Nothing a workload does not need: no package manager caches, shells, curl, compilers, sshd.",
|
|
],
|
|
"fast": """
|
|
podman build -t localhost/app:v2 . && podman images | grep app
|
|
podman history localhost/app:v2 # what each layer added
|
|
podman inspect localhost/app:v2 --format '{{.Config.User}} {{.Config.Env}}'
|
|
podman run --rm localhost/app:v2 id
|
|
# FROM golang:1.24-alpine AS build ... FROM gcr.io/distroless/static:nonroot
|
|
# COPY --from=build /out/app /app ; USER 65532 ; ENTRYPOINT ["/app"]
|
|
""",
|
|
"traps": [
|
|
"`USER` must come after the steps that need root (installing, chown) and before ENTRYPOINT.",
|
|
"`apt-get update` and `install` in separate RUN lines leave a stale cache layer; one RUN, and remove /var/lib/apt/lists in the same line.",
|
|
"`ADD` unpacks archives and fetches URLs - use COPY unless you mean that.",
|
|
],
|
|
"docs": ["https://kubernetes.io/docs/concepts/containers/images/",
|
|
"https://kubernetes.io/docs/concepts/security/security-checklist/#images"],
|
|
},
|
|
{
|
|
"id": "5.2", "key": "sbom", "domain": "SUPPLY",
|
|
"text": "Understand your supply chain (e.g. SBOM, CI/CD, artifact repositories)",
|
|
"tasks": [74, 75, 76, 77, 78],
|
|
"know": [
|
|
"An SBOM lists every package in an artifact with its version and licence, in SPDX or CycloneDX format. With it you can answer 'where do we run libfoo 1.2?' without pulling images.",
|
|
"Generate one from an image with `trivy image --format spdx-json|cyclonedx` or `bom generate`; scan one later with `trivy sbom`.",
|
|
"A tag can move; a digest (`image@sha256:...`) cannot. `status.containerStatuses[].imageID` shows what a Pod really runs.",
|
|
"The chain: source, build (CI), registry, admission, runtime. Each hand-over needs a check - signed artifacts, a trusted registry, admission control.",
|
|
],
|
|
"fast": """
|
|
trivy image --format spdx-json --output sbom.spdx.json IMAGE
|
|
trivy image --format cyclonedx --output sbom.cdx.json IMAGE
|
|
trivy sbom sbom.cdx.json --severity HIGH,CRITICAL
|
|
bom generate --image IMAGE --format json --output sbom.json
|
|
jq -r '.packages[] | .name+" "+.versionInfo' sbom.spdx.json | grep -i openssl
|
|
jq -r '.components[] | .name+" "+.version' sbom.cdx.json | grep -i openssl
|
|
k get pod -A -o jsonpath='{range .items[*]}{.metadata.namespace}/{.metadata.name}{"\\t"}{.status.containerStatuses[*].imageID}{"\\n"}{end}'
|
|
""",
|
|
"traps": [
|
|
"SPDX keeps packages under `.packages[]` with `versionInfo`; CycloneDX under `.components[]` with `version`.",
|
|
"The tag in a Pod spec says what was asked for, not what runs - compare digests.",
|
|
"`trivy image` scans by default; it only writes an SBOM when a `--format` for one is given - and its documentation is not on the exam's allowed list (bom's is): `trivy image --help | grep -A3 format`.",
|
|
],
|
|
"docs": ["https://kubernetes-sigs.github.io/bom/cli-reference/",
|
|
"https://kubernetes.io/docs/concepts/containers/images/",
|
|
"https://kubernetes.io/docs/tasks/administer-cluster/verify-signed-kubernetes-artifacts/"],
|
|
},
|
|
{
|
|
"id": "5.3", "key": "registries", "domain": "SUPPLY",
|
|
"text": "Secure your supply chain (permitted registries, sign and validate artifacts, etc.)",
|
|
"tasks": [8, 79, 80, 81, 82],
|
|
"know": [
|
|
"Allow only trusted registries at admission: a ValidatingAdmissionPolicy with a CEL expression over the Pod's images, the ImagePolicyWebhook admission plugin, or a policy engine (OPA Gatekeeper, Kyverno).",
|
|
"ImagePolicyWebhook: enable the plugin, point `--admission-control-config-file` at an AdmissionConfiguration that names a kubeconfig for the webhook backend, and set `defaultAllow: false` so nothing runs when the backend is down.",
|
|
"Pin by digest and verify signatures (cosign) so that what was built is what runs.",
|
|
"Check every container list: containers, initContainers and ephemeralContainers.",
|
|
],
|
|
"fast": """
|
|
k get validatingadmissionpolicy,validatingadmissionpolicybinding
|
|
# validations: [{expression: "object.spec.containers.all(c, c.image.startsWith('registry.corp/'))"}]
|
|
grep -n 'admission' /etc/kubernetes/manifests/kube-apiserver.yaml
|
|
cat /etc/kubernetes/policywebhook/admission_config.json # defaultAllow, kubeConfigFile
|
|
k get constrainttemplate; k get constraints
|
|
cosign verify --key cosign.pub IMAGE
|
|
k -n NS run probe --image=nginx:1.27 # the rejection message is your proof
|
|
""",
|
|
"traps": [
|
|
"A ValidatingAdmissionPolicy does nothing until a binding with `validationActions: [Deny]` exists.",
|
|
"The files ImagePolicyWebhook needs must be mounted into the API server Pod, and the kubeconfig path in the configuration is the path inside it.",
|
|
"`startsWith('registry.corp')` without the trailing slash also admits registry.corp.evil.io.",
|
|
],
|
|
"docs": ["https://kubernetes.io/docs/reference/access-authn-authz/validating-admission-policy/",
|
|
"https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#imagepolicywebhook",
|
|
"https://kubernetes.io/docs/concepts/containers/images/#using-a-private-registry"],
|
|
},
|
|
{
|
|
"id": "5.4", "key": "static", "domain": "SUPPLY",
|
|
"text": "Perform static analysis of user workloads and container images (e.g. Kubesec, KubeLinter)",
|
|
"tasks": [83, 84, 85, 86],
|
|
"know": [
|
|
"Static analysis reads a manifest, Dockerfile or image without running it. kubesec scores a manifest and lists what to add; KubeLinter lists checks a manifest fails; trivy scans images for known vulnerabilities and configuration files for misconfigurations.",
|
|
"What they all flag: privileged, root user, privilege escalation allowed, writable root filesystem, added capabilities, host namespaces and hostPath, missing resource limits, `latest` tags.",
|
|
"Scan the image that runs, by the name in the Pod spec, and act on what the task names: usually the HIGH and CRITICAL findings.",
|
|
],
|
|
"fast": """
|
|
kubesec scan pod.yaml | jq '.[0] | {score, critical: .scoring.critical, advise: [.scoring.advise[].id]}'
|
|
kube-linter lint deploy.yaml; kube-linter checks list | less
|
|
trivy image --severity HIGH,CRITICAL IMAGE; trivy image -q --severity CRITICAL --format json IMAGE | jq '[.Results[].Vulnerabilities[]?]|length'
|
|
trivy config .
|
|
k get pod -A -o jsonpath='{range .items[*]}{.metadata.namespace}/{.metadata.name}{"\\t"}{.spec.containers[*].image}{"\\n"}{end}'
|
|
""",
|
|
"traps": [
|
|
"kubesec scores one resource per document; a Deployment's settings are read from its Pod template.",
|
|
"A scanner reports what it finds in a file - the running object may differ. Apply the fixed file.",
|
|
"`--severity CRITICAL` hides HIGH: read which severities the task asks about.",
|
|
"None of these tools' own documentation may be opened in the exam: `trivy image --help`, `kubesec scan --help` and `kube-linter checks list` are the manual.",
|
|
],
|
|
"docs": ["https://kubernetes.io/docs/concepts/security/pod-security-standards/",
|
|
"https://kubernetes.io/docs/tasks/configure-pod-container/security-context/",
|
|
"https://kubernetes.io/docs/concepts/security/security-checklist/"],
|
|
},
|
|
# ---------------------------- 6. Monitoring, Logging and Runtime Security (20%) ----
|
|
{
|
|
"id": "6.1", "key": "falco", "domain": "RUNTIME",
|
|
"text": "Perform behavioral analytics to detect malicious activities",
|
|
"tasks": [103, 104, 105, 106, 107],
|
|
"know": [
|
|
"Falco watches system calls on a node and raises an alert when a rule's condition matches: a shell in a container, a write below /etc, a package manager run, an unexpected outbound connection.",
|
|
"A rule has `rule`, `desc`, `condition`, `output`, `priority`; macros and lists are reusable pieces of conditions. Shipped rules live in /etc/falco/falco_rules.yaml; your changes go in /etc/falco/falco_rules.local.yaml, which is read later and wins.",
|
|
"Output fields name what happened and where: %evt.time, %user.name, %proc.name, %proc.cmdline, %fd.name, %container.id, %container.name, %k8s.pod.name, %k8s.ns.name.",
|
|
"From an alert to the workload: the container id leads to the Pod with `crictl ps --id` and `crictl pods`.",
|
|
],
|
|
"fast": """
|
|
systemctl status falco-modern-bpf; journalctl -u falco-modern-bpf --since '5 min ago' | grep -i 'shell\\|Warning\\|Error'
|
|
falco --list | less # every field a condition or output can use
|
|
falco -r /etc/falco/falco_rules.local.yaml -M 30 > /opt/course/out.log # run 30 s with these rules
|
|
grep -n -B2 -A12 'rule: Terminal shell in container' /etc/falco/falco_rules.yaml
|
|
crictl ps --id CONTAINER_ID; crictl pods --id POD_ID
|
|
k -n NS scale deploy D --replicas=0
|
|
""",
|
|
"traps": [
|
|
"Edit the local file, not the shipped one, and restart Falco - a syntax error stops it from starting (`falco --validate FILE`).",
|
|
"To change a shipped rule, repeat it in the local file with an `override:` block (or redefine it whole).",
|
|
"Falco runs on the node where the event happens: look on the node the Pod runs on.",
|
|
],
|
|
"docs": ["https://falco.org/docs/concepts/rules/basic-elements/",
|
|
"https://falco.org/docs/reference/rules/supported-fields/",
|
|
"https://falco.org/docs/concepts/rules/overriding/"],
|
|
},
|
|
{
|
|
"id": "6.2", "key": "threats", "domain": "RUNTIME",
|
|
"text": "Detect threats within physical infrastructure, apps, networks, data, users and workloads",
|
|
"tasks": [87, 88, 89, 90],
|
|
"know": [
|
|
"Work from a symptom to a workload: a process (`ps`, /proc), a listening or outbound connection (`ss`), a file that changed, an unusual Pod spec (privileged, hostPath, hostPID, runs as root).",
|
|
"From a process on the node to its Pod: the pid's cgroup or `crictl inspect` (info.pid) names the container, `crictl ps` the Pod.",
|
|
"From a Pod to its processes without a shell in the image: `crictl inspect`, /proc/<pid>/root for its filesystem, `kubectl debug` for an ephemeral container.",
|
|
],
|
|
"fast": """
|
|
crictl ps; crictl inspect -o go-template --template '{{.info.pid}}' CONTAINER_ID
|
|
cat /proc/PID/cgroup; ls -l /proc/PID/exe; tr '\\0' ' ' < /proc/PID/cmdline
|
|
ss -tnp | grep -v 127.0.0.1 # connections with their processes
|
|
k get pod -A -o json | jq -r '.items[]|select(any(.spec.containers[];.securityContext.privileged))|.metadata.namespace+"/"+.metadata.name'
|
|
k -n NS debug -it POD --image=busybox:1.36 --target=CONTAINER
|
|
""",
|
|
"traps": [
|
|
"A process name proves little - an attacker names a miner `nginx`. Check the binary's path and hash.",
|
|
"Deleting a Pod a controller owns just brings it back: scale or delete the owner.",
|
|
],
|
|
"docs": ["https://kubernetes.io/docs/tasks/debug/debug-cluster/crictl/",
|
|
"https://kubernetes.io/docs/tasks/debug/debug-application/debug-running-pod/"],
|
|
},
|
|
{
|
|
"id": "6.3", "key": "investigate", "domain": "RUNTIME",
|
|
"text": "Investigate and identify phases of attack and bad actors within the environment",
|
|
"tasks": [91, 92, 93, 94],
|
|
"know": [
|
|
"Reconstruct the sequence: how they got in (a leaked token, an exposed service), what they looked at (secrets, RBAC), how they raised their privileges (a new binding, a privileged Pod), how they stay (a CronJob, a static Pod, a webhook, a new ServiceAccount).",
|
|
"The audit log has who (`user.username`, `impersonatedUser`), from where (`sourceIPs`), what (`verb`, `objectRef`) and the outcome (`responseStatus.code`). jq makes it searchable.",
|
|
"Persistence hides in places `kubectl get all` does not show: ClusterRoleBindings, Secrets, CronJobs, static Pod manifests, admission webhooks.",
|
|
],
|
|
"fast": """
|
|
jq -c 'select(.objectRef.resource=="secrets" and .verb=="get")|[.requestReceivedTimestamp,.user.username,.sourceIPs[0],.objectRef.namespace+"/"+.objectRef.name]' audit.log
|
|
jq -r 'select(.verb=="create" and .objectRef.resource=="clusterrolebindings")|.user.username+" "+.objectRef.name' audit.log
|
|
jq -r '.user.username' audit.log | sort | uniq -c | sort -rn | head
|
|
k get clusterrolebinding -o wide --sort-by=.metadata.creationTimestamp | tail
|
|
k get cronjob,ds -A; ls -l /etc/kubernetes/manifests/
|
|
""",
|
|
"traps": [
|
|
"One request appears once per stage (RequestReceived, ResponseComplete) - count one stage only.",
|
|
"A ServiceAccount is the identity that was used, not necessarily the actor: find who created or stole its token.",
|
|
],
|
|
"docs": ["https://kubernetes.io/docs/tasks/debug/debug-cluster/audit/",
|
|
"https://kubernetes.io/docs/reference/config-api/apiserver-audit.v1/"],
|
|
},
|
|
{
|
|
"id": "6.4", "key": "immutability", "domain": "RUNTIME",
|
|
"text": "Ensure immutability of containers at runtime",
|
|
"tasks": [95, 96, 97, 98],
|
|
"know": [
|
|
"An immutable container cannot change itself after start: `readOnlyRootFilesystem: true`, with an emptyDir for the few paths the program must write.",
|
|
"Also needed: not privileged, `allowPrivilegeEscalation: false`, no writable hostPath - each of those lets a process change the node instead.",
|
|
"Configuration can be frozen too: `immutable: true` on a ConfigMap or Secret.",
|
|
"Enforce it rather than hope: Pod Security, a ValidatingAdmissionPolicy, and RBAC without pods/exec.",
|
|
],
|
|
"fast": """
|
|
k get pod -A -o json | jq -r '.items[]|select(any(.spec.containers[];.securityContext.readOnlyRootFilesystem!=true))|.metadata.namespace+"/"+.metadata.name'
|
|
k -n NS exec POD -- touch /probe # 'Read-only file system' is the answer you want
|
|
k -n NS set volume deploy D --add --name=tmp --type=emptyDir --mount-path=/tmp
|
|
k -n NS patch cm C -p '{"immutable": true}'
|
|
""",
|
|
"traps": [
|
|
"readOnlyRootFilesystem is per container - set it on every container, init containers included.",
|
|
"Most images write somewhere (nginx: /var/cache/nginx and /var/run): find the path in the Pod's log and mount an emptyDir there.",
|
|
"`immutable: true` cannot be undone; the object has to be deleted and created again.",
|
|
],
|
|
"docs": ["https://kubernetes.io/docs/tasks/configure-pod-container/security-context/",
|
|
"https://kubernetes.io/docs/concepts/configuration/configmap/#configmap-immutable"],
|
|
},
|
|
{
|
|
"id": "6.5", "key": "audit", "domain": "RUNTIME",
|
|
"text": "Use Kubernetes audit logs to monitor access",
|
|
"tasks": [9, 99, 100, 101, 102],
|
|
"know": [
|
|
"An audit Policy is a list of rules; the first rule that matches a request decides its level: None, Metadata (who, what, when), Request (plus the request body) or RequestResponse (plus the response).",
|
|
"A rule matches on users, userGroups, verbs, resources (group, resources, resourceNames), namespaces and nonResourceURLs; `omitStages` drops stages such as RequestReceived.",
|
|
"API server flags: `--audit-policy-file`, `--audit-log-path`, `--audit-log-maxage` (days), `--audit-log-maxbackup` (files), `--audit-log-maxsize` (MB). Policy file and log directory are hostPath mounts of the static Pod.",
|
|
"Never log Secrets above Metadata: Request level writes their contents into the log.",
|
|
],
|
|
"fast": """
|
|
vi /etc/kubernetes/audit/policy.yaml; vi /etc/kubernetes/manifests/kube-apiserver.yaml
|
|
watch crictl ps # wait for kube-apiserver to return
|
|
tail -f /var/log/kubernetes/audit/audit.log | jq -c '[.level,.verb,.user.username,.objectRef.resource,.objectRef.name]'
|
|
jq -c 'select(.objectRef.resource=="secrets")' /var/log/kubernetes/audit/audit.log | tail -1
|
|
crictl logs $(crictl ps -a --name kube-apiserver -q | head -1) 2>&1 | tail # why it does not start
|
|
""",
|
|
"traps": [
|
|
"Order matters: a catch-all rule placed first swallows every rule after it.",
|
|
"Core resources are group \"\" - write `group: \"\"`, not `group: core`.",
|
|
"Without the two mounts the API server fails to start, and `kubectl` cannot tell you why: use crictl on the node.",
|
|
"A changed policy file is only read at start: touch the manifest (or move it out and back) to restart the API server.",
|
|
],
|
|
"docs": ["https://kubernetes.io/docs/tasks/debug/debug-cluster/audit/",
|
|
"https://kubernetes.io/docs/reference/config-api/apiserver-audit.v1/"],
|
|
},
|
|
]
|