vumast3r
1cd963e8b5
RESTORE_BASELINE ran `sed -i` on the scheduler and controller-manager manifests before every session, whether anything needed fixing or not. sed writes its temp file (sedXXXXXX) into /etc/kubernetes/manifests, the kubelet loads it as one more static pod and kills the running one, so every session start sent both into CrashLoopBackOff for up to minutes - while that session's tasks were being seeded. New namespaces got no default ServiceAccount, so their bare Pods were rejected, and Pods sat unscheduled. The practice cluster's kubelet journal showed the sed churn at each session start, and kube-controller-manager had restarted 10 times. Manifests are now edited through `medit`: only when an edit changes the content, and via a hidden temp file (the kubelet ignores dot-files) moved into place. mkns waits 90 s for the ServiceAccount instead of 30 and fails loudly instead of silently. Verified on a throwaway cluster: three no-op baselines cause zero restarts; a scheduler command typo, an injected controller-manager flag and an apiserver command typo are all still repaired. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mhxm4rsUfuE5HoytYbu2YW
28 lines
1.3 KiB
Python
28 lines
1.3 KiB
Python
"""Shared helpers for CKA task banks."""
|
|
|
|
NS = r"""
|
|
mkns() {
|
|
$K create ns "$1" --dry-run=client -o yaml | $K apply -f - >/dev/null
|
|
# Wait for the namespace's `default` ServiceAccount before returning. The controller creates it
|
|
# asynchronously, and a Pod applied in the gap is rejected outright:
|
|
# error looking up service account <ns>/default: serviceaccount "default" not found
|
|
# Setups that create a namespace and immediately apply Pods lose that race under load,
|
|
# which fails the seed and costs the candidate a whole task. It belongs here, not in each task.
|
|
# 90 s, not 30: the ServiceAccount controller runs in kube-controller-manager, and a manager that
|
|
# is restarting (it takes ~60 s to be back and leading) left bare Pods rejected with no trace.
|
|
for _ in $(seq 1 90); do $K -n "$1" get sa default >/dev/null 2>&1 && return 0; sleep 1; done
|
|
echo "namespace $1 still has no default ServiceAccount after 90 s - is kube-controller-manager running?" >&2
|
|
return 1
|
|
}
|
|
"""
|
|
|
|
def C(name, pts, script):
|
|
return {"name": name, "pts": pts, "script": script}
|
|
|
|
def ns_pick(r, base):
|
|
"""Namespace name with a random suffix so the same task never lands on the same namespace twice."""
|
|
return f"{base}-{r.choice(['alpha', 'bravo', 'delta', 'echo', 'kilo', 'nova', 'omega', 'zulu'])}"
|
|
|
|
def name_pick(r, base):
|
|
return f"{base}-{r.randint(10, 99)}"
|