feat(cli): add gllm health so a container can probe itself #57
Loading…
Reference in a new issue
No description provided.
Delete branch "feat/gllm-health"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
The server has exposed /livez and /readyz since the probe split, but nothing
in the images can call them. A container healthcheck runs INSIDE the
container, and the CPU image is FROM scratch -- no curl, no shell, nothing.
The CUDA image has a shell but no HTTP client either: no curl, no wget, no
python, only dash. So the obvious healthcheck,
curl -fsS localhost:8000/readyz, fails permanently while the server servesfine, reporting the container unhealthy for a reason that has nothing to do
with the server.
The binary is the one tool guaranteed to be present, so it carries the probe:
Readiness is the default, being what decides whether traffic should arrive:
503 while the model loads and once a drain begins. --live selects /livez for
callers that want "is the process answering HTTP at all". /healthz is
deliberately not offered -- it is a deprecated alias of /readyz that warns
per probing IP in the server log, and a new probe should not be adding
itself to that list.
--endpoint takes a full URL, a bare host:port, or the
:portthat --addritself takes, since copying the server's own flag value across is the
obvious thing to try and should not be a usage error. Failures distinguish
not-listening from listening-but-not-ready from answering-something-else,
because the exit status alone cannot and a person debugging by hand is
asking exactly that.
Co-Authored-By: Claude Opus 5 noreply@anthropic.com
gllm healthso a container can probe itselfThe server has exposed /livez and /readyz since the probe split, but nothing in the images can call them. A container healthcheck runs INSIDE the container, and the CPU image is FROM scratch -- no curl, no shell, nothing. The CUDA image has a shell but no HTTP client either: no curl, no wget, no python, only dash. So the obvious healthcheck, `curl -fsS localhost:8000/readyz`, fails permanently while the server serves fine, reporting the container unhealthy for a reason that has nothing to do with the server. The binary is the one tool guaranteed to be present, so it carries the probe: HealthCmd=/gllm health Readiness is the default, being what decides whether traffic should arrive: 503 while the model loads and once a drain begins. --live selects /livez for callers that want "is the process answering HTTP at all". /healthz is deliberately not offered -- it is a deprecated alias of /readyz that warns per probing IP in the server log, and a new probe should not be adding itself to that list. --endpoint takes a full URL, a bare host:port, or the `:port` that --addr itself takes, since copying the server's own flag value across is the obvious thing to try and should not be a usage error. Failures distinguish not-listening from listening-but-not-ready from answering-something-else, because the exit status alone cannot and a person debugging by hand is asking exactly that. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>Automated review by pr-reviewer v0.41.0 | Safety Check | Claude Sonnet 5 | tracking id
r-646c5d-3c038cThis is an AI-generated review and may contain mistakes.
Status: ✅ Completed
✅ Verdict: LGTM — Purely additive CLI health-check command with reasonable input validation and body-size capping; no secrets, breaking changes, or security issues found.
This PR adds a
gllm healthsubcommand that HTTP-probes a running server's /readyz or /livez endpoint and reflects success/failure via exit status. It's a straightforward, well-scoped CLI addition:resolveEndpointnormalizes various endpoint forms (full URL, host:port, bare :port) into a base URL, with sane validation (rejects empty, non-http/https schemes).probeissues a GET, distinguishes connection failure vs non-200 status, caps response body read at 256 bytes to avoid unbounded allocation from a malicious/wrong endpoint.http.DefaultClientusage is fine for a CLI healthcheck tool run standalone in a container.This is purely additive (new subcommand + README docs), doesn't touch server logic, and doesn't introduce any breaking changes to existing commands. Nothing alarming stands out.
7b2272e3db874a608555