feat(engine): response debug preview, unlimited previews in files, and closing lines for unfinished requests #107
Loading…
Reference in a new issue
No description provided.
Delete branch "feat/debug-response-preview"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
gllm servealready has--debug-with-prompt-preview, which adds the text of the first few prompt tokens to the debugengine: request receivedline. This adds the response-side counterpart, lets either preview be unlimited, and gives requests that do not complete a closing log line.Response preview
--debug-with-response-previewadds aresponse_previewfield to theengine: request completeline: the text of the first few generated tokens, rendered the same way as the prompt preview (control tokens by name rather than dropped).--debug-response-preview-size=Nsets how many tokens (default 8). It is off by default, since it logs response content.The request-complete line is INFO, but the preview is a debug aid, so like the prompt preview it only appears at
--log-level debug.Unlimited previews
--debug-prompt-preview-size=-1and--debug-response-preview-size=-1mean the whole prompt or response. That is too much for a log line, so an unlimited preview is written to a file named for the request id, and the log line carries the path in place of the preview field:<request_id>.prompt.txtprompt_fileonengine: request received<request_id>.response.txtresponse_fileon the request's closing line--debug-preview-dirnames the directory. Without it, the engine makes a fresh directory withos.MkdirTemp--gllm-previews-*in$TMPDIR, else/tmp-- on first use, and logs its path once. A fixed name in a shared temp dir would be predictable, so another user could create it first and read or redirect what is written;MkdirTemppicks an unused name and makes the directory owner-only. Files are written 0600. Request ids are minted by the engine, never taken from the client, so they are safe file names.Nothing prunes the directory. A failed write is logged as a warning and does not fail the request.
Closing lines for cancelled and failed requests
Every request that reaches the engine has an id -- the server mints it and sends it in
X-Request-Idbefore generating -- but only a completed request logged it at the end. A cancelled request logged nothing afterrequest received; a failed one logged at most the server's ERROR line, which has no id, and a too-large or not-implemented request logged nothing at all.Each now gets an INFO closing line with
request_id,prompt_tokens,completion_tokensanderr:engine: request cancelledwhen the request context ended (the client went away).engine: request failedfor any other error. A prompt that fails to tokenize gets this line too, with onlyrequest_idanderr, since it was never admitted.These lines are INFO whatever the error: the server still decides what deserves its ERROR line. Both carry the response preview the same way
request completedoes, so a stopped request shows what it had generated -- and with an unlimited preview, every admitted request ends up with a response file, empty if it produced nothing.cancelreturns the generated tokens, copied under the engine lock, for this.Requests rejected before an id is minted (an undecodable body, a wrong model name) are unchanged: no id, no closing line, no files.
Behaviour changes
0still means "use the default", but a negative size, which used to be treated the same way, now means unlimited.Testing
go build ./...,go vet ./...,gofmtandgo test ./...are clean, and the engine and server tests pass under-race. New engine tests cover the response preview and its size cap, its gating on the option and the debug level, unlimited previews written to a named directory, the default temp directory (one per engine, mode 0700, logged once), and the cancelled and failed closing lines with their response files. Not exercised against a real checkpoint or on a GPU.🤖 Generated with Claude Code
Automated review by pr-reviewer v0.52.3 | Safety Check | Nemotron 3 Nano | tracking id
r-bf65c4-7ef5fbThis is an AI-generated review and may contain mistakes.
Status: ❌ Failed
Review failed. Tracking id
r-bf65c4-7ef5fb— see logs for details.Comment
@pr-reviewer-bot retryto try again.feat(engine): add a response debug preview, and unlimited previews written to filesto feat(engine): response debug preview, unlimited previews in files, and closing lines for unfinished requests