feat(server): report usage on the final stream chunk

rcsheets commented

2026-07-22 23:52:13 +00:00

Owner

Streaming responses carried no token accounting: usage only existed on
the non-streaming response shapes. A client (or an aggregating proxy,
which is the motivating case) reassembling a stream into a plain
response had nothing to carry over.

Both /v1/completions and /v1/chat/completions now set usage on the
final chunk -- the same values the non-streaming response would have
carried -- and only there, so delta chunks are unchanged.

Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com

Streaming responses carried no token accounting: usage only existed on the non-streaming response shapes. A client (or an aggregating proxy, which is the motivating case) reassembling a stream into a plain response had nothing to carry over. Both /v1/completions and /v1/chat/completions now set usage on the final chunk -- the same values the non-streaming response would have carried -- and only there, so delta chunks are unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

rcsheets added 1 commit

2026-07-22 23:52:13 +00:00

feat(server): report usage on the final stream chunk

ci / test_and_build (pull_request) Has been cancelled

Details

b51b9c75f8

Streaming responses carried no token accounting: usage only existed on
the non-streaming response shapes. A client (or an aggregating proxy,
which is the motivating case) reassembling a stream into a plain
response had nothing to carry over.

Both /v1/completions and /v1/chat/completions now set usage on the
final chunk -- the same values the non-streaming response would have
carried -- and only there, so delta chunks are unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

pr-reviewer-bot commented

2026-07-22 23:52:15 +00:00

Collaborator

Automated review by pr-reviewer v0.37.0 | Safety Check | Claude Sonnet 5 | tracking id r-6157af-194a72
This is an AI-generated review and may contain mistakes.

Status: ✅ Completed

⚠️ The primary review (Mistral Small) failed. The review below is from Claude Sonnet 5, which reviewed the same diff.

✅ Verdict: LGTM — Small, well-tested feature addition with no signs of bugs or security issues.

Straightforward, well-scoped change: adds a Usage pointer field to both streaming chunk types and populates it only on the final SSE chunk via an existing usage(res) helper, mirroring non-streaming behavior. Tests were updated to assert usage is present only on the last chunk and absent elsewhere for both completions and chat completions endpoints. No signs of leaked secrets, injection risk, or breaking changes — omitempty keeps the field backward-compatible for clients not expecting it. Diff is small and consistent with the stated intent.

*Automated review by [pr-reviewer](https://git.brooktrails.org/brooktrails/pr-reviewer) v0.37.0 | Safety Check | Claude Sonnet 5 | tracking id `r-6157af-194a72`* *This is an AI-generated review and may contain mistakes.* **Status:** ✅ Completed --- > ⚠️ The primary review (**Mistral Small**) failed. The review below is from **Claude Sonnet 5**, which reviewed the same diff. **✅ Verdict: LGTM** — Small, well-tested feature addition with no signs of bugs or security issues. Straightforward, well-scoped change: adds a `Usage` pointer field to both streaming chunk types and populates it only on the final SSE chunk via an existing `usage(res)` helper, mirroring non-streaming behavior. Tests were updated to assert usage is present only on the last chunk and absent elsewhere for both completions and chat completions endpoints. No signs of leaked secrets, injection risk, or breaking changes — `omitempty` keeps the field backward-compatible for clients not expecting it. Diff is small and consistent with the stated intent.

rcsheets force-pushed feat/stream-usage from b51b9c75f8

ci / test_and_build (pull_request) Has been cancelled

Details

to 19d4ca806b

ci / test_and_build (pull_request) Successful in 22s

Details

2026-07-22 23:52:18 +00:00

Compare

rcsheets merged commit 7d5e98d7f6 into main

2026-07-22 23:57:10 +00:00

rcsheets deleted branch feat/stream-usage

2026-07-22 23:57:10 +00:00