fix(proxy): reassemble reasoning_content in aggregated chat completions #19

Merged

rcsheets merged 1 commit from fix/aggregate-reasoning into main

2026-10-02 19:59:32 +00:00

rcsheets commented

2026-10-02 19:47:52 +00:00

Owner

Problem

With aggregate: true, SLP turns a non-streaming chat completion into an upstream stream and reassembles it. The reassembly kept only delta.content. A reasoning model streams its chain of thought first, in reasoning_content deltas, so:

the reassembled message had no reasoning_content, although the same request sent to the upstream without streaming returns one;
a turn that ran out of max_tokens mid-thought came back as "content": "" with a full completion_tokens count and nothing to show where the tokens went.

usage is carried verbatim, so completion_tokens_details.reasoning_tokens already crossed over; only the text was lost.

Change

aggregateStream collects delta.reasoning_content per choice and emits it as the message's reasoning_content, beside content rather than inside it. The field is omitted when a turn had no reasoning deltas, matching the upstream's own non-streaming response. Reasoning bytes count toward maxAggregateBytes.

Streaming clients and text completions are unaffected.

Verification

TestAggregateChatCompletionReasoning covers reasoning followed by an answer, reasoning with no answer, and a turn with no reasoning (key absent). go build ./..., go vet ./... and go test ./... pass.

Related: brooktrails/gllm#109 fixes the gllm bug that made constrained answers land in the reasoning field in the first place.

🤖 Generated with Claude Code

## Problem With `aggregate: true`, SLP turns a non-streaming chat completion into an upstream stream and reassembles it. The reassembly kept only `delta.content`. A reasoning model streams its chain of thought first, in `reasoning_content` deltas, so: - the reassembled message had no `reasoning_content`, although the same request sent to the upstream without streaming returns one; - a turn that ran out of `max_tokens` mid-thought came back as `"content": ""` with a full `completion_tokens` count and nothing to show where the tokens went. `usage` is carried verbatim, so `completion_tokens_details.reasoning_tokens` already crossed over; only the text was lost. ## Change `aggregateStream` collects `delta.reasoning_content` per choice and emits it as the message's `reasoning_content`, beside `content` rather than inside it. The field is omitted when a turn had no reasoning deltas, matching the upstream's own non-streaming response. Reasoning bytes count toward `maxAggregateBytes`. Streaming clients and text completions are unaffected. ## Verification `TestAggregateChatCompletionReasoning` covers reasoning followed by an answer, reasoning with no answer, and a turn with no reasoning (key absent). `go build ./...`, `go vet ./...` and `go test ./...` pass. Related: brooktrails/gllm#109 fixes the gllm bug that made constrained answers land in the reasoning field in the first place. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

rcsheets added 1 commit

2026-10-02 19:47:52 +00:00

fix(proxy): reassemble reasoning_content in aggregated chat completions

ci / check (pull_request) Has been cancelled

Details

c38635d5a4

An aggregated chat completion kept only delta.content. A reasoning model
streams its chain of thought first, in reasoning_content deltas, so the
reassembled message lost it -- and a turn that ran out of budget
mid-thought came back as an empty message with a full completion token
count and nothing to show where the tokens went.

Reasoning deltas are now collected into the message's reasoning_content,
beside content rather than inside it, and count toward the aggregate size
cap. The field is omitted for a turn with no reasoning deltas, as the
upstream's own non-streaming response omits it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

pr-reviewer-bot commented

2026-10-02 19:47:59 +00:00

Collaborator

Automated review by pr-reviewer v0.52.3 | Safety Check | Granite | tracking id r-c00a82-f0bbb4
This is an AI-generated review and may contain mistakes.

Status: ❌ Failed

Review failed. Tracking id r-c00a82-f0bbb4 — see logs for details.

Comment @pr-reviewer-bot retry to try again.

*Automated review by [pr-reviewer](https://git.brooktrails.org/brooktrails/pr-reviewer) v0.52.3 | Safety Check | Granite | tracking id `r-c00a82-f0bbb4`* *This is an AI-generated review and may contain mistakes.* **Status:** ❌ Failed --- Review failed. Tracking id `r-c00a82-f0bbb4` — see logs for details. Comment `@pr-reviewer-bot retry` to try again.

rcsheets force-pushed fix/aggregate-reasoning from c38635d5a4

ci / check (pull_request) Has been cancelled

Details

to b9957c8d84

ci / check (pull_request) Successful in 47s

Details

2026-10-02 19:48:18 +00:00

Compare

rcsheets referenced this pull request from brooktrails/pr-reviewer

2026-10-02 19:49:42 +00:00

feat: per-config thinking switch and stored reasoning for gllm #118

rcsheets merged commit 50adf3360a into main