docs(perf): use idle-card numbers for the GQA attention result #130

Merged
rcsheets merged 1 commit from docs/attn-gqa-clean-numbers into main 2026-11-04 02:19:49 +00:00
Owner

#129 merged with the original profile, taken while a live gllm serve shared the Blackwell. That server has now exited, so re-measure on the idle card and correct the section:

  • tiled flash kernel 3.39 ms -> 1.00 ms (3.4x, was reported as 3.72 -> 1.03, 3.6x)
  • DRAM throughput 26.6% -> 58.3%; achieved occupancy 75.8% -> 56.9%
  • end-to-end launch, host staging included: 4.72 -> 1.41 ms (Nemotron 3, 16 x 30k) and 0.62 -> 0.35 ms (Granite 4.2 8b, 8 x 7k)
  • eligible warps per scheduler is 1.2, not 0.3; the latency-bound conclusion holds

The Granite number also records why the win shrinks with the group factor: at group 4 L2 already served part of the duplicated traffic.

No code changes in this PR. The #129 description was updated with the same clean numbers.

`#129` merged with the original profile, taken while a live `gllm serve` shared the Blackwell. That server has now exited, so re-measure on the idle card and correct the section: - tiled flash kernel 3.39 ms -> 1.00 ms (**3.4x**, was reported as 3.72 -> 1.03, 3.6x) - DRAM throughput 26.6% -> 58.3%; achieved occupancy 75.8% -> 56.9% - end-to-end launch, host staging included: 4.72 -> 1.41 ms (Nemotron 3, 16 x 30k) and 0.62 -> 0.35 ms (Granite 4.2 8b, 8 x 7k) - eligible warps per scheduler is 1.2, not 0.3; the latency-bound conclusion holds The Granite number also records why the win shrinks with the group factor: at group 4 L2 already served part of the duplicated traffic. No code changes in this PR. The #129 description was updated with the same clean numbers.
docs(perf): replace the contended GQA numbers with idle-card ones
All checks were successful
ci / test_and_build (pull_request) Successful in 41s
14ad623cd6
The live server that shared the Blackwell during the first measurement has
exited, so re-run the profile and benchmark on the idle card: 3.39 ms ->
1.00 ms kernel time (3.4x, not 3.6x), DRAM 26.6% -> 58.3%, occupancy
75.8% -> 56.9%. Drop the caveat and add the end-to-end launch times for
Nemotron 3 and Granite 4.2 8b, which show the win shrinking with the group
factor because L2 already served part of the duplicated traffic.

The clean run also puts eligible warps at 1.2 per scheduler, not 0.3, so
correct that while keeping the latency-bound conclusion. The register-count
comparison for ATTN_MAX_HEADS_PER_WARP no longer quotes the contended
sweep's absolute times.
Collaborator

Automated review by pr-reviewer v0.54.0 | Safety Check | Nemotron 3 Nano | tracking id r-e99cdb-4c779d
This is an AI-generated review and may contain mistakes.

Status: ❌ Failed


Review failed. Tracking id r-e99cdb-4c779d — see logs for details.

Comment @pr-reviewer-bot retry to try again.

<!-- pr-reviewer:review --> *Automated review by [pr-reviewer](https://git.brooktrails.org/brooktrails/pr-reviewer) v0.54.0 | Safety Check | Nemotron 3 Nano | tracking id `r-e99cdb-4c779d`* *This is an AI-generated review and may contain mistakes.* **Status:** ❌ Failed --- Review failed. Tracking id `r-e99cdb-4c779d` — see logs for details. Comment `@pr-reviewer-bot retry` to try again.
rcsheets deleted branch docs/attn-gqa-clean-numbers 2026-11-04 02:19:49 +00:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
brooktrails/gllm!130
No description provided.