refactor: rename identifiers that differed from a neighbour only by case #103

Merged
rcsheets merged 1 commit from style/case-only-names into main 2026-10-01 09:00:16 +00:00
Owner

Problem

Two places use a pair of names in one scope that differ only in capitalisation, so they read as one name:

  • kvcache.Manager.matchBlocks holds the block size in B next to a block pointer b.
  • The grouped NVFP4 GEMV kernel and its launcher read the rows field of gllm_nvfp4_group beside the ROWS template parameter.

Change

  • B becomes size in matchBlocks.
  • The rows field of gllm_nvfp4_group becomes nrows, in gllm_kernels.h, kernels.cu and the Go caller that fills the struct (grouped.go).

Pairs left alone on purpose: a generic test helper's type parameter T beside its *testing.T argument t, and the GLLM_TARGET_SMS macro beside the gllm_target_sms() function that returns it.

Merge order

This branch is based on #102 (perf/nvfp4-gemv-rows), which rewrites the kernel lines the field rename touches. Until #102 merges, this PR also shows its commit; merge #102 first.

Verification

  • The PTX that nvcc generates from kernels.cu is identical before and after for sm_86 and sm_120, so the device code is unchanged.
  • go test ./internal/kvcache/ passes.
  • go build ./..., the cuda-tagged build of cmd/gllm and the cuda-tagged test binary for internal/backend/cuda all compile.

Not run: the CUDA tests on a GPU. The host-side launcher change is the field rename only.

🤖 Generated with Claude Code

## Problem Two places use a pair of names in one scope that differ only in capitalisation, so they read as one name: - `kvcache.Manager.matchBlocks` holds the block size in `B` next to a block pointer `b`. - The grouped NVFP4 GEMV kernel and its launcher read the `rows` field of `gllm_nvfp4_group` beside the `ROWS` template parameter. ## Change - `B` becomes `size` in `matchBlocks`. - The `rows` field of `gllm_nvfp4_group` becomes `nrows`, in `gllm_kernels.h`, `kernels.cu` and the Go caller that fills the struct (`grouped.go`). Pairs left alone on purpose: a generic test helper's type parameter `T` beside its `*testing.T` argument `t`, and the `GLLM_TARGET_SMS` macro beside the `gllm_target_sms()` function that returns it. ## Merge order This branch is based on #102 (`perf/nvfp4-gemv-rows`), which rewrites the kernel lines the field rename touches. Until #102 merges, this PR also shows its commit; merge #102 first. ## Verification - The PTX that nvcc generates from `kernels.cu` is identical before and after for sm_86 and sm_120, so the device code is unchanged. - `go test ./internal/kvcache/` passes. - `go build ./...`, the `cuda`-tagged build of `cmd/gllm` and the `cuda`-tagged test binary for `internal/backend/cuda` all compile. Not run: the CUDA tests on a GPU. The host-side launcher change is the field rename only. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
perf(cuda): compute several output rows per warp in the NVFP4 GEMV
All checks were successful
ci / test_and_build (pull_request) Successful in 32s
ff5944a07c
A profiled Nemotron 3 Nano decode step (8 requests) spent 70% of its
23 ms in the two wide-load NVFP4 GEMV kernels, with the GPU busy 86% of
the step. A new microbenchmark (BenchmarkCUDANVFP4GEMV and
BenchmarkCUDANVFP4GEMVGrouped) showed why: launch time was linear in
the products, about 1.5e12 a second at every shape and row count, and
not in the weight bytes. A kernel that only read the same weights took
2.5 us of a 49 us 8-row launch. Each warp computed one output row, so
every product paid for a quarter of an activation load and every warp
fetched each activation row for itself; pointing those loads at a small
window alone cut the launch to 27 us.

gemv_vec_rows replaces gemv_vec_row: a warp computes OUTS consecutive
output rows (a template parameter), each lane carrying that many weight
rows through a chunk, so one activation load feeds OUTS products per
column. Row counts above 8 run as
tiles of 8, each a full pass over the weights, so the accumulators keep
a fixed size. Every (row, output) value is computed by the same
operations in the same order as before, so the results are
bit-identical for any OUTS.

More rows per warp means proportionally fewer warps, and a launch short
of warps slows down again: 8 rows through 3712x2688 took 49 us at one
row per warp, 35 at 2, 29 at 4 and 44 at 8. gemv_rows_per_warp picks
the largest count that leaves every SM four warps, capped at 2 for a single matrix with one or two
rows (which gains nothing from more) and at 4 otherwise; a grouped
launch has a set of warps per expert and usually takes 8.

Measured on the RTX PRO 6000 Blackwell (300 W cap), launch times from
the benchmark, which re-reads the same weights and so serves them from
L2:

  3712x2688, 8 rows                      49 -> 30 us
  3712x2688, 32 rows                    254 -> 116 us
  2688x4096, 8 rows                      51 -> 34 us
  grouped 1856x2688, 8 tokens/37 experts 152 -> 112 us
  grouped 1856x2688, 32 tokens/48 experts 554 -> 263 us
  one row, any shape                     unchanged to 15% faster

On an RTX 3070, 3712x2688 at 8 rows went 247 -> 108 us.

Nemotron 3 Nano NVFP4, 8 concurrent requests x 256 tokens: 383 -> 437
tok/s, 0.52 -> 0.44 J/token of GPU energy. Eight sequential greedy
requests (four ~5k-token prefills, four short chats) produced
byte-identical replies before and after. GEMV outputs over five shapes,
ten row counts and two grouped layouts were bit-identical between the
old and new kernels on both GPUs, and identical between the GPUs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
refactor: rename identifiers that differed from a neighbour only by case
All checks were successful
ci / test_and_build (pull_request) Successful in 31s
ed94cd5876
Two places used a pair of names in one scope that differ only in
capitalisation, which reads as one name:

- kvcache.Manager.matchBlocks held the block size in B next to a block
  pointer b. B is now size.
- The grouped NVFP4 GEMV kernel and its launcher read the rows field of
  gllm_nvfp4_group beside the ROWS template parameter. The field is now
  nrows, in the header, the kernel source and the Go caller that fills
  the struct.

No behaviour changes: the PTX nvcc generates for sm_86 and sm_120 is
identical before and after, and the kvcache tests pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Collaborator

Automated review by pr-reviewer v0.52.3 | Safety Check | Ministral 3 Instruct | tracking id r-be1f8e-c768ea
This is an AI-generated review and may contain mistakes.

Status: ❌ Failed


This review couldn't be completed: that model isn't loaded on the inference service right now, and no alternate model was able to review it either. Consider splitting this PR into smaller changes. Tracking id r-be1f8e-c768ea.

Comment @pr-reviewer-bot retry to try again.

<!-- pr-reviewer:review --> *Automated review by [pr-reviewer](https://git.brooktrails.org/brooktrails/pr-reviewer) v0.52.3 | Safety Check | Ministral 3 Instruct | tracking id `r-be1f8e-c768ea`* *This is an AI-generated review and may contain mistakes.* **Status:** ❌ Failed --- This review couldn't be completed: that model isn't loaded on the inference service right now, and no alternate model was able to review it either. Consider splitting this PR into smaller changes. Tracking id `r-be1f8e-c768ea`. Comment `@pr-reviewer-bot retry` to try again.
rcsheets deleted branch style/case-only-names 2026-10-01 09:00:16 +00:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
brooktrails/gllm!103
No description provided.