docs: energy metering at inference time #120

Merged
rcsheets merged 1 commit from docs/energy-metering into main 2026-10-09 23:59:24 +00:00
Owner

Adds docs/energy-metering.md, a self-contained design document for how gllm measures and bills energy at inference time.

It covers:

  • Goals and non-goals. Energy is measured from hardware counters, not estimated. It is billed the same two ways as compute time, conserved across requests, free for the step loop, and omitted (never 0 J) when unmetered. Marginal energy, power control and whole-node energy are out of scope.
  • The constraint. NVML's board energy counter updates in ~2 J steps every ~100 ms and costs ~1.5 ms per read, which rules out reading it around each ~20 ms decode step.
  • Design. The energy.Meter interface, the 25 ms sampler and interpolating Timeline, per-step billing over the model.Forward window, the single extrapolation estimate, and the batch vs. proportional per-request split.
  • Metrics. gllm_*_energy_joules_total vs. ..._attributed_joules_total, what the gap between them means, and the perfstats energy fields.
  • NVML integration. Runtime dlopen, an optional energy call, and device binding by UUID.
  • CPU energy. The existing Options.CPUEnergy slot and the planned privileged RAPL helper.
  • Validation and interpretation. Results recorded when the feature landed in #96, test coverage, and guidance on reading the numbers.

Documentation only; no code changes.

🤖 Generated with Claude Code

Adds `docs/energy-metering.md`, a self-contained design document for how gllm measures and bills energy at inference time. It covers: - **Goals and non-goals.** Energy is measured from hardware counters, not estimated. It is billed the same two ways as compute time, conserved across requests, free for the step loop, and omitted (never 0 J) when unmetered. Marginal energy, power control and whole-node energy are out of scope. - **The constraint.** NVML's board energy counter updates in ~2 J steps every ~100 ms and costs ~1.5 ms per read, which rules out reading it around each ~20 ms decode step. - **Design.** The `energy.Meter` interface, the 25 ms sampler and interpolating `Timeline`, per-step billing over the `model.Forward` window, the single extrapolation estimate, and the batch vs. proportional per-request split. - **Metrics.** `gllm_*_energy_joules_total` vs. `..._attributed_joules_total`, what the gap between them means, and the perfstats energy fields. - **NVML integration.** Runtime `dlopen`, an optional energy call, and device binding by UUID. - **CPU energy.** The existing `Options.CPUEnergy` slot and the planned privileged RAPL helper. - **Validation and interpretation.** Results recorded when the feature landed in #96, test coverage, and guidance on reading the numbers. Documentation only; no code changes. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
docs: energy metering at inference time
All checks were successful
ci / test_and_build (pull_request) Successful in 50s
471f41b327
A self-contained design doc for how gllm measures and bills energy:
the NVML counter constraints that rule out per-step reads, the
sampler and interpolating timeline, per-step and per-request billing,
the total vs. attributed metrics, NVML binding by UUID, the pending
CPU (RAPL) source, validation results, and how to read the numbers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Collaborator

Automated review by pr-reviewer v0.54.0 | Safety Check | Nemotron 3 Nano | tracking id r-c97fc9-55ebac
This is an AI-generated review and may contain mistakes.

Status: ❌ Failed


Review failed. Tracking id r-c97fc9-55ebac — see logs for details.

Comment @pr-reviewer-bot retry to try again.

<!-- pr-reviewer:review --> *Automated review by [pr-reviewer](https://git.brooktrails.org/brooktrails/pr-reviewer) v0.54.0 | Safety Check | Nemotron 3 Nano | tracking id `r-c97fc9-55ebac`* *This is an AI-generated review and may contain mistakes.* **Status:** ❌ Failed --- Review failed. Tracking id `r-c97fc9-55ebac` — see logs for details. Comment `@pr-reviewer-bot retry` to try again.
rcsheets deleted branch docs/energy-metering 2026-10-09 23:59:24 +00:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
brooktrails/gllm!120
No description provided.