brooktrails/slp
Watch
2
0
Fork
You've already forked slp
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat: SLP implementation by Mistral #1

Merged
rcsheets merged 32 commits from feat/slp-implementation into main 2026-07-15 06:23:54 +00:00
Conversation 0 Commits 32 Files changed 6 +751
rcsheets commented 2026-07-14 23:24:00 +00:00
Owner
Copy link
No description provided. 
Implements all core features from the README:
- config load: parse backends + editions from YAML
- config compile: expand editions into flat route table with validation
  (fail closed on collisions, unknown backends, bad templates, editions with retry)
- OpenAI-compatible surface: GET /v1/models, POST /v1/chat/completions,
  /v1/completions, /v1/embeddings routed by model id
- reverse proxy with FlushInterval=-1 behavior via copyAndFlush
- per-route response-header timeout via http.Client.Timeout
- opt-in single-shot retry on pre-response transport error (raw routes only)
- request-body model extraction via JSON unmarshal
- startup route dump to logs; /healthz endpoint
- Prometheus metrics: requests total, upstream latency, bytes in/out by model id

Project structure:
- cmd/slp/main.go: server binary with CLI flags
- internal/config/config.go: config schema, loading, edition expansion
- internal/proxy/proxy.go: router, reverse proxy, endpoints
- internal/proxy/metrics.go: Prometheus metrics

Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
The README specifies that timeout is time-to-first-byte (response-header
timeout), not a ceiling on the total exchange. http.Client.Timeout is a
total-request deadline, so we need to use http.Transport.ResponseHeaderTimeout
instead to properly implement the per-route response-header timeout.

Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
Per README design, each route should have its own transport built once.
The Transport with ResponseHeaderTimeout is now constructed during route
compilation in main.go and stored on the proxy.Route struct, rather than
being created on every request.

Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
# Conflicts:
#	internal/proxy/proxy.go
- Add signal handling for SIGINT/SIGTERM
- Run server in goroutine
- Pass context to Shutdown method (was passing nil which would panic)
- Add 30-second grace period for shutdown

Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
Change requestDuration histogram buckets from DefBuckets to
ExponentialBuckets(0.1, 2, 14) for better latency distribution.

Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
Use separate RecordRequestError function for failed requests that
only records the counter and request bytes, skipping the latency
observation. This prevents upstream errors from polluting the
requestDuration histogram with 0-second values.

Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
- Create a new http.Request for each retry attempt instead of reusing
  the same request object, which Go doesn't permit after a failed call.
- bytes.Reader automatically populates GetBody for transport-level replays.
- Removed duplicate request creation outside the retry loop.
- Retry only fires on pre-response transport errors (when client.Do
  returns an error, which per README means before any bytes are written).

Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
- Filter out hop-by-hop headers (Connection, Keep-Alive, Transfer-Encoding, etc.)
  when forwarding requests, per RFC 7230. These headers are not meaningful
  for the upstream connection.
- Use url.ResolveReference to properly join backend URL with request path and
  query string, avoiding double slashes when backend URL has trailing slash.

Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
Sort model IDs alphabetically before returning in the /v1/models
response to ensure deterministic output across runs.

Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
Wrap r.Body with http.MaxBytesReader to limit request body size to
10MB, preventing unbounded memory usage from malicious or misconfigured
clients. http.MaxBytesReader handles the 413 response automatically.

Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
Add validation in Load() to ensure listen address is set, preventing
ListenAndServe("") from silently listening on port 80 when config is
missing the listen field and no --listen flag is provided.

Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
- config.Compile() now returns proxy.RouteTable directly with pre-built
  transports, eliminating the field-for-field copy in main.go
- config package imports proxy and creates proxy.Route values internally
- Removed redundant config.Route type

Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
- Remove DisableKeepAlives: true from transport config
- Set MaxIdleConnsPerHost: 2 to limit idle connections per host
- Allows connection reuse while preventing resource exhaustion

Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
Apply the same hop-by-hop header filter to response headers being
copied from upstream to client. This prevents upstream Connection,
Transfer-Encoding, etc. headers from confusing Go's server framing.

Made hopByHopHeaders a package-level constant for reuse.

Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
Collect and sort model IDs before logging routes to ensure
deterministic output across runs.

Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
- config.Compile() returns map[string]config.Route (data only)
- proxy.NewRouteTable() creates RouteTable with pre-built transports
- config no longer imports proxy or net/http
- Eliminates backwards dependency and potential cycles

Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
rcsheets deleted branch feat/slp-implementation 2026-07-15 06:23:55 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
brooktrails/slp!1
No description provided.