SLP - Simple LLM Proxy

An LLM Proxy that is not very smart. Presents one OpenAI-compatible surface in front of several OpenAI-compatible backends, routes each request to exactly one of them by model ID, and is otherwise too dumb to touch the bytes in flight.

SLP is copydesk's counterpart on the same shelf: copydesk is the editor who reads every line before it prints; SLP is the runner who carries the copy to the right desk and is constitutionally incapable of reading it. Every feature it does not have is a class of bug that cannot corrupt a response.

It exists because routing on the model field means looking inside the request body, which Caddy and nginx will not do without a custom module -- and once a module is parsing the body it has stopped being config and started being a program. SLP is that program, kept as small as the job allows. Put it behind Caddy: TLS, HTTP/2, and ACME stay Caddy's job; body-aware routing is SLP's.

What a request does

client ── POST /v1/chat/completions {"model":"mistral-7b-edited"} ──> SLP
  1. read     buffer the (small) request body
  2. look up  model id -> upstream, from the compiled route table
  3. rename   if the route declares one, swap in the upstream's own model
              name -- the only field SLP ever writes (see "Model names")
  4. forward  reverse-proxy to that upstream, byte for byte
              (FlushInterval = -1: SSE frames and reasoning_content pass
               through untouched; SLP never parses the response)

No pools, no failover. GET /v1/models is answered from the route table directly -- SLP never fans out on the request path, so a slow upstream (copydesk mid-verification) cannot stall the catalog. The one place SLP talks to a backend unprompted is model-name discovery, which runs on its own schedule in the background and never blocks a request or the catalog.

Routing is by exact model id

One model id maps to one upstream. SLP does not alias, group, or load-balance ids. This is deliberate: mistral-7b and mistral-7b-edited are different products with different latency contracts -- the whole point of the edited path is that it is not interchangeable with the raw one -- so they stay distinct routes to distinct upstreams. A backend that wants two dials behind one id is a smarter program's job.

Model names: the exposed id is not always the upstream's id

The id SLP exposes is a product name you choose. The id a backend answers to is whatever it was loaded with, which is frequently a build artifact:

{"error":{"message":"model \"mistral-small-4\" does not exist;
                     this server serves \"Mistral-Small-4-119B-2603-NVFP4\""}}

A client that reads GET /v1/models and echoes back what it saw will hit this every time, so the two names have to be reconciled somewhere. Renaming the route to match the backend leaks a quantization format into the catalog and breaks the moment the weights are rebuilt. SLP reconciles it instead, by asking the backend what it calls itself.

Each raw backend gets a supervisor goroutine that probes its /v1/models. If the backend serves exactly one model -- the normal case for a single-model vLLM or gllm -- that name becomes the route's rewrite rule, and the model field is swapped on the way past. Nothing else in the body is touched.

client sends  {"model":"mistral-small-4", ...}
upstream gets {"model":"Mistral-Small-4-119B-2603-NVFP4", ...}

Discovery never gates startup. The supervisors run alongside the server, not before it. A backend that is down, or a vLLM still loading a 119B model, is retried indefinitely (1s backing off to 30s) without SLP waiting on it; the route simply forwards unchanged until a name is known, which is exactly what it would do if discovery did not exist. A backend outage cannot become an SLP outage, and boot order in the cluster stops mattering.

Once a name is known the supervisor backs off to a five-minute check and is only watching for the name to change. When a weights rebuild renames the model under a running SLP, the rewrite rule follows it -- no restart, no config edit.

Ambiguity never guesses. If a backend serves zero or several models, SLP cannot know which one the route means, so it keeps the last known good name (or forwards unchanged if it never had one) and says so loudly:

discovery: WARNING: mistral-small-4: backend serves 2 models [model-a model-b],
  expected exactly 1; keeping last known name "Mistral-Small-4-119B-2603-NVFP4"
  -- requests may be routed to a stale model

with slp_upstream_model_ambiguous{model_id="..."} going to 1 for the duration. That gauge is the one to alert on: it is the only state where SLP keeps serving using a name it can no longer confirm. Also exported: slp_discovery_probes_total{model_id,result} (ok/ambiguous/error), slp_upstream_model_changes_total, and slp_upstream_model_info, which carries the live mapping in its labels.

Pinning a name

An explicit model: on a backend overrides discovery and skips probing entirely -- for a multi-model backend that discovery structurally cannot disambiguate, an upstream that does not serve /v1/models, or a name you do not want drifting:

backends:
  - name: mistral-small-4                      # what clients see and send
    url: http://vllm.c2-talos:8000             # discovered; nothing to declare
    timeout: 60s
  - name: llama-70b
    url: http://gllm-2.c2-talos:8000
    model: Llama-3.3-70B-Instruct-AWQ          # pinned; never probed
    timeout: 120s

Startup logs show which routes are pinned and which are still resolving:

llama-70b       -> http://gllm-2.c2-talos:8000 as "Llama-3.3-70B-Instruct-AWQ"
mistral-small-4 -> http://vllm.c2-talos:8000 (discovering)

Editions are never probed. A copydesk instance is launched as --serve-as mistral-small-4-edited and expects the exposed id, so editions forward it unchanged unless they declare their own model: template (over .Name, .URL, .Model) -- useful for a pass-through wrapper that hands the body to the real backend.

Asking what an id resolves to: GET /v1/models?debug

Startup logs and slp_upstream_model_info both carry the live mapping, but neither is at hand when a client is being told its model does not exist. ?debug answers from the same route table the plain catalog is built from, still without touching a backend:

{"object": "list", "data": [
  {"id": "mistral-small-4",        "upstream_model": "Mistral-Small-4-119B-2603-NVFP4",
   "discovery": "discovered", "last_probe": "ok",
   "last_probe_at": "2026-07-27T09:14:02Z",
   "timeout": "1m0s", "retry": false, "aggregate": false},
  {"id": "llama-70b",              "discovery": "pending",
   "timeout": "2m0s", "retry": true,  "aggregate": false},
  {"id": "qwen-32b",               "upstream_model": "Qwen3-32B-AWQ",
   "discovery": "pinned",
   "timeout": "1m30s", "retry": false, "aggregate": false},
  {"id": "mistral-small-4-edited", "discovery": "passthrough",
   "timeout": "5m0s", "retry": false, "aggregate": true}
]}

discovery says where the name came from, which is the fast way to read a missing upstream_model: discovered (probed and current), pending (still probing, forwarding the exposed id meanwhile -- or, on a gated route, refusing), pinned (declared in config, never probed), passthrough (an edition, serving the exposed id), unloaded (model-gated, and not the model the backend is serving -- see expect). Gated routes, and editions of them, also carry loaded; gated backends carry their expect pattern. ?debug, ?debug=1, and ?debug=yes all work; only an explicit false or 0 is a no.

The upstream URL is not in there. /v1/models is unauthenticated -- it is the one endpoint a client is supposed to read -- and naming backend hosts would undo the reticence the error path already keeps, where a dial failure tells the log the host, port, and resolved IP and tells the client "upstream request failed". Probe failures are the same: last_probe says error, and the log says which address refused. The debug catalog answers "what does SLP send as the model name", not "where does SLP send it".

Without the parameter the response is unchanged -- ids and nothing else, so an OpenAI client parses it exactly as before.

Several models, one GPU: expect

Trying models on a single GPU means one backend URL that serves a different model depending on what was loaded last. Declared as one backend per model, discovery alone gets this wrong in the worst way: every backend on that URL probes the same /v1/models, sees the one loaded model, and adopts its name. A request for qwen-32b is rewritten to whatever Mistral is loaded and comes back 200.

expect: makes a backend model-gated. It is a glob (Go path.Match syntax) naming the upstream model the backend is for. Matching is case-sensitive, like the upstream names themselves; spell out a character class ("[Qq]wen3-32[Bb]*") if a pattern should accept either case:

backends:
  - name: qwen-32b
    url: http://gpu0:8000
    expect: "Qwen3-32B*"
  - name: mistral-small-4
    url: http://gpu0:8000
    expect: "Mistral-Small-4-*"

A gated route is probed like any discovered route, but serves only while exactly one model the backend reports matches its pattern -- and rewrites to that name. Otherwise it refuses, before anything reaches the backend:

HTTP/1.1 503 Service Unavailable
Retry-After: 15

model "mistral-small-4" is not currently loaded on its backend (request id: slpq...)

The client learns that its model is not loaded, not which one is; ?debug is where to look that up. Editions of a gated backend are gated with it -- the filter in front of an unloaded model has nothing to filter.

Probes decide whether a route serves, never what the route is. The route table is still exactly what config compiled: no route is added, removed, or repointed, and nothing takes the place of a model that is not loaded. A refused request is refused, not rerouted.

Swaps are noticed quickly. A gated route is rechecked every 15 seconds rather than every five minutes. A refused request, or a 404 from a backend SLP believed was serving the model, also asks for an immediate recheck -- in the background; the request itself never waits on a probe. And a route that sees its model swapped in or out prompts the other gated routes on the same URL to look too, since one of them was probably just loaded. Rechecks prompted by requests are held to one a second per route, so a client retrying in a tight loop cannot turn into a probe per request.

A failed probe changes nothing. A backend restarting mid-swap keeps each route's last known state. That is safe because a gated route rewrites to the exact name it last saw: a backend that has meanwhile loaded something else rejects that name rather than answering as the wrong model.

expect also resolves a multi-model backend that plain discovery reports as ambiguous: several models served is fine so long as exactly one matches. Two matches is still ambiguous, warned about loudly, and the fix is a narrower pattern.

None of this happens without expect. A backend without it behaves exactly as before. Several ungated backends sharing a URL is allowed -- it is how you spell "whatever is loaded right now" -- but SLP logs a startup warning naming them, since it is more often a missing expect than a choice. expect and model: together are a startup error: a pinned name is never probed, so there is nothing to match.

Exported alongside the discovery metrics: slp_upstream_model_loaded{model_id} (1 serving, 0 refusing, for gated routes only), slp_discovery_probes_total{result="unloaded"}, and slp_requests_total{status="not_loaded"} for refusals.

Telling clients what is loaded: X-SLP-Catalog: v1

The plain GET /v1/models lists every configured id whether or not it is loaded -- the catalog is the route table, and gating does not change it. A client that wants to know which ids will be served right now can opt in to a versioned extension by sending X-SLP-Catalog: v1:

{"object": "list", "data": [
  {"id": "llama-70b",       "slp": {"loaded": null}},
  {"id": "mistral-small-4", "slp": {"loaded": false}},
  {"id": "qwen-32b",        "slp": {"loaded": true}}
]}

loaded is true or false on a gated route (false until its first probe confirms the model, since requests are refused until then) and null on an ungated one, which always forwards. SLP's fields live under one slp key so they cannot collide with anything OpenAI adds.

The response echoes X-SLP-Catalog: v1 only when it served that version, so the echo is the client's confirmation. A version SLP does not know gets the plain catalog, no echo, and a fixed warning -- the requested value is never reflected, since the client knows what it sent:

X-SLP-Catalog-Warning: unsupported version supplied in request; supported: v1

Every /v1/models response carries Vary: X-SLP-Catalog. Unlike ?debug, which is a diagnostic view free to change, the extension is a contract; a change to its shape is a new version. ?debug takes precedence when both are present.

Editions: declare the transform once

The one place SLP is allowed to be clever is config, and only at load time. If you have three backends and want a copyedited variant of each, you should not have to write the same upstream three times.

An edition is a derived model id (base name + suffix) pointing at a derived upstream (a URL template over the backend's fields). It compiles, before the server starts, into ordinary flat routes. The runtime never sees an edition -- only the expanded table.

listen: ":8080"
log_requests: false      # true logs every forwarded request, not just failures

backends:
  - name: mistral-7b
    url: http://gllm.c2-talos:8000        # origin only; SLP forwards the request path verbatim
    timeout: 60s
  - name: mistral-small
    url: http://vllm.c2-talos:8000
    timeout: 60s
  - name: llama-70b
    url: http://gllm-2.c2-talos:8000
    timeout: 120s

editions:
  - suffix: -edited
    url: http://copydesk-{{.Name}}.c2-talos:8100   # template over backend fields: .Name, .URL
    timeout: 300s
    retry: false
    apply_to: ["*"]        # or a subset: [mistral-7b, llama-70b]

That compiles to six routes:

mistral-7b            -> http://gllm.c2-talos:8000
mistral-small         -> http://vllm.c2-talos:8000
llama-70b             -> http://gllm-2.c2-talos:8000
mistral-7b-edited     -> http://copydesk-mistral-7b.c2-talos:8100
mistral-small-edited  -> http://copydesk-mistral-small.c2-talos:8100
llama-70b-edited      -> http://copydesk-llama-70b.c2-talos:8100

The exposed id (mistral-7b-edited) is exactly what the corresponding copydesk instance is launched with (--serve-as mistral-7b-edited), so the two agree by construction. SLP knows nothing about copydesk specifically; an edition is generic, and copydesk is just its first consumer. A -json edition through some grammar-constrained wrapper would be the same shape.

Compilation fails closed. Duplicate model ids, an edition naming a backend that does not exist, or a template that does not resolve are startup errors, not runtime surprises. SLP either serves a fully-resolved table or refuses to start; there is no third state.

timeout is time-to-first-byte, not a ceiling

Per-route timeout is the response-header timeout -- how long SLP waits for the upstream to begin responding -- not a cap on the whole exchange. copydesk can spend five minutes on retrieval and verification, and a streaming backend can stream for as long as it likes; SLP only gives up if nothing has come back at all. This is why editions get a generous value.

When a route does time out, the log and the 502 body both name the length that expired, so you know which timeout: to raise without going back to config:

[slpq2ff15ba63230a0d1ddec5efe] Attempt 1/1 for model llama-70b failed: timed out after 120s: Post "http://gllm-2.c2-talos:8000/v1/chat/completions": net/http: timeout awaiting response headers

aggregate: streaming upstream on the client's behalf

timeout being time-to-first-byte has one blind spot: a non-streaming completion sends no bytes at all -- headers included -- until the last token is generated. On a slow model the whole generation happens inside the response-header timeout, and the knob stops working: sized for the worst-case generation it no longer detects a hung backend, sized as a liveness check it kills legitimate requests.

aggregate: true on a route closes the gap by making the upstream request a streaming one regardless of what the client asked for. When the client wanted plain JSON, SLP drains the SSE stream itself and reassembles the non-streaming response: deltas concatenated (content, and a reasoning model's reasoning_content beside it), usage, finish_reason, and gllm metadata carried over from the final chunk. Upstream headers now arrive at time-to-first-token, so timeout is a liveness check again and generation may take as long as it takes.

backends:
  - name: mistral-small-4
    url: http://newton.c2.brooktrails.us:8000
    timeout: 120s        # now bounds time-to-first-token, not the generation
    aggregate: true

This is SLP's one deliberate exception to never touching response bytes, which is why it is opt-in per route. The edges are handled conservatively:

• A client that asked to stream is untouched -- pass-through as always.
• Requests in the logprobs family pass through unconverted (gllm rejects logprobs with streaming); they keep the whole-generation timeout exposure, acceptable for a rare debugging shape.
• An upstream error status, or a backend that ignores stream: true, passes through byte for byte.
• A late in-band error event -- streaming commits its 200 before generation finishes, so failures arrive as an event -- is handed back as a 502 with the upstream's error object. A plain pass-through stream cannot do that; the aggregator can because it has written nothing to the client yet.
• A stream that dies before data: [DONE] is a 502, never a silently truncated 200.

usage in the reassembled response depends on the upstream putting usage on its final chunk, as gllm does. An upstream that never sends it in-stream yields responses without a usage field.

aggregate on an edition is its own declaration, never inherited from the backend: the edition's upstream is a different program with its own streaming behavior.

logging forwarded requests

Failures are always logged. Successful forwards are not, since that is a line per completion -- turn them on with log_requests: true at the top level of the config, or --log-requests for a one-off run:

[slpq2ff15ba63230a0d1ddec5efe] POST /v1/chat/completions: mistral-small-4 as "Mistral-Small-4-119B-2603-NVFP4" -> http://vllm.c2-talos:8000 200 in 1.482s (412 B up, 8317 B down) upstream=gdqz3f8a1c04b7e25d918a6f2c3b

The latency is time to first byte, the same measure as slp_upstream_latency_seconds -- for a streaming response it is when the stream started, not when it finished.

errors: the log gets the detail, the client gets the fact

A transport failure names the upstream's hostname, port, and resolved IP:

Post "http://gllm-2.c2-talos:8000/v1/chat/completions": dial tcp 10.69.60.58:8000: connect: connection refused

That is the route table talking, and the route table is SLP's business. A client behind Caddy generally cannot reach those addresses, cannot act on them, and did not ask which desk the copy went to. So the description above goes to the log, and the 502 body says only:

upstream request failed (request id: slpq2ff15ba63230a0d1ddec5efe)

Timeouts are the exception: the 502 body names the length that expired (upstream request timed out after 120s), because that is SLP's own configuration rather than backend topology, and it is the one detail here a caller can act on -- wait longer, or send less.

request ids

Every response carries X-Request-Id, and every log line for that request is prefixed with it. Ids match the shape gllm uses -- a four-letter prefix that never occurs in the hex body, then 12 random bytes -- so one expression finds either: (gdqz|slpq)[0-9a-f]{24}.

The prefix says who minted it, because that says how far the request got:

id	meaning
gdqz...	gllm's own id. It accepted the request; its logs have the rest.
slpq...	SLP's id, surfaced only when there is no upstream response. The request never reached the backend.

On a successful forward the upstream's X-Request-Id overwrites SLP's in the header copy, so a gdqz id reaches the client whenever gllm actually saw the request. log_requests records it as upstream=gdqz..., which is what makes an SLP log line joinable to a gllm log line for the same request.

SLP never adopts an inbound X-Request-Id. gllm ignores the request header too -- it always mints its own -- so honouring one would buy no correlation across the hop, and it would put an untrusted client string into the log. It is not dropped silently, though: the value is recorded on each request-scoped log line as client="...", and the first one seen logs a line saying it was ignored and why. Inbound values are truncated and always logged quoted, so a caller cannot forge log lines with a newline.

retry

Off by default, and copydesk editions should leave it off -- an edition with retry: true may be undesirable in many cases, since a retry on a filtered route not only adds latency; it re-runs the filter. In the case of copydesk this is a whole retrieve-and-verify pass, doubling the writer/verifier spend. On raw routes, retry: true permits a single re-dial on a transport-level error before any bytes are written. SLP never retries once the response has begun, and never retries on an HTTP status. That is the entire retry story.

The dashboard: /_slp/

/metrics is for Prometheus and ?debug is for curl. /_slp/ is for a person: one page that says what SLP is doing, how it is configured, and what it has been up to lately.

• Now: how many routes are serving, how many requests are in flight, and per route whether it is serving or refusing, the model name it sends upstream and where that name came from, and how its last discovery probe went.
• Configured: each route's timeout, retry, aggregate, and expect, whether it is a backend or an edition of one, plus the listen address, version, uptime, and logging setting.
• Lately: requests per minute over the last hour (served, turned away, failed), the last 50 requests with their result, time to first byte, total time, sizes, and request ids, and the last 30 discovery events -- a model loaded or swapped out, a name changed, probes failing or recovering.

The page refreshes itself every five seconds, holds still while you have text selected, and works as a plain snapshot with scripts off.

It is on by default, and served only to clients on the RFC 1918 private networks:

dashboard:
  enabled: true            # false turns it off; /_slp/ is then a 404
  debug: false             # true keeps and shows request and response bodies; see below
  allow:                   # CIDRs or bare addresses; this is the default
    - 10.0.0.0/8
    - 172.16.0.0/12
    - 192.168.0.0/16

allow replaces the default rather than adding to it. Loopback is not in the default, so curl localhost:8080/_slp/ on the SLP host is a 404 until you add 127.0.0.0/8 -- deliberately, because a reverse proxy on the same host makes every client on the internet look like 127.0.0.1. Add 100.64.0.0/10 for Tailscale, or an IPv6 prefix; a list that does not parse is a startup error.

Behind a proxy, the forwarded client has to be allowed too. SLP is meant to sit behind Caddy, and from there every request arrives from Caddy's private address. So the connecting address must be in allow, and so must every address in X-Forwarded-For, X-Real-IP, and Forwarded. Those headers are only ever used to refuse: a client can add entries to them but cannot remove the one its proxy appends, so forging them gains nothing, and one that does not parse is a refusal. This holds for a proxy that sets one of those headers, as Caddy and nginx's usual configuration do. A proxy that forwards without naming the client leaves SLP nothing to check, and should either not route /_slp/ or have the dashboard turned off.

A client that is not allowed gets the same 404 as an SLP with no dashboard. The log says why, at most once a minute:

dashboard: refused a request from 10.0.0.2, forwarding for X-Forwarded-For "203.0.113.9": not in dashboard.allow [10.0.0.0/8 172.16.0.0/12 192.168.0.0/16]; answered 404.

What the dashboard does not show

The allow list is a network boundary, not authentication, so the page is built to have nothing on it worth more than that -- unless you turn on debug mode, which is off by default:

• No backend addresses. The same rule as ?debug and the error path: model names yes, hosts, ports, and URLs no.
• Nothing a client sent. No client addresses, headers, or bodies, no client-supplied X-Request-Id, and not the model name of a request that matched no route -- that is shown as "unknown model" and counted.
• No error text. A failed request is "backend unreachable" or "backend timed out"; the transport error, which names the upstream, stays in the log. The request id on the page is how you find it there.

What it does show beyond ?debug is traffic: counts, timings, sizes, and SLP's and the backend's own request ids. It is read-only -- GET only, nothing on it changes anything -- and it is served under a content security policy that allows no inline script or style.

The history behind it is a few fixed-size buffers in memory, kept only while the dashboard is enabled and gone on restart. Prometheus is still the record.

Debug mode: the request and the response themselves

Everything above holds with debug mode off, which is the default. Turning it on is the deliberate exception, for when "backend unreachable" and a request id are not enough:

dashboard:
  debug: true

or --dashboard-debug for a one-off run -- turn-on only, like --log-requests. With it on, each request id in the recent requests table is a link to a page for that request:

• What the client sent: the request line, its headers, its body, and the address it came from.
• What SLP did with it: the upstream URL it was sent to, whether the model field was rewritten and to what, whether stream was forced for aggregation, and the transport error of each attempt that failed.
• What the client got back: status, headers, and body as SLP wrote them -- the backend's response passed through, SLP's reassembly of an aggregated stream, or SLP's own error.

An id is a link only when there is a page behind it. With debug off there are none; a request whose body could not be read has no id to link. A request that has aged out of the last 50, or predates a restart, answers its old link with a 404 that says so.

With debug on, anyone who can open the dashboard can read prompts and completions. The allow list is still the only gate, and it is still a network boundary rather than a login -- so debug mode is for a network you would hand those to, for as long as you are looking, and both the dashboard and the startup log say when it is on. Backend addresses appear on these pages too.

Credentials are kept out regardless. A header whose name looks like one (Authorization, Cookie, Set-Cookie, anything with token, secret, password, credential, or api-key in it) is stored as [redacted, 21 bytes] -- redacted when captured, not when shown, so the value is never in memory to leak. Bodies are not redacted; a key pasted into a prompt is a prompt.

Recording never changes a response: the bytes, status, headers, and flushing the client sees are the same with debug on or off. It costs memory instead -- up to 128 KiB of each body is kept, for the last 50 requests, so about 13 MB at worst. A longer body is shown truncated and says so. A whole JSON body is indented for reading, which moves whitespace and nothing else. With debug off none of this runs: no body is copied and no response is wrapped.

Status

Roughly in dependency order:

• config load: parse backends + editions
• config compile: expand editions into a flat route table; fail closed on collisions / unknown backends / bad templates / editions with retry
• OpenAI-compatible surface: GET /v1/models from the table (?debug adds the upstream model name and route state per id, never the upstream URL); POST /v1/chat/completions (and /v1/completions, /v1/embeddings) routed by model id
• reverse proxy with FlushInterval = -1 (verbatim streaming, reasoning_content preserved)
• per-route response-header timeout via the upstream transport
• opt-in single-shot retry on pre-response transport error (raw routes only)
• opt-in stream aggregation (aggregate: true): non-streaming completions converted to upstream streaming requests and reassembled, so the route timeout bounds time-to-first-token; in-band stream errors surface as real error statuses
• request-body model extraction (small struct decode, not a full parse)
• startup route dump to logs; /healthz
• Prometheus metrics: requests and upstream latency by model id
• upstream model-name discovery: per-backend supervisor goroutine, infinite retry until found, slow drift monitoring after, explicit model: to pin; loud logs and metrics when a backend goes ambiguous
• model-gated routes (expect:): serve only while the expected model is loaded, 503 otherwise; fast swap detection via request and sibling nudges; opt-in X-SLP-Catalog: v1 loaded status
• read-only dashboard at /_slp/: route state, configuration, and recent requests and discovery events; on by default, RFC 1918 clients only, never backend addresses or anything a client sent
• opt-in dashboard debug mode (dashboard.debug): a page per recent request with the request, where it was sent, and the response, linked from its id; credentials in headers redacted at capture

Layout

cmd/slp/          server binary
internal/config/    file schema, load, edition expansion, validation
internal/proxy/     model-id router, reverse proxy, /v1/models, /_slp/ dashboard

Design decisions

• Dumb at runtime, by construction. The request path is read-model-id, look-up, forward. No response parsing means no response corruption: SLP's worst case is that it fails to forward, never that it mangles something mid-flight.
• Smart config compiles to a dumb table. All expansion happens once, at load. The runtime's whole world is map[modelId]upstream. Discovery is the single exception: it updates one field of one route, atomically, and cannot add, remove, or repoint a route. On a model-gated route it also decides whether that route serves -- which can only ever make a route refuse, never send a request somewhere config did not.
• Stateless. Config file in, routes out. No database, no cache, no admin surface. (Nothing here is stateful, so nothing here is Postgres.) The discovered model names are the only runtime state routing depends on, they are derived rather than authoritative, and losing them on restart costs one probe. The dashboard's recent history is a window, not a record: nothing reads it but the page, and a restart empties it.
• Discovery degrades to the old behavior, never to a worse one. A route with no name yet forwards the body exactly as SLP did before discovery existed. Every failure mode -- backend down, catalog unparseable, several models served -- lands on pass-through or on the last known good name, and none of them can stop SLP from starting or serving. A model-gated route refuses instead of passing through, because on a shared backend pass-through is the misroute; that is opted into per backend with expect, and a backend without it keeps the old behavior exactly.
• Separate blast radius from everything it fronts. SLP falling over must not take a backend down; it holds no state a backend depends on, and it degrades to "not running," never to "running wrong."
• Behind Caddy, not replacing it. TLS, ACME, and HTTP/2 stay upstream. SLP is only the body-aware routing Caddy structurally cannot do.

Non-goals

• Not an inference engine. Tokens belong to gllm/vLLM.
• Not a load balancer. One id, one upstream. No pools, no groups, no health-checking backends in or out of a rotation. expect sits right at this line without crossing it: a gated route whose model is not loaded is refused, and nothing is substituted for it. There is no rotation to be in or out of.
• Not auth or metering. No keys, no budgets, no tenancy, no admin UI -- the dashboard shows and never sets, and its allow list is a network boundary, not a login. When you need per-key spend tracking across consumers you do not trust, you have outgrown SLP and want something like LiteLLM; SLP should say so rather than grow toward it.
• Not a guardrails framework. No policy engine, no request inspection beyond reading the model id.
• Not a response processor. Anything that reads or edits a response is the filter's job. SLP forwards; it does not review.

Prior art

LiteLLM is the feature-complete version of this idea and the thing to reach for once the tenancy and provider-adapter machinery earns its weight; SLP is what is left when every backend is already OpenAI-compatible and you want the router to be invisible. Caddy and nginx cannot route on a JSON body field without a custom module, which is the gap SLP fills. Bifrost sits in the same "fast Go gateway" niche with more features; SLP trades them away on purpose.

Running

slp --config slp.yaml