Quickstart
Point your SDK at the host, attach a policy, and read the trace it returns.
Four steps from your existing SDK to a traced routing decision. No model name is hardcoded; the decision travels with the request as a policy.
-
Point your SDK at the host.unhardcoded is OpenAI-compatible. Change the
baseURLto your host; everything else in the SDK stays the same.import OpenAI from "openai"; const client = new OpenAI({ baseURL: "https://<your-host>/v1", // your reference host, or the managed host apiKey: process.env.UNHARDCODED_KEY, }); -
Build a policy and send it with the call.Generate a
policy_irin your backend and attach it tocreate(). Themodelfield is a free-form label used only to group traces; routing comes from the attached policy, so any string works.// built in your backend, at request time, a plain JSON term const policy_ir = [ "policy", ["and", ["meets_req"], ["not", ["is", "disabled"]], ["has_cap", "supports_tools"], ["cmp", "bench_intelligence", "ge", 0.5]], // filter ["neg", ["normalize", ["field", "price_out"]]], // cheapest survivor ["argmax"], ["id"], ["always", { "action": "next_candidate" }], ]; const res = await client.chat.completions.create({ model: "policy:support", // free-form trace label, not a route policy_ir, messages, }); -
The host routes the call.It validates and fingerprints the policy, evaluates it over the live catalog, and runs the cheapest model that passes over your own provider keys.
-
Read the trace.Alongside the completion, the response carries a
trace: the chosen model, the candidates it ranked and rejected, and the policy fingerprint.{ "chosen": { "model_family": "deepseek-v4-pro", "price_out": 1.5 }, "trace": { "policy_fingerprint": "301140696-1054914287", "rejected": [{ "model_family": "deepseek-v4-flash", "reason": "cmp bench_intelligence ge 0.5" }], "total_latency_ms": 425 } }
Running it. The MIT-licensed reference host and runnable examples ship with the public repo. The format and semantics documented below are the spec that host implements, so anything here is reproducible against your own catalog.