Loading measured results…
An open, 421M-parameter Laya decision model reads each LLM request and picks the cheapest model tier that will still get it right — in milliseconds. Its output distribution is a policy, so we train it with RL on real outcomes and you can watch it learn. One base_url change routes your agents through it, lowering Alan's dollars per merged PR.
Watch reinforcement learning reshape the policy
Every point is a prompt, placed by what the frozen base model thinks it means. Colour is the tier the policy picks; size is its confidence. Press play to step through training checkpoints and watch regions flip from expensive to cheap as the reward comes in.
Can you pick the cheapest model that gets it right?
Real held-out prompts, real graded outcomes from all three models. Laya has seen none of them.
One knob from cheap to careful
Each λ trains a separate policy. Slide it to move along the cost/accuracy curve; the hero animation follows. Baselines are fixed points — including the cheap embedding baselines a reviewer would ask for first.
Route your own request
Point any OpenAI SDK at the proxy. Every request goes to the cheapest tier Laya thinks can handle it; agent sessions stay on one tier, tool calls pass through.
from openai import OpenAI
client = OpenAI(base_url="https://YOUR-DEPLOYMENT/v1", api_key="unused") # ← the one line
r = client.chat.completions.create(
model="auto", # or "auto-cheap" / "auto-quality"
messages=[{"role": "user", "content": "Summarise this diff in one line"}],
)
print(r.choices[0].message.content)Response headers show what happened: x-router-tier, x-router-confidence, x-router-latency-ms, x-estimated-savings-usd.
Request → decision → outcome → reward
A decision model's output is a policy
Laya scores every option at its own [MASK] marker in one non-autoregressive forward pass. A softmax over the three tier options is a full action distribution π(a|s) — no sampling of tokens, no generation. That makes policy-gradient RL cheap and clean: one pass gives log π for every action.
Reward = right answer, minus what it cost
Each training prompt was run on all three tiers and graded automatically (math answers, unit tests, multiple choice, agent bug-fix and CI-triage tasks). RL samples an action, looks up only that arm's outcome — the online bandit setting — and updates with REINFORCE, a running-mean baseline and an entropy bonus. Sweeping λ gives a family of policies: the cost/quality knob.
What it isn't
An offline experiment on benchmark-style prompts with a small test set and one seed. Confidence intervals are shown everywhere; the strongest cheap baseline (per-tier logistic regression on embeddings) is plotted next to it, including where it wins.