Six layers between your agent and the model
What a reliability layer for AI agents actually is, what each of the six does to a request, and why we built it as a gateway you point your agent at instead of a library you import.
An agent is a loop. It reads a goal, asks a model what to do, runs a tool, feeds the result back, and asks again. Most of the ways that loop fails are not about the model being unintelligent. The model answers in text when it should have called a tool. A tool call arrives with a trailing comma. The conversation grows until it no longer fits. The agent calls the same function nine times and burns the budget. A user says in March that they moved, and by June the agent has forgotten.
Each of these is a small, repeatable failure, and each has a small, repeatable fix. Rakuvo is the place where those fixes live, so that you do not rewrite them inside every agent you build.
#A gateway, not a library
Rakuvo sits between your agent and the model provider. You change one setting, the base URL your agent sends requests to, and the requests now pass through six layers on the way out and the answer passes back through them on the way in. It speaks the OpenAI-style endpoints (chat completions and the Responses API), so an agent written against those needs no code change.
We chose a gateway over a library for three reasons.
- It works on any agent. A library only helps code written in one language that imports it. A gateway helps the agent you built, the one a vendor built, and the one a teammate wrote in a language you do not use.
- It can be changed without a deploy. Settings live in a config that the gateway pulls from the control plane. Switch a layer off in the dashboard and gateways pick it up within about 30 seconds, with no restart and no new release of your agent.
- It sees everything. Because every call passes through one place, every call can be traced, costed and attributed to an agent, whether or not that agent was written with observability in mind.
#The path of a request
A request meets the layers in a fixed order.
- Memory. Reads the new user message, extracts facts, and puts the ones that matter to the question in front of the model.
- Context. Keeps the prompt small as a conversation grows.
- Prompt. Adds a short instruction when the request offers tools.
- Model policy. Decides which model answers and how hard it may think.
- Tools. Checks and repairs the tool calls that come back.
- Loop guard. Watches the sequence of calls for an agent going in circles.
Memory
Instead of storing raw text and searching it, memory keeps a temporal graph of facts: subject, predicate, object, the turn it came from, and a date. When a newer fact replaces an older one, the old one is kept as history and marked as no longer true. That is what lets it answer both "where does she live now?" and "where did she live in March?". Memory is kept per user, so one person's facts never reach another's prompt.
Context
Long agent sessions fill up with tool results the model no longer needs. Context replaces old results with one-line placeholders that tell the model how to fetch them again, and leaves the newest results in full. It masks in steps, so the start of the prompt stays stable and the provider's prompt cache keeps working. On a long run that outgrows the model's window it is the difference between finishing and a window error.
Prompt
A short instruction, added only to requests that offer tools: use the tool instead of answering from memory. It is the smallest layer and, on the right models, the one with the largest measured effect.
Model policy
The cheapest-first idea: try a small model, and move to a stronger one only when the small one visibly fails. It can also set how much a reasoning model is allowed to think. We measured this layer live, and it did not pay for itself in the setup we tested, so it is off until you turn it on.
Tools
Every tool call is checked against the tool's own schema. Small, common mistakes (a markdown fence around the JSON, a trailing comma, a number sent as a string) are repaired instead of failing the request. Optionally a small judge model can veto a call that makes no sense for the conversation, for example a tool called when the user asked a general question.
Loop guard
It watches the sequence of tool calls. A repeat, a cycle or a repeated message first earns a nudge in the prompt, and a run that keeps going is stopped with an honest report.
#What "reliable" means here
We do not use the word as a promise. Each layer is tested against a public benchmark with the same questions run with and without it, and we report the change with its interval. Some layers help a lot on some models and do nothing on others, and the product's defaults follow the measurements: the Prompt layer has an automatic mode that applies only to models where we saw a gain.
You choose which layers run. A team that only wants stuck-agent protection turns on one layer and leaves five off, and the trace for every call shows which layers acted and why.
#Where to look next
The dashboard shows the layers as the path a request takes, with an action count for each, so the answer to "what is Rakuvo doing to my traffic?" is one page, not a guess. The benchmarks are on the Why page.