Concepts
How it works
One pipeline, many layers, and an engine that owns every retry.
#The pipeline
your agent -> Rakuvo gateway -> model provider (OpenAI-compatible) | |- Before hooks layers run in order and may edit the request |- provider call '- After hooks layers run in reverse order and may inspect the response, ask for a retry, or add notesA layer implements two methods, Before and After. Because After runs in reverse, the layer closest to the model sees the response first.
On the way in the layers run in this order: model policy, effort, memory, context, tool pruning, loop guard, prompt, guard, verify.
#The engine owns retries
A layer never retries a call itself. It sets a retry flag, and the engine re-runs the whole pipeline on a fresh copy of the original request, up to a limit. Two consequences:
- Two layers cannot fight each other over a response.
- A retry never applies a prompt edit twice.
#Everything is traced
Layers write a note for every action and report the tokens they spend on their own model calls (the judge, a summariser, memory extraction). Traces carry per-attempt tokens, cache hits, latency and the layer notes, so the cost of a layer is always visible. It is charged to the layer, not hidden in the bill.
#Fail open
A judge that is down, slow or confused never blocks a response. A layer acts at most once per request unless its page says otherwise.
#Streaming
The gateway accepts streaming requests. The layers run on the complete response, which is then sent to the client as an event stream, so a streamed reply arrives after the model has finished instead of token by token.