Making Composio production-ready

Roma connects to Gmail, Google Calendar, Linear, Notion, GitHub and a few others through Composio. One integration, and the model can suddenly reach hundreds of tools it didn't have yesterday. That part took an afternoon.

Everything since has been building the layers that make it safe, fast and legible. None of them are Composio's fault — an aggregator's job is reach, and it does that well. But if you're about to wire one in, here's what you'll find yourself building.

1. A registry, because the vendor's catalogue is not your product

GitHub publishes 893 tool slugs. Gmail publishes 63. Shipping those as-is means handing a model a delete button because it happened to come in the box.

So each toolkit has a registry: every slug we offer, in an action group (read · create · edit · delete · send), with the ones we refuse written down and why. Two things read it. The tool-router session is filtered by it, so the model can't discover what we don't offer. The execute guard enforces it, so a slug named directly is still refused. Narrow what's discoverable, refuse what's called.

The distinction that keeps this sane: exposure is our decision, permission is the user's. A denied slug is unreachable and has no settings row. A permission is only ever "ask me first or don't" about something already exposed. So a permission bug can never widen the surface — the worst it does is skip a confirmation.

2. Permissions by action group, not by slug

Nobody wants 893 switches. Nobody wants 17 either. The question a person actually has is coarser than a slug: may it read my mail, may it send. One preference per (connection, group), and every slug in the group inherits it.

send is its own group for one reason: it reaches other people under your name and there's no undo.

3. The model shouldn't see the tools at all

We use the meta path — the model searches for a tool, then executes it — for integrations, while our own CRUD tools stay flat. That's ~24 tools per turn no matter how many apps you've connected, versus a count that grows linearly. It cut about 10k tokens off the prompt.

It costs one extra round-trip on integration writes. Fine: those are the rare path, and the smaller prompt pays for it everywhere else.

4. beforeExecute is where correctness lives

Three modifiers run before any call leaves.

The interesting one is the entity validator. The failure it exists for: user says "file a bug for the Windows team", the workspace has no Windows team, and the model picks "Roma Web" — visually similar, confidently wrong, written silently. Same shape for "assign it to me" landing on a colleague.

Prompt edits did not fix this. Confident-but-wrong isn't reachable by instruction. What fixed it was structural: before a write, check that the chosen entity's name shares a word with what the user actually said. No overlap, no write — throw an error rich enough that the model asks instead. It now says "I couldn't find a team matching 'Windows' — did you mean…" and lists the real ones.

There are deliberate bypasses (your configured default team; self-assignment when you didn't say "assign"), and it reads entities established earlier in the conversation, not just the last message. That second part came from a real false positive.

5. afterExecute does two different jobs — keep them apart

The same response feeds two consumers with opposite needs.

The UI gets a canonical envelope: small, normalised, built from the full response, so clients render any tool's result generically instead of hand-writing a card per slug.

The model gets the raw response trimmed in place, because tool results accumulate across rounds and that's the dominant per-turn cost. A Notion write chain once hit 213k cumulative input tokens and 429'd our provider's rate limit. Trimmers are per-toolkit, opt-in, passthrough by default — you only write one when a toolkit is measurably verbose.

Conflate these two and you either ship an ugly card or a rate limit.

6. Don't trim the aggregator's own responses

We tried. The search response carries recommended plan steps that read like prose bloat, and stripping them broke cold-path writes — the model lost the "fetch the schema before you insert" instruction it was relying on and ballooned to seven calls discovering it the hard way.

Reverted the same day. Some verbosity is load-bearing.

7. Per-app guidance, just in time

Every app has quirks worth telling the model — but the system prompt is cached, and anything you put there is paid for on every single turn, including the ones that never touch that app.

So per-toolkit hints are appended to the search result instead. The model only reads Notion's guidance on a turn that went looking for Notion. Zero cost to the cached prefix, and the advice arrives at the moment it's actionable.

8. Response shapes are not stable, and the failure is silent

Twice now, an upstream response has quietly changed which key holds the array. Once a calendar list moved to items. Once a team list came back as a paginated envelope naming no collection at all.

The second one is the instructive failure. Three catalog fetches run in parallel, each degrading to an empty array on its own so one outage can't poison the rest. Teams silently stored zero while members and projects populated normally. Nothing errored. The only symptom was a count on a settings screen nobody was looking at — for months.

Two rules came out of it. Capture response shapes from a live call, never from documentation. And anything that degrades to empty needs a "found nothing in a non-empty response" warning, or it's invisible by construction.

Amusingly, the vendor's own pitfall notes describe the inverse shape for that exact endpoint. Both are live. Handle both.

9. Health is a state, not an exception

A revoked OAuth grant stays structurally "active" on the vendor's side. If you render your connection list from that flag alone, you show a green dot over something dead.

So health is derived from what the last real call actually did, and a broken connection turns its row into a repair instead of a status. The failure users report otherwise isn't "it's broken" — it's "it says it's connected."

What I'd tell someone starting

The aggregator gets you reach immediately. Everything above is the difference between reach and something you'd hand to a person with a real inbox.

If I could only keep three: the registry, because it's the boundary everything else hangs off. The entity validator, because it's the one class of bug prompting genuinely cannot reach. And capturing response shapes from live calls, because I've now paid for that lesson twice and the second one hid for months.