IBM's open-source CUGA agent harness skips the plumbing, goes straight to the prompt

Most agentic apps begin with a week of plumbing before the agent does anything useful. You pick a framework, wire up a model client, write tool adapters, build some way to stream state to a UI, and somewhere in there you also decide what the agent is actually for. The interesting part arrives last.

CUGA inverts that. It's IBM's open-source agent harness that handles the planning, the execution loop, the tool calls, and the state plumbing for you. What's left is the part that's actually yours: which tools the agent can reach, and what you tell it to do. To show how this works in practice, the team built cuga-apps: two dozen small, working apps, each a single FastAPI file wrapping one CugaAgent, from a movie recommender to an IBM Cloud architecture advisor. They're meant to be read and copied. You can browse the live gallery.

What CUGA saves you from writing

CUGA answers the fair question: what does it save you from writing? The orchestration around a model that you'd otherwise rebuild every time. It plans before it acts, then executes with a mix of tool calls and generated code via CodeAct. On a long task that runs twenty steps, CUGA holds intermediate results and runs a reflection step that can catch a bad call and re-plan instead of barreling ahead. That machinery is why it has topped agent benchmarks like AppWorld and WebArena.

You also set the cost/latency tradeoff from config rather than code: Fast, Balanced, and Accurate reasoning modes, with code execution in whatever sandbox you trust: local, Docker/Podman, or E2B cloud. Most harnesses assume a frontier model sits underneath and lean on it to recover when a plan goes sideways; CUGA does that work itself. The planning, the reflection step, the variable-tracking that keeps a long run on course, that's the harness carrying load the model would otherwise have to, which is what lets a smaller open-weight model hold up where it normally wouldn't.

A single-file agent: the IBM Cloud advisor

Here's the IBM Cloud advisor, an agent that recommends real IBM Cloud services for an architecture. The whole thing fits in one file: a main.py with the agent factory, the tools, and the prompt, plus a small UI.

The whole agent comes down to this:

def make_agent():
    from cuga import CugaAgent
    from _llm import create_llm
    return CugaAgent(
        model=create_llm(
            provider=os.getenv("LLM_PROVIDER"),
            model=os.getenv("LLM_MODEL"),
        ),
        tools=_make_tools(),
        special_instructions=_SYSTEM,
        cuga_folder=str(_DIR / ".cuga"),
    )

Four arguments. The model comes from a small factory called create_llm that speaks to OpenAI, Anthropic, watsonx, LiteLLM, or Ollama depending on an environment variable. Nothing in the app code knows which model sits behind it. The two arguments that carry the app are tools and special_instructions.

The tools mix a local function with a hosted one. There's a pattern here that holds across every app: a split between MCP tools and inline tools. Generic, stateless capabilities come from shared MCP servers; anything specific to this app gets defined inline as a normal Python function. The cloud advisor's prompt tells the agent to search the catalog before naming any service, recommend three to seven services with each one's role in the design, and never invent service names.

Governance built in, not bolted on

A demo agent that searches a catalog is low-stakes. Point the same pattern at something that writes files, runs shell commands, or touches production, and the question changes: how do you stop it doing something you'll regret? CUGA answers this in the runtime, not in a wrapper you add afterward. The open-source agent ships a policy system with six policy types: Intent Guards, Tool Guides, Tool Approval, Output Formatters, Human-in-the-Loop, and the CustomPolicy escape hatch.

An Intent Guard checks the request before the agent picks a tool. Tool Approval runs after the agent has generated its code and inspects which tools that code uses. And Output Formatter fires only once the final message exists. Triggers go past keyword matching too, they're held in a sqlite-vec store and matched semantically, so a policy fires on what the user means, not just on an exact keyword.

From one agent to many: multi-agent delegation and skills

When one agent would drown in its own context, you split the work. A CugaSupervisor delegates to specialist CugaAgents, each with its own tools, prompt, and isolated context, and the supervisor only ever reasons about which specialist to hand a subtask to. Its planning surface stays small no matter how many tools sit underneath. Specialists can be local or external agents reached over A2A.

The other extension packages know-how rather than tools: Agent Skills, a folder with a SKILL.md playbook the agent pulls into context only when a task calls for it. Ouroboros, a lead-gen app, has a supervisor over seven specialists: scout, site auditor, voice-of-customer, person finder, stack scanner, revenue estimator, and a pitch-email writer. Each specialist is one skill loaded into a CugaAgent.

Production path: Sovereign Core

Because the harness is small, open source, model-agnostic, and already governs itself, the agent you wrote on your laptop is the same agent that runs in a locked-down deployment. IBM Sovereign Core builds on this. It runs CUGA agents under Boundary Isolation: data, control plane, and execution engine inside the same logical boundary. Deployments default to gpt-oss-120b running fully air-gapped within your infrastructure, and tools reach only private VNETs with per-tool approval. Every reasoning step emits OpenTelemetry traces into a Grafana Tempo backend that stays in-tenant, with no telemetry phoning home.

The developer takeaway stands on its own: an agentic app can be one file you hold in your head. The tools and the prompt are the only parts you really write. The apps are a library to learn from, and when the stakes rise, the governance is already in the runtime.