Costaff Secondary Agent Infrastructure

The Problem

The first autonomous agent proved the workflow: a principal-facing assistant could read a Jira backlog, claim work, coordinate over NATS, use credential-brokered GitHub access, run structured Lobster pipelines, and close tickets with evidence. But a single agent is not a team. It creates bottlenecks, hides design assumptions, and makes independent review awkward because the same identity that implemented the change is tempted to verify its own work.

Adding a second agent sounds simple: deploy another pod, give it the same tools, and point it at the same backlog. That is exactly the dangerous version.

A secondary agent needs to be operationally capable while remaining distinct:

• It must have its own identity, workspace, audit trail, and communication keys.
• It must not inherit broad credentials just because the first agent had them.
• It needs reliable task dispatch and review participation without racing the primary agent.
• It needs isolated worktrees so one agent’s repo state cannot corrupt another’s work.
• It needs enough shared standards to collaborate, but not so much shared memory that private context bleeds across principals or roles.

The goal of the Costaff work was to turn “spin up another agent” into a repeatable infrastructure pattern: a second autonomous worker that can contribute real implementation work while preserving separation of duties.

Architecture Overview

Costaff is a secondary agent stack built around five boundaries:

1. Runtime boundary — a separate OpenClaw agent runtime with its own workspace and process lifecycle.
2. Credential boundary — Vault-backed secrets and GitHub App permissions scoped to the repos and actions Costaff actually needs.
3. Messaging boundary — NATS and WebSocket bridge access using Costaff’s own agent identity, not the primary agent’s credentials.
4. Work boundary — Jira labels, task ownership, and isolated git worktrees prevent duplicate claims and dirty-tree collisions.
5. Review boundary — Costaff can implement, review, or respond as a distinct participant in the autonomous delivery loop.

That shape is the important part. Costaff is not “Fiducian with a different name.” It is a separately provisioned actor that shares protocols, not state.

Provisioning a Second Agent Without Copying the First

The provisioning work started with the obvious cluster pieces: a deployment, service account, runtime configuration, persistent workspace, and network access to the same internal tools. The harder part was deciding what not to copy.

The primary agent’s configuration had accumulated organic access over months of operation. Some of that access was essential for any agent, some was principal-specific, and some existed only because early development favored convenience over least privilege. Costaff forced those assumptions into the open.

Representative work across FAD-1308, FAD-1383, FAD-1384, FAD-1418, and FAD-1423 separated the reusable pattern from the individual agent:

• Kubernetes runtime resources define the agent as its own workload.
• Vault paths provide only the credentials required for Costaff’s role.
• RBAC grants are explicit rather than inherited from a shared “agent” bucket.
• Workspace seed files and skills are federated from canonical sources, not copied by hand.
• GitHub access flows through brokered short-lived tokens rather than long-lived static secrets.

The result is boring in the best way: a new agent can be reproduced from manifests and federation steps instead of tribal memory.

Credential and RBAC Design

The key design rule: shared tools do not imply shared credentials.

Costaff needs access to Jira, GitHub, NATS, and internal workflow services, but each grant is scoped to the role it performs. For GitHub, that means broker-issued installation tokens with repository-level permission maps. For cluster access, it means service-account RBAC and Vault injection rather than mounting a broad all-agent secret. For messaging, it means agent identity and signing material associated with Costaff specifically.

This made the system slightly more complex up front, but it paid off immediately. When later work needed a repo-specific runner credential for the portfolio site, the Costaff pattern already had the right instinct: create a separate scoped credential rather than widening a shared one. That same “separate blast radius” principle runs through the secondary-agent design.

Publicly, the important lesson is not the name of any secret or private path. It is the pattern:

• Prefer short-lived brokered tokens over durable personal tokens.
• Prefer repo-scoped grants over organization-wide grants.
• Prefer per-agent service accounts over shared automation identities.
• Treat credential expansion as an architecture decision, not an incidental implementation detail.

NATS and WebSocket Bridge Integration

Autonomous agents coordinate through durable messaging. Costaff needed to participate in that fabric as a first-class peer, not a shadow session.

The NATS side gives agents asynchronous request/response, review handoffs, dispatch nudges, and audit-friendly conversation references. The WebSocket bridge provides real-time wake paths for session-bound work. Costaff’s bridge work focused on making those paths reliable under the constraints that matter for a secondary agent:

• Messages must identify Costaff as the sender, not the primary agent.
• Dispatches must wake the right runtime and not disappear into a stale session.
• Replies need enough conversation metadata for other agents to reconcile them later.
• Operational failures should surface as retryable delivery problems, not silent missed work.

This is where the distinction between “chat” and “coordination protocol” matters. Costaff can send human-readable summaries, but the machine-to-machine work uses structured messages, signed identity, and durable correlation. That lets the team treat agent communication as infrastructure rather than vibes.

Dispatch and Work Ownership

A second agent increases throughput only if it does not create coordination debt. Two agents racing the same ticket is worse than one agent working slowly.

Costaff’s Jira integration follows the same autonomous work planning model used by the rest of the system: tickets advertise capability and routing labels, dispatch selects eligible work, and a claim path converts available work into assigned work. The board remains the human-visible source of truth, but the claim step is treated like a distributed-systems boundary.

The practical conventions are simple:

• agent:costaff marks work Costaff can take.
• work:available means the ticket can enter pickup selection.
• work:assigned means a worker has claimed it.
• In Progress work is a resume state, not an invitation for another session to start over.
• Human review labels and decision labels pause automation rather than bypassing it.

Those labels are not just board decoration. They are the coordination language between Jira, dispatch, NATS wake events, and the agent runtime.

Worktree Isolation

One of the less glamorous but most important parts of the Costaff work was git hygiene. Agents routinely operate in short-lived or isolated sessions. They may be interrupted by context limits, tool failures, review requests, or unrelated urgent tasks. If multiple agents share a checkout, dirty state becomes a hidden coupling point.

Costaff uses isolated worktrees and explicit branch naming so each ticket’s code state is recoverable and attributable. That supports:

• Independent implementation branches.
• Review branches that do not overwrite implementer work.
• Cleaner commit-first enforcement.
• Safer recovery after a session restart.
• Easier handoff between Costaff, Fiducian, and Claude Code.

This is one of those infrastructure choices that looks excessive until the first time it saves a half-completed branch from being trampled by another autonomous session.

ACP Tasking and Human-Supervised Go-Live

Costaff also became a testbed for ACP-style tasking: bounded agent sessions with a clear objective, output contract, write scope, and verification brief. That matters because secondary agents should be easy to aim without granting them open-ended initiative.

The go-live path intentionally staged capability:

1. Read-only inspection and report generation.
2. Limited implementation against well-scoped tickets.
3. Review participation with authenticated diffs.
4. Direct backlog pickup once the WIP and dispatch gates were trusted.
5. Operational reliability work once real incidents exposed gaps.

This staged rollout prevented a common automation mistake: measuring success by whether the agent can do anything, instead of whether it can do the right thing safely under failure.

Operational Reliability Lessons

The Costaff tickets include a lot of “plumbing” work because secondary agents expose every assumption the first agent was quietly relying on. Representative fixes across FAD-876, FAD-1035, FAD-1055, FAD-1290, FAD-1436, FAD-1437, and FAD-1440 hardened the system around:

• Skill federation drift between canonical repos and live agent pods.
• Dedicated model/API budgeting so one agent cannot starve another.
• Retryable dispatch when a bridge or session misses a wake event.
• Review workflow portability across agent hosts.
• Better evidence trails when work is completed by one actor and verified by another.
• Explicit blocked states when a task needs a credential or architecture decision.

The pattern is familiar from traditional distributed systems: adding a second node does not just double capacity. It reveals every implicit singleton.

What This Demonstrates

A useful agent team needs identity separation, not just more model calls. The value of Costaff is not that it can produce text or run commands. The value is that it can act as a separate worker with its own permissions, state, audit trail, and review responsibility.

Least privilege is easier when designed before scale. Retrofitting a second agent forced credential boundaries to become explicit. That work now makes future agents easier to add safely.

The backlog is the coordination surface. Jira labels and statuses are not a thin UI over hidden automation. They are the visible contract that lets humans, dispatch drivers, and agents agree on who owns what.

Operational reliability is a feature. NATS delivery, worktree isolation, model budgets, and skill federation are not side quests. They are what make a secondary agent dependable enough to use.

Current State

Costaff is now represented as a distinct secondary-agent system with its own routing labels, credential posture, NATS participation, worktree isolation, and review/implementation role. The implementation matured from provisioning and bridge setup into real production contribution: Costaff can prepare pull requests, participate in review dispatch, surface credential-scope blockers, and hand work back into the shared autonomous delivery loop.

The next frontier is not merely adding more agents. It is making each additional agent cheaper to provision, easier to constrain, and more useful as an independent participant in a small autonomous engineering team.