From Agent Demo to Production

Who this is for

You have a HazelJS demo that sometimes completes a real task with @Tool methods. Stakeholders want it “in prod next sprint.” This guide sequences Agent OS work so you do not scale autonomy past control.

Pair with the checklist lesson: /learn/agentic-development/from-demo-to-production-checklist. Capability map: Agent OS.

Reality check

Ask:

1. What side effects can tools perform today?
2. What happens on retry after a crash mid-tool?
3. Who gets paged when it misbehaves at 2am?
4. What is the monthly cost ceiling (costRoute / budgets)?
5. How do you know a prompt or DNA change did not break describeAgent fixtures?

If answers are vague, you have a prototype—not a service.

Agent OS capability map (use these names)

Phase A — Contain (1–3 days)

Goal: prevent catastrophic autonomy.

• Remove or disable irreversible tools
• Enforce maxSteps and loop.maxIterations in runtime
• Add a global kill switch (feature flag)
• Lock credentials to least privilege / capabilities
• Turn on timeline logging with redaction (defaultPiiMaskPolicies)

Ship containment before polish.

await runtime.execute('support-desk', message, {
  loop: {
    maxIterations: 4,
    stages: ['plan', 'execute', 'critique', 'validate'],
  },
  costRoute: { qualityBias: 0.35, maxCostUsd: 0.05 },
});

Phase B — Make writes safe (3–10 days)

For every write tool:

1. Side-effect classification
2. Idempotency key strategy in the adapter
3. requiresApproval: true + PolicyEngine require_approval
4. durableSuspend: true and resume with approveAndResume
5. Compensating action notes

const store = createDurableRunStore('.hazel/runs');

const runtime = new AgentRuntime({
  llmProvider,
  durableSuspend: true,
  runRepository: store.runRepository,
  checkpointService: store.checkpointService,
  humanTaskService: store.humanTaskService,
  policyEngine: new PolicyEngine([
    {
      id: 'refund-hitl',
      tool: 'processRefund',
      effect: 'require_approval',
      priority: 20,
    },
  ]),
});

const waiting = await runtime.execute('support-desk', 'Refund ORD-100');
await runtime.approveAndResume(waiting.executionId, {
  approved: true,
  approvedBy: 'ops@example.com',
});

Failure drill: kill the process after a successful external write but before checkpoint; fix until resume is safe (/patterns/durable-agent-run, /patterns/human-approval-gate).

For SQL process stores: createSqlDurableRunStore(prisma) with Agent OS Prisma models from the package example schema.

Phase C — Contracts, recovery, governance

Wire the remaining execute options from the Agent OS guide:

await runtime.execute('support-desk', message, {
  loop: { maxIterations: 4, successScore: 90 },
  contract: {
    name: 'support-desk-slo',
    maxLatencyMs: 30_000,
    fallbackAgent: 'safe-desk',
  },
  recovery: {
    maxRetries: 2,
    fallbackAgent: 'safe-desk',
    steps: ['retry', 'fallback_agent', 'fail'],
  },
  costRoute: { qualityBias: 0.35, maxCostUsd: 0.05 },
  governance: {
    action: 'agent.execute',
    roles: ['agent:run'],
    residency: 'eu',
    compliancePacks: ['soc2'],
  },
});

Also configure GovernanceGate / defaultAgentGovernance on AgentModule.forRoot when running inside a DI app.

Phase D — Prove quality (ongoing)

• Expand golden set: happy paths, tool errors, injection-like observations
• Put describeAgent suites in CI
• Optional digital twin / canary (runDigitalTwin / shouldRunCanary) before full traffic
• Track success rate, human takeover rate, cost per success

Phase E — Operate with Inspector

• Timeline store (FileTimelineStore) and getTimeline
• Inspector SSE / run endpoints (/__hazel/agents/:name/…)
• Dashboards and alerts (tool errors, budget exhaustion, approval lag)
• Runbook: pause agent, revoke tool auth, export timeline
• Named owners and an SLO—even a soft one tied to contract
• Security review against OWASP LLM Top 10

Phase F — DNA and expand autonomy deliberately

1. exportAgentDna for prompt, tools metadata, policies, contracts
2. Review DNA in PRs; smoke with hazel agent run ./agent.dna.json "hello" --mock / bootstrapRuntimeFromDna
3. Raise autonomy only when metrics are green: broader reads → reversible writes → gated irreversible → higher budgets → memory layers with retention (/patterns/memory-layers)
4. Document each raise as a DNA version (/guides/design-an-agent-manifest)

Remember: DNA is configuration + contract; TypeScript @Tool handlers still live in the app.

Organizational pitfalls

• Demo-driven deadlines that skip kill switches and durableSuspend
• Shared admin API keys “just for the hackathon”
• No owner after the building team rotates
• Eval theater — three anecdotes instead of describeAgent fixtures

Done when

• Checklist blockers cleared or formally accepted with expiry
• On-call can pause the agent and approve/reject HITL without the original author
• A DNA/prompt change that breaks a golden task fails CI

Next

• /guides/design-a-production-agent-runtime
• /guides/design-an-agent-manifest
• /comparisons/agent-framework-vs-runtime-vs-agent-os
• Roadmap: /roadmap