ACAgentic Craft

Intermediate~28 minHazelJS

describeAgent Golden Suites

Author @hazeljs/testing describeAgent cases with expectTools, assertAgentResult, and runAgentSuite wiring.

Authors
editorial-team
Published
Last reviewed
Progress is stored locally in this browser.

Direct answer

describeAgent + expectTools + assertAgentResult + runAgentSuite are the primary reliability harness for HazelJS agents. Prefer tool-trace assertions over brittle prose equality.

Docs: Testing. Guide: /guides/build-an-evaluation-harness.

Worked example

typescript
import {
  describeAgent,
  runAgentSuite,
  assertAgentResult,
  expectTools,
} from '@hazeljs/testing';
import { describe, it, expect } from 'vitest';

const suite = describeAgent('Support Agent', ({ test }) => {
  test('status ask uses lookup', async ({ run }) => {
    const result = await run('Status of ORD-1001?');
    expectTools(result, ['lookupOrder']);
    assertAgentResult(result, {
      maxLatencyMs: 8_000,
      maxCostUsd: 0.05,
      outputIncludes: 'ORD-1001',
    });
  });

  test('refund proposes lookup then refund tool', async ({ run }) => {
    const result = await run('Refund ORD-1001 please');
    expectTools(result, ['lookupOrder', 'processRefund'], 0.5);
  });
});

it('support agent regression', async () => {
  const { failed, errors } = await runAgentSuite(suite, {
    agentName: 'support-agent',
    run: async (input) => {
      const out = await runtime.execute('support-agent', input, {
        maxSteps: 8,
        budget: { maxCostUsd: 0.1 },
      });
      return {
        output: out.response ?? '',
        durationMs: out.duration,
        // map tool trace fields per testing docs / your result shape
        tools: out.tools ?? out.toolCalls,
        costUsd: out.costUsd,
      };
    },
  });
  expect(failed, errors?.join('\n')).toBe(0);
});

describeAgent does not replace Vitest/Jest globals—call runAgentSuite inside it, or use bindAgentSuite as documented.

Artifact: Five golden tests including one forbidden-tool case

Sources

Related