Labsco
mastra-ai logo

testing-core-processors

★ 25,837

by mastra-ai · part of mastra-ai/mastra

🧰 Not standalone. This skill ships with mastra-ai/mastra and only works together with that tool — install the tool first, then add this skill.

This is the playbook your agent receives when the skill activates — you don't need to read it to use the skill, but it's here to audit before installing.

Testing Core Error Processors

How to write integration tests for error processors in packages/core/src/processors/.

Mock Model Pattern

Use MockLanguageModelV2 from @internal/ai-sdk-v5/test to simulate API errors and verify retry behavior.

import { APICallError } from '@internal/ai-sdk-v5';
import { convertArrayToReadableStream, MockLanguageModelV2 } from '@internal/ai-sdk-v5/test';

// Track calls and captured prompts
let callCount = 0;
const receivedPrompts: any[] = [];

const model = new MockLanguageModelV2({
  doGenerate: async ({ prompt }) => {
    callCount++;
    receivedPrompts.push(JSON.parse(JSON.stringify(prompt)));
    if (callCount === 1) {
      throw new APICallError({
        message: '...',
        url: '...',
        requestBodyValues: {},
        statusCode: 400,
        responseBody: '...',
        isRetryable: false,
      });
    }
    return {
      rawCall: { rawPrompt: null, rawSettings: {} },
      finishReason: 'stop',
      usage: { inputTokens: 10, outputTokens: 20, totalTokens: 30 },
      content: [{ type: 'text', text: 'response' }],
      warnings: [],
    };
  },
  doStream: async ({ prompt }) => {
    // Same error logic as doGenerate
    // IMPORTANT: Stream response must include all event types:
    return {
      rawCall: { rawPrompt: null, rawSettings: {} },
      warnings: [],
      stream: convertArrayToReadableStream([
        { type: 'stream-start', warnings: [] },
        { type: 'response-metadata', id: 'id-0', modelId: 'mock-model', timestamp: new Date(0) },
        { type: 'text-start', id: 'text-1' },
        { type: 'text-delta', id: 'text-1', delta: 'response text' },
        { type: 'text-end', id: 'text-1' },
        { type: 'finish', finishReason: 'stop', usage: { inputTokens: 10, outputTokens: 20, totalTokens: 30 } },
      ]),
    };
  },
});

Stream Mock Gotcha

The stream mock format requires stream-start, response-metadata, text-start, text-delta, text-end, and finish events. Using only text-delta + finish (the minimal format) will result in empty text output because the AI SDK expects the full event sequence. See prefill-error-recovery.test.ts for the reference pattern.

Test Structure

For each error processor, write at minimum:

  1. Happy path: Processor catches the target error, modifies messages, retries successfully
    • Assert: agent.generate() succeeds, mock called 2x, retry prompt has expected modifications
  2. Control test: Same scenario without the processor — error propagates
    • Assert: agent.generate() throws the expected error
  3. Selectivity test: Processor ignores unrelated errors (e.g. rate limit 429)
    • Assert: Error propagates, mock called only 1x

Passing Messages with Tool Calls

When seeding conversation history for tool-related tests, pass messages as the second argument to agent.generate() or agent.stream() using the AI SDK message format:

const messages = [
  { role: 'user', content: 'Do something' },
  { role: 'assistant', content: [{ type: 'tool-call', toolCallId: 'some-id', toolName: 'myTool', args: {} }] },
  { role: 'tool', content: [{ type: 'tool-result', toolCallId: 'some-id', toolName: 'myTool', result: 'done' }] },
];
await agent.generate(messages);

Devin Secrets Needed

None for mock-based integration tests. For live API tests:

  • ANTHROPIC_API_KEY — for testing against real Anthropic API
  • OPENROUTER_API_KEY — for testing provider switching scenarios