When my agent started making five tool calls across three steps (classifying intent, querying data, then rendering a chart), debugging failures from log lines alone felt like reading a book; but with all the pages shuffled. I needed to see the full execution flow at a glance.

I use Braintrust for tracing - but honestly, still had a pretty tough time easily "playing back" a series of requests. So I ended up building a TraceCollector that records every event in the agent's lifecycle, persists traces in memory, and feeds them to a separate Vite + React debugger app that renders them as interactive flow graphs.

The problem

Log-based debugging works fine when your agent makes one tool call per request. But once you've got multi-step agentic loops (classification → data query → chart rendering → error correction → retry), a flat list of log lines loses the structure. You can't easily see which tool calls happened in which step, what the classifier decided, or where things went wrong. I found myself constantly reconstructing the execution flow manually, which was slow and error-prone.

The solution: Structured event tracing

The TraceCollector class in apps/api/src/lib/trace.ts is a lightweight event emitter that captures timestamped events throughout the request:

export type TraceEventType =
  | "request_received"
  | "messages_converted"
  | "first_message_detected"
  | "tools_resolved"
  | "tools_created"
  | "system_prompt_built"
  | "stream_started"
  | "step_finished"
  | "tool_called"
  | "tool_result"
  | "stream_completed"
  | "error_caught";
 
export class TraceCollector {
  private trace: Trace;
 
  constructor(metadata: Record<string, unknown> = {}) {
    this.trace = {
      id: randomUUID(),
      conversationId: null,
      startedAt: Date.now(),
      completedAt: null,
      events: [],
      metadata,
    };
  }
 
  emit(type: TraceEventType, data: Record<string, unknown> = {}): void {
    this.trace.events.push({
      id: randomUUID(),
      traceId: this.trace.id,
      type,
      timestamp: Date.now(),
      data,
      durationMs: Date.now() - this.trace.startedAt,
    });
  }
 
  complete(): Trace {
    this.trace.completedAt = Date.now();
    return this.trace;
  }
}

Each event captures what happened, when it happened, how long since the request started, and arbitrary data relevant to that event. The event types form a natural DAG; request_received leads to messages_converted, which leads to tools_resolved, and so on.

Instrumenting the chat route

The collector is instantiated at the top of every request and threaded through the entire pipeline. Here's how it's wired into the chat route (apps/api/src/routes/chat.ts):

chatRoutes.post("/", async (c) => {
  const collector = new TraceCollector();
  const requestId = randomUUID();
 
  // Step 1: Convert messages
  const { modelMessages, isFirstMessage, latestUserText } = await convertAndTruncateMessages(
    parsed.data.messages,
  );
  collector.emit("messages_converted", {
    modelMessageCount: modelMessages.length,
  });
 
  // Step 2: Resolve tools
  const { allTools, resolvedCategories, resolveReason } = await resolveAndCreateTools(
    latestUserText,
    DEFAULT_USER_ID,
  );
  collector.emit("tools_resolved", {
    categories: [...resolvedCategories],
    reason: resolveReason,
  });
 
  // Step 3: Build system prompt
  const system = buildSystemPrompt({
    hasWorkout: Boolean(workoutId),
    isFirstMessage,
  });
  collector.emit("system_prompt_built", { promptLength: system.length });
 
  // Step 4: Stream
  collector.emit("stream_started", {
    model: modelId,
    maxOutputTokens: modelMaxOutputTokens,
    reasoningBudget: supportsReasoning ? (REASONING_BUDGET[modelId] ?? 8000) : 0,
  });
 
  // ... tool calls and results are emitted from the step handler ...
 
  // On completion
  collector.emit("stream_completed", { usage: usage ?? {} });
  addTrace(collector.complete());
});

The step handler emits tool_called and tool_result events for each individual tool invocation, plus step_finished for each agentic step, and error_caught when things go wrong. Each event includes rich context: which tools were loaded, what the classifier decided, how many steps completed.

Audit logging with sanitization

Every tool call is also written to a PostgreSQL audit log, but with sensitive data stripped. The sanitizeValue function handles this:

function sanitizeValue(value: unknown, depth = 0): unknown {
  if (depth > 5) return "[nested]";
  if (typeof value === "string") {
    if (value.startsWith("data:")) return "[base64_data]";
    if (value.length > MAX_AUDIT_STRING_LENGTH)
      return `${value.slice(0, MAX_AUDIT_STRING_LENGTH)}…[truncated]`;
    return value;
  }
  if (Array.isArray(value)) return value.map((v) => sanitizeValue(v, depth + 1));
  if (typeof value === "object") {
    const out: Record<string, unknown> = {};
    for (const [k, v] of Object.entries(value)) {
      out[k] = sanitizeValue(v, depth + 1);
    }
    return out;
  }
  return String(value);
}

Base64 data URIs (from photo processing) are replaced with [base64_data]. Long strings are truncated at 200 characters. Deep nesting is capped at 5 levels. This keeps audit logs queryable without storing megabytes of image data.

The audit writes are fire-and-forget; they never block the main request:

export async function writeAuditLog(entry: AuditEntry): Promise<void> {
  try {
    await db.insert(auditLogs).values({ ... });
  } catch (error) {
    logger.warn('[Audit] Failed to write audit log', { ... });
  }
}

What if tracing itself breaks?

The collector is designed to never interfere with the actual request. Every emit() call is a simple array push; no async operations, no I/O. If the collector throws (which it shouldn't, but defensive programming), the catch block around the trace completion just logs a warning and moves on. The user's request still completes normally.

The audit log writes are similarly isolated. A failed write gets logged as a warning but doesn't abort the request. I've had a few of these in production (usually transient Postgres connection issues), and they've never impacted the user experience.

Extracting error context

When an error occurs, the trace already contains valuable context. The extractErrorContext function harvests it for structured error reporting:

function extractErrorContext(collector: TraceCollector): Record<string, unknown> {
  const events = collector.getTrace().events;
  const context: Record<string, unknown> = {};
 
  for (const e of events) {
    switch (e.type) {
      case "stream_started":
        context.model = e.data.model;
        break;
      case "tools_created":
        context.toolsLoaded = e.data.toolNames;
        break;
      case "tools_resolved":
        context.resolvedCategories = e.data.categories;
        break;
    }
  }
 
  context.stepsCompleted = events.filter((e) => e.type === "step_finished").length;
  context.toolCallsCompleted = events.filter((e) => e.type === "tool_called").length;
  return context;
}

When an error is caught, the error_caught event includes this context; so you can immediately see which model was running, which tools were loaded, how many steps completed and what the classifier decided, all in one structured object.

The debugger app

The apps/agent-debugger directory contains a separate Vite + React application that visualizes traces. It renders the event sequence as a flow graph (using Dagre for layout and XYFlow for rendering) with an event timeline and detail panel.

The debugger reads traces from the API's in-memory trace store and renders each event as a node in the graph. Tool calls fan out from step nodes. Error nodes are highlighted in red. You can click any node to see its full data payload.

What this ended up getting me

The combination of structured tracing, sanitized audit logs and visual debugging gave me three levels of observability:

Real-time debugging with the flow graph showing exactly where a request went wrong
Post-hoc analysis through audit logs in PostgreSQL, queryable for patterns (which tools fail most? which models produce the most repair attempts?)
Error triage via the extracted error context, giving me everything I need in a single structured object

The TraceCollector itself is under 80 lines. The real investment is in consistently emitting events at every decision point in the pipeline. Once you have that discipline, debugging agent behavior becomes tractable.

What's still rough

In-memory trace storage is great for fast iteration, but it's not ideal for longer retention or cross-instance debugging. If the API restarts, traces are gone. That's fine for now, but I'd eventually like a durable trace backend without making the hot path heavier.

TL;DR

If your agent has multi-step tool chains, emit structured trace events at every decision point and build a visual debugger to render them. The upfront cost is tiny compared to the time you'll save debugging production failures.

Takeaways

Make tracing zero-cost to the request. Synchronous array pushes only; no async, no I/O. If tracing can't fail silently, it shouldn't be in the hot path.

Sanitize audit data at write time, not read time. Stripping base64 data and truncating strings before they hit Postgres means the audit table stays queryable and reasonably sized.

Build the debugger as a separate app. Keeping it decoupled from the main API meant I could iterate on the visualization without touching production code. It reads from the same trace store but has its own deployment.

Building a visual debugger for agent tool chains