Sign in
Skip to solution
hardAI Engineering

How do you validate and repair structured LLM output at runtime?

480 views
01

Understand the problem

Schema validation, bounded retries with error feedback, and safe fallbacks when repair fails.

validationstructured-outputrepair-loopreliability
02

Attempt it yourself

Sketch your approach before reading the solution — that's what interviews test.

Nudge consolestandby

Stuck? Beam a request up — the console returns a conceptual nudge that guides your logic without spoiling the implementation.

03

Study the solution

Never trust parsed output: validate against a schema (Zod/Pydantic/JSON Schema) including semantic rules (enums, ranges, cross-field constraints) that syntax alone misses. On failure, run a bounded repair loop — re-prompt with the original output plus the specific validation errors, at most once or twice — then fall ba

Solution ready — 2 min read

Classified // press E to declassify

04

Read the code

Bounded repair with semantic checks
async function extractInvoice(text: string): Promise<Invoice> {
  let lastErrors = "";
  for (let attempt = 0; attempt < 3; attempt++) {
    const raw = await model.extract(text, lastErrors);       // errors appended on retry
    const parsed = InvoiceSchema.safeParse(raw);
    if (parsed.success) {
      const sem = await semanticChecks(parsed.data);          // vendor exists? totals add up?
      if (sem.ok) { metrics.repairAttempts(attempt); return parsed.data; }
      lastErrors = sem.errors.join("; ");
    } else {
      lastErrors = parsed.error.issues.map((i) => i.path + ": " + i.message).join("; ");
    }
  }
  await reviewQueue.push({ text, lastErrors });               // deterministic fallback
  throw new NeedsHumanReview();
}
05

Join the discussion

Discussion (0)

Sign in to join the discussion.

No responses yet. Be the first to share what you think.

Transmission complete // awaiting log

KEEP THE
STREAK ALIVE.

Dossier 73 of 80 decoded in the AI Engineering track. One more won't hurt.

Back to track