Deterministic JSON Extraction & Schema Guardrails for Production AI Pipelines

Engineers introducing LLMs into production AI Agents or enterprise automation pipelines often experience the same reversal of fortune.

During local development and demos, a System Prompt says, “Return only valid JSON, with no extra text.” With gpt-4o or Claude 3.5 Sonnet, all 50 test samples produce clean JSON. Confident, the team deploys it to parse commercial invoices, extract on-chain transaction intents, and dispatch orders to microservices.

By day 3 of handling tens of thousands of real requests daily, the downstream order database is broadly stalled:

  • One response reaches the model’s maximum Token limit and is truncated at {"amount": 1500, "recipient": "0x89. Unclosed delimiters make JSON.parse() throw an uncaught syntax exception;
  • Another response unexpectedly wraps its JSON in a Markdown block, json \n ... \n , crashing strict deserialization middleware;
  • Worse, “silent semantic contamination” produces syntactically valid JSON with the wrong financial representation. The model expresses 1.5 USDC as the quoted string "1.5", and a downstream service performs JavaScript concatenation, "100" + "1.5" = "1001.5", creating an overpayment dozens of times larger than intended.

LLMs are probabilistic sampling systems; enterprise backend services and smart contracts are deterministic state machines.

Prompt engineering alone cannot guarantee perfect structured output. A reliable pipeline built on probabilistic responses needs three deterministic guardrail layers: Constrained Decoding → Syntactic Repair → Semantic Invariants.


1. Three Destructive Forms of Structured-Output Failure

Error logs from millions of daily inference calls show three progressively deeper failure levels:

+----------------------------------------------------------------------------------------------------+
|                                  Three LLM Structured-Output Failure Levels                         |
+----------------------------------------------------------------------------------------------------+

   [ Syntactic Breakage ]
   - Token truncation leaves a missing `}` or `]`
   - Extra prose ("Here is the JSON you requested:") or unclosed Markdown fences
   - Unescaped quotes inside strings (e.g. `"description": "He said "Hello""`)
   - Result: `JSON.parse()` throws SyntaxError and the service returns 500.

   --------------------------------------------------------------------------------------------------

   [ Schema Drift ]
   - Hallucinated synonyms for field names (`recipientAddress` becomes `to_address` or `wallet`)
   - A single-element array becomes a standalone object
   - Integer fields become floats or strings
   - Result: TypeScript assumptions break, causing downstream `Cannot read properties of undefined`.

   --------------------------------------------------------------------------------------------------

   [ Semantic Invariant Breach ]
   - Valid structure and types, invalid business logic: item amounts do not equal the total
   - Token Decimals in a transfer intent do not match the target chain
   - Authorized amount exceeds the system's hard safety cap
   - Result: silent financial failures and loss-of-funds vulnerabilities.
+----------------------------------------------------------------------------------------------------+

2. Three-Tier Deterministic Guardrails

In fone.cc full-stack delivery, no model response is directly parsed and trusted. Every AI Agent inference passes through a strict validation funnel:

+----------------------------------------------------------------------------------------------------+
|                                  Three-Tier Deterministic Guardrail Pipeline                        |
+----------------------------------------------------------------------------------------------------+

   [ Layer 1: Constrained Decoding & Structured Outputs ]
   - Require the official API's `response_format: { type: "json_schema", strict: true }`
   - Use context-free grammar (CFG) and Logit bias to block illegal token sampling at generation time
                 │
                 ▼ (For open models without Strict Schema support or unexpected Token truncation)
   [ Layer 2: Syntactic Repair & JSON Fixer ]
   - Strip Markdown fences (````json ... ````) and surrounding prose with compiled regex
   - Close incomplete quotes, brackets, and truncated commas (Partial JSON AST Recovery)
                 │
                 ▼
   [ Layer 3: Zod / Pydantic Semantic Invariant Gate ]
   - Strict field types and formats (EVM address regex, UTC ISO timestamps, positive-integer bounds)
   - Cross-field assertions (`refine(data => sum(items) === data.total)`)
                 │
                 ├── Valid ──> Release safely to downstream services or on-chain transaction pipelines
                 │
                 └── Invalid ──> [Targeted Self-Correction]
                                   Still invalid after 2 retries ──> Dead Letter Queue (DLQ) for human review
+----------------------------------------------------------------------------------------------------+

3. TypeScript + Zod Guardrails and a Fault-Tolerant AST Parser

This high-performance implementation uses TypeScript, zod, and streaming AST repair logic for Next.js, NestJS, or standalone AI Agent runtimes:

// ai-pipeline/src/guardrails/json-repair-engine.ts
import { z } from 'zod';

/**
 * Production JSON sanitizer: remove Markdown markers and surrounding prose
 */
export function sanitizeRawLLMResponse(raw: string): string {
  if (!raw) return '';

  let cleaned = raw.trim();

  // 1. Extract the payload from ```json ... ``` or ``` ... ```
  const markdownCodeBlockRegex = /```(?:json)?\s*([\s\S]*?)\s*```/i;
  const match = cleaned.match(markdownCodeBlockRegex);
  if (match && match[1]) {
    cleaned = match[1].trim();
  }

  // 2. Find the first JSON opening delimiter ({ or [) and last closing delimiter (} or ])
  const firstBrace = cleaned.indexOf('{');
  const firstBracket = cleaned.indexOf('[');
  let startIdx = -1;

  if (firstBrace !== -1 && firstBracket !== -1) {
    startIdx = Math.min(firstBrace, firstBracket);
  } else {
    startIdx = firstBrace !== -1 ? firstBrace : firstBracket;
  }

  const lastBrace = cleaned.lastIndexOf('}');
  const lastBracket = cleaned.lastIndexOf(']');
  const endIdx = Math.max(lastBrace, lastBracket);

  if (startIdx !== -1 && endIdx !== -1 && endIdx >= startIdx) {
    cleaned = cleaned.substring(startIdx, endIdx + 1);
  }

  return cleaned;
}

/**
 * Automatically close truncated JSON (AST recovery for Max Tokens truncation)
 */
export function repairTruncatedJSON(truncatedJson: string): string {
  let repaired = truncatedJson.trim();

  // If the output ends inside a string (odd quote count), append a double quote
  const quoteCount = (repaired.match(/(?<!\\)"/g) || []).length;
  if (quoteCount % 2 !== 0) {
    repaired += '"';
  }

  // Count unmatched braces and brackets
  const openBraces = (repaired.match(/\{/g) || []).length;
  const closeBraces = (repaired.match(/\}/g) || []).length;
  const openBrackets = (repaired.match(/\[/g) || []).length;
  const closeBrackets = (repaired.match(/\]/g) || []).length;

  // Remove a dangling comma (e.g. strip the final `,` from `{"name": "Alice",` before closing)
  repaired = repaired.replace(/,\s*$/, '');

  // Close unmatched brackets and braces
  for (let i = 0; i < openBrackets - closeBrackets; i++) {
    repaired += ']';
  }
  for (let i = 0; i < openBraces - closeBraces; i++) {
    repaired += '}';
  }

  return repaired;
}

Next, define a strict Zod schema with business Invariants:

// ai-pipeline/src/guardrails/schemas/payment-intent.schema.ts
import { z } from 'zod';
import { isAddress } from 'viem';

// Business invariants: item sum must equal total; recipients must use valid EVM checksum addresses
export const PaymentIntentItemSchema = z.object({
  recipient: z.string().refine(val => isAddress(val), {
    message: 'Recipient must be a valid 20-byte EVM address',
  }),
  rawAmountUnits: z.string().regex(/^\d+$/, 'rawAmountUnits must be an unsigned integer string'),
  description: z.string().min(1).max(256),
});

export const BatchPaymentIntentSchema = z.object({
  chainId: z.number().int().positive(),
  tokenAddress: z.string().refine(val => isAddress(val), {
    message: 'tokenAddress must be a valid EVM contract address',
  }),
  totalRawAmount: z.string().regex(/^\d+$/, 'totalRawAmount must be an integer string'),
  items: z.array(PaymentIntentItemSchema).min(1).max(50),
}).refine(data => {
  // Core invariant: verify item sums to prevent hallucinated overpayment or underpayment
  const sumOfItems = data.items.reduce((acc, item) => acc + BigInt(item.rawAmountUnits), 0n);
  return sumOfItems === BigInt(data.totalRawAmount);
}, {
  message: 'CRITICAL INVARIANT BREACH: Sum of batch items does not equal totalRawAmount',
  path: ['totalRawAmount'],
});

export type BatchPaymentIntent = z.infer<typeof BatchPaymentIntentSchema>;

4. Targeted Self-Correction

When the third layer detects a Schema error or broken invariant, simply failing the whole request back to the user creates a poor experience.

A production system should use a Targeted Patching Prompt: return Zod’s exact error paths and expected rules to the model so it can make microsecond-level local corrections instead of retrying blindly:

// ai-pipeline/src/guardrails/execution-pipeline.ts
import { sanitizeRawLLMResponse, repairTruncatedJSON } from './json-repair-engine';
import { BatchPaymentIntentSchema, BatchPaymentIntent } from './schemas/payment-intent.schema';

export async function executeGuardedAIExtraction(
  initialPrompt: string,
  llmInvoker: (prompt: string) => Promise<string>,
  maxRetries = 2
): Promise<BatchPaymentIntent> {
  let currentPrompt = initialPrompt;

  for (let attempt = 1; attempt <= maxRetries + 1; attempt++) {
    const rawOutput = await llmInvoker(currentPrompt);

    // Step 1: Sanitize with regex
    let sanitized = sanitizeRawLLMResponse(rawOutput);

    // Step 2: Try normal parsing; on failure, repair truncated AST
    let parsedJson: any;
    try {
      parsedJson = JSON.parse(sanitized);
    } catch (parseErr) {
      try {
        const repaired = repairTruncatedJSON(sanitized);
        parsedJson = JSON.parse(repaired);
      } catch (fatalSyntaxErr) {
        if (attempt > maxRetries) {
          throw new Error(`Fatal JSON syntax error after ${maxRetries} retries: ${rawOutput}`);
        }
        // Build a syntax-targeted repair prompt
        currentPrompt = `${initialPrompt}\n\n【ERROR】Your previous output was malformed JSON. Fix syntax and return ONLY valid JSON: ${rawOutput}`;
        continue;
      }
    }

    // Step 3: Enforce Zod business invariants
    const validationResult = BatchPaymentIntentSchema.safeParse(parsedJson);

    if (validationResult.success) {
      return validationResult.data; // Validation passed; release the result
    } else {
      if (attempt > maxRetries) {
        throw new Error(
          `Semantic invariant validation failed: ${JSON.stringify(validationResult.error.format())}`
        );
      }

      // Build targeted semantic correction feedback identifying the invalid fields
      const errorFeedback = validationResult.error.issues
        .map(issue => `Field [${issue.path.join('.')}] failed: ${issue.message}`)
        .join('; ');

      console.warn(`[Guardrails] Attempt ${attempt} failed validation: ${errorFeedback}. Retrying with patch...`);

      currentPrompt = `${initialPrompt}\n\n【CORRECTION REQUIRED】Your previous response violated strict business invariants: ${errorFeedback}. Previous payload: ${JSON.stringify(parsedJson)}. Fix the numbers/fields and return corrected JSON only.`;
    }
  }

  throw new Error('Exhausted all guardrail remediation retries.');
}

5. Comparison: Prompting, JSON Mode, and Three-Tier Guardrails

The approaches differ in reliability, throughput, and protection against financial loss:

Dimension A: Prompt Only (“Return JSON”) B: OpenAI / DeepSeek JSON Mode Only C: Three-Tier Deterministic Guardrails
Markdown contamination Very high (models often add “`json markers) Moderate (occasional Markdown leakage) Fully immune (compiled regex stripping)
Token-truncation tolerance None (truncation always creates syntax errors) None (truncation corrupts syntax) Excellent (AST bracket balancing repairs output)
Semantic invariants No defense (cannot detect mismatched item totals) No defense (API guarantees syntax, not business logic) Code-enforced assertions (Zod Refine rejects violations in milliseconds)
Recovery Blind full retries, doubling API cost Unknown errors left to developer try-catch Targeted correction, identifying the specific unbalanced field
Downstream safety Very low (can cascade through microservice clusters) Moderate Financial-grade safety (output strictly matches TypeScript compile-time types)

6. Integration with Agent Wallets and Multi-Tenant SaaS

Deterministic JSON extraction and semantic guards connect nondeterministic AI with deterministic financial systems, acting as a gate in several deployments:

  1. Agent wallet orchestration: Combined with Web3 agent calldata parameter allowlists and ERC-4337 dual-checkpoint spending guards, model-extracted transfer intents must first pass the Zod invariants here. rawAmountUnits must contain no floating-point strings before the intent becomes an on-chain UserOperation;
  2. High-concurrency order APIs: With idempotent order APIs scaled from SQLite to MySQL, AI-parsed product records have their structure and amount consistency validated before entering distributed transactions, preventing ledger contamination.

Architecture Consulting and Source-Code Delivery

Deterministic JSON Extraction & Schema Guardrails for Production AI Pipelines

If you are developing industry-specific agent workflows, AI reconciliation, customer-service ordering, or complex production RAG pipelines and want to prevent occasional model failures from cascading through downstream services: