
Engineers introducing LLMs into production AI Agents or enterprise automation pipelines often experience the same reversal of fortune.
During local development and demos, a System Prompt says, “Return only valid JSON, with no extra text.” With gpt-4o or Claude 3.5 Sonnet, all 50 test samples produce clean JSON. Confident, the team deploys it to parse commercial invoices, extract on-chain transaction intents, and dispatch orders to microservices.
By day 3 of handling tens of thousands of real requests daily, the downstream order database is broadly stalled:
- One response reaches the model’s maximum Token limit and is truncated at
{"amount": 1500, "recipient": "0x89. Unclosed delimiters makeJSON.parse()throw an uncaught syntax exception; - Another response unexpectedly wraps its JSON in a Markdown block,
json \n ... \n, crashing strict deserialization middleware; - Worse, “silent semantic contamination” produces syntactically valid JSON with the wrong financial representation. The model expresses
1.5 USDCas the quoted string"1.5", and a downstream service performs JavaScript concatenation,"100" + "1.5" = "1001.5", creating an overpayment dozens of times larger than intended.
LLMs are probabilistic sampling systems; enterprise backend services and smart contracts are deterministic state machines.
Prompt engineering alone cannot guarantee perfect structured output. A reliable pipeline built on probabilistic responses needs three deterministic guardrail layers: Constrained Decoding → Syntactic Repair → Semantic Invariants.
1. Three Destructive Forms of Structured-Output Failure
Error logs from millions of daily inference calls show three progressively deeper failure levels:
+----------------------------------------------------------------------------------------------------+
| Three LLM Structured-Output Failure Levels |
+----------------------------------------------------------------------------------------------------+
[ Syntactic Breakage ]
- Token truncation leaves a missing `}` or `]`
- Extra prose ("Here is the JSON you requested:") or unclosed Markdown fences
- Unescaped quotes inside strings (e.g. `"description": "He said "Hello""`)
- Result: `JSON.parse()` throws SyntaxError and the service returns 500.
--------------------------------------------------------------------------------------------------
[ Schema Drift ]
- Hallucinated synonyms for field names (`recipientAddress` becomes `to_address` or `wallet`)
- A single-element array becomes a standalone object
- Integer fields become floats or strings
- Result: TypeScript assumptions break, causing downstream `Cannot read properties of undefined`.
--------------------------------------------------------------------------------------------------
[ Semantic Invariant Breach ]
- Valid structure and types, invalid business logic: item amounts do not equal the total
- Token Decimals in a transfer intent do not match the target chain
- Authorized amount exceeds the system's hard safety cap
- Result: silent financial failures and loss-of-funds vulnerabilities.
+----------------------------------------------------------------------------------------------------+
2. Three-Tier Deterministic Guardrails
In fone.cc full-stack delivery, no model response is directly parsed and trusted. Every AI Agent inference passes through a strict validation funnel:
+----------------------------------------------------------------------------------------------------+
| Three-Tier Deterministic Guardrail Pipeline |
+----------------------------------------------------------------------------------------------------+
[ Layer 1: Constrained Decoding & Structured Outputs ]
- Require the official API's `response_format: { type: "json_schema", strict: true }`
- Use context-free grammar (CFG) and Logit bias to block illegal token sampling at generation time
│
▼ (For open models without Strict Schema support or unexpected Token truncation)
[ Layer 2: Syntactic Repair & JSON Fixer ]
- Strip Markdown fences (````json ... ````) and surrounding prose with compiled regex
- Close incomplete quotes, brackets, and truncated commas (Partial JSON AST Recovery)
│
▼
[ Layer 3: Zod / Pydantic Semantic Invariant Gate ]
- Strict field types and formats (EVM address regex, UTC ISO timestamps, positive-integer bounds)
- Cross-field assertions (`refine(data => sum(items) === data.total)`)
│
├── Valid ──> Release safely to downstream services or on-chain transaction pipelines
│
└── Invalid ──> [Targeted Self-Correction]
Still invalid after 2 retries ──> Dead Letter Queue (DLQ) for human review
+----------------------------------------------------------------------------------------------------+
3. TypeScript + Zod Guardrails and a Fault-Tolerant AST Parser
This high-performance implementation uses TypeScript, zod, and streaming AST repair logic for Next.js, NestJS, or standalone AI Agent runtimes:
// ai-pipeline/src/guardrails/json-repair-engine.ts
import { z } from 'zod';
/**
* Production JSON sanitizer: remove Markdown markers and surrounding prose
*/
export function sanitizeRawLLMResponse(raw: string): string {
if (!raw) return '';
let cleaned = raw.trim();
// 1. Extract the payload from ```json ... ``` or ``` ... ```
const markdownCodeBlockRegex = /```(?:json)?\s*([\s\S]*?)\s*```/i;
const match = cleaned.match(markdownCodeBlockRegex);
if (match && match[1]) {
cleaned = match[1].trim();
}
// 2. Find the first JSON opening delimiter ({ or [) and last closing delimiter (} or ])
const firstBrace = cleaned.indexOf('{');
const firstBracket = cleaned.indexOf('[');
let startIdx = -1;
if (firstBrace !== -1 && firstBracket !== -1) {
startIdx = Math.min(firstBrace, firstBracket);
} else {
startIdx = firstBrace !== -1 ? firstBrace : firstBracket;
}
const lastBrace = cleaned.lastIndexOf('}');
const lastBracket = cleaned.lastIndexOf(']');
const endIdx = Math.max(lastBrace, lastBracket);
if (startIdx !== -1 && endIdx !== -1 && endIdx >= startIdx) {
cleaned = cleaned.substring(startIdx, endIdx + 1);
}
return cleaned;
}
/**
* Automatically close truncated JSON (AST recovery for Max Tokens truncation)
*/
export function repairTruncatedJSON(truncatedJson: string): string {
let repaired = truncatedJson.trim();
// If the output ends inside a string (odd quote count), append a double quote
const quoteCount = (repaired.match(/(?<!\\)"/g) || []).length;
if (quoteCount % 2 !== 0) {
repaired += '"';
}
// Count unmatched braces and brackets
const openBraces = (repaired.match(/\{/g) || []).length;
const closeBraces = (repaired.match(/\}/g) || []).length;
const openBrackets = (repaired.match(/\[/g) || []).length;
const closeBrackets = (repaired.match(/\]/g) || []).length;
// Remove a dangling comma (e.g. strip the final `,` from `{"name": "Alice",` before closing)
repaired = repaired.replace(/,\s*$/, '');
// Close unmatched brackets and braces
for (let i = 0; i < openBrackets - closeBrackets; i++) {
repaired += ']';
}
for (let i = 0; i < openBraces - closeBraces; i++) {
repaired += '}';
}
return repaired;
}
Next, define a strict Zod schema with business Invariants:
// ai-pipeline/src/guardrails/schemas/payment-intent.schema.ts
import { z } from 'zod';
import { isAddress } from 'viem';
// Business invariants: item sum must equal total; recipients must use valid EVM checksum addresses
export const PaymentIntentItemSchema = z.object({
recipient: z.string().refine(val => isAddress(val), {
message: 'Recipient must be a valid 20-byte EVM address',
}),
rawAmountUnits: z.string().regex(/^\d+$/, 'rawAmountUnits must be an unsigned integer string'),
description: z.string().min(1).max(256),
});
export const BatchPaymentIntentSchema = z.object({
chainId: z.number().int().positive(),
tokenAddress: z.string().refine(val => isAddress(val), {
message: 'tokenAddress must be a valid EVM contract address',
}),
totalRawAmount: z.string().regex(/^\d+$/, 'totalRawAmount must be an integer string'),
items: z.array(PaymentIntentItemSchema).min(1).max(50),
}).refine(data => {
// Core invariant: verify item sums to prevent hallucinated overpayment or underpayment
const sumOfItems = data.items.reduce((acc, item) => acc + BigInt(item.rawAmountUnits), 0n);
return sumOfItems === BigInt(data.totalRawAmount);
}, {
message: 'CRITICAL INVARIANT BREACH: Sum of batch items does not equal totalRawAmount',
path: ['totalRawAmount'],
});
export type BatchPaymentIntent = z.infer<typeof BatchPaymentIntentSchema>;
4. Targeted Self-Correction
When the third layer detects a Schema error or broken invariant, simply failing the whole request back to the user creates a poor experience.
A production system should use a Targeted Patching Prompt: return Zod’s exact error paths and expected rules to the model so it can make microsecond-level local corrections instead of retrying blindly:
// ai-pipeline/src/guardrails/execution-pipeline.ts
import { sanitizeRawLLMResponse, repairTruncatedJSON } from './json-repair-engine';
import { BatchPaymentIntentSchema, BatchPaymentIntent } from './schemas/payment-intent.schema';
export async function executeGuardedAIExtraction(
initialPrompt: string,
llmInvoker: (prompt: string) => Promise<string>,
maxRetries = 2
): Promise<BatchPaymentIntent> {
let currentPrompt = initialPrompt;
for (let attempt = 1; attempt <= maxRetries + 1; attempt++) {
const rawOutput = await llmInvoker(currentPrompt);
// Step 1: Sanitize with regex
let sanitized = sanitizeRawLLMResponse(rawOutput);
// Step 2: Try normal parsing; on failure, repair truncated AST
let parsedJson: any;
try {
parsedJson = JSON.parse(sanitized);
} catch (parseErr) {
try {
const repaired = repairTruncatedJSON(sanitized);
parsedJson = JSON.parse(repaired);
} catch (fatalSyntaxErr) {
if (attempt > maxRetries) {
throw new Error(`Fatal JSON syntax error after ${maxRetries} retries: ${rawOutput}`);
}
// Build a syntax-targeted repair prompt
currentPrompt = `${initialPrompt}\n\n【ERROR】Your previous output was malformed JSON. Fix syntax and return ONLY valid JSON: ${rawOutput}`;
continue;
}
}
// Step 3: Enforce Zod business invariants
const validationResult = BatchPaymentIntentSchema.safeParse(parsedJson);
if (validationResult.success) {
return validationResult.data; // Validation passed; release the result
} else {
if (attempt > maxRetries) {
throw new Error(
`Semantic invariant validation failed: ${JSON.stringify(validationResult.error.format())}`
);
}
// Build targeted semantic correction feedback identifying the invalid fields
const errorFeedback = validationResult.error.issues
.map(issue => `Field [${issue.path.join('.')}] failed: ${issue.message}`)
.join('; ');
console.warn(`[Guardrails] Attempt ${attempt} failed validation: ${errorFeedback}. Retrying with patch...`);
currentPrompt = `${initialPrompt}\n\n【CORRECTION REQUIRED】Your previous response violated strict business invariants: ${errorFeedback}. Previous payload: ${JSON.stringify(parsedJson)}. Fix the numbers/fields and return corrected JSON only.`;
}
}
throw new Error('Exhausted all guardrail remediation retries.');
}
5. Comparison: Prompting, JSON Mode, and Three-Tier Guardrails
The approaches differ in reliability, throughput, and protection against financial loss:
| Dimension | A: Prompt Only (“Return JSON”) | B: OpenAI / DeepSeek JSON Mode Only | C: Three-Tier Deterministic Guardrails |
|---|---|---|---|
| Markdown contamination | Very high (models often add “`json markers) | Moderate (occasional Markdown leakage) | Fully immune (compiled regex stripping) |
| Token-truncation tolerance | None (truncation always creates syntax errors) | None (truncation corrupts syntax) | Excellent (AST bracket balancing repairs output) |
| Semantic invariants | No defense (cannot detect mismatched item totals) | No defense (API guarantees syntax, not business logic) | Code-enforced assertions (Zod Refine rejects violations in milliseconds) |
| Recovery | Blind full retries, doubling API cost | Unknown errors left to developer try-catch | Targeted correction, identifying the specific unbalanced field |
| Downstream safety | Very low (can cascade through microservice clusters) | Moderate | Financial-grade safety (output strictly matches TypeScript compile-time types) |
6. Integration with Agent Wallets and Multi-Tenant SaaS
Deterministic JSON extraction and semantic guards connect nondeterministic AI with deterministic financial systems, acting as a gate in several deployments:
- Agent wallet orchestration: Combined with Web3 agent calldata parameter allowlists and ERC-4337 dual-checkpoint spending guards, model-extracted transfer intents must first pass the Zod invariants here.
rawAmountUnitsmust contain no floating-point strings before the intent becomes an on-chain UserOperation; - High-concurrency order APIs: With idempotent order APIs scaled from SQLite to MySQL, AI-parsed product records have their structure and amount consistency validated before entering distributed transactions, preventing ledger contamination.
Architecture Consulting and Source-Code Delivery

If you are developing industry-specific agent workflows, AI reconciliation, customer-service ordering, or complex production RAG pipelines and want to prevent occasional model failures from cascading through downstream services:
- Visit Fone client deliveries and system demos to explore our production deterministic AI pipelines and guardrails;
- Read our 12-Point Engineering and Security Audit from Prototype to Production;
- Book a one-to-one technical discussion through the Fone architecture consulting channel for a delivery package with Zod semantic-validation templates, AST repair middleware, and self-correction prompts.