Asking a language model for JSON can sound simple: list the fields, request a format, and parse the answer. But text that is valid JSON is not automatically useful data. A dependable workflow separates two jobs: constrain the shape of the output, then check whether the values make sense for the application.
Define what “valid” means
There are two different validation layers. The first is structural: is the response valid JSON, and does it contain the expected fields and types? The second is semantic: do the values make sense under the rules of the business process? JSON Schema defines structure and constraints, but application-level semantic validation may still be required. This distinction is important because an output can match a schema while still failing the meaning required by your product.
This article therefore recommends treating JSON Schema as a structural contract, not as a final quality judgment. First describe what the application accepts. Then list the rules that the schema cannot express clearly and enforce those rules in application code.
Keep the schema small and explicit
The current JSON Schema specification is Draft 2020-12, according to the official specification. That does not mean every provider or API supports every feature in the specification. OpenAI’s documentation says that strict schema adherence supports only a subset of JSON Schema, while Gemini’s documentation says that Structured Outputs supports a subset of JSON Schema; see the OpenAI reference and the Gemini documentation. Keeping a schema shallow and provider-compatible is therefore this article’s design recommendation, not a universal compatibility guarantee.
Make required fields explicit, choose narrow types, and use allowed values when the categories are known. Gemini documentation recommends clear descriptions, strong typing, and explicit enums as techniques for improving structured-output reliability; see the documented guidance. Use a rule that rejects unexpected properties when the provider supports it, but do not assume that support merely because JSON Schema describes the option.
Here is a small editorial example, not a universal provider contract:
{
"type": "object",
"properties": {
"status": {"type": "string", "enum": ["new", "review", "closed"]},
"summary": {"type": "string"}
},
"required": ["status", "summary"]
}This example defines shape only. It does not prove that the summary is accurate or that the selected status is correct for the business context.
Use structured output controls instead of prompting alone
OpenAI documentation recommends json_schema-based Structured Outputs over the older json_object JSON mode when supported; see the Chat Completions API reference. Gemini documents its own Structured Outputs capability and the subset of JSON Schema it supports; see the Gemini documentation. In practice, use the provider’s structured-output interface to constrain the shape rather than relying only on a prompt such as “return JSON only.”
This does not mean the feature solves every problem. Build a path for an unsupported schema feature or a difference in provider capability. If part of the schema is incompatible, simplify it or split the operation into stages according to your own tests. Do not turn a compatibility test into a claim that every schema feature will work.
Validate after the response arrives
Separate processing into stages that can be diagnosed:
- Parse: convert the response into a JSON value and record parsing failure separately.
- Validate structure: run the value through a JSON Schema validator compatible with the schema you selected.
- Validate meaning: check application rules, such as relationships between fields, allowed internal values, or whether a date fits the transaction context.
- Choose an operational result: accept the record, send it to review, or retry according to a stated policy.
The second and third stages are not interchangeable. A field can be a valid string and pass the schema while still violating a business rule. Gemini documentation recommends application-level value validation because schema-compliant output can still be semantically incorrect; see the source guidance.
Make incomplete cases visible
Your system should distinguish among a refusal, a truncated response, an unsupported schema, invalid JSON, structural validation failure, and semantic validation failure. This is a design checklist proposed by this article, not a guaranteed feature list for any provider.
- Store the error reason and the schema version used.
- Do not silently replace a missing value with a guess.
- Set bounded, observable retry conditions.
- Send cases requiring judgment to a clearly defined review path.
If you retry, test whether the cause of failure actually changed. Repeating the same request without changing the relevant condition may be an unhelpful operational pattern; treat that as a design risk to measure in your environment.
Separate JSON validity from data confidence
Use a small schema to test structure, then add semantic tests covering expected values and ambiguous cases. Include missing fields, an invalid enum value, a correctly typed value that violates a business rule, and an incomplete response. Do not present the results of these tests as general proof of a model or provider’s performance; they are local evaluation tools for your workflow.
This article’s engineering recommendation is to keep the raw response, validation results, and accept-or-review decision linked in an audit record, while reducing sensitive data according to your own system policy. That is a design suggestion, not a security claim about any provider.
A practical summary
For reliable structured JSON, start with a small schema, make required fields, types, and allowed values explicit, use Structured Outputs when the provider supports it, and then perform both structural and semantic validation in application code. Remember that Draft 2020-12 is the current JSON Schema specification and that Structured Outputs implementations may support only subsets of it; consult the official specification, the OpenAI documentation, and the Gemini documentation. “Valid JSON” should be a checkable stage in the workflow, not the end of validation.