> ## Documentation Index
> Fetch the complete documentation index at: https://docs.perceptron.inc/llms.txt
> Use this file to discover all available pages before exploring further.

# Structured outputs

> Choose native annotations, constrained final answers, or function arguments for your application.

Use the format that matches how your application will consume the result:

| Need                                         | Format                                             | Returned in                                                       |
| -------------------------------------------- | -------------------------------------------------- | ----------------------------------------------------------------- |
| Spatial or temporal grounding                | Native point, box, polygon, clip, and track markup | `message.content`                                                 |
| A final answer with a known data shape       | JSON Schema through `response_format`              | `message.content` as a JSON string                                |
| A short answer matching a pattern            | Top-level `regex`                                  | `message.content`                                                 |
| A request to execute an application function | Function definitions in `tools`                    | `message.tool_calls`, with argument strings to parse and validate |

Native annotations are documented in [annotation format](/perceptron-mk1.5/concepts/annotations). They are not automatically converted to JSON objects by the API. Function-call schemas are also separate from final-response constraints: `function.strict` is accepted but does not enforce argument validity. `function.arguments` is a string that can contain invalid JSON; parse it and validate it against your function schema before execution.

## Request a JSON Schema response

Install `perceptron>=0.4.0` and `jsonschema`, and set `PERCEPTRON_API_KEY`. This example constrains the answer and validates it locally before use:

```python theme={null}
import json

from jsonschema import validate
from perceptron import Client, image

client = Client()
schema = {
    "type": "object",
    "properties": {
        "description": {"type": "string"},
        "visible_objects": {"type": "array", "items": {"type": "string"}},
        "unreadable_text_present": {"type": "boolean"},
    },
    "required": ["description", "visible_objects", "unreadable_text_present"],
    "additionalProperties": False,
}
messages = [{
    "role": "user",
    "content": [
        image(
            "https://raw.githubusercontent.com/perceptron-ai-inc/perceptron/"
            "main/cookbook/_shared/assets/capabilities/qna/studio_scene.webp"
        ),
        "Describe this scene using the requested JSON fields.",
    ],
}]
response = client.chat.completions.create(
    model="perceptron-mk1.5",
    messages=messages,
    max_completion_tokens=1024,
    response_format={
        "type": "json_schema",
        "json_schema": {"name": "scene", "strict": True, "schema": schema},
    },
)
choice = response.choices[0]
if choice.finish_reason != "stop":
    raise RuntimeError(f"Incomplete structured answer: {choice.finish_reason}")
result = json.loads(choice.message.content or "")
validate(instance=result, schema=schema)
print(result)
```

Define required fields and allowed values explicitly. Use numeric bounds when a field represents a bounded quantity, and include an appropriate unknown or empty value when the image may not contain the requested information. A valid schema constrains the structure; it does not establish that the model's interpretation is correct.

## Define the response with Pydantic

Pydantic v2 can generate the JSON Schema and validate the response with the same Python model. Install it with `pip install "pydantic>=2,<3"`, then run this after defining `client` and the image `messages` above:

```python theme={null}
from typing import Literal

from pydantic import BaseModel, ConfigDict, Field


class SceneAnalysis(BaseModel):
    model_config = ConfigDict(extra="forbid")

    scene_type: Literal["indoor", "outdoor", "mixed", "unknown"]
    main_subjects: list[str] = Field(description="Subjects visible in the image; an empty list if none.")
    time_of_day: Literal["morning", "afternoon", "evening", "night", "unknown"] = Field(
        description="Use unknown when the image does not establish the time of day."
    )


response = client.chat.completions.create(
    model="perceptron-mk1.5",
    messages=messages + [{
        "role": "user",
        "content": "Analyze the scene using the requested fields. Do not guess the time of day.",
    }],
    max_completion_tokens=1024,
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "scene_analysis",
            "strict": True,
            "schema": SceneAnalysis.model_json_schema(),
        },
    },
)
choice = response.choices[0]
if choice.finish_reason != "stop" or choice.message.tool_calls:
    raise RuntimeError("The model did not return a complete scene analysis")
analysis = SceneAnalysis.model_validate_json(choice.message.content or "", strict=True)
print(analysis.scene_type)
print(analysis.main_subjects)
print(analysis.time_of_day)
```

All three fields are required, `extra="forbid"` rejects unexpected fields, and the literal types restrict allowed values. `model_validate_json` raises `ValidationError` for malformed JSON or values that do not match the model. Handle that failure before storing the result or using it in another workflow.

## Use regex for a short answer

Run this after the client and image messages above. Pass `regex` directly to the SDK's message API:

```python theme={null}
import re

response = client.chat.completions.create(
    model="perceptron-mk1.5",
    messages=messages + [{"role": "user", "content": "Is a desk visible? Answer yes or no."}],
    max_completion_tokens=32,
    regex="yes|no",
)
choice = response.choices[0]
answer = choice.message.content or ""
if choice.finish_reason != "stop" or re.fullmatch(r"yes|no", answer) is None:
    raise ValueError("The model did not return a complete yes/no answer")
print(answer)
```

Use one response constraint per request. Keep regex patterns small and test them against the answer shapes you intend to allow.

## Combine tools with a constrained final answer

A request cannot declare non-empty function `tools` together with a `json_schema` response format or `regex`. The SDK raises `BadRequestError` for this combination before sending the request.

First complete the [tool loop](/perceptron-mk1.5/guides/tool-calling). Append every tool result to the conversation. Then request the final answer with your response constraint and **omit `tools`**, `tool_choice`, and `parallel_tool_calls` for that request. The completed tool history can remain in `messages`.

Save the function-calling example as `inventory.py`. Its `run()` function returns the completed conversation, including the lookup results and assistant answer. Save the following as `inventory_report.py` in the same directory:

```python theme={null}
import json

from jsonschema import validate

from inventory import client, run


stock_schema = {
    "type": "object",
    "properties": {
        "quantity": {"type": "integer", "minimum": 0},
        "in_stock": {"type": "boolean"},
    },
    "required": ["quantity", "in_stock"],
    "additionalProperties": False,
}
report_schema = {
    "type": "object",
    "properties": {"BALL-RED": stock_schema, "BALL-BLUE": stock_schema},
    "required": ["BALL-RED", "BALL-BLUE"],
    "additionalProperties": False,
}

history = run()
history.append({
    "role": "user",
    "content": (
        "Turn the inventory lookup results into a JSON report for BALL-RED and BALL-BLUE. "
        "Use the quantities already retrieved. A product is in stock when its quantity is positive."
    ),
})
response = client.chat.completions.create(
    model="perceptron-mk1.5",
    messages=history,
    max_completion_tokens=1024,
    response_format={
        "type": "json_schema",
        "json_schema": {"name": "inventory_report", "strict": True, "schema": report_schema},
    },
)
choice = response.choices[0]
if choice.finish_reason != "stop" or choice.message.tool_calls:
    raise RuntimeError("The model did not return a complete inventory report")
report = json.loads(choice.message.content or "")
validate(instance=report, schema=report_schema)
print(json.dumps(report, indent=2))
```

With `PERCEPTRON_API_KEY` set as in the function-calling guide, install the dependencies and run the report script:

```bash theme={null}
pip install "perceptron>=0.4.0" jsonschema
python inventory_report.py
```

The script runs the inventory lookup workflow, then makes one additional request for the report. With the example inventory, the report should contain:

```json theme={null}
{
  "BALL-RED": {"quantity": 12, "in_stock": true},
  "BALL-BLUE": {"quantity": 0, "in_stock": false}
}
```

Always validate function names and arguments in your application before dispatch. A constrained final answer does not retroactively validate a tool call or its result.

## Validate completed streams

With `stream=True`, individual `delta.content` chunks can end inside a JSON string or annotation tag. Use `stream.get_final_completion()` to assemble the response, check `response.complete` and the finish reason, then parse and validate the final text. Treat `finish_reason: "length"`, API errors, or a lost connection as incomplete output.

The SDK requests usage by default for Perceptron streams. Usage can arrive in a final chunk with `choices: []`; the assembled completion includes it. If you process chunks directly, consume usage independently of choice deltas. The [video tracking stream example](/perceptron-mk1.5/capabilities/video-tracking#stream-a-tracking-answer) demonstrates streaming; replace annotation parsing with JSON or regex validation for constrained responses.