Structured outputs

Use the format that matches how your application will consume the result:

Need	Format	Returned in
Spatial or temporal grounding	Native point, box, polygon, clip, and track markup	`message.content`
A final answer with a known data shape	JSON Schema through `response_format`	`message.content` as a JSON string
A short answer matching a pattern	Top-level `regex`	`message.content`
A request to execute an application function	Function definitions in `tools`	`message.tool_calls`, with argument strings to parse and validate

Native annotations are documented in annotation format. They are not automatically converted to JSON objects by the API. Function-call schemas are also separate from final-response constraints: function.strict is accepted but does not enforce argument validity. function.arguments is a string that can contain invalid JSON; parse it and validate it against your function schema before execution.

Request a JSON Schema response

Install perceptron>=0.4.0 and jsonschema, and set PERCEPTRON_API_KEY. This example constrains the answer and validates it locally before use:

import json

from jsonschema import validate
from perceptron import Client, image

client = Client()
schema = {
    "type": "object",
    "properties": {
        "description": {"type": "string"},
        "visible_objects": {"type": "array", "items": {"type": "string"}},
        "unreadable_text_present": {"type": "boolean"},
    },
    "required": ["description", "visible_objects", "unreadable_text_present"],
    "additionalProperties": False,
}
messages = [{
    "role": "user",
    "content": [
        image(
            "https://raw.githubusercontent.com/perceptron-ai-inc/perceptron/"
            "main/cookbook/_shared/assets/capabilities/qna/studio_scene.webp"
        ),
        "Describe this scene using the requested JSON fields.",
    ],
}]
response = client.chat.completions.create(
    model="perceptron-mk1.5",
    messages=messages,
    max_completion_tokens=1024,
    response_format={
        "type": "json_schema",
        "json_schema": {"name": "scene", "strict": True, "schema": schema},
    },
)
choice = response.choices[0]
if choice.finish_reason != "stop":
    raise RuntimeError(f"Incomplete structured answer: {choice.finish_reason}")
result = json.loads(choice.message.content or "")
validate(instance=result, schema=schema)
print(result)

Define required fields and allowed values explicitly. Use numeric bounds when a field represents a bounded quantity, and include an appropriate unknown or empty value when the image may not contain the requested information. A valid schema constrains the structure; it does not establish that the model’s interpretation is correct.

Define the response with Pydantic

Pydantic v2 can generate the JSON Schema and validate the response with the same Python model. Install it with pip install "pydantic>=2,<3", then run this after defining client and the image messages above:

from typing import Literal

from pydantic import BaseModel, ConfigDict, Field


class SceneAnalysis(BaseModel):
    model_config = ConfigDict(extra="forbid")

    scene_type: Literal["indoor", "outdoor", "mixed", "unknown"]
    main_subjects: list[str] = Field(description="Subjects visible in the image; an empty list if none.")
    time_of_day: Literal["morning", "afternoon", "evening", "night", "unknown"] = Field(
        description="Use unknown when the image does not establish the time of day."
    )


response = client.chat.completions.create(
    model="perceptron-mk1.5",
    messages=messages + [{
        "role": "user",
        "content": "Analyze the scene using the requested fields. Do not guess the time of day.",
    }],
    max_completion_tokens=1024,
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "scene_analysis",
            "strict": True,
            "schema": SceneAnalysis.model_json_schema(),
        },
    },
)
choice = response.choices[0]
if choice.finish_reason != "stop" or choice.message.tool_calls:
    raise RuntimeError("The model did not return a complete scene analysis")
analysis = SceneAnalysis.model_validate_json(choice.message.content or "", strict=True)
print(analysis.scene_type)
print(analysis.main_subjects)
print(analysis.time_of_day)

All three fields are required, extra="forbid" rejects unexpected fields, and the literal types restrict allowed values. model_validate_json raises ValidationError for malformed JSON or values that do not match the model. Handle that failure before storing the result or using it in another workflow.

Use regex for a short answer

Run this after the client and image messages above. Pass regex directly to the SDK’s message API:

import re

response = client.chat.completions.create(
    model="perceptron-mk1.5",
    messages=messages + [{"role": "user", "content": "Is a desk visible? Answer yes or no."}],
    max_completion_tokens=32,
    regex="yes|no",
)
choice = response.choices[0]
answer = choice.message.content or ""
if choice.finish_reason != "stop" or re.fullmatch(r"yes|no", answer) is None:
    raise ValueError("The model did not return a complete yes/no answer")
print(answer)

Use one response constraint per request. Keep regex patterns small and test them against the answer shapes you intend to allow.

Combine tools with a constrained final answer

A request cannot declare non-empty function tools together with a json_schema response format or regex. The SDK raises BadRequestError for this combination before sending the request. First complete the tool loop. Append every tool result to the conversation. Then request the final answer with your response constraint and omit tools, tool_choice, and parallel_tool_calls for that request. The completed tool history can remain in messages. Save the function-calling example as inventory.py. Its run() function returns the completed conversation, including the lookup results and assistant answer. Save the following as inventory_report.py in the same directory:

import json

from jsonschema import validate

from inventory import client, run


stock_schema = {
    "type": "object",
    "properties": {
        "quantity": {"type": "integer", "minimum": 0},
        "in_stock": {"type": "boolean"},
    },
    "required": ["quantity", "in_stock"],
    "additionalProperties": False,
}
report_schema = {
    "type": "object",
    "properties": {"BALL-RED": stock_schema, "BALL-BLUE": stock_schema},
    "required": ["BALL-RED", "BALL-BLUE"],
    "additionalProperties": False,
}

history = run()
history.append({
    "role": "user",
    "content": (
        "Turn the inventory lookup results into a JSON report for BALL-RED and BALL-BLUE. "
        "Use the quantities already retrieved. A product is in stock when its quantity is positive."
    ),
})
response = client.chat.completions.create(
    model="perceptron-mk1.5",
    messages=history,
    max_completion_tokens=1024,
    response_format={
        "type": "json_schema",
        "json_schema": {"name": "inventory_report", "strict": True, "schema": report_schema},
    },
)
choice = response.choices[0]
if choice.finish_reason != "stop" or choice.message.tool_calls:
    raise RuntimeError("The model did not return a complete inventory report")
report = json.loads(choice.message.content or "")
validate(instance=report, schema=report_schema)
print(json.dumps(report, indent=2))

With PERCEPTRON_API_KEY set as in the function-calling guide, install the dependencies and run the report script:

pip install "perceptron>=0.4.0" jsonschema
python inventory_report.py

The script runs the inventory lookup workflow, then makes one additional request for the report. With the example inventory, the report should contain:

{
  "BALL-RED": {"quantity": 12, "in_stock": true},
  "BALL-BLUE": {"quantity": 0, "in_stock": false}
}

Always validate function names and arguments in your application before dispatch. A constrained final answer does not retroactively validate a tool call or its result.

Validate completed streams

With stream=True, individual delta.content chunks can end inside a JSON string or annotation tag. Use stream.get_final_completion() to assemble the response, check response.complete and the finish reason, then parse and validate the final text. Treat finish_reason: "length", API errors, or a lost connection as incomplete output. The SDK requests usage by default for Perceptron streams. Usage can arrive in a final chunk with choices: []; the assembled completion includes it. If you process chunks directly, consume usage independently of choice deltas. The video tracking stream example demonstrates streaming; replace annotation parsing with JSON or regex validation for constrained responses.

Get Started

Capabilities

Developer Guides

Scaling & deployment

Best practices

Request a JSON Schema response

Define the response with Pydantic

Use regex for a short answer

Combine tools with a constrained final answer

Validate completed streams

​Request a JSON Schema response

​Define the response with Pydantic

​Use regex for a short answer

​Combine tools with a constrained final answer

​Validate completed streams

Request a JSON Schema response

Define the response with Pydantic

Use regex for a short answer

Combine tools with a constrained final answer

Validate completed streams