Request basics

A chat request is an ordered list of messages. Each message has a role; user content can combine text with image, video, or audio parts. Keep this structure explicit when building reusable Python functions. See Python setup for client installation and configuration.

Build a reusable image request

Install perceptron>=0.4.0 and set PERCEPTRON_API_KEY. This complete example reuses the same client and returns the conversation so a follow-up can include its original image:

import os

from perceptron import Client, image

client = Client(
    api_key=os.environ["PERCEPTRON_API_KEY"],
    provider="perceptron",
    timeout=60.0,
)


def answer(messages):
    response = client.chat.completions.create(
        model="perceptron-mk1.5",
        messages=messages,
        max_completion_tokens=1024,
    )
    choice = response.choices[0]
    if choice.finish_reason != "stop" or choice.message.tool_calls:
        raise RuntimeError(f"No completed text answer: {choice.finish_reason}")
    return choice.message.to_dict()


def describe(image_url, question):
    messages = [
        {"role": "system", "content": "Answer using visible evidence. State uncertainty."},
        {
            "role": "user",
            "content": [
                image(image_url),
                {"type": "text", "text": question},
            ],
        },
    ]
    messages.append(answer(messages))
    return messages


history = describe(
    "https://raw.githubusercontent.com/perceptron-ai-inc/perceptron/"
    "main/cookbook/_shared/assets/capabilities/qna/studio_scene.webp",
    "Describe this coastal scene and the visible objects.",
)
print(history[-1].get("content") or "")

history.append({"role": "user", "content": "Where is the sailboat relative to the shoreline?"})
history.append(answer(history))
print(history[-1].get("content") or "")

The API receives the history supplied on each call. A follow-up does not automatically recover the previous image or answer. Append the full assistant message to retain its answer, reasoning_content, and any tool_calls in your client-side history. For requests with tool declarations, tool calls, or tool results, send reasoning_content back with the assistant turn. On ordinary requests without tools or tool traffic, the gateway omits historical reasoning_content from the model’s input. The previous media and assistant answer still need to be included for a follow-up.

Choose message roles and content parts

Part	Purpose
`system` message	Application instructions, such as output expectations and uncertainty handling.
`user` message	A question, media, and any labeled reference examples.
`assistant` message	A previous model turn, including its calls or reasoning when present.
`tool` message	The result of a model-issued function call, with its matching `tool_call_id`.
`image_url` / `video_url`	Media supplied by URL or a supported data URL.
`image_file_id` / `video_file_id`	Media previously uploaded through the Files API.
`video_frames`	A timestamped group of frames; see video inputs.
`input_audio`	Bare base64 audio bytes with `format: "wav"`, `"mp3"`, or `"flac"`.
`audio_url` / `audio_file_id`	Audio supplied by HTTP(S)/data URL or a previously uploaded file; see Audio.

A text part containing a URL is only text. Use the matching media part to make the content available as an image, video, or audio recording. Audio parts belong in user messages. The SDK’s image(), video(), and audio() helpers encode local paths or bytes inline. They also accept URLs, file_id=..., or a File returned by client.files.upload(). They do not upload through the Files API automatically. Order matters for asset selectors. User media and tool-returned images share one zero-based sequence that continues across messages and turns. An annotation’s asset_idx refers to media available at that point in the conversation; media added later cannot change its target. Removing an earlier media occurrence changes subsequent indices.

Choose how to consume the answer

Desired result	Request and handling
Text	Give a clear instruction; read `message.content` after checking completion.
Spatial or temporal annotations	Pass `vision_config={"annotation_format": "box"}`; parse and validate annotation markup.
A JSON record	Supply a JSON Schema through `response_format` and validate the result.
A short pattern-constrained answer	Pass `regex` directly and validate the completed text.
A function call	Declare `tools`, then use the caller-executed tool loop.

For example, an ordinary box request adds vision_config={"annotation_format": "box"} to client.chat.completions.create(...) and asks for the objects to locate. Use reasoning_effort as a top-level argument when reasoning is needed. Output guidance does not establish that a prediction is correct. Check selectors, geometry, schemas, and task-specific constraints before using the result. Tool declarations and JSON Schema/regex final-answer constraints cannot be combined in the same request; the structured-report example shows the separate final step.

Inspect a request before sending it

Build messages as an ordinary Python value so you can inspect roles, content types, ordering, and parameters in a debugger before calling the client. Keep API keys, private URLs, base64 media, and sensitive conversation content out of routine logs. Record the model, completion status, and reported usage for troubleshooting. Bound conversation growth and output with the tokenization guide, and use Error messages for failures and retries.

Get Started

Capabilities

Developer Guides

Scaling & deployment

Best practices

Build a reusable image request

Choose message roles and content parts

Choose how to consume the answer

Inspect a request before sending it

​Build a reusable image request

​Choose message roles and content parts

​Choose how to consume the answer

​Inspect a request before sending it

Build a reusable image request

Choose message roles and content parts

Choose how to consume the answer

Inspect a request before sending it