> ## Documentation Index
> Fetch the complete documentation index at: https://docs.perceptron.inc/llms.txt
> Use this file to discover all available pages before exploring further.

# Chat Completions

## Overview

The Chat Completions API is fully compatible with OpenAI's chat completions specification, supporting both text-only and multimodal (vision) requests. Use it to generate responses from Perceptron Mk1.

Perceptron Mk1 triggers thinking and structured grounding through the typed `vision_config` body field.

## `vision_config`

For `perceptron-mk1`, pass a top-level `vision_config` object alongside `messages`:

| Field                  | Values                                       | Purpose                                                              |
| ---------------------- | -------------------------------------------- | -------------------------------------------------------------------- |
| `annotation_format`    | `"point"` / `"box"` / `"polygon"` / `"clip"` | Grounded output format. `clip` is video-only.                        |
| `enable_thinking`      | `true` / `false`                             | Chain-of-thought reasoning.                                          |
| `internal_tools.focus` | `true` / `false`                             | Enable the focus tool — model can zoom into regions. **Image only.** |

### When to `enable_thinking`

* **On** for text Q\&A, captioning, OCR, and video clipping (`annotation_format: "clip"`).
* **Off** for spatial detection (`annotation_format` in `"point"`, `"box"`, `"polygon"`).

### Example: Grounded detection

```bash theme={null}
curl https://api.perceptron.inc/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $PERCEPTRON_API_KEY" \
  -d '{
    "model": "perceptron-mk1",
    "messages": [
      { "role": "user",
        "content": [
          { "type": "image_url",
            "image_url": { "url": "<image-url>" } },
          { "type": "text",
            "text": "Find every worker wearing PPE." }
        ]
      }
    ],
    "vision_config": { "annotation_format": "box" }
  }'
```

### Example: Video clipping

```bash theme={null}
curl https://api.perceptron.inc/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $PERCEPTRON_API_KEY" \
  -d '{
    "model": "perceptron-mk1",
    "messages": [
      { "role": "user",
        "content": [
          { "type": "video_url",
            "video_url": { "url": "<video-url>" } },
          { "type": "text",
            "text": "Clip the moment the worker scans the package." }
        ]
      }
    ],
    "vision_config": { "annotation_format": "clip", "enable_thinking": true }
  }'
```

### Example: Video from pre-decoded frames

If you've already sampled frames client-side, pass them inline with a `video_frames` content part instead of a single `video_url`. Each frame carries an `image_url` (HTTP(S) URL or base64 data URL) and a `timestamp_ms` offset from the start of the clip. Provide between two and 256 frames, ordered by non-decreasing `timestamp_ms`. For optimal performance, follow a uniform sampling strategy aligned with the [Video Token Counting](/perceptron-mk1/guides/tokenization#video-token-counting) guide.

```bash theme={null}
curl https://api.perceptron.inc/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $PERCEPTRON_API_KEY" \
  -d '{
    "model": "perceptron-mk1",
    "messages": [
      { "role": "user",
        "content": [
          { "type": "video_frames",
            "video_frames": {
              "frames": [
                { "image_url": { "url": "<frame-url>" }, "timestamp_ms": 0 },
                { "image_url": { "url": "<frame-url>" }, "timestamp_ms": 500 }
              ]
            }
          },
          { "type": "text",
            "text": "What changes between these frames?" }
        ]
      }
    ],
    "vision_config": { "enable_thinking": true }
  }'
```

### Example: Image reasoning with focus

```bash theme={null}
curl https://api.perceptron.inc/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $PERCEPTRON_API_KEY" \
  -d '{
    "model": "perceptron-mk1",
    "messages": [
      { "role": "user",
        "content": [
          { "type": "image_url",
            "image_url": { "url": "<image-url>" } },
          { "type": "text",
            "text": "What is the serial number on the device in the corner?" }
        ]
      }
    ],
    "vision_config": {
      "enable_thinking": true,
      "internal_tools": { "focus": true }
    }
  }'
```

***

## Streaming

Set `"stream": true` to receive Server-Sent Events (SSE). To get token usage, also set `stream_options.include_usage: true` — when enabled, usage is attached to the final chunk (the one with `finish_reason: "stop"`), immediately before `data: [DONE]`.

```bash theme={null}
curl https://api.perceptron.inc/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $PERCEPTRON_API_KEY" \
  -d '{
    "model": "perceptron-mk1",
    "messages": [
      { "role": "user",
        "content": [
          { "type": "image_url",
            "image_url": { "url": "<image-url>" } },
          { "type": "text", "text": "Describe this scene in detail." }
        ]
      }
    ],
    "vision_config": { "enable_thinking": true },
    "stream": true,
    "stream_options": { "include_usage": true }
  }'
```

## Best Practices

1. **Thinking pairs well with text and clipping; not with spatial detection.** Turn `enable_thinking` on for text Q\&A, captioning, OCR, and `annotation_format: "clip"`. Turn it off for `"point"`, `"box"`, and `"polygon"`.
2. **Leave `temperature` unset.** The default is `0.0` (deterministic). Only set a non-zero value if you want more varied outputs.
3. **Image format**: HTTP(S) URLs and base64 data URLs are both supported. MIME types: `image/png`, `image/jpeg`, `image/webp`, `video/mp4`, `video/webm`.
4. **Inline frames vs. `video_url`**: Send a whole clip with `video_url`, or — if you've already sampled frames — pass them inline with `video_frames` (two to 256 frames, `timestamp_ms` non-decreasing). Inline frames give you precise control over exactly which frames the model sees; for optimal performance, sample uniformly in line with the [Video Token Counting](/perceptron-mk1/guides/tokenization#video-token-counting) guide.
5. **Token limits**: 32K context, 8K output.

## Limits

| Limit             | Value              |
| ----------------- | ------------------ |
| Requests          | 300/min            |
| Request body size | 20 MB              |
| Media upload      | 20 GB per 48 hours |

<Callout type="tip">
  For large images, resize client-side before uploading. See the [Tokenization guide](/perceptron-mk1/guides/tokenization) for optimization tips.
</Callout>


## OpenAPI

````yaml api-reference/openapi.json POST /v1/chat/completions
openapi: 3.1.0
info:
  title: Perceptron API
  contact:
    name: Perceptron API Support
    email: support@perceptron.inc
  version: 1.0.0
servers:
  - url: https://api.perceptron.inc
security: []
tags:
  - name: Chat Completions
    description: Chat completions API (OpenAI-compatible)
  - name: Detection
    description: Native Perceptron image detection API
  - name: Models
    description: Model listing and metadata API
  - name: Files
    description: File upload, listing, retrieval, and deletion API
paths:
  /v1/chat/completions:
    post:
      tags:
        - Chat Completions
      operationId: handle_chat_completions
      requestBody:
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/CreateChatCompletionRequest'
        required: true
      responses:
        '200':
          description: Chat completion generated successfully.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/CreateChatCompletionResponse'
            text/event-stream:
              schema:
                $ref: '#/components/schemas/CreateChatCompletionStreamResponse'
        '400':
          description: The request was invalid or could not be processed.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/OpenAIErrorResponse'
              example:
                error:
                  code: null
                  message: Model 'test' does not support video input
                  param: null
                  type: invalid_request_error
        '401':
          description: Authentication failed. Invalid or missing API key.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/OpenAIErrorResponse'
              example:
                error:
                  code: null
                  message: Invalid API key
                  param: null
                  type: authentication_error
        '429':
          description: >-
            Rate limit exceeded or quota exceeded. Too many requests or
            insufficient credits.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/OpenAIErrorResponse'
              example:
                error:
                  code: rate_limit_exceeded
                  message: >-
                    Organization rate limit exceeded (300 requests/minute).
                    Please retry after 30 seconds.
                  param: null
                  type: rate_limit_error
        '500':
          description: Internal server error while processing the request.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/OpenAIErrorResponse'
              example:
                error:
                  code: null
                  message: The server had an error while processing your request.
                  param: null
                  type: server_error
      security:
        - ApiKeyAuth: []
components:
  schemas:
    CreateChatCompletionRequest:
      type: object
      required:
        - messages
        - model
      properties:
        frequency_penalty:
          type:
            - number
            - 'null'
          format: float
          description: >-
            Positive values discourage the model from repeating previously used
            tokens.
          maximum: 2
          minimum: -2
        max_completion_tokens:
          type:
            - integer
            - 'null'
          format: int32
          description: >-
            Maximum number of completion tokens to generate.


            **Model-specific limits:**

            - **Isaac 0.1**: The combined total of input tokens and output
            tokens must not exceed 8192 tokens.
          minimum: 0
        messages:
          type: array
          items:
            $ref: '#/components/schemas/ChatCompletionRequestMessage'
          description: >-
            Conversation history listed in order. Supported roles: `system`,
            `user`, `assistant`.
        model:
          type: string
          description: >-
            The model to invoke. Available options: `isaac-0.1`, `isaac-0.2-1b`,
            `isaac-0.2-2b-preview`, `perceptron-mk1`.
        presence_penalty:
          type:
            - number
            - 'null'
          format: float
          description: Positive values encourage the model to introduce new concepts.
          maximum: 2
          minimum: -2
        regex:
          type:
            - string
            - 'null'
          description: Regex pattern for constrained generation.
        response_format:
          oneOf:
            - type: 'null'
            - $ref: '#/components/schemas/ChatCompletionResponseFormat'
              description: >-
                An object specifying the format that the model must output.

                Setting to `{ "type": "json_schema", "json_schema": {...} }`
                enables Structured Outputs

                which ensures the model will match your supplied JSON schema.
        stream:
          type:
            - boolean
            - 'null'
          description: >-
            Set to `true` for SSE streaming. When omitted, the API returns a
            single JSON response.
          default: false
        stream_options:
          oneOf:
            - type: 'null'
            - $ref: '#/components/schemas/ChatCompletionStreamOptions'
              description: >-
                Optional streaming flags. Token usage is always reported in the
                final chunk of a streaming response.
        temperature:
          type:
            - number
            - 'null'
          format: float
          description: >-
            Sampling temperature. Lower values yield deterministic replies;
            higher values explore more creative outputs.


            **Model-specific recommendations:**

            - **Isaac 0.1**: Default and recommended value is `0.0`.
          maximum: 2
          minimum: 0
        top_k:
          type:
            - integer
            - 'null'
          format: int32
          description: Top-k sampling. The model samples from the top k most likely tokens.
          minimum: 0
        top_p:
          type:
            - number
            - 'null'
          format: float
          description: >-
            Nucleus sampling probability. The model samples from the smallest
            token set whose cumulative probability exceeds this threshold.
          maximum: 1
          exclusiveMinimum: 0
        vision_config:
          oneOf:
            - type: 'null'
            - $ref: '#/components/schemas/VisionConfig'
              description: >-
                Perceptron vision-model controls (thinking, spatial output
                format,

                internal-tool toggles). Only supported on Perceptron-owned
                models.
    CreateChatCompletionResponse:
      type: object
      description: Non-streaming response body when `stream=false`.
      required:
        - id
        - choices
        - created
        - model
        - object
      properties:
        choices:
          type: array
          items:
            $ref: '#/components/schemas/ChatChoice'
        created:
          type: integer
          format: int64
          minimum: 0
        id:
          type: string
        model:
          type: string
        object:
          type: string
        usage:
          oneOf:
            - type: 'null'
            - $ref: '#/components/schemas/CompletionUsage'
    CreateChatCompletionStreamResponse:
      type: object
      description: SSE response payload when `stream=true`.
      required:
        - id
        - choices
        - created
        - model
        - object
      properties:
        choices:
          type: array
          items:
            $ref: '#/components/schemas/ChatChoiceStream'
        created:
          type: integer
          format: int64
          minimum: 0
        id:
          type: string
        model:
          type: string
        object:
          type: string
        usage:
          oneOf:
            - type: 'null'
            - $ref: '#/components/schemas/CompletionUsage'
    OpenAIErrorResponse:
      type: object
      description: OpenAI-compatible error response format.
      required:
        - error
      properties:
        error:
          $ref: '#/components/schemas/OpenAIErrorDetail'
    ChatCompletionRequestMessage:
      oneOf:
        - allOf:
            - $ref: '#/components/schemas/ChatCompletionRequestSystemMessage'
            - type: object
              required:
                - role
              properties:
                role:
                  type: string
                  enum:
                    - system
        - allOf:
            - $ref: '#/components/schemas/ChatCompletionRequestUserMessage'
            - type: object
              required:
                - role
              properties:
                role:
                  type: string
                  enum:
                    - user
        - allOf:
            - $ref: '#/components/schemas/ChatCompletionRequestAssistantMessage'
            - type: object
              required:
                - role
              properties:
                role:
                  type: string
                  enum:
                    - assistant
      description: >-
        Author role of the message as defined by the OpenAI Chat Completions
        spec.
    ChatCompletionResponseFormat:
      oneOf:
        - type: object
          required:
            - type
          properties:
            type:
              type: string
              enum:
                - text
        - type: object
          required:
            - json_schema
            - type
          properties:
            json_schema:
              $ref: '#/components/schemas/ChatCompletionResponseFormatJsonSchema'
            type:
              type: string
              enum:
                - json_schema
    ChatCompletionStreamOptions:
      type: object
      description: Streaming flags accepted by the API.
      properties:
        include_usage:
          type:
            - boolean
            - 'null'
          description: Include token usage metrics with streaming responses.
    VisionConfig:
      type: object
      description: >-
        Perceptron vision-model controls. Only honored by Perceptron-owned
        models.
      properties:
        annotation_format:
          oneOf:
            - type: 'null'
            - $ref: '#/components/schemas/AnnotationFormat'
              description: >-
                Annotation format the model should emit (`point`, `box`,
                `polygon`, or `clip`).
        enable_thinking:
          type:
            - boolean
            - 'null'
          description: Toggle reasoning ("thinking") on supported models.
        internal_tools:
          oneOf:
            - type: 'null'
            - $ref: '#/components/schemas/InternalTools'
              description: Internal-tool toggles.
    ChatChoice:
      type: object
      description: Individual completion sampled by the model.
      required:
        - index
        - message
      properties:
        finish_reason:
          oneOf:
            - type: 'null'
            - $ref: '#/components/schemas/FinishReason'
        index:
          type: integer
          format: int32
          minimum: 0
        message:
          $ref: '#/components/schemas/ChatCompletionResponseMessage'
    CompletionUsage:
      type: object
      description: Token accounting emitted with every completion.
      required:
        - prompt_tokens
        - completion_tokens
        - total_tokens
      properties:
        completion_tokens:
          type: integer
          format: int32
          minimum: 0
        prompt_tokens:
          type: integer
          format: int32
          minimum: 0
        total_tokens:
          type: integer
          format: int32
          minimum: 0
    ChatChoiceStream:
      type: object
      description: Streaming choice emitted per SSE chunk.
      required:
        - index
      properties:
        delta:
          oneOf:
            - type: 'null'
            - $ref: '#/components/schemas/ChatCompletionStreamResponseDelta'
        finish_reason:
          oneOf:
            - type: 'null'
            - $ref: '#/components/schemas/FinishReason'
        index:
          type: integer
          format: int32
          minimum: 0
    OpenAIErrorDetail:
      type: object
      description: OpenAI-compatible error detail.
      required:
        - message
      properties:
        code:
          type:
            - string
            - 'null'
        message:
          type: string
        param:
          type:
            - string
            - 'null'
        type:
          type:
            - string
            - 'null'
    ChatCompletionRequestSystemMessage:
      type: object
      description: Single chat message within the request payload.
      required:
        - content
      properties:
        content:
          $ref: '#/components/schemas/ChatCompletionRequestSystemMessageContent'
    ChatCompletionRequestUserMessage:
      type: object
      description: Single chat message within the request payload.
      required:
        - content
      properties:
        content:
          $ref: '#/components/schemas/ChatCompletionRequestUserMessageContent'
    ChatCompletionRequestAssistantMessage:
      type: object
      description: Single chat message within the request payload.
      properties:
        content:
          oneOf:
            - type: 'null'
            - $ref: >-
                #/components/schemas/ChatCompletionRequestAssistantMessageContent
    ChatCompletionResponseFormatJsonSchema:
      type: object
      required:
        - name
      properties:
        name:
          type: string
        schema: {}
        strict:
          type:
            - boolean
            - 'null'
          description: >-
            When true, the model output will strictly adhere to the provided
            schema.
    AnnotationFormat:
      type: string
      description: Annotation format the model should emit alongside text output.
      enum:
        - point
        - box
        - polygon
        - clip
    InternalTools:
      type: object
      description: Internal-tool toggles for Perceptron vision models.
      properties:
        focus:
          type:
            - boolean
            - 'null'
          description: When true, allows the model to invoke its internal "focus" tool.
    FinishReason:
      type: string
      enum:
        - stop
        - length
        - tool_error
        - tool_limit
    ChatCompletionResponseMessage:
      type: object
      description: Message object returned in the assistant's response.
      required:
        - role
      properties:
        content:
          type:
            - string
            - 'null'
        reasoning_content:
          type:
            - string
            - 'null'
        role:
          $ref: '#/components/schemas/Role'
    ChatCompletionStreamResponseDelta:
      type: object
      description: Delta payload for streaming responses.
      properties:
        content:
          type:
            - string
            - 'null'
        reasoning_content:
          type:
            - string
            - 'null'
        role:
          oneOf:
            - type: 'null'
            - $ref: '#/components/schemas/Role'
    ChatCompletionRequestSystemMessageContent:
      oneOf:
        - type: string
        - type: array
          items:
            $ref: '#/components/schemas/ChatCompletionRequestSystemMessageContentPart'
      description: >-
        Chat completion message content as either a string or structured content
        array.
    ChatCompletionRequestUserMessageContent:
      oneOf:
        - type: string
        - type: array
          items:
            $ref: '#/components/schemas/ChatCompletionRequestUserMessageContentPart'
      description: >-
        Chat completion message content as either a string or structured content
        array.
    ChatCompletionRequestAssistantMessageContent:
      oneOf:
        - type: string
        - type: array
          items:
            $ref: >-
              #/components/schemas/ChatCompletionRequestAssistantMessageContentPart
      description: >-
        Chat completion message content as either a string or structured content
        array.
    Role:
      type: string
      enum:
        - system
        - user
        - assistant
    ChatCompletionRequestSystemMessageContentPart:
      oneOf:
        - allOf:
            - $ref: '#/components/schemas/ChatCompletionRequestMessageContentPartText'
            - type: object
              required:
                - type
              properties:
                type:
                  type: string
                  enum:
                    - text
    ChatCompletionRequestUserMessageContentPart:
      oneOf:
        - allOf:
            - $ref: '#/components/schemas/ChatCompletionRequestMessageContentPartText'
            - type: object
              required:
                - type
              properties:
                type:
                  type: string
                  enum:
                    - text
        - allOf:
            - $ref: >-
                #/components/schemas/ChatCompletionRequestMessageContentPartImage
            - type: object
              required:
                - type
              properties:
                type:
                  type: string
                  enum:
                    - image_url
        - allOf:
            - $ref: >-
                #/components/schemas/ChatCompletionRequestMessageContentPartVideo
            - type: object
              required:
                - type
              properties:
                type:
                  type: string
                  enum:
                    - video_url
        - allOf:
            - $ref: >-
                #/components/schemas/ChatCompletionRequestMessageContentPartVideoFrames
            - type: object
              required:
                - type
              properties:
                type:
                  type: string
                  enum:
                    - video_frames
        - allOf:
            - $ref: >-
                #/components/schemas/ChatCompletionRequestMessageContentPartImageFileId
            - type: object
              required:
                - type
              properties:
                type:
                  type: string
                  enum:
                    - image_file_id
        - allOf:
            - $ref: >-
                #/components/schemas/ChatCompletionRequestMessageContentPartVideoFileId
            - type: object
              required:
                - type
              properties:
                type:
                  type: string
                  enum:
                    - video_file_id
    ChatCompletionRequestAssistantMessageContentPart:
      oneOf:
        - allOf:
            - $ref: '#/components/schemas/ChatCompletionRequestMessageContentPartText'
            - type: object
              required:
                - type
              properties:
                type:
                  type: string
                  enum:
                    - text
    ChatCompletionRequestMessageContentPartText:
      type: object
      description: Text chunk inside a structured message.
      required:
        - text
      properties:
        text:
          type: string
    ChatCompletionRequestMessageContentPartImage:
      type: object
      description: Image content part containing a URL.
      required:
        - image_url
      properties:
        image_url:
          $ref: '#/components/schemas/ImageUrl'
    ChatCompletionRequestMessageContentPartVideo:
      type: object
      description: Video content part containing a URL.
      required:
        - video_url
      properties:
        video_url:
          $ref: '#/components/schemas/VideoUrl'
    ChatCompletionRequestMessageContentPartVideoFrames:
      type: object
      description: >-
        Content part carrying raw video frames the caller has already decoded,
        as an

        alternative to a single `video_url`.
      required:
        - video_frames
      properties:
        video_frames:
          $ref: '#/components/schemas/VideoFrames'
    ChatCompletionRequestMessageContentPartImageFileId:
      type: object
      description: Image content part referencing an uploaded file by id.
      required:
        - image_file_id
      properties:
        image_file_id:
          $ref: '#/components/schemas/ImageFileId'
    ChatCompletionRequestMessageContentPartVideoFileId:
      type: object
      description: Video content part referencing an uploaded file by id.
      required:
        - video_file_id
      properties:
        video_file_id:
          $ref: '#/components/schemas/VideoFileId'
    ImageUrl:
      type: object
      description: Inline image reference (an HTTP(S) URL or a base64 data URL).
      required:
        - url
      properties:
        url:
          type: string
    VideoUrl:
      type: object
      description: Video asset referenced inside structured content arrays.
      required:
        - url
      properties:
        url:
          type: string
    VideoFrames:
      type: object
      description: An ordered sequence of decoded video frames passed inline.
      required:
        - frames
      properties:
        frames:
          type: array
          items:
            $ref: '#/components/schemas/VideoFrame'
          minItems: 2
    ImageFileId:
      type: object
      description: Reference to an uploaded image file by its id (e.g. `file-abc...`).
      required:
        - file_id
      properties:
        file_id:
          type: string
    VideoFileId:
      type: object
      description: Reference to an uploaded video file by its id (e.g. `file-abc...`).
      required:
        - file_id
      properties:
        file_id:
          type: string
    VideoFrame:
      type: object
      description: >-
        A single decoded video frame: an image (HTTP(S) URL or base64 data URL)
        plus

        its timestamp within the clip.
      required:
        - image_url
        - timestamp_ms
      properties:
        image_url:
          $ref: '#/components/schemas/ImageUrl'
        timestamp_ms:
          type: integer
          format: int64
          description: >-
            Offset of this frame from the start of the clip, in milliseconds.
            Timestamps must

            be non-negative and non-decreasing across the `frames` array.
          minimum: 0
  securitySchemes:
    ApiKeyAuth:
      type: http
      scheme: bearer
      description: Bearer token authentication using your Perceptron API key

````