> ## Documentation Index
> Fetch the complete documentation index at: https://docs.perceptron.inc/llms.txt
> Use this file to discover all available pages before exploring further.
# Detect
## Overview
The Detect API returns grounded object detections for a single image. Use it when you want a direct detection workflow without building a chat-completions prompt.
Requests can include text `categories`, annotated positive `exemplars`, both together, or no targets for exhaustive object detection. You can also include `negative_exemplars`, but only alongside at least one category or positive exemplar. Coordinates for exemplar annotations and returned detections are image pixels.
## Examples
### Detect all objects as boxes
Omit `categories` and `exemplars` to ask the model to return the objects it finds in the image. This is useful for exploratory labeling or scene inventory. Box output is the default.
```bash theme={null}
curl https://api.perceptron.inc/v1/detect \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $PERCEPTRON_API_KEY" \
-d '{
"media": {
"type": "image",
"image_url": "https://example.com/warehouse.jpg"
},
"config": {
"output": "box",
"enable_thinking": false
}
}'
```
### Detect target categories as boxes
Pass `categories` when you know the labels you want returned. Each detection `mention` is one of the requested categories.
```bash theme={null}
curl https://api.perceptron.inc/v1/detect \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $PERCEPTRON_API_KEY" \
-d '{
"media": {
"type": "image",
"image_url": "https://example.com/warehouse.jpg"
},
"categories": ["person", "hard_hat"],
"config": {
"output": "box",
"enable_thinking": false
}
}'
```
### Detect one category as points
Set `config.output` to `"point"` when you want one pixel point per detection. With exactly one unique category and no exemplars, `/v1/detect` uses a direct point prompt and forces every returned `mention` to that category.
```bash theme={null}
curl https://api.perceptron.inc/v1/detect \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $PERCEPTRON_API_KEY" \
-d '{
"media": {
"type": "image",
"image_url": "https://example.com/warehouse.jpg"
},
"categories": ["forklift"],
"config": {
"output": "point",
"enable_thinking": false
}
}'
```
### Detect all objects as points
You can also request point output without categories or exemplars. The route still uses exhaustive object detection internally, then returns the center point for each detected box or polygon.
```bash theme={null}
curl https://api.perceptron.inc/v1/detect \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $PERCEPTRON_API_KEY" \
-d '{
"media": {
"type": "image",
"image_url": "https://example.com/warehouse.jpg"
},
"config": {
"output": "point"
}
}'
```
### Detect from exemplars as boxes
Use `exemplars` when the target is visual or domain-specific. Exemplar annotations use pixel coordinates in the exemplar image. If you omit `categories`, the exemplar annotation `mention` values define the target labels.
```bash theme={null}
curl https://api.perceptron.inc/v1/detect \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $PERCEPTRON_API_KEY" \
-d '{
"media": {
"type": "image",
"image_url": "https://example.com/loading-dock.jpg"
},
"exemplars": [
{
"media": {
"type": "image",
"image_url": "data:image/png;base64,..."
},
"annotations": [
{
"mention": "damaged_box",
"point_box": {
"top_left": { "x": 124, "y": 80 },
"bottom_right": { "x": 310, "y": 240 }
}
}
]
}
],
"config": {
"output": "box",
"enable_thinking": true
}
}'
```
### Detect from exemplars as points
For exemplar-guided point output, set `config.output` to `"point"`. Positive exemplars can use either `point` or `point_box` annotations. The route uses the exemplar labels to find matching objects, then returns a `point` for each detection.
```bash theme={null}
curl https://api.perceptron.inc/v1/detect \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $PERCEPTRON_API_KEY" \
-d '{
"media": {
"type": "image",
"image_url": "https://example.com/loading-dock.jpg"
},
"categories": ["damaged_box"],
"exemplars": [
{
"media": {
"type": "image",
"image_url": "data:image/png;base64,..."
},
"annotations": [
{
"mention": "damaged_box",
"point": { "x": 218, "y": 160 }
},
{
"mention": "damaged_box",
"point_box": {
"top_left": { "x": 124, "y": 80 },
"bottom_right": { "x": 310, "y": 240 }
}
}
]
}
],
"config": {
"output": "point",
"enable_thinking": true
}
}'
```
`enable_thinking` is optional and defaults to `false`. Set it to `true` when you want a supported detect model to use reasoning internally; the native response still returns only `detections` and `finish_reason`.
## Supported permutations
| Request shape | `config.output` | Behavior |
| ---------------------------------------------------------------- | -------------------- | -------------------------------------------------------------------------------------------------------------------------------------- |
| `media` only | `"box"` or omitted | Exhaustive object detection; returns `point_box` for every parsed object. |
| `media` only | `"point"` | Exhaustive object detection; returns `point` values converted from parsed boxes or polygons. |
| One or more `categories` entries, no positive exemplars | `"box"` | Category detection; returns `point_box` values. Duplicate categories are deduplicated for prompting. |
| One unique `categories` entry, no positive or negative exemplars | `"point"` | Direct point prompt; every returned `mention` is forced to that category. |
| Multiple unique `categories` entries | `"point"` | Category detection; returns centroid `point` values from parsed boxes or polygons. |
| Positive `exemplars`, with or without `categories` | `"box"` | Exemplar-guided detection; returns `point_box` values. If `categories` is omitted, exemplar `mention` values define the target labels. |
| Positive `exemplars`, with or without `categories` | `"point"` | Exemplar-guided detection; returns `point` values. Exemplar annotations may use `point` or `point_box`. |
| `negative_exemplars` with categories or positive exemplars | `"box"` or `"point"` | Uses negative images as examples of what not to detect. Negative exemplars do not define labels. |
| `negative_exemplars` only | Any | Invalid request; include at least one category or positive exemplar. |
## Example response
### Box response
```json theme={null}
{
"detections": [
{
"mention": "hard_hat",
"point_box": {
"top_left": { "x": 384.0, "y": 108.0 },
"bottom_right": { "x": 768.0, "y": 540.0 }
}
}
],
"finish_reason": "complete"
}
```
### Point response
```json theme={null}
{
"detections": [
{
"mention": "forklift",
"point": { "x": 512.0, "y": 320.0 }
}
],
"finish_reason": "complete"
}
```
## Notes
* `/v1/detect` supports image inputs only.
* `config.output` can be `"box"` or `"point"`; the default is `"box"`. The public value is `"box"`, not `"bbox"`.
* The response does not include confidence scores.
* `finish_reason` is `"complete"` or `"max_tokens"`.
* When `categories` is set, every positive exemplar annotation `mention` must exactly match one of those categories.
* Exemplar annotations must include exactly one of `point` or `point_box`.
* Point outputs may come from parsed points directly, or from the centroid of a parsed box or polygon.
* `config.enable_thinking` can be `true` or `false` for box output, point output, category requests, and exemplar-guided requests. It does not change the response schema.
## Limits
| Limit | Value |
| ------------------------ | ----- |
| `categories` entries | 256 |
| Positive `exemplars` | 64 |
| `negative_exemplars` | 64 |
| Annotations per exemplar | 256 |
See the API reference below for the complete request and response schemas.
## OpenAPI
````yaml api-reference/openapi.json POST /v1/detect
openapi: 3.1.0
info:
title: Perceptron API
contact:
name: Perceptron API Support
email: support@perceptron.inc
version: 1.0.0
servers:
- url: https://api.perceptron.inc
security: []
tags:
- name: Chat Completions
description: Chat completions API (OpenAI-compatible)
- name: Detection
description: Native Perceptron image detection API
- name: Models
description: Model listing and metadata API
- name: Files
description: File upload, listing, retrieval, and deletion API
paths:
/v1/detect:
post:
tags:
- Detection
operationId: handle_detect
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/DetectRequest'
required: true
responses:
'200':
description: Detection completed successfully.
content:
application/json:
schema:
$ref: '#/components/schemas/DetectResponse'
'400':
description: Invalid detection request.
content:
application/json:
schema:
$ref: '#/components/schemas/PerceptronErrorResponse'
'401':
description: Authentication failed. Invalid or missing API key.
content:
application/json:
schema:
$ref: '#/components/schemas/PerceptronErrorResponse'
'429':
description: Rate limit exceeded.
content:
application/json:
schema:
$ref: '#/components/schemas/PerceptronErrorResponse'
'500':
description: Internal error while processing detection.
content:
application/json:
schema:
$ref: '#/components/schemas/PerceptronErrorResponse'
security:
- ApiKeyAuth: []
components:
schemas:
DetectRequest:
type: object
required:
- media
properties:
categories:
type:
- array
- 'null'
items:
type: string
description: >-
Category names to detect. Entries must be non-empty and must not
contain commas.
config:
$ref: '#/components/schemas/DetectConfig'
exemplars:
type:
- array
- 'null'
items:
$ref: '#/components/schemas/DetectExemplar'
media:
$ref: '#/components/schemas/DetectMedia'
negative_exemplars:
type:
- array
- 'null'
items:
$ref: '#/components/schemas/DetectNegativeExemplar'
additionalProperties: false
DetectResponse:
type: object
required:
- detections
- finish_reason
properties:
detections:
type: array
items:
$ref: '#/components/schemas/Detection'
finish_reason:
$ref: '#/components/schemas/DetectFinishReason'
additionalProperties: false
PerceptronErrorResponse:
type: object
description: Native Perceptron error response format.
required:
- error
properties:
error:
$ref: '#/components/schemas/PerceptronErrorDetail'
DetectConfig:
type: object
properties:
enable_thinking:
type: boolean
description: Toggle reasoning ("thinking") on supported detect models.
default: false
output:
$ref: '#/components/schemas/DetectOutput'
additionalProperties: false
DetectExemplar:
type: object
required:
- media
- annotations
properties:
annotations:
type: array
items:
$ref: '#/components/schemas/DetectPointPrimitive'
media:
$ref: '#/components/schemas/DetectMedia'
DetectMedia:
oneOf:
- type: object
required:
- image_url
- type
properties:
image_url:
type: string
description: Image URL or data URL. Video is not supported by /v1/detect yet.
type:
type: string
enum:
- image
DetectNegativeExemplar:
type: object
required:
- media
properties:
media:
$ref: '#/components/schemas/DetectMedia'
description: Image known not to contain requested target objects.
Detection:
type: object
required:
- mention
properties:
mention:
type: string
point:
oneOf:
- type: 'null'
- $ref: '#/components/schemas/DetectPoint'
description: Point in target-image pixel coordinates.
point_box:
oneOf:
- type: 'null'
- $ref: '#/components/schemas/DetectPointBox'
description: Bounding box in target-image pixel coordinates.
additionalProperties: false
DetectFinishReason:
type: string
enum:
- complete
- max_tokens
PerceptronErrorDetail:
type: object
description: Native Perceptron error detail.
required:
- message
properties:
code:
type:
- string
- 'null'
message:
type: string
param:
type:
- string
- 'null'
type:
type:
- string
- 'null'
DetectOutput:
type: string
enum:
- box
- point
DetectPointPrimitive:
type: object
required:
- mention
properties:
index:
type:
- integer
- 'null'
format: int64
label:
type:
- string
- 'null'
mention:
type: string
description: >-
Category/target identity for this exemplar annotation. Must not
contain commas.
point:
oneOf:
- type: 'null'
- $ref: '#/components/schemas/DetectPoint'
description: Exemplar point annotation in exemplar-image pixel coordinates.
point_box:
oneOf:
- type: 'null'
- $ref: '#/components/schemas/DetectPointBox'
description: Exemplar box annotation in exemplar-image pixel coordinates.
timestamp:
type:
- number
- 'null'
format: double
DetectPoint:
type: object
required:
- x
- 'y'
properties:
x:
type: number
format: double
description: X coordinate in image pixels.
'y':
type: number
format: double
description: Y coordinate in image pixels.
DetectPointBox:
type: object
required:
- top_left
- bottom_right
properties:
bottom_right:
$ref: '#/components/schemas/DetectPoint'
top_left:
$ref: '#/components/schemas/DetectPoint'
securitySchemes:
ApiKeyAuth:
type: http
scheme: bearer
description: Bearer token authentication using your Perceptron API key
````