# Detect
Source: https://docs.perceptron.inc/capabilities/detect
api-reference/openapi.json POST /v1/detect
Return grounded object detections for a single image from text categories, annotated exemplars, or exhaustive detection.
## Overview
The Detect API returns grounded object detections for a single image. Use it when you want a direct detection workflow without building a chat-completions prompt.
Requests can include text `categories`, annotated positive `exemplars`, both together, or no targets for exhaustive object detection. You can also include `negative_exemplars`, but only alongside at least one category or positive exemplar. Coordinates for exemplar annotations and returned detections are image pixels.
## Examples
### Detect all objects as boxes
Omit `categories` and `exemplars` to ask the model to return the objects it finds in the image. This is useful for exploratory labeling or scene inventory. Box output is the default.
```bash theme={null}
curl https://api.perceptron.inc/v1/detect \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $PERCEPTRON_API_KEY" \
-d '{
"media": {
"type": "image",
"image_url": "https://example.com/warehouse.jpg"
},
"config": {
"output": "box",
"reasoning_effort": "high"
}
}'
```
![Box output example showing detected packages in a delivery van]()
### Detect target categories as boxes
Pass `categories` when you know the labels you want returned. Each detection `mention` is one of the requested categories.
```bash theme={null}
curl https://api.perceptron.inc/v1/detect \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $PERCEPTRON_API_KEY" \
-d '{
"media": {
"type": "image",
"image_url": "https://example.com/warehouse.jpg"
},
"categories": ["person", "hard_hat"],
"config": {
"output": "box",
"reasoning_effort": "none"
}
}'
```
### Detect one category as points
Set `config.output` to `"point"` when you want one pixel point per detection. With exactly one unique category and no exemplars, `/v1/detect` uses a direct point prompt and forces every returned `mention` to that category.
```bash theme={null}
curl https://api.perceptron.inc/v1/detect \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $PERCEPTRON_API_KEY" \
-d '{
"media": {
"type": "image",
"image_url": "https://example.com/warehouse.jpg"
},
"categories": ["forklift"],
"config": {
"output": "point",
"reasoning_effort": "none"
}
}'
```
### Detect all objects as points
You can also request point output without categories or exemplars. The route still uses exhaustive object detection internally, then returns the center point for each detected box or polygon.
```bash theme={null}
curl https://api.perceptron.inc/v1/detect \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $PERCEPTRON_API_KEY" \
-d '{
"media": {
"type": "image",
"image_url": "https://example.com/warehouse.jpg"
},
"config": {
"output": "point"
}
}'
```
![Point output example showing centroid points for detected packages]()
### Detect from exemplars as boxes
Use `exemplars` when the target is visual or domain-specific. Exemplar annotations use pixel coordinates in the exemplar image. If you omit `categories`, the exemplar annotation `mention` values define the target labels.
```bash theme={null}
curl https://api.perceptron.inc/v1/detect \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $PERCEPTRON_API_KEY" \
-d '{
"media": {
"type": "image",
"image_url": "https://example.com/loading-dock.jpg"
},
"exemplars": [
{
"media": {
"type": "image",
"image_url": "data:image/png;base64,..."
},
"annotations": [
{
"mention": "damaged_box",
"point_box": {
"top_left": { "x": 124, "y": 80 },
"bottom_right": { "x": 310, "y": 240 }
}
}
]
}
],
"config": {
"output": "box"
}
}'
```
### Detect from exemplars as points
For exemplar-guided point output, set `config.output` to `"point"`. Positive exemplars can use either `point` or `point_box` annotations. The route uses the exemplar labels to find matching objects, then returns a `point` for each detection.
```bash theme={null}
curl https://api.perceptron.inc/v1/detect \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $PERCEPTRON_API_KEY" \
-d '{
"media": {
"type": "image",
"image_url": "https://example.com/loading-dock.jpg"
},
"categories": ["damaged_box"],
"exemplars": [
{
"media": {
"type": "image",
"image_url": "data:image/png;base64,..."
},
"annotations": [
{
"mention": "damaged_box",
"point": { "x": 218, "y": 160 }
},
{
"mention": "damaged_box",
"point_box": {
"top_left": { "x": 124, "y": 80 },
"bottom_right": { "x": 310, "y": 240 }
}
}
]
}
],
"config": {
"output": "point"
}
}'
```
## Configure reasoning
Use **`config.reasoning_effort`** with `none`, `minimal`, `low`, `medium`, or `high`. The default is `none`; `minimal` currently uses the same reasoning level as `low`. The setting belongs inside `config` on `/v1/detect`, unlike the top-level field on chat completions and Multilook. An unsupported value returns HTTP 400.
The effect depends on the detection mode:
* **No categories or exemplars:** a value other than `none` enables model reasoning, for either box or point output. The first example uses `high`. Reasoning stays internal; the response contains `detections` and `finish_reason`, without reasoning text.
* **Categories with box output, without positive or negative exemplars:** a value other than `none` selects a detection workflow that can inspect additional image regions. It does not enable model reasoning, and the non-`none` tiers do not select different reasoning levels in this mode.
* **Category-based point output or any exemplar-guided request:** the field is accepted, but model reasoning remains disabled and the selected tier does not change the detection workflow.
For older clients, `config.enable_thinking` remains a deprecated alias: `true` maps to `high`, and `false` maps to `none`. **An explicit `config.reasoning_effort` takes precedence** when both are supplied. For example, `{"reasoning_effort": "none", "enable_thinking": true}` keeps the resolved effort at `none`. Prefer sending only `reasoning_effort`.
## Supported permutations
| Request shape | `config.output` | Behavior |
| ---------------------------------------------------------------- | -------------------- | -------------------------------------------------------------------------------------------------------------------------------------- |
| `media` only | `"box"` or omitted | Exhaustive object detection; returns `point_box` for every parsed object. |
| `media` only | `"point"` | Exhaustive object detection; returns `point` values converted from parsed boxes or polygons. |
| One or more `categories` entries, no positive exemplars | `"box"` | Category detection; returns `point_box` values. Duplicate categories are deduplicated for prompting. |
| One unique `categories` entry, no positive or negative exemplars | `"point"` | Direct point prompt; every returned `mention` is forced to that category. |
| Multiple unique `categories` entries | `"point"` | Category detection; returns centroid `point` values from parsed boxes or polygons. |
| Positive `exemplars`, with or without `categories` | `"box"` | Exemplar-guided detection; returns `point_box` values. If `categories` is omitted, exemplar `mention` values define the target labels. |
| Positive `exemplars`, with or without `categories` | `"point"` | Exemplar-guided detection; returns `point` values. Exemplar annotations may use `point` or `point_box`. |
| `negative_exemplars` with categories or positive exemplars | `"box"` or `"point"` | Uses negative images as examples of what not to detect. Negative exemplars do not define labels. |
| `negative_exemplars` only | Any | Invalid request; include at least one category or positive exemplar. |
## Example response
### Box response
```json theme={null}
{
"detections": [
{
"mention": "hard_hat",
"point_box": {
"top_left": { "x": 384.0, "y": 108.0 },
"bottom_right": { "x": 768.0, "y": 540.0 }
}
}
],
"finish_reason": "complete"
}
```
### Point response
```json theme={null}
{
"detections": [
{
"mention": "forklift",
"point": { "x": 512.0, "y": 320.0 }
}
],
"finish_reason": "complete"
}
```
## Notes
* `/v1/detect` supports image inputs only.
* `config.output` can be `"box"` or `"point"`; the default is `"box"`. The public value is `"box"`, not `"bbox"`.
* The response does not include confidence scores.
* `finish_reason` is `"complete"` or `"max_tokens"`.
* When `categories` is set, every positive exemplar annotation `mention` must exactly match one of those categories.
* Exemplar annotations must include exactly one of `point` or `point_box`.
* Point outputs may come from parsed points directly, or from the centroid of a parsed box or polygon.
* `config.reasoning_effort` does not change the response schema; its effect on detection depends on the request mode described above.
## Limits
| Limit | Value |
| ------------------------ | ----- |
| `categories` entries | 256 |
| Positive `exemplars` | 64 |
| `negative_exemplars` | 64 |
| Annotations per exemplar | 256 |
See the API reference below for the complete request and response schemas.
# Create chat completion
Source: https://docs.perceptron.inc/perceptron-mk1.5/api-reference/chat-completions/create-chat-completion
/api-reference/openapi.json post /v1/chat/completions
Generate a response for a text or multimodal conversation, using the OpenAI chat completions request and response format.
# Create multilook chat completions
Source: https://docs.perceptron.inc/perceptron-mk1.5/api-reference/chat-completions/create-multilook-chat-completions
/api-reference/openapi.json post /v1/chat/completions/multilook
Run several independent prompts over one shared, prefilled context.
# Detect objects
Source: https://docs.perceptron.inc/perceptron-mk1.5/api-reference/detection/detect-objects
/api-reference/openapi.json post /v1/detect
Return grounded object detections for a single image from text categories, annotated exemplars, or exhaustive detection.
# Delete a file
Source: https://docs.perceptron.inc/perceptron-mk1.5/api-reference/files/delete-a-file
/api-reference/openapi.json delete /v1/files/{file_id}
Permanently deletes a file. Once deleted, the file can no longer be
retrieved, downloaded, or referenced in requests. Returns 404 if no file
with the given id exists.
# Download a file's contents
Source: https://docs.perceptron.inc/perceptron-mk1.5/api-reference/files/download-a-files-contents
/api-reference/openapi.json get /v1/files/{file_id}/content
Returns the raw bytes with the file's stored `Content-Type` header.
# List files
Source: https://docs.perceptron.inc/perceptron-mk1.5/api-reference/files/list-files
/api-reference/openapi.json get /v1/files
Returns a paginated list of files belonging to the caller's organization.
Cursor pagination via `after`; see the `limit` and `order` parameters for defaults.
# Retrieve a file
Source: https://docs.perceptron.inc/perceptron-mk1.5/api-reference/files/retrieve-a-file
/api-reference/openapi.json get /v1/files/{file_id}
Returns metadata for a single file (size, content type, purpose, etc.).
Use the `/v1/files/{file_id}/content` endpoint to download the bytes.
# Upload a file
Source: https://docs.perceptron.inc/perceptron-mk1.5/api-reference/files/upload-a-file
/api-reference/openapi.json post /v1/files
Accepts a single file via multipart upload alongside a `purpose` field that
declares how the file will be used. Subject to per-file and per-organization
storage limits.
# List available models
Source: https://docs.perceptron.inc/perceptron-mk1.5/api-reference/models/list-available-models
/api-reference/openapi.json get /v1/models
Returns a list of models available in the API. Use the `extended` query parameter
to get additional metadata about each model's capabilities and supported features.
# Retrieve a model
Source: https://docs.perceptron.inc/perceptron-mk1.5/api-reference/models/retrieve-a-model
/api-reference/openapi.json get /v1/models/{model_id}
Returns a specific model by its ID. Use the `extended` query parameter
to get additional metadata about the model's capabilities and supported features.
# Prompting reference
Source: https://docs.perceptron.inc/perceptron-mk1.5/best-practices/prompting-reference
Choose clear instructions, reasoning controls, and output formats for Mk1.5 tasks.
State what to inspect, which evidence matters, and how the answer will be used. Put reusable application instructions in a system message and the current question with its media in a user message. Keep text copied from documents or tool results distinct from your application's instructions.
## Task prompts
Use these as starting points, then evaluate them on representative inputs.
| Task | Example instruction | Output control |
| ------------------ | --------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------- |
| Concise caption | “Describe the main objects and their arrangement in one sentence.” | Text |
| Detailed caption | “Describe the visible objects, their relationships, and the setting. Separate observations from uncertain interpretations.” | Text, or `box` for grounded mentions |
| OCR | “Transcribe all readable text in reading order. Preserve line breaks and mark unreadable spans as unknown.” | Text |
| Table extraction | “Extract the table as CSV, preserving column order. Leave unreadable cells empty.” | Text; validate the table locally |
| General detection | “Locate each visible object with a separate labeled bounding box.” | `box` |
| Category detection | “Locate helmets, vests, and goggles. Use those labels for the matching objects.” | `box`, `point`, or `polygon` |
| Visual Q\&A | “Which safety equipment is visible? Cite the regions supporting your answer.” | `box` |
| Counting | “Count the visible cars, excluding buses. Place one point on each car you count.” | `point` |
| One video event | “Identify the moment the ball passes through the hoop. Return a clip annotation in seconds.” | `clip` |
| Repeated events | “Find every interval where the person opens the door. Return one clip annotation per occurrence.” | `clip` |
| Temporal evidence | “Does the reference product appear in the video? Return a clip that supports the answer; say if no match is visible.” | `clip` |
| Object tracking | “Track the red ball with timed boxes inside one track. Report only observed positions.” | `box`, plus a prompt requesting `