> ## Documentation Index
> Fetch the complete documentation index at: https://docs.perceptron.inc/llms.txt
> Use this file to discover all available pages before exploring further.

# Prompting reference

> Choose clear instructions, reasoning controls, and output formats for Mk1.5 tasks.

State what to inspect, which evidence matters, and how the answer will be used. Put reusable application instructions in a system message and the current question with its media in a user message. Keep text copied from documents or tool results distinct from your application's instructions.

## Task prompts

Use these as starting points, then evaluate them on representative inputs.

| Task               | Example instruction                                                                                                         | Output control                                                               |
| ------------------ | --------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------- |
| Concise caption    | “Describe the main objects and their arrangement in one sentence.”                                                          | Text                                                                         |
| Detailed caption   | “Describe the visible objects, their relationships, and the setting. Separate observations from uncertain interpretations.” | Text, or `box` for grounded mentions                                         |
| OCR                | “Transcribe all readable text in reading order. Preserve line breaks and mark unreadable spans as unknown.”                 | Text                                                                         |
| Table extraction   | “Extract the table as CSV, preserving column order. Leave unreadable cells empty.”                                          | Text; validate the table locally                                             |
| General detection  | “Locate each visible object with a separate labeled bounding box.”                                                          | `box`                                                                        |
| Category detection | “Locate helmets, vests, and goggles. Use those labels for the matching objects.”                                            | `box`, `point`, or `polygon`                                                 |
| Visual Q\&A        | “Which safety equipment is visible? Cite the regions supporting your answer.”                                               | `box`                                                                        |
| Counting           | “Count the visible cars, excluding buses. Place one point on each car you count.”                                           | `point`                                                                      |
| One video event    | “Identify the moment the ball passes through the hoop. Return a clip annotation in seconds.”                                | `clip`                                                                       |
| Repeated events    | “Find every interval where the person opens the door. Return one clip annotation per occurrence.”                           | `clip`                                                                       |
| Temporal evidence  | “Does the reference product appear in the video? Return a clip that supports the answer; say if no match is visible.”       | `clip`                                                                       |
| Object tracking    | “Track the red ball with timed boxes inside one track. Report only observed positions.”                                     | `box`, plus a prompt requesting `<track>`                                    |
| Transcription      | “Transcribe the speech. Mark unclear words as \[unclear] instead of guessing.”                                              | Text; supply a supported [audio input](/perceptron-mk1.5/capabilities/audio) |
| Audio summary      | “Summarize the stated decisions and unresolved questions in this recording.”                                                | Text                                                                         |
| Audiovisual Q\&A   | “Describe what is visible and what is audible, keeping the two kinds of evidence separate.”                                 | Text; enable the video's soundtrack                                          |

For spatial and temporal output, set `vision_config.annotation_format` to the value in the table. These prompts request model-generated annotations; they do not guarantee exhaustive detection, correct counts, or continuous visibility.

## Set the API controls explicitly

| Field                                 | Values or shape                            | Purpose                                                                                                        |
| ------------------------------------- | ------------------------------------------ | -------------------------------------------------------------------------------------------------------------- |
| `reasoning_effort`                    | `none`, `minimal`, `low`, `medium`, `high` | Choose reasoning effort; evaluate quality and latency for your task.                                           |
| `vision_config.annotation_format`     | `point`, `box`, `polygon`, `clip`          | Request spatial geometry or temporal annotations.                                                              |
| `vision_config.enable_audio_in_video` | Boolean; default `false`                   | Analyze soundtracks alongside frames for every video in the request. Standalone audio does not need this flag. |
| `max_completion_tokens`               | Integer within the model's limits          | Bound output, including reasoning and the final answer.                                                        |
| `response_format`                     | JSON Schema response format                | Constrain a final answer to a data structure.                                                                  |
| `regex`                               | Pattern string at the top level            | Constrain a short final answer to a pattern.                                                                   |
| `tools`                               | Function declarations                      | Describe functions that your application may execute.                                                          |

`track` is a container in annotation markup, not an `annotation_format` value. Use `clip` for moments or intervals, and spatial annotations inside tracks for positions over time. Coordinates use the normalized 0–1000 grid; generated annotation timestamps use seconds. See [annotations](/perceptron-mk1.5/concepts/annotations).

Use `reasoning_effort` for new requests and leave the older thinking override unset. Reasoning and grounding are separate choices: do not assume all spatial tasks require reasoning off, or that a higher effort guarantees a better answer. See [reasoning](/perceptron-mk1.5/capabilities/thinking) and [tokenization](/perceptron-mk1.5/guides/tokenization) for budgeting.

## Ground a specific question

Set `PERCEPTRON_API_KEY`, then send the image, an observable task, and the requested annotation format together:

```bash theme={null}
curl --fail-with-body --silent --show-error \
  https://api.perceptron.inc/v1/chat/completions \
  -H "Authorization: Bearer $PERCEPTRON_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "perceptron-mk1.5",
    "messages": [{
      "role": "user",
      "content": [
        {"type": "image_url", "image_url": {"url": "https://raw.githubusercontent.com/perceptron-ai-inc/perceptron/main/cookbook/_shared/assets/capabilities/detection/ppe_line.webp"}},
        {"type": "text", "text": "Locate each visible helmet and safety vest. Label every box with the item type. If an item is not visible, do not invent a box."}
      ]
    }],
    "vision_config": {"annotation_format": "box"},
    "reasoning_effort": "high",
    "max_completion_tokens": 2048
  }'
```

Read the answer from `choices[0].message.content` and inspect `finish_reason` before parsing it. Native annotations remain markup within that text; the response does not turn them into a separate detections array.

## Provide examples and identify the target

When an object is hard to describe, supply a reference image and, when available, a verified annotation showing the intended region. Keep category labels consistent between demonstrations and the query. Accurate examples are more useful than many uncertain ones; review a model-generated exemplar before reusing it as a reference.

Label the role of each input explicitly: for example, “Asset 0 is the reference object; locate matching objects in asset 1.” Request `asset_idx` in annotations and preserve the media ordering when resending history. The attribute is optional, and the model may omit it despite the request; without an explicit or inherited selector, the annotation refers to the last asset passed to the model. A reference image plus a target is useful comparison context; an annotated demonstration additionally shows the desired geometry or answer format. See [multiple assets](/perceptron-mk1.5/guides/multiple-assets).

For video, describe a visible event precisely and distinguish a single moment from a duration. Ask for separate clips for repeated events. If you provide a product reference followed by a video, request evidence in the video rather than an annotation on the reference image. Keep input `timestamp_ms` values distinct from generated timestamps in seconds. For denser object tracks, follow [High Fidelity Object Tracking](/perceptron-mk1.5/guides/high-fidelity-object-tracking) to combine model waypoints with optical flow and appearance tracking.

## Choose a final-answer format

“Return JSON” is an instruction, not a response constraint. Use a JSON Schema `response_format` when your application requires known fields and types, then validate the completed answer locally. Include an unknown or empty value for information that may not be visible. For document transcription, ask explicitly for reading order, tables, headings, or Markdown rather than relying on an unstated layout convention.

Keep native annotation markup and a constrained JSON report as distinct output contracts. Function arguments are another contract: function `strict` does not enforce their schema. A request cannot declare non-empty `tools` together with JSON Schema or regex constrained output. Finish the tool loop, then request a constrained report without tool declarations; completed tool history can remain. See [structured outputs](/perceptron-mk1.5/capabilities/structured-outputs).

## Give tools a clear job

Describe what each function does, when it is useful, and the meanings of its arguments. Ask the model to use returned evidence and report missing information honestly. Your application validates names and arguments, checks permissions, executes the function, and returns the result; a prompt does not grant execution permission.

Preserve call IDs, completed tool results, and assistant reasoning when continuing a tool conversation. Set limits on rounds, time, tokens, and tool calls, and check completion before using any answer. See [tool calling](/perceptron-mk1.5/guides/tool-calling), [building agents](/perceptron-mk1.5/guides/tool-agents), and [security](/perceptron-mk1.5/best-practices/security).