> ## Documentation Index
> Fetch the complete documentation index at: https://docs.perceptron.inc/llms.txt
> Use this file to discover all available pages before exploring further.

# Image captioning

> Generate concise descriptions, detailed captions, and captions with spatial grounding.

Use Perceptron Mk1.5 to describe an image for accessibility text, catalog metadata, search, or review. Specify the audience and desired detail in the prompt. A concise caption identifies the main subject; a detailed caption can describe its surroundings and relationships.

## Choose the caption style

Install `perceptron>=0.4.0` and set `PERCEPTRON_API_KEY`. This example sends the same public street image in two independent requests, one for each style.

```python theme={null}
import os

from perceptron import Client, image

client = Client(api_key=os.environ["PERCEPTRON_API_KEY"])
image_url = (
    "https://raw.githubusercontent.com/perceptron-ai-inc/perceptron/"
    "main/cookbook/_shared/assets/capabilities/caption/suburban_street.webp"
)


def caption(prompt, grounded=False):
    response = client.chat.completions.create(
        model="perceptron-mk1.5",
        messages=[{
            "role": "user",
            "content": [
                image(image_url),
                {"type": "text", "text": prompt},
            ],
        }],
        reasoning_effort="high",
        max_completion_tokens=2048,
        vision_config={"annotation_format": "box"} if grounded else {},
    )
    choice = response.choices[0]
    if choice.finish_reason != "stop" or choice.message.tool_calls:
        raise RuntimeError(f"Incomplete caption: {choice.finish_reason}")
    return choice.message.content or ""


prompts = {
    "concise": (
        "Write one sentence of alt text identifying the main visible subjects and "
        "setting. Include only details supported by the image."
    ),
    "detailed": (
        "Describe this image in one detailed paragraph. Cover the main subjects, "
        "their positions, the foreground and background, and visible relationships. "
        "Do not invent events, identities, or details outside the image."
    ),
}
for style, prompt in prompts.items():
    print(f"{style}: {caption(prompt)}")
```

The output wording will vary. Adjust the instruction rather than treating a style name as an API parameter: `concise` and `detailed` are labels in this example application.

## Ground the caption in regions

Request boxes for the subjects mentioned in the description. Run this after the setup above:

```python theme={null}
from pathlib import Path

grounded_caption = caption(
    "Describe the main visible subjects and their surroundings. Interleave the "
    "description with flat point_box annotations for the subjects you mention, "
    "without collections or tracks. Give each box a descriptive mention and "
    "asset_idx=0. Use normalized coordinates.",
    grounded=True,
)
print(grounded_caption)
Path("response.txt").write_text(grounded_caption, encoding="utf-8")
```

An illustrative grounded phrase looks like:

```html theme={null}
The scene includes <point_box mention="vehicle" asset_idx="0"> (100,400) (350,700) </point_box> beside the road.
```

This is a format illustration, not measured output for the sample image. A `mention` describes the cited region. Boxes use two normalized 0–1000 coordinates: top-left, then bottom-right. Convert using the dimensions of the asset selected by `asset_idx`; do not treat the numbers as source pixels.

Use the shared [rendering guide](/perceptron-mk1.5/guides/rendering-annotations) to turn `response.txt` into an overlay. The [annotation reference](/perceptron-mk1.5/concepts/annotations) also covers points, polygons, and collections. For captions comparing several images, request explicit selectors and follow [multiple-asset ordering](/perceptron-mk1.5/guides/multiple-assets).

## Write captions for their destination

| Destination      | Prompt guidance                                                                           |
| ---------------- | ----------------------------------------------------------------------------------------- |
| Alt text         | State the important visual information concisely and avoid repeating nearby page text.    |
| Product metadata | Ask for observable color, shape, material cues, and markings; allow unknown values.       |
| Visual search    | Describe distinguishing objects, actions, and relationships using consistent terminology. |
| Human review     | Include spatial citations so a reviewer can inspect the evidence behind a description.    |

For metadata that must be parsed, define a JSON Schema with [structured outputs](/perceptron-mk1.5/capabilities/structured-outputs). Asking for JSON in a caption prompt alone does not enforce a schema. Use [image Q\&A](/perceptron-mk1.5/capabilities/image-qa) when you need an answer to a specific question rather than an overall description.