> ## Documentation Index
> Fetch the complete documentation index at: https://docs.perceptron.inc/llms.txt
> Use this file to discover all available pages before exploring further.

# In-context learning (image)

> Use labeled visual examples to guide detection in a new image.

In-context learning gives Perceptron Mk1.5 examples of the visual concept you want to find. Supply a reference image, a label, and a box around the reference object, then ask for matching objects in a target image. The examples guide that request; they do not train or permanently update the model.

Use one reference for a specific object or several labeled references to distinguish classes. This is useful when a visual example communicates the appearance more clearly than a description alone.

## Detect from labeled examples

This example uses reviewed boxes around a cat and a dog in the sample reference images. It runs once with only the dog reference, then with both classes. The target image shows a dog; providing a cat reference does not mean a cat must appear in the answer.

Install the `perceptron>=0.4.0` Python package and set `PERCEPTRON_API_KEY` before running:

```python theme={null}
import os
from html import escape

from perceptron import Client, image

client = Client(api_key=os.environ["PERCEPTRON_API_KEY"])
asset_base = (
    "https://raw.githubusercontent.com/perceptron-ai-inc/perceptron/"
    "main/cookbook/_shared/assets/in-context-learning/multi"
)
cat_reference = {
    "url": f"{asset_base}/classA.jpg",
    "label": "cat",
    "box": (316, 136, 703, 906),
}
dog_reference = {
    "url": f"{asset_base}/classB.webp",
    "label": "dog",
    "box": (161, 48, 666, 980),
}
target_url = f"{asset_base}/cat_dog_input.png"


def detect_from_examples(examples, target_url):
    content = [{"type": "text", "text": (
        "The following images and box annotations are labeled references. "
        "Use them to recognize the same classes in the final target image."
    )}]
    for asset_idx, example in enumerate(examples):
        x1, y1, x2, y2 = example["box"]
        label = escape(example["label"], quote=True)
        content.extend([
            image(example["url"]),
            {"type": "text", "text": (
                f'Reference: <point_box mention="{label}" asset_idx="{asset_idx}"> '
                f"({x1},{y1}) ({x2},{y2}) </point_box>"
            )},
        ])

    target_idx = len(examples)
    content.extend([
        image(target_url),
        {"type": "text", "text": (
            f"This is the target image, asset_idx={target_idx}. "
            "Find all visible instances of the reference classes here. "
            "Return flat point_box annotations, without collections, using the reference labels and "
            f"asset_idx={target_idx}, with normalized 0–1000 coordinates. "
            "Do not annotate the reference images. Do not assume every class "
            "is present; say when no matching object is visible."
        )},
    ])
    response = client.chat.completions.create(
        model="perceptron-mk1.5",
        messages=[{"role": "user", "content": content}],
        reasoning_effort="high",
        max_completion_tokens=2048,
        vision_config={"annotation_format": "box"},
    )
    choice = response.choices[0]
    if choice.finish_reason != "stop":
        raise RuntimeError(f"Incomplete detection answer: {choice.finish_reason}")
    return choice.message.content or ""


print("One reference:")
print(detect_from_examples([dog_reference], target_url))

print("Multiple classes:")
print(detect_from_examples([cat_reference, dog_reference], target_url))
```

In the first call, the dog reference is asset `0` and the target is asset `1`. In the second, the cat is asset `0`, the dog is asset `1`, and the target is asset `2`. Each call has its own supplied conversation, so its numbering starts again at zero. See [multiple assets](/perceptron-mk1.5/guides/multiple-assets) before adapting this to a conversation with earlier media.

The answer is annotation markup in `message.content`. Check that each returned box refers to the target asset before drawing it. The [annotation reference](/perceptron-mk1.5/concepts/annotations) explains coordinate conversion, collections, and validation.

To draw the boxes, save one call's returned string as `response.txt`, download the target image, and follow [rendering annotations](/perceptron-mk1.5/guides/rendering-annotations). Use `--asset-idx 1 --last-asset-idx 1` for the single-reference answer or `--asset-idx 2 --last-asset-idx 2` for the multiclass answer. Keep the two answers separate because their indices differ. When `asset_idx` is omitted and no selector is inherited, it defaults to the last asset available when that answer was produced, which is the target image in both examples.

## Prepare your own references

Replace each reference URL, label, and box together. The sample coordinates belong only to the sample images. Use the normalized 0–1000 grid, with the top-left and bottom-right corners enclosing the object you intend to teach. If you obtain candidate boxes from a previous model response, inspect and correct them before using them as examples.

* Choose references with a clear view of the object, including the features that distinguish it from similar objects.
* Use consistent class labels across examples and the target request. Multiple views of one class should share a label.
* Include confusing alternatives as separately labeled references when distinguishing them matters.
* Keep the target image separate from the examples, and request output only for its `asset_idx`.

For the same reference-to-query pattern across time, see [in-context learning for video](/perceptron-mk1.5/capabilities/in-context-learning-video). For ordinary detection without examples, see [object detection](/perceptron-mk1.5/capabilities/object-detection).