> ## Documentation Index
> Fetch the complete documentation index at: https://docs.perceptron.inc/llms.txt
> Use this file to discover all available pages before exploring further.

# In-context learning (video)

> Show a reference image and find matching visual evidence in a video.

Use a reference image to show Perceptron Mk1.5 what to look for in a video. The request combines the example, an explanation of which features matter, and a query video. This can help with product matching, finding a familiar object, or reviewing footage for a visually defined condition.

The example guides the current request. Keep it in the supplied conversation when asking follow-up questions that depend on it.

## Find a reference product on a shelf

This example supplies an image of a cereal package, then asks whether a matching package is visible in a shelf video. It asks for temporal evidence and permits an uncertain answer when packaging details cannot be read clearly. Visibility in this recording does not establish a store's current inventory.

Install the `perceptron>=0.4.0` Python package and set `PERCEPTRON_API_KEY` before running:

```python theme={null}
import os

from perceptron import Client, image, video

client = Client(api_key=os.environ["PERCEPTRON_API_KEY"])
asset_base = (
    "https://raw.githubusercontent.com/perceptron-ai-inc/perceptron/"
    "main/cookbook/_shared/assets/capabilities/in-context-learning-video"
)
response = client.chat.completions.create(
    model="perceptron-mk1.5",
    messages=[{
        "role": "user",
        "content": [
            image(f"{asset_base}/mini_wheats.jpeg"),
            {"type": "text", "text": (
                "Asset 0 is the reference product. Look for a matching cereal "
                "package using its visible branding, product name, and package "
                "design. Similar colors alone do not establish a match."
            )},
            video(f"{asset_base}/cereal_short.mp4"),
            {"type": "text", "text": (
                "Asset 1 is the shelf video to inspect. Is a package matching "
                "the reference visible? Explain the visible evidence and cite "
                "the matching interval or intervals with clip annotations, "
                "asset_idx=1, and timestamps in explicit seconds. "
                "Only cite intervals in the video, not the reference image. "
                "If no match is visible, say that. If resolution or occlusion "
                "prevents a confident comparison, explain the uncertainty "
                "instead of claiming the product is absent."
            )},
        ],
    }],
    reasoning_effort="high",
    max_completion_tokens=2048,
    vision_config={"annotation_format": "clip"},
)
choice = response.choices[0]
if choice.finish_reason != "stop":
    raise RuntimeError(f"Incomplete comparison answer: {choice.finish_reason}")
print(choice.message.content or "")
```

The reference image is asset `0`; the query video is asset `1`. Temporal annotations must therefore select asset `1`. This is an illustrative matching interval, not measured output for the sample video:

```html theme={null}
<clip mention="matching reference package visible" asset_idx="1" t="2.0 seconds 4.5 seconds" />
```

Validate the selected asset before using a timestamp in a player. If you add more references or retain earlier media in the conversation, update the selectors according to [multiple-asset ordering](/perceptron-mk1.5/guides/multiple-assets).

## Make the comparison precise

* State whether you want the same class, the same product design, or the same individual object. Those are different matching tasks.
* Use a reference where the distinguishing features are visible. If it contains several objects, identify the intended one with a reviewed box, as in [image in-context learning](/perceptron-mk1.5/capabilities/in-context-learning-image).
* For several reference concepts, label each one and request separate evidence for each. Do not assume every reference has a match in the query video.
* Treat a missed or obscured match as uncertainty about the supplied footage. Selected frames do not show every instant of the source video.

## Choose the output you need

Request [clip annotations](/perceptron-mk1.5/capabilities/video-clipping) when the result should seek to supporting evidence. Request [video tracks](/perceptron-mk1.5/capabilities/video-tracking) when you need the matching object's position over time; the track belongs to the video asset, not the reference image.

If a track has an uncertain interval, [request new model observations around the gap](/perceptron-mk1.5/guides/high-fidelity-object-tracking#recover-a-gap-with-new-model-observations) before continuing local tracking. Keep crop coordinates and source timestamps associated with their own assets when combining the results.