> ## Documentation Index
> Fetch the complete documentation index at: https://docs.perceptron.inc/llms.txt
> Use this file to discover all available pages before exploring further.

# Video Q&A

> Ask about goals, actions, and outcomes observed in a video.

Video Q\&A lets you ask about the actions, sequence, and outcome of an episode. Start with a concrete question, and ask the model to separate visible evidence from any inferred intent. For example, an assembly video may show several actions toward a goal without showing whether the final assembly succeeds.

## Identify a goal and its subgoals

The following request asks about a robot-assembly episode. It requests a concise account of the overall goal, the observed subgoals in order, and whether the ending supports a completion claim.

Install the `perceptron>=0.4.0` Python package and set `PERCEPTRON_API_KEY` before running:

```python theme={null}
import os

from perceptron import Client, video

client = Client(api_key=os.environ["PERCEPTRON_API_KEY"])
video_url = (
    "https://raw.githubusercontent.com/perceptron-ai-inc/perceptron/"
    "main/cookbook/_shared/assets/capabilities/video-qa/robot_assembly.mp4"
)
messages = [{
    "role": "user",
    "content": [
        video(video_url),
        {"type": "text", "text": (
            "Watch this robot-assembly episode. State the likely overall goal, "
            "then list the observed subgoals in chronological order. "
            "For each subgoal, describe the visible action that supports it. "
            "Distinguish an inferred intention from something directly visible. "
            "Finally, say whether the video shows the overall goal being "
            "completed, remaining incomplete, or an uncertain outcome."
        )},
    ],
}]
response = client.chat.completions.create(
    model="perceptron-mk1.5",
    messages=messages,
    reasoning_effort="high",
    max_completion_tokens=2048,
)
choice = response.choices[0]
if choice.finish_reason != "stop":
    raise RuntimeError(f"Incomplete video answer: {choice.finish_reason}")
answer = choice.message.content or ""
print(answer)
```

The answer is natural language. `reasoning_effort` controls reasoning for the request; it does not require you to display a reasoning trace in your application. If you need fixed fields for downstream processing, define a [JSON Schema](/perceptron-mk1.5/capabilities/structured-outputs) for fields such as `goal`, `subgoals`, and `outcome`.

## Ask a follow-up grounded in time

A follow-up question can use the first answer while keeping the original video in the supplied history. Run this after the example above to ask for evidence near the end of the episode:

```python theme={null}
messages.extend([
    {"role": "assistant", "content": answer},
    {"role": "user", "content": (
        "Recheck the ending of the video. Which visible action most directly "
        "supports your outcome assessment? Describe it and cite its interval "
        "with a clip annotation for asset_idx=0, using explicit seconds. "
        "If the ending is inconclusive, explain what evidence is missing."
    )},
])
follow_up = client.chat.completions.create(
    model="perceptron-mk1.5",
    messages=messages,
    reasoning_effort="high",
    max_completion_tokens=2048,
    vision_config={"annotation_format": "clip"},
)
choice = follow_up.choices[0]
if choice.finish_reason != "stop":
    raise RuntimeError(f"Incomplete follow-up answer: {choice.finish_reason}")
print(choice.message.content or "")
```

The video remains asset `0`: the assistant answer and follow-up add text but no new media. The service receives the full `messages` list on each request; it does not retain this history automatically.

## Combine audible and visible evidence

The robot-assembly sample above has no audio track. For a question that depends on sound, supply a video with audio and set `vision_config.enable_audio_in_video` to `true` on each request that needs it. This example uses a basketball clip with a soundtrack; run it after creating `client` above:

```python theme={null}
response = client.chat.completions.create(
    model="perceptron-mk1.5",
    messages=[{
        "role": "user",
        "content": [
            video(
                "https://raw.githubusercontent.com/perceptron-ai-inc/perceptron/"
                "main/cookbook/_shared/assets/capabilities/video-clipping/mj_shot_short.mp4"
            ),
            {"type": "text", "text": (
                "What happens during the basketball play, and what does the soundtrack "
                "add to that account? Separate visible actions from audible speech or "
                "reactions. Mark unclear speech instead of guessing the words."
            )},
        ],
    }],
    reasoning_effort="high",
    max_completion_tokens=2048,
    vision_config={"enable_audio_in_video": True},
)
choice = response.choices[0]
if choice.finish_reason != "stop":
    raise RuntimeError(f"Incomplete audiovisual answer: {choice.finish_reason}")
print(choice.message.content or "")
```

Soundtrack analysis is off by default. A video without an audio stream is still processed visually; an audio decode error is not evidence of silence. For standalone recordings, use the [audio inputs](/perceptron-mk1.5/capabilities/audio). See [video soundtracks](/perceptron-mk1.5/capabilities/video-understanding#analyze-video-soundtracks) for supported video inputs and limits.

## Choose the question and evidence

* Ask for observable actions when labeling a task sequence. Use separate language for inferred goals and observed completion.
* Narrow the question or supply a relevant excerpt when a long video contains many unrelated activities. Sampled frames can miss brief actions.
* Use [video clipping](/perceptron-mk1.5/capabilities/video-clipping) for temporal evidence, or [video tracking](/perceptron-mk1.5/capabilities/video-tracking) when the answer needs an object's changing position.
* Use [Multilook](/perceptron-mk1.5/guides/multilook) for independent questions over the same video. Keep dependent follow-ups in a conversation so they include the earlier answer.

See [video understanding](/perceptron-mk1.5/capabilities/video-understanding) for supported video inputs and timestamped frames.