> ## Documentation Index
> Fetch the complete documentation index at: https://docs.perceptron.inc/llms.txt
> Use this file to discover all available pages before exploring further.

# OCR

> Extract product names, prices, and document text as structured data or grounded spans.

Perceptron Mk1.5 can read text in images and extract the fields relevant to your application. Ask for a transcription, a product-and-price table, or a constrained JSON record. Request spatial annotations when you need to show where a text span came from.

## Extract product names and prices

Install `perceptron>=0.4.0` and `jsonschema`, and set `PERCEPTRON_API_KEY`. This example uses the public grocery-label image. It preserves printed prices as strings so currency symbols and decimal formatting are not lost, and allows `null` when a requested field is unreadable.

```python theme={null}
import json
import os

from jsonschema import validate
from perceptron import Client, image

client = Client(api_key=os.environ["PERCEPTRON_API_KEY"])
image_url = (
    "https://raw.githubusercontent.com/perceptron-ai-inc/perceptron/"
    "main/cookbook/_shared/assets/capabilities/ocr/grocery_labels.webp"
)
schema = {
    "type": "object",
    "properties": {
        "items": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "product": {"type": ["string", "null"]},
                    "price": {"type": ["string", "null"]},
                },
                "required": ["product", "price"],
                "additionalProperties": False,
            },
        },
    },
    "required": ["items"],
    "additionalProperties": False,
}
response = client.chat.completions.create(
    model="perceptron-mk1.5",
    messages=[{
        "role": "user",
        "content": [
            image(image_url),
            {"type": "text", "text": (
                "Extract the visible product names and their associated listed prices "
                "in reading order. Preserve spelling, currency symbols, and decimal "
                "separators exactly as printed. Use null for a field you cannot read; "
                "do not guess or pair a product with an unrelated price. Return an "
                "empty items array if there are no relevant labels."
            )},
        ],
    }],
    reasoning_effort="high",
    max_completion_tokens=2048,
    response_format={
        "type": "json_schema",
        "json_schema": {"name": "product_prices", "strict": True, "schema": schema},
    },
)
choice = response.choices[0]
if choice.finish_reason != "stop" or choice.message.tool_calls:
    raise RuntimeError(f"Incomplete OCR response: {choice.finish_reason}")
result = json.loads(choice.message.content or "")
validate(instance=result, schema=schema)
print(json.dumps(result, indent=2))
```

An illustrative response shape is:

```json theme={null}
{
  "items": [
    {"product": "Example product", "price": "$3.49"},
    {"product": "Another product", "price": null}
  ]
}
```

These entries are invented to demonstrate the schema, not a transcription of the sample image. Validate extracted values against the image when accuracy matters. JSON Schema controls structure, not whether the reading or product-price association is correct. See [structured outputs](/perceptron-mk1.5/capabilities/structured-outputs) for other schema and validation patterns.

## Extract a Markdown table

For a human-readable result, request Markdown instead of a JSON Schema response. Run this after the client and image setup above:

```python theme={null}
response = client.chat.completions.create(
    model="perceptron-mk1.5",
    messages=[{
        "role": "user",
        "content": [
            image(image_url),
            {"type": "text", "text": (
                "Read the visible grocery labels and return a Markdown table with "
                "Product and Price columns, one row per label in reading order. "
                "Preserve the printed text and currency formatting. Write [unreadable] "
                "for unclear text instead of guessing. Do not add code fences."
            )},
        ],
    }],
    reasoning_effort="high",
    max_completion_tokens=2048,
)
choice = response.choices[0]
if choice.finish_reason != "stop" or choice.message.tool_calls:
    raise RuntimeError("The Markdown extraction did not complete")
print(choice.message.content or "")
```

For a document page, change the instruction to “Transcribe the visible text in reading order as Markdown; preserve headings, lists, and tables.” If your application needs HTML, request semantic headings, paragraphs, and tables in the prompt. These formats express document structure rather than exact pixel layout. Use the JSON Schema workflow when downstream code requires a fixed set of fields.

## Locate the text spans

Use a separate annotation request when you want boxes around the evidence. This example asks the model to put each transcribed span in its box's `mention` attribute:

```python theme={null}
from pathlib import Path

response = client.chat.completions.create(
    model="perceptron-mk1.5",
    messages=[{
        "role": "user",
        "content": [
            image(image_url),
            {"type": "text", "text": (
                "Locate the visible product-name and price text spans. Return a "
                "flat point_box for each span, with the transcribed text as its mention "
                "and asset_idx=0. Use normalized coordinates, without collections or tracks. Do not invent text "
                "that is too small or obscured to read."
            )},
        ],
    }],
    reasoning_effort="high",
    max_completion_tokens=2048,
    vision_config={"annotation_format": "box"},
)
choice = response.choices[0]
if choice.finish_reason != "stop" or choice.message.tool_calls:
    raise RuntimeError("The grounded OCR answer did not complete")
annotations = choice.message.content or ""
print(annotations)
Path("response.txt").write_text(annotations, encoding="utf-8")
```

Boxes use normalized 0–1000 coordinates, with top-left followed by bottom-right. `asset_idx="0"` selects the supplied image. Follow [rendering annotations](/perceptron-mk1.5/guides/rendering-annotations) to draw `response.txt` over it, and validate attributes and geometry using the [annotation reference](/perceptron-mk1.5/concepts/annotations). OCR markup and constrained JSON are distinct output formats; the API does not automatically attach boxes to the JSON records from the earlier request.

## Improve extraction quality

* State the fields and reading order you need. Distinguish a shelf price, a unit price, and a promotional price when several numbers appear near one product.
* Preserve identifiers, prices, dates, and other exact text as strings until your application applies its own normalization rules.
* Use an image with enough detail for the smallest relevant text. If you supply a crop as well as the original, treat it as another asset and keep the crop-to-source mapping.
* For several pages or labels, keep their order and request explicit `asset_idx` on grounded spans. See [multiple assets](/perceptron-mk1.5/guides/multiple-assets).