jev-eyes

Give Jev eyes — honestly.

Jev, TypeSafe's System One model, is text-only. It has no world knowledge and does not see images. Any demo where Jev "looks at" a screenshot is doing perception somewhere else — usually hidden.

jev-eyes does it in the open. It turns an image into the text state Jev can actually decide over, locally, and hands it back as a plain dict. The state is the return value: you always see exactly what Jev saw.

pip install jev-eyes               # until the PyPI release lands: pip install git+https://github.com/LeddoEngano/jev-eyes
jev-eyes see screenshot.png --show # no API key needed

Rendered from real output of the command above — nothing typed in by hand. docs/demo.tape reproduces it as a true recording with vhs.

In your code: one new line

from jev_eyes import see
from typesafe_sdk import TypeSafeClient, Noul, Choice

result = TypeSafeClient().system_one(
    state=see("screenshot.png"),          # ← that's the whole integration
    questions={
        "decorative": Noul(instructions="Is this image purely decorative, carrying no information?"),
        "kind": Choice(instructions="What is this image?",
                       criteria={"screenshot": None, "chart": None, "photo": None, "document": None}),
        "needs_vision": Noul(instructions="Is the extracted text insufficient to describe what the image shows?"),
    },
)
result.answers["decorative"].noul      # float 0..1 — Jev's calibrated probability
result.answers["kind"].choice          # str; .probabilities has every option
result.answers["needs_vision"].noul    # your gate: escalate to a VLM only when this is high

see() never needs a key. ask(image, questions) does both steps when TYPESAFE_API_KEY is set.

Related MCP server: Image Reader MCP

What Jev receives

Real output on the 1440×900 banking screen committed at examples/sample_screen.png. Nothing was edited — reproduce it with jev-eyes see examples/sample_screen.png --no-blocks --cols 80.

text — OCR in reading order; same-row blocks joined by two spaces:

Conta corrente
Início  Saldo disponível  Fatura do cartão
Extrato  R$ 4.820,15  Fecha em 3 dias · R$ 1.284,77
Cartões  Limite disponível R$ 1.200,00
Investimentos  atualizado às 14:32· agência 0341 · conta 88192-4
Ajustes
Últimas transações
18 set  Mercado Pão de Açúcar  -R$ 214,90
17 set  Transferência PIX recebida  +R$ 1.500,00
17 set  Assinatura Spotify Premium  -R$ 34,90
16 set  Posto Ipiranga Centro  -R$ 180,00
15 set  Farmácia Droga Raia  -R$ 62,45
14 set  Salário  +R$ 7.200,00
Transferir  Extrato

layout — the same words at their true position on a character grid. This is what lets Jev answer where things are ("which region is the main content?", "is the error next to the form?") without a single pixel reaching it:

            Conta corrente

 Início       Saldo disponível              Fatura do cartão
 Extrato      R$ 4.820,15                   Fecha em 3 dias · R$ 1.284,77
 Cartões                                    Limite disponível R$ 1.200,00
 Investimentos atualizado às 14:32· agência 0341 · conta 88192-4
 Ajustes
              Últimas transações
              18 set Mercado Pão de Açúcar               -R$ 214,90

              17 set Transferência PIX recebida          +R$ 1.500,00
              17 set Assinatura Spotify Premium          -R$ 34,90

              16 set Posto Ipiranga Centro               -R$ 180,00
              15 set Farmácia Droga Raia                 -R$ 62,45
              14 set Salário                             +R$ 7.200,00

              Transferir Extrato

Also in the state: image (size, source), blocks ([x, y, w, h] boxes with OCR confidence) and, if installed, labels. see(img, compact=True) keeps only image, text and top label names when tokens matter more than positions.

Why this and not…

• …ASCII art? It only becomes legible at ~200 columns, where it costs 15× the tokens of OCR for a blurry copy of what OCR returns exactly — and a text model does not read ASCII the way your eyes do. Measured in benchmarks/.

• …a vision-language model? Slower, and it generates prose that Jev then has to re-read. Keep one for what OCR and layout cannot answer; the needs_vision question above is the gate.

• …training a classifier? Jev decides; it does not recognise. Recognition belongs in the perception layer, where pretrained encoders already do it for free.

Optional: zero-shot labels

pip install 'jev-eyes[labels]'      # SigLIP via torch + transformers, ~1 GB

see("photo.jpg", labels=["a flower", "a dog", "a car"])

Scores are a softmax over the candidates you pass — a relative ranking, not a calibrated probability, and it says nothing about labels you did not offer. On the banking screen above, the default vocabulary ranks it as "a text document" (0.70), then "a blank or empty screen" (0.12) and "a table of data" (0.08) — a fair description of a white synthetic UI, and a reminder that labels are hints for Jev, not truth.

For agents

The repo ships a skill in the same format TypeSafe uses for Jev itself:

npx skills add LeddoEngano/jev-eyes --skill jev-eyes     # or copy skills/jev-eyes into your agent's skills dir

And an MCP server:

pip install 'jev-eyes[mcp]'
claude mcp add jev-eyes -- jev-eyes mcp

Tools: see(image_path, labels?, compact?, layout?, cols?) and ask(image_path, questions, ...). Both return the state alongside any notes about missing backends, so an agent can never silently pretend.

Honest limits

• OCR on CPU takes seconds, not milliseconds — about 1.5–2.5 s per full-resolution screenshot on an 8-core laptop, measured. Crop to the region you care about, cache by image hash, or downscale to ~1280 px. An Apple Vision backend is on the roadmap.

• Labels cost a model load — the first call in a process loads ~400 MB (around a minute); after that ~200 ms per image on Apple Silicon, ~2.8 s on plain CPU.

• The default OCR is PP-OCR (PP-OCRv6, keeps accents). pip install 'jev-eyes[tesseract]' and ocr="tesseract" if you prefer Tesseract.

• Jev decides; it does not describe. If your task is "describe this image", you need a vision-language model, not Jev.

• jev-eyes is an independent project, not affiliated with or endorsed by TypeSafe AI.

Roadmap

JavaScript package for @typesafe-ai/sdk users · Apple Vision OCR backend · accessibility-tree input for browsers · perceptual-hash cache · PDF pages · a labelled evaluation set so benchmarks/jev_accuracy.py has real numbers.

Development

python -m venv .venv && .venv/bin/pip install -e '.[dev]'
.venv/bin/pytest -q -m "not slow"

MIT.