Process an image file or URL using OCR to extract text.

Uses EasyOCR for robust scene-text extraction. Excels at photos, signage, watermarks, logos, and printed text.

This is the right tool for any text in an image. EasyOCR handles both of the cases that used to need separate routing -- dense printed text and stylized, cursive, logo or low-contrast text -- so there is no longer a reason to send the second kind to query_image, which paraphrases rather than transcribing.

A page laid out in side-by-side columns (a form, meeting notes, a resume) is detected automatically: each column is OCR'd separately and joined in reading order, so fields from different columns don't get interleaved.

An image with no text returns an empty result, and that is the correct answer rather than a failure -- do not re-ask a captioning or VQA model to produce text for it, which is how invented text gets into a payload.

Set detail=true to get confidence scores and bounding boxes. This is highly recommended for checking generative image artifacts: if an image contains gibberish text, the confidence scores will drop significantly.

Describe what an image shows, as one detailed prose caption.

The default choice for "what is this a picture of". Returns a single paragraph covering the scene as a whole, with no coordinates.

Reach for a different tool when the question is narrower: query_image to ask something specific about the image, dense_region_caption to get a separate caption and box for each thing in it, and ocr to transcribe text rather than describe it. Returns one caption per page for a PDF.

Do not trust any text this quotes back. A caption that mentions a name, brand or label is describing it, not transcribing it, and Florence-2 misspells text here that it reads correctly under ocr -- it rendered a logo reading "FusionVisionMCP" as "FusionVisionMP" mid-caption while ocr read the same image exactly. When a specific piece of text matters, confirm it with ocr, which handles stylized and cursive text as well as printed text, rather than quoting this.

Set verify_text=true to have this confirmation done for you: the tool also runs the OCR-with-region head and returns each verbatim text span alongside its box, so a name the caption quoted can be checked against what was actually read without a second call. It then goes further and corrects the close misses: any caption token that is similar to (but not identical to) a verbatim OCR span is substituted with the verbatim text in a caption_corrected copy, and every such change is listed in corrections (with the quoted-in-caption token, the verbatim OCR text, the box and the similarity) so the substitution is auditable. The return shape becomes one dict per page ({caption, text_regions, corrections, caption_corrected}) when this is set. The correction is best-effort and only fires for high-similarity same-word matches; the raw corrections list is always present so a caller can audit or reject any change.

Set auto_verify_text=true to automatically detect when a caption quotes embedded text or signage and run specialist OCR (EasyOCR) on upscaled crops of the text regions, providing consensus metrics (text_consensus) and warnings (caption_text_warning).

Run a raw Florence-2 task token against an image (escape hatch).

prompt must be a Florence-2 task token, not an instruction: '', '', '' and the like. Passing plain English ("describe this image") does not fail — it returns confident nonsense, because the model has no such task and decodes the words as one anyway.

Only for task tokens the named tools do not already cover. Prefer caption, ocr, detect_objects and dense_region_caption: they wrap the common tokens, parse the structured output into usable fields, and document where each one misleads. This returns raw text either way.

Locate a named object in an image, as bounding boxes and center points.

Returns bboxes ([x1, y1, x2, y2] each), points (the center of each box) and labels, all index-aligned -- so use this whether you want regions or coordinates; the centers come free with the boxes.

If return_annotated is true, the return value becomes a dictionary containing the regular results plus an annotated_image_path key.

The count of results is NOT a reliable count of objects on an ambiguous class name: Florence-2 can return several overlapping results for one physical object (a whole-animal box plus sub-part boxes all labelled 'wing'), or a single result spanning two touching instances (two fused blades labelled once as 'sword blade'). Prefer a more specific object_name, and treat results as candidates to inspect, not a tally. Use count_objects when you actually need "how many" -- it runs a detection head that emits one region per instance, which this head does not. Neither can separate heavily overlapping instances, so a count of 1 from either means "could not separate", not "there is one".

Boxes cannot answer whether two objects actually touch or whether one is inside another -- they overlap the moment one object is merely in front of another. Use spatial_relations for that.

A maximally vague object_name ("object", "thing", "item", ...) on a scene with nothing distinctive to point to gets a near-full-frame box filtered out rather than returned as a match -- Florence-2's grounding head has no explicit "nothing here" output for that case and falls back to the whole image otherwise. This only applies to the exact generic vocabulary itself: a specific noun that genuinely fills the frame (a close-up of wood, asked for "wood") is returned unfiltered.

Caption every salient region of an image at once, with bounding boxes.

Use this to inventory an image without knowing in advance what is in it -- it returns bboxes and labels for each region it finds, discovering the objects itself. That is the difference from detect_objects, which needs you to name the object you are looking for, and from caption, which describes the whole scene in prose with no coordinates.

Count how many instances of a named object an image contains.

Use this, not detect_objects, whenever the question is "how many". detect_objects returns however many regions Florence-2's grounding head emits, which is not a tally: it collapses several repeated, undifferentiated parts into one box (every petal of a flower, both halves of a fused blade) and conversely splits one object into overlapping sub-part boxes.

Returns count, plus bboxes/points/labels/scores for the instances found, index-aligned and in the same pixel-space convention detect_objects uses. scores are per-detection confidences, so a count resting on marginal detections is visible rather than implied. group_boxes_dropped counts detections that enclosed the whole arrangement rather than one instance.

Accuracy is measured, not assumed: against a fixture suite whose synthetic counts are exact by construction, this is right on 9 of 10 positive cases with a mean error of 0.1, while holding all 8 single-object controls. Separated, touching, dense, two-instance and awkwardly-named cases are reliable.

Two measured limits, worth checking before trusting a count:

Heavy overlap undercounts. Eight identical shapes in a ring count as 8 when separated or touching, but 6 once they overlap by roughly two-thirds of their width. A count below what you expect means "could not separate them", not a real tally -- and it will undercount rather than overcount.

Some evidence is simply not in the picture. On a photo of a paper flower whose petals overlap, this returns 1 at every resolution from 128px to 768px, as does every other approach tried -- other detectors, outline geometry, tiled crops, and interior-colour analysis alike. That flower's outline is 98% convex and its interior contrast is near zero, so nothing measurable distinguishes the petals. Ask query_image for a count in that situation and treat it as an estimate.

When only one region is found, a silhouette block is added measuring how many repeated lobes that region's outline contains. count and silhouette.lobes come from different methods and neither overrides the other: count: 1 beside silhouette.lobes: 8 means the detector could not separate the instances while the outline shows eight cores. agreement: true means a second, independent estimator matched it; shattered or clipped mean the number should not be used at all.

Set consensus=true (the default) for two extra fields that surface structural ambiguity instead of hiding it:

• consensus: {detector_count, region_label_count, agree} -- a second count from Florence-2's dense region captioner (a different head), which tally how many region labels contain the object name. Agreement is real evidence; disagreement is a warning.

• separable: "yes" if the count is a real tally (the detector separated instances, or a single region the silhouette confirms is one lobe), "no" if the detector collapsed while the outline shows several lobes (the overlapping-petal case: nothing local can count it honestly), "unknown" when there's no silhouette check to confirm either way. Read count with this in mind: a count: 1 with separable: "no" is a collapse, not a tally.

When separable is "no" but the outline still carries the lobe pattern (silhouette.by_radial > 1), an estimates block is added reporting that outline count as an actionable number -- it's a measurement (angular notches in the silhouette), not a judgment, so it stays within "measure, don't judge". count is never overwritten; estimates.outline is the number the outline supports, with a basis string saying so. Set vqa_estimate=true to also ask Moondream2 "how many ?" and attach Additionally, every result reports semantic_ambiguity (bool) and count_semantics ("measured_tally", "minimum_visible_instances", or "unverified_tally"). When semantic_ambiguity is true, an ambiguity payload is attached detailing why the count represents a collapsed or ambiguous structure rather than a confirmed discrete tally.

A maximally vague object_name ("object", "thing", "item", ...) is guarded the same way detect_objects is: Grounding DINO has the same fallback-to- the-whole-frame failure mode on a name with nothing distinctive to point to, so a near-full-frame box is dropped rather than counted as a real instance. clip_art=true gets the equivalent guard through Florence-2's own exclude_full_frame. Only the exact generic vocabulary triggers this; a specific noun that genuinely fills the frame is counted unfiltered.

Measure how named objects in an image sit relative to one another.

Locates each object, segments it, and reports measurements that are hard to judge by eye: whether two things actually touch, how many pixels apart they are, how much of one lies inside the other and how deeply, plus each object's own elongation, straightness and end-to-end width profile.

This reports geometry, not verdicts — it does not decide what is wrong. Interpret the numbers against what the scene ought to look like: a hand and the grip it holds that come back separate with a large gap are not in contact; a hand overlapping a shield with a_inside_b near 1.0 and a large embed_depth is buried in the shield face rather than gripping its rim; an elongated object whose end_symmetry is near 1.0 is equally wide at both ends, unlike a blade that tapers to a point at one end only.

Useful for checking whether a generated or edited image holds together physically, for verifying that an object is where it should be relative to another, and for any question of contact, containment or clearance that a bounding box cannot answer — boxes overlap whenever one object is simply in front of another.

Takes the single best-scoring match per name, so this assumes one instance of each named object. Asked for 'red circle'/'blue circle'/'green circle' on a scene with one of each, the detector returned the same three boxes for every query — color alone doesn't reliably discriminate same-shaped objects — but the correctly-matching box scored highest every time, which is what this relies on. For several instances of one kind of thing, give them distinguishing names, or use count_objects for a tally instead.

A maximally vague name in objects ("object", "thing", "item", ...) is guarded the same way detect_objects/count_objects are: a near-full-frame box from a name with nothing distinctive to point to is dropped rather than treated as a located match, so it doesn't get reported here as the "best-scoring" box for that name.

Ask a free-form question about an image (visual question answering).

This is the right tool for reading photo watermarks, logos, signage, or any cursive/stylized/low-contrast text — ask e.g. "What does the text/watermark say, exactly?". The ocr tool misreads that kind of text confidently; prefer this one for it instead.

Moondream2 is a small model and is documented to answer open-ended judgment questions ("describe anything wrong in this image") with a flat "None" on images that all had real visible defects, and to give the same yes/no answer across genuinely different images -- a default response, not a real observation. Set check_consistency=true to make that visible: the tool also asks a rephrased control question and returns, per image, {answer, control_answer, consistent, confidence}. confidence is "low" in either of two failure modes: the two answers agree on a short default-looking token ("None", "Yes", "Nothing", ...), which is the signature of a flat default rather than a genuine observation; or they substantively contradict each other, which makes either answer weaker evidence than it looks alone. "normal" requires substantive answers that agree. A low result on a judgment question means you should not trust the answer without independent confirmation. When the answer is low-confidence, the tool also tries to route to the measurement that actually answers the question: it classifies the question's wording and, if a measurable category applies and the object names parse from the wording, runs that tool's measurement (spatial_relations for a contact/containment question, count_objects for "how many", ocr for a text-reading question, detect_objects for "which is largest/smallest") and attaches it as cross_check. The "largest/smallest" case detects every instance of the named object and picks the extremum by bounding-box area, returning its box -- so "which circle is biggest" resolves to coordinates, not a repeated guess. The cross-check is omitted when no measurement applies or the names can't be parsed -- it never guesses.

Run one operation across many images in a single call.

The batch form of caption, ocr, detect_objects, count_objects, dense_region_caption and query_image -- pick which with operation. Use it when the same question applies to a whole set of images, since it costs one round trip instead of one per image and loads each model once for the whole run.

Failures are isolated per image: a missing file or an unreachable URL is reported as its own {"src", "success": false, "error"} entry and the rest of the batch still runs. Results come back in the order given.

For a single image, call the named tool directly -- its arguments are checked up front rather than depending on operation.

Rate how aesthetically pleasing an image looks, independent of its content.

Uses a CLIP-based predictor trained on human aesthetic ratings (the LAION "improved aesthetic predictor"). Reflects visual qualities like lighting, composition and clarity — not whether the subject matter is correct or matches a prompt. A technically accurate but flatly-lit, cluttered photo can score low; a blurry but beautifully lit one can score comparatively higher.

In v0.8.0, the returned payload is enriched. The legacy score is duplicated as photographic_aesthetic. If a Reasoner is configured at the server level, an artistic_judgment is generated. If the server is installed with the [iqa] extra, a technical_quality score (0-100) is included via the MUSIQ ONNX model.

Returns one {"score": float, "rating": str} object per page/image. score is roughly on a 1-10 scale; rating buckets it coarsely for quick triage — read score for anything comparative.

Its training set was photographic, so it rates photographs, not fine art: celebrated paintings and illustrations score middling (Hokusai's "The Great Wave" comes back around 5.8) without that meaning anything is wrong with them. Use it to compare like with like — several shots of the same subject, or successive edits of one image — and do not read a single absolute score as a verdict on quality.

Set style_context=true to also get the medium the score is being read in. The CLIP backbone (already loaded for scoring) classifies the image as a photograph, oil painting, digital illustration, etc., and that style plus its style_distribution are added to each result. This is the local-model answer to the photography bias: it doesn't make the head understand fine art, but it tells you that the score is for a non-photographic medium, so you read it with the documented caveat instead of as an absolute verdict.

Set compare_with to a reference image to switch to relative mode: both images are scored and the result becomes one entry per image page carrying {image, reference, delta, preferred}, with preferred being "image" / "reference" / "tie" (tie when |delta| < 0.05). This is the predictor's calibrated use -- like-with-like comparison -- so it sidesteps the absolute photography bias that makes a single score misleading across media. With style_context=true, both media are classified and a cross_medium_warning is added when they differ (cross-medium comparison is out of calibrated scope). The absolute score is not recalibrated; the relative delta is the actionable output.

Critique an image's composition: framing, and, for low-scoring images, why it looks off.

Combines score_aesthetics (numeric quality), a rule-of-thirds/centeredness check on the main subject's bounding box, and — only when the aesthetic score is below low_score_threshold — a Moondream2 VQA explanation of what specifically looks unbalanced. Use it over score_aesthetics alone when you need to know why a shot is weak and where its subject sits in the frame, rather than just how it scores.

Returns image_size, subject_box, aesthetics, and framing (with thirds_offset near 0 meaning the subject sits on a rule-of-thirds power point, and center_offset near 0 meaning it is dead-center instead).

Pass target_subject whenever you know what the subject is — from your own context or a prior caption call. Auto-detection picks the largest, most central region and degrades on busy scenes that fill the frame, where no single region is the subject. If nothing can be located, returns a soft-failure shape (image size, aesthetic score, and a "note") rather than raising. Only the first page of a PDF is assessed.

Set style_context=true to also get the image's medium (style plus style_distribution), so the aesthetic score is read in the context of its medium — the documented photography bias means a non-photographic medium should not be judged by the raw score.

Set compare_with to a reference image to switch to relative mode: both images are critiqued and the result becomes {image, reference, delta, preferred} (tie when |delta| < 0.05), plus a cross_medium_warning when style_context=true and the two media differ. Same like-with-like framing as score_aesthetics' compare_with; the absolute score is not recalibrated, the relative delta is.