ui2.scene_match
Classify screenshots or crops against candidate scene/pattern labels with CLIP similarity, returning zero-shot scores without training.
Instructions
场景/花纹零样本分类(小模型层):给候选标签短语(如 '设置页面'/'聊天列表页'),CLIP 类模型计算图文相似度返回各候选得分——加新场景=加一个标签,无需训练。定位:色彩/布局等像素统计概括不了的花纹与场景语义。需可选依赖:pip install 'mobile-agent-harness[scene]';模型经 MAH_SCENE_MODEL 配置(中文标签用 OFA-Sys/chinese-clip-vit-base-p16,英文标签默认 openai/clip-vit-base-patch32)
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| box | No | [l,t,r,b] 只对裁剪区域分类 | |
| top | No | ||
| from | No | 截图路径,缺省最近一次 vision.screenshot | |
| candidates | Yes | 候选场景/花纹标签短语(封闭集合声明) |