figmar/BrowserSkill
1
0
Fork 0
mirror of https://github.com/Tencent/BrowserSkill.git synced 2026-09-10 13:01:23 +08:00
Code Issues Packages Projects Releases Wiki Activity

feat(vom): two channels to deal with canvas

Browse Source
This commit is contained in:
Ljy-0827
2026-09-08 10:54:33 +08:00
parent 672b17cea8
commit 905ba6a8a6
6 changed files with 45 additions and 21 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/bsk-cli/skill/SKILL.md
+9 -4
View File
@@ -73,10 +73,15 @@ CSS-only hover menu looks like from here. It hovers a bounded set of likely trig
few seconds and touches the live page; once you know which element hides the menu, `bsk hover <ref>`
is cheaper and more precise.
When `bsk observe` renders `@eN surface ... [visual-only; ...]`, the ref identifies a rendered
canvas surface, not an interactable DOM control. Use
`bsk screenshot --ref @eN --session <id>` to inspect its visible crop. Do not pass a visual surface
ref to click, fill, hover, or select; those commands intentionally reject screenshot-only refs.
When `bsk observe` renders
`@eN surface ... [visual-only; requires=image-understanding; ...]`, the ref identifies rendered
canvas content that is not represented by the text observation, not an interactable DOM control.
Keep using any reliable DOM/AX text and controls that appear alongside it. If you can actually
inspect image output, use `bsk screenshot --ref @eN --session <id>` to obtain the visible crop. If
you lack multimodal or image-reading capability, do not take a screenshot and pretend to know its
contents, and never guess coordinates; tell the user that they need to switch to a model with
image-understanding capability. Do not pass a visual surface ref to click, fill, hover, or select;
those commands intentionally reject screenshot-only refs.
Escalate page reading only as needed:
crates/bsk-cli/src/cli/render_error.rs
+1 -1
View File
@@ -303,7 +303,7 @@ pub fn info_for_error(code: ErrorCode, data: Option<&serde_json::Value>) -> Rend
(ErrorCode::PermissionDenied, reason::REF_CAPABILITY_DENIED) => RenderInfo {
summary: "snapshot ref does not support this operation",
hint: Some(
"visual surface refs are screenshot-only; run `bsk screenshot --ref <ref> --session <id>`",
"visual surface refs are screenshot-only; inspect `bsk screenshot --ref <ref> --session <id>` with image-understanding capability",
),
exit_code: base.exit_code,
},