mirror of
https://github.com/Tencent/BrowserSkill.git
synced 2026-09-10 13:01:23 +08:00
docs: document scroll-to primitive contract
This commit is contained in:
@@ -5,6 +5,13 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/).
|
||||
|
||||
Starting from 0.2.0, CLI / Extension / DSH Plugin share the same version number.
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Added
|
||||
|
||||
- [Scroll-to element primitive](docs/scroll-to.md) across CLI, Extension and DSH Plugin,
|
||||
with ancestor-clipped visible bounds, iframe support and cooperative cancellation
|
||||
|
||||
## [0.2.1] - 2026-09-09
|
||||
|
||||
### Changed
|
||||
|
||||
@@ -207,6 +207,9 @@ through the [plugin](#deepseek-harness-plugin): the agent calls injected
|
||||
|
||||
## For Developers
|
||||
|
||||
The [scroll-to primitive reference](docs/scroll-to.md) covers its CLI, protocol
|
||||
and plugin entry points, visible bounds and interruption behavior.
|
||||
|
||||
The repository is a Cargo + pnpm workspace:
|
||||
|
||||
- `crates/bsk-cli` — `bsk` CLI and local daemon
|
||||
|
||||
@@ -177,6 +177,8 @@ Agent 不直接与浏览器通信。它通过 `bsk` CLI 下发浏览器任务;
|
||||
|
||||
## 面向开发者
|
||||
|
||||
[scroll-to 原语说明](docs/scroll-to.md)介绍 CLI、协议和插件入口,以及可见区域、错误和中断语义。
|
||||
|
||||
本仓库是 Cargo + pnpm workspace:
|
||||
|
||||
- `crates/bsk-cli` — `bsk` CLI 与本地 daemon
|
||||
|
||||
@@ -55,7 +55,7 @@ Use this default loop:
|
||||
```text
|
||||
bsk navigate <url> --session <id>
|
||||
bsk observe --session <id>
|
||||
bsk click|hover|focus|blur|fill|select|press ... --session <id>
|
||||
bsk click|hover|scroll-to|focus|blur|fill|select|press ... --session <id>
|
||||
bsk observe --session <id> # after navigation or a meaningful DOM change
|
||||
```
|
||||
|
||||
@@ -63,7 +63,7 @@ bsk observe --session <id> # after navigation or a meaningful DOM ch
|
||||
Use a fresh element ref for iframe/shadow-root targets; CSS selectors search the main document.
|
||||
The result is the visible border-box portion's bounds in top-level viewport CSS pixels after
|
||||
ancestor clipping. Partial visibility is enough; hidden or fully clipped targets fail with
|
||||
`element_not_visible`. The result does not guarantee that other elements do not cover the target.
|
||||
`permission_denied` and `data.reason=element_not_visible`. This does not test occlusion by other elements.
|
||||
For a specific tab or deadline: `bsk scroll-to @e3 --session <id> --tab-id 42 --timeout 5s`.
|
||||
|
||||
`bsk focus <ref>` explicitly focuses a target; `bsk blur <ref>` removes focus and reports whether
|
||||
|
||||
@@ -84,6 +84,11 @@ Shared Rust types + JSON Schema generation. TypeScript mirrors frame shapes in
|
||||
4. Extension dispatcher validates sandbox rules, invokes CDP via `BrowserDriver`.
|
||||
5. Response travels CLI ← daemon ← extension; CLI prints result and exits.
|
||||
|
||||
The [scroll-to primitive reference](scroll-to.md) documents `tool.scroll_to`,
|
||||
including its CLI/plugin mappings, visible-bounds contract and cancellation
|
||||
behavior. It follows the same routing path and is classified as a browser
|
||||
mutation for session queueing and user-interruption gating.
|
||||
|
||||
## Session and sandbox model
|
||||
|
||||
- **Session** = opaque ID (4 lowercase letters in v0.1) + dedicated **Agent Window**
|
||||
|
||||
@@ -0,0 +1,131 @@
|
||||
# Scroll-to primitive
|
||||
|
||||
`scroll-to` brings an existing element and its containing frames into view, then
|
||||
returns the visible portion's bounds. Use it to reveal a target before inspecting
|
||||
the page or taking a screenshot. It accepts an element target rather than a pixel
|
||||
distance or scroll direction.
|
||||
|
||||
| Entry point | Name |
|
||||
| --- | --- |
|
||||
| CLI | `bsk scroll-to` |
|
||||
| Wire protocol | `tool.scroll_to` |
|
||||
| DeepSeek Harness | `browser_interact` with `action: "scroll-to"` |
|
||||
|
||||
## CLI
|
||||
|
||||
Use a session you started and a fresh ref from an observation. The ids below are
|
||||
examples; substitute the session, tab and ref returned by your own calls.
|
||||
|
||||
```sh
|
||||
bsk scroll-to @e3 --session abcd --tab-id 42 --timeout 5s --json
|
||||
bsk scroll-to --selector '#details' --session abcd
|
||||
```
|
||||
|
||||
Supply exactly one target: a positional ref or selector, `--ref`, or `--selector`.
|
||||
Refs accept `@e3` and `e3`. CSS selectors search only the main document; use refs
|
||||
for elements in iframes (including out-of-process iframes) or shadow roots.
|
||||
Refs belong to one session and tab and may become stale after page changes.
|
||||
|
||||
`--session` is required. `--tab-id` defaults to the Agent Window's active tab.
|
||||
`--timeout` defaults to `30s` and must be positive; durations such as `5000ms` and
|
||||
`5s` are accepted. The command operates on Agent Window tabs, including user tabs
|
||||
explicitly borrowed into that window.
|
||||
|
||||
Example JSON output:
|
||||
|
||||
```json
|
||||
{
|
||||
"tab_id": 42,
|
||||
"used_ref": "e3",
|
||||
"x": 10,
|
||||
"y": 20,
|
||||
"width": 100,
|
||||
"height": 80
|
||||
}
|
||||
```
|
||||
|
||||
`used_ref` or `used_selector` identifies the resolved target. The optional
|
||||
`dialogs` array reports JavaScript dialogs handled during the call.
|
||||
|
||||
## Result contract
|
||||
|
||||
`x` and `y` are the upper-left corner, and `width` and `height` are the size of
|
||||
the visible border-box portion's bounding rectangle. All four values use
|
||||
**top-level viewport CSS pixels**, after scrolling and clipping by ancestor
|
||||
containers and frame viewports. They are not document or screenshot pixel
|
||||
coordinates.
|
||||
|
||||
Partial visibility is sufficient: a 400px-high element clipped by an 80px-high
|
||||
scroll container can succeed with `height: 80`. Hidden or fully clipped targets
|
||||
fail. The rectangle is not an occlusion or hit test: another element can cover
|
||||
the target, and the rectangle's center is not a guaranteed click point. Layout
|
||||
can change after the result; use fresh refs for subsequent interactions.
|
||||
|
||||
## Protocol and plugin
|
||||
|
||||
The CLI sends a normal request through the daemon to the extension:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "scroll-1",
|
||||
"method": "tool.scroll_to",
|
||||
"params": {
|
||||
"session_id": "abcd",
|
||||
"ref": "@e3",
|
||||
"tab_id": 42,
|
||||
"timeout_ms": 5000
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Protocol callers must supply exactly one nonempty `ref` or `selector`.
|
||||
`tab_id` is optional; `timeout_ms` is a positive integer and defaults to 30000.
|
||||
These target constraints are enforced by the handler. See the generated
|
||||
[parameter schema](../crates/bsk-protocol/schema/tool_scroll_to_params.json) and
|
||||
[result schema](../crates/bsk-protocol/schema/tool_scroll_to_result.json) for field
|
||||
types. The wire response wraps the result above in `{ "id": "scroll-1", "result": ... }`.
|
||||
|
||||
The plugin exposes the same action with its usual camelCase parameters:
|
||||
|
||||
```json
|
||||
{
|
||||
"action": "scroll-to",
|
||||
"session": "abcd",
|
||||
"target": "@e3",
|
||||
"tabId": 42,
|
||||
"timeoutMs": 5000
|
||||
}
|
||||
```
|
||||
|
||||
Call this through `browser_interact` using a plugin-owned session. Its result
|
||||
contains `session`, `tabId`, `x`, `y`, `width` and `height` with the same bounds
|
||||
semantics. Omitted `session` uses the plugin's current session.
|
||||
|
||||
## Errors and interruption
|
||||
|
||||
The wire response uses the normal `error.code`, `error.message` and optional
|
||||
`error.data.reason` fields. Common cases are:
|
||||
|
||||
| Code | `data.reason` | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `invalid_params` | — | Missing/conflicting target or invalid timeout/tab id |
|
||||
| `not_found` | `ref_not_found` | Ref is unknown, expired or belongs to another tab |
|
||||
| `not_found` | `selector_not_found` | Main-document selector matched no element |
|
||||
| `permission_denied` | `agent_window_scope` | Target tab has not been borrowed into the Agent Window |
|
||||
| `permission_denied` | `element_not_visible` | No visible area remains after scrolling |
|
||||
| `cancelled` | — | The call was cancelled |
|
||||
| `timeout` | — | The action deadline expired |
|
||||
| `cdp_failed` | varies | Browser command, frame geometry or visibility measurement failed |
|
||||
|
||||
The protocol classifies scroll-to as a browser mutation, so it uses the existing
|
||||
per-session queue and user-interruption gate. The extension releases any retained
|
||||
hover state before dispatch. Cancellation and deadline checks surround each CDP
|
||||
step, including frame lookup, scrolling and measurement. Once detected, they
|
||||
prevent further scrolling and a successful response; remote-object cleanup still
|
||||
runs. Already performed scrolling is not rolled back, so inspect the current
|
||||
page before retrying an interrupted call.
|
||||
|
||||
The implementation is scoped to scroll-to: it reuses frame scrolling and
|
||||
coordinate projection, with a separate visibility measurement that accounts for
|
||||
ancestor clipping and hidden frame owners. Shared click, hover and screenshot
|
||||
geometry is unchanged.
|
||||
@@ -58,6 +58,9 @@ the `bsk` CLI and browser extension separately when a release requires it.
|
||||
| `browser_tabs` | `list`, `create`, `select`, `close`, `borrow`, `return` | Manage Agent Window tabs and temporarily borrow user tabs. |
|
||||
| `browser_assist` | `resize`, `emulate`, `request-help` | Resize or emulate the browser and pause for human-only steps. |
|
||||
|
||||
For `browser_interact` with `action: "scroll-to"`, see the
|
||||
[scroll-to reference](../../docs/scroll-to.md) for parameters, visible bounds and errors.
|
||||
|
||||
Arbitrary page-script evaluation and interaction recording are not supported.
|
||||
|
||||
## Multi-session model
|
||||
|
||||
+2
-2
@@ -55,7 +55,7 @@ Use this default loop:
|
||||
```text
|
||||
bsk navigate <url> --session <id>
|
||||
bsk observe --session <id>
|
||||
bsk click|hover|focus|blur|fill|select|press ... --session <id>
|
||||
bsk click|hover|scroll-to|focus|blur|fill|select|press ... --session <id>
|
||||
bsk observe --session <id> # after navigation or a meaningful DOM change
|
||||
```
|
||||
|
||||
@@ -63,7 +63,7 @@ bsk observe --session <id> # after navigation or a meaningful DOM ch
|
||||
Use a fresh element ref for iframe/shadow-root targets; CSS selectors search the main document.
|
||||
The result is the visible border-box portion's bounds in top-level viewport CSS pixels after
|
||||
ancestor clipping. Partial visibility is enough; hidden or fully clipped targets fail with
|
||||
`element_not_visible`. The result does not guarantee that other elements do not cover the target.
|
||||
`permission_denied` and `data.reason=element_not_visible`. This does not test occlusion by other elements.
|
||||
For a specific tab or deadline: `bsk scroll-to @e3 --session <id> --tab-id 42 --timeout 5s`.
|
||||
|
||||
`bsk focus <ref>` explicitly focuses a target; `bsk blur <ref>` removes focus and reports whether
|
||||
|
||||
Reference in New Issue
Block a user