mirror of
https://github.com/Tencent/BrowserSkill.git
synced 2026-09-10 13:01:23 +08:00
Compare commits
3
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
a3bb81bfab | ||
|
|
187b088013 | ||
|
|
5046b5ab44 |
@@ -147,20 +147,26 @@ Start a new Agent session and write a prompt that needs the browser, for example
|
|||||||
Using [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) (`dsh`)?
|
Using [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) (`dsh`)?
|
||||||
BrowserSkill ships a first-class dsh plugin on npm as
|
BrowserSkill ships a first-class dsh plugin on npm as
|
||||||
[`@wxg-prc-cpg/browser-skill-dsh-plugin`](https://www.npmjs.com/package/@wxg-prc-cpg/browser-skill-dsh-plugin).
|
[`@wxg-prc-cpg/browser-skill-dsh-plugin`](https://www.npmjs.com/package/@wxg-prc-cpg/browser-skill-dsh-plugin).
|
||||||
It injects native `browser_*` tools (no shelling out to `bsk`) and a live Web UI
|
It gives the agent native `browser_*` tools and a live view of its browser sessions
|
||||||
overlay of each Agent Window.
|
in the Web UI. The plugin runs `bsk` on the agent's behalf.
|
||||||
|
|
||||||
Add it to a dsh profile, then start that profile:
|
Install the `bsk` CLI and connect the browser extension first. Then add the plugin
|
||||||
|
to a dsh profile and start it (replace `web` with your profile name):
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
dsh plugin --profile web add @wxg-prc-cpg/browser-skill-dsh-plugin
|
dsh plugin --profile web add @wxg-prc-cpg/browser-skill-dsh-plugin
|
||||||
dsh --profile web
|
dsh --profile web
|
||||||
```
|
```
|
||||||
|
|
||||||
The plugin carries its own copy of the skill, so `bsk install-skill` is not needed
|
The plugin includes the `browser-skill` skill, so `bsk install-skill` is not needed
|
||||||
for dsh — but the `bsk` CLI and the browser extension are still prerequisites. See
|
for dsh. Installed plugins do not update automatically. To upgrade this plugin:
|
||||||
the [plugin README](packages/dsh-plugin-browserskill/README.md) for the tool list,
|
|
||||||
configuration, and the observation overlay.
|
```sh
|
||||||
|
dsh plugin --profile web update @wxg-prc-cpg/browser-skill-dsh-plugin --latest
|
||||||
|
```
|
||||||
|
|
||||||
|
Restart the profile after upgrading. See the
|
||||||
|
[plugin README](packages/dsh-plugin-browserskill/README.md) for usage and configuration.
|
||||||
|
|
||||||
## How It Works
|
## How It Works
|
||||||
|
|
||||||
|
|||||||
+9
-3
@@ -125,16 +125,22 @@ bsk install-skill
|
|||||||
|
|
||||||
## DeepSeek Harness 插件
|
## DeepSeek Harness 插件
|
||||||
|
|
||||||
在用 [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness)(`dsh`)?BrowserSkill 提供了官方 dsh 插件,已发布到 npm:[`@wxg-prc-cpg/browser-skill-dsh-plugin`](https://www.npmjs.com/package/@wxg-prc-cpg/browser-skill-dsh-plugin)。它会注入原生 `browser_*` 工具(无需再通过 Shell 调用 `bsk`),并在 Web UI 中实时观察每个 Agent Window。
|
在用 [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness)(`dsh`)?BrowserSkill 提供了官方 dsh 插件,已发布到 npm:[`@wxg-prc-cpg/browser-skill-dsh-plugin`](https://www.npmjs.com/package/@wxg-prc-cpg/browser-skill-dsh-plugin)。它为 Agent 提供原生 `browser_*` 工具,由插件代为调用 `bsk`,并在 Web UI 中实时展示浏览器会话。
|
||||||
|
|
||||||
把它装进某个 dsh profile,然后启动该 profile:
|
先安装 `bsk` CLI 并连接浏览器扩展,再将插件装进 dsh profile 并启动(将 `web` 替换为你的 profile 名称):
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
dsh plugin --profile web add @wxg-prc-cpg/browser-skill-dsh-plugin
|
dsh plugin --profile web add @wxg-prc-cpg/browser-skill-dsh-plugin
|
||||||
dsh --profile web
|
dsh --profile web
|
||||||
```
|
```
|
||||||
|
|
||||||
插件自带 skill,所以在 dsh 下无需执行 `bsk install-skill`;但 `bsk` CLI 和浏览器扩展仍是前置条件。工具清单、配置项与观察浮层见[插件 README](packages/dsh-plugin-browserskill/README.md)。
|
插件自带 `browser-skill` skill,所以在 dsh 下无需执行 `bsk install-skill`。已安装的插件不会自动更新;升级此插件请运行:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
dsh plugin --profile web update @wxg-prc-cpg/browser-skill-dsh-plugin --latest
|
||||||
|
```
|
||||||
|
|
||||||
|
升级后重启该 profile。用法与配置见[插件 README](packages/dsh-plugin-browserskill/README.md)。
|
||||||
|
|
||||||
## 工作原理
|
## 工作原理
|
||||||
|
|
||||||
|
|||||||
@@ -1,13 +1,51 @@
|
|||||||
# dsh-plugin-browserskill
|
# BrowserSkill for DeepSeek Harness
|
||||||
|
|
||||||
npm: [`@wxg-prc-cpg/browser-skill-dsh-plugin`](https://www.npmjs.com/package/@wxg-prc-cpg/browser-skill-dsh-plugin)
|
[](https://www.npmjs.com/package/@wxg-prc-cpg/browser-skill-dsh-plugin)
|
||||||
|
|
||||||
A [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) (dsh) tool plugin that exposes
|
Use [BrowserSkill](https://github.com/Tencent/BrowserSkill) in
|
||||||
[BrowserSkill](https://github.com/Tencent/BrowserSkill) (`bsk`) browser automation to the model.
|
[DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) (`dsh`) to browse
|
||||||
|
websites, fill forms, and capture screenshots through native `browser_*` tools.
|
||||||
|
Browser tasks run in Agent Windows, with a live view in the dsh Web UI.
|
||||||
|
|
||||||
Each action maps to one `bsk <cmd> --json` invocation: the plugin spawns the bsk CLI, parses its
|
## Installation
|
||||||
structured JSON output, and returns a canonical typed value. The bsk daemon, browser, and browser
|
|
||||||
extension keep owning the actual browser control — this package is a thin, well-typed bridge.
|
Before installing the plugin:
|
||||||
|
|
||||||
|
- Install [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) and
|
||||||
|
[pnpm](https://pnpm.io/installation), which dsh uses to manage plugins.
|
||||||
|
- Install the `bsk` CLI and connect the BrowserSkill extension in Chrome or Edge.
|
||||||
|
Follow the [BrowserSkill setup guide](https://github.com/Tencent/BrowserSkill#quick-start).
|
||||||
|
- Make sure `bsk` is on the `PATH` used to start dsh, or set `bskPath` in the plugin configuration.
|
||||||
|
|
||||||
|
Install the plugin into the `web` profile, then start it:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
dsh plugin --profile web add @wxg-prc-cpg/browser-skill-dsh-plugin
|
||||||
|
dsh --profile web
|
||||||
|
```
|
||||||
|
|
||||||
|
Replace `web` with your profile name if you use a different profile. The plugin
|
||||||
|
includes the `browser-skill` skill; no separate `bsk install-skill` step is needed.
|
||||||
|
|
||||||
|
In a conversation, try:
|
||||||
|
|
||||||
|
```text
|
||||||
|
/browser-skill open example.com and summarize the page.
|
||||||
|
```
|
||||||
|
|
||||||
|
By default, the browser tools become available when the skill is invoked.
|
||||||
|
|
||||||
|
## Updating
|
||||||
|
|
||||||
|
Installed plugins do not update automatically. To upgrade this plugin to npm's
|
||||||
|
`latest` version, including versions outside the profile's saved dependency range:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
dsh plugin --profile web update @wxg-prc-cpg/browser-skill-dsh-plugin --latest
|
||||||
|
```
|
||||||
|
|
||||||
|
Restart that dsh profile after upgrading. This command updates the plugin; update
|
||||||
|
the `bsk` CLI and browser extension separately when a release requires it.
|
||||||
|
|
||||||
## Tools
|
## Tools
|
||||||
|
|
||||||
@@ -20,23 +58,7 @@ extension keep owning the actual browser control — this package is a thin, wel
|
|||||||
| `browser_tabs` | `list`, `create`, `select`, `close`, `borrow`, `return` | Manage Agent Window tabs and temporarily borrow user tabs. |
|
| `browser_tabs` | `list`, `create`, `select`, `close`, `borrow`, `return` | Manage Agent Window tabs and temporarily borrow user tabs. |
|
||||||
| `browser_assist` | `resize`, `emulate`, `request-help` | Resize or emulate the browser and pause for human-only steps. |
|
| `browser_assist` | `resize`, `emulate`, `request-help` | Resize or emulate the browser and pause for human-only steps. |
|
||||||
|
|
||||||
`bsk evaluate` and `bsk record` are intentionally not exposed by this plugin: arbitrary page
|
Arbitrary page-script evaluation and interaction recording are not supported.
|
||||||
evaluation is a higher-risk capability, while recording is long-running and needs a dedicated
|
|
||||||
DeepSeek Harness lifecycle before it can be added safely.
|
|
||||||
|
|
||||||
## Agent skill (progressive disclosure)
|
|
||||||
|
|
||||||
Beyond the tools, the plugin publishes the **`browser-skill` agent skill** through the harness's
|
|
||||||
official skill seam (`ctx.skills.register`): the catalog entry (name + routing description) is
|
|
||||||
resident in `<available_skills>`, and the body is loaded only when the model invokes the `skill`
|
|
||||||
tool. Its single source is the DSH-specific `skill/SKILL.md`, which documents only structured
|
|
||||||
`browser_*` calls and their plugin semantics. The repository-root CLI skill is intentionally not
|
|
||||||
concatenated: its command examples belong to a different execution interface and would bypass
|
|
||||||
the plugin's ownership, live observation UI, cancellation, and cleanup path if followed directly.
|
|
||||||
The build rejects internal CLI-name leakage, command-line code blocks, unknown browser tools, and
|
|
||||||
missing supported browser tools before embedding the Markdown. Registration and every pre-step
|
|
||||||
catalog snapshot are pure in-memory reads (no disk/process/daemon); compositions without the skill
|
|
||||||
seam degrade silently.
|
|
||||||
|
|
||||||
## Multi-session model
|
## Multi-session model
|
||||||
|
|
||||||
@@ -55,136 +77,73 @@ an explicit `session` argument naming a foreign or unknown id is rejected, the `
|
|||||||
`browser_session` shows plugin-created sessions only (no daemon-wide view), and stop/unload cleanup
|
`browser_session` shows plugin-created sessions only (no daemon-wide view), and stop/unload cleanup
|
||||||
can never touch a session owned by another program.
|
can never touch a session owned by another program.
|
||||||
|
|
||||||
## Installation
|
|
||||||
|
|
||||||
The plugin follows the standard dsh bundle layout (`dsh.bundle` manifest + `cordis.patch.yml`):
|
|
||||||
|
|
||||||
```sh
|
|
||||||
dsh plugin --profile <name> add @wxg-prc-cpg/browser-skill-dsh-plugin
|
|
||||||
dsh --profile <name>
|
|
||||||
```
|
|
||||||
|
|
||||||
Prerequisite: the `bsk` CLI must be installed and on `PATH`, and the BrowserSkill browser extension
|
|
||||||
(Chrome or Edge) must be connected — see the
|
|
||||||
[BrowserSkill README](https://github.com/Tencent/BrowserSkill). When bsk is missing, tool calls fail
|
|
||||||
with install guidance instead of a bare spawn error.
|
|
||||||
|
|
||||||
## Configuration
|
## Configuration
|
||||||
|
|
||||||
All fields are optional and validated through the plugin's Schemastery `Config`:
|
After installing the plugin, edit your profile's `cordis.patch.yml`. For the `web`
|
||||||
|
profile, the default location is `~/.dsh/profiles/web/cordis.patch.yml`. If you set
|
||||||
|
`DSH_HOME`, use `$DSH_HOME/profiles/web/cordis.patch.yml` instead. Replace `web` with
|
||||||
|
your profile name as needed.
|
||||||
|
|
||||||
|
If the file contains only comments and `[]`, keep the comments and replace `[]`
|
||||||
|
with the YAML below. If it already contains patch entries, add this entry to the
|
||||||
|
existing list or edit its existing `id: browserskill` entry. Keep a single
|
||||||
|
top-level YAML list. This overrides the plugin registered by the installed bundle:
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
# cordis.patch.yml override example
|
- id: browserskill
|
||||||
- insert:
|
|
||||||
- id: browserskill
|
|
||||||
name: "@wxg-prc-cpg/browser-skill-dsh-plugin"
|
|
||||||
config:
|
config:
|
||||||
bskPath: bsk # path to the bsk binary (default: resolve from PATH)
|
bskPath: bsk
|
||||||
defaultTimeoutMs: 120000
|
defaultTimeoutMs: 120000
|
||||||
maxSessions: 5
|
maxSessions: 5
|
||||||
# observationEnabled: true # live PiP/overlay observation (below)
|
observationEnabled: true
|
||||||
# thumbnailIntervalMs: 1500 # frame cadence while a session is active
|
thumbnailIntervalMs: 1500
|
||||||
# idleIntervalMs: 8000 # idle cadence / recent-activity window
|
idleIntervalMs: 8000
|
||||||
# lazyTools: true # reveal browser_* tools only after the skill is invoked
|
lazyTools: true
|
||||||
```
|
```
|
||||||
|
|
||||||
- **`lazyTools` (default `true`)** — the final progressive-disclosure stage: the six
|
Change `bskPath` to the full path of your CLI binary if it is not on dsh's `PATH`.
|
||||||
`browser_*` tool schemas stay OUT of the system prompt (zero schema tokens) until the
|
A patch replaces the entry's entire `config` object, so keep all overrides you need
|
||||||
`browser-skill` skill is actually invoked — the skill catalog entry is the only
|
together in that object.
|
||||||
advertisement. One successful invocation (model tool call, or a `/browser-skill` user
|
|
||||||
gesture) registers the whole suite for the rest of the process; repeated invocations are
|
|
||||||
no-ops, and sessions resumed with a past invocation in their durable log reveal the suite
|
|
||||||
on entry. Set `false` for the legacy always-on registration.
|
|
||||||
|
|
||||||
## Observation overlay (PiP mini-window)
|
Configuration changes follow `dsh.profile.patchReload` in the profile's
|
||||||
|
`package.json`: `live` (the default for `web`) applies changes when you save the
|
||||||
|
patch file; `startup` requires restarting the profile. Restart after upgrading
|
||||||
|
the plugin in either case.
|
||||||
|
|
||||||
When the plugin runs inside the dsh Web UI, an **observation overlay** floats over the app
|
All fields are optional; omitted fields use the defaults below:
|
||||||
(registered into the `shell.overlay` seat): a breathing thumbnail per owned session plus its
|
|
||||||
current action and elapsed time. The card docks at the top-right of the content area (clear of
|
|
||||||
the composer and the shell's header controls) and wears the **BrowserSkill product family's own
|
|
||||||
look**: the overlay reuses `@browser-skill/ui` components (`Button`, `cn`) and its oklch design
|
|
||||||
tokens (`--card`, `--primary`, `--destructive`, `--ring`, …), status dots spec'd after the
|
|
||||||
extension popup's connection indicator, and Remix icons. The BSK utility sheet is compiled
|
|
||||||
scoped under the `.bsk-obs` root class (`scripts/build-client-css.mjs`), so the overlay looks
|
|
||||||
like the BrowserSkill extension without leaking a single selector into the host shell — and the
|
|
||||||
shell's theme cannot bleed back in.
|
|
||||||
|
|
||||||
- **Lifecycle**: hidden while the plugin owns no sessions; appears on the first
|
| Option | Default | Purpose |
|
||||||
`browser_session` start action; disappears when all sessions stop (or the plugin unloads).
|
| --- | --- | --- |
|
||||||
- **Focus view**: status row (green/idle/red dot + session + action + mm:ss), the latest page
|
| `bskPath` | `bsk` | Path to the CLI binary. |
|
||||||
frame (while viewed, refreshes every ~1.5s while active, ~8s when idle; the last good frame stays on
|
| `defaultTimeoutMs` | `120000` | Default command timeout in milliseconds. |
|
||||||
stage while the next one loads, and is kept on errors so the card does not flash),
|
| `maxSessions` | `5` | Maximum concurrent sessions started by this plugin. |
|
||||||
and a compact icon toolbar (Interrupt + Pop out, hover for the label).
|
| `observationEnabled` | `true` | Enable live browser observation. |
|
||||||
- **Interrupt**: one click kills the in-flight bsk command of the focus session (same semantics
|
| `thumbnailIntervalMs` | `1500` | Screenshot interval for active sessions, in milliseconds. |
|
||||||
as the chat Stop button — the current action fails, the agent run may continue). Strip items
|
| `idleIntervalMs` | `8000` | Screenshot interval for idle sessions and the recent-activity window, in milliseconds. |
|
||||||
carry their own hover interrupt button.
|
| `lazyTools` | `true` | Reveal the browser tools when the skill is invoked. Set `false` to register them at startup. |
|
||||||
- **Multi-session strip**: every session gets a tile (mini frame + id + status dot); focus
|
|
||||||
auto-follows the most recently active session; clicking a tile pins focus (pin badge, click
|
|
||||||
again to release); errored sessions get a red edge without stealing focus; sessions the daemon
|
|
||||||
lost are greyed out; prolonged daemon/browser outage shows "browser unavailable" and greys
|
|
||||||
the interrupt button until captures recover.
|
|
||||||
- **Drag & resize**: drag the header to move the card; drag any of the four
|
|
||||||
corners to resize (min 240×180, max 80% of the viewport; no visible grip).
|
|
||||||
Both are remembered for the page lifetime.
|
|
||||||
- **Pop out (PiP)**: upgrades the card into a native Document PiP window (requires a user
|
|
||||||
gesture, per browser rules), sized from the current card; closing the PiP falls back to the
|
|
||||||
in-page card with state intact. Browsers without Document PiP simply hide the button.
|
|
||||||
- **Screenshot demand**: the client requests periodic screenshots while an observation view is
|
|
||||||
visible, using the PiP document's visibility when popped out. Hidden or collapsed views can
|
|
||||||
keep subscribing to state without requesting screenshots. The first screenshot subscriber
|
|
||||||
starts capture scheduling; the last one's departure cancels timers and skips queued captures.
|
|
||||||
An already running capture may finish. Demand applies to the service's owned sessions, not just
|
|
||||||
the selected session.
|
|
||||||
- **Wire**: `GET /bsk-observation/events` is the live synchronization channel (SSE). Every connection,
|
|
||||||
including reconnects, starts with `{ type: "snapshot", sessions, available }`, followed by
|
|
||||||
`upsert`, `remove`, `reset`, and `availability` events on the same stream. Replace local state
|
|
||||||
with each snapshot before applying subsequent events. Use `?thumbnails=0` for state-only
|
|
||||||
subscriptions or `?thumbnails=1` to request screenshots; omitting the parameter defaults to
|
|
||||||
requesting screenshots for compatibility. Closing the stream releases its screenshot demand.
|
|
||||||
`GET /bsk-observation/state` remains a one-time `{ sessions, available }` read and does not
|
|
||||||
request screenshots. It has no ordering guarantee relative to a separate SSE connection;
|
|
||||||
live clients should initialize and resynchronize from the stream's snapshot instead.
|
|
||||||
The host also serves `POST /bsk-observation/interrupt`, `POST /bsk-observation/stop`, and
|
|
||||||
`GET /bsk-observation/thumbnail/<attachmentId>` through the dsh `webServer` route seam
|
|
||||||
(dsh 0.1's Typert Remote pipeline is closed to out-of-tree packages). All commands for one
|
|
||||||
session — tool calls and frame captures alike — run through a per-session FIFO, because the
|
|
||||||
daemon accepts only one unfinished command per session.
|
|
||||||
- **Programmatic subscription**: `ObservationService.subscribe(listener, { thumbnails: false })`
|
|
||||||
subscribes to state without requesting screenshots. On an active service, the listener receives
|
|
||||||
the initial `snapshot` synchronously, before `subscribe` returns, then receives subsequent
|
|
||||||
changes. `thumbnails` defaults to `true`, so callers should choose it explicitly. The returned
|
|
||||||
unsubscribe function is safe to call repeatedly and releases this subscription's screenshot
|
|
||||||
demand. Subscribing after service disposal is a no-op.
|
|
||||||
- **Trust model**: these routes expose live screenshots (and an interrupt write), so they
|
|
||||||
replicate the browser-trust fence dsh applies to its own `/api` routes: the request Host
|
|
||||||
must be a loopback authority (`localhost`, `127.0.0.0/8`, `[::1]`), a present Origin must
|
|
||||||
match the Host, `sec-fetch-site: cross-site` is refused, and POST requires an
|
|
||||||
`application/json` body (cross-site simple requests can never satisfy that). The channel is
|
|
||||||
therefore built for **loopback-only serving** — binding the dsh web server to `0.0.0.0` and
|
|
||||||
reaching it through a LAN address will (deliberately) fail the fence; do not put these
|
|
||||||
routes behind a non-loopback reverse proxy without adding your own authentication.
|
|
||||||
- Configure with `observationEnabled` / `thumbnailIntervalMs` / `idleIntervalMs`.
|
|
||||||
|
|
||||||
## Behavior notes
|
With `lazyTools: true`, only the skill's catalog entry is initially advertised to
|
||||||
|
the model. The six `browser_*` tool schemas are added to the system prompt after
|
||||||
|
the `browser-skill` skill is successfully invoked, either by the model or through
|
||||||
|
`/browser-skill`. Set `lazyTools: false` to make the tools available immediately.
|
||||||
|
|
||||||
- **Cancellation**: aborting a tool call (`exec.signal`) kills the underlying bsk child process,
|
## Live browser view
|
||||||
matching BrowserSkill's cooperative tool-cancellation model.
|
|
||||||
- **UI cards**: calls render as terminal cards (command line as title, output as the completed
|
The dsh Web UI shows the plugin's browser sessions in a floating panel. If your
|
||||||
card). Screenshots additionally attach the image itself when the host mounts an attachment store
|
profile provides the `dsh-better-sidebar` integration, the view appears in a
|
||||||
and the active model route declares image input; otherwise the PNG path is returned.
|
**Browser Skill** sidebar tab instead.
|
||||||
- **Web UI toolview (browser half)**: the package is dual-face. `dsh.client` (platform `web`) ships
|
|
||||||
`lib/client.cjs`, which registers a keyed `tool.call.toolview` view for `browser_inspect`. The
|
- See the current action, elapsed time, and recent screenshot for each session.
|
||||||
custom view keeps a terminal block for every inspect action and, when a screenshot result carries
|
- Select a session to focus on it. The sidebar view follows the current conversation.
|
||||||
an image block, resolves the durable attachment through the client session's authorized
|
- Use **Interrupt** to cancel the current browser command. The agent may continue
|
||||||
`readAttachment` RPC and renders it with the shared `MessageImage` thumbnail/lightbox atoms.
|
with another action afterward.
|
||||||
The other five tools keep the stock terminal card. The bundle follows the dsh client
|
- Drag or resize the floating panel, or use **Pop out** to open a Picture-in-Picture
|
||||||
contract: a CJS closure factory handed to `window.__ModuleLoader__.load`, platform modules
|
window in browsers that support it.
|
||||||
(`react`, `dsh-client-ui-*`) external, everything else inlined, CSS Modules compiled by
|
- Periodic screenshots are requested while a browser observation view is visible.
|
||||||
lightningcss.
|
Configure the active and idle intervals with the options above.
|
||||||
- **Errors**: non-zero bsk exits surface the CLI's JSON error envelope (`code`, `message`, `hint`)
|
|
||||||
so the model gets the daemon's actionable guidance.
|
The observation endpoints require a loopback address such as `localhost` or
|
||||||
- **Long-running work** (e.g. `bsk record`) is not backgrounded via `ctx.jobs` yet — tracked as a
|
`127.0.0.1`. Access through a LAN hostname or non-loopback reverse proxy is not supported.
|
||||||
follow-up.
|
|
||||||
|
|
||||||
## Development
|
## Development
|
||||||
|
|
||||||
@@ -195,22 +154,47 @@ pnpm --filter @wxg-prc-cpg/browser-skill-dsh-plugin test # unit tests mock b
|
|||||||
pnpm --filter @wxg-prc-cpg/browser-skill-dsh-plugin build # tsdown -> lib/
|
pnpm --filter @wxg-prc-cpg/browser-skill-dsh-plugin build # tsdown -> lib/
|
||||||
```
|
```
|
||||||
|
|
||||||
|
See the [development notes](https://github.com/Tencent/BrowserSkill/blob/main/packages/dsh-plugin-browserskill/docs/development.md)
|
||||||
|
for skill registration, tool results, and observation APIs in the current source.
|
||||||
|
|
||||||
## Publishing
|
## Publishing
|
||||||
|
|
||||||
The GitHub Actions workflow **Release dsh plugin** publishes this package to npm
|
The [Release dsh plugin workflow](https://github.com/Tencent/BrowserSkill/blob/main/.github/workflows/release-dsh-plugin.yml)
|
||||||
as `@wxg-prc-cpg/browser-skill-dsh-plugin`. The Cordis plugin id, client
|
publishes the package and this README to npm. Pushing a `dsh-plugin-vX.Y.Z` tag
|
||||||
bundle registration, and `cordis.patch.yml` name are the same specifier so
|
triggers it; ordinary commits to `main` do not.
|
||||||
dsh can import and materialize the plugin without a ModuleLoader id mismatch.
|
|
||||||
|
|
||||||
Trigger it by pushing a tag that matches `package.json`'s `version`:
|
1. Commit this README and any other changes intended for the release.
|
||||||
|
2. From the repository root, run the
|
||||||
|
[release script](https://github.com/Tencent/BrowserSkill/blob/main/scripts/release.mjs)
|
||||||
|
with a new stable version, replacing `<version>` below. The script updates the
|
||||||
|
CLI, extension, and DSH plugin versions, commits the version changes, and creates
|
||||||
|
their release tags:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
git tag dsh-plugin-v0.1.2
|
node scripts/release.mjs <version>
|
||||||
git push origin dsh-plugin-v0.1.2
|
```
|
||||||
```
|
|
||||||
|
|
||||||
Or run the workflow from the Actions tab (`workflow_dispatch`). The job reads the
|
3. Push the version commit to the release branch, then push the DSH plugin tag
|
||||||
`NPM_TOKEN` secret from the `npm-publish` GitHub Environment.
|
created by the script, using the same `<version>`:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
git push origin HEAD
|
||||||
|
git push origin dsh-plugin-v<version>
|
||||||
|
```
|
||||||
|
|
||||||
|
This publishes the DSH plugin. Push the CLI and extension tags separately when
|
||||||
|
those components are ready for release.
|
||||||
|
|
||||||
|
The workflow checks the version, runs typechecks and tests, builds the package,
|
||||||
|
and publishes it to npm.
|
||||||
|
|
||||||
|
You can also run the workflow manually from GitHub Actions on the intended release
|
||||||
|
ref. Both triggers require an unpublished version and the `NPM_TOKEN` secret in the
|
||||||
|
`npm-publish` GitHub Environment.
|
||||||
|
|
||||||
|
npm updates the package README only when a new version is published, including
|
||||||
|
for documentation-only changes. Published versions cannot be overwritten. See
|
||||||
|
[npm's README update rules](https://docs.npmjs.com/about-package-readme-files/).
|
||||||
|
|
||||||
## License
|
## License
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,90 @@
|
|||||||
|
# DSH plugin development notes
|
||||||
|
|
||||||
|
For installation, usage, and configuration, see the [plugin README](../README.md).
|
||||||
|
These notes describe the current source on this branch; a published npm version may
|
||||||
|
not include later changes.
|
||||||
|
|
||||||
|
## Tool execution
|
||||||
|
|
||||||
|
The plugin runs the `bsk` CLI and parses its JSON output. The daemon and browser
|
||||||
|
extension perform the browser operations. The published package name is also the
|
||||||
|
Cordis plugin id, client bundle registration id, and `cordis.patch.yml` plugin name.
|
||||||
|
Keep these identifiers aligned so dsh can load both halves of the plugin.
|
||||||
|
|
||||||
|
## Agent skill
|
||||||
|
|
||||||
|
Beyond the tools, the plugin publishes the **`browser-skill` agent skill** through the harness's
|
||||||
|
official skill seam (`ctx.skills.register`): the catalog entry (name + routing description) is
|
||||||
|
resident in `<available_skills>`, and the body is loaded only when the model invokes the `skill`
|
||||||
|
tool. Its single source is the DSH-specific `skill/SKILL.md`, which documents only structured
|
||||||
|
`browser_*` calls and their plugin semantics. The repository-root CLI skill is intentionally not
|
||||||
|
concatenated: its command examples belong to a different execution interface and would bypass
|
||||||
|
the plugin's ownership, live observation UI, cancellation, and cleanup path if followed directly.
|
||||||
|
The build rejects internal CLI-name leakage, command-line code blocks, unknown browser tools, and
|
||||||
|
missing supported browser tools before embedding the Markdown. Registration and every pre-step
|
||||||
|
catalog snapshot are pure in-memory reads (no disk/process/daemon); compositions without the skill
|
||||||
|
seam degrade silently.
|
||||||
|
|
||||||
|
With `lazyTools: true`, the six tool schemas stay out of the system prompt until
|
||||||
|
the skill is successfully invoked. Registration happens once and lasts until the
|
||||||
|
plugin is unloaded. Repeated invocations do not register duplicate tools. Entering
|
||||||
|
a resumed conversation whose history contains a successful skill invocation also
|
||||||
|
registers the tools. Setting `lazyTools: false` registers them at plugin startup.
|
||||||
|
|
||||||
|
## Observation subscriptions and routes
|
||||||
|
|
||||||
|
- **Screenshot demand**: the client requests periodic screenshots while an observation view is
|
||||||
|
visible, using the PiP document's visibility when popped out. Hidden or collapsed views can
|
||||||
|
keep subscribing to state without requesting screenshots. The first screenshot subscriber
|
||||||
|
starts capture scheduling; the last one's departure cancels timers and skips queued captures.
|
||||||
|
An already running capture may finish. Demand applies to the service's owned sessions, not just
|
||||||
|
the selected session.
|
||||||
|
- **Wire**: `GET /bsk-observation/events` is the live synchronization channel (SSE). Every connection,
|
||||||
|
including reconnects, starts with `{ type: "snapshot", sessions, available }`, followed by
|
||||||
|
`upsert`, `remove`, `reset`, and `availability` events on the same stream. Replace local state
|
||||||
|
with each snapshot before applying subsequent events. Use `?thumbnails=0` for state-only
|
||||||
|
subscriptions or `?thumbnails=1` to request screenshots; omitting the parameter defaults to
|
||||||
|
requesting screenshots for compatibility. Closing the stream releases its screenshot demand.
|
||||||
|
`GET /bsk-observation/state` remains a one-time `{ sessions, available }` read and does not
|
||||||
|
request screenshots. It has no ordering guarantee relative to a separate SSE connection;
|
||||||
|
live clients should initialize and resynchronize from the stream's snapshot instead.
|
||||||
|
The host also serves `POST /bsk-observation/interrupt`, `POST /bsk-observation/stop`, and
|
||||||
|
`GET /bsk-observation/thumbnail/<attachmentId>` through the dsh `webServer` route seam
|
||||||
|
(dsh 0.1's Typert Remote pipeline is closed to out-of-tree packages). All commands for one
|
||||||
|
session — tool calls and frame captures alike — run through a per-session FIFO, because the
|
||||||
|
daemon accepts only one unfinished command per session.
|
||||||
|
- **Programmatic subscription**: `ObservationService.subscribe(listener, { thumbnails: false })`
|
||||||
|
subscribes to state without requesting screenshots. On an active service, the listener receives
|
||||||
|
the initial `snapshot` synchronously, before `subscribe` returns, then receives subsequent
|
||||||
|
changes. `thumbnails` defaults to `true`, so callers should choose it explicitly. The returned
|
||||||
|
unsubscribe function is safe to call repeatedly and releases this subscription's screenshot
|
||||||
|
demand. Subscribing after service disposal is a no-op.
|
||||||
|
- **Trust model**: these routes expose live screenshots (and an interrupt write), so they
|
||||||
|
replicate the browser-trust fence dsh applies to its own `/api` routes: the request Host
|
||||||
|
must be a loopback authority (`localhost`, `127.0.0.0/8`, `[::1]`), a present Origin must
|
||||||
|
match the Host, `sec-fetch-site: cross-site` is refused, and POST requires an
|
||||||
|
`application/json` body (cross-site simple requests can never satisfy that). The channel is
|
||||||
|
therefore built for **loopback-only serving** — binding the dsh web server to `0.0.0.0` and
|
||||||
|
reaching it through a LAN address will (deliberately) fail the fence; do not put these
|
||||||
|
routes behind a non-loopback reverse proxy without adding your own authentication.
|
||||||
|
|
||||||
|
## Tool results and cancellation
|
||||||
|
|
||||||
|
- **Cancellation**: aborting a tool call (`exec.signal`) kills the underlying bsk child process,
|
||||||
|
matching BrowserSkill's cooperative tool-cancellation model.
|
||||||
|
- **UI cards**: calls render as terminal cards (command line as title, output as the completed
|
||||||
|
card). Screenshots additionally attach the image itself when the host mounts an attachment store
|
||||||
|
and the active model route declares image input; otherwise the PNG path is returned.
|
||||||
|
- **Web UI toolview (browser half)**: the package is dual-face. `dsh.client` (platform `web`) ships
|
||||||
|
`lib/client.cjs`, which registers a keyed `tool.call.toolview` view for `browser_inspect`. The
|
||||||
|
custom view keeps a terminal block for every inspect action and, when a screenshot result carries
|
||||||
|
an image block, resolves the durable attachment through the client session's authorized
|
||||||
|
`readAttachment` RPC and renders it with the shared `MessageImage` thumbnail/lightbox atoms.
|
||||||
|
The other five tools keep the stock terminal card. The bundle follows the dsh client
|
||||||
|
contract: a CJS closure factory handed to `window.__ModuleLoader__.load`, platform modules
|
||||||
|
(`react`, `dsh-client-ui-*`) external, everything else inlined, CSS Modules compiled by
|
||||||
|
lightningcss.
|
||||||
|
- **Errors**: non-zero bsk exits surface the CLI's JSON error envelope (`code`, `message`, `hint`)
|
||||||
|
so the model gets the daemon's actionable guidance.
|
||||||
|
- **Long-running work** (e.g. `bsk record`) is not backgrounded via `ctx.jobs` yet — tracked as a
|
||||||
|
follow-up.
|
||||||
Reference in New Issue
Block a user