Compare commits

..
Author SHA1 Message Date
BB-fat edb149611b ci: publish the dsh plugin to npm as @wxg-prc-cpg/browser-skill-dsh-plugin
Add a Release dsh plugin workflow (dsh-plugin-v* tags or workflow_dispatch)
that reads NPM_TOKEN from the GitHub Environment of the same name. The
Cordis plugin id stays dsh-plugin-browserskill; only the npm package name
changes.
2026-08-17 10:00:13 +00:00
BB-fat 14c5884cbc style(dsh-plugin-browserskill): fix overlay CSS stylelint errors
Add the missing blank line before the actions comment, and hoist
.tool-button:focus-visible above the wrap-hover rules so specificity
no longer descends.
2026-08-17 09:52:09 +00:00
BB-fat e7aad9bc16 style(dsh-plugin-browserskill): wrap lines to satisfy biome
CI's frontend job failed on formatter-only diffs in the overlay and the
skill-content assertion.
2026-08-17 09:48:27 +00:00
BB-fat 3fb9e9f581 fix(dsh-plugin-browserskill): compact overlay toolbar and four-corner resize
Use icon-only actions with a short hover tip, swap the interrupt glyph to
RiStopCircleLine, and let every corner resize the card without a grip icon.
2026-08-17 07:50:53 +00:00
BB-fat 9854ee2f45 fix(dsh-plugin-browserskill): stop observation overlay from flashing between frames
Hold the last ready thumbnail until the next one decodes, and swap the
img src in place instead of remounting through a placeholder.
2026-08-17 07:27:28 +00:00
BB-fat 51085b960b fix(browser-skill): route DSH usage through injected tools 2026-08-17 07:04:36 +00:00
BB-fat 51df868008 chore(dsh-plugin-browserskill): prepare the package for npm publishing
- @browser-skill/ui and @remixicon/react move to devDependencies (the client
  bundle inlines both; the host face never imports them — verified).
- Drop private, add license: MIT, ship the repo-root MIT LICENSE in files.
- peerDependencies: add the runtime-required @deepseek-ai/schemastery, and
  the previously dangling meta entries @deepseek-ai/dsh-attachment and
  @deepseek-ai/dsh-llm (optional, matching actual type-level use).
- prepack runs the full build; npm pack now yields exactly 7 files
  (lib/index.mjs, lib/index.d.mts, lib/client.cjs, cordis.patch.yml,
  README.md, LICENSE, package.json).
- publint clean: the client face ships as lib/client.cjs (CJS extension in
  a type:module package); add lint:publint script. Built declarations pass a
  NodeNext consumer typecheck (strict, skipLibCheck false).
2026-08-14 17:13:48 +00:00
BB-fat 43ebfab7ba fix(dsh-plugin-browserskill): register the lazy suite from any fiber
Cordis contexts are fiber-owned: assigning onto an Object.create() overlay of
the apply-time context fails with 'cannot set property tools in multiple
fibers' when the reveal fires from the tools/result listener fiber (caught by
the e2e: the skill invocation matched but the suite never registered).
registerTools now tracks disposers through a plain closure over the captured
context — no context mutation anywhere. Also documents lazyTools in the
README and hardens the reveal against a failed registration (log and stay
hidden instead of latching).

Verified end to end against the real stack: first request carries 0 browser_*
schemas (skill catalog entry only); after the scripted skill invocation the
next request carries exactly the 11 browser_* tools and browser_session_start
dispatches for real.
2026-08-14 16:58:30 +00:00
BB-fat ce11325de2 feat(dsh-plugin-browserskill): reveal tool schemas only after skill invocation
Progressive disclosure, final stage: with lazyTools on (default), the eleven
browser_* tool schemas stay out of the system prompt — the browser-skill
catalog entry is the sole advertisement. One successful skill invocation
(model tool call via tools/result, or a /browser-skill user gesture via the
session append feed) registers the suite for the process lifetime, idempotent
on repeats; session resume is covered by scanning durable events for a past
tool/call+tool/result pair (or skill-invocation message) on session entry and
at apply time. lazyTools: false keeps the legacy always-on registration.

Investigation (recorded in the ticket): tools/result carries normalized
name/arguments with an isError discriminant; dsh 0.1 has no official global
visibility switch (ctx.tools.restrict is agent-scoped and throws from a plain
context), making conditional registration the intended pattern; per-step
assembly picks the new suite up on the next step, and visibility changes are
first-class inputs of tool-skill's catalog digest. registerTools now returns
the combined disposer via a prototype-preserving tracking overlay; the skill
prelude tells the model the browser_* tools unlock on skill load.

Tests: +8 (hidden-before/reveal/idempotent/gesture/history hit+miss/dispose/
scanner pairing/two-state apply wiring). 124/124 green.
2026-08-14 16:43:51 +00:00
BB-fat 4d1dcd122f feat(dsh-plugin-browserskill): publish the BSK skill through ctx.skills
The plugin exposed only tools; the BrowserSkill agent skill never reached
the harness's <available_skills> catalog. Register it as an embedded runtime
skill (progressive disclosure: catalog entry resident, body on skill-tool
invocation) via the official ctx.skills seam — silent no-op when the
composition lacks it.

Content is build-time assembled so there is one source of truth:
skill/prelude.md (committed, dsh-specific: tool<->CLI map, owned-session
semantics, plugin-only overrides) + the canonical repo-root skill/SKILL.md
verbatim (the same file crates/bsk-cli/build.rs mirrors; frontmatter
stripped, name/description supplied by the registration). Generated into
src/skill-content.generated.ts (gitignored) — registration and every
pre-step snapshot are pure in-memory reads: no disk, no process, no daemon.

Scripts: build:skill chained into build/test/typecheck (same pattern as
build:css); biome excludes the generated artifacts. Tests: +3 (catalog
content, in-memory weight, silent degradation). 116/116 green.
2026-08-14 16:17:54 +00:00
BB-fat a4e668542d fix(dsh-plugin-browserskill): address review findings (leaks, trust fence, queue race)
Blocking:
- B1: delete screenshot scratch PNGs — observation frames reuse one fixed
  per-session path and unlink after every read (finally); browser_screenshot
  unlinks once the bytes are in the attachment store (kept only when the file
  itself is the model-facing artifact).
- B2: replicate dsh's browser-trust fence on /bsk-observation/*: loopback
  Host only, Origin must match Host, sec-fetch-site: cross-site refused, POST
  requires application/json; README documents the loopback trust premise and
  the 0.0.0.0 warning.

Major:
- M1: KeyedExecutor tail chains previous+task (allSettled) so a task aborted
  while queued cannot release the next one into the running session.
- M2: client tracks the live frame per session — replacing/removing/reset
  revokes the old blob URL at once; loads settling after replacement never
  resurrect it.

Minor: drop replaced thumbRefs (+on remove); beginAction fires inside the
queue (no label overwrite while queued, no idle flash); half-initialized
session cleanup stops through the queue with one retry; SSE (re)open refetches
/state (+ snapshot flag renamed subscribed); install probe uses --version (no
daemon spawn); emulate mobile documents the width+height requirement (the
daemon refuses it alone — verified).

Nit: single BskRunOptions declaration; tails map entry dropped on drain; SSE
cleanup on res close; popOut catches requestWindow rejections; resize handle
gains keyboard control (arrows, 16px steps, aria value attrs).

Tests: +14 (fence rules, scratch lifecycle, queue race, blob revocation,
instrumentation timing, SSE resync). 113/113 green.
2026-08-14 11:56:02 +00:00
BB-fat 52aa356153 test(dsh-plugin-browserskill): build the scoped css before vitest
The client entry imports the generated bsk-ui.nomodule.css; the file is
gitignored, so a fresh CI checkout could not resolve it. Generate it as part
of the test script.
2026-08-14 11:05:05 +00:00
BB-fat d1001e14b2 feat(dsh-plugin-browserskill): reskin the overlay with BrowserSkill's own UI system
The previous pass aligned the observation overlay with the dsh shell; the
product call is the opposite direction — the floating card should read as a
BrowserSkill surface, not a shell-native widget.

- Reuse @browser-skill/ui directly (Button, cn); status dots spec'd after the
  extension popup's ConnectionStatusIndicator; Remix icons for
  interrupt/pin/warn/pop-out.
- Compile the BSK tailwind utility sheet scoped under the .bsk-obs root class
  (scripts/build-client-css.mjs + postcss-prefix-selector) and ship the oklch
  design tokens on the same scope, so nothing leaks into the host shell and
  the shell theme cannot bleed back. tsdown injects .nomodule.css verbatim
  (minified, unhashed) alongside the hashed CSS modules.
- BSK ships no shared tooltip, so the one-time interrupt semantics hint is a
  card-spec bubble (bg-card/border/12px radius), retired after first use.
- Tests: pin react/react-dom to this package's 18.x copies in vitest — the ui
  package sources peer on react ^19 and would otherwise emit react-19
  (transitional) elements that the react-18 renderer rejects. Production is
  unaffected (react stays external, provided by the shell).
- stylelint: ignore the generated bsk-ui.nomodule.css.
2026-08-14 10:00:37 +00:00
BB-fat eb71ddc1da docs(dsh-plugin-browserskill): note the top-right docking and shell-native styling 2026-08-14 08:10:05 +00:00
BB-fat b07b8fec65 fix(dsh-plugin-browserskill): dock below the shell top bar
The plain top:16 default overlapped the shell's Session log action;
dock the card and capsule at the content area's top-right (64px) so
neither collides with the header controls.
2026-08-14 08:06:28 +00:00
BB-fat 47070fdc14 feat(dsh-plugin-browserskill): default the overlay to the top-right + dsh-native styling
Review follow-ups: the card/capsule now default to the top-right (clear
of the composer), and the overlay reads as part of the shell instead of
a foreign widget:

- shared primitives everywhere: Button (outline/ghost sm) for
  Interrupt/Pop out with the shell's danger hover token, Tooltip for
  the one-time interrupt semantics hint (native span anchor — Button
  does not forward refs), StateDot for status, and outline icons
  (Stop/RightUp/ChevronDown/Warning) replacing text glyphs;
- every color/surface/shadow/label now uses --dsw-alias-* semantic
  tokens (bg-overlay, border-l1/l2, state-*-primary, label-*,
  interactive-bg-hover[-danger], brand-primary focus rings,
  dsw-shadow-lv1/lv2) — no invented tokens, no literal colors (dsh 0.1
  ships no spacing/radius scale; those stay px like dsh's own code);
- strip item focus ring uses brand-primary; focus-visible and
  reduced-motion preserved.
2026-08-14 07:57:07 +00:00
BB-fat 916229dc62 docs(dsh-plugin-browserskill): README section for the observation overlay 2026-08-14 07:05:39 +00:00
BB-fat dde707b558 fix(dsh-plugin-browserskill): report interrupt-killed commands as interrupted
A child killed via killFor (overlay interrupt) exits with a null code
and empty output, which used to render as a duplicated 'bsk x failed:
bsk x failed'. Report it as interrupted instead.
2026-08-14 07:01:44 +00:00
BB-fat e2d07574d7 fix(dsh-plugin-browserskill): prefix routes register without a trailing slash
The webserver prefix matcher joins prefix + '/', so a registered
'/bsk-observation/thumbnail/' never matched real paths.
2026-08-14 06:54:01 +00:00
BB-fat 461d204ca3 fix(dsh-plugin-browserskill): serve observation thumbnails over the plugin's own route
The session-authorized client RPC (readAttachment) refuses images that
no session log references — observation frames are plugin-owned runtime
data, so every overlay thumbnail failed with ATTACHMENT_NOT_REFERENCED.
The host now keeps the full attachment ref per frame and serves bytes
through GET /bsk-observation/thumbnail/<id> (verified readImage path);
the client loader is a plain fetch of that route.
2026-08-14 06:49:15 +00:00
BB-fat bc5d9610a3 fix(dsh-plugin-browserskill): serialize per-session commands (observation vs tool calls)
The daemon accepts only one unfinished command per session; the
thumbnail loop racing a model tool call produced 'session already has
an unfinished command' failures on both sides (found by the Phase 1-C
e2e: navigate/snapshot/click all errored while captures kept running).
Add a KeyedExecutor: every command for a session — tool calls and
observation captures alike — runs FIFO; queued tasks reject early on
abort, running tasks keep the runner's signal-driven kill.
2026-08-14 06:41:49 +00:00
BB-fat 2955260d11 fix(dsh-plugin-browserskill): register observation routes once webServer appears
ctx.get('webServer') at apply time raced the web composition's service
registration (the fallback SPA then answered /bsk-observation/* with
index.html). Ride ctx.inject(['webServer'], …) instead: the callback
runs when the service is provided and never runs in headless
compositions, keeping route registration timing-safe and optional.
2026-08-14 06:20:18 +00:00
BB-fat ce89e95211 feat(dsh-plugin-browserskill): multi-session strip, focus pin, and edge states (Phase 1-C)
- Strip (meeting-style multi-session layout): one item per session
  (mini frame + id + status dot), horizontal row with scroll overflow;
  hover reveals a per-item interrupt button that acts without
  refocusing; click pins the focus view (pin badge, click again to
  release); auto-follow picks the most recently active session but
  never steals focus for errored or dead sessions (red edge / greyed
  item instead).
- Edge states: the host flips an availability flag after repeated
  global capture failures (client shows 'browser unavailable', keeps
  the last frame, greys interrupt, and recovers on the next success);
  a session_not_found envelope marks the session dead (no more frame
  requests, grey strip item, removable as usual); lastError now clears
  when the next action starts or succeeds.
- State/SSE wire gains { available } and an 'availability' event;
  SessionObservation gains dead?.

Tests (+8, 97 total): host availability flip, dead marking +
instrumentation drop + clean removal, lastError lifecycle; client
strip render/pin/unpin/auto-follow, error no-steal, unavailable strip,
dead grey, strip-item interrupt without refocus.
2026-08-14 05:58:53 +00:00
BB-fat 24faa10659 feat(dsh-plugin-browserskill): observation overlay — floating card, interrupt, and PiP (Phase 1-B)
Client half of the PiP observation window, wired to the host
ObservationService over the /bsk-observation HTTP+SSE seam:

- ObservationClientStore: initial state fetch + SSE increments +
  on-demand thumbnail blob loading (session-authorized readAttachment,
  same path as the toolview) + interrupt POST. All I/O injectable.
- ObservationOverlay registered into the shell.overlay list seat (the
  sanctioned frame-wide surface; root is off-limits). Hidden with no
  owned sessions; appears on first start; vanishes when all stop.
- Focus view: status row (green/grey/red dot + session + action + mm:ss
  ticking), breathing thumbnail (new frames fade in, failures keep the
  last frame with a warning badge, pre-navigate placeholder), and the
  action area.
- Interrupt: single click, no confirm; greys into 'Interrupting…' and
  back; disabled when nothing is in flight; one-time tooltip explains
  the semantics (stops the current action only; the run continues).
- Card: drag-move via the header, corner-handle resize (min 240x180,
  max 80% viewport), both clamped and kept for the page lifetime;
  collapsible status capsule.
- PiP: Pop out (user gesture) opens Document PiP with the card's
  current size, portals the same content in, and clones the document's
  style nodes; pagehide falls back to the card with state intact;
  unsupported browsers hide the button; nothing auto-pops.

Tests (+17, 89 total): store (fetch/SSE apply/malformed frames/
thumbnail lifecycle/interrupt wire/stop cleanup) and overlay (hidden->
visible lifecycle, status row, thumbnail via loader, interrupt
disabled/active/hint-once, capsule, resize clamps, move clamps, PiP
unsupported, PiP pop/fallback). Styling follows web-styling (dsw-alias
tokens, CSS Modules, focus-visible, reduced-motion).
2026-08-14 05:33:34 +00:00
BB-fat a7bc8bacf6 feat(dsh-plugin-browserskill): host-side ObservationService for the PiP overlay (Phase 1-A)
Per-session live observation for owned sessions only, feeding the
upcoming client overlay:

- State model: SessionObservation (sessionId/url/action/since/
  thumbnailAttachmentId/lastError), one entry per owned session;
  added on start, removed on stop, cleared on dispose.
- Instrumentation: the shared runBsk wrapper begins/ends an action
  (tool name mapped to a verb) around every model-facing call and tags
  the child with its session id; action end triggers an immediate
  thumbnail refresh.
- Thumbnail loop: 1.5s cadence while active, 8s idle downclock, x3
  failure backoff, silent frame retention on errors; captures run
  OUTSIDE the instrumentation (no action events, no current-pointer
  movement) and land in the attachment store by reference.
- Interrupt: runner.killFor(tag) kills exactly the tagged in-flight bsk
  children — chat-Stop-equivalent semantics for the current or a named
  owned session; foreign ids and the no-in-flight case return false.
- Remote seam: dsh 0.1's Typert Remote pipeline and forwarded-event
  allowlist are closed to out-of-tree packages, so state/events/
  interrupt are served over the documented webServer route seam:
  GET /bsk-observation/state, GET /bsk-observation/events (SSE),
  POST /bsk-observation/interrupt; routes mount only when a webServer
  service exists (headless compositions skip them).
- Config: observationEnabled (true), thumbnailIntervalMs (1500),
  idleIntervalMs (8000).

Tests (+13, 72 total): state machine + event stream, cadence fast/idle/
backoff, headless no-store silence, observation-traffic isolation,
interrupt routing (default/specified/none/foreign), HTTP route
handlers, disabled-webServer no-op.
2026-08-14 04:59:26 +00:00
BB-fat 3d050e0f26 feat(dsh-plugin-browserskill): strict ownership boundary — foreign sessions are invisible
Tighten the shared-daemon boundary from 'cleanup only touches owned
sessions' to 'the plugin only ever sees and operates on sessions it
created':

- browser_session_list no longer queries the daemon at all; it returns
  the plugin's own session table (with the current marker), so foreign
  sessions cannot even be enumerated through the plugin.
- Every tool's optional session argument must name a plugin-created
  session; foreign/unknown ids are rejected with a clear error before
  any command reaches the daemon (the reference-adoption channel is
  gone).
- Unchanged: dispose stops exactly the owned set, stop requires
  ownership, the start reservation protocol, and handle-precise child
  kills.

Tests updated (59): explicit-foreign rejection without daemon contact,
registry-only listing, reservation race, stale-handle-tolerant dispose.
e2e: a manually started foreign session was rejected on reference,
absent from session_list, refused on stop, and survived dispose while
both plugin sessions were cleaned up.
2026-08-14 04:12:38 +00:00
BB-fat afdc398730 fix(dsh-plugin-browserskill): strict session ownership on a shared daemon
Review hardening for daemon-sharing deployments (other agents,
terminals, or dsh instances on the same bsk daemon):

- Ownership split: only sessions created by browser_session_start are
  'owned'. Explicitly referenced foreign sessions are still tracked for
  the current-session pointer, but browser_session_stop refuses them
  and unload cleanup stops exactly the owned set — no path can stop a
  session this plugin did not create.
- Start race: the capacity check is now a synchronous
  reserveStart/completeStart/abandonStart protocol, so two concurrent
  starts can never both pass the cap (check-and-reserve is atomic on
  the event loop); a rejected or failed start never leaks a session.
- Stale handles: dispose already swallowed per-stop errors; the owned
  set is computed at dispose time so an externally stopped session is a
  no-op, not a failure.

Tests (+7): reservation race (two concurrent starts, cap 1 -> exactly
one spawn), foreign-stop refusal (no stop command reaches the daemon),
reference adoption never becoming owned, resolveForStop pointer
stability, and an apply-level dispose test proving only owned sessions
are stopped while stale stops are tolerated. e2e: a manually started
foreign session survived the plugin's full lifecycle while both plugin
sessions were cleaned up.
2026-08-14 03:45:52 +00:00
BB-fat 6b96fcca32 feat(dsh-plugin-browserskill): add the web client half with a browser_screenshot toolview
Make the package dual-face: a dsh.client declaration (platform 'web')
plus an exports["./client"] bundle built in dsh's closure-factory shape
(window.__ModuleLoader__.load handoff, platform modules external,
everything else inlined, CSS Modules via lightningcss).

The client registers a keyed 'tool.call.toolview' view for
browser_screenshot that keeps the terminal block (command + output) and,
when the settled result carries an image block, resolves the durable
attachment through the client session's authorized readAttachment RPC
and renders it with the shared MessageImage thumbnail/lightbox atoms.
Path-only results render unchanged; every other browser_* tool keeps
the stock terminal card.

Tests: view-model derivation (running/ok/error, image vs path-only),
component rendering through @testing-library/react + happy-dom, and the
registration key. biome.json now excludes the package's lib/ build
output (same treatment as dist/).
2026-08-14 02:54:33 +00:00
BB-fat 0771292336 fix(dsh-plugin-browserskill): keep the terminal result card for multi-block results
browser_screenshot on an image-capable route returns text + image
blocks; the shared result presenter required exactly one text block and
bailed to the generic card (tool name + raw args). Project the first
text block instead so the terminal card shows in both modes.
2026-08-13 18:06:57 +00:00
BB-fat 54869cabd0 fix(dsh-plugin-browserskill): include the bsk error hint in the thrown message
The daemon's JSON error envelope carries an actionable hint (e.g.
'choose an input, textarea, or contenteditable element from the latest
snapshot'), but the thrown BskError only embedded the message field, so
the model-facing error text never showed the hint. Append it.
2026-08-13 17:56:16 +00:00
BB-fat 2de7c5c6ff fix(dsh-plugin-browserskill): enforce the session cap before spawning
The concurrency cap was checked in registry.add() AFTER
`bsk session start` had already created the session, so a rejected
start leaked a live session outside the plugin's tracking (found by the
real-browser e2e: the rejected third session stayed in
`bsk session list` after dispose). Check capacity before spawning;
registry.add() keeps the same check as a backstop.
2026-08-13 15:27:29 +00:00
BB-fat 703fc64399 feat(packages): add dsh-plugin-browserskill — DeepSeek Harness tools for bsk
Add a new monorepo package that registers BrowserSkill (bsk) browser
automation as model-visible tools in DeepSeek Harness (dsh):

- 11 tools mapping to bsk CLI --json commands: session start/stop/list,
  navigate, snapshot, observe, click, fill, press, screenshot, emulate
- multi-session support: optional session arg with a current-session
  pointer, session id echoed in every result, configurable concurrency
  cap (default 5), and dispose-time cleanup of plugin-started sessions
- cancellation: exec.signal aborts kill the underlying bsk child process
- terminal-style UI cards (command line as the call card, output as the
  result card); screenshots inline the PNG through the host attachment
  store when the model route accepts image input, else return the path
- config: bskPath / defaultTimeoutMs / maxSessions via Schemastery schema
- install guidance when the bsk binary is missing; bsk JSON error
  envelopes (code/message/hint) surfaced to the model
- unit tests mock the bsk runner (no real browser required); the root
  lint script now also runs the package's typecheck + tests so the
  existing CI frontend job covers the new package (a dedicated CI job
  diff is provided in the PR; the pushing token lacks workflow scope)
2026-08-13 15:04:52 +00:00
48 changed files with 11244 additions and 32 deletions
+105
View File
@@ -0,0 +1,105 @@
name: Release dsh plugin
on:
push:
tags:
- dsh-plugin-v*
workflow_dispatch:
permissions:
contents: read
concurrency:
group: release-dsh-plugin-${{ github.ref }}
cancel-in-progress: false
env:
PACKAGE_NAME: "@wxg-prc-cpg/browser-skill-dsh-plugin"
PACKAGE_DIR: packages/dsh-plugin-browserskill
jobs:
resolve:
runs-on: ubuntu-latest
outputs:
version: ${{ steps.version.outputs.version }}
tag: ${{ steps.version.outputs.tag }}
steps:
- uses: actions/checkout@v6
- name: Resolve plugin version
id: version
shell: bash
run: |
set -euo pipefail
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
VERSION="$(node -p "require('./${{ env.PACKAGE_DIR }}/package.json').version")"
else
VERSION="${GITHUB_REF_NAME#dsh-plugin-v}"
fi
VERSION="${VERSION#v}"
echo "version=${VERSION}" >> "$GITHUB_OUTPUT"
echo "tag=dsh-plugin-v${VERSION}" >> "$GITHUB_OUTPUT"
guard:
needs: resolve
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- name: Verify tag matches package.json name and version
shell: bash
run: |
set -euo pipefail
PKG_NAME="$(node -p "require('./${{ env.PACKAGE_DIR }}/package.json').name")"
PKG_VERSION="$(node -p "require('./${{ env.PACKAGE_DIR }}/package.json').version")"
RELEASE_VERSION="${{ needs.resolve.outputs.version }}"
if [ "$PKG_NAME" != "$PACKAGE_NAME" ]; then
echo "package.json name (${PKG_NAME}) does not match expected ${PACKAGE_NAME}"
exit 1
fi
if [ "$PKG_VERSION" != "$RELEASE_VERSION" ]; then
echo "${PACKAGE_DIR}/package.json version (${PKG_VERSION}) does not match release version (${RELEASE_VERSION})"
exit 1
fi
echo "Version guard passed: ${PKG_NAME}@${RELEASE_VERSION}"
publish:
needs: [resolve, guard]
runs-on: ubuntu-latest
# Secret NPM_TOKEN is stored on the GitHub Environment also named NPM_TOKEN.
environment: NPM_TOKEN
permissions:
contents: read
steps:
- uses: actions/checkout@v6
- uses: pnpm/action-setup@v6
- uses: actions/setup-node@v6
with:
node-version: 22
cache: pnpm
registry-url: https://registry.npmjs.org
scope: "@wxg-prc-cpg"
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Typecheck plugin
run: pnpm --filter "${{ env.PACKAGE_NAME }}" typecheck
- name: Test plugin
run: pnpm --filter "${{ env.PACKAGE_NAME }}" test
- name: Publish to npm
working-directory: ${{ env.PACKAGE_DIR }}
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
run: |
set -euo pipefail
if [ -z "${NODE_AUTH_TOKEN:-}" ]; then
echo "::error::Environment secret NPM_TOKEN is not configured"
exit 1
fi
pnpm publish --access public --no-git-checks
echo "Published ${PACKAGE_NAME}@${{ needs.resolve.outputs.version }}"
+3
View File
@@ -3,3 +3,6 @@
**/.output/**
**/target/**
**/node_modules/**
# Generated client CSS (tailwind compile output); built by
# packages/dsh-plugin-browserskill/scripts/build-client-css.mjs.
**/dsh-plugin-browserskill/src/client/bsk-ui.nomodule.css
@@ -104,27 +104,6 @@ describe("reduceTraceSteps", () => {
});
});
it("keeps committed empty fill steps", () => {
const { steps } = reduceTraceSteps(
[
{
op: "fill",
target: { tag: "input", role: "textbox", name: "Search query" },
value: "",
page_url: "https://example.com/search",
},
],
"https://example.com/search",
);
expect(steps).toEqual([
expect.objectContaining({
op: "fill",
value: "",
}),
]);
});
it("maps select navigated_to onto effect.navigated_to (page id)", () => {
const { pages, steps } = reduceTraceSteps(
[
+1
View File
@@ -18,6 +18,7 @@ export function shouldRecordPress(
}
function shouldIncludeDraft(step: DraftTraceStep): boolean {
if (step.op === "fill" && !(step.value ?? "").trim() && !step.redacted) return false;
if (step.op === "press" && !shouldRecordPress(step.key, step.modifiers)) return false;
return true;
}
+5
View File
@@ -39,6 +39,11 @@
"**",
"!**/node_modules/**",
"!**/dist/**",
"!packages/dsh-plugin-browserskill/lib/**",
"!packages/dsh-plugin-browserskill/src/skill-content.generated.ts",
"!packages/dsh-plugin-browserskill/src/client/bsk-ui.nomodule.css",
"!packages/dsh-plugin-browserskill/.bsk-ui.raw.css",
"!packages/dsh-plugin-browserskill/.bsk-ui.input.css",
"!**/target/**",
"!**/.wxt/**",
"!**/.output/**",
+1 -1
View File
@@ -9,7 +9,7 @@
"ext:dev": "pnpm --filter @browser-skill/extension dev",
"ext:test": "pnpm --filter @browser-skill/extension test",
"vom:test": "pnpm --filter @browser-skill/vom test",
"lint": "biome check . && stylelint \"**/*.css\"",
"lint": "biome check . && stylelint \"**/*.css\" && pnpm --filter @wxg-prc-cpg/browser-skill-dsh-plugin typecheck && pnpm --filter @wxg-prc-cpg/browser-skill-dsh-plugin test",
"format": "biome format --write ."
},
"devDependencies": {
@@ -0,0 +1,5 @@
lib/
src/client/bsk-ui.nomodule.css
.bsk-ui.input.css
.bsk-ui.raw.css
src/skill-content.generated.ts
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 Tencent
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+194
View File
@@ -0,0 +1,194 @@
# dsh-plugin-browserskill
npm: [`@wxg-prc-cpg/browser-skill-dsh-plugin`](https://www.npmjs.com/package/@wxg-prc-cpg/browser-skill-dsh-plugin)
A [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) (dsh) tool plugin that exposes
[BrowserSkill](https://github.com/Tencent/BrowserSkill) (`bsk`) browser automation to the model.
Each tool maps to one `bsk <cmd> --json` invocation: the plugin spawns the bsk CLI, parses its
structured JSON output, and returns a canonical typed value. The bsk daemon, browser, and Chrome
extension keep owning the actual browser control — this package is a thin, well-typed bridge.
## Tools
| Tool | bsk command | Purpose |
| --- | --- | --- |
| `browser_session_start` | `bsk session start` | Open an Agent Window session; optional initial URL, window size, and device emulation preset. Returns the session id and makes it the current session. |
| `browser_session_stop` | `bsk session stop` | Stop a session (the current one by default) and close its Agent Window. Only plugin-created sessions can be stopped. |
| `browser_session_list` | — (registry only) | List the sessions this plugin created, marking the current one. Foreign daemon sessions are never shown. |
| `browser_navigate` | `bsk navigate` | Navigate the active tab, with `waitUntil` / timeout control. |
| `browser_snapshot` | `bsk snapshot` | Indented aria-tree snapshot with `@eN` refs for interaction tools. |
| `browser_observe` | `bsk observe` | Semantic VOM observation (read-only) with `@eN` refs. |
| `browser_click` | `bsk click` | Click a snapshot ref or CSS selector (button / click-count options). |
| `browser_fill` | `bsk fill` | Fill an input / textarea / contenteditable (clears first by default). |
| `browser_press` | `bsk press` | Dispatch a key or combo, optionally focusing a target first. |
| `browser_screenshot` | `bsk screenshot` | PNG capture of the tab or a ref-cropped element; inlines the image when the deployment supports image input, otherwise returns a file path. |
| `browser_emulate` | `bsk emulate` | Apply or clear mobile device emulation on the active tab. |
## Agent skill (progressive disclosure)
Beyond the tools, the plugin publishes the **`browser-skill` agent skill** through the harness's
official skill seam (`ctx.skills.register`): the catalog entry (name + routing description) is
resident in `<available_skills>`, and the body is loaded only when the model invokes the `skill`
tool. The body is assembled at build time from two parts, so there is exactly one source of
truth: a dsh-specific prelude (`skill/prelude.md` — tool↔CLI map, owned-session semantics,
plugin-only overrides) followed verbatim by the canonical CLI skill (`skill/SKILL.md` at the
repo root — the same file `crates/bsk-cli/build.rs` mirrors for the CLI package; workflows,
stop-when-done rules, refs usage, sandbox rules). Registration and every pre-step catalog
snapshot are pure in-memory reads (no disk/process/daemon); compositions without the skill
seam degrade silently.
## Multi-session model
One agent conversation can drive several browser sessions at once:
- `browser_session_start` returns the session id and makes it the **current session**.
- Every operation tool accepts an optional `session` argument. When omitted, the call acts on the
current session (the one most recently started or used); when given, that session becomes current.
- Every tool result echoes the session it actually acted on, so the model never has to guess.
- The number of concurrent sessions started through the plugin is capped (`maxSessions`, default 5).
- Unloading the plugin stops every session it started and kills in-flight bsk processes.
**Ownership boundary**: the bsk daemon may be shared with other agents, terminals, or dsh
instances. The plugin therefore only ever sees and operates on sessions it created itself —
an explicit `session` argument naming a foreign or unknown id is rejected, `browser_session_list`
shows plugin-created sessions only (no daemon-wide view), and stop/unload cleanup can never touch
a session owned by another program.
## Installation
The plugin follows the standard dsh bundle layout (`dsh.bundle` manifest + `cordis.patch.yml`):
```sh
dsh plugin --profile <name> add @wxg-prc-cpg/browser-skill-dsh-plugin
dsh --profile <name>
```
Prerequisite: the `bsk` CLI must be installed and on `PATH`, and the BrowserSkill Chrome extension
must be connected — see the [BrowserSkill README](https://github.com/Tencent/BrowserSkill). When bsk
is missing, tool calls fail with install guidance instead of a bare spawn error.
## Configuration
All fields are optional and validated through the plugin's Schemastery `Config`:
```yaml
# cordis.patch.yml override example
- insert:
- id: browserskill
name: dsh-plugin-browserskill
config:
bskPath: bsk # path to the bsk binary (default: resolve from PATH)
defaultTimeoutMs: 120000
maxSessions: 5
# observationEnabled: true # live PiP/overlay observation (below)
# thumbnailIntervalMs: 1500 # frame cadence while a session is active
# idleIntervalMs: 8000 # idle cadence / recent-activity window
# lazyTools: true # reveal browser_* tools only after the skill is invoked
```
- **`lazyTools` (default `true`)** — the final progressive-disclosure stage: the eleven
`browser_*` tool schemas stay OUT of the system prompt (zero schema tokens) until the
`browser-skill` skill is actually invoked — the skill catalog entry is the only
advertisement. One successful invocation (model tool call, or a `/browser-skill` user
gesture) registers the whole suite for the rest of the process; repeated invocations are
no-ops, and sessions resumed with a past invocation in their durable log reveal the suite
on entry. Set `false` for the legacy always-on registration.
## Observation overlay (PiP mini-window)
When the plugin runs inside the dsh Web UI, an **observation overlay** floats over the app
(registered into the `shell.overlay` seat): a breathing thumbnail per owned session plus its
current action and elapsed time. The card docks at the top-right of the content area (clear of
the composer and the shell's header controls) and wears the **BrowserSkill product family's own
look**: the overlay reuses `@browser-skill/ui` components (`Button`, `cn`) and its oklch design
tokens (`--card`, `--primary`, `--destructive`, `--ring`, …), status dots spec'd after the
extension popup's connection indicator, and Remix icons. The BSK utility sheet is compiled
scoped under the `.bsk-obs` root class (`scripts/build-client-css.mjs`), so the overlay looks
like the BrowserSkill extension without leaking a single selector into the host shell — and the
shell's theme cannot bleed back in.
- **Lifecycle**: hidden while the plugin owns no sessions; appears on the first
`browser_session_start`; disappears when all sessions stop (or the plugin unloads).
- **Focus view**: status row (green/idle/red dot + session + action + mm:ss), the latest page
frame (refreshes every ~1.5s while active, ~8s when idle; the last good frame stays on
stage while the next one loads, and is kept on errors so the card does not flash),
and a compact icon toolbar (Interrupt + Pop out, hover for the label).
- **Interrupt**: one click kills the in-flight bsk command of the focus session (same semantics
as the chat Stop button — the current action fails, the agent run may continue). Strip items
carry their own hover interrupt button.
- **Multi-session strip**: every session gets a tile (mini frame + id + status dot); focus
auto-follows the most recently active session; clicking a tile pins focus (pin badge, click
again to release); errored sessions get a red edge without stealing focus; sessions the daemon
lost are greyed out; prolonged daemon/browser outage shows "browser unavailable" and greys
the interrupt button until captures recover.
- **Drag & resize**: drag the header to move the card; drag any of the four
corners to resize (min 240×180, max 80% of the viewport; no visible grip).
Both are remembered for the page lifetime.
- **Pop out (PiP)**: upgrades the card into a native Document PiP window (requires a user
gesture, per browser rules), sized from the current card; closing the PiP falls back to the
in-page card with state intact. Browsers without Document PiP simply hide the button.
- **Wire**: the host serves `GET /bsk-observation/state`, `GET /bsk-observation/events` (SSE),
`POST /bsk-observation/interrupt`, and `GET /bsk-observation/thumbnail/<attachmentId>` over
the dsh `webServer` route seam (dsh 0.1's Typert Remote pipeline is closed to out-of-tree
packages). All commands for one session — tool calls and frame captures alike — run through a
per-session FIFO, because the daemon accepts only one unfinished command per session.
- **Trust model**: these routes expose live screenshots (and an interrupt write), so they
replicate the browser-trust fence dsh applies to its own `/api` routes: the request Host
must be a loopback authority (`localhost`, `127.0.0.0/8`, `[::1]`), a present Origin must
match the Host, `sec-fetch-site: cross-site` is refused, and POST requires an
`application/json` body (cross-site simple requests can never satisfy that). The channel is
therefore built for **loopback-only serving** — binding the dsh web server to `0.0.0.0` and
reaching it through a LAN address will (deliberately) fail the fence; do not put these
routes behind a non-loopback reverse proxy without adding your own authentication.
- Configure with `observationEnabled` / `thumbnailIntervalMs` / `idleIntervalMs`.
## Behavior notes
- **Cancellation**: aborting a tool call (`exec.signal`) kills the underlying bsk child process,
matching BrowserSkill's cooperative tool-cancellation model.
- **UI cards**: calls render as terminal cards (command line as title, output as the completed
card). Screenshots additionally attach the image itself when the host mounts an attachment store
and the active model route declares image input; otherwise the PNG path is returned.
- **Web UI toolview (browser half)**: the package is dual-face. `dsh.client` (platform `web`) ships
`lib/client.js`, which registers a keyed `tool.call.toolview` view for `browser_screenshot`. The
custom view keeps the terminal block (command + output) and, when the settled result carries an
image block, resolves the durable attachment through the client session's authorized
`readAttachment` RPC and renders it with the shared `MessageImage` thumbnail/lightbox atoms.
Every other `browser_*` tool keeps the stock terminal card. The bundle follows the dsh client
contract: a CJS closure factory handed to `window.__ModuleLoader__.load`, platform modules
(`react`, `dsh-client-ui-*`) external, everything else inlined, CSS Modules compiled by
lightningcss.
- **Errors**: non-zero bsk exits surface the CLI's JSON error envelope (`code`, `message`, `hint`)
so the model gets the daemon's actionable guidance.
- **Long-running work** (e.g. `bsk record`) is not backgrounded via `ctx.jobs` yet — tracked as a
follow-up.
## Development
```sh
pnpm install
pnpm --filter @wxg-prc-cpg/browser-skill-dsh-plugin typecheck
pnpm --filter @wxg-prc-cpg/browser-skill-dsh-plugin test # unit tests mock bsk; no browser needed
pnpm --filter @wxg-prc-cpg/browser-skill-dsh-plugin build # tsdown -> lib/
```
## Publishing
The GitHub Actions workflow **Release dsh plugin** publishes this package to npm
as `@wxg-prc-cpg/browser-skill-dsh-plugin`. The Cordis plugin id stays
`dsh-plugin-browserskill`.
Trigger it by pushing a tag that matches `package.json`'s `version`:
```sh
git tag dsh-plugin-v0.1.0
git push origin dsh-plugin-v0.1.0
```
Or run the workflow from the Actions tab (`workflow_dispatch`). The job reads
`NPM_TOKEN` from the GitHub Environment of the same name.
## License
MIT
@@ -0,0 +1,3 @@
- insert:
- id: browserskill
name: dsh-plugin-browserskill
@@ -0,0 +1,130 @@
{
"name": "@wxg-prc-cpg/browser-skill-dsh-plugin",
"description": "DeepSeek Harness tool plugin that exposes BrowserSkill (bsk) browser automation to the model",
"version": "0.1.0",
"license": "MIT",
"publishConfig": {
"access": "public",
"registry": "https://registry.npmjs.org/"
},
"repository": {
"type": "git",
"url": "git+https://github.com/Tencent/BrowserSkill.git",
"directory": "packages/dsh-plugin-browserskill"
},
"homepage": "https://github.com/Tencent/BrowserSkill/tree/main/packages/dsh-plugin-browserskill#readme",
"bugs": {
"url": "https://github.com/Tencent/BrowserSkill/issues"
},
"keywords": [
"dsh-plugin",
"deepseek-harness",
"browserskill",
"browser-automation"
],
"type": "module",
"main": "lib/index.mjs",
"types": "lib/index.d.mts",
"exports": {
".": {
"types": "./lib/index.d.mts",
"default": "./lib/index.mjs"
},
"./client": {
"default": "./lib/client.cjs"
},
"./package.json": "./package.json"
},
"files": [
"lib",
"cordis.patch.yml",
"LICENSE"
],
"dsh": {
"bundle": {
"patch": "./cordis.patch.yml"
},
"client": {
"platform": "web",
"inject": [
"@deepseek-ai/dsh-client-runtime",
"@deepseek-ai/dsh-client-ui-tool",
"@deepseek-ai/dsh-client-ui-layout"
]
}
},
"scripts": {
"build": "node scripts/build-client-css.mjs && node scripts/build-skill-content.mjs && tsdown",
"typecheck": "pnpm run build:skill && tsc --noEmit",
"test": "pnpm run build:css && pnpm run build:skill && vitest run",
"build:css": "node scripts/build-client-css.mjs",
"build:skill": "node scripts/build-skill-content.mjs",
"prepack": "pnpm run build",
"lint:publint": "publint"
},
"peerDependencies": {
"@deepseek-ai/cordis": "^4.0.1",
"@deepseek-ai/dsh-attachment": "^0.1.0-rc.6",
"@deepseek-ai/dsh-client-runtime": "^0.1.0-rc.6",
"@deepseek-ai/dsh-client-ui-attachment": "^0.1.0-rc.6",
"@deepseek-ai/dsh-client-ui-primitives": "^0.1.0-rc.6",
"@deepseek-ai/dsh-client-ui-tool": "^0.1.0-rc.6",
"@deepseek-ai/dsh-llm": "^0.1.0-rc.6",
"@deepseek-ai/dsh-tools": "^0.1.0-rc.6",
"@deepseek-ai/schemastery": "^3.18.1",
"react": "^18.2.0"
},
"peerDependenciesMeta": {
"@deepseek-ai/dsh-attachment": {
"optional": true
},
"@deepseek-ai/dsh-client-runtime": {
"optional": true
},
"@deepseek-ai/dsh-client-ui-attachment": {
"optional": true
},
"@deepseek-ai/dsh-client-ui-primitives": {
"optional": true
},
"@deepseek-ai/dsh-client-ui-tool": {
"optional": true
},
"@deepseek-ai/dsh-llm": {
"optional": true
},
"react": {
"optional": true
}
},
"devDependencies": {
"@deepseek-ai/cordis": "^4.0.1",
"@deepseek-ai/dsh-agent": "^0.1.0-rc.6",
"@deepseek-ai/dsh-attachment": "^0.1.0-rc.6",
"@deepseek-ai/dsh-client-runtime": "^0.1.0-rc.6",
"@deepseek-ai/dsh-client-ui-attachment": "^0.1.0-rc.6",
"@deepseek-ai/dsh-client-ui-layout": "^0.1.0-rc.6",
"@deepseek-ai/dsh-client-ui-primitives": "^0.1.0-rc.6",
"@deepseek-ai/dsh-client-ui-tool": "^0.1.0-rc.6",
"@deepseek-ai/dsh-llm": "^0.1.0-rc.6",
"@deepseek-ai/dsh-tools": "^0.1.0-rc.6",
"@deepseek-ai/schemastery": "^3.18.1",
"@tailwindcss/cli": "^4.3.3",
"@testing-library/react": "^16.1.0",
"@types/node": "^22.10.0",
"@types/react": "~18.3.1",
"@types/react-dom": "^18.3.0",
"happy-dom": "^15.11.7",
"lightningcss": "^1.30.0",
"postcss": "^8.5.0",
"postcss-prefix-selector": "^2.1.1",
"react": "^18.2.0",
"react-dom": "^18.2.0",
"tailwindcss": "^4.3.3",
"tsdown": "^0.22.14",
"typescript": "^5.8.0",
"vitest": "^4.1.6",
"@browser-skill/ui": "workspace:*",
"@remixicon/react": "^4.9.0"
}
}
@@ -0,0 +1,72 @@
#!/usr/bin/env node
/**
* Build the scoped BrowserSkill-UI utility sheet for the dsh client bundle.
*
* The overlay reuses @browser-skill/ui components (shadcn-style, Tailwind v4
* utility classes). Those classes need a stylesheet — but a global utility
* sheet inside the dsh page would collide with the host UI, so the compiled
* output is scope-prefixed under `.bsk-obs` (the overlay root class):
* 1. tailwindcss compiles the utilities used by packages/ui + our client;
* 2. postcss-prefix-selector nests every rule under .bsk-obs.
* Token values themselves live in src/client/bsk-tokens.nomodule.css
* (hand-mirrored from packages/ui/src/styles/tailwind.css).
*
* Output: src/client/bsk-ui.nomodule.css (gitignored build artifact).
*/
import { execFileSync } from "node:child_process";
import { mkdirSync, readFileSync, writeFileSync } from "node:fs";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
import postcss from "postcss";
import prefixer from "postcss-prefix-selector";
const pkgRoot = dirname(dirname(fileURLToPath(import.meta.url)));
const input = join(pkgRoot, ".bsk-ui.input.css");
const rawOut = join(pkgRoot, ".bsk-ui.raw.css");
const finalOut = join(pkgRoot, "src/client/bsk-ui.nomodule.css");
// The canonical sheet can't be imported directly (its own `@import
// "tailwindcss"` must resolve from a package that has it), so we inline the
// canonical @theme block read from packages/ui at build time.
const canonical = readFileSync(join(pkgRoot, "..", "ui", "src", "styles", "tailwind.css"), "utf8");
const themeBlock = canonical.match(/@theme inline \{[\s\S]*?\n\}/)?.[0];
if (themeBlock === undefined) throw new Error("canonical @theme inline block not found");
writeFileSync(
input,
[
'@import "tailwindcss";',
"@custom-variant dark (&:is(.dark *));",
themeBlock,
'@source "../ui/src";',
'@source "./src/client";',
"",
].join("\n"),
);
execFileSync(
process.execPath,
[
join(pkgRoot, "node_modules", "@tailwindcss", "cli", "dist", "index.mjs"),
"-i",
input,
"-o",
rawOut,
],
{ cwd: pkgRoot, stdio: "inherit" },
);
const result = await postcss([
prefixer({
prefix: ".bsk-obs",
transform(prefix, selector, prefixedSelector) {
// Root-level anchors collapse onto the scope root itself.
if (selector === ":root" || selector === "html" || selector === "body") return prefix;
if (selector.startsWith(prefix)) return selector;
return prefixedSelector;
},
}),
]).process(readFileSync(rawOut, "utf8"), { from: rawOut });
mkdirSync(dirname(finalOut), { recursive: true });
writeFileSync(finalOut, result.css);
console.log(`bsk-ui sheet: ${result.css.length} bytes -> ${finalOut}`);
@@ -0,0 +1,44 @@
// Assemble the plugin's skill content: a dsh-specific routing prelude, the
// canonical CLI skill body verbatim (repo-root skill/SKILL.md — the same source
// crates/bsk-cli's build.rs mirrors), then a dsh routing reminder postlude. The
// original's YAML frontmatter is stripped: name/description are supplied by the
// registration.
// Output: src/skill-content.generated.ts (gitignored, built before tsdown and
// before vitest) so the markdown ships inlined in the bundle and registration
// is a pure in-memory read — never a per-step disk access.
import { readFileSync, writeFileSync } from "node:fs";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
const here = dirname(fileURLToPath(import.meta.url));
const pkg = join(here, "..");
const prelude = readFileSync(join(pkg, "skill", "prelude.md"), "utf8");
const upstream = readFileSync(join(pkg, "..", "..", "skill", "SKILL.md"), "utf8");
const postlude = readFileSync(join(pkg, "skill", "postlude.md"), "utf8");
function stripFrontmatter(markdown) {
if (!markdown.startsWith("---\n")) return markdown;
const end = markdown.indexOf("\n---\n", 4);
return end === -1 ? markdown : markdown.slice(end + 5).replace(/^\s*\n/, "");
}
const NAME = "browser-skill";
const DESCRIPTION =
"Browser automation against the user's logged-in Chromium through this plugin's injected browser_* tools; never invoke bsk through bash or shell. " +
"tools (bsk engine): visit and read pages, fill forms, click through flows, scrape data, " +
"smoke-test UI changes. Use when a task needs a real browser; requires the bsk CLI and the " +
"browser-skill extension connected.";
const content = `${prelude.trim()}\n\n${stripFrontmatter(upstream).trim()}\n\n${postlude.trim()}\n`;
const banner =
"// GENERATED by scripts/build-skill-content.mjs — do not edit.\n" +
"// Sources: skill/prelude.md + ../../../skill/SKILL.md + skill/postlude.md.\n";
writeFileSync(
join(pkg, "src", "skill-content.generated.ts"),
`${banner}export const BSK_SKILL_NAME = ${JSON.stringify(NAME)};\n` +
`export const BSK_SKILL_DESCRIPTION = ${JSON.stringify(DESCRIPTION)};\n` +
`export const BSK_SKILL_MARKDOWN = ${JSON.stringify(content)};\n`,
);
console.log(`skill-content.generated.ts written (${content.length} chars)`);
@@ -0,0 +1,10 @@
---
## DSH routing reminder
The CLI-oriented reference above describes behavior, not the execution mechanism in
DeepSeek Harness. For every browser operation, call the injected `browser_*` model
tool. Do not translate these examples into `bash` commands and do not execute `bsk`
directly; the plugin invokes `bsk` internally. In particular, always begin with
`browser_session_start`, pass its returned session id to subsequent `browser_*`
tools, and finish with `browser_session_stop`.
@@ -0,0 +1,71 @@
# browser-skill (dsh edition)
## DSH tool-routing requirement (highest priority)
This skill is running inside DeepSeek Harness, where the plugin has injected
structured `browser_*` model tools. **Always call those injected `browser_*` tools
for browser work. Never use `bash`, `shell`, `exec`, or another tool to invoke the
`bsk` CLI directly.** The CLI command names shown below are only conceptual
mappings for understanding the tools; they are not instructions to run commands.
If a `browser_*` tool is available, using the CLI directly is incorrect because it
bypasses plugin session ownership, observation UI, tool cards, and cleanup.
Use the injected tool whose name matches the operation, for example:
`browser_session_start`, `browser_navigate`, `browser_observe`, `browser_snapshot`,
`browser_click`, `browser_fill`, `browser_press`, `browser_screenshot`, and
`browser_session_stop`. Pass arguments as the tool schema specifies (not CLI flags).
Drive the user's **real Chromium browser** (logins, cookies, Agent Window isolation)
through this plugin's `browser_*` tools. Each injected tool is implemented by the
plugin using the `bsk` engine internally — the model must not invoke that engine
itself. The workflow, judgment rules, and safety constraints remain identical to
the canonical CLI skill.
> **Tool availability**: invoking this skill (which you just did) makes the plugin's
> `browser_*` tools available for the rest of the session — they are injected on
> successful skill load. Until then they are deliberately hidden, so do not attempt
> a browser action before loading this skill.
## Tool ↔ CLI map
The tools below are thin, structured wrappers over the bsk CLI verbs of the same
name; `session` parameters take the 4-letter session id (no `--session` flag
spelling here):
| dsh tool | bsk equivalent |
|---|---|
| `browser_session_start` / `browser_session_stop` / `browser_session_list` | `bsk session start/stop/list` |
| `browser_navigate` | `bsk navigate` |
| `browser_snapshot` / `browser_observe` | `bsk snapshot` / `bsk observe` |
| `browser_click` / `browser_fill` / `browser_press` | `bsk click` / `bsk fill` / `bsk press` |
| `browser_screenshot` | `bsk screenshot` |
| `browser_emulate` | `bsk emulate` |
(`bsk console` / `bsk network` have no tool counterpart in this plugin yet.)
Plugin-specific semantics that override or narrow the CLI rules below:
- **Owned sessions only**: this plugin sees and acts on just the sessions IT
started via `browser_session_start`. `browser_session_list` shows only those
(with the current marker); passing a session id created by another program
(or person) sharing the bsk daemon is refused outright — never retry those
with the same id.
- **Current session**: omitted `session` parameters resolve to the most
recently started or used OWNED session. With several sessions in flight,
pass `session` explicitly to avoid surprises.
- **Screenshots**: when the model route accepts images, `browser_screenshot`
returns the PNG inline as an image attachment; otherwise it returns the
saved file path.
- **`browser_emulate mobile`**: requires `width`+`height` — the bsk daemon
refuses `--mobile` without viewport dimensions.
- **Live observation**: in the dsh Web UI, every owned session is watched by
the floating overlay (thumbnails, action, interrupt) — observational side
traffic is free and never steals your command turn; just keep acting.
The canonical CLI skill body follows verbatim (source of truth: `skill/SKILL.md`
in the BrowserSkill repo, mirrored by build into `crates/bsk-cli/skill/SKILL.md`).
Read `bsk <verb> --session <id>` as "the `browser_<verb>` tool with
`session: <id>`"; everything else — when to use, the stop-when-done rules, the
interaction loop, refs, sandbox rules — applies unchanged.
---
@@ -0,0 +1,370 @@
/* Observation overlay skin, BrowserSkill family: colors/radius ride the BSK
design tokens (--card/--border/--primary/--destructive/--ring/--radius,
defined on the .bsk-obs scope root by bsk-tokens.nomodule.css — values
mirrored from packages/ui/src/styles/tailwind.css). Layout-only here;
component chrome (buttons etc.) comes from @browser-skill/ui plus the
scope-prefixed utility sheet. The shell overlay layer is click-through, so
the card and capsule opt back into pointer events themselves. */
.card {
position: fixed;
z-index: 40;
display: flex;
flex-direction: column;
pointer-events: auto;
background: var(--card);
border: 1px solid var(--border);
border-radius: var(--radius);
box-shadow: 0 8px 28px rgb(0 0 0 / 18%);
overflow: hidden;
color: var(--card-foreground);
}
.body {
display: flex;
flex-direction: column;
height: 100%;
min-height: 0;
font-size: 12px;
background: var(--card);
color: var(--card-foreground);
}
.header {
display: flex;
align-items: center;
gap: 6px;
padding: 6px 8px;
cursor: move;
user-select: none;
touch-action: none;
border-bottom: 1px solid var(--border);
}
.status-text {
flex: 1;
min-width: 0;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
font-weight: 500;
}
.icon-button {
flex: none;
display: inline-flex;
align-items: center;
border: none;
background: none;
padding: 2px;
border-radius: calc(var(--radius) - 6px);
font: inherit;
color: var(--muted-foreground);
cursor: pointer;
}
.icon-button:hover {
color: var(--foreground);
background: var(--accent);
}
.stage {
position: relative;
flex: 1;
min-height: 0;
display: flex;
align-items: center;
justify-content: center;
background: var(--background);
overflow: hidden;
}
/* Fade only on first mount. The <img> is reused across frames (src swap,
no React key) so this animation must not replay on every breath. */
.thumb {
max-width: 100%;
max-height: 100%;
object-fit: contain;
animation: bsk-obs-fade-in 240ms ease-out;
}
@keyframes bsk-obs-fade-in {
from {
opacity: 0;
}
to {
opacity: 1;
}
}
.placeholder {
color: var(--muted-foreground);
font-size: 12px;
}
.badge {
position: absolute;
top: 6px;
right: 6px;
width: 16px;
height: 16px;
border-radius: 50%;
display: inline-flex;
align-items: center;
justify-content: center;
color: oklch(0.7 0.19 60);
border: 1px solid oklch(0.7 0.19 60);
background: var(--card);
}
.actions {
position: relative;
z-index: 3;
display: flex;
align-items: center;
justify-content: space-between;
gap: 4px;
/* A little inset so icons are not flush with the resize corners. */
padding: 4px 8px;
min-height: 32px;
border-top: 1px solid var(--border);
}
.tool-wrap {
position: relative;
display: inline-flex;
}
.tool-button {
flex: none;
display: inline-flex;
align-items: center;
justify-content: center;
width: 28px;
height: 28px;
padding: 0;
border: none;
border-radius: calc(var(--radius) - 4px);
background: none;
color: var(--muted-foreground);
cursor: pointer;
}
.tool-button:disabled {
opacity: 0.4;
cursor: default;
}
.tool-button:focus-visible {
outline: 2px solid var(--ring);
outline-offset: 2px;
}
.tool-danger {
color: var(--destructive);
}
/* Hover the wrap (not just :hover on the button) so the tooltip target and
the left-corner resize overlay still light the icon up. */
.tool-wrap:hover .tool-button:not(:disabled) {
color: var(--foreground);
background: var(--accent);
}
.tool-wrap:hover .tool-danger:not(:disabled) {
color: var(--destructive);
background: color-mix(in oklch, var(--destructive) 18%, var(--card));
}
.tool-wrap:hover .tool-button:disabled {
background: var(--accent);
}
.hint {
position: absolute;
bottom: calc(100% + 4px);
left: 0;
width: max-content;
max-width: 220px;
padding: 4px 6px;
border-radius: calc(var(--radius) - 4px);
background: var(--card);
border: 1px solid var(--border);
color: var(--muted-foreground);
font-size: 11px;
line-height: 1.35;
box-shadow: 0 4px 14px rgb(0 0 0 / 16%);
z-index: 41;
}
.hint[data-align="end"] {
left: auto;
right: 0;
}
/* Invisible hit targets — no grip glyph; all four corners resize. */
.resize-handle {
position: absolute;
width: 16px;
height: 16px;
touch-action: none;
z-index: 2;
}
.resize-handle[data-corner="nw"] {
top: 0;
left: 0;
cursor: nwse-resize;
}
.resize-handle[data-corner="ne"] {
top: 0;
right: 0;
cursor: nesw-resize;
}
.resize-handle[data-corner="sw"] {
bottom: 0;
left: 0;
cursor: nesw-resize;
}
.resize-handle[data-corner="se"] {
bottom: 0;
right: 0;
cursor: nwse-resize;
}
.capsule {
position: fixed;
right: 16px;
top: 64px;
z-index: 40;
display: inline-flex;
align-items: center;
gap: 6px;
padding: 6px 12px;
pointer-events: auto;
border-radius: 999px;
border: 1px solid var(--border);
background: var(--card);
box-shadow: 0 4px 14px rgb(0 0 0 / 16%);
font-size: 12px;
color: var(--muted-foreground);
cursor: pointer;
}
.capsule:hover {
background: var(--accent);
}
.strip {
display: flex;
gap: 6px;
padding: 4px 8px;
overflow-x: auto;
border-top: 1px solid var(--border);
}
.strip-item {
position: relative;
flex: none;
border-radius: calc(var(--radius) - 6px);
border: 1px solid var(--border);
}
.strip-item[data-focused] {
border-color: var(--primary);
}
.strip-item[data-state="error"] {
border-color: var(--destructive);
}
.strip-item[data-state="dead"] {
opacity: 0.45;
}
.strip-main {
display: flex;
align-items: center;
gap: 4px;
padding: 3px 6px;
border: none;
border-radius: calc(var(--radius) - 6px);
background: none;
font: inherit;
font-size: 11px;
color: var(--muted-foreground);
cursor: pointer;
}
.strip-main:hover {
background: var(--accent);
}
.strip-thumb {
width: 28px;
height: 20px;
overflow: hidden;
border-radius: 3px;
background: var(--background);
display: inline-flex;
}
.strip-thumb img {
width: 100%;
height: 100%;
object-fit: cover;
}
.strip-thumb-empty {
width: 100%;
height: 100%;
}
.strip-id {
max-width: 64px;
overflow: hidden;
text-overflow: ellipsis;
}
.pin-badge {
color: var(--primary);
}
.strip-interrupt {
position: absolute;
top: -7px;
right: -7px;
width: 16px;
height: 16px;
border-radius: 50%;
display: inline-flex;
align-items: center;
justify-content: center;
border: 1px solid var(--destructive);
background: var(--card);
color: var(--destructive);
padding: 0;
cursor: pointer;
}
.strip-interrupt:disabled {
opacity: 0.4;
cursor: default;
}
.icon-button:focus-visible,
.capsule:focus-visible,
.strip-main:focus-visible,
.strip-interrupt:focus-visible {
outline: 2px solid var(--ring);
outline-offset: 2px;
}
@media (prefers-reduced-motion: reduce) {
.thumb {
animation: none;
}
}
@@ -0,0 +1,612 @@
// Observation overlay: the default in-app carrier for live session watching.
// Expanded = draggable/resizable floating card (top-right, out of the
// composer's way); collapsed = a status capsule; "pop out" upgrades the same
// content to a native Document PiP window (user gesture required by the
// browser). Multi-session renders a meeting-style strip under the focus view.
// Visuals follow the BrowserSkill product family: @browser-skill/ui components
// and oklch tokens on a .bsk-obs scope root, so the card reads as BSK's own
// surface without leaking styles into (or inheriting themes from) the shell.
import { cn } from "@browser-skill/ui";
import {
RiArrowDownSLine,
RiErrorWarningLine,
RiPictureInPicture2Line,
RiPushpinFill,
RiStopCircleLine,
} from "@remixicon/react";
// remixicon's component types target @types/react 19 while the dsh shell
// runs react 18 — a compile-time-only recast keeps the 18 typecheck happy.
type IconComponent = (props: { size?: number | string; className?: string }) => ReactNode;
const asIcon = (component: unknown): IconComponent => component as IconComponent;
const IconStop = asIcon(RiStopCircleLine);
const IconPip = asIcon(RiPictureInPicture2Line);
const IconDown = asIcon(RiArrowDownSLine);
const IconWarn = asIcon(RiErrorWarningLine);
const IconPin = asIcon(RiPushpinFill);
import {
type ReactNode,
type PointerEvent as ReactPointerEvent,
useCallback,
useEffect,
useRef,
useState,
useSyncExternalStore,
} from "react";
import { createPortal } from "react-dom";
import type { SessionObservation } from "../observation";
import css from "./ObservationOverlay.module.css";
import type { ObservationClientStore } from "./observation-store";
/** Minimal Document PiP surface (TS lib.dom lacks it). */
interface DocumentPip {
requestWindow(options?: { width?: number; height?: number }): Promise<Window>;
}
interface Point {
x: number;
y: number;
}
interface Size {
w: number;
h: number;
}
const DEFAULT_SIZE: Size = { w: 320, h: 240 };
const MIN_SIZE: Size = { w: 240, h: 180 };
const EDGE_MARGIN = 16;
/** Default dock: top-right, just under the shell's top bar (no spacing tokens exist in dsh 0.1). */
const TOP_OFFSET = 64;
function pipApi(): DocumentPip | undefined {
if (typeof window === "undefined") return undefined;
return (window as unknown as { documentPictureInPicture?: DocumentPip }).documentPictureInPicture;
}
function clampSize(size: Size, viewport: Size): Size {
const maxW = viewport.w * 0.8;
const maxH = viewport.h * 0.8;
return {
w: Math.min(Math.max(size.w, MIN_SIZE.w), maxW),
h: Math.min(Math.max(size.h, MIN_SIZE.h), maxH),
};
}
function clampPos(pos: Point, size: Size, viewport: Size): Point {
return {
x: Math.min(Math.max(pos.x, 0), Math.max(0, viewport.w - size.w)),
y: Math.min(Math.max(pos.y, 0), Math.max(0, viewport.h - size.h)),
};
}
export type ResizeCorner = "nw" | "ne" | "sw" | "se";
/** Grow/shrink from one corner, keeping the opposite corner planted. */
export function applyResize(
base: Point & Size,
corner: ResizeCorner,
dx: number,
dy: number,
viewport: Size,
): { pos: Point; size: Size } {
const nextW = corner === "ne" || corner === "se" ? base.w + dx : base.w - dx;
const nextH = corner === "sw" || corner === "se" ? base.h + dy : base.h - dy;
const size = clampSize({ w: nextW, h: nextH }, viewport);
const x = corner === "nw" || corner === "sw" ? base.x + base.w - size.w : base.x;
const y = corner === "nw" || corner === "ne" ? base.y + base.h - size.h : base.y;
return { pos: clampPos({ x, y }, size, viewport), size };
}
const CORNER_LABEL: Record<ResizeCorner, string> = {
nw: "top left",
ne: "top right",
sw: "bottom left",
se: "bottom right",
};
function formatElapsed(sinceMs: number, nowMs: number): string {
const total = Math.max(0, Math.floor((nowMs - sinceMs) / 1000));
const mm = String(Math.floor(total / 60)).padStart(2, "0");
const ss = String(total % 60).padStart(2, "0");
return `${mm}:${ss}`;
}
/**
* Auto-follow focus: the most recently touched session, but never yank focus
* to a session whose latest action failed (errors flag the strip item, they
* do not steal the stage) or one already reported dead.
*/
export function focusOf(sessions: readonly SessionObservation[]): SessionObservation | undefined {
if (sessions.length === 0) return undefined;
const byRecency = [...sessions].sort((a, b) => b.since - a.since);
const healthy = byRecency.find((s) => s.lastError === undefined && s.dead !== true);
return healthy ?? byRecency[0];
}
export function statusOf(obs: SessionObservation): "active" | "idle" | "error" {
if (obs.lastError !== undefined && obs.action === "idle") return "error";
return obs.action === "idle" ? "idle" : "active";
}
/** Compact toolbar icon: no label, hover bubble for the name / semantics. */
function IconAction(props: {
label: string;
hint: string;
disabled?: boolean;
danger?: boolean;
align?: "start" | "end";
onClick: () => void;
children: ReactNode;
}) {
const [open, setOpen] = useState(false);
return (
<span
className={css["tool-wrap"]}
onPointerEnter={() => setOpen(true)}
onPointerLeave={() => setOpen(false)}
>
<button
type="button"
className={cn(css["tool-button"], props.danger === true && css["tool-danger"])}
disabled={props.disabled}
aria-label={props.label}
onClick={props.onClick}
>
{props.children}
</button>
{open ? (
<span className={css.hint} data-align={props.align} role="tooltip">
{props.hint}
</span>
) : null}
</span>
);
}
/** Flat status dot, specced after the BSK popup's ConnectionStatusIndicator. */
function StatusDot({ state }: { state: "active" | "idle" | "error" | "dead" }) {
const color =
state === "active"
? "bg-emerald-500"
: state === "error"
? "bg-red-500"
: state === "dead"
? "bg-amber-500"
: "bg-muted-foreground/40";
return (
<span
className={cn("size-2 shrink-0 rounded-full ring-2 ring-background", color)}
data-state={state}
aria-hidden
/>
);
}
/** One strip item: mini frame, id, status dot, hover interrupt, pin toggle. */
function StripItem(props: {
store: ObservationClientStore;
obs: SessionObservation;
pinned: boolean;
focused: boolean;
onTogglePin: (sessionId: string) => void;
}) {
const { store, obs, pinned, focused, onTogglePin } = props;
const [hover, setHover] = useState(false);
const thumbId = obs.thumbnailAttachmentId;
useEffect(() => {
store.ensureThumbnail(thumbId);
}, [store, thumbId]);
const thumb = store.getSnapshot().displayFrames[obs.sessionId];
const state = obs.dead === true ? "dead" : statusOf(obs);
return (
<div
className={css["strip-item"]}
data-state={state}
data-focused={focused || undefined}
data-pinned={pinned || undefined}
onPointerEnter={() => setHover(true)}
onPointerLeave={() => setHover(false)}
>
<button
type="button"
className={css["strip-main"]}
aria-label={`${pinned ? "Unpin" : "Pin"} session ${obs.sessionId}`}
aria-pressed={pinned}
onClick={() => onTogglePin(obs.sessionId)}
>
<span className={css["strip-thumb"]}>
{thumb?.url !== undefined ? (
<img src={thumb.url} alt="" />
) : (
<span className={css["strip-thumb-empty"]} />
)}
</span>
<span className={css["strip-id"]}>{obs.sessionId}</span>
<StatusDot state={state} />
{pinned ? <IconPin size={9} className={css["pin-badge"]} aria-label="pinned" /> : null}
</button>
{hover && !pinned && obs.dead !== true ? (
<button
type="button"
className={css["strip-interrupt"]}
aria-label={`Interrupt session ${obs.sessionId}`}
disabled={obs.action === "idle"}
onClick={(event) => {
event.stopPropagation();
void store.interrupt(obs.sessionId);
}}
>
<IconStop size={10} />
</button>
) : null}
</div>
);
}
/** The floating card / PiP shared content. */
function OverlayBody(props: {
store: ObservationClientStore;
focus: SessionObservation | undefined;
sessions: readonly SessionObservation[];
available: boolean;
pinnedId: string | null;
onTogglePin: (sessionId: string) => void;
now: number;
onPopOut?: (() => void) | undefined;
onCollapse?: (() => void) | undefined;
inPip: boolean;
onHeaderPointerDown?: (event: ReactPointerEvent<HTMLDivElement>) => void;
}) {
const {
store,
focus,
sessions,
available,
pinnedId,
onTogglePin,
now,
onPopOut,
onCollapse,
inPip,
onHeaderPointerDown,
} = props;
const [interrupting, setInterrupting] = useState(false);
const thumbId = focus?.thumbnailAttachmentId;
useEffect(() => {
store.ensureThumbnail(thumbId);
}, [store, thumbId]);
// Paint the last good frame while the next attachment decodes — swapping
// to a placeholder (and remounting <img> with a fade) is what made the
// card flash on every breath / action-end capture.
const thumb =
focus !== undefined ? store.getSnapshot().displayFrames[focus.sessionId] : undefined;
const displayUrl = thumb?.url;
const canInterrupt =
available &&
!interrupting &&
focus !== undefined &&
focus.action !== "idle" &&
focus.dead !== true;
const onInterrupt = (): void => {
if (!canInterrupt || focus === undefined) return;
setInterrupting(true);
void store.interrupt(focus.sessionId).finally(() => setInterrupting(false));
};
const statusText = !available
? "browser unavailable"
: focus === undefined
? "no session"
: `${focus.sessionId} · ${focus.action === "idle" ? "idle" : focus.action} · ${formatElapsed(focus.since, now)}`;
const state = !available ? "error" : focus !== undefined ? statusOf(focus) : "idle";
return (
<div className={cn(css.body, "bsk-obs")} data-state={state} data-in-pip={inPip || undefined}>
<div
className={css.header}
data-testid="obs-header"
onPointerDown={onHeaderPointerDown}
role="presentation"
>
<StatusDot state={state === "error" ? "error" : state === "active" ? "active" : "idle"} />
<span className={css["status-text"]}>{statusText}</span>
{onCollapse !== undefined ? (
<button
type="button"
className={css["icon-button"]}
aria-label="Collapse"
onClick={onCollapse}
>
<IconDown size={14} />
</button>
) : null}
</div>
<div className={css.stage}>
{displayUrl !== undefined ? (
<img
className={css.thumb}
src={displayUrl}
alt={`session ${focus?.sessionId ?? ""} view`}
/>
) : (
<div className={css.placeholder}>
{!available
? "last frame kept"
: thumb?.status === "error"
? "frame unavailable"
: "waiting for page"}
</div>
)}
{thumb?.status === "error" ? (
<span className={css.badge} aria-label="thumbnail failed">
<IconWarn size={12} />
</span>
) : null}
</div>
{sessions.length >= 2 ? (
<div className={css.strip} data-testid="obs-strip" role="list">
{sessions.map((obs) => (
<StripItem
key={obs.sessionId}
store={store}
obs={obs}
pinned={pinnedId === obs.sessionId}
focused={focus?.sessionId === obs.sessionId}
onTogglePin={onTogglePin}
/>
))}
</div>
) : null}
<div className={css.actions}>
<IconAction
label={interrupting ? "Interrupting…" : "Interrupt the current browser action"}
hint="Stop the current browser action."
disabled={!canInterrupt}
danger
onClick={onInterrupt}
>
<IconStop size={16} />
</IconAction>
{onPopOut !== undefined ? (
<IconAction
label="Pop out into a mini window"
hint="Pop out into a mini window"
align="end"
onClick={onPopOut}
>
<IconPip size={16} />
</IconAction>
) : null}
</div>
</div>
);
}
/** Clone the host document's style/link nodes into a PiP window. */
function cloneStylesInto(pipWindow: Window): void {
for (const node of document.querySelectorAll('link[rel="stylesheet"], style')) {
pipWindow.document.head.appendChild(node.cloneNode(true));
}
}
export function ObservationOverlay({ store }: { store: ObservationClientStore }) {
const snapshot = useSyncExternalStore(store.subscribe, store.getSnapshot);
useEffect(() => {
store.start();
return () => store.stop();
}, [store]);
const [collapsed, setCollapsed] = useState(false);
const [pos, setPos] = useState<Point | null>(null);
const [size, setSize] = useState<Size>(DEFAULT_SIZE);
const [pipWindow, setPipWindow] = useState<Window | null>(null);
const [pinnedId, setPinnedId] = useState<string | null>(null);
const dragRef = useRef<{
kind: "move" | "resize";
corner?: ResizeCorner;
startX: number;
startY: number;
base: Point & Size;
} | null>(null);
// Elapsed-time ticker: 1s while anything is active.
const anyActive = snapshot.sessions.some((s) => s.action !== "idle");
const [now, setNow] = useState(() => Date.now());
useEffect(() => {
if (!anyActive) return;
const timer = setInterval(() => setNow(Date.now()), 1000);
return () => clearInterval(timer);
}, [anyActive]);
// Focus: the pinned session wins while it still exists; otherwise auto-follow.
const pinned =
pinnedId !== null ? snapshot.sessions.find((s) => s.sessionId === pinnedId) : undefined;
const focus = pinned ?? focusOf(snapshot.sessions);
const onTogglePin = useCallback((sessionId: string) => {
setPinnedId((current) => (current === sessionId ? null : sessionId));
}, []);
const viewport = (): Size => ({ w: window.innerWidth, h: window.innerHeight });
const onPointerMove = useCallback((event: PointerEvent) => {
const drag = dragRef.current;
if (drag === null) return;
const dx = event.clientX - drag.startX;
const dy = event.clientY - drag.startY;
if (drag.kind === "move") {
setPos(
clampPos(
{ x: drag.base.x + dx, y: drag.base.y + dy },
{ w: drag.base.w, h: drag.base.h },
viewport(),
),
);
} else if (drag.corner !== undefined) {
const next = applyResize(drag.base, drag.corner, dx, dy, viewport());
setPos(next.pos);
setSize(next.size);
}
}, []);
const onPointerUp = useCallback(() => {
dragRef.current = null;
document.removeEventListener("pointermove", onPointerMove);
document.removeEventListener("pointerup", onPointerUp);
}, [onPointerMove]);
const cardOrigin = (): Point & Size => {
const vp = viewport();
return {
x: pos?.x ?? vp.w - size.w - EDGE_MARGIN,
y: pos?.y ?? TOP_OFFSET,
w: size.w,
h: size.h,
};
};
const beginMove = (event: ReactPointerEvent) => {
event.preventDefault();
const rect = (
event.currentTarget.closest("[data-obs-card]") as HTMLElement | null
)?.getBoundingClientRect();
const origin = cardOrigin();
const base = {
x: rect?.left ?? origin.x,
y: rect?.top ?? origin.y,
w: rect !== undefined && rect.width > 0 ? rect.width : origin.w,
h: rect !== undefined && rect.height > 0 ? rect.height : origin.h,
};
dragRef.current = { kind: "move", startX: event.clientX, startY: event.clientY, base };
setPos({ x: base.x, y: base.y });
document.addEventListener("pointermove", onPointerMove);
document.addEventListener("pointerup", onPointerUp);
};
const beginResize = (corner: ResizeCorner) => (event: ReactPointerEvent) => {
event.preventDefault();
event.stopPropagation();
const rect = (
event.currentTarget.closest("[data-obs-card]") as HTMLElement | null
)?.getBoundingClientRect();
const origin = cardOrigin();
const base = {
x: rect?.left ?? origin.x,
y: rect?.top ?? origin.y,
w: rect !== undefined && rect.width > 0 ? rect.width : origin.w,
h: rect !== undefined && rect.height > 0 ? rect.height : origin.h,
};
dragRef.current = {
kind: "resize",
corner,
startX: event.clientX,
startY: event.clientY,
base,
};
setPos({ x: base.x, y: base.y });
document.addEventListener("pointermove", onPointerMove);
document.addEventListener("pointerup", onPointerUp);
};
const popOut = async (): Promise<void> => {
const pip = pipApi();
if (pip === undefined) return;
try {
const win = await pip.requestWindow({ width: size.w, height: size.h });
cloneStylesInto(win);
win.addEventListener("pagehide", () => setPipWindow(null));
setPipWindow(win);
} catch {
// requestWindow rejects without a user gesture (or when the window was
// closed mid-request): stay on the in-page card, no state change.
}
};
// Hidden while no owned session exists (and no PiP is up).
if (snapshot.sessions.length === 0 && pipWindow === null) return null;
const body = (
<OverlayBody
store={store}
focus={focus}
sessions={snapshot.sessions}
available={snapshot.available}
pinnedId={pinned !== undefined ? pinnedId : null}
onTogglePin={onTogglePin}
now={now}
inPip={pipWindow !== null}
onPopOut={pipWindow === null && pipApi() !== undefined ? () => void popOut() : undefined}
onCollapse={pipWindow === null ? () => setCollapsed(true) : undefined}
onHeaderPointerDown={pipWindow === null ? beginMove : undefined}
/>
);
if (pipWindow !== null) {
return createPortal(body, pipWindow.document.body);
}
if (collapsed) {
const state = focus !== undefined ? statusOf(focus) : "idle";
return (
<button
type="button"
className={cn(css.capsule, "bsk-obs")}
data-state={state}
aria-label="Expand browser observation overlay"
onClick={() => setCollapsed(false)}
>
<StatusDot state={state} />
<span className={css["capsule-text"]}>
{snapshot.sessions.length} session{snapshot.sessions.length === 1 ? "" : "s"}
{focus !== undefined && focus.action !== "idle"
? ` · ${focus.action} · ${formatElapsed(focus.since, now)}`
: ""}
</span>
</button>
);
}
const style: React.CSSProperties =
pos !== null
? { left: pos.x, top: pos.y, width: size.w, height: size.h }
: { right: EDGE_MARGIN, top: TOP_OFFSET, width: size.w, height: size.h };
return (
<div className={cn(css.card, "bsk-obs")} style={style} data-obs-card data-testid="obs-card">
{body}
{(["nw", "ne", "sw", "se"] as const).map((corner) => (
<div
key={corner}
className={css["resize-handle"]}
data-corner={corner}
data-testid={`obs-resize-${corner}`}
aria-label={`Resize overlay from the ${CORNER_LABEL[corner]}`}
role="separator"
aria-valuenow={size.w}
aria-valuetext={`${Math.round(size.w)} by ${Math.round(size.h)} pixels`}
aria-valuemin={MIN_SIZE.w}
aria-valuemax={Math.round(viewport().w * 0.8)}
tabIndex={corner === "se" ? 0 : -1}
onPointerDown={beginResize(corner)}
onKeyDown={
corner === "se"
? (event) => {
const step = 16;
const dx =
event.key === "ArrowRight" ? step : event.key === "ArrowLeft" ? -step : 0;
const dy = event.key === "ArrowDown" ? step : event.key === "ArrowUp" ? -step : 0;
if (dx === 0 && dy === 0) return;
event.preventDefault();
const next = applyResize(cardOrigin(), "se", dx, dy, viewport());
setPos(next.pos);
setSize(next.size);
}
: undefined
}
/>
))}
</div>
);
}
@@ -0,0 +1,57 @@
/* browser_screenshot toolview: disclosure row over a terminal block plus the
screenshot image. Semantic tokens only (--dsw-alias-*), no literal colors. */
.card {
display: flex;
flex-direction: column;
}
.summary {
min-width: 0;
margin-left: 6px;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
color: var(--dsw-alias-label-secondary);
}
.card[data-state="error"] .summary {
color: var(--dsw-alias-label-error);
}
.body {
display: flex;
flex-direction: column;
gap: 8px;
padding: 4px 0 4px 22px;
}
.image-wrap {
display: flex;
}
.inspect-button {
align-self: flex-start;
border: none;
background: none;
padding: 0;
font: inherit;
font-size: 12px;
color: var(--dsw-alias-label-tertiary);
cursor: pointer;
}
.inspect-button:hover {
color: var(--dsw-alias-label-secondary);
}
.inspect-button:focus-visible {
outline: 2px solid var(--dsw-alias-border-focus);
outline-offset: 2px;
border-radius: 2px;
}
@media (prefers-reduced-motion: reduce) {
.inspect-button {
transition: none;
}
}
@@ -0,0 +1,161 @@
// browser_screenshot keyed toolview: the terminal card's command line and
// output stay exactly as the host renders them, and a result carrying an
// image block additionally renders the screenshot itself (thumbnail with the
// shared click-to-open lightbox). Pure function of the frozen call/result
// slice — live and replay paths render identically.
import type { ImageAttachmentRef } from "@deepseek-ai/dsh-attachment";
import {
type ImageLoader,
MessageImage,
type MessageImageLabels,
} from "@deepseek-ai/dsh-client-ui-attachment";
import {
DisclosureRow,
StateDot,
type StateDotState,
TerminalBlock,
type TerminalBlockLabels,
} from "@deepseek-ai/dsh-client-ui-primitives";
import type { ToolCallViewProps } from "@deepseek-ai/dsh-client-ui-tool/client";
import { useState } from "react";
import css from "./ScreenshotToolView.module.css";
/** Resolved loader bound to the owning session at registration time. */
export type ScreenshotImageLoader = (attachment: ImageAttachmentRef) => Promise<string>;
export type ScreenshotToolViewProps = ToolCallViewProps & {
loadImage: ScreenshotImageLoader;
};
type ViewState = "running" | "ok" | "error";
interface ViewModel {
readonly state: ViewState;
readonly command: string;
readonly output: string | null;
readonly image: ImageAttachmentRef | null;
readonly summary: string;
}
const TERMINAL_LABELS: Partial<TerminalBlockLabels> = {
running: "Running",
failed: "Failed",
done: "Done",
copy: "Copy",
copied: "Copied",
noOutput: "No output",
collapseAria: "Collapse output",
collapse: "Collapse",
expandAria: (hidden) => `Expand the remaining ${hidden} output lines`,
expand: (hidden) => `… ${hidden} more lines`,
};
const IMAGE_LABELS: MessageImageLabels = {
image: "screenshot",
open: "Open the original screenshot",
openNamed: (label) => `Open screenshot ${label}`,
loading: "Loading…",
loadFailed: "Load failed — retry",
lightbox: { dialog: "Screenshot preview", close: "Close preview" },
};
function firstLine(text: string): string {
const newline = text.indexOf("\n");
return newline === -1 ? text : text.slice(0, newline);
}
/** Rebuild the command line from the logged arguments (mirrors the host presenter). */
function commandOf(argsRaw: string, callId: string): string {
try {
const args = JSON.parse(argsRaw) as { session?: unknown; ref?: unknown };
const parts = ["bsk", "screenshot", "--session"];
parts.push(
typeof args.session === "string" && args.session !== "" ? args.session : "(current)",
);
if (typeof args.ref === "string" && args.ref !== "") parts.push("--ref", args.ref);
return parts.join(" ");
} catch {
// Streaming can expose a truncated JSON prefix; it is still the best label.
return argsRaw === "" ? `bsk screenshot (${callId})` : `bsk screenshot ${firstLine(argsRaw)}`;
}
}
/** Derive the display model from the frozen block only. */
export function viewModelOf(block: ToolCallViewProps["block"]): ViewModel {
const settled = "kind" in block;
const argsRaw = (settled ? block.call?.argsRaw : block.argsRaw) ?? "";
const command = commandOf(argsRaw, block.callId);
if (!settled) {
return { state: "running", command, output: null, image: null, summary: command };
}
const textBlock = block.content.find((item) => item.type === "text");
const imageBlock = block.content.find((item) => item.type === "image");
const output = textBlock !== undefined && textBlock.type === "text" ? textBlock.text : null;
const image =
imageBlock !== undefined && imageBlock.type === "image" ? imageBlock.attachment : null;
const state: ViewState = block.isError ? "error" : "ok";
const summary = output !== null ? firstLine(output) : command;
return { state, command, output, image, summary };
}
function dotState(state: ViewState): StateDotState {
switch (state) {
case "running":
return "ongoing";
case "error":
return "error";
default:
return "done";
}
}
/**
* Render one browser_screenshot call: a disclosure row over a terminal block
* (command + output) plus the screenshot image when the result carries one.
*/
export function ScreenshotToolView({ block, cwd, inspect, loadImage }: ScreenshotToolViewProps) {
const model = viewModelOf(block);
const [expanded, setExpanded] = useState(false);
const expandable = model.state === "running" || model.output !== null || model.image !== null;
const open = expanded && expandable;
return (
<div className={css.card} data-tool="browser_screenshot" data-state={model.state}>
<DisclosureRow
icon={<StateDot state={dotState(model.state)} />}
title="Screenshot"
open={open}
expandable={expandable}
onToggle={() => setExpanded((value) => !value)}
expandOnRowClick
previewChevron
collapsedContent={<span className={css.summary}>{model.summary}</span>}
>
<div className={css.body}>
<TerminalBlock
command={model.command}
cwd={cwd ?? undefined}
output={model.output ?? undefined}
running={model.state === "running"}
labels={TERMINAL_LABELS}
/>
{model.image !== null ? (
<div className={css["image-wrap"]}>
<MessageImage
attachment={model.image}
load={loadImage}
variant="single"
labels={IMAGE_LABELS}
/>
</div>
) : null}
{inspect !== undefined ? (
<button type="button" className={css["inspect-button"]} onClick={inspect}>
Inspect
</button>
) : null}
</div>
</DisclosureRow>
</div>
);
}
@@ -0,0 +1,49 @@
/*
* BrowserSkill UI design tokens, scoped to the observation overlay root.
* VALUES MIRROR packages/ui/src/styles/tailwind.css (:root and .dark) — keep
* in sync with that file; it is the canonical source. Scoped here (instead of
* :root) so the dsh host theme can neither leak in nor be polluted by us.
* Dark follows prefers-color-scheme, same as the extension popup.
*/
.bsk-obs {
color-scheme: light dark;
--background: oklch(0.99 0.005 50);
--foreground: oklch(0.14 0.03 50);
--card: oklch(0.995 0.003 50 / 0.82);
--card-foreground: oklch(0.14 0.03 50);
--primary: oklch(0.62 0.22 45);
--primary-foreground: oklch(0.99 0.01 50);
--secondary: oklch(0.96 0.04 65);
--secondary-foreground: oklch(0.25 0.08 65);
--muted: oklch(0.96 0.01 50);
--muted-foreground: oklch(0.55 0.04 50);
--accent: oklch(0.96 0.04 65);
--accent-foreground: oklch(0.25 0.08 65);
--destructive: oklch(0.6 0.25 25);
--destructive-foreground: oklch(0.99 0.01 50);
--border: oklch(0.9 0.025 60);
--input: oklch(0.9 0.025 60);
--ring: oklch(0.62 0.22 45);
--radius: 0.75rem;
}
@media (prefers-color-scheme: dark) {
.bsk-obs {
--background: oklch(0.2 0.02 50);
--foreground: oklch(0.98 0.01 50);
--card: oklch(0.24 0.02 50 / 0.82);
--card-foreground: oklch(0.98 0.01 50);
--primary: oklch(0.7 0.18 45);
--primary-foreground: oklch(0.14 0.03 50);
--secondary: oklch(0.32 0.03 60);
--secondary-foreground: oklch(0.98 0.01 50);
--muted: oklch(0.3 0.02 50);
--muted-foreground: oklch(0.75 0.03 50);
--accent: oklch(0.32 0.03 60);
--accent-foreground: oklch(0.98 0.01 50);
--destructive: oklch(0.45 0.2 25);
--destructive-foreground: oklch(0.98 0.01 50);
--border: oklch(0.35 0.02 60);
--input: oklch(0.35 0.02 60);
--ring: oklch(0.7 0.18 45);
}
}
@@ -0,0 +1,84 @@
/**
* dsh-plugin-browserskill browser half: the `browser_screenshot` keyed
* toolview plus the observation overlay (live thumbnails + interrupt) floating
* over the shell via the `shell.overlay` seat.
*/
import type { ImageAttachmentRef } from "@deepseek-ai/dsh-attachment";
import type { ClientContext, ISessions, SessionId } from "@deepseek-ai/dsh-client-runtime/client";
// Type-only: pulls the 'shell.overlay' SlotMap merge into scope.
import type {} from "@deepseek-ai/dsh-client-ui-layout/client";
import type { ToolCallViewProps } from "@deepseek-ai/dsh-client-ui-tool/client";
import { createElement } from "react";
// Scope-prefixed BSK design tokens and utility sheet (injected verbatim as
// <style> tags; selectors stay unhashed so `cn(..., "bsk-obs")` roots match).
import "./bsk-tokens.nomodule.css";
import "./bsk-ui.nomodule.css";
import { ObservationOverlay } from "./ObservationOverlay";
import { type EventSourceLike, ObservationClientStore } from "./observation-store";
import { ScreenshotToolView } from "./ScreenshotToolView";
/** Required services: slots, session-scoped attachment reads, and the overlay seat. */
export const inject = ["slots", "sessions"];
/** Resolve one durable image attachment into a browser blob URL. */
async function loadSessionImage(
sessions: ISessions,
sessionId: SessionId,
attachment: ImageAttachmentRef,
): Promise<string> {
const session = sessions.binding(sessionId)?.session;
if (session === undefined) {
throw new Error(`screenshot toolview: session "${String(sessionId)}" is not bound`);
}
const result = await session.readAttachment(attachment.attachmentId);
if (!result.ok) {
throw new Error(
`screenshot toolview: readAttachment failed: ${result.error.code}: ${result.error.message}`,
);
}
const bytes = Uint8Array.from(result.value.data);
return URL.createObjectURL(
new Blob([bytes.buffer as ArrayBuffer], { type: result.value.attachment.mediaType }),
);
}
/**
* Thumbnail loader for the overlay: frames are plugin-owned runtime data
* (never referenced by a session log, so the session-authorized RPC refuses
* them), served by the host over the plugin's own route.
*/
async function overlayImageLoader(attachmentId: string): Promise<string> {
const res = await fetch(`/bsk-observation/thumbnail/${encodeURIComponent(attachmentId)}`);
if (!res.ok) throw new Error(`thumbnail fetch failed: ${res.status}`);
return URL.createObjectURL(await res.blob());
}
/**
* Client plugin body: register the keyed toolview and the observation overlay.
* @param ctx - client root context.
*/
export function apply(ctx: ClientContext): void {
const sessions = ctx.get("sessions") as unknown as ISessions;
ctx.slots.inject("tool.call.toolview", () =>
ctx.slots.register(
{ name: "tool.call.toolview", key: "browser_screenshot" },
(props: ToolCallViewProps) =>
createElement(ScreenshotToolView, {
...props,
loadImage: (attachment: ImageAttachmentRef) =>
loadSessionImage(sessions, props.sessionId, attachment),
}),
),
);
const store = new ObservationClientStore({
fetchFn: (url, init) => fetch(url, init),
eventSourceFactory: (url) => new EventSource(url) as unknown as EventSourceLike,
loadImage: overlayImageLoader,
});
ctx.slots.inject("shell.overlay", () =>
ctx.slots.register({ name: "shell.overlay", id: "bsk-observation" }, () =>
createElement(ObservationOverlay, { store }),
),
);
}
@@ -0,0 +1,294 @@
/**
* Client-side data layer for the observation overlay: initial state fetch,
* SSE increment stream (with a state refetch on every (re)open — events that
* fired while the stream was down are otherwise lost forever), on-demand
* thumbnail blob loading through the session-authorized readAttachment path,
* and the interrupt call. The last ready blob per session is held until its
* replacement decodes so the overlay can swap frames in place. All I/O is
* injected so tests never touch a network.
*/
import type { ObservationEvent, SessionObservation } from "../observation";
export interface EventSourceLike {
onmessage: ((event: { data: string }) => void) | null;
/** Fires on the initial connect AND on every automatic reconnect. */
onopen?: (() => void) | null;
close(): void;
}
export interface ObservationClientDeps {
fetchFn: (
url: string,
init?: { method?: string; body?: string },
) => Promise<{
ok: boolean;
json(): Promise<unknown>;
}>;
eventSourceFactory: (url: string) => EventSourceLike;
/** Resolve one attachment id to a displayable blob URL. */
loadImage: (attachmentId: string) => Promise<string>;
}
export interface ThumbnailState {
status: "loading" | "ready" | "error";
url?: string;
}
export interface OverlaySnapshot {
readonly sessions: readonly SessionObservation[];
/** Whether the SSE increment stream is currently subscribed. */
readonly subscribed: boolean;
readonly thumbnails: Readonly<Record<string, ThumbnailState>>;
/**
* sessionId → the frame the overlay should paint. While a newer attachment
* is still loading (or failed), this keeps the last good URL so the card
* never drops to a placeholder between breaths.
*/
readonly displayFrames: Readonly<Record<string, ThumbnailState>>;
/** False when the host reports the browser/daemon as unreachable. */
readonly available: boolean;
}
const STATE_URL = "/bsk-observation/state";
const EVENTS_URL = "/bsk-observation/events";
const INTERRUPT_URL = "/bsk-observation/interrupt";
function revoke(url: string | undefined): void {
if (url !== undefined && typeof URL.revokeObjectURL === "function") {
URL.revokeObjectURL(url);
}
}
export class ObservationClientStore {
private sessions = new Map<string, SessionObservation>();
private thumbs = new Map<string, ThumbnailState>();
/** sessionId → the attachment id currently advertised for it. */
private readonly thumbBySession = new Map<string, string>();
/** sessionId → last successfully decoded frame (held across the next load). */
private readonly lastReady = new Map<string, { attachmentId: string; url: string }>();
private listeners = new Set<() => void>();
private events: EventSourceLike | undefined;
private snapshot: OverlaySnapshot = {
sessions: [],
subscribed: false,
thumbnails: {},
displayFrames: {},
available: true,
};
private available = true;
private started = false;
constructor(private readonly deps: ObservationClientDeps) {}
subscribe = (listener: () => void): (() => void) => {
this.listeners.add(listener);
return () => this.listeners.delete(listener);
};
getSnapshot = (): OverlaySnapshot => this.snapshot;
private publish(): void {
this.snapshot = {
sessions: [...this.sessions.values()],
subscribed: this.events !== undefined,
thumbnails: Object.fromEntries(this.thumbs),
displayFrames: this.buildDisplayFrames(),
available: this.available,
};
for (const listener of [...this.listeners]) listener();
}
/** Frame the overlay should paint for one session (last good while next loads). */
private frameFor(sessionId: string): ThumbnailState | undefined {
const attachmentId = this.thumbBySession.get(sessionId);
const current = attachmentId !== undefined ? this.thumbs.get(attachmentId) : undefined;
if (current?.status === "ready" && current.url !== undefined) return current;
const held = this.lastReady.get(sessionId);
if (held !== undefined) {
return {
status: current?.status === "error" ? "error" : "ready",
url: held.url,
};
}
return current;
}
private buildDisplayFrames(): Record<string, ThumbnailState> {
const frames: Record<string, ThumbnailState> = {};
for (const sessionId of this.sessions.keys()) {
const frame = this.frameFor(sessionId);
if (frame !== undefined) frames[sessionId] = frame;
}
return frames;
}
private sessionOf(attachmentId: string): string | undefined {
for (const [sessionId, id] of this.thumbBySession) {
if (id === attachmentId) return sessionId;
}
return undefined;
}
/** (Re)pull the full state: initial load and every SSE (re)open. */
private refreshState(): void {
void this.deps
.fetchFn(STATE_URL)
.then(async (res) => {
if (!res.ok) return;
const body = (await res.json()) as {
sessions?: SessionObservation[];
available?: boolean;
};
const sessions = body.sessions ?? [];
this.sessions = new Map(sessions.map((s) => [s.sessionId, s]));
// Prune thumbnails of sessions that vanished while we were away.
const alive = new Set(sessions.map((s) => s.sessionId));
for (const sessionId of [...this.thumbBySession.keys()]) {
if (!alive.has(sessionId)) this.dropThumb(sessionId);
}
for (const s of sessions) this.trackThumb(s.sessionId, s.thumbnailAttachmentId);
if (typeof body.available === "boolean") this.available = body.available;
this.publish();
})
.catch(() => {});
}
/** Initial fetch + SSE subscription. Idempotent. */
start(): void {
if (this.started) return;
this.started = true;
this.refreshState();
const events = this.deps.eventSourceFactory(EVENTS_URL);
events.onmessage = (message) => {
let event: ObservationEvent;
try {
event = JSON.parse(message.data) as ObservationEvent;
} catch {
return;
}
this.apply(event);
};
events.onopen = () => {
// Reconnects lose every event fired during the outage — resync.
if (this.events === events) this.refreshState();
};
this.events = events;
this.publish();
}
stop(): void {
this.events?.close();
this.events = undefined;
this.started = false;
for (const thumb of this.thumbs.values()) revoke(thumb.url);
this.thumbs.clear();
this.thumbBySession.clear();
this.lastReady.clear();
this.sessions.clear();
this.publish();
}
/** Forget one session's tracked + held frames, revoking their blob URLs. */
private dropThumb(sessionId: string): void {
const attachmentId = this.thumbBySession.get(sessionId);
const held = this.lastReady.get(sessionId);
this.thumbBySession.delete(sessionId);
this.lastReady.delete(sessionId);
if (attachmentId !== undefined) {
revoke(this.thumbs.get(attachmentId)?.url);
this.thumbs.delete(attachmentId);
}
if (held !== undefined && held.attachmentId !== attachmentId) {
revoke(held.url);
this.thumbs.delete(held.attachmentId);
}
}
/**
* Track the frame a session currently advertises. The last *ready* blob is
* kept until the replacement decodes — dropping it on the upsert is what
* made the overlay flash a placeholder between breaths.
*/
private trackThumb(sessionId: string, attachmentId: string | undefined): void {
const previous = this.thumbBySession.get(sessionId);
if (previous === attachmentId) return;
if (attachmentId !== undefined) this.thumbBySession.set(sessionId, attachmentId);
else this.thumbBySession.delete(sessionId);
const heldId = this.lastReady.get(sessionId)?.attachmentId;
if (previous !== undefined && previous !== heldId) {
revoke(this.thumbs.get(previous)?.url);
this.thumbs.delete(previous);
}
}
private apply(event: ObservationEvent): void {
if (event.type === "reset") {
this.sessions.clear();
for (const thumb of this.thumbs.values()) revoke(thumb.url);
this.thumbs.clear();
this.thumbBySession.clear();
this.lastReady.clear();
} else if (event.type === "remove" && event.session !== undefined) {
this.sessions.delete(event.session.sessionId);
this.dropThumb(event.session.sessionId);
} else if (event.type === "upsert" && event.session !== undefined) {
this.sessions.set(event.session.sessionId, event.session);
this.trackThumb(event.session.sessionId, event.session.thumbnailAttachmentId);
} else if (event.type === "availability") {
this.available = event.available;
}
this.publish();
}
/**
* Ensure a thumbnail load is in flight for one attachment reference. New
* frames replace the old URL only after they decode; failures keep the last
* good frame (the caller renders `status: 'error'` as a small badge over
* the old image).
*/
ensureThumbnail(attachmentId: string | undefined): void {
if (attachmentId === undefined || this.thumbs.has(attachmentId)) return;
this.thumbs.set(attachmentId, { status: "loading" });
this.publish();
this.deps.loadImage(attachmentId).then(
(url) => {
if (!this.thumbs.has(attachmentId)) {
// Replaced or removed while loading: never resurrect (or leak) it.
revoke(url);
return;
}
this.thumbs.set(attachmentId, { status: "ready", url });
const sessionId = this.sessionOf(attachmentId);
if (sessionId !== undefined) {
const previous = this.lastReady.get(sessionId);
if (previous !== undefined && previous.attachmentId !== attachmentId) {
revoke(previous.url);
this.thumbs.delete(previous.attachmentId);
}
this.lastReady.set(sessionId, { attachmentId, url });
}
this.publish();
},
() => {
this.thumbs.set(attachmentId, { status: "error" });
this.publish();
},
);
}
/** Interrupt the current or named session's in-flight call. */
async interrupt(sessionId?: string): Promise<boolean> {
try {
const res = await this.deps.fetchFn(INTERRUPT_URL, {
method: "POST",
body: JSON.stringify(sessionId === undefined ? {} : { sessionId }),
});
if (!res.ok) return false;
const body = (await res.json()) as { interrupted?: boolean };
return body.interrupted === true;
} catch {
return false;
}
}
}
+6
View File
@@ -0,0 +1,6 @@
declare module "*.module.css" {
const classes: Record<string, string>;
export default classes;
}
declare module "*.css";
@@ -0,0 +1,83 @@
/**
* Screenshot delivery. The canonical tool value stays plain JSON (a PNG file
* path plus pixel metadata); when the host mounts a durable attachment store
* AND the calling route declares image input, the PNG bytes are additionally
* committed through `ctx.attachments` so the render step can attach the image
* itself to the tool result. Any uncertainty (no store, unknown route,
* text-only model) falls back to the path-only form.
*/
import type { Context } from "@deepseek-ai/cordis";
import type { ImageAttachmentRef } from "@deepseek-ai/dsh-attachment";
import type { ToolExecution } from "@deepseek-ai/dsh-tools";
/** Structural view of the attachment seam (avoids a runtime dependency). */
interface AttachmentLike {
imageLimits: {
mediaTypes: readonly string[];
maxImageBytes: number;
maxMessageImageBytes: number;
};
saveImage(input: {
data: Uint8Array;
mediaType: string;
name?: string;
}): Promise<ImageAttachmentRef>;
}
interface LlmLike {
resolveModelInfo(
provider: string,
model: string,
signal?: AbortSignal,
): Promise<{ inputModalities?: readonly string[] }>;
}
/**
* Commit screenshot bytes to the host attachment store when the composition
* supports durable images on the current model route.
* @returns the durable reference, or undefined to stay in path-only mode.
*/
export async function trySaveScreenshot(
ctx: Context,
exec: ToolExecution,
data: Uint8Array,
name: string,
): Promise<ImageAttachmentRef | undefined> {
const attachments = ctx.get("attachments") as AttachmentLike | undefined;
if (attachments === undefined) return undefined;
if (!attachments.imageLimits.mediaTypes.includes("image/png")) return undefined;
if (
data.byteLength >
Math.min(attachments.imageLimits.maxImageBytes, attachments.imageLimits.maxMessageImageBytes)
) {
return undefined;
}
if (!(await isImageCapableRoute(ctx, exec))) return undefined;
try {
return await attachments.saveImage({ data, mediaType: "image/png", name });
} catch {
// A store hiccup must not fail the tool call; the PNG path still works.
return undefined;
}
}
/**
* Best-effort check that the calling route's model accepts image input,
* mirroring dsh-tool-fs `read_image`. Unknown route / missing llm service
* answers false (refuse to attach) so history never gains an image block a
* text-only adapter cannot replay.
*/
async function isImageCapableRoute(ctx: Context, exec: ToolExecution): Promise<boolean> {
const llm = ctx.get("llm") as LlmLike | undefined;
const provider =
exec.agent?.session.requestHeader()?.config.provider ?? exec.agent?.options.provider;
const model = exec.agent?.session.requestHeader()?.config.model ?? exec.agent?.options.model;
if (llm === undefined || provider === undefined || model === undefined) return false;
try {
const info = await llm.resolveModelInfo(provider, model, exec.signal);
return info.inputModalities?.includes("image") ?? false;
} catch {
return false;
}
}
@@ -0,0 +1,155 @@
/**
* dsh-plugin-browserskill — a DeepSeek Harness bundle that registers
* BrowserSkill (`bsk`) browser automation as model-visible tools.
*
* Each tool spawns `bsk <cmd> --json`, parses the structured output, and
* returns a canonical JSON value. The plugin tracks the sessions it starts so
* one agent conversation can drive several browsers at once; unloading the
* plugin stops every tracked session and kills any in-flight bsk children.
*
* @module dsh-plugin-browserskill
*/
import type { Context } from "@deepseek-ai/cordis";
import Schema from "@deepseek-ai/schemastery";
import { armLazyTools } from "./lazy-tools";
import { ObservationService } from "./observation";
import { registerObservationRoutes } from "./observation-http";
import { KeyedExecutor } from "./queue";
import { type BskRunner, createBskRunner } from "./runner";
import { SessionRegistry } from "./sessions";
import { registerBskSkill } from "./skill";
import { type PluginConfig, registerTools } from "./tools";
export const name = "dsh-plugin-browserskill";
export const inject = ["tools"];
/** Runtime configuration schema (validated and defaulted by Cordis). */
export const Config = Schema.object({
bskPath: Schema.string()
.default("bsk")
.description("Path to the bsk CLI binary (defaults to resolving `bsk` from PATH)."),
defaultTimeoutMs: Schema.number()
.default(120_000)
.description("Default per-command timeout in milliseconds."),
maxSessions: Schema.number()
.default(5)
.description("Maximum number of concurrent browser sessions started through this plugin."),
observationEnabled: Schema.boolean()
.default(true)
.description("Track per-session observation state (action/url/thumbnail) for the PiP overlay."),
thumbnailIntervalMs: Schema.number()
.default(1500)
.description("Thumbnail refresh cadence for active sessions (milliseconds)."),
idleIntervalMs: Schema.number()
.default(8000)
.description("Thumbnail refresh cadence for idle sessions; also the recent-activity window."),
lazyTools: Schema.boolean()
.default(true)
.description(
"Reveal the browser_* tools only after the browser-skill skill is invoked (default true); " +
"false registers the full suite at load.",
),
});
export type Config = PluginConfig;
/** Test seams: swap the process runner (unit tests never spawn a real bsk). */
export interface ApplyOptions {
runnerFactory?: (bskPath: string) => BskRunner;
}
export function apply(
ctx: Context,
config: Partial<PluginConfig> = {},
options: ApplyOptions = {},
): void {
const resolved: PluginConfig = {
bskPath: config.bskPath ?? "bsk",
defaultTimeoutMs: config.defaultTimeoutMs ?? 120_000,
maxSessions: config.maxSessions ?? 5,
observationEnabled: config.observationEnabled ?? true,
thumbnailIntervalMs: config.thumbnailIntervalMs ?? 1500,
idleIntervalMs: config.idleIntervalMs ?? 8000,
lazyTools: config.lazyTools ?? true,
};
const runner = options.runnerFactory?.(resolved.bskPath) ?? createBskRunner(resolved.bskPath);
const registry = new SessionRegistry(resolved.maxSessions);
const queue = new KeyedExecutor();
const observation = new ObservationService({
ctx,
runner,
registry,
queue,
options: {
enabled: resolved.observationEnabled,
thumbnailIntervalMs: resolved.thumbnailIntervalMs,
idleIntervalMs: resolved.idleIntervalMs,
},
});
// Progressive disclosure of the BSK agent skill (catalog entry resident,
// body on demand) through the official skill seam; silent no-op when the
// composition lacks it. With lazyTools on, the skill entry is initially the
// ONLY model-visible advertisement — the tool suite reveals itself on a
// successful skill invocation (or a session whose history already has one).
const unregisterSkill = registerBskSkill(ctx);
const removeSuite = resolved.lazyTools
? armLazyTools(ctx, () =>
registerTools({ ctx, runner, registry, config: resolved, observation, queue }),
)
: registerTools({ ctx, runner, registry, config: resolved, observation, queue });
// Route registration rides ctx.inject: the webServer service may be provided
// AFTER this plugin loads, and in headless compositions it never appears (the
// callback simply never runs, leaving the rest of the plugin unaffected).
let removeRoutes: () => void = () => {};
ctx.inject(["webServer"], (injected) => {
removeRoutes = registerObservationRoutes(injected, observation);
return () => removeRoutes();
});
// Non-blocking install probe: warn early when bsk is missing instead of
// failing the first tool call with a bare spawn error. Uses --version on
// purpose — it answers without starting the daemon (`bsk status` would
// ensure-spawn one, an expensive side effect for a probe).
runner.run(["--version"], { timeoutMs: 10_000 }).then(
() => {},
(error: unknown) => {
const detail = error instanceof Error ? error.message : String(error);
console.warn(
`[${name}] bsk probe failed (${detail}); browser tools will report install guidance until the bsk CLI is available`,
);
},
);
// Unload cleanup: kill in-flight children, then stop every session this
// plugin OWNS (created via browser_session_start). Referenced or unknown
// sessions belonging to other programs on the shared daemon are never
// touched; per-stop failures (already stopped externally, daemon restart)
// are swallowed so one stale handle cannot abort the rest.
ctx.effect(() => {
return () => {
removeSuite();
unregisterSkill();
removeRoutes();
observation.dispose();
runner.killAll();
const stops = registry
.ownedIds()
.map((sessionId) =>
runner.run(["session", "stop", sessionId], { timeoutMs: 15_000 }).catch(() => {}),
);
return Promise.all(stops).then(() => {});
};
});
}
export type { ObservationEvent, ObservationOptions, SessionObservation } from "./observation";
export { ObservationService } from "./observation";
export { registerObservationRoutes } from "./observation-http";
export { KeyedExecutor } from "./queue";
export type { BskRunner, BskRunResult, SpawnImpl } from "./runner";
export { BskError, createBskRunner } from "./runner";
export { SessionRegistry } from "./sessions";
export type { PluginConfig, ToolDeps } from "./tools";
export { registerTools } from "./tools";
@@ -0,0 +1,153 @@
/**
* Lazy tool-schema injection ("progressive disclosure", final stage): with
* `lazyTools` on (the default), the 11 browser_* tool schemas stay OUT of the
* system prompt until the `browser-skill` skill has actually been invoked —
* the skill catalog entry is the only advertisement. One successful
* invocation (model tool call, or a user's `/browser-skill` gesture) reveals
* the whole suite for the rest of the process lifetime; repeated invocations
* are idempotent no-ops. Session resume is covered by scanning durable
* session events for a past successful invocation (tool/call + tool/result
* pair, or a skill-invocation sourced message) when a session is entered.
*
* Verified against dsh 0.1 (recorded in the PR ticket):
* - `tools/result(exec, result)`: exec carries normalized `name`/`arguments`,
* result is discriminated by `isError` — the model-invocation trigger.
* - There is NO official global tool-visibility switch (`ctx.tools.restrict`
* is agent-scoped and throws from a plain host context), so conditional
* registration + this hook is the intended pattern; the registry supports
* mid-flight register/dispose with an unfiltered `tools/change` notice, and
* tool-skill's per-step catalog digest treats visibility changes as a
* first-class cache-invalidation input — the suite simply appears in the
* NEXT step's assembly.
* - Durable events: `tool/call` {callId, name, arguments(JSON string)} pairs
* with `tool/result` {message: {callId, isError}} — the history signal.
*/
import type { Context } from "@deepseek-ai/cordis";
import { BSK_SKILL_NAME } from "./skill-content.generated";
/** Structural view of the pieces of the session seam we consume. */
interface SessionLike {
events: readonly SessionEventLike[];
}
interface SessionEventLike {
type: string;
data?: unknown;
}
interface SessionsLike {
list(): SessionLike[];
}
interface ToolResultExecutionLike {
name: string;
arguments?: unknown;
}
/** Parse a tool arguments payload that may be normalized (object) or raw JSON. */
function skillNameOf(args: unknown): string | undefined {
if (typeof args === "string") {
try {
return skillNameOf(JSON.parse(args));
} catch {
return undefined;
}
}
if (typeof args === "object" && args !== null && "name" in args) {
const name = (args as { name?: unknown }).name;
return typeof name === "string" ? name : undefined;
}
return undefined;
}
function isSkillInvocationMessage(data: unknown): boolean {
if (typeof data !== "object" || data === null) return false;
const source = (data as { source?: unknown }).source;
if (typeof source !== "object" || source === null) return false;
const { kind, name } = source as { kind?: unknown; name?: unknown };
return kind === "skill-invocation" && name === BSK_SKILL_NAME;
}
/**
* A durable log proves the skill was successfully invoked when a successful
* `tool/result` pairs a `tool/call` for skill/browser-skill — or when a
* `/browser-skill` user gesture landed as a skill-invocation message.
*/
export function hasSuccessfulSkillInvocation(events: readonly SessionEventLike[]): boolean {
const skillCallIds = new Set<unknown>();
for (const event of events) {
if (event.type !== "tool/call") continue;
const data = event.data as { callId?: unknown; name?: unknown; arguments?: unknown };
if (data?.name === "skill" && skillNameOf(data.arguments) === BSK_SKILL_NAME) {
skillCallIds.add(data.callId);
}
}
for (const event of events) {
if (event.type === "tool/result") {
const message = (event.data as { message?: unknown })?.message;
if (typeof message !== "object" || message === null) continue;
const { callId, isError } = message as { callId?: unknown; isError?: unknown };
if (isError === false && skillCallIds.has(callId)) return true;
}
if (isSkillInvocationMessage(event.data)) return true;
}
return false;
}
/**
* Arm the lazy reveal. Returns a disposer tearing down listeners and — when
* the reveal already happened — the tool suite itself.
* @param registerSuite - registers the 11 browser_* tools, returns their disposer.
*/
export function armLazyTools(ctx: Context, registerSuite: () => () => void): () => void {
let suiteDisposer: (() => void) | undefined;
const disposers: (() => void)[] = [];
const ensureSuite = (): void => {
if (suiteDisposer !== undefined) return;
try {
suiteDisposer = registerSuite();
} catch (error) {
// A failed reveal must not strand the plugin: stay hidden, log, retry on
// the next trigger instead of latching a half-registered suite.
suiteDisposer = undefined;
console.warn(
`[dsh-plugin-browserskill] lazy tool registration failed: ${error instanceof Error ? error.message : String(error)}`,
);
}
};
// Live trigger: a successful model invocation of skill/browser-skill.
const onToolResult = (exec: ToolResultExecutionLike, result: { isError: boolean }): void => {
if (result.isError) return;
if (exec.name !== "skill") return;
if (skillNameOf(exec.arguments) === BSK_SKILL_NAME) ensureSuite();
};
disposers.push(ctx.on("tools/result" as never, onToolResult as never));
const scanSession = (session: SessionLike): void => {
try {
if (hasSuccessfulSkillInvocation(session.events)) ensureSuite();
} catch {
// A session object that cannot be read must never break plugin startup.
}
};
// Live gesture/append feed: covers /browser-skill user gestures (no tool
// call happens on that path) landing as skill-invocation messages.
const onSessionEvent = (_session: SessionLike, event: SessionEventLike): void => {
if (isSkillInvocationMessage(event?.data)) ensureSuite();
};
disposers.push(ctx.on("session/event" as never, onSessionEvent as never));
// History restore: sessions entered from now on, plus any already live.
const onSessionCreated = (session: SessionLike): void => scanSession(session);
disposers.push(ctx.on("session/created" as never, onSessionCreated as never));
const sessions = ctx.get("sessions") as SessionsLike | null | undefined;
if (sessions != null && typeof sessions.list === "function") {
for (const session of sessions.list()) scanSession(session);
}
return () => {
for (const dispose of disposers.splice(0)) dispose();
suiteDisposer?.();
};
}
@@ -0,0 +1,196 @@
/**
* HTTP/SSE exposure of the ObservationService for the client half. The Typert
* Remote pipeline and the forwarded-event allowlist are both closed to
* out-of-tree packages in dsh 0.1 (generation runs over dsh's own ts.Program;
* the event allowlist lives in dsh-api-remotes), so the plugin serves its
* observation channel through the documented `webServer` route seam instead:
*
* GET /bsk-observation/state → { sessions, available }
* GET /bsk-observation/events → SSE stream of ObservationEvent
* POST /bsk-observation/interrupt → body {sessionId?} → {interrupted: boolean}
* GET /bsk-observation/thumbnail/<attachmentId> → image bytes
*
* Routes exist only when a webServer service is mounted (web composition);
* headless deployments skip registration entirely.
*/
import type { IncomingMessage, ServerResponse } from "node:http";
import type { Context } from "@deepseek-ai/cordis";
import type { ObservationService } from "./observation";
/** Structural view of the dsh-host-webserver route seam. */
interface WebServerLike {
register(route: {
kind: "exact" | "prefix";
path: string;
handler: (req: IncomingMessage, res: ServerResponse) => void | Promise<void>;
}): () => void;
}
const ROUTE_BASE = "/bsk-observation";
const SSE_HEARTBEAT_MS = 15_000;
function sendJson(res: ServerResponse, status: number, body: unknown): void {
res.writeHead(status, { "content-type": "application/json; charset=utf-8" });
res.end(JSON.stringify(body));
}
/**
* Browser-trust fence, mirroring the one dsh applies to its /api routes (ours
* live outside that prefix, so the checks are replicated here):
* - Host must be a loopback authority (localhost / 127.0.0.0/8 / [::1]) — the
* observation channel exposes live screenshots and must never answer a LAN
* or DNS-rebound name;
* - a present Origin must be same-host with the Host header (blocks cross-site
* reads), and `sec-fetch-site: cross-site` is refused outright;
* - POST must be `application/json` — anything a cross-site *simple request*
* can send (form/plain) never reaches the handler, which kills CSRF.
*/
function fenceViolation(req: IncomingMessage): string | undefined {
const host = req.headers.host ?? "";
const hostname = /^\[.*\](?::\d+)?$/.test(host)
? host.slice(1, host.indexOf("]"))
: host.split(":")[0];
const isLoopback =
hostname === "localhost" ||
hostname.endsWith(".localhost") ||
hostname === "::1" ||
/^127\.\d{1,3}\.\d{1,3}\.\d{1,3}$/.test(hostname);
if (!isLoopback) return "host is not a loopback authority";
const origin = req.headers.origin;
if (origin !== undefined && origin !== "null") {
let originHost: string | undefined;
try {
originHost = new URL(origin).host;
} catch {
return "unparseable Origin header";
}
if (originHost !== host) return "Origin does not match Host";
}
if (req.headers["sec-fetch-site"] === "cross-site") return "sec-fetch-site: cross-site";
if (req.method === "POST") {
const contentType = req.headers["content-type"] ?? "";
if (!/^\s*application\/json\s*(;|$)/.test(contentType)) {
return "POST requires an application/json body";
}
}
return undefined;
}
/** Run the fence; returns true when the request was rejected (handled). */
function fenceRejected(req: IncomingMessage, res: ServerResponse): boolean {
const violation = fenceViolation(req);
if (violation === undefined) return false;
sendJson(res, 403, { error: `forbidden: ${violation}` });
return true;
}
/**
* Register the observation routes. No-op (with a console note) when the
* composition has no web server.
* @returns disposer removing the routes.
*/
export function registerObservationRoutes(
ctx: Context,
observation: ObservationService,
): () => void {
const webServer = ctx.get("webServer") as WebServerLike | undefined;
if (webServer === undefined) {
return () => {};
}
const disposers = [
webServer.register({
kind: "exact",
path: `${ROUTE_BASE}/state`,
handler: (req, res) => {
if (req.method !== "GET") {
sendJson(res, 405, { error: "method not allowed" });
return;
}
if (fenceRejected(req, res)) return;
sendJson(res, 200, {
sessions: observation.getState(),
available: observation.isAvailable(),
});
},
}),
webServer.register({
kind: "exact",
path: `${ROUTE_BASE}/events`,
handler: (req, res) => {
if (req.method !== "GET") {
sendJson(res, 405, { error: "method not allowed" });
return;
}
if (fenceRejected(req, res)) return;
res.writeHead(200, {
"content-type": "text/event-stream",
"cache-control": "no-cache",
connection: "keep-alive",
});
const unsubscribe = observation.subscribe((event) => {
res.write(`data: ${JSON.stringify(event)}\n\n`);
});
const heartbeat = setInterval(() => res.write(": heartbeat\n\n"), SSE_HEARTBEAT_MS);
res.on("close", () => {
clearInterval(heartbeat);
unsubscribe();
});
},
}),
webServer.register({
kind: "exact",
path: `${ROUTE_BASE}/interrupt`,
handler: (req, res) => {
if (req.method !== "POST") {
sendJson(res, 405, { error: "method not allowed" });
return;
}
if (fenceRejected(req, res)) return;
let body = "";
req.on("data", (chunk: Buffer | string) => {
body += chunk;
});
req.on("end", () => {
let sessionId: string | undefined;
try {
const parsed = JSON.parse(body || "{}") as { sessionId?: unknown };
if (typeof parsed.sessionId === "string" && parsed.sessionId !== "") {
sessionId = parsed.sessionId;
}
} catch {
sendJson(res, 400, { error: "invalid JSON body" });
return;
}
sendJson(res, 200, { interrupted: observation.interrupt(sessionId) });
});
},
}),
webServer.register({
kind: "prefix",
path: `${ROUTE_BASE}/thumbnail`,
handler: async (req, res) => {
if (req.method !== "GET") {
sendJson(res, 405, { error: "method not allowed" });
return;
}
if (fenceRejected(req, res)) return;
const pathname = decodeURIComponent(new URL(req.url ?? "/", "http://x").pathname);
const attachmentId = pathname.slice(`${ROUTE_BASE}/thumbnail/`.length);
const frame = await observation.readThumbnail(attachmentId);
if (frame === undefined) {
sendJson(res, 404, { error: "unknown thumbnail" });
return;
}
res.writeHead(200, {
"content-type": frame.mediaType,
"cache-control": "no-cache",
});
res.end(Buffer.from(frame.data));
},
}),
];
return () => {
for (const dispose of disposers) dispose();
};
}
@@ -0,0 +1,421 @@
/**
* ObservationService: per-owned-session live observation state for the PiP
* overlay — current action, page url, and a breathing thumbnail carried by
* attachment reference. Strict ownership boundary applies throughout: only
* sessions in the plugin's SessionRegistry (owned by construction) ever get
* an observation entry, and interrupt/kill paths can only reach children this
* plugin spawned.
*
* Observation traffic isolation: thumbnail captures run through the runner
* directly (never through the tool-level instrumentation), emit no action
* events, and never move the registry's current pointer.
*/
import { readFile, unlink } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import type { Context } from "@deepseek-ai/cordis";
import type { KeyedExecutor } from "./queue";
import type { BskRunner } from "./runner";
import type { SessionRegistry } from "./sessions";
/** One owned session's live observation record (wire-stable shape). */
export interface SessionObservation {
sessionId: string;
/** Last settled page URL, when any navigation completed. */
url?: string;
/** Current action ('idle' when nothing is in flight). */
action: string;
/** Epoch ms when the current action started (elapsed time is client-side). */
since: number;
/** Latest thumbnail, by attachment-store reference (never bytes on the wire). */
thumbnailAttachmentId?: string;
/** Summary of the most recent failed action (drives the red status dot). */
lastError?: string;
/**
* The daemon reports this session as gone (e.g. daemon restart): the strip
* greys it out until it is stopped/removed. No more frames are requested.
*/
dead?: boolean;
}
/** Incremental event carried to subscribers (SSE on the wire). */
export type ObservationEvent =
| { type: "upsert" | "remove" | "reset"; session?: SessionObservation }
| { type: "availability"; available: boolean };
export interface ObservationOptions {
enabled: boolean;
/** Fast cadence while a session is active or recently was. */
thumbnailIntervalMs: number;
/** Slow cadence for idle sessions; also the "recently active" window. */
idleIntervalMs: number;
}
/** Structural view of the optional attachment seam (absent in headless compositions). */
interface AttachmentLike {
saveImage(input: {
data: Uint8Array;
mediaType: string;
name?: string;
}): Promise<ImageAttachmentRefLike>;
readImage(
ref: ImageAttachmentRefLike,
): Promise<{ data: Uint8Array; attachment: ImageAttachmentRefLike }>;
}
/** The attachment reference fields the store verifies reads against. */
export interface ImageAttachmentRefLike {
attachmentId: unknown;
mediaType: string;
bytes: number;
width: number;
height: number;
name?: string;
}
/** Injectable clock/scheduler bits for tests. */
export interface ObservationScheduler {
setTimeout: (fn: () => void, ms: number) => unknown;
clearTimeout: (handle: unknown) => void;
now: () => number;
}
const DEFAULT_SCHEDULER: ObservationScheduler = {
setTimeout: (fn, ms) => {
const timer = setTimeout(fn, ms);
// Never hold the event loop open for a frame refresh.
if (typeof timer === "object" && typeof timer.unref === "function") timer.unref();
return timer;
},
clearTimeout: (h) => clearTimeout(h as Parameters<typeof clearTimeout>[0]),
now: () => Date.now(),
};
/** Max consecutive capture failures before a session drops to the idle cadence. */
const FAILURE_BACKOFF_THRESHOLD = 3;
export class ObservationService {
private readonly observations = new Map<string, SessionObservation>();
private readonly listeners = new Set<(event: ObservationEvent) => void>();
private readonly captureTimers = new Map<string, unknown>();
private readonly captureInFlight = new Set<string>();
private readonly captureFailures = new Map<string, number>();
private readonly lastActivity = new Map<string, number>();
/** attachmentId → full reference (the store requires it for verified reads). */
private readonly thumbRefs = new Map<string, ImageAttachmentRefLike>();
/** Consecutive capture failures across ALL sessions (daemon-level signal). */
private globalFailures = 0;
private available = true;
private disposed = false;
constructor(
private readonly deps: {
ctx: Context;
runner: BskRunner;
registry: SessionRegistry;
queue: KeyedExecutor;
options: ObservationOptions;
scheduler?: ObservationScheduler;
},
) {}
private get scheduler(): ObservationScheduler {
return this.deps.scheduler ?? DEFAULT_SCHEDULER;
}
/** All current entries (client initial/resync snapshot). */
getState(): SessionObservation[] {
return [...this.observations.values()].map((entry) => ({ ...entry }));
}
/** Whether the browser side looks reachable (drives the "browser unavailable" state). */
isAvailable(): boolean {
return this.available;
}
private setAvailable(available: boolean): void {
if (this.available === available) return;
this.available = available;
this.emit({ type: "availability", available });
}
/** Subscribe to incremental changes; returns an unsubscribe function. */
subscribe(listener: (event: ObservationEvent) => void): () => void {
this.listeners.add(listener);
return () => this.listeners.delete(listener);
}
private emit(event: ObservationEvent): void {
for (const listener of [...this.listeners]) {
try {
listener(event);
} catch {
// A faulty subscriber must not starve the others.
}
}
}
private put(entry: SessionObservation): void {
this.observations.set(entry.sessionId, entry);
this.emit({ type: "upsert", session: { ...entry } });
}
/** Register a fresh owned session (called from browser_session_start). */
addSession(sessionId: string, url?: string): void {
if (!this.deps.options.enabled || this.disposed) return;
this.put({
sessionId,
...(url !== undefined ? { url } : {}),
action: "idle",
since: this.scheduler.now(),
});
this.lastActivity.set(sessionId, this.scheduler.now());
this.scheduleCapture(sessionId, 0);
}
/** Drop a session (called from browser_session_stop). */
removeSession(sessionId: string): void {
this.cancelCapture(sessionId);
this.captureFailures.delete(sessionId);
this.lastActivity.delete(sessionId);
const entry = this.observations.get(sessionId);
if (entry?.thumbnailAttachmentId !== undefined) {
this.thumbRefs.delete(entry.thumbnailAttachmentId);
}
if (this.observations.delete(sessionId)) {
this.emit({
type: "remove",
session: { sessionId, action: "idle", since: this.scheduler.now() },
});
}
}
/** Mark an action starting on a session (tool entry instrumentation). */
beginAction(sessionId: string, action: string): void {
if (!this.deps.options.enabled || this.disposed) return;
const entry = this.observations.get(sessionId);
if (entry === undefined || entry.dead === true) return;
const now = this.scheduler.now();
this.lastActivity.set(sessionId, now);
// A fresh action clears the previous error marker (red dot means "last action failed").
const next: SessionObservation = { ...entry, action, since: now };
delete next.lastError;
this.put(next);
}
/**
* Mark the current action settling (tool exit instrumentation). Triggers an
* immediate thumbnail refresh — action-driven first, timer as fallback.
*/
endAction(sessionId: string, error?: string): void {
if (!this.deps.options.enabled || this.disposed) return;
const entry = this.observations.get(sessionId);
if (entry === undefined || entry.dead === true) return;
const now = this.scheduler.now();
this.lastActivity.set(sessionId, now);
const next: SessionObservation = { ...entry, action: "idle", since: now };
if (error !== undefined) next.lastError = error;
else delete next.lastError;
this.put(next);
this.scheduleCapture(sessionId, 0);
}
/** Record the settled page URL (navigate success / start with url). */
setUrl(sessionId: string, url: string): void {
if (!this.deps.options.enabled || this.disposed) return;
const entry = this.observations.get(sessionId);
if (entry === undefined) return;
this.put({ ...entry, url });
}
/**
* Interrupt the in-flight call of one session (default: the registry's
* current). Kills exactly the bsk children this plugin spawned for that
* session — same user-visible semantics as the chat Stop button (the in-flight
* tool call fails; the agent flow may continue).
* @returns whether an in-flight call was actually interrupted.
*/
interrupt(sessionId?: string): boolean {
const target = sessionId ?? this.deps.registry.current();
if (target === undefined || !this.deps.registry.isOwned(target)) return false;
return this.deps.runner.killFor(target) > 0;
}
/**
* Read one captured thumbnail back through the store's verified path.
* Powers the plugin's own HTTP thumbnail route — frames are plugin-owned
* runtime data, never referenced by any session log, so the
* session-authorized client RPC cannot serve them.
*/
async readThumbnail(
attachmentId: string,
): Promise<{ data: Uint8Array; mediaType: string } | undefined> {
const ref = this.thumbRefs.get(attachmentId);
if (ref === undefined) return undefined;
const attachments = this.deps.ctx.get("attachments") as AttachmentLike | undefined;
if (attachments === undefined) return undefined;
try {
const stored = await attachments.readImage(ref);
return { data: stored.data, mediaType: ref.mediaType };
} catch {
return undefined;
}
}
/** Tear down all state and timers (plugin dispose). */
dispose(): void {
this.disposed = true;
for (const sessionId of [...this.captureTimers.keys()]) this.cancelCapture(sessionId);
this.captureTimers.clear();
this.captureInFlight.clear();
this.captureFailures.clear();
this.lastActivity.clear();
this.thumbRefs.clear();
this.observations.clear();
this.emit({ type: "reset" });
this.listeners.clear();
}
// ------------------------------------------------------------------
// Thumbnail loop
// ------------------------------------------------------------------
private cancelCapture(sessionId: string): void {
const timer = this.captureTimers.get(sessionId);
if (timer !== undefined) {
this.scheduler.clearTimeout(timer);
this.captureTimers.delete(sessionId);
}
}
/** Schedule the next capture for a session; `delayMs` 0 means "as soon as the event loop allows". */
private scheduleCapture(sessionId: string, delayMs?: number): void {
if (!this.deps.options.enabled || this.disposed) return;
const entry = this.observations.get(sessionId);
if (entry === undefined || entry.dead === true) return;
this.cancelCapture(sessionId);
const now = this.scheduler.now();
const lastSeen = this.lastActivity.get(sessionId) ?? 0;
const failures = this.captureFailures.get(sessionId) ?? 0;
const { thumbnailIntervalMs, idleIntervalMs } = this.deps.options;
const cadence =
now - lastSeen < idleIntervalMs && failures < FAILURE_BACKOFF_THRESHOLD
? thumbnailIntervalMs
: idleIntervalMs;
const delay = delayMs ?? cadence;
const timer = this.scheduler.setTimeout(() => {
this.captureTimers.delete(sessionId);
void this.capture(sessionId);
}, delay);
this.captureTimers.set(sessionId, timer);
}
/**
* Capture one frame: `bsk screenshot --json` through the runner, bytes into
* the attachment store, reference onto the observation. Runs OUTSIDE the
* tool instrumentation on purpose — no action events, no registry writes.
* Failures keep the previous frame and back off silently.
*/
private async capture(sessionId: string): Promise<void> {
if (this.disposed) return;
const current = this.observations.get(sessionId);
if (current === undefined || current.dead === true) return;
if (this.captureInFlight.has(sessionId)) return;
const attachments = this.deps.ctx.get("attachments") as AttachmentLike | undefined;
if (attachments === undefined) {
// No attachment store (headless composition): thumbnails stay absent.
return;
}
this.captureInFlight.add(sessionId);
// One fixed scratch path per session (overwritten each frame) and always
// unlinked after the bytes are read — /tmp must not grow without bound.
const outPath = join(tmpdir(), `bsk-obs-${sessionId}.png`);
let writtenPath = outPath;
try {
const result = await this.deps.queue.run(sessionId, () =>
this.deps.runner.run(["screenshot", "--session", sessionId, "--out", outPath], {
timeoutMs: 15_000,
tag: `observation:${sessionId}`,
}),
);
if (result.code !== 0) {
let code: string | undefined;
try {
code = (JSON.parse(result.stdout) as { code?: string }).code;
} catch {
code = undefined;
}
if (code === "session_not_found") {
this.markDead(sessionId);
return;
}
throw new Error(`screenshot exited ${result.code}`);
}
const reply = JSON.parse(result.stdout) as { path?: string };
writtenPath = reply.path ?? outPath;
const data = await readFile(writtenPath);
const ref = await attachments.saveImage({
data,
mediaType: "image/png",
name: `observation-${sessionId}.png`,
});
const entry = this.observations.get(sessionId);
if (entry === undefined) return;
this.captureFailures.delete(sessionId);
this.globalFailures = 0;
this.setAvailable(true);
// Drop the replaced frame's reference — the store keeps the bytes, but
// this map (and the old id's read path) must not grow without bound.
if (entry.thumbnailAttachmentId !== undefined) {
this.thumbRefs.delete(entry.thumbnailAttachmentId);
}
this.thumbRefs.set(String(ref.attachmentId), ref);
this.put({ ...entry, thumbnailAttachmentId: String(ref.attachmentId) });
} catch {
// Silent by design: keep the previous frame, count toward backoff.
this.captureFailures.set(sessionId, (this.captureFailures.get(sessionId) ?? 0) + 1);
this.globalFailures += 1;
if (this.globalFailures >= FAILURE_BACKOFF_THRESHOLD) this.setAvailable(false);
} finally {
await unlink(writtenPath).catch(() => {});
this.captureInFlight.delete(sessionId);
if (!this.disposed && this.observations.has(sessionId)) this.scheduleCapture(sessionId);
}
}
/** Mark a session dead: grey it out, stop asking for frames, keep the entry for the strip. */
private markDead(sessionId: string): void {
const entry = this.observations.get(sessionId);
if (entry === undefined || entry.dead === true) return;
this.cancelCapture(sessionId);
this.put({ ...entry, dead: true, action: "idle" });
}
}
/** Map a bsk command label onto its observation action verb. */
export function actionForLabel(label: string): string {
switch (label) {
case "session start":
return "starting";
case "session stop":
return "stopping";
case "navigate":
return "navigating";
case "snapshot":
return "snapshotting";
case "observe":
return "observing";
case "click":
return "clicking";
case "fill":
return "filling";
case "press":
return "pressing";
case "screenshot":
return "capturing";
case "emulate":
return "emulating";
default:
return label;
}
}
@@ -0,0 +1,63 @@
/**
* Per-key FIFO executor. The bsk daemon serializes commands per session (a
* second command while one is unfinished is rejected), so every plugin
* command — model-facing tool calls AND observation captures — funnels
* through one queue per session. A queued task rejects early if its abort
* signal fires before it starts; once running, cancellation is owned by the
* task's own signal handling (the runner kills the child).
*/
export class KeyedExecutor {
private readonly tails = new Map<string, Promise<void>>();
/** Run `fn` after every previously queued task for `key` settled. */
run<T>(key: string, fn: () => Promise<T>, signal?: AbortSignal): Promise<T> {
const previous = this.tails.get(key) ?? Promise.resolve();
const task = new Promise<T>((resolve, reject) => {
let settled = false;
const onAbort = () => {
if (!settled) {
settled = true;
reject(abortError());
}
};
signal?.addEventListener("abort", onAbort, { once: true });
void previous.then(() => {
if (settled) return;
if (signal?.aborted) {
settled = true;
reject(abortError());
return;
}
// Running now: cancellation flows through the task's own signal.
signal?.removeEventListener("abort", onAbort);
fn().then(
(value) => {
settled = true;
resolve(value);
},
(error: unknown) => {
settled = true;
reject(error);
},
);
});
});
// The stored tail never rejects, so one failure never strands the queue;
// and it chains BOTH the previous tail and this task, so a task aborted
// while still queued cannot let a later task overlap the one before it.
const tail = Promise.allSettled([previous, task]).then(() => {});
this.tails.set(key, tail);
// Drop the entry once the queue drains, so the map tracks live keys only.
void tail.then(() => {
if (this.tails.get(key) === tail) this.tails.delete(key);
});
return task;
}
}
function abortError(): Error {
const error = new Error("tool call aborted");
error.name = "AbortError";
return error;
}
@@ -0,0 +1,216 @@
/**
* Process runner for the `bsk` CLI. Every model-facing tool in this plugin
* maps to one `bsk <cmd> --json` invocation: spawn the child, capture
* stdout/stderr, honor the dsh cancellation signal by killing the child, and
* map the CLI's JSON error envelope onto a thrown `BskError`.
*/
import { type ChildProcess, spawn } from "node:child_process";
/** Shape of the JSON error envelope `bsk --json` prints on failure. */
export interface BskErrorBody {
code?: string;
message?: string;
hint?: string;
exit_code?: number;
data?: unknown;
}
/** A failed `bsk` invocation (non-zero exit, timeout, or spawn failure). */
export class BskError extends Error {
readonly code?: string;
readonly hint?: string;
readonly exitCode?: number | null;
readonly timedOut: boolean;
constructor(
message: string,
options: { code?: string; hint?: string; exitCode?: number | null; timedOut?: boolean } = {},
) {
super(message);
this.name = "BskError";
this.code = options.code;
this.hint = options.hint;
this.exitCode = options.exitCode;
this.timedOut = options.timedOut ?? false;
}
}
export interface BskRunResult {
/** Process exit code, or null when killed by a signal / never started. */
code: number | null;
stdout: string;
stderr: string;
timedOut: boolean;
aborted: boolean;
}
export interface BskRunOptions {
signal?: AbortSignal;
timeoutMs?: number;
/** Opaque routing tag (e.g. a session id) enabling per-tag kills. */
tag?: string;
}
/** Minimal spawn signature so tests can substitute a fake child process. */
export type SpawnImpl = (command: string, args: string[]) => ChildProcess;
export interface BskRunner {
/** Run `bsk <args...> --json` and collect its output. */
run(args: string[], options?: BskRunOptions): Promise<BskRunResult>;
/** Kill every in-flight child (used when the plugin unloads). */
killAll(): void;
/** Kill only the in-flight children carrying this tag; returns how many were killed. */
killFor(tag: string): number;
}
const KILL_GRACE_MS = 2000;
export function createBskRunner(bskPath: string, spawnImpl: SpawnImpl = spawn): BskRunner {
const live = new Map<ChildProcess, string | undefined>();
function killChild(child: ChildProcess): void {
if (child.exitCode !== null || child.signalCode !== null) return;
child.kill("SIGTERM");
const force = setTimeout(() => {
if (child.exitCode === null && child.signalCode === null) child.kill("SIGKILL");
}, KILL_GRACE_MS);
force.unref();
}
return {
run(args, options = {}) {
return new Promise<BskRunResult>((resolve, reject) => {
let child: ChildProcess;
try {
child = spawnImpl(bskPath, [...args, "--json"]);
} catch (error) {
reject(error);
return;
}
live.set(child, options.tag);
let stdout = "";
let stderr = "";
let timedOut = false;
let aborted = false;
child.stdout?.on("data", (chunk: Buffer | string) => {
stdout += chunk;
});
child.stderr?.on("data", (chunk: Buffer | string) => {
stderr += chunk;
});
const timeoutMs = options.timeoutMs;
const timer =
timeoutMs !== undefined && timeoutMs > 0
? setTimeout(() => {
timedOut = true;
killChild(child);
}, timeoutMs)
: undefined;
timer?.unref();
const onAbort = () => {
aborted = true;
killChild(child);
};
if (options.signal?.aborted) {
onAbort();
} else {
options.signal?.addEventListener("abort", onAbort, { once: true });
}
const settle = () => {
if (timer !== undefined) clearTimeout(timer);
options.signal?.removeEventListener("abort", onAbort);
live.delete(child);
};
child.on("error", (error) => {
settle();
reject(error);
});
child.on("close", (code) => {
settle();
resolve({ code, stdout, stderr, timedOut, aborted });
});
});
},
killAll() {
for (const child of live.keys()) killChild(child);
},
killFor(tag: string) {
let killed = 0;
for (const [child, childTag] of live) {
if (childTag === tag) {
killChild(child);
killed += 1;
}
}
return killed;
},
};
}
/** True when the spawn failure means the bsk binary itself is missing. */
export function isCommandNotFound(error: unknown): boolean {
return (
typeof error === "object" &&
error !== null &&
"code" in error &&
(error as { code?: unknown }).code === "ENOENT"
);
}
/** Install guidance shown when the bsk CLI cannot be spawned. */
export function bskInstallMessage(bskPath: string): string {
return (
`the bsk CLI ("${bskPath}") was not found. BrowserSkill must be installed and on PATH ` +
"for browser tools to work — install it from https://github.com/Tencent/BrowserSkill " +
"(see the README install script or `cargo install`), then retry."
);
}
/**
* Interpret one finished run: throw `BskError` on timeout / non-zero exit
* (parsing the CLI's JSON error envelope when present), otherwise parse and
* return the stdout JSON payload.
*/
export function parseBskJson(result: BskRunResult, commandLabel: string): unknown {
if (result.timedOut) {
throw new BskError(`bsk ${commandLabel} timed out`, { timedOut: true });
}
const body = result.stdout.trim();
// Killed by our own interrupt (SIGTERM from killFor), not by the abort path:
// say so instead of doubling the generic label into the message.
if (result.code === null && !result.aborted && !result.timedOut) {
throw new BskError(`bsk ${commandLabel} was interrupted (process killed)`);
}
if (result.code !== 0) {
let parsed: BskErrorBody | undefined;
try {
parsed = JSON.parse(body) as BskErrorBody;
} catch {
parsed = undefined;
}
const message =
parsed?.message ?? (result.stderr.trim() || body || `bsk ${commandLabel} failed`);
// Surface the envelope's actionable hint in the model-facing message; a
// hint the model cannot see cannot be followed.
const withHint = parsed?.hint !== undefined ? `${message} (hint: ${parsed.hint})` : message;
throw new BskError(`bsk ${commandLabel} failed: ${withHint}`, {
code: parsed?.code,
hint: parsed?.hint,
exitCode: result.code,
});
}
try {
return JSON.parse(body);
} catch {
throw new BskError(
`bsk ${commandLabel} did not produce JSON output: ${body.slice(0, 200) || "(empty)"}`,
{ exitCode: result.code },
);
}
}
@@ -0,0 +1,161 @@
/**
* Tracks the bsk sessions this plugin created and the "current" session
* pointer used when a tool call omits its optional `session` arg. The pointer
* moves to whatever session was most recently started, stopped, or operated
* on, so a model can run several browsers side by side without repeating the
* id on every call.
*
* Strict ownership boundary (the bsk daemon may be shared with other agents,
* terminals, or dsh instances): the registry ONLY ever holds sessions created
* by this plugin's browser_session_start. Tools cannot see or act on any
* other session — an explicit `session` argument naming a foreign id is an
* error, the list tool shows owned sessions only, and stop/unload cleanup
* can never touch a session this plugin did not create.
*/
export interface TrackedSession {
sessionId: string;
browserInstanceId?: string;
startedAtMs: number;
/** Always true: only plugin-created sessions enter the registry at all. */
owned: boolean;
}
export class SessionRegistry {
private readonly sessions = new Map<string, TrackedSession>();
private currentId: string | undefined;
/**
* Slots reserved by in-flight starts. reserveStart/completeStart/abandonStart
* run synchronously around the async spawn, so concurrent starts can never
* both pass the capacity check (check-and-reserve is atomic on the event loop).
*/
private pendingStarts = 0;
constructor(private readonly maxSessions: number) {}
/**
* Reserve a start slot synchronously, BEFORE spawning.
* @throws when the configured concurrency cap (tracked + in-flight) is reached.
*/
reserveStart(): void {
if (this.sessions.size + this.pendingStarts >= this.maxSessions) {
throw new Error(
`session limit reached (${this.maxSessions} concurrent sessions); ` +
"stop one with browser_session_stop before starting another",
);
}
this.pendingStarts += 1;
}
/** Give back a reservation after a start that never produced a session. */
abandonStart(): void {
this.pendingStarts = Math.max(0, this.pendingStarts - 1);
}
/**
* Register a freshly started session, consuming its reservation, and make
* it current.
*/
completeStart(session: Omit<TrackedSession, "owned">): void {
this.pendingStarts = Math.max(0, this.pendingStarts - 1);
// Backstop only: with the reservation protocol above this never fires.
if (!this.sessions.has(session.sessionId) && this.sessions.size >= this.maxSessions) {
throw new Error(
`session limit reached (${this.maxSessions} concurrent sessions); ` +
"stop one with browser_session_stop before starting another",
);
}
this.sessions.set(session.sessionId, { ...session, owned: true });
this.currentId = session.sessionId;
}
/** Forget a session; falls back to the most recent remaining one. */
remove(sessionId: string): void {
this.sessions.delete(sessionId);
if (this.currentId === sessionId) {
const rest = [...this.sessions.values()];
this.currentId = rest.length > 0 ? rest[rest.length - 1].sessionId : undefined;
}
}
/** Mark an owned session as most recently used (recency order refresh). */
private touch(sessionId: string): void {
const existing = this.sessions.get(sessionId);
if (existing === undefined) return;
this.sessions.delete(sessionId);
this.sessions.set(sessionId, existing);
this.currentId = sessionId;
}
/** The current session id, if any. */
current(): string | undefined {
return this.currentId;
}
/** Owned sessions in least- to most-recently-used order. */
list(): TrackedSession[] {
return [...this.sessions.values()];
}
/** Ids of owned sessions — the exact set unload cleanup is allowed to stop. */
ownedIds(): string[] {
return this.list()
.filter((session) => session.owned)
.map((session) => session.sessionId);
}
/** Whether the session was created by this plugin. */
isOwned(sessionId: string): boolean {
return this.sessions.get(sessionId)?.owned === true;
}
size(): number {
return this.sessions.size;
}
/** Shared not-yours error for foreign or unknown session ids. */
private foreignError(sessionId: string, toolName: string): Error {
return new Error(
`${toolName}: session "${sessionId}" does not belong to this plugin — only sessions ` +
"created by browser_session_start are visible and operable here",
);
}
/**
* Resolve the session a tool call acts on: an explicit `session` argument
* must name an owned session (and becomes current); omitted falls back to
* the current session. Foreign ids are rejected, never adopted.
* @throws on foreign/unknown ids, or when no session exists.
*/
resolve(explicit: string | undefined, toolName: string): string {
if (explicit !== undefined && explicit.trim().length > 0) {
if (!this.isOwned(explicit)) throw this.foreignError(explicit, toolName);
this.touch(explicit);
return explicit;
}
const current = this.current();
if (current === undefined) {
throw new Error(
`${toolName} needs a session but none is active — start one with browser_session_start`,
);
}
this.touch(current);
return current;
}
/**
* Resolve the session a STOP call acts on. Same ownership rule as every
* other tool; a rejected stop never moves the current pointer.
*/
resolveForStop(explicit: string | undefined): string {
const candidate =
explicit !== undefined && explicit.trim().length > 0 ? explicit : this.current();
if (candidate === undefined) {
throw new Error(
"browser_session_stop needs a session but none is active — start one with browser_session_start",
);
}
if (!this.isOwned(candidate)) throw this.foreignError(candidate, "browser_session_stop");
return candidate;
}
}
@@ -0,0 +1,45 @@
/**
* Progressive disclosure of the BrowserSkill agent skill through the harness's
* official skill seam (`ctx.skills`). The catalog entry (name + description)
* is resident; the body is loaded only when the model invokes the `skill`
* tool. The markdown is build-time assembled into a static module (dsh
* prelude + the canonical CLI skill body verbatim), so registration and every
* pre-step catalog snapshot are pure in-memory reads — no disk, no process,
* no daemon.
*/
import type { Context } from "@deepseek-ai/cordis";
import {
BSK_SKILL_DESCRIPTION,
BSK_SKILL_MARKDOWN,
BSK_SKILL_NAME,
} from "./skill-content.generated";
/** Structural view of the skill registry seam (absent in bare compositions). */
interface SkillsLike {
register(skill: {
name: string;
description: string;
content: string;
source?: string;
}): () => void;
}
/**
* Publish the BSK skill as an embedded runtime skill. Returns the unregistration
* disposer. A composition without the `skills` service (or with dsh-tool-skill
* retired) leaves the rest of the plugin unaffected — the no-op is silent.
*/
export function registerBskSkill(ctx: Context): () => void {
const skills = ctx.get("skills") as SkillsLike | undefined | null;
if (skills == null || typeof skills.register !== "function") {
return () => {};
}
return skills.register({
name: BSK_SKILL_NAME,
description: BSK_SKILL_DESCRIPTION,
content: BSK_SKILL_MARKDOWN,
// Prompt-visible origin bucket: packaged with a plugin, not user/project files.
source: "bundled",
});
}
@@ -0,0 +1,961 @@
/**
* Model-facing tool definitions. Each tool maps to one `bsk <cmd> --json`
* invocation: spawn via the runner, parse the JSON envelope, and return a
* canonical value whose `output.render` produces the model-facing text.
* UI cards reuse the terminal style: the pending card is the command line,
* the completed card carries the output.
*/
import { readFile, unlink } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import type { Context } from "@deepseek-ai/cordis";
import type { ContentBlock } from "@deepseek-ai/dsh-llm";
import {
defineTool,
type ToolDefinition,
type ToolResult,
type ToolRunContext,
} from "@deepseek-ai/dsh-tools";
import { trySaveScreenshot } from "./image";
import { actionForLabel, type ObservationService } from "./observation";
import type { KeyedExecutor } from "./queue";
import { type BskRunner, bskInstallMessage, isCommandNotFound, parseBskJson } from "./runner";
import type { SessionRegistry } from "./sessions";
/** Plugin configuration resolved from the Schemastery schema in index.ts. */
export interface PluginConfig {
bskPath: string;
defaultTimeoutMs: number;
maxSessions: number;
observationEnabled: boolean;
thumbnailIntervalMs: number;
idleIntervalMs: number;
/**
* Lazy tool-schema injection: when true (default), the browser_* tools are
* registered only after the `browser-skill` skill has been successfully
* invoked (skill catalog entry advertises alone until then); when false,
* the suite is registered at apply time (legacy always-on behavior).
*/
lazyTools: boolean;
}
export interface ToolDeps {
ctx: Context;
runner: BskRunner;
registry: SessionRegistry;
config: PluginConfig;
observation: ObservationService;
/** Per-session FIFO: the daemon rejects a second command while one is unfinished. */
queue: KeyedExecutor;
}
/** Device presets supported by `bsk emulate --device`. */
const DEVICE_PRESETS = [
"iphone-14",
"iphone-14-pro-max",
"iphone-se",
"pixel-7",
"galaxy-s23",
"ipad-mini",
"galaxy-tab-s8",
] as const;
const SESSION_PARAM = {
type: "string",
description:
"bsk session id to act on; must be one created by this plugin's browser_session_start. " +
"Omit to use the current session (the one most recently started or used).",
} as const;
/**
* Run one bsk command and return its parsed JSON payload, or throw.
* `observeSession` marks the run as model-facing work on that session: it is
* instrumented into the observation service (action begin/end) and tagged so
* interrupt() can kill exactly this child. Observation traffic itself never
* passes an observeSession.
*/
async function runBsk(
deps: ToolDeps,
exec: ToolRunContext,
args: string[],
label: string,
observeSession?: string,
): Promise<unknown> {
// beginAction must fire only when the task actually starts inside the
// per-session queue — marking it while still queued would overwrite the
// in-flight action's label (and flash it idle on a queued abort).
let began = false;
let actionError: string | undefined;
try {
let result;
try {
const runOnce = () => {
if (observeSession !== undefined) {
began = true;
deps.observation.beginAction(observeSession, actionForLabel(label));
}
return deps.runner.run(args, {
signal: exec.signal,
timeoutMs: deps.config.defaultTimeoutMs,
...(observeSession !== undefined ? { tag: observeSession } : {}),
});
};
result =
observeSession !== undefined
? await deps.queue.run(observeSession, runOnce, exec.signal)
: await runOnce();
} catch (error) {
if (isCommandNotFound(error)) {
throw new Error(bskInstallMessage(deps.config.bskPath));
}
throw error;
}
if (result.aborted) {
const error = new Error("tool call aborted");
error.name = "AbortError";
throw error;
}
return parseBskJson(result, label);
} catch (error) {
actionError = error instanceof Error ? error.message.split("\n")[0] : String(error);
throw error;
} finally {
if (observeSession !== undefined && began) {
deps.observation.endAction(observeSession, actionError);
}
}
}
/** Build the command line shown on the pending terminal card (pure). */
function cmdline(deps: ToolDeps, args: string[]): string {
return [deps.config.bskPath, ...args].join(" ");
}
/** Shared completed-card presenter: terminal card with the rendered text. */
function presentTerminalResult(_args: never, result: ToolResult) {
// Multi-block results (e.g. screenshot text + image) still project their
// text block onto the terminal card.
const block = result.content.find((b) => b.type === "text");
if (block === undefined || block.type !== "text") return undefined;
if (result.isError) return undefined;
return { card: "terminal" as const, output: block.text, exitCode: 0 };
}
function abortError(): Error {
const error = new Error("tool call aborted");
error.name = "AbortError";
return error;
}
/** Register the full browser tool suite; returns the combined unregister disposer. */
export function registerTools(deps: ToolDeps): () => void {
const { ctx, registry } = deps;
// Track every registration without touching the cordis context itself
// (contexts are fiber-owned; the lazy path registers from a listener fiber).
const disposers: (() => void)[] = [];
const register = (definition: ToolDefinition): (() => void) => {
const dispose = ctx.tools.register(definition);
// Test doubles occasionally return the registry map instead of a disposer.
if (typeof dispose === "function") disposers.push(dispose);
return dispose;
};
register(
defineTool({
name: "browser_session_start",
description:
"Start a new browser session: opens an Agent Window in the connected browser and returns " +
"its session id. The new session becomes the current session for subsequent browser_* calls. " +
"Optionally navigate to an initial URL and/or apply a mobile device emulation preset.",
parameters: {
url: {
type: "string",
description: "Initial URL to navigate to after the session starts.",
},
width: {
type: "integer",
description: "Agent Window outer width in CSS pixels (100..=7680). Requires height.",
},
height: {
type: "integer",
description: "Agent Window outer height in CSS pixels (100..=7680). Requires width.",
},
noFocus: {
type: "boolean",
description: "Open the Agent Window in the background without stealing focus.",
},
browser: {
type: "string",
description:
"Target browser instance id (only needed when multiple browsers are connected).",
},
device: {
type: "string",
enum: DEVICE_PRESETS,
description: "Mobile device emulation preset applied to the tab after start.",
},
},
output: {
schema: {
type: "object",
additionalProperties: false,
properties: {
sessionId: { type: "string", required: true },
browserInstanceId: { type: "string", required: true },
url: { type: "string" },
device: { type: "string" },
},
},
render: (_args, value) => [
{
type: "text",
text:
`started browser session ${value.sessionId}` +
(value.url !== undefined ? ` and navigated to ${value.url}` : "") +
(value.device !== undefined ? ` (device: ${value.device})` : ""),
},
],
},
async execute(args, exec) {
if (args.url !== undefined && args.url.trim().length === 0) {
throw new Error("url must be a non-empty string");
}
if ((args.width === undefined) !== (args.height === undefined)) {
throw new Error("width and height must be given together");
}
// Reserve the slot BEFORE spawning: check-and-reserve is synchronous,
// so concurrent starts can never both pass the cap and leak a session.
registry.reserveStart();
const startArgs = ["session", "start"];
if (args.width !== undefined && args.height !== undefined) {
startArgs.push("--width", String(args.width), "--height", String(args.height));
}
if (args.noFocus === true) startArgs.push("--no-focus");
if (args.browser !== undefined) startArgs.push("--browser", args.browser);
let reply: { session_id: string; browser_instance_id: string };
try {
reply = (await runBsk(deps, exec, startArgs, "session start")) as typeof reply;
} catch (error) {
registry.abandonStart();
throw error;
}
registry.completeStart({
sessionId: reply.session_id,
browserInstanceId: reply.browser_instance_id,
startedAtMs: Date.now(),
});
deps.observation.addSession(reply.session_id, args.url);
try {
if (args.device !== undefined) {
await runBsk(
deps,
exec,
["emulate", "--session", reply.session_id, "--device", args.device],
"emulate",
reply.session_id,
);
}
if (args.url !== undefined) {
await runBsk(
deps,
exec,
["navigate", "--session", reply.session_id, args.url],
"navigate",
reply.session_id,
);
}
} catch (error) {
// A half-initialized session must not leak: stop it before surfacing.
// The stop funnels through the session's queue (an in-flight command
// — e.g. an observation capture — would otherwise make the daemon
// refuse it), with one retry for good measure.
for (let attempt = 0; attempt < 2; attempt++) {
try {
const stopped = await deps.queue.run(reply.session_id, () =>
deps.runner.run(["session", "stop", reply.session_id], { timeoutMs: 30_000 }),
);
if (stopped.code === 0) break;
} catch {
// retry once, then give up quietly (registry/observation are
// cleaned below either way; the daemon reaps orphans).
}
}
registry.remove(reply.session_id);
deps.observation.removeSession(reply.session_id);
throw error;
}
return {
sessionId: reply.session_id,
browserInstanceId: reply.browser_instance_id,
...(args.url !== undefined ? { url: args.url } : {}),
...(args.device !== undefined ? { device: args.device } : {}),
};
},
presentCall: (args) => ({
card: "terminal",
title: cmdline(deps, [
"session",
"start",
...(args.device !== undefined ? ["+ emulate", args.device] : []),
...(args.url !== undefined ? ["+ navigate", args.url] : []),
]),
description: "Start a browser session",
}),
presentResult: presentTerminalResult,
}),
);
register(
defineTool({
name: "browser_session_stop",
description:
"Stop a browser session and close its Agent Window. Stops the given session, or the " +
"current session when `session` is omitted. Only sessions created by browser_session_start " +
"can be stopped — sessions owned by other programs sharing the bsk daemon are refused.",
parameters: { session: SESSION_PARAM },
output: {
schema: {
type: "object",
additionalProperties: false,
properties: { stopped: { type: "string", required: true } },
},
render: (_args, value) => [
{ type: "text", text: `stopped browser session ${value.stopped}` },
],
},
async execute(args, exec) {
const sessionId = registry.resolveForStop(args.session);
await runBsk(deps, exec, ["session", "stop", sessionId], "session stop", sessionId);
registry.remove(sessionId);
deps.observation.removeSession(sessionId);
return { stopped: sessionId };
},
presentCall: (args) => ({
card: "terminal",
title: cmdline(deps, ["session", "stop", args.session ?? "(current session)"]),
description: "Stop a browser session",
}),
presentResult: presentTerminalResult,
}),
);
register(
defineTool({
name: "browser_session_list",
description:
"List the browser sessions created by this plugin. Sessions owned by other programs on the " +
"shared bsk daemon are not visible here. The session marked `current` is the one browser_* " +
"tools act on when no explicit `session` is passed.",
parameters: {},
output: {
schema: {
type: "object",
additionalProperties: false,
properties: {
sessions: {
type: "array",
required: true,
items: {
type: "object",
additionalProperties: false,
properties: {
sessionId: { type: "string", required: true },
browserInstanceId: { type: "string", required: true },
current: { type: "boolean", required: true },
},
},
},
},
},
render: (_args, value) => [
{
type: "text",
text:
value.sessions.length === 0
? "no active browser sessions"
: value.sessions
.map(
(s) =>
`${s.sessionId} (browser ${s.browserInstanceId})${s.current ? " [current]" : ""}`,
)
.join("\n"),
},
],
},
isConcurrencySafe: () => true,
// Registry-only by design: no daemon call, so foreign sessions on a
// shared daemon can never even be SEEN through this tool.
async execute() {
const current = registry.current();
return {
sessions: registry.list().map((entry) => ({
sessionId: entry.sessionId,
browserInstanceId: entry.browserInstanceId ?? "",
current: entry.sessionId === current,
})),
};
},
presentCall: () => ({
card: "terminal",
title: cmdline(deps, ["session", "list"]),
description: "List browser sessions",
}),
presentResult: presentTerminalResult,
}),
);
register(
defineTool({
name: "browser_navigate",
description:
"Navigate the session's active tab to a URL and wait for a page lifecycle phase " +
"(default: load). Returns the final URL after redirects.",
parameters: {
url: { type: "string", required: true, description: "Destination URL." },
session: SESSION_PARAM,
waitUntil: {
type: "string",
enum: ["load", "domcontentloaded", "networkidle", "commit"],
description: "Lifecycle phase to wait for (default: load).",
},
timeoutMs: {
type: "integer",
description: "Navigation wait timeout in milliseconds (default 30000).",
},
},
output: {
schema: {
type: "object",
additionalProperties: false,
properties: {
session: { type: "string", required: true },
tabId: { type: "integer", required: true },
url: { type: "string", required: true },
finalUrl: { type: "string" },
reached: { type: "string", required: true },
},
},
render: (_args, value) => [
{
type: "text",
text:
`[session ${value.session}] navigated to ${value.finalUrl ?? value.url} (reached: ${value.reached})` +
(value.reached === "timeout"
? " — the wait timed out before the requested phase"
: ""),
},
],
},
async execute(args, exec) {
if (args.url.trim().length === 0) throw new Error("url must be a non-empty string");
const sessionId = registry.resolve(args.session, "browser_navigate");
const cmdArgs = ["navigate", "--session", sessionId];
if (args.waitUntil !== undefined) cmdArgs.push("--wait-until", args.waitUntil);
if (args.timeoutMs !== undefined) cmdArgs.push("--timeout", `${args.timeoutMs}ms`);
cmdArgs.push(args.url);
const reply = (await runBsk(deps, exec, cmdArgs, "navigate", sessionId)) as {
tab_id: number;
url: string;
final_url?: string;
reached: string;
};
deps.observation.setUrl(sessionId, reply.final_url ?? reply.url);
return {
session: sessionId,
tabId: reply.tab_id,
url: reply.url,
...(reply.final_url !== undefined ? { finalUrl: reply.final_url } : {}),
reached: reply.reached,
};
},
presentCall: (args) => ({
card: "terminal",
title: cmdline(deps, ["navigate", "--session", args.session ?? "(current)", args.url]),
description: "Navigate to a URL",
}),
presentResult: presentTerminalResult,
}),
);
const registerObservationTool = (kind: "snapshot" | "observe") => {
const isSnapshot = kind === "snapshot";
const name = isSnapshot ? "browser_snapshot" : "browser_observe";
const description = isSnapshot
? "Capture an indented aria-tree snapshot of the session's active tab. Interactive elements " +
"carry @eN refs for browser_click / browser_fill. Prefer browser_observe for a richer semantic view."
: "Produce a semantic VOM observation of the session's active tab (roles, states, perception " +
"probes) with @eN refs for browser_click / browser_fill. Read-only: never submits input.";
register(
defineTool({
name,
description,
parameters: {
session: SESSION_PARAM,
maxDepth: { type: "integer", description: "Cap on tree depth before truncating." },
maxTokens: {
type: "integer",
description: "Soft cap on rendered tokens (~4 chars/token).",
},
},
output: {
schema: {
type: "object",
additionalProperties: false,
properties: {
session: { type: "string", required: true },
tabId: { type: "integer", required: true },
text: { type: "string", required: true },
refCount: { type: "integer", required: true },
truncated: { type: "boolean", required: true },
},
},
render: (_args, value) => [
{
type: "text",
text:
value.text.length > 0
? value.text + (value.truncated ? "\n(truncated — re-run with looser caps)" : "")
: "(empty observation — page may still be loading)",
},
],
},
isConcurrencySafe: () => true,
async execute(args, exec) {
const sessionId = registry.resolve(args.session, name);
const cmdArgs = [kind, "--session", sessionId];
if (args.maxDepth !== undefined) cmdArgs.push("--max-depth", String(args.maxDepth));
if (args.maxTokens !== undefined) cmdArgs.push("--max-tokens", String(args.maxTokens));
const reply = (await runBsk(deps, exec, cmdArgs, kind, sessionId)) as {
text: string;
ref_count: number;
tab_id: number;
truncated?: boolean;
};
return {
session: sessionId,
tabId: reply.tab_id,
text: reply.text,
refCount: reply.ref_count,
truncated: reply.truncated ?? false,
};
},
presentCall: (args) => ({
card: "terminal",
title: cmdline(deps, [kind, "--session", args.session ?? "(current)"]),
description: isSnapshot ? "Capture an aria snapshot" : "Observe the page semantically",
}),
presentResult: presentTerminalResult,
}),
);
};
registerObservationTool("snapshot");
registerObservationTool("observe");
register(
defineTool({
name: "browser_click",
description:
"Click an element in the session's active tab. Target is a snapshot ref (@e3) from the " +
"last browser_snapshot/browser_observe, or a CSS selector.",
parameters: {
target: {
type: "string",
required: true,
description: "Snapshot ref (@e3 / e3) or CSS selector of the element to click.",
},
session: SESSION_PARAM,
button: {
type: "string",
enum: ["left", "middle", "right"],
description: "Mouse button (default: left).",
},
clickCount: {
type: "integer",
description: "Number of consecutive presses (double-click = 2).",
},
},
output: {
schema: {
type: "object",
additionalProperties: false,
properties: {
session: { type: "string", required: true },
tabId: { type: "integer", required: true },
x: { type: "number", required: true },
y: { type: "number", required: true },
},
},
render: (_args, value) => [
{
type: "text",
text: `[session ${value.session}] clicked at (${value.x}, ${value.y}) on tab ${value.tabId}`,
},
],
},
async execute(args, exec) {
if (args.target.trim().length === 0) throw new Error("target must be a non-empty string");
const sessionId = registry.resolve(args.session, "browser_click");
const cmdArgs = ["click", "--session", sessionId];
if (args.button !== undefined) cmdArgs.push("--button", args.button);
if (args.clickCount !== undefined) cmdArgs.push("--click-count", String(args.clickCount));
cmdArgs.push(args.target);
const reply = (await runBsk(deps, exec, cmdArgs, "click", sessionId)) as {
tab_id: number;
x: number;
y: number;
};
return { session: sessionId, tabId: reply.tab_id, x: reply.x, y: reply.y };
},
presentCall: (args) => ({
card: "terminal",
title: cmdline(deps, ["click", args.target, "--session", args.session ?? "(current)"]),
description: "Click an element",
}),
presentResult: presentTerminalResult,
}),
);
register(
defineTool({
name: "browser_fill",
description:
"Fill an input / textarea / contenteditable element, clearing it first by default. " +
"Target is a snapshot ref (@e3) or a CSS selector.",
parameters: {
target: {
type: "string",
required: true,
description: "Snapshot ref (@e3 / e3) or CSS selector of the field.",
},
value: { type: "string", required: true, description: "Text to type into the element." },
session: SESSION_PARAM,
noClear: {
type: "boolean",
description: "Skip the default wipe-the-field-first pass (append instead).",
},
},
output: {
schema: {
type: "object",
additionalProperties: false,
properties: {
session: { type: "string", required: true },
tabId: { type: "integer", required: true },
valueLength: { type: "integer", required: true },
},
},
render: (_args, value) => [
{
type: "text",
text: `[session ${value.session}] filled field on tab ${value.tabId} (${value.valueLength} chars)`,
},
],
},
async execute(args, exec) {
if (args.target.trim().length === 0) throw new Error("target must be a non-empty string");
const sessionId = registry.resolve(args.session, "browser_fill");
const cmdArgs = ["fill", "--session", sessionId, "--value", args.value];
if (args.noClear === true) cmdArgs.push("--no-clear");
cmdArgs.push(args.target);
const reply = (await runBsk(deps, exec, cmdArgs, "fill", sessionId)) as {
tab_id: number;
value_length: number;
};
return { session: sessionId, tabId: reply.tab_id, valueLength: reply.value_length };
},
presentCall: (args) => ({
card: "terminal",
title: cmdline(deps, ["fill", args.target, "--session", args.session ?? "(current)"]),
description: `Fill a field with ${args.value.length} chars`,
}),
presentResult: presentTerminalResult,
}),
);
register(
defineTool({
name: "browser_press",
description:
"Dispatch a keyboard key or combo (Enter, Escape, ArrowDown, Ctrl+A, …) to the session's " +
"active tab, optionally focusing a target element first.",
parameters: {
key: {
type: "string",
required: true,
description: "Key spec: a CDP key name (Enter, a, ArrowLeft) or a combo (Ctrl+A).",
},
session: SESSION_PARAM,
target: {
type: "string",
description: "Snapshot ref (@e3) or CSS selector to focus before pressing.",
},
holdMs: {
type: "integer",
description: "Hold the key down for N milliseconds between keyDown and keyUp.",
},
},
output: {
schema: {
type: "object",
additionalProperties: false,
properties: {
session: { type: "string", required: true },
tabId: { type: "integer", required: true },
key: { type: "string", required: true },
},
},
render: (_args, value) => [
{
type: "text",
text: `[session ${value.session}] pressed ${value.key} on tab ${value.tabId}`,
},
],
},
async execute(args, exec) {
if (args.key.trim().length === 0) throw new Error("key must be a non-empty string");
const sessionId = registry.resolve(args.session, "browser_press");
const cmdArgs = ["press", "--session", sessionId];
if (args.target !== undefined) {
if (args.target.trim().length === 0) throw new Error("target must be a non-empty string");
// bsk distinguishes ref vs selector via flags; positional detection is CLI-side for
// click/fill only, so pass the explicit flag that matches the target's shape.
if (/^@?e\d+$/.test(args.target)) {
cmdArgs.push("--ref", args.target);
} else {
cmdArgs.push("--selector", args.target);
}
}
if (args.holdMs !== undefined) cmdArgs.push("--hold-ms", String(args.holdMs));
cmdArgs.push(args.key);
const reply = (await runBsk(deps, exec, cmdArgs, "press", sessionId)) as {
tab_id: number;
key: string;
};
return { session: sessionId, tabId: reply.tab_id, key: reply.key };
},
presentCall: (args) => ({
card: "terminal",
title: cmdline(deps, ["press", args.key, "--session", args.session ?? "(current)"]),
description: "Press a key",
}),
presentResult: presentTerminalResult,
}),
);
register(
defineTool({
name: "browser_screenshot",
description:
"Capture a PNG screenshot of the session's active tab, or crop to a snapshot ref element. " +
"Returns the image itself when the deployment supports image input, otherwise a file path.",
parameters: {
session: SESSION_PARAM,
ref: {
type: "string",
description: "Snapshot ref (@e3) from the last snapshot/observe; crops to that element.",
},
},
output: {
schema: {
type: "object",
additionalProperties: false,
properties: {
session: { type: "string", required: true },
tabId: { type: "integer", required: true },
path: { type: "string", required: true },
width: { type: "integer", required: true },
height: { type: "integer", required: true },
byteSize: { type: "integer", required: true },
image: {
type: "object",
additionalProperties: false,
properties: {
attachmentId: { type: "string", required: true },
mediaType: { type: "string", required: true, const: "image/png" },
bytes: { type: "integer", required: true },
width: { type: "integer", required: true },
height: { type: "integer", required: true },
name: { type: "string" },
},
},
},
},
render: (_args, value) => {
const text =
value.image !== undefined
? `[session ${value.session}] screenshot of tab ${value.tabId} (${value.width}x${value.height}px)`
: `[session ${value.session}] screenshot saved to ${value.path} (${value.width}x${value.height}px, ${value.byteSize} bytes) — this deployment cannot inline images; read the file to view it`;
const blocks: ContentBlock[] = [{ type: "text", text }];
if (value.image !== undefined) {
blocks.push({
type: "image",
// Structurally an ImageAttachmentRef; branded without a runtime import.
attachment: {
attachmentId: value.image.attachmentId,
mediaType: value.image.mediaType,
bytes: value.image.bytes,
width: value.image.width,
height: value.image.height,
...(value.image.name !== undefined ? { name: value.image.name } : {}),
} as never,
});
}
return blocks;
},
},
isConcurrencySafe: () => true,
async execute(args, exec) {
const sessionId = registry.resolve(args.session, "browser_screenshot");
const outPath = join(
tmpdir(),
`bsk-screenshot-${Date.now()}-${Math.random().toString(36).slice(2, 8)}.png`,
);
const cmdArgs = ["screenshot", "--session", sessionId, "--out", outPath];
if (args.ref !== undefined) cmdArgs.push("--ref", args.ref);
// The scratch PNG is only kept when it IS the model-facing artifact
// (no attachment store to inline through); once the bytes are saved
// into the store — and on every failure/abort path — it is deleted.
let keepFile = false;
let writtenPath: string | undefined;
try {
const reply = (await runBsk(deps, exec, cmdArgs, "screenshot", sessionId)) as {
tab_id: number;
width: number;
height: number;
path: string;
byte_size: number;
};
writtenPath = reply.path;
const data = await readFile(reply.path);
if (exec.signal.aborted) throw abortError();
const ref = await trySaveScreenshot(deps.ctx, exec, data, `screenshot-${sessionId}.png`);
keepFile = ref === undefined;
return {
session: sessionId,
tabId: reply.tab_id,
path: reply.path,
width: reply.width,
height: reply.height,
byteSize: reply.byte_size,
...(ref !== undefined
? {
image: {
attachmentId: String(ref.attachmentId),
mediaType: "image/png" as const,
bytes: ref.bytes,
width: ref.width,
height: ref.height,
...(ref.name !== undefined ? { name: ref.name } : {}),
},
}
: {}),
};
} finally {
if (!keepFile) await unlink(writtenPath ?? outPath).catch(() => {});
}
},
presentCall: (args) => ({
card: "terminal",
title: cmdline(deps, [
"screenshot",
"--session",
args.session ?? "(current)",
...(args.ref !== undefined ? ["--ref", args.ref] : []),
]),
description: "Capture a screenshot",
}),
presentResult: presentTerminalResult,
}),
);
register(
defineTool({
name: "browser_emulate",
description:
"Apply (or clear) mobile device emulation on the session's active tab: viewport metrics, " +
"user agent, and touch. Overrides are per-tab and not inherited by new tabs.",
parameters: {
session: SESSION_PARAM,
device: {
type: "string",
enum: DEVICE_PRESETS,
description: "Built-in device preset.",
},
width: { type: "integer", description: "Viewport width in CSS pixels (requires height)." },
height: { type: "integer", description: "Viewport height in CSS pixels (requires width)." },
mobile: {
type: "boolean",
description:
"Emulate a mobile viewport. Requires width+height — the bsk daemon refuses " +
"--mobile without viewport dimensions, so this flag cannot be used alone.",
},
off: { type: "boolean", description: "Clear every emulation override on the tab." },
},
output: {
schema: {
type: "object",
additionalProperties: false,
properties: {
session: { type: "string", required: true },
tabId: { type: "integer", required: true },
cleared: { type: "boolean", required: true },
},
},
render: (_args, value) => [
{
type: "text",
text: value.cleared
? `[session ${value.session}] cleared emulation on tab ${value.tabId}`
: `[session ${value.session}] applied emulation on tab ${value.tabId}`,
},
],
},
async execute(args, exec) {
const sessionId = registry.resolve(args.session, "browser_emulate");
if (args.off === true) {
if (args.device !== undefined || args.width !== undefined || args.height !== undefined) {
throw new Error("off is mutually exclusive with device/width/height");
}
} else if (args.device === undefined && args.width === undefined) {
throw new Error(
"nothing to apply: pass device or width+height (mobile also requires width+height), or off",
);
}
if ((args.width === undefined) !== (args.height === undefined)) {
throw new Error("width and height must be given together");
}
const cmdArgs = ["emulate", "--session", sessionId];
if (args.off === true) {
cmdArgs.push("--off");
} else {
if (args.device !== undefined) cmdArgs.push("--device", args.device);
if (args.width !== undefined && args.height !== undefined) {
cmdArgs.push("--width", String(args.width), "--height", String(args.height));
}
if (args.mobile === true) cmdArgs.push("--mobile");
}
const reply = (await runBsk(deps, exec, cmdArgs, "emulate", sessionId)) as {
tab_id: number;
cleared: boolean;
};
return { session: sessionId, tabId: reply.tab_id, cleared: reply.cleared };
},
presentCall: (args) => ({
card: "terminal",
title: cmdline(deps, [
"emulate",
"--session",
args.session ?? "(current)",
...(args.off === true ? ["--off"] : []),
...(args.device !== undefined ? ["--device", args.device] : []),
]),
description: "Emulate a device environment",
}),
presentResult: presentTerminalResult,
}),
);
return () => {
for (const dispose of disposers.splice(0)) dispose();
};
}
@@ -0,0 +1,375 @@
// @vitest-environment happy-dom
// ObservationOverlay: visibility lifecycle, focus view, interrupt interaction,
// hover tooltip, drag move/resize from every corner with clamps, collapse capsule, and PiP
// upgrade/fallback (mocked documentPictureInPicture). NOTE: use RTL's waitFor
// (act-flushing), not vi.waitFor, when asserting store-driven UI updates.
import { cleanup, fireEvent, render, screen, waitFor } from "@testing-library/react";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { applyResize, ObservationOverlay } from "../../src/client/ObservationOverlay";
import { type EventSourceLike, ObservationClientStore } from "../../src/client/observation-store";
import type { SessionObservation } from "../../src/observation";
const BUSY: SessionObservation = { sessionId: "s1", action: "clicking", since: Date.now() - 7000 };
interface Harness {
store: ObservationClientStore;
es: () => EventSourceLike;
push: (sessions: SessionObservation[]) => void;
emitRaw: (event: unknown) => void;
fetches: { url: string; init?: { body?: string } }[];
loadImage: ReturnType<typeof vi.fn>;
}
function makeHarness(initial: SessionObservation[]): Harness {
const fetches: Harness["fetches"] = [];
let es: EventSourceLike | undefined;
let current = initial;
const loadImage = vi.fn(async (id: string) => `blob:${id}`);
const store = new ObservationClientStore({
fetchFn: async (url: string, init?: { body?: string }) => {
fetches.push({ url, init });
if (url === "/bsk-observation/state") {
return { ok: true, json: async () => ({ sessions: current }) };
}
return { ok: true, json: async () => ({ interrupted: true }) };
},
eventSourceFactory: () => {
es = { onmessage: null, close: vi.fn() };
return es;
},
loadImage,
});
return {
store,
es: () => {
if (es === undefined) throw new Error("no EventSource yet");
return es;
},
push: (sessions) => {
current = sessions;
for (const s of sessions) {
es?.onmessage?.({ data: JSON.stringify({ type: "upsert", session: s }) });
}
if (sessions.length === 0) es?.onmessage?.({ data: JSON.stringify({ type: "reset" }) });
},
emitRaw: (event) => {
es?.onmessage?.({ data: JSON.stringify(event) });
},
fetches,
loadImage,
};
}
afterEach(cleanup);
describe("ObservationOverlay", () => {
beforeEach(() => {
delete (window as unknown as Record<string, unknown>).documentPictureInPicture;
});
it("renders nothing with no sessions, appears when one starts", async () => {
const h = makeHarness([]);
const { container } = render(<ObservationOverlay store={h.store} />);
await waitFor(() => expect(h.es).not.toThrow());
expect(container.firstChild).toBeNull();
h.push([{ sessionId: "s1", action: "idle", since: Date.now() }]);
await screen.findByText(/s1 · idle/);
expect(screen.getByTestId("obs-card")).toBeTruthy();
});
it("shows the status row and a placeholder without a thumbnail", async () => {
const h = makeHarness([BUSY]);
render(<ObservationOverlay store={h.store} />);
await screen.findByText(/s1 · clicking · 00:0/);
expect(screen.getByText("waiting for page")).toBeTruthy();
});
it("loads and renders the thumbnail through the store", async () => {
const h = makeHarness([{ ...BUSY, thumbnailAttachmentId: "att-9" }]);
render(<ObservationOverlay store={h.store} />);
await waitFor(() => expect(h.loadImage).toHaveBeenCalledWith("att-9"));
await screen.findByRole("img", { name: "session s1 view" });
});
it("keeps the current frame on stage while the next thumbnail loads", async () => {
let resolveNext: ((url: string) => void) | undefined;
const h = makeHarness([{ ...BUSY, thumbnailAttachmentId: "att-1" }]);
h.loadImage.mockImplementation(async (id: string) => {
if (id === "att-2") {
return new Promise<string>((resolve) => {
resolveNext = resolve;
});
}
return `blob:${id}`;
});
render(<ObservationOverlay store={h.store} />);
const img = await screen.findByRole("img", { name: "session s1 view" });
expect(img.getAttribute("src")).toBe("blob:att-1");
h.push([{ ...BUSY, thumbnailAttachmentId: "att-2" }]);
await waitFor(() => expect(h.loadImage).toHaveBeenCalledWith("att-2"));
// Same <img> node, same src — no placeholder flash, no remount/fade.
expect(screen.queryByText("waiting for page")).toBeNull();
expect(screen.getByRole("img", { name: "session s1 view" })).toBe(img);
expect(img.getAttribute("src")).toBe("blob:att-1");
resolveNext?.("blob:att-2");
await waitFor(() => expect(img.getAttribute("src")).toBe("blob:att-2"));
expect(screen.getByRole("img", { name: "session s1 view" })).toBe(img);
});
it("interrupt: disabled while idle, active call posts and shows progress", async () => {
const h = makeHarness([{ sessionId: "s1", action: "idle", since: Date.now() }]);
render(<ObservationOverlay store={h.store} />);
const button = await screen.findByRole("button", { name: /Interrupt the current/ });
expect(button).toHaveProperty("disabled", true);
h.push([BUSY]);
await waitFor(() => expect(button).toHaveProperty("disabled", false));
fireEvent.click(button);
await waitFor(() =>
expect(h.fetches.some((f) => f.url === "/bsk-observation/interrupt")).toBe(true),
);
});
it("shows a hover tooltip on the interrupt icon and keeps the button icon-only", async () => {
const h = makeHarness([BUSY]);
render(<ObservationOverlay store={h.store} />);
const button = await screen.findByRole("button", { name: /Interrupt the current/ });
expect(button.textContent ?? "").not.toMatch(/Interrupt/i);
fireEvent.pointerEnter(button.parentElement as HTMLElement);
const tip = await screen.findByRole("tooltip");
expect(tip.textContent).toBe("Stop the current browser action.");
fireEvent.pointerLeave(button.parentElement as HTMLElement);
expect(screen.queryByRole("tooltip")).toBeNull();
fireEvent.pointerEnter(button.parentElement as HTMLElement);
expect(screen.getByRole("tooltip")).toBeTruthy();
});
it("collapses to a capsule and expands back", async () => {
const h = makeHarness([BUSY]);
render(<ObservationOverlay store={h.store} />);
await screen.findByText(/s1 · clicking/);
fireEvent.click(screen.getByRole("button", { name: "Collapse" }));
const capsule = await screen.findByRole("button", {
name: "Expand browser observation overlay",
});
expect(capsule.textContent).toContain("clicking");
fireEvent.click(capsule);
await screen.findByText(/s1 · clicking/);
});
it("resizes within min/max clamps", async () => {
const h = makeHarness([BUSY]);
render(<ObservationOverlay store={h.store} />);
const card = await screen.findByTestId("obs-card");
card.getBoundingClientRect = () =>
({
x: 0,
y: 0,
left: 0,
top: 0,
width: 320,
height: 240,
right: 320,
bottom: 240,
toJSON: () => {},
}) as DOMRect;
expect(Number.parseFloat(card.style.width)).toBe(320);
fireEvent.pointerDown(screen.getByTestId("obs-resize-se"), { clientX: 100, clientY: 100 });
fireEvent.pointerMove(document, { clientX: 0, clientY: 0 });
fireEvent.pointerUp(document);
expect(Number.parseFloat(card.style.width)).toBe(240); // min width clamp
fireEvent.pointerDown(screen.getByTestId("obs-resize-se"), { clientX: 100, clientY: 100 });
fireEvent.pointerMove(document, { clientX: 5000, clientY: 5000 });
fireEvent.pointerUp(document);
expect(Number.parseFloat(card.style.width)).toBe(window.innerWidth * 0.8);
});
it("exposes an invisible resize hit target on every corner", async () => {
const h = makeHarness([BUSY]);
render(<ObservationOverlay store={h.store} />);
await screen.findByTestId("obs-card");
for (const corner of ["nw", "ne", "sw", "se"]) {
expect(screen.getByTestId(`obs-resize-${corner}`)).toBeTruthy();
}
});
it("moves the card with header drags, clamped to the viewport", async () => {
const h = makeHarness([BUSY]);
render(<ObservationOverlay store={h.store} />);
const card = await screen.findByTestId("obs-card");
fireEvent.pointerDown(screen.getByTestId("obs-header"), { clientX: 50, clientY: 50 });
fireEvent.pointerMove(document, { clientX: -500, clientY: -500 });
fireEvent.pointerUp(document);
expect(card.style.left).toBe("0px");
expect(card.style.top).toBe("0px");
});
it("hides Pop out when Document PiP is unsupported", async () => {
const h = makeHarness([BUSY]);
render(<ObservationOverlay store={h.store} />);
await screen.findByText(/s1 · clicking/);
expect(screen.queryByRole("button", { name: /Pop out/ })).toBeNull();
});
it("pops out with the card's current size and falls back on pagehide", async () => {
const h = makeHarness([BUSY]);
const pipDoc = document.implementation.createHTMLDocument("pip");
const listeners = new Map<string, (() => void)[]>();
const requestWindow = vi.fn(async (options?: { width?: number; height?: number }) => {
void options;
return {
document: pipDoc,
addEventListener: (name: string, fn: () => void) => {
listeners.set(name, [...(listeners.get(name) ?? []), fn]);
},
} as unknown as Window;
});
(window as unknown as Record<string, unknown>).documentPictureInPicture = { requestWindow };
render(<ObservationOverlay store={h.store} />);
const popout = await screen.findByRole("button", { name: /Pop out/ });
fireEvent.click(popout);
await waitFor(() => expect(requestWindow).toHaveBeenCalledWith({ width: 320, height: 240 }));
// Content now renders inside the PiP document.
await waitFor(() => expect(pipDoc.body.textContent).toContain("s1"));
// Closing the PiP returns to the in-page card with state preserved.
for (const fn of listeners.get("pagehide") ?? []) fn();
await screen.findByTestId("obs-card");
});
it("renders the strip for two sessions and pins focus on click", async () => {
const older: SessionObservation = {
sessionId: "s1",
action: "idle",
since: Date.now() - 60000,
};
const newer: SessionObservation = { sessionId: "s2", action: "clicking", since: Date.now() };
const h = makeHarness([older, newer]);
render(<ObservationOverlay store={h.store} />);
// Auto-follows the most recently active session.
await screen.findByText(/s2 · clicking/);
const strip = screen.getByTestId("obs-strip");
expect(strip.textContent).toContain("s1");
expect(strip.textContent).toContain("s2");
// Pin s1: focus moves and stays even when s2 gets newer activity.
fireEvent.click(screen.getByRole("button", { name: "Pin session s1" }));
await screen.findByText(/s1 · idle/);
h.push([{ ...newer, since: Date.now() + 5000, action: "filling" }]);
await waitFor(() => expect(screen.queryByText(/s2 · filling/)).toBeNull());
expect(screen.getByText(/s1 · idle/)).toBeTruthy();
// Unpin: auto-follow resumes.
fireEvent.click(screen.getByRole("button", { name: "Unpin session s1" }));
await screen.findByText(/s2 · filling/);
});
it("flags an errored strip item without stealing focus", async () => {
const s1: SessionObservation = { sessionId: "s1", action: "idle", since: Date.now() - 30000 };
const s2: SessionObservation = { sessionId: "s2", action: "idle", since: Date.now() - 10000 };
const h = makeHarness([s1, s2]);
render(<ObservationOverlay store={h.store} />);
await screen.findByText(/s2 · idle/);
// s2's latest action failed: red edge on its strip item, focus stays on s2 already…
// now push an even-more-recent ERROR for s1 — focus must NOT jump to it.
h.push([
{ ...s2, since: Date.now() - 5000 },
{ ...s1, since: Date.now(), lastError: "click failed" },
]);
await waitFor(() =>
expect(
screen.getByRole("button", { name: "Pin session s1" }).closest("[data-state]"),
).toHaveProperty("dataset", expect.objectContaining({ state: "error" })),
);
expect(screen.getByText(/s2 · idle/)).toBeTruthy();
});
it("shows browser-unavailable with a kept frame and a greyed interrupt", async () => {
const h = makeHarness([{ ...BUSY, thumbnailAttachmentId: "att-1" }]);
render(<ObservationOverlay store={h.store} />);
await waitFor(() => expect(h.loadImage).toHaveBeenCalledWith("att-1"));
await screen.findByRole("img", { name: "session s1 view" });
h.emitRaw({ type: "availability", available: false });
await screen.findByText("browser unavailable");
// Last frame stays on stage; interrupt is greyed.
expect(screen.getByRole("img", { name: "session s1 view" })).toBeTruthy();
expect(screen.getByRole("button", { name: /Interrupt the current/ })).toHaveProperty(
"disabled",
true,
);
});
it("greys out a dead session in the strip and skips it for focus", async () => {
const s1: SessionObservation = { sessionId: "s1", action: "idle", since: Date.now() - 20000 };
const s2: SessionObservation = { sessionId: "s2", action: "idle", since: Date.now() - 10000 };
const h = makeHarness([s1, s2]);
render(<ObservationOverlay store={h.store} />);
await screen.findByText(/s2 · idle/);
h.push([{ ...s1, dead: true, since: Date.now() }, s2]);
await waitFor(() =>
expect(
screen.getByRole("button", { name: "Pin session s1" }).closest("[data-state]"),
).toHaveProperty("dataset", expect.objectContaining({ state: "dead" })),
);
// Dead sessions never take the stage.
expect(screen.getByText(/s2 · idle/)).toBeTruthy();
});
it("interrupts a strip session directly on hover without focusing it", async () => {
const s1: SessionObservation = {
sessionId: "s1",
action: "clicking",
since: Date.now() - 30000,
};
const s2: SessionObservation = { sessionId: "s2", action: "idle", since: Date.now() };
const h = makeHarness([s1, s2]);
render(<ObservationOverlay store={h.store} />);
await screen.findByText(/s2 · idle/);
const item = screen.getByRole("button", { name: "Pin session s1" }).closest("div");
if (item === null) throw new Error("no strip item");
fireEvent.pointerEnter(item);
const mini = await screen.findByRole("button", { name: "Interrupt session s1" });
fireEvent.click(mini);
await waitFor(() =>
expect(
h.fetches.some(
(f) => f.url === "/bsk-observation/interrupt" && f.init?.body === '{"sessionId":"s1"}',
),
).toBe(true),
);
// Focus did not move to s1.
expect(screen.getByText(/s2 · idle/)).toBeTruthy();
});
});
describe("applyResize", () => {
const base = { x: 200, y: 100, w: 320, h: 240 };
const viewport = { w: 1000, h: 800 };
it("grows the planted opposite corner from each handle", () => {
expect(applyResize(base, "se", 20, 10, viewport)).toEqual({
pos: { x: 200, y: 100 },
size: { w: 340, h: 250 },
});
expect(applyResize(base, "nw", -20, -10, viewport)).toEqual({
pos: { x: 180, y: 90 },
size: { w: 340, h: 250 },
});
expect(applyResize(base, "ne", 20, -10, viewport)).toEqual({
pos: { x: 200, y: 90 },
size: { w: 340, h: 250 },
});
expect(applyResize(base, "sw", -20, 10, viewport)).toEqual({
pos: { x: 180, y: 100 },
size: { w: 340, h: 250 },
});
});
it("clamps to the minimum size without drifting the planted corner", () => {
expect(applyResize(base, "se", -200, -200, viewport)).toEqual({
pos: { x: 200, y: 100 },
size: { w: 240, h: 180 },
});
expect(applyResize(base, "nw", 200, 200, viewport)).toEqual({
pos: { x: 280, y: 160 },
size: { w: 240, h: 180 },
});
});
});
@@ -0,0 +1,236 @@
// ObservationClientStore: initial fetch + SSE application, thumbnail loading
// lifecycle, interrupt wire shape. All I/O faked.
import { describe, expect, it, vi } from "vitest";
import { type EventSourceLike, ObservationClientStore } from "../../src/client/observation-store";
import type { ObservationEvent, SessionObservation } from "../../src/observation";
const OBS_IDLE: SessionObservation = { sessionId: "s1", action: "idle", since: 1000 };
const OBS_BUSY: SessionObservation = { sessionId: "s1", action: "clicking", since: 2000 };
function harness(state: SessionObservation[] = []) {
const fetches: { url: string; init?: { method?: string; body?: string } }[] = [];
const esInstances: EventSourceLike[] = [];
const fetchFn = vi.fn(async (url: string, init?: { method?: string; body?: string }) => {
fetches.push({ url, init });
if (url === "/bsk-observation/state") {
return { ok: true, json: async () => ({ sessions: state }) };
}
return { ok: true, json: async () => ({ interrupted: true }) };
});
const eventSourceFactory = (url: string) => {
const es: EventSourceLike = { onmessage: null, close: vi.fn() };
esInstances.push(es);
return es;
};
const loadImage = vi.fn(async (id: string) => `blob:url-${id}`);
const store = new ObservationClientStore({ fetchFn, eventSourceFactory, loadImage });
return { store, fetches, esInstances, loadImage };
}
function emit(es: EventSourceLike, event: ObservationEvent): void {
es.onmessage?.({ data: JSON.stringify(event) });
}
describe("ObservationClientStore", () => {
it("loads the initial state and applies SSE increments", async () => {
const { store, esInstances } = harness([OBS_IDLE]);
const seen: number[] = [];
store.subscribe(() => seen.push(store.getSnapshot().sessions.length));
store.start();
await vi.waitFor(() => expect(store.getSnapshot().sessions).toHaveLength(1));
expect(esInstances).toHaveLength(1);
emit(esInstances[0], { type: "upsert", session: OBS_BUSY });
expect(store.getSnapshot().sessions[0].action).toBe("clicking");
emit(esInstances[0], {
type: "upsert",
session: { sessionId: "s2", action: "idle", since: 3 },
});
expect(store.getSnapshot().sessions).toHaveLength(2);
emit(esInstances[0], {
type: "remove",
session: { sessionId: "s2", action: "idle", since: 0 },
});
expect(store.getSnapshot().sessions).toHaveLength(1);
emit(esInstances[0], { type: "reset" });
expect(store.getSnapshot().sessions).toHaveLength(0);
expect(seen.length).toBeGreaterThan(0);
});
it("ignores malformed SSE frames", () => {
const { store, esInstances } = harness([]);
store.start();
esInstances[0].onmessage?.({ data: "not json" });
expect(store.getSnapshot().sessions).toHaveLength(0);
});
it("loads each thumbnail once and reports ready", async () => {
const { store, loadImage } = harness();
store.ensureThumbnail("att-1");
store.ensureThumbnail("att-1");
expect(loadImage).toHaveBeenCalledTimes(1);
await vi.waitFor(() =>
expect(store.getSnapshot().thumbnails["att-1"]).toEqual({
status: "ready",
url: "blob:url-att-1",
}),
);
});
it("marks failed thumbnail loads as error", async () => {
const { store, loadImage } = harness();
loadImage.mockRejectedValueOnce(new Error("nope"));
store.ensureThumbnail("att-bad");
await vi.waitFor(() => expect(store.getSnapshot().thumbnails["att-bad"].status).toBe("error"));
});
it("posts interrupt with and without a session id", async () => {
const { store, fetches } = harness();
expect(await store.interrupt()).toBe(true);
expect(await store.interrupt("s1")).toBe(true);
const calls = fetches.filter((f) => f.url === "/bsk-observation/interrupt");
expect(JSON.parse(calls[0].init?.body ?? "")).toEqual({});
expect(JSON.parse(calls[1].init?.body ?? "")).toEqual({ sessionId: "s1" });
});
it("returns false when interrupt fails", async () => {
const failing = new ObservationClientStore({
fetchFn: async () => ({ ok: false, json: async () => ({}) }),
eventSourceFactory: () => ({ onmessage: null, close: () => {} }),
loadImage: async () => "blob:x",
});
expect(await failing.interrupt("s1")).toBe(false);
});
it("stop closes the stream and clears state", async () => {
const { store, esInstances } = harness([OBS_IDLE]);
store.start();
await vi.waitFor(() => expect(store.getSnapshot().sessions).toHaveLength(1));
store.stop();
expect(esInstances[0].close).toHaveBeenCalled();
expect(store.getSnapshot().sessions).toHaveLength(0);
});
});
describe("thumbnail blob URL lifecycle", () => {
function withRevokeSpy() {
const revoke = vi.fn();
(URL as unknown as { revokeObjectURL: unknown }).revokeObjectURL = revoke;
return revoke;
}
it("holds the last ready frame until the replacement decodes", async () => {
const revoke = withRevokeSpy();
const { store, esInstances } = harness([{ ...OBS_IDLE, thumbnailAttachmentId: "a1" }]);
store.start();
await vi.waitFor(() => expect(store.getSnapshot().sessions).toHaveLength(1));
store.ensureThumbnail("a1");
await vi.waitFor(() =>
expect(store.getSnapshot().thumbnails["a1"]).toEqual({ status: "ready", url: "blob:url-a1" }),
);
emit(esInstances[0], {
type: "upsert",
session: { ...OBS_IDLE, thumbnailAttachmentId: "a2" },
});
// Old blob stays painted so the overlay does not flash a placeholder.
expect(store.getSnapshot().thumbnails["a1"]).toEqual({
status: "ready",
url: "blob:url-a1",
});
expect(store.getSnapshot().displayFrames.s1).toEqual({
status: "ready",
url: "blob:url-a1",
});
expect(revoke).not.toHaveBeenCalled();
store.ensureThumbnail("a2");
await vi.waitFor(() =>
expect(store.getSnapshot().displayFrames.s1).toEqual({
status: "ready",
url: "blob:url-a2",
}),
);
expect(store.getSnapshot().thumbnails["a1"]).toBeUndefined();
expect(revoke).toHaveBeenCalledWith("blob:url-a1");
store.stop();
});
it("keeps the last good frame in displayFrames when the next load fails", async () => {
const { store, esInstances, loadImage } = harness([
{ ...OBS_IDLE, thumbnailAttachmentId: "a1" },
]);
store.start();
await vi.waitFor(() => expect(store.getSnapshot().sessions).toHaveLength(1));
store.ensureThumbnail("a1");
await vi.waitFor(() => expect(store.getSnapshot().thumbnails["a1"].status).toBe("ready"));
loadImage.mockRejectedValueOnce(new Error("nope"));
emit(esInstances[0], {
type: "upsert",
session: { ...OBS_IDLE, thumbnailAttachmentId: "a2" },
});
store.ensureThumbnail("a2");
await vi.waitFor(() => expect(store.getSnapshot().thumbnails["a2"]?.status).toBe("error"));
expect(store.getSnapshot().displayFrames.s1).toEqual({
status: "error",
url: "blob:url-a1",
});
store.stop();
});
it("remove and reset revoke the session's tracked frame", async () => {
const revoke = withRevokeSpy();
const { store, esInstances } = harness([{ ...OBS_IDLE, thumbnailAttachmentId: "a1" }]);
store.start();
await vi.waitFor(() => expect(store.getSnapshot().sessions).toHaveLength(1));
store.ensureThumbnail("a1");
await vi.waitFor(() => expect(store.getSnapshot().thumbnails["a1"].status).toBe("ready"));
emit(esInstances[0], {
type: "remove",
session: { sessionId: "s1", action: "idle", since: 0 },
});
expect(revoke).toHaveBeenCalledWith("blob:url-a1");
expect(store.getSnapshot().thumbnails["a1"]).toBeUndefined();
store.stop();
});
it("a load that finishes after replacement never resurrects the old frame", async () => {
const revoke = withRevokeSpy();
const { store, esInstances, loadImage } = harness([
{ ...OBS_IDLE, thumbnailAttachmentId: "a1" },
]);
let resolveA1!: (url: string) => void;
loadImage.mockImplementation((id: string) =>
id === "a1"
? new Promise<string>((resolve) => {
resolveA1 = resolve;
})
: Promise.resolve(`blob:url-${id}`),
);
store.start();
await vi.waitFor(() => expect(store.getSnapshot().sessions).toHaveLength(1));
store.ensureThumbnail("a1");
// the frame is replaced while a1 is still loading
emit(esInstances[0], {
type: "upsert",
session: { ...OBS_IDLE, thumbnailAttachmentId: "a2" },
});
resolveA1("blob:url-a1-late");
await new Promise((resolve) => setImmediate(resolve));
expect(store.getSnapshot().thumbnails["a1"]).toBeUndefined();
expect(revoke).toHaveBeenCalledWith("blob:url-a1-late");
store.stop();
});
});
describe("SSE (re)open resync", () => {
it("refetches the full state when the stream (re)opens", async () => {
const { store, fetches, esInstances } = harness([OBS_IDLE]);
store.start();
await vi.waitFor(() => expect(store.getSnapshot().sessions).toHaveLength(1));
const before = fetches.filter((f) => f.url === "/bsk-observation/state").length;
esInstances[0].onopen?.();
await vi.waitFor(() =>
expect(fetches.filter((f) => f.url === "/bsk-observation/state").length).toBe(before + 1),
);
store.stop();
});
});
@@ -0,0 +1,154 @@
// @vitest-environment happy-dom
// browser_screenshot keyed toolview: view-model derivation from frozen blocks,
// image/path-only/running/error rendering, the registration key, and the
// session-bound attachment loader.
import type { RunningToolCall, ToolResultNode } from "@deepseek-ai/dsh-client-runtime/client";
import { cleanup, render, screen, waitFor } from "@testing-library/react";
import { afterEach, describe, expect, it, vi } from "vitest";
import { apply } from "../../src/client/index";
import {
ScreenshotToolView,
type ScreenshotToolViewProps,
viewModelOf,
} from "../../src/client/ScreenshotToolView";
const ATTACHMENT = {
attachmentId: "sha256:abc123",
mediaType: "image/png",
bytes: 8,
width: 800,
height: 457,
name: "screenshot-s1.png",
} as never;
function runningBlock(argsRaw: string): RunningToolCall {
return {
callId: "c1",
name: "browser_screenshot",
argsRaw,
turn: 1,
step: 1,
time: 0,
callView: null,
subCalls: [],
} as never;
}
function settledBlock(argsRaw: string, content: unknown[], isError = false): ToolResultNode {
return {
kind: "tool-result",
seq: 1,
time: 0,
callId: "c1",
call: { name: "browser_screenshot", argsRaw },
callTime: 0,
content,
isError,
callView: null,
resultView: null,
subCalls: [],
} as never;
}
const IMAGE_TEXT = "[session s1] screenshot of tab 7 (800x457px)";
const PATH_TEXT = "[session s1] screenshot saved to /tmp/shot.png (800x457px, 8 bytes)";
afterEach(cleanup);
describe("viewModelOf", () => {
it("derives the running model from the call frame", () => {
const model = viewModelOf(runningBlock('{"session":"s1"}'));
expect(model.state).toBe("running");
expect(model.command).toBe("bsk screenshot --session s1");
expect(model.image).toBeNull();
});
it("marks settled results carrying an image block", () => {
const block = settledBlock("{}", [
{ type: "text", text: IMAGE_TEXT },
{ type: "image", attachment: ATTACHMENT },
]);
const model = viewModelOf(block);
expect(model.state).toBe("ok");
expect(model.image).toBe(ATTACHMENT);
expect(model.output).toBe(IMAGE_TEXT);
expect(model.command).toBe("bsk screenshot --session (current)");
});
it("keeps the path-only form image-free", () => {
const model = viewModelOf(settledBlock("{}", [{ type: "text", text: PATH_TEXT }]));
expect(model.state).toBe("ok");
expect(model.image).toBeNull();
expect(model.summary).toBe(PATH_TEXT);
});
it("marks error results", () => {
const model = viewModelOf(settledBlock("{}", [{ type: "text", text: "Error: boom" }], true));
expect(model.state).toBe("error");
});
});
describe("ScreenshotToolView", () => {
const baseProps = { callId: "c1", toolName: "browser_screenshot", openFile: () => {} };
const renderView = (
block: ScreenshotToolViewProps["block"],
loadImage: ScreenshotToolViewProps["loadImage"],
) => {
const props = { ...baseProps, block, loadImage } as unknown as ScreenshotToolViewProps;
return render(<ScreenshotToolView {...props} />);
};
it("renders the screenshot image through the loader", async () => {
const block = settledBlock("{}", [
{ type: "text", text: IMAGE_TEXT },
{ type: "image", attachment: ATTACHMENT },
]);
const loadImage = vi.fn(async () => "blob:mock-url");
renderView(block, loadImage);
// Collapsed row shows the summary; expand to reach the image.
expect(screen.getByText(IMAGE_TEXT)).toBeTruthy();
const toggle = screen.getByRole("button", { name: /screenshot/i });
toggle.click();
await waitFor(() => expect(loadImage).toHaveBeenCalledWith(ATTACHMENT));
await screen.findByRole("img", { name: "screenshot-s1.png" });
});
it("renders the path-only form without touching the loader", async () => {
const block = settledBlock("{}", [{ type: "text", text: PATH_TEXT }]);
const loadImage = vi.fn(async () => "blob:unused");
renderView(block, loadImage);
screen.getByRole("button", { name: /screenshot/i }).click();
await screen.findByText(PATH_TEXT, { exact: false });
expect(loadImage).not.toHaveBeenCalled();
});
it("renders the running form without output", () => {
const loadImage = vi.fn();
renderView(runningBlock("{}"), loadImage);
expect(screen.getByText("bsk screenshot --session (current)")).toBeTruthy();
expect(loadImage).not.toHaveBeenCalled();
});
});
describe("client plugin registration", () => {
it("registers the browser_screenshot toolview and the shell observation overlay", () => {
const registrations: { name: string; key?: string; id?: string }[] = [];
const sessions = { binding: () => undefined };
const ctx = {
get: (key: string) => (key === "sessions" ? sessions : undefined),
slots: {
inject: (_name: string, fn: () => unknown) => fn(),
register: (slot: { name: string; key?: string; id?: string }, view: unknown) => {
registrations.push(slot);
expect(typeof view).toBe("function");
},
},
};
apply(ctx as never);
expect(registrations).toEqual([
{ name: "tool.call.toolview", key: "browser_screenshot" },
{ name: "shell.overlay", id: "bsk-observation" },
]);
});
});
@@ -0,0 +1,94 @@
// Unload-cleanup ownership discipline: dispose must stop exactly the sessions
// this plugin created — never referenced/foreign sessions on the shared
// daemon — and a stale handle (already stopped elsewhere, daemon restarted)
// must not abort the remaining stops.
import type { ToolDefinition, ToolRunContext } from "@deepseek-ai/dsh-tools";
import { describe, expect, it } from "vitest";
import { apply } from "../src/index";
import type { BskRunOptions, BskRunResult } from "../src/runner";
interface FakeCall {
args: string[];
}
function ok(payload: unknown): BskRunResult {
return { code: 0, stdout: JSON.stringify(payload), stderr: "", timedOut: false, aborted: false };
}
function fail(message: string): BskRunResult {
return {
code: 4,
stdout: JSON.stringify({ code: "session_not_found", message }),
stderr: "",
timedOut: false,
aborted: false,
};
}
function makeExec(): ToolRunContext {
return {
callId: "t",
name: "t",
signal: new AbortController().signal,
} as unknown as ToolRunContext;
}
describe("dispose cleanup ownership", () => {
it("stops only plugin-owned sessions and tolerates stale handles", async () => {
const calls: FakeCall[] = [];
const startReplies = [
{ session_id: "own1", browser_instance_id: "b1" },
{ session_id: "own2", browser_instance_id: "b1" },
];
const runner = {
async run(args: string[], _options: BskRunOptions = {}): Promise<BskRunResult> {
calls.push({ args });
const joined = args.join(" ");
if (joined.startsWith("status")) return ok({});
if (joined.startsWith("session start")) return ok(startReplies.shift());
if (joined.startsWith("snapshot")) {
return ok({ text: "x", ref_count: 1, tab_id: 7, truncated: false });
}
if (joined === "session stop own1") return ok({});
if (joined === "session stop own2") {
// Stale handle: already stopped by someone else / daemon restarted.
return fail("session not found");
}
return fail(`unexpected command: ${joined}`);
},
killAll() {},
killFor: () => 0,
};
const tools = new Map<string, ToolDefinition>();
const disposers: Array<() => Promise<void>> = [];
const ctx = {
tools: { register: (def: ToolDefinition) => tools.set(def.name, def) },
get: () => undefined,
// No webServer in this harness: the inject callback never runs.
inject: () => {},
effect: (fn: () => () => Promise<void>) => {
disposers.push(fn());
},
};
apply(ctx as never, { maxSessions: 5, lazyTools: false }, { runnerFactory: () => runner });
const start = tools.get("browser_session_start");
const snapshot = tools.get("browser_snapshot");
if (start === undefined || snapshot === undefined) throw new Error("tools not registered");
await start.execute({}, makeExec());
await start.execute({}, makeExec());
// A foreign session cannot even be referenced any more.
await expect(snapshot.execute({ session: "ext9" }, makeExec())).rejects.toThrow(
/does not belong to this plugin/,
);
await Promise.all(disposers.map((dispose) => dispose()));
const stops = calls
.map((call) => call.args.join(" "))
.filter((joined) => joined.startsWith("session stop"));
expect(stops.sort()).toEqual(["session stop own1", "session stop own2"]);
expect(stops.some((joined) => joined.includes("ext9"))).toBe(false);
});
});
@@ -0,0 +1,229 @@
// Lazy tool-schema injection: suite hidden before a successful skill
// invocation, revealed by the model trigger / user gesture / session history,
// idempotent on repeats, and torn down cleanly. Plus the apply-level
// lazyTools two-state wiring.
import type { ToolDefinition, ToolRunContext } from "@deepseek-ai/dsh-tools";
import { describe, expect, it, vi } from "vitest";
import { apply } from "../src/index";
import { armLazyTools, hasSuccessfulSkillInvocation } from "../src/lazy-tools";
import type { BskRunOptions, BskRunResult } from "../src/runner";
function fakeEventCtx(sessions?: { list(): { events: unknown[] }[] }) {
const listeners = new Map<string, (...args: never[]) => void>();
const ctx = {
on: (event: string, listener: (...args: never[]) => void) => {
listeners.set(event, listener);
return () => listeners.delete(event);
},
get: (key: string) => (key === "sessions" ? sessions : undefined),
};
return { ctx: ctx as never, listeners };
}
function callListeners(
listeners: Map<string, (...args: never[]) => void>,
event: string,
...args: unknown[]
): void {
(listeners.get(event) as ((...a: unknown[]) => void) | undefined)?.(...args);
}
describe("armLazyTools", () => {
it("keeps the suite hidden until a successful skill invocation reveals it", () => {
const { ctx, listeners } = fakeEventCtx();
const registerSuite = vi.fn(() => () => {});
armLazyTools(ctx, registerSuite);
expect(registerSuite).not.toHaveBeenCalled();
// wrong tool
callListeners(listeners, "tools/result", { name: "bash", arguments: {} }, { isError: false });
// wrong skill name
callListeners(
listeners,
"tools/result",
{ name: "skill", arguments: { name: "other-skill" } },
{ isError: false },
);
// failed invocation
callListeners(
listeners,
"tools/result",
{ name: "skill", arguments: { name: "browser-skill" } },
{ isError: true },
);
expect(registerSuite).not.toHaveBeenCalled();
// the real trigger (arguments may arrive as a raw JSON string)
callListeners(
listeners,
"tools/result",
{ name: "skill", arguments: '{"name":"browser-skill"}' },
{ isError: false },
);
expect(registerSuite).toHaveBeenCalledTimes(1);
});
it("is idempotent across repeated invocations", () => {
const { ctx, listeners } = fakeEventCtx();
const registerSuite = vi.fn(() => () => {});
armLazyTools(ctx, registerSuite);
for (let i = 0; i < 3; i++) {
callListeners(
listeners,
"tools/result",
{ name: "skill", arguments: { name: "browser-skill" } },
{ isError: false },
);
}
expect(registerSuite).toHaveBeenCalledTimes(1);
});
it("reveals on a /browser-skill user gesture (skill-invocation message)", () => {
const { ctx, listeners } = fakeEventCtx();
const registerSuite = vi.fn(() => () => {});
armLazyTools(ctx, registerSuite);
callListeners(
listeners,
"session/event",
{ events: [] },
{
type: "user/message",
data: { source: { kind: "skill-invocation", name: "browser-skill" } },
},
);
expect(registerSuite).toHaveBeenCalledTimes(1);
});
it("restores from session history: hit at boot, on session entry, and miss", () => {
const hitEvents = [
{
type: "tool/call",
data: { callId: "c1", name: "skill", arguments: '{"name":"browser-skill"}' },
},
{ type: "tool/result", data: { message: { callId: "c1", isError: false } } },
];
// boot-time live sessions are scanned
const boot = fakeEventCtx({ list: () => [{ events: hitEvents }] });
const bootRegister = vi.fn(() => () => {});
armLazyTools(boot.ctx, bootRegister);
expect(bootRegister).toHaveBeenCalledTimes(1);
// a session entered later with the same proof
const { ctx, listeners } = fakeEventCtx({ list: () => [] });
const registerSuite = vi.fn(() => () => {});
armLazyTools(ctx, registerSuite);
callListeners(listeners, "session/created", { events: hitEvents });
expect(registerSuite).toHaveBeenCalledTimes(1);
// miss: failed result proves nothing
const miss = fakeEventCtx({
list: () => [
{
events: [
{
type: "tool/call",
data: { callId: "c1", name: "skill", arguments: '{"name":"browser-skill"}' },
},
{ type: "tool/result", data: { message: { callId: "c1", isError: true } } },
],
},
],
});
const missRegister = vi.fn(() => () => {});
armLazyTools(miss.ctx, missRegister);
expect(missRegister).not.toHaveBeenCalled();
});
it("disposes listeners and the revealed suite", () => {
const { ctx, listeners } = fakeEventCtx();
const suiteDispose = vi.fn();
const disarm = armLazyTools(ctx, () => suiteDispose);
callListeners(
listeners,
"tools/result",
{ name: "skill", arguments: { name: "browser-skill" } },
{ isError: false },
);
disarm();
expect(suiteDispose).toHaveBeenCalledTimes(1);
expect(listeners.size).toBe(0);
});
});
describe("hasSuccessfulSkillInvocation", () => {
it("pairs call and result by callId and honors gestures", () => {
expect(hasSuccessfulSkillInvocation([])).toBe(false);
expect(
hasSuccessfulSkillInvocation([
{
type: "tool/call",
data: { callId: "x", name: "skill", arguments: { name: "browser-skill" } },
},
{ type: "tool/result", data: { message: { callId: "x", isError: false } } },
]),
).toBe(true);
// result for an unrelated call id does not count
expect(
hasSuccessfulSkillInvocation([
{
type: "tool/call",
data: { callId: "x", name: "skill", arguments: { name: "browser-skill" } },
},
{ type: "tool/result", data: { message: { callId: "y", isError: false } } },
]),
).toBe(false);
expect(
hasSuccessfulSkillInvocation([
{
type: "user/message",
data: { source: { kind: "skill-invocation", name: "browser-skill" } },
},
]),
).toBe(true);
});
});
describe("lazyTools wiring in apply()", () => {
function applyHarness(config: Record<string, unknown>) {
const tools = new Map<string, ToolDefinition>();
const listeners = new Map<string, (...args: never[]) => void>();
const ctx = {
tools: { register: (def: ToolDefinition) => tools.set(def.name, def) },
get: () => undefined,
inject: () => {},
effect: () => {},
on: (event: string, listener: (...args: never[]) => void) => {
listeners.set(event, listener);
return () => listeners.delete(event);
},
};
const runner = {
async run(): Promise<BskRunResult> {
return { code: 0, stdout: "{}", stderr: "", timedOut: false, aborted: false };
},
killAll() {},
killFor: () => 0,
};
apply(ctx as never, config, { runnerFactory: () => runner as never });
return { tools, listeners };
}
it("lazyTools: false registers the suite at apply time", () => {
const { tools } = applyHarness({ lazyTools: false });
expect(tools.has("browser_session_start")).toBe(true);
expect(tools.has("browser_emulate")).toBe(true);
});
it("lazyTools: true hides the suite until the skill fires", () => {
const { tools, listeners } = applyHarness({ lazyTools: true });
expect(tools.has("browser_session_start")).toBe(false);
callListeners(
listeners,
"tools/result",
{ name: "skill", arguments: { name: "browser-skill" } },
{ isError: false },
);
expect(tools.has("browser_session_start")).toBe(true);
});
});
@@ -0,0 +1,644 @@
// ObservationService host tests: state machine, cadence/backoff, observation-
// traffic isolation, interrupt routing, the owned boundary, and the HTTP/SSE
// interface. All bsk runs are faked; the scheduler is a manual timer queue.
import { existsSync, mkdtempSync, writeFileSync } from "node:fs";
import type { IncomingMessage, ServerResponse } from "node:http";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { describe, expect, it } from "vitest";
import {
actionForLabel,
type ObservationEvent,
type ObservationScheduler,
ObservationService,
} from "../src/observation";
import { registerObservationRoutes } from "../src/observation-http";
import { KeyedExecutor } from "../src/queue";
import type { BskRunner, BskRunOptions, BskRunResult } from "../src/runner";
import { SessionRegistry } from "../src/sessions";
const OPTIONS = { enabled: true, thumbnailIntervalMs: 1500, idleIntervalMs: 8000 };
/** Manual timer queue + controllable clock. */
function fakeScheduler() {
let now = 1_000_000;
const timers: { handle: object; fn: () => void; ms: number }[] = [];
const scheduler: ObservationScheduler & {
advance: (ms: number) => void;
runNext: () => void;
pending: () => number[];
} = {
now: () => now,
setTimeout: (fn, ms) => {
const handle = { fn, ms };
timers.push({ handle, fn, ms });
return handle;
},
clearTimeout: (h) => {
const index = timers.findIndex((t) => t.handle === h);
if (index >= 0) timers.splice(index, 1);
},
advance: (ms) => {
now += ms;
},
runNext: () => {
const next = timers.shift();
next?.fn();
},
pending: () => timers.map((t) => t.ms),
};
return scheduler;
}
interface FakeRunner extends BskRunner {
calls: { args: string[]; options: BskRunOptions }[];
killed: string[];
}
/** Runner that answers screenshot with a real temp PNG and records kills. */
function fakeRunner(
opts: { screenshotFails?: boolean; screenshotNotFound?: boolean; killCount?: number } = {},
): FakeRunner {
const calls: FakeRunner["calls"] = [];
const killed: string[] = [];
return {
calls,
killed,
async run(args: string[], options: BskRunOptions = {}): Promise<BskRunResult> {
calls.push({ args, options });
if (args[0] === "screenshot") {
if (opts.screenshotNotFound) {
return {
code: 4,
stdout: JSON.stringify({ code: "session_not_found", message: "no such session" }),
stderr: "",
timedOut: false,
aborted: false,
};
}
if (opts.screenshotFails) {
return { code: 1, stdout: "", stderr: "boom", timedOut: false, aborted: false };
}
const outIndex = args.indexOf("--out") + 1;
const out = args[outIndex];
writeFileSync(out, Buffer.from([0x89, 0x50, 0x4e, 0x47]));
return {
code: 0,
stdout: JSON.stringify({
tab_id: 7,
width: 4,
height: 4,
format: "png",
path: out,
byte_size: 4,
}),
stderr: "",
timedOut: false,
aborted: false,
};
}
return { code: 0, stdout: "{}", stderr: "", timedOut: false, aborted: false };
},
killAll() {},
killFor(tag: string) {
killed.push(tag);
return opts.killCount ?? 1;
},
};
}
function fakeCtx(attachments?: unknown) {
return { get: (key: string) => (key === "attachments" ? attachments : undefined) } as never;
}
function setup(opts: {
runner?: FakeRunner;
attachments?: unknown;
registry?: SessionRegistry;
scheduler?: ReturnType<typeof fakeScheduler>;
}) {
const registry = opts.registry ?? new SessionRegistry(5);
const runner = opts.runner ?? fakeRunner();
const scheduler = opts.scheduler ?? fakeScheduler();
const attachments =
"attachments" in opts
? opts.attachments
: {
saveImage: async (input: { data: Uint8Array; name?: string }) => ({
attachmentId: "att-1",
mediaType: "image/png",
bytes: input.data.byteLength,
width: 4,
height: 4,
name: input.name,
}),
readImage: async (ref: { attachmentId: unknown }) => ({
data: new Uint8Array([1, 2, 3]),
attachment: ref,
}),
};
const service = new ObservationService({
ctx: fakeCtx(attachments),
runner,
registry,
queue: new KeyedExecutor(),
options: OPTIONS,
scheduler,
});
const events: ObservationEvent[] = [];
service.subscribe((event) => events.push(event));
return { service, registry, runner, scheduler, events };
}
/** Drive the registry through a real start so the session is owned. */
function own(registry: SessionRegistry, sessionId: string): void {
registry.reserveStart();
registry.completeStart({ sessionId, startedAtMs: 1 });
}
async function flushMicrotasks(): Promise<void> {
// setImmediate yields to the poll phase so real fs reads inside capture() settle.
for (let i = 0; i < 20; i++) await new Promise((resolve) => setImmediate(resolve));
}
/** Poll until the condition holds (capture completion is real async I/O). */
async function waitFor(cond: () => boolean, timeoutMs = 2000): Promise<void> {
const start = Date.now();
while (!cond()) {
if (Date.now() - start > timeoutMs) throw new Error("waitFor timeout");
await new Promise((resolve) => setImmediate(resolve));
}
}
describe("state machine", () => {
it("tracks add/action/end/url/remove and emits upsert/remove/reset events", () => {
const { service, events } = setup({});
service.addSession("s1");
expect(service.getState()).toEqual([{ sessionId: "s1", action: "idle", since: 1_000_000 }]);
service.beginAction("s1", "navigating");
expect(service.getState()[0].action).toBe("navigating");
service.endAction("s1");
expect(service.getState()[0].action).toBe("idle");
service.setUrl("s1", "https://example.com");
expect(service.getState()[0].url).toBe("https://example.com");
service.endAction("s1", "click failed: no ref");
expect(service.getState()[0].lastError).toBe("click failed: no ref");
service.removeSession("s1");
expect(service.getState()).toEqual([]);
expect(events.map((e) => e.type)).toEqual([
"upsert", // add
"upsert", // begin
"upsert", // end
"upsert", // setUrl
"upsert", // end with error
"remove",
]);
service.dispose();
expect(events[events.length - 1].type).toBe("reset");
});
it("ignores instrumentation for unknown sessions (owned-only by construction)", () => {
const { service } = setup({});
service.beginAction("foreign", "clicking");
service.endAction("foreign");
service.setUrl("foreign", "https://x");
expect(service.getState()).toEqual([]);
});
});
describe("thumbnail cadence", () => {
it("captures immediately on add and after action end, then fast-cadences while active", async () => {
const scheduler = fakeScheduler();
const runner = fakeRunner();
const { service } = setup({ runner, scheduler });
const captures = () => runner.calls.filter((c) => c.args[0] === "screenshot").length;
service.addSession("s1");
// add schedules an immediate (0ms) capture.
expect(scheduler.pending()).toEqual([0]);
scheduler.runNext();
await waitFor(() => captures() === 1 && scheduler.pending().length === 1);
expect(service.getState()[0].thumbnailAttachmentId).toBe("att-1");
// Just captured with fresh activity: next frame on the fast cadence.
expect(scheduler.pending()).toEqual([1500]);
scheduler.runNext();
await waitFor(() => captures() === 2 && scheduler.pending().length === 1);
expect(scheduler.pending()).toEqual([1500]);
});
it("downclocks to the idle cadence when the session has been quiet", async () => {
const scheduler = fakeScheduler();
const runner = fakeRunner();
const { service, scheduler: sch } = setup({ scheduler, runner });
const captures = () => runner.calls.filter((c) => c.args[0] === "screenshot").length;
service.addSession("s1");
sch.runNext();
await waitFor(() => captures() === 1 && sch.pending().length === 1);
// Fresh activity: fast cadence.
expect(sch.pending()).toEqual([1500]);
// Go quiet beyond the idle window before the next frame fires.
sch.advance(9000);
sch.runNext();
await waitFor(() => captures() === 2 && sch.pending().length === 1);
expect(sch.pending()).toEqual([8000]);
});
it("backs off to the idle cadence after repeated capture failures, keeping the old frame", async () => {
const scheduler = fakeScheduler();
const runner = fakeRunner({ screenshotFails: true });
const { service } = setup({ runner, scheduler });
const captures = () => runner.calls.filter((c) => c.args[0] === "screenshot").length;
service.addSession("s1");
for (let i = 1; i <= 4; i++) {
scheduler.runNext();
await waitFor(() => captures() === i && scheduler.pending().length === 1);
}
// 4 attempts (1 immediate + 3 fast), then backoff to the idle cadence.
expect(captures()).toBe(4);
expect(scheduler.pending()).toEqual([8000]);
expect(service.getState()[0].thumbnailAttachmentId).toBeUndefined();
});
it("stays silent without an attachment store (headless composition)", async () => {
const scheduler = fakeScheduler();
const runner = fakeRunner();
const { service } = setup({ runner, scheduler, attachments: undefined });
service.addSession("s1");
scheduler.runNext();
await flushMicrotasks();
expect(runner.calls.filter((c) => c.args[0] === "screenshot")).toHaveLength(0);
});
});
describe("observation traffic isolation", () => {
it("captures never emit action changes nor touch the registry", async () => {
const scheduler = fakeScheduler();
const registry = new SessionRegistry(5);
own(registry, "s1");
const { service, events } = setup({ registry, scheduler });
service.addSession("s1");
events.length = 0;
scheduler.runNext();
await waitFor(() => events.filter((e) => e.type === "upsert").length === 1);
// The capture upsert carries the thumbnail but the action stays idle…
const captureEvents = events.filter((e) => e.type === "upsert");
expect(captureEvents).toHaveLength(1);
const captureEvent = captureEvents[0];
if (captureEvent.type !== "upsert") throw new Error("unreachable");
expect(captureEvent.session?.action).toBe("idle");
expect(captureEvent.session?.thumbnailAttachmentId).toBe("att-1");
// …and the registry's current pointer never moved through observation.
expect(registry.current()).toBe("s1");
expect(registry.list()[0].startedAtMs).toBe(1);
});
});
describe("interrupt routing", () => {
it("interrupts the current session by default", () => {
const registry = new SessionRegistry(5);
own(registry, "s1");
const runner = fakeRunner();
const { service } = setup({ registry, runner });
expect(service.interrupt()).toBe(true);
expect(runner.killed).toEqual(["s1"]);
});
it("interrupts the specified owned session", () => {
const registry = new SessionRegistry(5);
own(registry, "s1");
own(registry, "s2");
const runner = fakeRunner();
const { service } = setup({ registry, runner });
expect(service.interrupt("s1")).toBe(true);
expect(runner.killed).toEqual(["s1"]);
});
it("returns false with no current session and for foreign ids without killing", () => {
const registry = new SessionRegistry(5);
own(registry, "s1");
const runner = fakeRunner();
const { service } = setup({ registry, runner });
expect(service.interrupt("foreign")).toBe(false);
registry.remove("s1");
expect(service.interrupt()).toBe(false);
expect(runner.killed).toEqual([]);
});
});
describe("HTTP/SSE interface", () => {
interface RecordedRoute {
path: string;
handler: (req: IncomingMessage, res: ServerResponse) => void | Promise<void>;
}
function routeHarness() {
const routes = new Map<string, RecordedRoute>();
const webServer = {
register(route: RecordedRoute & { kind: string }) {
routes.set(route.path, route);
return () => routes.delete(route.path);
},
};
return { routes, webServer };
}
function fakeRes() {
const chunks: string[] = [];
const closeHandlers: (() => void)[] = [];
const res = {
status: 0,
writeHead(status: number) {
res.status = status;
return res;
},
write(chunk: string) {
chunks.push(chunk);
},
end(chunk?: string) {
if (chunk !== undefined) chunks.push(chunk);
},
on(event: string, fn: () => void) {
if (event === "close") closeHandlers.push(fn);
},
close: () => closeHandlers.forEach((fn) => fn()),
body: () => chunks.join(""),
};
return {
res: res as unknown as ServerResponse & {
status: number;
body: () => string;
close: () => void;
},
chunks,
};
}
/** Loopback GET/POST request stubs the fence accepts. */
function fakeReq(overrides: Record<string, unknown> = {}): IncomingMessage {
return {
method: "GET",
headers: { host: "127.0.0.1:3999" },
on: (event: string, fn: (chunk?: string) => void) => {
if (event === "data") fn(JSON.stringify({ sessionId: "s1" }));
if (event === "end") fn();
},
...overrides,
} as unknown as IncomingMessage;
}
it("serves state, streams events, and routes interrupt", async () => {
const registry = new SessionRegistry(5);
own(registry, "s1");
const runner = fakeRunner();
const { service } = setup({ registry, runner });
service.addSession("s1");
const { routes, webServer } = routeHarness();
const ctx = { get: (key: string) => (key === "webServer" ? webServer : undefined) } as never;
const dispose = registerObservationRoutes(ctx, service);
// state
const stateRoute = routes.get("/bsk-observation/state");
expect(stateRoute).toBeDefined();
const stateRes = fakeRes();
await stateRoute?.handler(fakeReq(), stateRes.res as never);
expect(stateRes.res.status).toBe(200);
expect(JSON.parse(stateRes.res.body()).sessions[0].sessionId).toBe("s1");
// events (SSE): one write per service event
const eventsRoute = routes.get("/bsk-observation/events");
const eventsRes = fakeRes();
await eventsRoute?.handler(fakeReq(), eventsRes.res as never);
service.beginAction("s1", "clicking");
expect(eventsRes.res.body()).toContain('"action":"clicking"');
eventsRes.res.close();
// interrupt (POST {sessionId})
const interruptRoute = routes.get("/bsk-observation/interrupt");
const interruptRes = fakeRes();
const postReq = fakeReq({
method: "POST",
headers: { host: "127.0.0.1:3999", "content-type": "application/json" },
});
await interruptRoute?.handler(postReq, interruptRes.res as never);
expect(JSON.parse(interruptRes.res.body())).toEqual({ interrupted: true });
dispose();
expect(routes.size).toBe(0);
service.dispose();
});
it("fences off non-loopback, cross-site, and simple-request traffic", async () => {
const registry = new SessionRegistry(5);
own(registry, "s1");
const { service } = setup({ registry });
service.addSession("s1");
const { routes, webServer } = routeHarness();
const ctx = { get: (key: string) => (key === "webServer" ? webServer : undefined) } as never;
const dispose = registerObservationRoutes(ctx, service);
const stateRoute = routes.get("/bsk-observation/state");
const interruptRoute = routes.get("/bsk-observation/interrupt");
const getState = async (headers: Record<string, string>) => {
const res = fakeRes();
await stateRoute?.handler(fakeReq({ headers }), res.res as never);
return res.res.status;
};
// Host must be a loopback authority.
expect(await getState({ host: "192.168.1.20:3999" })).toBe(403);
expect(await getState({ host: "attacker.example" })).toBe(403);
expect(await getState({ host: "localhost:3999" })).toBe(200);
expect(await getState({ host: "[::1]:3999" })).toBe(200);
// A present Origin must match the Host (DNS-rebinding reads die here).
expect(await getState({ host: "127.0.0.1:3999", origin: "http://evil.example" })).toBe(403);
expect(await getState({ host: "127.0.0.1:3999", origin: "http://127.0.0.1:3999" })).toBe(200);
// Explicitly cross-site fetches are refused even without an Origin.
expect(await getState({ host: "127.0.0.1:3999", "sec-fetch-site": "cross-site" })).toBe(403);
expect(await getState({ host: "127.0.0.1:3999", "sec-fetch-site": "same-origin" })).toBe(200);
// POST must be application/json — a cross-site simple request can never comply.
const post = async (contentType: string | undefined) => {
const res = fakeRes();
const req = fakeReq({
method: "POST",
headers: {
host: "127.0.0.1:3999",
...(contentType !== undefined ? { "content-type": contentType } : {}),
},
});
await interruptRoute?.handler(req, res.res as never);
return res.res.status;
};
expect(await post("text/plain")).toBe(403);
expect(await post("application/x-www-form-urlencoded")).toBe(403);
expect(await post(undefined)).toBe(403);
expect(await post("application/json; charset=utf-8")).toBe(200);
dispose();
service.dispose();
});
it("registers nothing when no webServer is mounted", () => {
const { service } = setup({});
const ctx = { get: () => undefined } as never;
const dispose = registerObservationRoutes(ctx, service);
expect(typeof dispose).toBe("function");
dispose();
service.dispose();
});
});
describe("availability and dead sessions", () => {
it("flips availability off after repeated global failures and back on success", async () => {
const scheduler = fakeScheduler();
const runner = fakeRunner({ screenshotFails: true });
const { service, events } = setup({ runner, scheduler });
service.addSession("s1");
for (let i = 1; i <= 3; i++) {
scheduler.runNext();
// the frame's scratch-file unlink lands before the next timer is queued
await waitFor(
() =>
runner.calls.filter((c) => c.args[0] === "screenshot").length === i &&
scheduler.pending().length === 1,
);
}
expect(service.isAvailable()).toBe(false);
expect(events.some((e) => e.type === "availability" && e.available === false)).toBe(true);
service.dispose();
});
it("marks a session dead on session_not_found and stops asking for frames", async () => {
const scheduler = fakeScheduler();
const runner = fakeRunner({ screenshotNotFound: true });
const { service, events } = setup({ runner, scheduler });
service.addSession("s1");
scheduler.runNext();
await waitFor(() => events.some((e) => e.type === "upsert" && e.session?.dead === true));
expect(service.getState()[0].dead).toBe(true);
expect(service.getState()[0].action).toBe("idle");
// No further captures are scheduled for a dead session.
expect(scheduler.pending()).toEqual([]);
// Instrumentation is dropped for dead sessions.
service.beginAction("s1", "clicking");
expect(service.getState()[0].action).toBe("idle");
// A dead session leaves cleanly.
service.removeSession("s1");
expect(service.getState()).toEqual([]);
});
it("clears lastError on the next action and keeps it on failure", () => {
const { service } = setup({});
service.addSession("s1");
service.endAction("s1", "click failed");
expect(service.getState()[0].lastError).toBe("click failed");
service.beginAction("s1", "clicking");
expect(service.getState()[0].lastError).toBeUndefined();
service.endAction("s1");
expect(service.getState()[0].lastError).toBeUndefined();
service.endAction("s1", "fill failed");
expect(service.getState()[0].lastError).toBe("fill failed");
service.dispose();
});
});
describe("thumbnail read-back", () => {
it("serves captured frames through readThumbnail and 404s unknown ids", async () => {
const scheduler = fakeScheduler();
const runner = fakeRunner();
const { service } = setup({ runner, scheduler });
service.addSession("s1");
scheduler.runNext();
await waitFor(() => service.getState()[0]?.thumbnailAttachmentId !== undefined);
const frame = await service.readThumbnail("att-1");
expect(frame?.mediaType).toBe("image/png");
expect([...(frame?.data ?? [])]).toEqual([1, 2, 3]);
expect(await service.readThumbnail("nope")).toBeUndefined();
service.dispose();
});
});
describe("actionForLabel", () => {
it("maps tool labels onto observation verbs", () => {
expect(actionForLabel("navigate")).toBe("navigating");
expect(actionForLabel("screenshot")).toBe("capturing");
expect(actionForLabel("session start")).toBe("starting");
});
});
describe("scratch files and thumbnail references", () => {
it("reuses one scratch path per session and deletes it after reading", async () => {
const scheduler = fakeScheduler();
const runner = fakeRunner();
const { service } = setup({ runner, scheduler });
service.addSession("s1");
// first frame (scheduled with delay 0)
scheduler.runNext();
await waitFor(
() =>
service.getState()[0]?.thumbnailAttachmentId !== undefined &&
scheduler.pending().length === 1,
);
const firstOut = runner.calls.filter((c) => c.args[0] === "screenshot").at(-1)?.args;
const pathOf = (args: string[]) => args[args.indexOf("--out") + 1];
expect(pathOf(firstOut ?? [])).toBe(join(tmpdir(), "bsk-obs-s1.png"));
expect(existsSync(pathOf(firstOut ?? []))).toBe(false);
// second frame: same path, deleted again
service.endAction("s1");
scheduler.runNext();
await waitFor(
() =>
runner.calls.filter((c) => c.args[0] === "screenshot").length >= 2 &&
scheduler.pending().length === 1,
);
const secondOut = runner.calls.filter((c) => c.args[0] === "screenshot").at(-1)?.args;
expect(pathOf(secondOut ?? [])).toBe(join(tmpdir(), "bsk-obs-s1.png"));
await waitFor(() => !existsSync(pathOf(secondOut ?? [])));
service.dispose();
});
it("drops the replaced frame reference and clears it on remove", async () => {
let counter = 0;
const attachments = {
saveImage: async (input: { data: Uint8Array; name?: string }) => {
counter += 1;
return {
attachmentId: `att-${counter}`,
mediaType: "image/png",
bytes: input.data.byteLength,
width: 4,
height: 4,
name: input.name,
};
},
readImage: async (ref: { attachmentId: unknown }) => ({
data: new Uint8Array([1, 2, 3]),
attachment: ref,
}),
};
const scheduler = fakeScheduler();
const { service } = setup({ scheduler, attachments });
service.addSession("s1");
scheduler.runNext();
await waitFor(
() =>
service.getState()[0]?.thumbnailAttachmentId === "att-1" &&
scheduler.pending().length === 1,
);
service.endAction("s1");
scheduler.runNext();
await waitFor(
() =>
service.getState()[0]?.thumbnailAttachmentId === "att-2" &&
scheduler.pending().length === 1,
);
// the replaced frame's read path is gone
expect(await service.readThumbnail("att-1")).toBeUndefined();
expect((await service.readThumbnail("att-2"))?.mediaType).toBe("image/png");
// removing the session clears its live reference too
service.removeSession("s1");
expect(await service.readThumbnail("att-2")).toBeUndefined();
service.dispose();
});
});
@@ -0,0 +1,85 @@
// KeyedExecutor: per-key FIFO guarantees, including the abort-while-queued
// race that used to release the tail early (letting a later task overlap the
// one still running on the same session).
import { describe, expect, it } from "vitest";
import { KeyedExecutor } from "../src/queue";
describe("KeyedExecutor", () => {
it("runs same-key tasks in FIFO order", async () => {
const queue = new KeyedExecutor();
const order: string[] = [];
const gate = new Promise<void>((resolve) => setTimeout(resolve, 5));
await Promise.all([
queue.run("k", async () => {
await gate;
order.push("a");
}),
queue.run("k", async () => {
order.push("b");
}),
queue.run("k", async () => {
order.push("c");
}),
]);
expect(order).toEqual(["a", "b", "c"]);
});
it("a task aborted while queued does not let the next one overlap the runner", async () => {
const queue = new KeyedExecutor();
let releaseA!: () => void;
const aRuns = new Promise<void>((resolve) => {
releaseA = resolve;
});
const events: string[] = [];
// A holds the session until releaseA.
const a = queue.run("s1", async () => {
events.push("a:start");
await aRuns;
events.push("a:end");
});
// B is queued behind A and aborted before A finishes.
const abortB = new AbortController();
const b = queue.run(
"s1",
async () => {
events.push("b:start");
},
abortB.signal,
);
// C is queued behind B.
const c = queue.run("s1", async () => {
events.push("c:start");
});
abortB.abort();
await expect(b).rejects.toMatchObject({ name: "AbortError" });
// Give C every chance to (wrongly) start while A is still in flight.
await new Promise((resolve) => setTimeout(resolve, 20));
expect(events).toEqual(["a:start"]);
releaseA();
await a;
await c;
expect(events).toEqual(["a:start", "a:end", "c:start"]);
});
it("a failed task never strands the queue", async () => {
const queue = new KeyedExecutor();
await expect(
queue.run("k", async () => {
throw new Error("boom");
}),
).rejects.toThrow("boom");
await expect(queue.run("k", async () => "ok")).resolves.toBe("ok");
});
it("drops the tail entry once the queue drains", async () => {
const queue = new KeyedExecutor();
const internals = queue as unknown as { tails: Map<string, Promise<void>> };
await queue.run("k", async () => {});
await new Promise((resolve) => setImmediate(resolve));
expect(internals.tails.size).toBe(0);
});
});
@@ -0,0 +1,142 @@
import type { ChildProcess } from "node:child_process";
import { EventEmitter } from "node:events";
import { describe, expect, it } from "vitest";
import { type BskRunResult, createBskRunner, isCommandNotFound, parseBskJson } from "../src/runner";
/** Minimal fake ChildProcess driven by the test. */
class FakeChild extends EventEmitter {
stdout = new EventEmitter();
stderr = new EventEmitter();
exitCode: number | null = null;
signalCode: string | null = null;
killedWith: string[] = [];
kill(signal: string): boolean {
if (this.exitCode !== null || this.signalCode !== null) return false;
this.killedWith.push(signal);
this.signalCode = signal;
queueMicrotask(() => this.emit("close", null));
return true;
}
finish(code: number, stdout = "", stderr = ""): void {
if (stdout) this.stdout.emit("data", stdout);
if (stderr) this.stderr.emit("data", stderr);
this.exitCode = code;
this.emit("close", code);
}
}
function fakeSpawn(children: FakeChild[]) {
return () => {
const child = children.shift();
if (child === undefined) throw new Error("no fake child queued");
return child as unknown as ChildProcess;
};
}
describe("createBskRunner", () => {
it("appends --json and collects stdout", async () => {
const child = new FakeChild();
const seen: string[][] = [];
const runner = createBskRunner("bsk", (cmd: string, args: string[]) => {
seen.push([cmd, ...args]);
return child as unknown as ChildProcess;
});
const promise = runner.run(["session", "list"]);
child.finish(0, '{"ok":true}');
const result = await promise;
expect(seen).toEqual([["bsk", "session", "list", "--json"]]);
expect(result).toMatchObject({
code: 0,
stdout: '{"ok":true}',
aborted: false,
timedOut: false,
});
});
it("kills the child when the abort signal fires", async () => {
const child = new FakeChild();
const runner = createBskRunner("bsk", fakeSpawn([child]));
const controller = new AbortController();
const promise = runner.run(["snapshot"], { signal: controller.signal });
controller.abort();
const result = await promise;
expect(result.aborted).toBe(true);
expect(child.killedWith).toContain("SIGTERM");
});
it("kills the child on timeout", async () => {
const child = new FakeChild();
const runner = createBskRunner("bsk", fakeSpawn([child]));
const result = await runner.run(["snapshot"], { timeoutMs: 5 });
expect(result.timedOut).toBe(true);
expect(child.killedWith.length).toBeGreaterThan(0);
});
it("killAll terminates in-flight children", async () => {
const child = new FakeChild();
const runner = createBskRunner("bsk", fakeSpawn([child]));
const promise = runner.run(["session", "list"]);
runner.killAll();
const result = await promise;
expect(child.killedWith).toContain("SIGTERM");
expect(result.code).toBeNull();
});
});
describe("parseBskJson", () => {
const base: BskRunResult = { code: 0, stdout: "", stderr: "", timedOut: false, aborted: false };
it("parses stdout JSON on success", () => {
expect(parseBskJson({ ...base, stdout: '{"a":1}' }, "status")).toEqual({ a: 1 });
});
it("maps the JSON error envelope on non-zero exit", () => {
const body = JSON.stringify({
code: "no_browser",
message: "no browser connected",
hint: "open Chrome",
});
try {
parseBskJson({ ...base, code: 3, stdout: body }, "session start");
expect.unreachable();
} catch (error) {
expect(error).toMatchObject({
name: "BskError",
code: "no_browser",
hint: "open Chrome",
exitCode: 3,
});
expect((error as Error).message).toContain("no browser connected");
expect((error as Error).message).toContain("hint: open Chrome");
}
});
it("falls back to stderr when the error body is not JSON", () => {
expect(() => parseBskJson({ ...base, code: 1, stderr: "boom" }, "x")).toThrow(/boom/);
});
it("reports killed-by-interrupt children (null exit code) as interrupted", () => {
expect(() => parseBskJson({ ...base, code: null }, "navigate")).toThrow(/interrupted/);
});
it("reports timeouts distinctly", () => {
expect(() => parseBskJson({ ...base, timedOut: true }, "x")).toThrow(/timed out/);
});
it("rejects non-JSON success output", () => {
expect(() => parseBskJson({ ...base, stdout: "not json" }, "x")).toThrow(
/did not produce JSON/,
);
});
});
describe("isCommandNotFound", () => {
it("detects ENOENT spawn failures", () => {
expect(
isCommandNotFound(Object.assign(new Error("spawn bsk ENOENT"), { code: "ENOENT" })),
).toBe(true);
expect(isCommandNotFound(new Error("other"))).toBe(false);
});
});
@@ -0,0 +1,108 @@
import { describe, expect, it } from "vitest";
import { SessionRegistry } from "../src/sessions";
/** Drive one full successful start through the reservation protocol. */
function start(registry: SessionRegistry, sessionId: string): void {
registry.reserveStart();
registry.completeStart({ sessionId, startedAtMs: Date.now() });
}
describe("SessionRegistry", () => {
it("tracks the current session across start and remove", () => {
const registry = new SessionRegistry(5);
expect(registry.current()).toBeUndefined();
start(registry, "s1");
expect(registry.current()).toBe("s1");
start(registry, "s2");
expect(registry.current()).toBe("s2");
registry.remove("s2");
expect(registry.current()).toBe("s1");
registry.remove("s1");
expect(registry.current()).toBeUndefined();
});
it("enforces the concurrency cap including in-flight starts", () => {
const registry = new SessionRegistry(2);
start(registry, "s1");
start(registry, "s2");
expect(() => registry.reserveStart()).toThrow(/session limit/);
registry.remove("s1");
registry.reserveStart();
registry.completeStart({ sessionId: "s3", startedAtMs: Date.now() });
expect(registry.size()).toBe(2);
});
it("counts reserved (not yet spawned) starts against the cap — the race case", () => {
const registry = new SessionRegistry(2);
registry.reserveStart();
registry.reserveStart();
// Both slots are in flight; a third concurrent start must reject even
// though no session has been registered yet.
expect(() => registry.reserveStart()).toThrow(/session limit/);
registry.abandonStart();
registry.reserveStart();
registry.completeStart({ sessionId: "s1", startedAtMs: Date.now() });
expect(registry.ownedIds()).toEqual(["s1"]);
});
it("resolve accepts an explicit owned session and makes it current", () => {
const registry = new SessionRegistry(5);
start(registry, "s1");
start(registry, "s2");
expect(registry.resolve("s1", "tool")).toBe("s1");
expect(registry.current()).toBe("s1");
});
it("resolve rejects foreign ids and never adopts them", () => {
const registry = new SessionRegistry(5);
start(registry, "s1");
expect(() => registry.resolve("foreign", "tool")).toThrow(/does not belong to this plugin/);
expect(registry.isOwned("foreign")).toBe(false);
expect(registry.ownedIds()).toEqual(["s1"]);
// A rejected resolve must not move the current pointer.
expect(registry.current()).toBe("s1");
});
it("resolve falls back to the current session", () => {
const registry = new SessionRegistry(5);
start(registry, "s1");
expect(registry.resolve(undefined, "tool")).toBe("s1");
});
it("resolve throws actionable guidance when no session exists", () => {
const registry = new SessionRegistry(5);
expect(() => registry.resolve(undefined, "browser_click")).toThrow(/browser_session_start/);
});
it("recency order survives remove of the current session", () => {
const registry = new SessionRegistry(5);
start(registry, "s1");
start(registry, "s2");
registry.resolve("s1", "tool");
registry.remove("s1");
expect(registry.current()).toBe("s2");
});
it("resolveForStop accepts owned sessions and refuses foreign ones", () => {
const registry = new SessionRegistry(5);
start(registry, "s1");
expect(registry.resolveForStop(undefined)).toBe("s1");
expect(registry.resolveForStop("s1")).toBe("s1");
expect(() => registry.resolveForStop("foreign")).toThrow(/does not belong to this plugin/);
expect(() => registry.resolveForStop("never-seen")).toThrow(/does not belong to this plugin/);
});
it("a refused stop does not move the current pointer", () => {
const registry = new SessionRegistry(5);
start(registry, "s1");
expect(() => registry.resolveForStop("foreign")).toThrow();
expect(registry.current()).toBe("s1");
});
it("ownedIds always equals every registered session (foreign ids never enter)", () => {
const registry = new SessionRegistry(5);
start(registry, "own1");
start(registry, "own2");
expect(registry.ownedIds().sort()).toEqual(["own1", "own2"]);
});
});
@@ -0,0 +1,74 @@
// BSK skill injection: registration wiring, catalog content, progressive-load
// weight, and silent degradation without the skill seam.
import { describe, expect, it } from "vitest";
import { registerBskSkill } from "../src/skill";
function fakeCtx(skills?: unknown) {
return { get: (key: string) => (key === "skills" ? skills : undefined) } as never;
}
describe("registerBskSkill", () => {
it("registers the catalog entry with the assembled body", () => {
const registrations: { skill: Record<string, unknown>; disposer: () => void }[] = [];
const skills = {
register(skill: Record<string, unknown>) {
const disposer = () => {
skill.__disposed = true;
};
registrations.push({ skill, disposer });
return disposer;
},
};
const unregister = registerBskSkill(fakeCtx(skills));
expect(registrations).toHaveLength(1);
const skill = registrations[0].skill;
expect(skill.name).toBe("browser-skill");
expect(typeof skill.description).toBe("string");
expect(String(skill.description)).toContain("browser_*");
// body = dsh prelude + canonical CLI skill body + dsh postlude
const content = String(skill.content);
expect(content).toContain("browser_observe");
expect(String(skill.description)).toContain("never invoke bsk through bash or shell");
expect(content).toContain("Always call those injected `browser_*` tools");
expect(content).toContain("Never use `bash`, `shell`, `exec`");
expect(content).toContain("## DSH routing reminder");
expect(content.trimEnd()).toMatch(
/always begin with[\s\S]*`browser_session_start`[\s\S]*`browser_session_stop`\.$/,
);
expect(content).toContain("Owned sessions only");
expect(content).toContain("Mandatory workflow");
expect(content).toContain("Refs invalidate after navigation");
expect(content.length).toBeGreaterThan(10_000);
expect(skill.source).toBe("bundled");
// the original YAML frontmatter must not leak into the body
expect(content.includes("name: browser-skill\ndescription:")).toBe(false);
// disposer passthrough
unregister();
expect(skill.__disposed).toBe(true);
});
it("keeps one in-memory copy: repeated reads share the same content", () => {
let captured: Record<string, unknown> | undefined;
const skills = {
register(skill: Record<string, unknown>) {
captured = skill;
return () => {};
},
};
registerBskSkill(fakeCtx(skills));
// Pre-step catalog snapshots re-read the registration: the body must be
// the identical in-memory string every time (no reload, no copy).
expect(captured?.content).toBe(captured?.content);
expect(typeof captured?.content).toBe("string");
});
it("degrades silently when the skills seam is absent or foreign", () => {
expect(() => registerBskSkill(fakeCtx())).not.toThrow();
expect(() => registerBskSkill(fakeCtx(null))).not.toThrow();
expect(() => registerBskSkill(fakeCtx({}))).not.toThrow();
const disposer = registerBskSkill(fakeCtx());
expect(typeof disposer).toBe("function");
expect(() => disposer()).not.toThrow();
});
});
@@ -0,0 +1,801 @@
import { existsSync, mkdtempSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import type { ToolDefinition, ToolRunContext } from "@deepseek-ai/dsh-tools";
import { describe, expect, it, vi } from "vitest";
import { ObservationService } from "../src/observation";
import { KeyedExecutor } from "../src/queue";
import type { BskRunner, BskRunOptions, BskRunResult } from "../src/runner";
import { SessionRegistry } from "../src/sessions";
import { type PluginConfig, registerTools } from "../src/tools";
const CONFIG: PluginConfig = {
bskPath: "bsk",
defaultTimeoutMs: 120_000,
maxSessions: 5,
observationEnabled: false,
thumbnailIntervalMs: 1500,
idleIntervalMs: 8000,
lazyTools: false,
};
/** An observation service with the feature off: instrumentation becomes a no-op. */
function disabledObservation(deps: { ctx: unknown; runner: BskRunner; registry: SessionRegistry }) {
return new ObservationService({
ctx: deps.ctx as never,
runner: deps.runner,
registry: deps.registry,
queue: new KeyedExecutor(),
options: { enabled: false, thumbnailIntervalMs: 1500, idleIntervalMs: 8000 },
});
}
interface FakeCall {
args: string[];
options: BskRunOptions;
}
/** Mark a canned answer as a per-call sequence (each call shifts one reply). */
function seq(items: unknown[]): { __seq: unknown[] } {
return { __seq: items };
}
/** A runner that answers canned JSON per command prefix and records calls. */
function fakeRunner(responses: Record<string, unknown>) {
const calls: FakeCall[] = [];
return {
calls,
runner: {
async run(args: string[], options: BskRunOptions = {}): Promise<BskRunResult> {
calls.push({ args, options });
if (options.signal?.aborted) {
return { code: null, stdout: "", stderr: "", timedOut: false, aborted: true };
}
for (const [prefix, response] of Object.entries(responses)) {
if (args.join(" ").startsWith(prefix)) {
if (response instanceof Error) throw response;
const reply =
typeof response === "object" && response !== null && "__seq" in response
? (response as { __seq: unknown[] }).__seq.shift()
: response;
return {
code: 0,
stdout: JSON.stringify(reply),
stderr: "",
timedOut: false,
aborted: false,
};
}
}
return {
code: 2,
stdout: JSON.stringify({
code: "unexpected",
message: `no canned response for: ${args.join(" ")}`,
}),
stderr: "",
timedOut: false,
aborted: false,
};
},
killAll() {},
killFor: () => 0,
},
};
}
function makeCtx(services: Record<string, unknown> = {}) {
const tools = new Map<string, ToolDefinition>();
const ctx = {
tools: { register: (def: ToolDefinition) => tools.set(def.name, def) },
get: (key: string) => services[key],
};
return { ctx, tools };
}
function makeExec(overrides: Record<string, unknown> = {}): ToolRunContext {
return {
callId: "test-call",
name: "test",
signal: new AbortController().signal,
...overrides,
} as unknown as ToolRunContext;
}
const SNAPSHOT_REPLY = {
text: '- button "OK" [ref=@e1]',
ref_count: 1,
tab_id: 7,
truncated: false,
};
function setup(
responses: Record<string, unknown>,
services: Record<string, unknown> = {},
maxSessions = 5,
) {
const { ctx, tools } = makeCtx(services);
const { runner, calls } = fakeRunner(responses);
const registry = new SessionRegistry(maxSessions);
const config = { ...CONFIG, maxSessions };
registerTools({
ctx: ctx as never,
runner: runner as BskRunner,
registry,
config,
observation: disabledObservation({ ctx, runner: runner as BskRunner, registry }),
queue: new KeyedExecutor(),
});
return { tools, calls, registry };
}
async function startSession(tools: Map<string, ToolDefinition>, sessionId = "s1") {
const tool = tools.get("browser_session_start");
if (tool === undefined) throw new Error("browser_session_start not registered");
return tool.execute({}, makeExec()) as Promise<{ sessionId: string }>;
}
const START_REPLY = (id: string) => ({ session_id: id, browser_instance_id: "chrome-1" });
describe("tool registration", () => {
it("registers the full browser tool suite", () => {
const { tools } = setup({});
expect([...tools.keys()].sort()).toEqual([
"browser_click",
"browser_emulate",
"browser_fill",
"browser_navigate",
"browser_observe",
"browser_press",
"browser_screenshot",
"browser_session_list",
"browser_session_start",
"browser_session_stop",
"browser_snapshot",
]);
});
it("validates model args through defineTool (missing required param)", async () => {
const { tools } = setup({});
const navigate = tools.get("browser_navigate");
await expect(navigate?.execute({}, makeExec())).rejects.toThrow(/invalid arguments/);
});
});
describe("browser_session_start", () => {
it("maps the start reply and tracks the session as current", async () => {
const { tools, registry } = setup({ "session start": START_REPLY("s1") });
const value = await startSession(tools);
expect(value.sessionId).toBe("s1");
expect(registry.current()).toBe("s1");
});
it("passes window size and opens the initial URL and device preset", async () => {
const { tools, calls } = setup({
"session start": START_REPLY("s1"),
emulate: { tab_id: 7, cleared: false },
navigate: { tab_id: 7, url: "https://example.com", reached: "load" },
});
const tool = tools.get("browser_session_start");
const value = (await tool?.execute(
{ width: 1280, height: 800, url: "https://example.com", device: "iphone-14", noFocus: true },
makeExec(),
)) as { sessionId: string; url: string; device: string };
expect(value).toMatchObject({
sessionId: "s1",
url: "https://example.com",
device: "iphone-14",
});
expect(calls[0].args).toEqual([
"session",
"start",
"--width",
"1280",
"--height",
"800",
"--no-focus",
]);
expect(calls[1].args).toEqual(["emulate", "--session", "s1", "--device", "iphone-14"]);
expect(calls[2].args).toEqual(["navigate", "--session", "s1", "https://example.com"]);
});
it("requires width and height together", async () => {
const { tools } = setup({ "session start": START_REPLY("s1") });
const tool = tools.get("browser_session_start");
await expect(tool?.execute({ width: 1280 }, makeExec())).rejects.toThrow(/together/);
});
it("cleans up a half-initialized session when the initial navigate fails", async () => {
const { tools, calls, registry } = setup({
"session start": START_REPLY("s1"),
"session stop": {},
});
// navigate has no canned response -> non-zero exit -> error.
const tool = tools.get("browser_session_start");
await expect(tool?.execute({ url: "https://example.com" }, makeExec())).rejects.toThrow(
/navigate/,
);
expect(calls.some((c) => c.args.join(" ") === "session stop s1")).toBe(true);
expect(registry.current()).toBeUndefined();
});
});
describe("multi-session behavior", () => {
const responses = {
"session start": START_REPLY("s1"),
snapshot: SNAPSHOT_REPLY,
};
it("operation tools default to the current session and echo it back", async () => {
const { tools, calls } = setup(responses);
await startSession(tools);
const snapshot = tools.get("browser_snapshot");
const value = (await snapshot?.execute({}, makeExec())) as {
session: string;
refCount: number;
};
expect(value).toMatchObject({ session: "s1", refCount: 1 });
expect(calls[1].args).toEqual(["snapshot", "--session", "s1"]);
});
it("an explicit owned session wins and becomes the new current session", async () => {
const { tools, calls, registry } = setup({
"session start": seq([START_REPLY("s1"), START_REPLY("s2")]),
snapshot: SNAPSHOT_REPLY,
});
await startSession(tools, "s1");
await startSession(tools, "s2");
const snapshot = tools.get("browser_snapshot");
const value = (await snapshot?.execute({ session: "s1" }, makeExec())) as {
session: string;
};
expect(value.session).toBe("s1");
expect(calls[2].args).toEqual(["snapshot", "--session", "s1"]);
expect(registry.current()).toBe("s1");
});
it("rejects an explicit foreign session without touching the daemon", async () => {
const { tools, calls } = setup(responses);
await startSession(tools);
const callsBefore = calls.length;
const snapshot = tools.get("browser_snapshot");
await expect(snapshot?.execute({ session: "other" }, makeExec())).rejects.toThrow(
/does not belong to this plugin/,
);
expect(calls.length).toBe(callsBefore);
});
it("fails with guidance when no session exists", async () => {
const { tools } = setup(responses);
const snapshot = tools.get("browser_snapshot");
await expect(snapshot?.execute({}, makeExec())).rejects.toThrow(/browser_session_start/);
});
it("enforces the configured session cap before spawning", async () => {
const { tools, calls } = setup(
{ "session start": seq([START_REPLY("s1"), START_REPLY("s2")]) },
{},
1,
);
await startSession(tools);
const start = tools.get("browser_session_start");
await expect(start?.execute({}, makeExec())).rejects.toThrow(/session limit/);
// A rejected start must not spawn another bsk session (no leaked session).
expect(calls.filter((c) => c.args.join(" ").startsWith("session start"))).toHaveLength(1);
});
it("shares the cap atomically across concurrent starts (reservation race)", async () => {
const { tools, calls } = setup(
{ "session start": seq([START_REPLY("s1"), START_REPLY("s2")]) },
{},
1,
);
const start = tools.get("browser_session_start");
if (start === undefined) throw new Error("not registered");
// Two truly concurrent starts against a cap of 1: exactly one may spawn.
const outcomes = await Promise.allSettled([
start.execute({}, makeExec()),
start.execute({}, makeExec()),
]);
const fulfilled = outcomes.filter((o) => o.status === "fulfilled");
const rejected = outcomes.filter((o) => o.status === "rejected");
expect(fulfilled).toHaveLength(1);
expect(rejected).toHaveLength(1);
expect(String((rejected[0] as PromiseRejectedResult).reason)).toMatch(/session limit/);
expect(calls.filter((c) => c.args.join(" ").startsWith("session start"))).toHaveLength(1);
});
it("refuses to stop a session the plugin did not create", async () => {
const { tools, calls } = setup(responses);
await startSession(tools);
const stop = tools.get("browser_session_stop");
await expect(stop?.execute({ session: "ext7" }, makeExec())).rejects.toThrow(
/does not belong to this plugin/,
);
// No stop command may reach the daemon for a foreign session.
expect(calls.some((c) => c.args.join(" ").startsWith("session stop"))).toBe(false);
});
});
describe("browser_session_stop / list", () => {
it("stops the current session and clears the registry", async () => {
const { tools, calls, registry } = setup({
"session start": START_REPLY("s1"),
"session stop": {},
});
await startSession(tools);
const stop = tools.get("browser_session_stop");
const value = (await stop?.execute({}, makeExec())) as { stopped: string };
expect(value.stopped).toBe("s1");
expect(calls[1].args).toEqual(["session", "stop", "s1"]);
expect(registry.current()).toBeUndefined();
});
it("lists only plugin-owned sessions with the current marker (no daemon view)", async () => {
const { tools, calls } = setup({
"session start": seq([START_REPLY("s1"), START_REPLY("s2")]),
});
await startSession(tools);
await startSession(tools);
const list = tools.get("browser_session_list");
const value = (await list?.execute({}, makeExec())) as {
sessions: { sessionId: string; current: boolean }[];
};
expect(value.sessions).toEqual([
{ sessionId: "s1", browserInstanceId: "chrome-1", current: false },
{ sessionId: "s2", browserInstanceId: "chrome-1", current: true },
]);
// Registry-only: listing must not call the daemon at all.
expect(calls.some((c) => c.args.join(" ").startsWith("session list"))).toBe(false);
});
});
describe("interaction tools", () => {
const responses = {
"session start": START_REPLY("s1"),
click: { tab_id: 7, used_ref: "@e1", x: 10, y: 20 },
fill: { tab_id: 7, used_ref: "@e2", value_length: 5 },
press: { tab_id: 7, key: "Enter", code: "Enter", modifiers: [] },
navigate: { tab_id: 7, url: "https://a.test", final_url: "https://b.test", reached: "load" },
emulate: { tab_id: 7, cleared: false },
};
it("browser_click maps target, button, and click count", async () => {
const { tools, calls } = setup(responses);
await startSession(tools);
const click = tools.get("browser_click");
const value = (await click?.execute(
{ target: "@e1", button: "right", clickCount: 2 },
makeExec(),
)) as { session: string; x: number; y: number };
expect(value).toMatchObject({ session: "s1", x: 10, y: 20 });
expect(calls[1].args).toEqual([
"click",
"--session",
"s1",
"--button",
"right",
"--click-count",
"2",
"@e1",
]);
});
it("browser_fill passes value and --no-clear", async () => {
const { tools, calls } = setup(responses);
await startSession(tools);
const fill = tools.get("browser_fill");
await fill?.execute({ target: "e2", value: "hello", noClear: true }, makeExec());
expect(calls[1].args).toEqual([
"fill",
"--session",
"s1",
"--value",
"hello",
"--no-clear",
"e2",
]);
});
it("browser_press picks --ref for snapshot refs and --selector otherwise", async () => {
const { tools, calls } = setup(responses);
await startSession(tools);
const press = tools.get("browser_press");
await press?.execute({ key: "Enter", target: "@e3" }, makeExec());
expect(calls[1].args).toEqual(["press", "--session", "s1", "--ref", "@e3", "Enter"]);
await press?.execute({ key: "Ctrl+A", target: "#field" }, makeExec());
expect(calls[2].args).toEqual(["press", "--session", "s1", "--selector", "#field", "Ctrl+A"]);
});
it("browser_navigate converts waitUntil and timeoutMs", async () => {
const { tools, calls } = setup(responses);
await startSession(tools);
const navigate = tools.get("browser_navigate");
const value = (await navigate?.execute(
{ url: "https://a.test", waitUntil: "networkidle", timeoutMs: 1500 },
makeExec(),
)) as { finalUrl: string; reached: string };
expect(value).toMatchObject({ finalUrl: "https://b.test", reached: "load" });
expect(calls[1].args).toEqual([
"navigate",
"--session",
"s1",
"--wait-until",
"networkidle",
"--timeout",
"1500ms",
"https://a.test",
]);
});
it("browser_emulate rejects off combined with overrides", async () => {
const { tools } = setup(responses);
await startSession(tools);
const emulate = tools.get("browser_emulate");
await expect(emulate?.execute({ off: true, device: "iphone-14" }, makeExec())).rejects.toThrow(
/mutually exclusive/,
);
});
});
describe("browser_screenshot", () => {
function pngFile(): string {
const dir = mkdtempSync(join(tmpdir(), "bsk-test-"));
const path = join(dir, "shot.png");
// Minimal PNG header bytes are enough: the plugin never decodes the image.
writeFileSync(path, Buffer.from([0x89, 0x50, 0x4e, 0x47, 1, 2, 3, 4]));
return path;
}
it("returns the PNG path when no attachment store is mounted", async () => {
const path = pngFile();
const { tools, calls } = setup({
"session start": START_REPLY("s1"),
screenshot: { tab_id: 7, width: 800, height: 600, format: "png", path, byte_size: 8 },
});
await startSession(tools);
const screenshot = tools.get("browser_screenshot");
const value = (await screenshot?.execute({ ref: "@e1" }, makeExec())) as {
path: string;
image?: unknown;
};
expect(value.path).toBe(path);
expect(value.image).toBeUndefined();
expect(calls[1].args.slice(0, 3)).toEqual(["screenshot", "--session", "s1"]);
expect(calls[1].args).toContain("--ref");
const rendered = screenshot?.output.render({ session: "s1" }, value as never);
expect(rendered?.every((block) => block.type === "text")).toBe(true);
});
it("commits the image through the attachment store on an image-capable route", async () => {
const path = pngFile();
const services = {
attachments: {
imageLimits: {
mediaTypes: ["image/png"],
maxImageBytes: 10_000_000,
maxMessageImageBytes: 10_000_000,
},
saveImage: async (input: { data: Uint8Array; mediaType: string; name?: string }) => ({
attachmentId: "att-1",
mediaType: input.mediaType,
bytes: input.data.byteLength,
width: 800,
height: 600,
name: input.name,
}),
},
llm: {
resolveModelInfo: async () => ({ inputModalities: ["text", "image"] }),
},
};
const { tools } = setup(
{
"session start": START_REPLY("s1"),
screenshot: { tab_id: 7, width: 800, height: 600, format: "png", path, byte_size: 8 },
},
services,
);
await startSession(tools);
const exec = makeExec({
agent: {
session: { requestHeader: () => ({ config: { provider: "deepseek", model: "vl" } }) },
options: {},
},
});
const screenshot = tools.get("browser_screenshot");
const value = (await screenshot?.execute({}, exec)) as {
image?: { attachmentId: string; mediaType: string };
};
expect(value.image).toMatchObject({ attachmentId: "att-1", mediaType: "image/png" });
const rendered = screenshot?.output.render({}, value as never) as { type: string }[];
expect(rendered.map((block) => block.type)).toEqual(["text", "image"]);
});
it("stays path-only when the model route is text-only", async () => {
const path = pngFile();
const services = {
attachments: {
imageLimits: { mediaTypes: ["image/png"], maxImageBytes: 1e7, maxMessageImageBytes: 1e7 },
saveImage: async () => {
throw new Error("must not be called");
},
},
llm: { resolveModelInfo: async () => ({ inputModalities: ["text"] }) },
};
const { tools } = setup(
{
"session start": START_REPLY("s1"),
screenshot: { tab_id: 7, width: 1, height: 1, format: "png", path, byte_size: 8 },
},
services,
);
await startSession(tools);
const exec = makeExec({
agent: {
session: { requestHeader: () => ({ config: { provider: "p", model: "m" } }) },
options: {},
},
});
const screenshot = tools.get("browser_screenshot");
const value = (await screenshot?.execute({}, exec)) as { image?: unknown; path: string };
expect(value.image).toBeUndefined();
expect(value.path).toBe(path);
});
});
describe("error and cancellation semantics", () => {
it("surfaces the bsk JSON error envelope with code and hint", async () => {
const { tools } = setup({
"session start": START_REPLY("s1"),
});
await startSession(tools);
const snapshot = tools.get("browser_snapshot");
await expect(snapshot?.execute({}, makeExec())).rejects.toThrow(/no canned response/);
});
it("turns an aborted child into an AbortError", async () => {
const { tools } = setup({ "session start": START_REPLY("s1") });
await startSession(tools);
const snapshot = tools.get("browser_snapshot");
const controller = new AbortController();
controller.abort();
const exec = makeExec({ signal: controller.signal });
await expect(snapshot?.execute({}, exec)).rejects.toMatchObject({ name: "AbortError" });
});
it("maps a missing bsk binary onto install guidance", async () => {
const { ctx, tools } = makeCtx();
const registry = new SessionRegistry(5);
const runner = {
async run(): Promise<BskRunResult> {
throw Object.assign(new Error("spawn bsk ENOENT"), { code: "ENOENT" });
},
killAll() {},
killFor: () => 0,
};
registerTools({
ctx: ctx as never,
runner,
registry,
config: CONFIG,
observation: disabledObservation({ ctx, runner, registry }),
queue: new KeyedExecutor(),
});
const start = tools.get("browser_session_start");
await expect(start?.execute({}, makeExec())).rejects.toThrow(/BrowserSkill must be installed/);
});
});
describe("presentation", () => {
it("presents calls as terminal cards with the bsk command line", () => {
const { tools } = setup({});
const click = tools.get("browser_click");
const view = click?.presentCall?.({ target: "@e1" }) as {
card: string;
title: string;
};
expect(view.card).toBe("terminal");
expect(view.title).toContain("bsk click @e1");
});
it("presents results as terminal cards carrying the rendered text", () => {
const { tools } = setup({});
const click = tools.get("browser_click");
const view = click?.presentResult?.({ target: "@e1" }, {
content: [{ type: "text", text: "clicked" }],
isError: false,
} as never) as { card: string; output: string; exitCode: number };
expect(view).toMatchObject({ card: "terminal", output: "clicked", exitCode: 0 });
});
it("projects the text block of multi-block results (screenshot text + image)", () => {
const { tools } = setup({});
const screenshot = tools.get("browser_screenshot");
const view = screenshot?.presentResult?.({}, {
content: [
{ type: "text", text: "[session s1] screenshot of tab 7" },
{ type: "image", attachment: { attachmentId: "a1" } },
],
isError: false,
} as never) as { card: string; output: string };
expect(view).toMatchObject({ card: "terminal", output: "[session s1] screenshot of tab 7" });
});
});
describe("screenshot scratch file lifecycle", () => {
/** Runner that honors --out like the real bsk (writes the PNG there). */
function outWritingRunner() {
const calls: FakeCall[] = [];
return {
calls,
runner: {
async run(args: string[], options: BskRunOptions = {}): Promise<BskRunResult> {
calls.push({ args, options });
if (args[0] === "session") {
return {
code: 0,
stdout: JSON.stringify({ session_id: "s1", browser_instance_id: "chrome-1" }),
stderr: "",
timedOut: false,
aborted: false,
};
}
if (args[0] === "screenshot") {
const out = args[args.indexOf("--out") + 1];
writeFileSync(out, Buffer.from([0x89, 0x50, 0x4e, 0x47]));
return {
code: 0,
stdout: JSON.stringify({
tab_id: 7,
width: 4,
height: 4,
format: "png",
path: out,
byte_size: 4,
}),
stderr: "",
timedOut: false,
aborted: false,
};
}
return { code: 2, stdout: "{}", stderr: "", timedOut: false, aborted: false };
},
killAll() {},
killFor: () => 0,
} as BskRunner,
};
}
function setupWithRunner(runner: BskRunner, services: Record<string, unknown> = {}) {
const { ctx, tools } = makeCtx(services);
const registry = new SessionRegistry(5);
registerTools({
ctx: ctx as never,
runner,
registry,
config: CONFIG,
observation: disabledObservation({ ctx, runner, registry }),
queue: new KeyedExecutor(),
});
return { tools, registry };
}
const IMAGE_SERVICES = {
attachments: {
imageLimits: { mediaTypes: ["image/png"], maxImageBytes: 1e7, maxMessageImageBytes: 1e7 },
saveImage: async (input: { data: Uint8Array; mediaType: string }) => ({
attachmentId: "att-1",
mediaType: input.mediaType,
bytes: input.data.byteLength,
width: 4,
height: 4,
}),
},
llm: { resolveModelInfo: async () => ({ inputModalities: ["text", "image"] }) },
};
it("deletes the scratch PNG once the bytes are in the attachment store", async () => {
const { runner, calls } = outWritingRunner();
const { tools } = setupWithRunner(runner, IMAGE_SERVICES);
await startSession(tools);
const exec = makeExec({
agent: {
session: { requestHeader: () => ({ config: { provider: "p", model: "vl" } }) },
options: {},
},
});
const screenshot = tools.get("browser_screenshot");
const value = (await screenshot?.execute({}, exec)) as { image?: unknown; path: string };
expect(value.image).toBeDefined();
const out = calls.find((c) => c.args[0] === "screenshot")?.args.at(-1);
expect(out).toBeDefined();
expect(existsSync(out ?? "")).toBe(false);
});
it("keeps the PNG only when it is the model-facing artifact (no store)", async () => {
const { runner, calls } = outWritingRunner();
const { tools } = setupWithRunner(runner);
await startSession(tools);
const screenshot = tools.get("browser_screenshot");
const value = (await screenshot?.execute({}, makeExec())) as { image?: unknown; path: string };
expect(value.image).toBeUndefined();
const out = calls.find((c) => c.args[0] === "screenshot")?.args.at(-1);
expect(existsSync(out ?? "")).toBe(true);
});
});
describe("observation action instrumentation timing", () => {
it("a queued call never overwrites the in-flight action label", async () => {
const { ctx, tools } = makeCtx();
const registry = new SessionRegistry(5);
let releaseNavigate!: () => void;
const navigateGate = new Promise<void>((resolve) => {
releaseNavigate = resolve;
});
const runner: BskRunner = {
async run(args: string[]): Promise<BskRunResult> {
if (args[0] === "session") {
return {
code: 0,
stdout: JSON.stringify({ session_id: "s1", browser_instance_id: "chrome-1" }),
stderr: "",
timedOut: false,
aborted: false,
};
}
if (args[0] === "navigate") {
await navigateGate;
return {
code: 0,
stdout: JSON.stringify({ tab_id: 7, url: "http://x" }),
stderr: "",
timedOut: false,
aborted: false,
};
}
return {
code: 0,
stdout: JSON.stringify({ text: "ok", ref_count: 0, tab_id: 7, truncated: false }),
stderr: "",
timedOut: false,
aborted: false,
};
},
killAll() {},
killFor: () => 0,
};
const queue = new KeyedExecutor();
const observation = new ObservationService({
ctx: ctx as never,
runner,
registry,
queue,
options: { enabled: true, thumbnailIntervalMs: 1500, idleIntervalMs: 8000 },
});
registerTools({ ctx: ctx as never, runner, registry, config: CONFIG, observation, queue });
await startSession(tools);
const navigate = tools.get("browser_navigate");
const snapshot = tools.get("browser_snapshot");
const a = navigate?.execute({ url: "http://x" }, makeExec()) as Promise<unknown>;
// A starts immediately (empty queue) and blocks on the gate.
await vi.waitFor(() =>
expect(observation.getState().find((s) => s.sessionId === "s1")?.action).toBe("navigating"),
);
// B queues behind A: the label must stay "navigating" until A settles.
const b = snapshot?.execute({}, makeExec()) as Promise<unknown>;
await new Promise((resolve) => setTimeout(resolve, 30));
expect(observation.getState().find((s) => s.sessionId === "s1")?.action).toBe("navigating");
releaseNavigate();
await a;
await vi.waitFor(() =>
expect(observation.getState().find((s) => s.sessionId === "s1")?.action).toBe("snapshotting"),
);
await b;
await vi.waitFor(() =>
expect(observation.getState().find((s) => s.sessionId === "s1")?.action).toBe("idle"),
);
observation.dispose();
});
});
@@ -0,0 +1,17 @@
{
"compilerOptions": {
"target": "ES2022",
"module": "ESNext",
"moduleResolution": "Bundler",
"jsx": "react-jsx",
"lib": ["ES2022", "DOM", "DOM.Iterable"],
"noEmit": true,
"esModuleInterop": true,
"forceConsistentCasingInFileNames": true,
"resolveJsonModule": true,
"strict": true,
"skipLibCheck": true,
"types": ["node"]
},
"include": ["src/**/*", "tests/**/*", "tsdown.config.ts", "vitest.config.ts"]
}
@@ -0,0 +1,128 @@
import { readFile } from "node:fs/promises";
import { basename, dirname, resolve as resolvePath } from "node:path";
import { transform } from "lightningcss";
import { defineConfig, type UserConfig } from "tsdown";
/**
* Two build faces of the dual-face package:
* - host: the Cordis tool plugin (lib/index.mjs, ESM, dsh packages external);
* - client: the Web UI bundle (lib/client.cjs, CJS in a type:module package) in dsh's closure-factory shape —
* the artifact registers itself through `window.__ModuleLoader__.load`, with
* platform modules resolved from the loader's frozen module table and every
* other dependency inlined. Mirrors deepseek-harness `packages/client/tsdown.client.ts`.
*/
/** Module specifiers the dsh web shell shares into its frozen module table. */
const CLIENT_EXTERNALS: readonly string[] = [
"react",
"react/jsx-runtime",
"react-dom",
"react-dom/client",
"@deepseek-ai/cordis",
"@deepseek-ai/dsh-client-ui-slots",
"@deepseek-ai/dsh-client-web-react",
"@deepseek-ai/dsh-client-ui-primitives",
"@deepseek-ai/dsh-client-ui-attachment",
"@deepseek-ai/dsh-client-schema-form",
"@deepseek-ai/dsh-client-runtime/client",
];
/** Wire/type layers with no shared runtime identity: safe to inline. */
const INLINE_SAFE = /^@deepseek-ai\/dsh-(host-apiproxy|session|llm|tools|brand|attachment)(\/|$)/;
const VENDORED_LIBRARY = /^@deepseek-ai\/(cosmokit|schemastery)(\/|$)/;
const CSS_VIRTUAL_PREFIX = "\0bsk-css:";
const CSS_VIRTUAL_SUFFIX = ".mjs";
const CLIENT_ID = "dsh-plugin-browserskill";
const client: UserConfig = {
name: `${CLIENT_ID}/client`,
entry: { client: "src/client/index.ts" },
outDir: "lib",
format: "cjs",
platform: "browser",
dts: false,
clean: false,
external: [...CLIENT_EXTERNALS],
// Anything not in the loader module table must inline; an unanswerable
// require() is a guaranteed runtime throw.
noExternal: (id: string) => (CLIENT_EXTERNALS.includes(id) ? undefined : true),
define: {
"process.env.NODE_ENV": JSON.stringify(process.env.NODE_ENV ?? "production"),
"import.meta.env.MODE": JSON.stringify(process.env.NODE_ENV ?? "production"),
"import.meta.env": JSON.stringify({ MODE: process.env.NODE_ENV ?? "production" }),
},
plugins: [
{
// Bundle purity gate: platform modules stay external, wire layers
// inline, every other @deepseek-ai value import is a build error.
name: "bsk-client-bundle-purity",
resolveId(source: string) {
if (!source.startsWith("@deepseek-ai/")) return null;
if (CLIENT_EXTERNALS.includes(source)) return null;
if (VENDORED_LIBRARY.test(source)) return null;
if (INLINE_SAFE.test(source)) return null;
throw new Error(
`client bundle purity: "${source}" is not a platform module or an inline-safe wire layer — ` +
"collaborate through cordis services (type-only imports are erased and never reach this gate)",
);
},
},
{
name: "bsk-css-modules-inline",
resolveId(source: string, importer: string | undefined) {
// `.module.css` gets hashed class maps; `.nomodule.css` is injected
// verbatim (minified) — used for the scope-prefixed BSK utility sheet
// and tokens, whose selectors must survive untouched.
if (!source.endsWith(".module.css") && !source.endsWith(".nomodule.css")) return null;
const abs = importer !== undefined ? resolvePath(dirname(importer), source) : source;
return CSS_VIRTUAL_PREFIX + abs + CSS_VIRTUAL_SUFFIX;
},
async load(virtualId: string) {
if (!virtualId.startsWith(CSS_VIRTUAL_PREFIX)) return null;
const fileId = virtualId.slice(CSS_VIRTUAL_PREFIX.length, -CSS_VIRTUAL_SUFFIX.length);
this.addWatchFile(fileId);
const source = await readFile(fileId);
const isModule = fileId.endsWith(".module.css");
const { code, exports: cssExports } = transform({
filename: fileId,
code: source,
...(isModule ? { cssModules: { pattern: "[hash]_[local]" } } : {}),
minify: true,
});
const classMap: Record<string, string> = {};
for (const [local, exp] of Object.entries(cssExports ?? {})) classMap[local] = exp.name;
return [
`const css = ${JSON.stringify(code.toString())};`,
`const tagId = ${JSON.stringify(`${CLIENT_ID}/${basename(fileId)}`)};`,
"if (typeof document !== 'undefined' && document.querySelector('style[data-plugin-css=' + JSON.stringify(tagId) + ']') === null) {",
" const tag = document.createElement('style');",
` tag.dataset.plugin = ${JSON.stringify(CLIENT_ID)};`,
" tag.dataset.pluginCss = tagId;",
" tag.textContent = css;",
" document.head.appendChild(tag);",
"}",
`export default ${JSON.stringify(classMap)};`,
].join("\n");
},
},
],
outputOptions: {
entryFileNames: "client.cjs",
banner: `window.__ModuleLoader__.load({ id: ${JSON.stringify(CLIENT_ID)}, factory: (require) => {`,
footer: "return module.exports; } });",
intro: "var module = { exports: {} }; var exports = module.exports;",
},
};
export default defineConfig([
{
entry: ["src/index.ts"],
format: ["esm"],
dts: true,
outDir: "lib",
// dsh host packages are provided by the profile the plugin is installed into.
external: [/^@deepseek-ai\//],
},
client,
]);
@@ -0,0 +1,32 @@
import { createRequire } from "node:module";
import path from "node:path";
import { defineConfig } from "vitest/config";
const require = createRequire(import.meta.url);
// @browser-skill/ui ships raw source and peers on react ^19, while the dsh
// shell — and therefore the production client bundle, where react is
// external — runs react 18. Pin every react import in tests to this
// package's react 18 copy, or elements created inside ui components come out
// as react-19 "transitional" elements that the react-18 renderer rejects.
const reactDir = path.dirname(require.resolve("react/package.json"));
const reactDomDir = path.dirname(require.resolve("react-dom/package.json"));
export default defineConfig({
resolve: {
alias: {
react: reactDir,
"react-dom": reactDomDir,
},
},
test: {
environment: "node",
include: ["tests/**/*.test.ts", "tests/**/*.test.tsx"],
server: {
deps: {
// dsh client packages ship ESM with CSS imports; run them through the
// transform pipeline (where CSS is stubbed) instead of Node's loader.
inline: [/@deepseek-ai\//],
},
},
},
});
+3373 -10
View File
File diff suppressed because it is too large Load Diff