Register bundled browser instructions at agent scope, prioritize foreground commands over thumbnail capture, and unify cancellation and session cleanup.
browser_screenshot declared every capture as image/png regardless of the
actual bytes. On Chromium/Edge builds where captureVisibleTab returns JPEG
bytes, the stored attachment reference recorded mediaType "image/png" against
JPEG bytes, so dsh-attachment-local's read-time metadata verification fails
with ATTACHMENT_CORRUPT ("Stored attachment metadata does not match its
reference"), surfacing as code UNKNOWN and triggering endless LLM retries.
- image.ts: sniff the capture's magic bytes (PNG/JPEG/WebP/GIF) and declare
that real media type to attachments.saveImage, instead of always "image/png".
- tools.ts: assemble the tool result from ref.mediaType (the store's verified
type) rather than re-declaring "image/png" as const, and relax the output
schema's const to the accepted media-type enum.
- tests: add a regression case asserting JPEG capture bytes surface as
image/jpeg.
Co-Authored-By: DeepSeek Harness <noreply@deepseek.com>
Cordis contexts are fiber-owned: assigning onto an Object.create() overlay of
the apply-time context fails with 'cannot set property tools in multiple
fibers' when the reveal fires from the tools/result listener fiber (caught by
the e2e: the skill invocation matched but the suite never registered).
registerTools now tracks disposers through a plain closure over the captured
context — no context mutation anywhere. Also documents lazyTools in the
README and hardens the reveal against a failed registration (log and stay
hidden instead of latching).
Verified end to end against the real stack: first request carries 0 browser_*
schemas (skill catalog entry only); after the scripted skill invocation the
next request carries exactly the 11 browser_* tools and browser_session_start
dispatches for real.
Progressive disclosure, final stage: with lazyTools on (default), the eleven
browser_* tool schemas stay out of the system prompt — the browser-skill
catalog entry is the sole advertisement. One successful skill invocation
(model tool call via tools/result, or a /browser-skill user gesture via the
session append feed) registers the suite for the process lifetime, idempotent
on repeats; session resume is covered by scanning durable events for a past
tool/call+tool/result pair (or skill-invocation message) on session entry and
at apply time. lazyTools: false keeps the legacy always-on registration.
Investigation (recorded in the ticket): tools/result carries normalized
name/arguments with an isError discriminant; dsh 0.1 has no official global
visibility switch (ctx.tools.restrict is agent-scoped and throws from a plain
context), making conditional registration the intended pattern; per-step
assembly picks the new suite up on the next step, and visibility changes are
first-class inputs of tool-skill's catalog digest. registerTools now returns
the combined disposer via a prototype-preserving tracking overlay; the skill
prelude tells the model the browser_* tools unlock on skill load.
Tests: +8 (hidden-before/reveal/idempotent/gesture/history hit+miss/dispose/
scanner pairing/two-state apply wiring). 124/124 green.
Blocking:
- B1: delete screenshot scratch PNGs — observation frames reuse one fixed
per-session path and unlink after every read (finally); browser_screenshot
unlinks once the bytes are in the attachment store (kept only when the file
itself is the model-facing artifact).
- B2: replicate dsh's browser-trust fence on /bsk-observation/*: loopback
Host only, Origin must match Host, sec-fetch-site: cross-site refused, POST
requires application/json; README documents the loopback trust premise and
the 0.0.0.0 warning.
Major:
- M1: KeyedExecutor tail chains previous+task (allSettled) so a task aborted
while queued cannot release the next one into the running session.
- M2: client tracks the live frame per session — replacing/removing/reset
revokes the old blob URL at once; loads settling after replacement never
resurrect it.
Minor: drop replaced thumbRefs (+on remove); beginAction fires inside the
queue (no label overwrite while queued, no idle flash); half-initialized
session cleanup stops through the queue with one retry; SSE (re)open refetches
/state (+ snapshot flag renamed subscribed); install probe uses --version (no
daemon spawn); emulate mobile documents the width+height requirement (the
daemon refuses it alone — verified).
Nit: single BskRunOptions declaration; tails map entry dropped on drain; SSE
cleanup on res close; popOut catches requestWindow rejections; resize handle
gains keyboard control (arrows, 16px steps, aria value attrs).
Tests: +14 (fence rules, scratch lifecycle, queue race, blob revocation,
instrumentation timing, SSE resync). 113/113 green.
The daemon accepts only one unfinished command per session; the
thumbnail loop racing a model tool call produced 'session already has
an unfinished command' failures on both sides (found by the Phase 1-C
e2e: navigate/snapshot/click all errored while captures kept running).
Add a KeyedExecutor: every command for a session — tool calls and
observation captures alike — runs FIFO; queued tasks reject early on
abort, running tasks keep the runner's signal-driven kill.
Per-session live observation for owned sessions only, feeding the
upcoming client overlay:
- State model: SessionObservation (sessionId/url/action/since/
thumbnailAttachmentId/lastError), one entry per owned session;
added on start, removed on stop, cleared on dispose.
- Instrumentation: the shared runBsk wrapper begins/ends an action
(tool name mapped to a verb) around every model-facing call and tags
the child with its session id; action end triggers an immediate
thumbnail refresh.
- Thumbnail loop: 1.5s cadence while active, 8s idle downclock, x3
failure backoff, silent frame retention on errors; captures run
OUTSIDE the instrumentation (no action events, no current-pointer
movement) and land in the attachment store by reference.
- Interrupt: runner.killFor(tag) kills exactly the tagged in-flight bsk
children — chat-Stop-equivalent semantics for the current or a named
owned session; foreign ids and the no-in-flight case return false.
- Remote seam: dsh 0.1's Typert Remote pipeline and forwarded-event
allowlist are closed to out-of-tree packages, so state/events/
interrupt are served over the documented webServer route seam:
GET /bsk-observation/state, GET /bsk-observation/events (SSE),
POST /bsk-observation/interrupt; routes mount only when a webServer
service exists (headless compositions skip them).
- Config: observationEnabled (true), thumbnailIntervalMs (1500),
idleIntervalMs (8000).
Tests (+13, 72 total): state machine + event stream, cadence fast/idle/
backoff, headless no-store silence, observation-traffic isolation,
interrupt routing (default/specified/none/foreign), HTTP route
handlers, disabled-webServer no-op.
Tighten the shared-daemon boundary from 'cleanup only touches owned
sessions' to 'the plugin only ever sees and operates on sessions it
created':
- browser_session_list no longer queries the daemon at all; it returns
the plugin's own session table (with the current marker), so foreign
sessions cannot even be enumerated through the plugin.
- Every tool's optional session argument must name a plugin-created
session; foreign/unknown ids are rejected with a clear error before
any command reaches the daemon (the reference-adoption channel is
gone).
- Unchanged: dispose stops exactly the owned set, stop requires
ownership, the start reservation protocol, and handle-precise child
kills.
Tests updated (59): explicit-foreign rejection without daemon contact,
registry-only listing, reservation race, stale-handle-tolerant dispose.
e2e: a manually started foreign session was rejected on reference,
absent from session_list, refused on stop, and survived dispose while
both plugin sessions were cleaned up.
Review hardening for daemon-sharing deployments (other agents,
terminals, or dsh instances on the same bsk daemon):
- Ownership split: only sessions created by browser_session_start are
'owned'. Explicitly referenced foreign sessions are still tracked for
the current-session pointer, but browser_session_stop refuses them
and unload cleanup stops exactly the owned set — no path can stop a
session this plugin did not create.
- Start race: the capacity check is now a synchronous
reserveStart/completeStart/abandonStart protocol, so two concurrent
starts can never both pass the cap (check-and-reserve is atomic on
the event loop); a rejected or failed start never leaks a session.
- Stale handles: dispose already swallowed per-stop errors; the owned
set is computed at dispose time so an externally stopped session is a
no-op, not a failure.
Tests (+7): reservation race (two concurrent starts, cap 1 -> exactly
one spawn), foreign-stop refusal (no stop command reaches the daemon),
reference adoption never becoming owned, resolveForStop pointer
stability, and an apply-level dispose test proving only owned sessions
are stopped while stale stops are tolerated. e2e: a manually started
foreign session survived the plugin's full lifecycle while both plugin
sessions were cleaned up.
browser_screenshot on an image-capable route returns text + image
blocks; the shared result presenter required exactly one text block and
bailed to the generic card (tool name + raw args). Project the first
text block instead so the terminal card shows in both modes.
The concurrency cap was checked in registry.add() AFTER
`bsk session start` had already created the session, so a rejected
start leaked a live session outside the plugin's tracking (found by the
real-browser e2e: the rejected third session stayed in
`bsk session list` after dispose). Check capacity before spawning;
registry.add() keeps the same check as a backstop.
Add a new monorepo package that registers BrowserSkill (bsk) browser
automation as model-visible tools in DeepSeek Harness (dsh):
- 11 tools mapping to bsk CLI --json commands: session start/stop/list,
navigate, snapshot, observe, click, fill, press, screenshot, emulate
- multi-session support: optional session arg with a current-session
pointer, session id echoed in every result, configurable concurrency
cap (default 5), and dispose-time cleanup of plugin-started sessions
- cancellation: exec.signal aborts kill the underlying bsk child process
- terminal-style UI cards (command line as the call card, output as the
result card); screenshots inline the PNG through the host attachment
store when the model route accepts image input, else return the path
- config: bskPath / defaultTimeoutMs / maxSessions via Schemastery schema
- install guidance when the bsk binary is missing; bsk JSON error
envelopes (code/message/hint) surfaced to the model
- unit tests mock the bsk runner (no real browser required); the root
lint script now also runs the package's typecheck + tests so the
existing CI frontend job covers the new package (a dedicated CI job
diff is provided in the PR; the pushing token lacks workflow scope)
Zhang GH