browser_screenshot declared every capture as image/png regardless of the
actual bytes. On Chromium/Edge builds where captureVisibleTab returns JPEG
bytes, the stored attachment reference recorded mediaType "image/png" against
JPEG bytes, so dsh-attachment-local's read-time metadata verification fails
with ATTACHMENT_CORRUPT ("Stored attachment metadata does not match its
reference"), surfacing as code UNKNOWN and triggering endless LLM retries.
- image.ts: sniff the capture's magic bytes (PNG/JPEG/WebP/GIF) and declare
that real media type to attachments.saveImage, instead of always "image/png".
- tools.ts: assemble the tool result from ref.mediaType (the store's verified
type) rather than re-declaring "image/png" as const, and relax the output
schema's const to the accepted media-type enum.
- tests: add a regression case asserting JPEG capture bytes surface as
image/jpeg.
Co-Authored-By: DeepSeek Harness <noreply@deepseek.com>
Stop writing overlay thumbnails into the durable attachment store. Each
session now holds only the current and previous PNG in process memory,
reuses the id when the hash is unchanged, and drops the rest on rotate,
remove, or dispose.
Progressive disclosure, final stage: with lazyTools on (default), the eleven
browser_* tool schemas stay out of the system prompt — the browser-skill
catalog entry is the sole advertisement. One successful skill invocation
(model tool call via tools/result, or a /browser-skill user gesture via the
session append feed) registers the suite for the process lifetime, idempotent
on repeats; session resume is covered by scanning durable events for a past
tool/call+tool/result pair (or skill-invocation message) on session entry and
at apply time. lazyTools: false keeps the legacy always-on registration.
Investigation (recorded in the ticket): tools/result carries normalized
name/arguments with an isError discriminant; dsh 0.1 has no official global
visibility switch (ctx.tools.restrict is agent-scoped and throws from a plain
context), making conditional registration the intended pattern; per-step
assembly picks the new suite up on the next step, and visibility changes are
first-class inputs of tool-skill's catalog digest. registerTools now returns
the combined disposer via a prototype-preserving tracking overlay; the skill
prelude tells the model the browser_* tools unlock on skill load.
Tests: +8 (hidden-before/reveal/idempotent/gesture/history hit+miss/dispose/
scanner pairing/two-state apply wiring). 124/124 green.
Blocking:
- B1: delete screenshot scratch PNGs — observation frames reuse one fixed
per-session path and unlink after every read (finally); browser_screenshot
unlinks once the bytes are in the attachment store (kept only when the file
itself is the model-facing artifact).
- B2: replicate dsh's browser-trust fence on /bsk-observation/*: loopback
Host only, Origin must match Host, sec-fetch-site: cross-site refused, POST
requires application/json; README documents the loopback trust premise and
the 0.0.0.0 warning.
Major:
- M1: KeyedExecutor tail chains previous+task (allSettled) so a task aborted
while queued cannot release the next one into the running session.
- M2: client tracks the live frame per session — replacing/removing/reset
revokes the old blob URL at once; loads settling after replacement never
resurrect it.
Minor: drop replaced thumbRefs (+on remove); beginAction fires inside the
queue (no label overwrite while queued, no idle flash); half-initialized
session cleanup stops through the queue with one retry; SSE (re)open refetches
/state (+ snapshot flag renamed subscribed); install probe uses --version (no
daemon spawn); emulate mobile documents the width+height requirement (the
daemon refuses it alone — verified).
Nit: single BskRunOptions declaration; tails map entry dropped on drain; SSE
cleanup on res close; popOut catches requestWindow rejections; resize handle
gains keyboard control (arrows, 16px steps, aria value attrs).
Tests: +14 (fence rules, scratch lifecycle, queue race, blob revocation,
instrumentation timing, SSE resync). 113/113 green.
The daemon accepts only one unfinished command per session; the
thumbnail loop racing a model tool call produced 'session already has
an unfinished command' failures on both sides (found by the Phase 1-C
e2e: navigate/snapshot/click all errored while captures kept running).
Add a KeyedExecutor: every command for a session — tool calls and
observation captures alike — runs FIFO; queued tasks reject early on
abort, running tasks keep the runner's signal-driven kill.
Per-session live observation for owned sessions only, feeding the
upcoming client overlay:
- State model: SessionObservation (sessionId/url/action/since/
thumbnailAttachmentId/lastError), one entry per owned session;
added on start, removed on stop, cleared on dispose.
- Instrumentation: the shared runBsk wrapper begins/ends an action
(tool name mapped to a verb) around every model-facing call and tags
the child with its session id; action end triggers an immediate
thumbnail refresh.
- Thumbnail loop: 1.5s cadence while active, 8s idle downclock, x3
failure backoff, silent frame retention on errors; captures run
OUTSIDE the instrumentation (no action events, no current-pointer
movement) and land in the attachment store by reference.
- Interrupt: runner.killFor(tag) kills exactly the tagged in-flight bsk
children — chat-Stop-equivalent semantics for the current or a named
owned session; foreign ids and the no-in-flight case return false.
- Remote seam: dsh 0.1's Typert Remote pipeline and forwarded-event
allowlist are closed to out-of-tree packages, so state/events/
interrupt are served over the documented webServer route seam:
GET /bsk-observation/state, GET /bsk-observation/events (SSE),
POST /bsk-observation/interrupt; routes mount only when a webServer
service exists (headless compositions skip them).
- Config: observationEnabled (true), thumbnailIntervalMs (1500),
idleIntervalMs (8000).
Tests (+13, 72 total): state machine + event stream, cadence fast/idle/
backoff, headless no-store silence, observation-traffic isolation,
interrupt routing (default/specified/none/foreign), HTTP route
handlers, disabled-webServer no-op.
Tighten the shared-daemon boundary from 'cleanup only touches owned
sessions' to 'the plugin only ever sees and operates on sessions it
created':
- browser_session_list no longer queries the daemon at all; it returns
the plugin's own session table (with the current marker), so foreign
sessions cannot even be enumerated through the plugin.
- Every tool's optional session argument must name a plugin-created
session; foreign/unknown ids are rejected with a clear error before
any command reaches the daemon (the reference-adoption channel is
gone).
- Unchanged: dispose stops exactly the owned set, stop requires
ownership, the start reservation protocol, and handle-precise child
kills.
Tests updated (59): explicit-foreign rejection without daemon contact,
registry-only listing, reservation race, stale-handle-tolerant dispose.
e2e: a manually started foreign session was rejected on reference,
absent from session_list, refused on stop, and survived dispose while
both plugin sessions were cleaned up.
Review hardening for daemon-sharing deployments (other agents,
terminals, or dsh instances on the same bsk daemon):
- Ownership split: only sessions created by browser_session_start are
'owned'. Explicitly referenced foreign sessions are still tracked for
the current-session pointer, but browser_session_stop refuses them
and unload cleanup stops exactly the owned set — no path can stop a
session this plugin did not create.
- Start race: the capacity check is now a synchronous
reserveStart/completeStart/abandonStart protocol, so two concurrent
starts can never both pass the cap (check-and-reserve is atomic on
the event loop); a rejected or failed start never leaks a session.
- Stale handles: dispose already swallowed per-stop errors; the owned
set is computed at dispose time so an externally stopped session is a
no-op, not a failure.
Tests (+7): reservation race (two concurrent starts, cap 1 -> exactly
one spawn), foreign-stop refusal (no stop command reaches the daemon),
reference adoption never becoming owned, resolveForStop pointer
stability, and an apply-level dispose test proving only owned sessions
are stopped while stale stops are tolerated. e2e: a manually started
foreign session survived the plugin's full lifecycle while both plugin
sessions were cleaned up.
browser_screenshot on an image-capable route returns text + image
blocks; the shared result presenter required exactly one text block and
bailed to the generic card (tool name + raw args). Project the first
text block instead so the terminal card shows in both modes.
The concurrency cap was checked in registry.add() AFTER
`bsk session start` had already created the session, so a rejected
start leaked a live session outside the plugin's tracking (found by the
real-browser e2e: the rejected third session stayed in
`bsk session list` after dispose). Check capacity before spawning;
registry.add() keeps the same check as a backstop.
Add a new monorepo package that registers BrowserSkill (bsk) browser
automation as model-visible tools in DeepSeek Harness (dsh):
- 11 tools mapping to bsk CLI --json commands: session start/stop/list,
navigate, snapshot, observe, click, fill, press, screenshot, emulate
- multi-session support: optional session arg with a current-session
pointer, session id echoed in every result, configurable concurrency
cap (default 5), and dispose-time cleanup of plugin-started sessions
- cancellation: exec.signal aborts kill the underlying bsk child process
- terminal-style UI cards (command line as the call card, output as the
result card); screenshots inline the PNG through the host attachment
store when the model route accepts image input, else return the path
- config: bskPath / defaultTimeoutMs / maxSessions via Schemastery schema
- install guidance when the bsk binary is missing; bsk JSON error
envelopes (code/message/hint) surfaced to the model
- unit tests mock the bsk runner (no real browser required); the root
lint script now also runs the package's typecheck + tests so the
existing CI frontend job covers the new package (a dedicated CI job
diff is provided in the PR; the pushing token lacks workflow scope)
