Keep the expanded command-reference tables from feat/upload-download
(including the new File Transfer section) over main's compact two-line
format.
Co-authored-by: Cursor <cursoragent@cursor.com>
The Document PiP window had no close/return entry once popped out —
users could not dismiss it from inside the window. Add an onClosePip
callback and render a close button in the PiP header; clicking it
calls pipWindow.close() and the content returns to its carrier.
The observation thumbnail publisher hardcoded mediaType as "image/png"
regardless of the actual capture bytes. Reuse the existing
sniffImageMediaType helper (exported from image.ts) so the HTTP
thumbnail endpoint declares the correct Content-Type when Chromium
returns JPEG captures.
Closes#153 (Bug 2 fully; Bug 1 was fixed in #135, this covers the
remaining observation-thumbnail path).
Co-authored-by: Cursor <cursoragent@cursor.com>
The borrow confirmation overlay was easily missed because the Agent
Window stole focus and the 5-second auto-deny was too aggressive for
users who had context-switched away.
- Focus the user window proactively when a borrow request is issued so
the overlay is immediately visible even when the Agent Window was
last focused.
- Raise CONFIRMATION_TIMEOUT_MS from 5 s to 60 s, giving users enough
time to notice and respond — the OS notification with
requireInteraction: true now has a matching window of usefulness.
Co-authored-by: Cursor <cursoragent@cursor.com>
main rewrote SKILL.md into a leaner core (#167) while this branch documented
the opt-in --probe-hover flag against the old structure. Keep main's rewritten
prose and re-apply the probe-hover guidance on top of it: the [has-submenu] /
[expanded] markers, when reaching for --probe-hover is warranted, and its slot
in the page-reading escalation ladder ahead of get-html/screenshot.
Co-authored-by: Cursor <cursoragent@cursor.com>
The previous wording asked the agent to notice when a page "looks like"
it hides content behind hover. That is not something an agent reading a
text tree can judge: hover-only content is invisible by definition, and
the marker that used to point at it came from the probing that is now off
by default.
Replaces it with a condition the agent can actually evaluate — an
expected control is missing and no [has-submenu] marker points at a
trigger — and places it on the escalation ladder ahead of get-html and
screenshot, which is where that dead end is currently reached.
Also documents [has-submenu]/[expanded] as the primary hover signal. It
comes from aria-haspopup in the accessibility tree, costs nothing, and is
unaffected by probing being off, so it covers menus that are marked up
correctly.
Every tool.observe moved the real cursor across the page to look for
hover-only menus and tooltips. Measured against real Chrome, that cost a
median of 2.5s and up to 4.7s per observation — 73% to 99% of total
observe wall clock — and returned 3 hits across 12 pages. A positive
control confirms the probes work; the yield on real pages simply does not
justify putting them on the default path.
observe now probes only when asked, via `probe_hover` (`--probe-hover`).
Agents that know which element hides a menu should keep using
`bsk hover <ref>`, which is both cheaper and precise. A held hover latch
still suppresses probing, since probing would move the cursor off the
element the caller is deliberately holding.
Correctness fixes that apply whether or not probing is requested:
- Hover probing ran between the DOM snapshot and the accessibility tree
fetch, so a hover that opened a menu left one half of an observation
describing the page before the change and the other half after it. Both
chains now run after all capture is complete.
- The overlay bypass toggled once per chain and swallowed restore
failures, which could leave the agent overlay hidden for the rest of
the tab's life. It is now reference counted across both chains, and a
failed restore is reported instead of discarded.
- Observations report whether they hovered the page and whether that
revealed content the static tree lacks, so callers can judge how far
the live page may have drifted from the returned snapshot.
Probe budgets were checked at the top of the loop, so a candidate could
start with 1ms remaining and still run a full settle window, overshooting
by up to 634ms. Candidates are now admitted only if they fit. Raising
MAX_HOVER_PROBE_MS to 2600 is part of that fix rather than a relaxation:
2000 was never the real ceiling, and enforcing it literally would have
silently dropped a candidate the positive control depends on.
Tooltip misses are cached too, with a shorter TTL than hits, so repeat
observations of the same page no longer re-probe every unlabelled control
from scratch.
The CLI skill had grown to 20.8 KB of command tables and flag catalogs that
`bsk --help` already answers at runtime, so every agent paid for them on every
task. Rewrite both skills around the durable rules an agent cannot rediscover
-- session lifecycle, the observe/act/observe loop, the Agent Window boundary,
and the stop-on-success contract -- and delegate flag details to the CLI's own
help output. The CLI skill drops to 6.9 KB (-67%) and the DSH skill to 5.1 KB.
Keep the parts that help discovery has no answer for: the literal
`[hover first: ...]` observation marker, the `request-help` outcome values, the
complete command-name inventory, and the required flags agents guess wrong
(`fill --value`, `select --value`, `screenshot --out`, `emulate --device`).
Also bind `--no-focus` explicitly to `bsk session start`, since a bare mention
led agents to pass it to `bsk navigate`.
Validated with 36 agent runs of the `core` eval suite against a real browser,
comparing the old and new instructions with everything else held constant.
Pass rate did not regress (the lean version was better on `form-controls`) and
token cost per task came out even. The inventory and flag signatures cut
`--help` round-trips from 1.7 to 0.7 per task and removed the invented
`bsk back` / `bsk forward` calls the first draft produced.
Tighten the DSH size assertion to a range so the budget is bounded from both
sides and accidental truncation fails the test.
Co-authored-by: Cursor <cursoragent@cursor.com>
SessionManager.classifyNewTab's `initializing` branch relied on a
windowInitializing flag that start() cleared *before* the home tab's
chrome.tabs.onCreated event was dispatched (the event is asynchronous), so
the branch never ran in production and the home tab's classification was
subject to a timing race. Match the home tab by its tab id instead — the id
is already returned by ensureActiveTab and is reliable regardless of event
ordering.
Also refresh the overlay-bridge docstring: overlay.ready now makes the
background *push* the per-tab overlay state (and the Agent Window also
pushes on tab create / control-mode change), rather than a request/reply.
tool.tab_create increments pendingAgentTabCount *before* chrome.tabs.create
so the onCreated listener can tell an agent tab from a user-opened tab. When
the create failed, no onCreated event arrived and the counter leaked, so the
next tab the user opened was misclassified as agent-created — the exact
confusion the counter exists to prevent, a variant of issue #57.
handleTabCreate now releases the slot on failure via
SessionManager.releaseAgentTabPending. An abort (cancelled) is excluded: the
tab did open (and was cleaned up), so its onCreated still consumes the slot.
Added a regression test asserting that after a failed tab_create, a
user-opened tab is still classified as 'user' (not 'agent').
- session_stop: exclude tabs still tracked in agentCreatedTabs from the
release decision so a tab that failed to close is no longer mistaken for
a user tab (which previously kept the window open and leaked the tab).
- Distinguish user-created tabs from agent-created tabs in the Agent Window
via a pending-tab counter consumed by chrome.tabs.onCreated, so tabs the
user opens themselves (new-tab button / Cmd+T / open in new tab) are never
taken over by the BrowserSkill control mask.
- background overlay: decide per-tab state BEFORE showing controls
(overlayStateForTab + pushOverlayStateForTab) so a user tab receives hidden
on its first overlay.ready ping instead of flashing control then hiding.
- AgentWindowApi.ensureActiveTab now resolves with the activated/created tab
id instead of void, so SessionManager can record homeTabId. This fixes the
tsc failure flagged in review (the interface previously returned
Promise<void> while callers treated the result as number | null).
- Wire the production tabManagement.tabs / tabsQuery deps into
tool.session_stop, both in the dispatcher and in the disconnect cleanup.
They were never injected, which left the agent-tab cleanup and the
window-release path as dead code outside tests (behaviour identical to
main). Added a dispatcher-level integration test that fails without the
wiring: a surviving user tab must release the window, not close it.
dsh 0.1.0-rc.8 imports and registers the client bundle by export const name, so a leftover dsh-plugin-browserskill id fails to load the 0.1.1 web UI.
Co-authored-by: Cursor <cursoragent@cursor.com>
The exit code reflects RPC success only, not JS success. Document that programmatic callers must use --json and check the .ok field to detect JS errors. Addresses the discoverability gap raised in issue #133.
Agents were stopping on popup and status copy that looked like a hard incompatibility. Keep the connection usable, reframe exit 5 as an unknown method, and ask users to upgrade instead of aborting.
Co-authored-by: Cursor <cursoragent@cursor.com>
Register bundled browser instructions at agent scope, prioritize foreground commands over thumbnail capture, and unify cancellation and session cleanup.
