drakezhang 05a36075b8 perf(vom): make active hover probing opt-in on observe
Every tool.observe moved the real cursor across the page to look for
hover-only menus and tooltips. Measured against real Chrome, that cost a
median of 2.5s and up to 4.7s per observation — 73% to 99% of total
observe wall clock — and returned 3 hits across 12 pages. A positive
control confirms the probes work; the yield on real pages simply does not
justify putting them on the default path.

observe now probes only when asked, via `probe_hover` (`--probe-hover`).
Agents that know which element hides a menu should keep using
`bsk hover <ref>`, which is both cheaper and precise. A held hover latch
still suppresses probing, since probing would move the cursor off the
element the caller is deliberately holding.

Correctness fixes that apply whether or not probing is requested:

- Hover probing ran between the DOM snapshot and the accessibility tree
  fetch, so a hover that opened a menu left one half of an observation
  describing the page before the change and the other half after it. Both
  chains now run after all capture is complete.
- The overlay bypass toggled once per chain and swallowed restore
  failures, which could leave the agent overlay hidden for the rest of
  the tab's life. It is now reference counted across both chains, and a
  failed restore is reported instead of discarded.
- Observations report whether they hovered the page and whether that
  revealed content the static tree lacks, so callers can judge how far
  the live page may have drifted from the returned snapshot.

Probe budgets were checked at the top of the loop, so a candidate could
start with 1ms remaining and still run a full settle window, overshooting
by up to 634ms. Candidates are now admitted only if they fit. Raising
MAX_HOVER_PROBE_MS to 2600 is part of that fix rather than a relaxation:
2000 was never the real ceiling, and enforcing it literally would have
silently dropped a candidate the positive control depends on.

Tooltip misses are cached too, with a shorter TTL than hits, so repeat
observations of the same page no longer re-probe every unlabelled control
from scratch.
2026-09-02 10:59:18 +08:00
2026-06-22 17:36:26 +08:00
2026-06-22 17:36:26 +08:00
2026-06-22 17:36:26 +08:00
2026-08-26 14:59:09 +08:00
2026-08-26 14:59:09 +08:00
2026-06-22 17:36:26 +08:00
2026-06-22 17:36:26 +08:00
2026-07-09 12:42:42 +08:00
2026-06-22 17:36:26 +08:00
2026-06-22 17:36:26 +08:00

BrowserSkill

BrowserSkill banner

Let AI agents use your browser without interrupting your work.

English · 中文

BrowserSkill connects Cursor, Claude Code, Codex, OpenClaw, CodeBuddy, WorkBuddy, Pi, Hermes Agent, DeepSeek Harness, and other AI agents to your already logged-in browser.

Need the agent to touch a tab you already have open? It must borrow that tab explicitly, return it when the task is done, and leave the rest of your browser alone.

https://github.com/user-attachments/assets/db782c92-b1d4-4aae-a255-039675937a90

BrowserSkill Advantages

  • Reuse real login state: Agents can work with sites you are already signed into, without separate test accounts.
  • Keep working uninterrupted: browser tasks run in a separate, visible Agent Window, so you can keep using your own browser.
  • Support any Agent: any Agent that can call a shell can use BrowserSkill through the bsk CLI, with no lock-in to a specific model, Agent framework, or harness.
  • Built-in human-in-loop: when a task hits captcha, login, confirmation dialogs, or other human-only steps, the Agent can ask you to take over and then continue afterwards.

Runtime Environment

BrowserSkill has two local runtime pieces: the bsk CLI/daemon and the browser extension.

Runtime Support
Operating systems macOS (Apple Silicon and Intel), Linux (x64 and ARM64), Windows x64
Browsers Chrome and Microsoft Edge are supported; other Chromium-based browsers are expected to work when they support unpacked Chromium extensions; Firefox is planned

Quick Start

Install with your Agent (recommended)

Already using Cursor, Claude Code, Codex, or another shell-capable agent? Just copy this one line and send it to your agent — it will install the CLI and skill for you, then walk you through loading the extension:

Set up browser-skill on this machine by following https://raw.githubusercontent.com/Tencent/BrowserSkill/main/AGENT_INSTALL.md
Manual install

Install the CLI, then install the extension from the Chrome Web Store or Edge Add-ons.

1. Install the bsk CLI

macOS / Linux (recommended — installs to ~/.local/bin):

curl -fsSL https://raw.githubusercontent.com/Tencent/BrowserSkill/main/install.sh | sh

Windows (PowerShell — installs to ~/.local/bin):

irm https://raw.githubusercontent.com/Tencent/BrowserSkill/main/install.ps1 | iex

Verify the binary:

bsk --version

2. Install the browser extension

Install BrowserSkill from your browser's store:

Browser Store listing
Chrome Chrome Web Store
Microsoft Edge Edge Add-ons

On other Chromium-based browsers, install the Chrome Web Store build.

3. Install the skill

BrowserSkill ships a skill that teaches your agent harness how to use bsk. For these harnesses, install it in one step:

Cursor
Cursor
Claude Code
Claude Code
Codex
Codex
OpenClaw
OpenClaw
CodeBuddy
CodeBuddy
WorkBuddy
WorkBuddy
Pi
Pi
Hermes Agent
Hermes Agent

bsk install-skill

Use Space to select the Agent harness you want to install into, then press Enter to install the skill. Run bsk install-skill --list to see internal variants and install paths.

Other shell-capable agent harnesses are supported too. Copy skill/SKILL.md into your harness's skills directory as browser-skill/SKILL.md to install the skill manually. DeepSeek Harness uses a dedicated plugin instead — see DeepSeek Harness plugin.

Start a new Agent session and write a prompt that needs the browser, for example:

/browser-skill open example.com and summarize what is on the page.

DeepSeek Harness plugin

Using DeepSeek Harness (dsh)? BrowserSkill ships a first-class dsh plugin on npm as @wxg-prc-cpg/browser-skill-dsh-plugin. It injects native browser_* tools (no shelling out to bsk) and a live Web UI overlay of each Agent Window.

Add it to a dsh profile, then start that profile:

dsh plugin --profile web add @wxg-prc-cpg/browser-skill-dsh-plugin
dsh --profile web

The plugin carries its own copy of the skill, so bsk install-skill is not needed for dsh — but the bsk CLI and the browser extension are still prerequisites. See the plugin README for the tool list, configuration, and the observation overlay.

How It Works

BrowserSkill is a local bridge between your agent harness and your browser.

flowchart TB
  subgraph Harness["Agent Harness"]
    Agent["Cursor / Claude Code / Codex / OpenClaw"]
  end

  subgraph Local["Your Machine"]
    CLI["bsk CLI"]
    Daemon["bsk daemon"]
    Extension["BrowserSkill extension"]
  end

  subgraph Browser["Browser Profile"]
    AgentWindow["Agent Window"]
    UserWindows["Your normal browser windows"]
  end

  Agent -->|"shell: bsk ..."| CLI
  CLI -->|"local IPC"| Daemon
  Daemon -->|"WebSocket on 127.0.0.1"| Extension
  Extension -->|"automates"| AgentWindow
  Extension -.->|"borrow tab only when asked"| UserWindows

  style AgentWindow fill:#fff4e6,stroke:#f59e0b,stroke-width:2px,color:#111827
  style UserWindows fill:#f8fafc,stroke:#cbd5e1,color:#334155

The agent never talks to the browser directly. It asks the bsk CLI to perform a browser task; the local daemon routes that request to the extension; the extension runs it in an Agent Window. DeepSeek Harness takes the same path through the plugin: the agent calls injected browser_* tools, and the plugin invokes bsk on its behalf.

For Developers

The repository is a Cargo + pnpm workspace:

  • crates/bsk-cli — bsk CLI and local daemon
  • crates/bsk-protocol — shared wire types and JSON schemas
  • apps/extension — browser extension
  • packages/ui and packages/i18n — shared extension UI support
  • packages/dsh-plugin-browserskill — DeepSeek Harness plugin (@wxg-prc-cpg/browser-skill-dsh-plugin)
  • evals/browser — deterministic local pages and agent-neutral browser capability evaluation

License

MIT

S
Description
BrowserSkill: Let AI agents use your real, logged-in browser without interrupting your work. CLI + extension for browser automation. (镜像自 GitHub Tencent/BrowserSkill)
Readme MIT
6.7 MiB
Languages
TypeScript 63.1%
Rust 33%
JavaScript 2.9%
PowerShell 0.4%
CSS 0.4%
Other 0.1%