mirror of
https://github.com/Tencent/BrowserSkill.git
synced 2026-09-10 13:01:23 +08:00
Compare commits
8
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
c74c119a82 | ||
|
|
b801bc1f62 | ||
|
|
bf59753aa3 | ||
|
|
a86cbf724f | ||
|
|
76d447caa0 | ||
|
|
14e5bbe4fc | ||
|
|
bef722107f | ||
|
|
6800f8fdfe |
Generated
+3
-2
@@ -134,9 +134,10 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "bsk"
|
||||
version = "0.1.11"
|
||||
version = "0.1.10"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"base64",
|
||||
"bsk-protocol",
|
||||
"clap",
|
||||
"console",
|
||||
@@ -168,7 +169,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "bsk-protocol"
|
||||
version = "0.1.11"
|
||||
version = "0.1.10"
|
||||
dependencies = [
|
||||
"schemars",
|
||||
"semver",
|
||||
|
||||
+2
-1
@@ -7,7 +7,7 @@ members = [
|
||||
default-members = ["crates/bsk-cli"]
|
||||
|
||||
[workspace.package]
|
||||
version = "0.1.11"
|
||||
version = "0.1.10"
|
||||
edition = "2024"
|
||||
rust-version = "1.85"
|
||||
license = "MIT"
|
||||
@@ -36,6 +36,7 @@ flate2 = "1.1.9"
|
||||
tar = "0.4.46"
|
||||
zip = { version = "7.2.0", default-features = false, features = ["deflate"] }
|
||||
sha2 = "0.11.0"
|
||||
base64 = "0.22"
|
||||
tempfile = "3"
|
||||
nix = { version = "0.29", features = ["signal", "process"] }
|
||||
rand = "0.8"
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# BrowserSkill — Privacy Policy
|
||||
|
||||
**Last updated:** May 25, 2026
|
||||
**Last updated:** August 20, 2026
|
||||
|
||||
This Privacy Policy describes how the **BrowserSkill** browser extension (the "Extension") handles information when you install and use it. BrowserSkill is published as part of the open-source [BrowserSkill](https://github.com/Tencent/BrowserSkill) project. The source code is publicly auditable.
|
||||
|
||||
@@ -14,7 +14,7 @@ BrowserSkill is a local automation bridge that lets AI coding agents (such as Cu
|
||||
|
||||
## 2. Single Purpose
|
||||
|
||||
The Extension's single purpose is to expose browser automation primitives (navigation, DOM observation, screenshots, clicks, form filling, tab management) to a locally running BrowserSkill daemon over a WebSocket connection on `127.0.0.1`, so that an AI agent invoked by the user can interact with web pages on the user's behalf.
|
||||
The Extension's single purpose is to expose browser automation primitives (navigation, DOM observation, screenshots, clicks, form filling, task-scoped file transfer, and tab management) to a locally running BrowserSkill daemon over a WebSocket connection on `127.0.0.1`, so that an AI agent invoked by the user can interact with web pages on the user's behalf.
|
||||
|
||||
## 3. Data the Extension Accesses
|
||||
|
||||
@@ -26,6 +26,7 @@ Depending on the commands the user (via their AI agent) sends to the local daemo
|
||||
| **User input simulated by the agent** | Mouse clicks, keystrokes, and form values that the AI agent dispatches through the Chrome DevTools Protocol (CDP). | Required to perform automation actions the user has asked the agent to do. |
|
||||
| **Tab and window metadata** | Tab IDs, URLs, titles, window IDs of the Agent Window and any tabs the user explicitly authorizes. | Required to target automation commands at the correct tab/window. |
|
||||
| **Local extension storage** | A randomly generated 8-character instance ID and an optional user-supplied label. | Used so the local daemon can recognize this browser instance across reconnects. No personal data is stored. |
|
||||
| **File transfers requested by the agent** | Local files explicitly supplied to `bsk upload`, and the file created by a single `bsk download` action. | Required to attach a task file to a web page or return a browser-generated download to the invoking local agent. |
|
||||
| **OS notifications** | Permission to display a system notification when the agent requests to "borrow" one of the user's existing tabs. | Required to obtain explicit, per-tab user consent before the agent touches any pre-existing tab. |
|
||||
|
||||
## 4. Data the Extension Does **Not** Collect
|
||||
@@ -34,7 +35,7 @@ BrowserSkill does **not**:
|
||||
|
||||
- Send any data to remote servers, the Extension's authors, or any third party.
|
||||
- Call any LLM, AI, or cloud API. The Extension contains no API keys, model identifiers, or remote endpoints.
|
||||
- Read or transmit cookies, browsing history, bookmarks, downloads, saved passwords, or autofill data.
|
||||
- Read or transmit cookies, browsing history, bookmarks, saved passwords, or autofill data. It observes only the download initiated by an active `bsk download` call, not download history generally.
|
||||
- Use webcam, microphone, geolocation, or any device sensor.
|
||||
- Include analytics, telemetry, crash reporting, advertising SDKs, or fingerprinting code.
|
||||
- Track users across websites or across sessions.
|
||||
@@ -50,6 +51,7 @@ The Extension requests the following Chrome permissions. Each is used solely for
|
||||
- **`alarms`** — Periodically wake the service worker to keep the local WebSocket connection alive.
|
||||
- **`idle`** — Detect when the device returns from idle/locked so the Extension can promptly re-establish the local WebSocket connection after the machine wakes. No idle data is stored or transmitted.
|
||||
- **`notifications`** — Show a system notification to obtain user approval before the agent borrows a user-owned tab.
|
||||
- **`downloads`** — Correlate and route the one browser download initiated by an active `bsk download` command. If that claimed transaction fails, BrowserSkill cancels an in-progress file or removes its completed temporary browser file. It is not used to enumerate download history or alter unclaimed downloads.
|
||||
- **`storage`** — Persist a random instance ID and optional label in `chrome.storage.local`.
|
||||
- **Host permission `<all_urls>`** — Inject a small status overlay (showing "Agent Active") on pages controlled by the agent, and enable automation across whatever sites the user directs the agent to. The Extension does **not** read or transmit page content from sites the agent is not actively driving.
|
||||
|
||||
@@ -61,6 +63,7 @@ All Extension activity stays on the user's local device. The only network traffi
|
||||
|
||||
- The instance ID and optional label persist in `chrome.storage.local` until the user uninstalls the Extension or clears extension storage.
|
||||
- Page content, screenshots, DOM snapshots, and other observed data are returned to the local daemon in response to commands and are **not retained by the Extension**. They live only as long as the agent's tool call.
|
||||
- Upload and download bytes are staged by the local daemon in a private, session-scoped directory. Download staging is removed after it is copied to the requested destination. Upload staging is retained until the session ends so a later form submission can still read the attached file. Remaining staging is removed when the session ends or disconnects, or when the daemon next starts after a crash.
|
||||
|
||||
## 8. User Control
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "@browser-skill/extension",
|
||||
"version": "0.1.7",
|
||||
"version": "0.1.6",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
|
||||
@@ -247,41 +247,6 @@ describe("App", () => {
|
||||
expect(copyButton.getAttribute("disabled")).not.toBeNull();
|
||||
expect(screen.getByText("连接后可用")).toBeTruthy();
|
||||
});
|
||||
|
||||
it("treats protocol drift as still connected and prompts an upgrade", async () => {
|
||||
mockUseConnectionState.mockReturnValue({
|
||||
snapshot: {
|
||||
...baseSnapshot,
|
||||
state: "version_skew",
|
||||
instanceId: "03c3e47f",
|
||||
handshake: {
|
||||
server: "bh",
|
||||
version: mockDaemonVersion,
|
||||
protocol_version: "1.0",
|
||||
},
|
||||
},
|
||||
statusState: "version_skew",
|
||||
setLabel,
|
||||
setConnectionEnabled,
|
||||
});
|
||||
|
||||
render(<App />);
|
||||
|
||||
expect(screen.getByText("已连接")).toBeTruthy();
|
||||
expect(screen.getByText("可升级")).toBeTruthy();
|
||||
expect(screen.queryByText("协议不一致")).toBeNull();
|
||||
expect(screen.queryByText("Action needed")).toBeNull();
|
||||
expect(screen.queryByText("兼容")).toBeNull();
|
||||
const warning = screen.getByText(/协议版本不同,请及时升级/);
|
||||
expect(warning.textContent).toContain("CLI 协议");
|
||||
expect(warning.textContent).toContain("扩展协议");
|
||||
|
||||
openRecordView();
|
||||
const copyButton = screen.getByRole("button", { name: "复制录制指令" });
|
||||
expect(copyButton.getAttribute("disabled")).toBeNull();
|
||||
fireEvent.click(copyButton);
|
||||
expect(navigator.clipboard.writeText).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
});
|
||||
|
||||
describe("control hints toggle", () => {
|
||||
|
||||
@@ -79,7 +79,6 @@ export function App() {
|
||||
}, [copiedTick]);
|
||||
|
||||
const isSkewed = statusState === "version_skew";
|
||||
const connectionLive = statusState === "connected" || isSkewed;
|
||||
const daemonVersion = snapshot.handshake?.version ?? "—";
|
||||
const daemonProtocol = snapshot.handshake?.protocol_version ?? "—";
|
||||
const extensionVersion = snapshot.extensionVersion || "—";
|
||||
@@ -91,7 +90,7 @@ export function App() {
|
||||
setCopiedInstanceId(true);
|
||||
};
|
||||
|
||||
const recordReady = connectionLive && Boolean(snapshot.instanceId);
|
||||
const recordReady = statusState === "connected" && Boolean(snapshot.instanceId);
|
||||
const recordPurpose = purposeDraft.trim();
|
||||
const recordStartUrl = startUrlDraft.trim();
|
||||
const recordCommand = snapshot.instanceId
|
||||
@@ -207,12 +206,12 @@ export function App() {
|
||||
</div>
|
||||
{isSkewed && (
|
||||
<p
|
||||
className="mt-2 text-xs leading-snug text-muted-foreground"
|
||||
className="mt-2 text-xs leading-snug text-amber-600 dark:text-amber-400"
|
||||
data-slot="popup-version-skew-warning"
|
||||
>
|
||||
{t("popup.versionSkewWarning", {
|
||||
extensionProtocol: PROTOCOL_VERSION,
|
||||
cliProtocol: daemonProtocol,
|
||||
daemonProtocol,
|
||||
})}
|
||||
</p>
|
||||
)}
|
||||
|
||||
@@ -232,6 +232,107 @@ describe("ToolDispatcher", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("bypasses and restores the control overlay for an upload trigger click", async () => {
|
||||
const sendMessage = vi.fn(async () => undefined);
|
||||
vi.stubGlobal("chrome", {
|
||||
tabs: {
|
||||
get: vi.fn(async () => ({ id: 7, windowId: 4242, active: true })),
|
||||
query: vi.fn(async () => [{ id: 7, windowId: 4242, active: true }]),
|
||||
sendMessage,
|
||||
},
|
||||
});
|
||||
const { transport, sent, deliver } = fakeTransport();
|
||||
const sessions = new SessionManager({
|
||||
agentWindow: {
|
||||
create: vi.fn(async () => 4242),
|
||||
remove: vi.fn(async () => {}),
|
||||
ensureActiveTab: vi.fn(async () => {}),
|
||||
},
|
||||
});
|
||||
const ctx = await sessions.start("aa11");
|
||||
ctx.refStore.set("e1", 123, { tabId: 7 });
|
||||
const send = vi.fn(async <T>(_tabId: number, method: string, params?: object) => {
|
||||
if (method === "Page.getLayoutMetrics") {
|
||||
return { cssLayoutViewport: { clientWidth: 1280, clientHeight: 720 } } as T;
|
||||
}
|
||||
if (method === "DOM.getContentQuads") {
|
||||
return { quads: [[0, 0, 20, 0, 20, 20, 0, 20]] } as T;
|
||||
}
|
||||
if (method === "DOM.resolveNode") {
|
||||
return { object: { objectId: "trigger-object" } } as T;
|
||||
}
|
||||
if (method === "DOM.describeNode") return { node: { backendNodeId: 456 } } as T;
|
||||
if (method === "Runtime.callFunctionOn") {
|
||||
const declaration = (params as { functionDeclaration?: string }).functionDeclaration ?? "";
|
||||
if (declaration.includes("count: state.inputs.length")) {
|
||||
return { result: { value: { count: 1, multiple: false } } } as T;
|
||||
}
|
||||
if (declaration.includes("inputs[0]")) {
|
||||
return { result: { objectId: "input-object" } } as T;
|
||||
}
|
||||
return { result: { value: true } } as T;
|
||||
}
|
||||
if (method === "Runtime.evaluate") {
|
||||
const expression = (params as { expression?: string }).expression ?? "";
|
||||
if (expression.includes("overlayDetails")) {
|
||||
return { result: { value: { hitIndex: 0 } } } as T;
|
||||
}
|
||||
if (expression.includes("overlayHostPresent")) {
|
||||
return {
|
||||
result: {
|
||||
value: { overlayHostPresent: true, overlayHostConnected: true },
|
||||
},
|
||||
} as T;
|
||||
}
|
||||
if (expression.includes("count:")) {
|
||||
return { result: { value: { count: 1, multiple: false } } } as T;
|
||||
}
|
||||
if (expression.includes("?.inputs[0]")) {
|
||||
return { result: { objectId: "input-object" } } as T;
|
||||
}
|
||||
return { result: { value: true } } as T;
|
||||
}
|
||||
return {} as T;
|
||||
});
|
||||
const cdp = {
|
||||
send,
|
||||
detachSession: vi.fn(async () => {}),
|
||||
ensureNetworkCapture: vi.fn(async () => {}),
|
||||
networkEntriesSince: vi.fn(() => ({
|
||||
tab_id: 7,
|
||||
entries: [],
|
||||
next_since: 0,
|
||||
truncated: false,
|
||||
})),
|
||||
setDeviceMetricsOverride: vi.fn(async () => {}),
|
||||
clearDeviceMetricsOverride: vi.fn(async () => {}),
|
||||
setUserAgentOverride: vi.fn(async () => {}),
|
||||
setTouchEmulationEnabled: vi.fn(async () => {}),
|
||||
};
|
||||
const dispatcher = new ToolDispatcher({ transport, sessions, cdp: cdp as TestDispatcherCdp });
|
||||
dispatcher.start();
|
||||
|
||||
deliver(
|
||||
makeRequest("tool.upload", {
|
||||
session_id: "aa11",
|
||||
ref: "@e1",
|
||||
files: [{ transfer_id: "tr_1", name: "test.png", staged_path: "/stage/test.png" }],
|
||||
}),
|
||||
);
|
||||
await flushMicrotasks();
|
||||
await vi.waitFor(() => expect(sent).toHaveLength(1));
|
||||
|
||||
expect(sent[0]).toMatchObject({ result: { tab_id: 7, file_names: ["test.png"] } });
|
||||
expect(sendMessage).toHaveBeenNthCalledWith(1, 7, {
|
||||
type: "bh-automation-bypass",
|
||||
enabled: true,
|
||||
});
|
||||
expect(sendMessage).toHaveBeenNthCalledWith(2, 7, {
|
||||
type: "bh-automation-bypass",
|
||||
enabled: false,
|
||||
});
|
||||
});
|
||||
|
||||
it("detaches CDP state before stopping a session", async () => {
|
||||
const { transport, sent, deliver } = fakeTransport();
|
||||
const sessions = new SessionManager({
|
||||
|
||||
@@ -0,0 +1,708 @@
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import { SessionManager } from "@/session-manager/manager";
|
||||
import { type DownloadsApi, handleDownload } from "../download";
|
||||
import { captureBrowserDownload } from "../download-capture";
|
||||
import { uploadThroughActivatedFileInput } from "../file-input-transaction";
|
||||
import type { ResolvedActionTarget } from "../interaction";
|
||||
import type { CdpRunner } from "../shared";
|
||||
import { handleUpload } from "../upload";
|
||||
|
||||
function sessions() {
|
||||
return new SessionManager({
|
||||
agentWindow: {
|
||||
create: vi.fn(async () => 100),
|
||||
remove: vi.fn(async () => {}),
|
||||
ensureActiveTab: vi.fn(async () => {}),
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
function tabsApi() {
|
||||
return {
|
||||
get: vi.fn(
|
||||
async (tabId: number) => ({ id: tabId, windowId: 100, active: true }) as chrome.tabs.Tab,
|
||||
),
|
||||
query: vi.fn(async () => [{ id: 4, windowId: 100, active: true } as chrome.tabs.Tab]),
|
||||
};
|
||||
}
|
||||
|
||||
function fakeEvent<T extends (...args: never[]) => unknown>() {
|
||||
const listeners = new Set<T>();
|
||||
return {
|
||||
addListener: (listener: T) => listeners.add(listener),
|
||||
removeListener: (listener: T) => listeners.delete(listener),
|
||||
emit: (...args: Parameters<T>) => {
|
||||
for (const listener of listeners) listener(...args);
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
function actionTarget(frameId?: string): ResolvedActionTarget {
|
||||
return {
|
||||
tab: { tabId: 4, windowId: 100, active: true },
|
||||
backendNodeId: 123,
|
||||
cdpTarget: { tabId: 4 },
|
||||
...(frameId ? { frameId } : {}),
|
||||
usedRef: "e3",
|
||||
};
|
||||
}
|
||||
|
||||
function uploadCdp(
|
||||
options: {
|
||||
inputCount?: number;
|
||||
multiple?: boolean;
|
||||
chooser?: { frameId?: string; backendNodeId?: number; mode?: string };
|
||||
pendingResolve?: boolean;
|
||||
} = {},
|
||||
) {
|
||||
const calls: Array<{ method: string; params?: object }> = [];
|
||||
let cdpEvent: Parameters<NonNullable<CdpRunner["onEvent"]>>[0] | undefined;
|
||||
const send = vi.fn(async (_tabId: number, method: string, params?: object) => {
|
||||
calls.push({ method, params });
|
||||
if (method === "Page.setInterceptFileChooserDialog") return {};
|
||||
if (method === "Page.getLayoutMetrics")
|
||||
return { cssLayoutViewport: { clientWidth: 1280, clientHeight: 720 } };
|
||||
if (method === "DOM.getContentQuads") return { quads: [[0, 0, 20, 0, 20, 20, 0, 20]] };
|
||||
if (method === "DOM.resolveNode") {
|
||||
if (options.pendingResolve) return new Promise<never>(() => {});
|
||||
return { object: { objectId: "trigger-object" } };
|
||||
}
|
||||
if (method === "Runtime.callFunctionOn") {
|
||||
const declaration = (params as { functionDeclaration?: string }).functionDeclaration ?? "";
|
||||
if (declaration.includes("Object.defineProperty")) return { result: { value: true } };
|
||||
if (declaration.includes("count: state.inputs.length")) {
|
||||
return {
|
||||
result: {
|
||||
value: {
|
||||
count: options.inputCount ?? 1,
|
||||
multiple: options.multiple ?? true,
|
||||
},
|
||||
},
|
||||
};
|
||||
}
|
||||
if (declaration.includes("inputs[0]")) {
|
||||
return { result: { objectId: "input-object" } };
|
||||
}
|
||||
return { result: { value: true } };
|
||||
}
|
||||
if (method === "DOM.describeNode") return { node: { backendNodeId: 456 } };
|
||||
if (
|
||||
method === "Input.dispatchMouseEvent" &&
|
||||
(params as { type?: string }).type === "mousePressed" &&
|
||||
options.chooser
|
||||
) {
|
||||
cdpEvent?.({ tabId: 4 }, "Page.fileChooserOpened", options.chooser);
|
||||
}
|
||||
return {};
|
||||
});
|
||||
const cdp: CdpRunner = {
|
||||
send: send as unknown as CdpRunner["send"],
|
||||
onEvent: (handler) => {
|
||||
cdpEvent = handler;
|
||||
return { dispose: vi.fn() };
|
||||
},
|
||||
};
|
||||
return {
|
||||
calls,
|
||||
emitChooser: (event: { frameId?: string; backendNodeId?: number; mode?: string }) =>
|
||||
cdpEvent?.({ tabId: 4 }, "Page.fileChooserOpened", event),
|
||||
cdp,
|
||||
};
|
||||
}
|
||||
|
||||
describe("file transfer tools", () => {
|
||||
it("captures the file input activated by the requested click and injects only staged paths", async () => {
|
||||
const manager = sessions();
|
||||
const ctx = await manager.start("s1");
|
||||
ctx.refStore.set("e3", 123, { tabId: 4 });
|
||||
const { cdp, calls } = uploadCdp();
|
||||
|
||||
const result = await handleUpload(
|
||||
manager,
|
||||
{
|
||||
session_id: "s1",
|
||||
ref: "@e3",
|
||||
files: [
|
||||
{ transfer_id: "tr_1", name: "one.png", staged_path: "/private/stage/one" },
|
||||
{ transfer_id: "tr_2", name: "two.png", staged_path: "/private/stage/two" },
|
||||
],
|
||||
},
|
||||
{ cdp, tabsApi: tabsApi() },
|
||||
);
|
||||
|
||||
expect(result).toMatchObject({ tab_id: 4, file_names: ["one.png", "two.png"] });
|
||||
expect(calls[0]).toEqual({
|
||||
method: "Page.setInterceptFileChooserDialog",
|
||||
params: { enabled: true },
|
||||
});
|
||||
expect(calls).toContainEqual({
|
||||
method: "Page.setInterceptFileChooserDialog",
|
||||
params: { enabled: false, cancel: true },
|
||||
});
|
||||
expect(calls).toContainEqual({
|
||||
method: "DOM.setFileInputFiles",
|
||||
params: { files: ["/private/stage/one", "/private/stage/two"], backendNodeId: 456 },
|
||||
});
|
||||
});
|
||||
|
||||
it("fails immediately when the trigger does not activate a file input", async () => {
|
||||
const { cdp } = uploadCdp({ inputCount: 0 });
|
||||
const result = await uploadThroughActivatedFileInput({
|
||||
cdp,
|
||||
actionTarget: actionTarget(),
|
||||
files: ["/private/stage/one"],
|
||||
timeoutMs: 100,
|
||||
trigger: async () => ({ tab_id: 4, x: 10, y: 10 }),
|
||||
});
|
||||
|
||||
expect(result).toMatchObject({
|
||||
code: "unsupported",
|
||||
data: { reason: "file_input_not_activated", phase: "resolve_input" },
|
||||
});
|
||||
});
|
||||
|
||||
it("fails before clicking when chooser interception is unavailable", async () => {
|
||||
const trigger = vi.fn(async () => ({ tab_id: 4, x: 10, y: 10 }));
|
||||
const cdp: CdpRunner = {
|
||||
send: vi.fn(async (_tabId: number, method: string) => {
|
||||
if (method === "Page.setInterceptFileChooserDialog") {
|
||||
throw new Error("method unavailable");
|
||||
}
|
||||
return {};
|
||||
}) as CdpRunner["send"],
|
||||
};
|
||||
|
||||
const result = await uploadThroughActivatedFileInput({
|
||||
cdp,
|
||||
actionTarget: actionTarget(),
|
||||
files: ["/private/stage/one"],
|
||||
timeoutMs: 100,
|
||||
trigger,
|
||||
});
|
||||
|
||||
expect(result).toMatchObject({
|
||||
code: "cdp_failed",
|
||||
data: { effect_state: "none", phase: "arm_interception" },
|
||||
});
|
||||
expect(trigger).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("uses an exact chooser event as an independent input-location signal", async () => {
|
||||
const { cdp, calls, emitChooser } = uploadCdp({ inputCount: 0 });
|
||||
const result = await uploadThroughActivatedFileInput({
|
||||
cdp,
|
||||
actionTarget: actionTarget("f1"),
|
||||
files: ["/private/stage/one"],
|
||||
timeoutMs: 100,
|
||||
trigger: async () => {
|
||||
emitChooser({ frameId: "f1", backendNodeId: 789, mode: "selectSingle" });
|
||||
return { tab_id: 4, x: 10, y: 10 };
|
||||
},
|
||||
});
|
||||
|
||||
expect(result).toMatchObject({ multiple: false });
|
||||
expect(calls).toContainEqual({
|
||||
method: "DOM.setFileInputFiles",
|
||||
params: { files: ["/private/stage/one"], backendNodeId: 789 },
|
||||
});
|
||||
});
|
||||
|
||||
it("reports a File System Access picker without waiting for a timeout", async () => {
|
||||
const { cdp, emitChooser } = uploadCdp({
|
||||
inputCount: 0,
|
||||
});
|
||||
const result = await uploadThroughActivatedFileInput({
|
||||
cdp,
|
||||
actionTarget: actionTarget("f1"),
|
||||
files: ["/private/stage/one"],
|
||||
timeoutMs: 100,
|
||||
trigger: async () => {
|
||||
emitChooser({ frameId: "f1", mode: "selectSingle" });
|
||||
return { tab_id: 4, x: 10, y: 10 };
|
||||
},
|
||||
});
|
||||
|
||||
expect(result).toMatchObject({
|
||||
code: "unsupported",
|
||||
message: "upload trigger invoked a non-input file picker",
|
||||
data: { reason: "file_input_not_activated", phase: "resolve_input" },
|
||||
});
|
||||
});
|
||||
|
||||
it("bounds a stuck file-input probe", async () => {
|
||||
const { cdp } = uploadCdp({ pendingResolve: true });
|
||||
|
||||
const result = await uploadThroughActivatedFileInput({
|
||||
cdp,
|
||||
actionTarget: actionTarget(),
|
||||
files: ["/private/stage/one"],
|
||||
timeoutMs: 5,
|
||||
trigger: vi.fn(),
|
||||
});
|
||||
|
||||
expect(result).toMatchObject({
|
||||
code: "timeout",
|
||||
data: { reason: "file_input_probe_failed", phase: "arm_input_probe" },
|
||||
});
|
||||
});
|
||||
|
||||
it("marks a timed-out file assignment unknown and detaches browser state", async () => {
|
||||
const fixture = uploadCdp();
|
||||
const originalSend = fixture.cdp.send;
|
||||
const detach = vi.fn(async () => {});
|
||||
fixture.cdp.detach = detach;
|
||||
fixture.cdp.send = vi.fn((tabId: number, method: string, params?: object) => {
|
||||
if (method === "DOM.setFileInputFiles") return new Promise<never>(() => {});
|
||||
return originalSend(tabId, method, params);
|
||||
}) as CdpRunner["send"];
|
||||
|
||||
const result = await uploadThroughActivatedFileInput({
|
||||
cdp: fixture.cdp,
|
||||
actionTarget: actionTarget(),
|
||||
files: ["/private/stage/one"],
|
||||
timeoutMs: 10,
|
||||
trigger: async () => ({ tab_id: 4, x: 10, y: 10 }),
|
||||
});
|
||||
|
||||
expect(result).toMatchObject({
|
||||
code: "timeout",
|
||||
data: { effect_state: "unknown", phase: "set_files" },
|
||||
});
|
||||
expect(detach).toHaveBeenCalledWith(4);
|
||||
});
|
||||
|
||||
it("routes one exact-target download through a browser-relative capability", async () => {
|
||||
const manager = sessions();
|
||||
const ctx = await manager.start("s1");
|
||||
ctx.refStore.set("e3", 123, { tabId: 4 });
|
||||
const onCreated = fakeEvent<(item: chrome.downloads.DownloadItem) => void>();
|
||||
const onChanged = fakeEvent<(delta: chrome.downloads.DownloadDelta) => void>();
|
||||
const onDeterminingFilename =
|
||||
fakeEvent<
|
||||
(
|
||||
item: chrome.downloads.DownloadItem,
|
||||
suggest: (suggestion?: chrome.downloads.DownloadFilenameSuggestion) => void,
|
||||
) => void | true
|
||||
>();
|
||||
const initial = {
|
||||
id: 9,
|
||||
url: "https://example.test/result.zip",
|
||||
finalUrl: "https://example.test/result.zip",
|
||||
filename: "result.zip",
|
||||
state: "in_progress",
|
||||
fileSize: -1,
|
||||
totalBytes: 12,
|
||||
mime: "application/zip",
|
||||
danger: "safe",
|
||||
} as chrome.downloads.DownloadItem;
|
||||
const completed = {
|
||||
...initial,
|
||||
filename: "/profile/Downloads/BrowserSkill/tr_1/result.zip",
|
||||
state: "complete",
|
||||
fileSize: 12,
|
||||
} as chrome.downloads.DownloadItem;
|
||||
const downloads: DownloadsApi = {
|
||||
onCreated,
|
||||
onChanged,
|
||||
onDeterminingFilename,
|
||||
search: vi.fn(async () => [completed]),
|
||||
cancel: vi.fn(async () => {}),
|
||||
removeFile: vi.fn(async () => {}),
|
||||
};
|
||||
let cdpEvent: Parameters<NonNullable<CdpRunner["onEvent"]>>[0] | undefined;
|
||||
let suggested: chrome.downloads.DownloadFilenameSuggestion | undefined;
|
||||
const send = vi.fn(async (_tabId: number, method: string, params?: object) => {
|
||||
if (method === "Page.getLayoutMetrics")
|
||||
return { cssLayoutViewport: { clientWidth: 1280, clientHeight: 720 } };
|
||||
if (method === "DOM.getContentQuads") return { quads: [[0, 0, 20, 0, 20, 20, 0, 20]] };
|
||||
if (
|
||||
method === "Input.dispatchMouseEvent" &&
|
||||
(params as { type?: string }).type === "mousePressed"
|
||||
) {
|
||||
cdpEvent?.({ tabId: 4 }, "Page.downloadWillBegin", {
|
||||
url: initial.url,
|
||||
suggestedFilename: "result.zip",
|
||||
});
|
||||
await new Promise<void>((resolve) => {
|
||||
onDeterminingFilename.emit(initial, (value) => {
|
||||
suggested = value;
|
||||
resolve();
|
||||
});
|
||||
});
|
||||
onCreated.emit(completed);
|
||||
}
|
||||
return {};
|
||||
});
|
||||
const cdp: CdpRunner = {
|
||||
send: send as unknown as CdpRunner["send"],
|
||||
onEvent: (handler) => {
|
||||
cdpEvent = handler;
|
||||
return { dispose: vi.fn() };
|
||||
},
|
||||
};
|
||||
|
||||
const result = await handleDownload(
|
||||
manager,
|
||||
{ session_id: "s1", ref: "@e3", browser_relative_dir: "BrowserSkill/tr_1" },
|
||||
{ cdp, tabsApi: tabsApi(), downloads },
|
||||
);
|
||||
|
||||
expect(suggested).toEqual({
|
||||
filename: "BrowserSkill/tr_1/result.zip",
|
||||
conflictAction: "overwrite",
|
||||
});
|
||||
expect(result).toMatchObject({
|
||||
tab_id: 4,
|
||||
suggested_filename: "result.zip",
|
||||
byte_size: 12,
|
||||
browser_path: completed.filename,
|
||||
});
|
||||
});
|
||||
|
||||
it("does not claim a download without an intent from the exact target", async () => {
|
||||
const onCreated = fakeEvent<(item: chrome.downloads.DownloadItem) => void>();
|
||||
const onChanged = fakeEvent<(delta: chrome.downloads.DownloadDelta) => void>();
|
||||
const onDeterminingFilename =
|
||||
fakeEvent<
|
||||
(
|
||||
item: chrome.downloads.DownloadItem,
|
||||
suggest: (suggestion?: chrome.downloads.DownloadFilenameSuggestion) => void,
|
||||
) => void | true
|
||||
>();
|
||||
const downloads: DownloadsApi = {
|
||||
onCreated,
|
||||
onChanged,
|
||||
onDeterminingFilename,
|
||||
search: vi.fn(async () => []),
|
||||
cancel: vi.fn(async () => {}),
|
||||
removeFile: vi.fn(async () => {}),
|
||||
};
|
||||
let cdpEvent: Parameters<NonNullable<CdpRunner["onEvent"]>>[0] | undefined;
|
||||
const cdp: CdpRunner = {
|
||||
send: vi.fn(async () => ({})) as unknown as CdpRunner["send"],
|
||||
onEvent: (handler) => {
|
||||
cdpEvent = handler;
|
||||
return { dispose: vi.fn() };
|
||||
},
|
||||
};
|
||||
const unrelated = {
|
||||
id: 17,
|
||||
url: "https://example.test/unrelated.zip",
|
||||
finalUrl: "https://example.test/unrelated.zip",
|
||||
filename: "unrelated.zip",
|
||||
state: "in_progress",
|
||||
} as chrome.downloads.DownloadItem;
|
||||
let defaultSuggestionCalled = false;
|
||||
|
||||
const result = await captureBrowserDownload({
|
||||
cdp,
|
||||
target: { tabId: 4, sessionId: "expected-child" },
|
||||
downloads,
|
||||
browserRelativeDir: "BrowserSkill/tr_1",
|
||||
timeoutMs: 5,
|
||||
trigger: async () => {
|
||||
cdpEvent?.({ tabId: 4, sessionId: "other-child" }, "Page.downloadWillBegin", {
|
||||
url: unrelated.url,
|
||||
suggestedFilename: unrelated.filename,
|
||||
});
|
||||
onDeterminingFilename.emit(unrelated, (suggestion) => {
|
||||
defaultSuggestionCalled = suggestion === undefined;
|
||||
});
|
||||
return { tab_id: 4, x: 10, y: 10 };
|
||||
},
|
||||
});
|
||||
|
||||
expect(defaultSuggestionCalled).toBe(true);
|
||||
expect(downloads.cancel).not.toHaveBeenCalled();
|
||||
expect(result).toMatchObject({
|
||||
code: "cdp_failed",
|
||||
data: { reason: "download_capture_failed" },
|
||||
});
|
||||
});
|
||||
|
||||
it("correlates a filename candidate that arrives before the CDP intent", async () => {
|
||||
const onCreated = fakeEvent<(item: chrome.downloads.DownloadItem) => void>();
|
||||
const onChanged = fakeEvent<(delta: chrome.downloads.DownloadDelta) => void>();
|
||||
const onDeterminingFilename =
|
||||
fakeEvent<
|
||||
(
|
||||
item: chrome.downloads.DownloadItem,
|
||||
suggest: (suggestion?: chrome.downloads.DownloadFilenameSuggestion) => void,
|
||||
) => void | true
|
||||
>();
|
||||
const initial = {
|
||||
id: 21,
|
||||
url: "https://example.test/candidate-first.bin",
|
||||
finalUrl: "https://example.test/candidate-first.bin",
|
||||
filename: "candidate-first.bin",
|
||||
state: "in_progress",
|
||||
fileSize: -1,
|
||||
totalBytes: 4,
|
||||
bytesReceived: 0,
|
||||
} as chrome.downloads.DownloadItem;
|
||||
const complete = {
|
||||
...initial,
|
||||
state: "complete",
|
||||
fileSize: 4,
|
||||
} as chrome.downloads.DownloadItem;
|
||||
const downloads: DownloadsApi = {
|
||||
onCreated,
|
||||
onChanged,
|
||||
onDeterminingFilename,
|
||||
search: vi.fn(async () => [complete]),
|
||||
cancel: vi.fn(async () => {}),
|
||||
removeFile: vi.fn(async () => {}),
|
||||
};
|
||||
let cdpEvent: Parameters<NonNullable<CdpRunner["onEvent"]>>[0] | undefined;
|
||||
const cdp: CdpRunner = {
|
||||
send: vi.fn(async () => ({})) as CdpRunner["send"],
|
||||
onEvent: (handler) => {
|
||||
cdpEvent = handler;
|
||||
return { dispose: vi.fn() };
|
||||
},
|
||||
};
|
||||
let suggestion: chrome.downloads.DownloadFilenameSuggestion | undefined;
|
||||
|
||||
const result = await captureBrowserDownload({
|
||||
cdp,
|
||||
target: { tabId: 4 },
|
||||
downloads,
|
||||
browserRelativeDir: "BrowserSkill/tr_21",
|
||||
timeoutMs: 1_000,
|
||||
trigger: async () => {
|
||||
const suggested = new Promise<void>((resolve) => {
|
||||
onDeterminingFilename.emit(initial, (value) => {
|
||||
suggestion = value;
|
||||
resolve();
|
||||
});
|
||||
});
|
||||
cdpEvent?.({ tabId: 4 }, "Page.downloadWillBegin", {
|
||||
url: initial.url,
|
||||
suggestedFilename: initial.filename,
|
||||
});
|
||||
await suggested;
|
||||
onCreated.emit(complete);
|
||||
return { tab_id: 4, x: 10, y: 10 };
|
||||
},
|
||||
});
|
||||
|
||||
expect(suggestion).toEqual({
|
||||
filename: "BrowserSkill/tr_21/candidate-first.bin",
|
||||
conflictAction: "overwrite",
|
||||
});
|
||||
expect(result).toMatchObject({ item: { id: 21, state: "complete" } });
|
||||
});
|
||||
|
||||
it("removes a completed download when the final size exceeds the limit", async () => {
|
||||
const onCreated = fakeEvent<(item: chrome.downloads.DownloadItem) => void>();
|
||||
const onChanged = fakeEvent<(delta: chrome.downloads.DownloadDelta) => void>();
|
||||
const onDeterminingFilename =
|
||||
fakeEvent<
|
||||
(
|
||||
item: chrome.downloads.DownloadItem,
|
||||
suggest: (suggestion?: chrome.downloads.DownloadFilenameSuggestion) => void,
|
||||
) => void | true
|
||||
>();
|
||||
const initial = {
|
||||
id: 22,
|
||||
url: "https://example.test/oversized.bin",
|
||||
finalUrl: "https://example.test/oversized.bin",
|
||||
filename: "oversized.bin",
|
||||
state: "in_progress",
|
||||
fileSize: -1,
|
||||
totalBytes: -1,
|
||||
bytesReceived: 0,
|
||||
} as chrome.downloads.DownloadItem;
|
||||
const complete = {
|
||||
...initial,
|
||||
filename: "/profile/Downloads/BrowserSkill/tr_22/oversized.bin",
|
||||
state: "complete",
|
||||
fileSize: 8,
|
||||
totalBytes: 8,
|
||||
bytesReceived: 8,
|
||||
} as chrome.downloads.DownloadItem;
|
||||
const downloads: DownloadsApi = {
|
||||
onCreated,
|
||||
onChanged,
|
||||
onDeterminingFilename,
|
||||
search: vi.fn(async () => [complete]),
|
||||
cancel: vi.fn(async () => {}),
|
||||
removeFile: vi.fn(async () => {}),
|
||||
};
|
||||
let cdpEvent: Parameters<NonNullable<CdpRunner["onEvent"]>>[0] | undefined;
|
||||
const cdp: CdpRunner = {
|
||||
send: vi.fn(async () => ({})) as CdpRunner["send"],
|
||||
onEvent: (handler) => {
|
||||
cdpEvent = handler;
|
||||
return { dispose: vi.fn() };
|
||||
},
|
||||
};
|
||||
|
||||
const result = await captureBrowserDownload({
|
||||
cdp,
|
||||
target: { tabId: 4 },
|
||||
downloads,
|
||||
browserRelativeDir: "BrowserSkill/tr_22",
|
||||
maxByteSize: 4,
|
||||
timeoutMs: 1_000,
|
||||
trigger: async () => {
|
||||
cdpEvent?.({ tabId: 4 }, "Page.downloadWillBegin", {
|
||||
url: initial.url,
|
||||
suggestedFilename: initial.filename,
|
||||
});
|
||||
await new Promise<void>((resolve) => {
|
||||
onDeterminingFilename.emit(initial, () => resolve());
|
||||
});
|
||||
onCreated.emit(initial);
|
||||
onChanged.emit({ id: initial.id, state: { current: "complete" } });
|
||||
return { tab_id: 4, x: 10, y: 10 };
|
||||
},
|
||||
});
|
||||
|
||||
expect(result).toMatchObject({
|
||||
code: "cdp_failed",
|
||||
data: { reason: "download_capture_failed", effect_state: "committed" },
|
||||
});
|
||||
expect(downloads.removeFile).toHaveBeenCalledWith(initial.id);
|
||||
expect(downloads.cancel).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("reconciles a download that completes while cancellation is being requested", async () => {
|
||||
const onCreated = fakeEvent<(item: chrome.downloads.DownloadItem) => void>();
|
||||
const onChanged = fakeEvent<(delta: chrome.downloads.DownloadDelta) => void>();
|
||||
const onDeterminingFilename =
|
||||
fakeEvent<
|
||||
(
|
||||
item: chrome.downloads.DownloadItem,
|
||||
suggest: (suggestion?: chrome.downloads.DownloadFilenameSuggestion) => void,
|
||||
) => void | true
|
||||
>();
|
||||
const initial = {
|
||||
id: 23,
|
||||
url: "https://example.test/racing.bin",
|
||||
finalUrl: "https://example.test/racing.bin",
|
||||
filename: "racing.bin",
|
||||
state: "in_progress",
|
||||
fileSize: -1,
|
||||
totalBytes: -1,
|
||||
bytesReceived: 0,
|
||||
} as chrome.downloads.DownloadItem;
|
||||
const complete = {
|
||||
...initial,
|
||||
filename: "/profile/Downloads/BrowserSkill/tr_23/racing.bin",
|
||||
state: "complete",
|
||||
fileSize: 4,
|
||||
totalBytes: 4,
|
||||
bytesReceived: 4,
|
||||
} as chrome.downloads.DownloadItem;
|
||||
const downloads: DownloadsApi = {
|
||||
onCreated,
|
||||
onChanged,
|
||||
onDeterminingFilename,
|
||||
search: vi.fn().mockResolvedValueOnce([initial]).mockResolvedValueOnce([complete]),
|
||||
cancel: vi.fn(async () => {
|
||||
throw new Error("download already complete");
|
||||
}),
|
||||
removeFile: vi.fn(async () => {}),
|
||||
};
|
||||
let cdpEvent: Parameters<NonNullable<CdpRunner["onEvent"]>>[0] | undefined;
|
||||
const cdp: CdpRunner = {
|
||||
send: vi.fn(async () => ({})) as CdpRunner["send"],
|
||||
onEvent: (handler) => {
|
||||
cdpEvent = handler;
|
||||
return { dispose: vi.fn() };
|
||||
},
|
||||
};
|
||||
|
||||
const result = await captureBrowserDownload({
|
||||
cdp,
|
||||
target: { tabId: 4 },
|
||||
downloads,
|
||||
browserRelativeDir: "BrowserSkill/tr_23",
|
||||
timeoutMs: 80,
|
||||
trigger: async () => {
|
||||
cdpEvent?.({ tabId: 4 }, "Page.downloadWillBegin", {
|
||||
url: initial.url,
|
||||
suggestedFilename: initial.filename,
|
||||
});
|
||||
await new Promise<void>((resolve) => {
|
||||
onDeterminingFilename.emit(initial, () => resolve());
|
||||
});
|
||||
onCreated.emit(initial);
|
||||
return { tab_id: 4, x: 10, y: 10 };
|
||||
},
|
||||
});
|
||||
|
||||
expect(result).toMatchObject({
|
||||
code: "cdp_failed",
|
||||
data: { reason: "download_capture_failed", effect_state: "committed" },
|
||||
});
|
||||
expect(downloads.cancel).toHaveBeenCalledWith(initial.id);
|
||||
expect(downloads.removeFile).toHaveBeenCalledWith(initial.id);
|
||||
expect(result).not.toMatchObject({ data: { cleanup_state: "failed" } });
|
||||
});
|
||||
|
||||
it("rejects ambiguous attribution without cancelling either unclaimed download", async () => {
|
||||
const onCreated = fakeEvent<(item: chrome.downloads.DownloadItem) => void>();
|
||||
const onChanged = fakeEvent<(delta: chrome.downloads.DownloadDelta) => void>();
|
||||
const onDeterminingFilename =
|
||||
fakeEvent<
|
||||
(
|
||||
item: chrome.downloads.DownloadItem,
|
||||
suggest: (suggestion?: chrome.downloads.DownloadFilenameSuggestion) => void,
|
||||
) => void | true
|
||||
>();
|
||||
const downloads: DownloadsApi = {
|
||||
onCreated,
|
||||
onChanged,
|
||||
onDeterminingFilename,
|
||||
search: vi.fn(async () => []),
|
||||
cancel: vi.fn(async () => {}),
|
||||
removeFile: vi.fn(async () => {}),
|
||||
};
|
||||
let cdpEvent: Parameters<NonNullable<CdpRunner["onEvent"]>>[0] | undefined;
|
||||
const cdp: CdpRunner = {
|
||||
send: vi.fn(async () => ({})) as CdpRunner["send"],
|
||||
onEvent: (handler) => {
|
||||
cdpEvent = handler;
|
||||
return { dispose: vi.fn() };
|
||||
},
|
||||
};
|
||||
const defaults: number[] = [];
|
||||
const candidate = (id: number) =>
|
||||
({
|
||||
id,
|
||||
url: "https://example.test/same.bin",
|
||||
finalUrl: "https://example.test/same.bin",
|
||||
filename: "same.bin",
|
||||
state: "in_progress",
|
||||
}) as chrome.downloads.DownloadItem;
|
||||
|
||||
const result = await captureBrowserDownload({
|
||||
cdp,
|
||||
target: { tabId: 4 },
|
||||
downloads,
|
||||
browserRelativeDir: "BrowserSkill/tr_ambiguous",
|
||||
timeoutMs: 100,
|
||||
trigger: async () => {
|
||||
cdpEvent?.({ tabId: 4 }, "Page.downloadWillBegin", {
|
||||
url: "https://example.test/same.bin",
|
||||
suggestedFilename: "same.bin",
|
||||
});
|
||||
for (const id of [31, 32]) {
|
||||
onDeterminingFilename.emit(candidate(id), (value) => {
|
||||
if (value === undefined) defaults.push(id);
|
||||
});
|
||||
}
|
||||
return { tab_id: 4, x: 10, y: 10 };
|
||||
},
|
||||
});
|
||||
|
||||
expect(defaults.sort()).toEqual([31, 32]);
|
||||
expect(downloads.cancel).not.toHaveBeenCalled();
|
||||
expect(result).toMatchObject({
|
||||
code: "cdp_failed",
|
||||
data: { effect_state: "unknown", phase: "attribution" },
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -4,6 +4,7 @@ import type { Transport } from "@/transport/transport";
|
||||
import type {
|
||||
ClickParams,
|
||||
ConsoleParams,
|
||||
DownloadParams,
|
||||
EmulateParams,
|
||||
EvaluateParams,
|
||||
FillParams,
|
||||
@@ -28,10 +29,12 @@ import type {
|
||||
ScreenshotParams,
|
||||
SelectParams,
|
||||
SnapshotParams,
|
||||
UploadParams,
|
||||
WaitForNavigationParams,
|
||||
} from "@/transport/types";
|
||||
import { isRequestFrame } from "@/transport/types";
|
||||
import { handleConsole } from "./console";
|
||||
import { handleDownload } from "./download";
|
||||
import { type EmulateCdpRunner, handleEmulate } from "./emulate";
|
||||
import { handleEvaluate } from "./evaluate";
|
||||
import { handleRequestHelp } from "./human-loop";
|
||||
@@ -79,6 +82,7 @@ import {
|
||||
type TabReturnParams,
|
||||
type TabSelectParams,
|
||||
} from "./tabs";
|
||||
import { handleUpload } from "./upload";
|
||||
import { handleWaitForNavigation } from "./waits";
|
||||
import { handleWindowResize, type WindowResizeParams } from "./window";
|
||||
|
||||
@@ -513,6 +517,40 @@ export class ToolDispatcher {
|
||||
),
|
||||
signal,
|
||||
);
|
||||
case "tool.upload":
|
||||
return this.withHoverReleaseForRequest(
|
||||
req.params as UploadParams,
|
||||
() =>
|
||||
this.cdp
|
||||
? handleUpload(this.sessions, req.params as UploadParams, {
|
||||
cdp: this.cdp,
|
||||
tabsApi: chromeTabsApi,
|
||||
signal,
|
||||
bypassOverlay,
|
||||
})
|
||||
: Promise.resolve({
|
||||
code: "unsupported",
|
||||
message: "upload requires CDP",
|
||||
} satisfies RpcError),
|
||||
signal,
|
||||
);
|
||||
case "tool.download":
|
||||
return this.withHoverReleaseForRequest(
|
||||
req.params as DownloadParams,
|
||||
() =>
|
||||
this.cdp
|
||||
? handleDownload(this.sessions, req.params as DownloadParams, {
|
||||
cdp: this.cdp,
|
||||
tabsApi: chromeTabsApi,
|
||||
signal,
|
||||
bypassOverlay,
|
||||
})
|
||||
: Promise.resolve({
|
||||
code: "unsupported",
|
||||
message: "download requires CDP",
|
||||
} satisfies RpcError),
|
||||
signal,
|
||||
);
|
||||
case "tool.evaluate":
|
||||
return handleEvaluate(
|
||||
this.sessions,
|
||||
@@ -719,6 +757,8 @@ function sessionIdForBrowserControlMethod(req: RequestFrame): string | null {
|
||||
case "tool.fill":
|
||||
case "tool.press":
|
||||
case "tool.select":
|
||||
case "tool.upload":
|
||||
case "tool.download":
|
||||
case "tool.evaluate":
|
||||
case "tool.observe":
|
||||
case "tool.request_help":
|
||||
|
||||
@@ -0,0 +1,364 @@
|
||||
// Order-independent coordinator for one browser download. CDP supplies the
|
||||
// exact target/frame intent while chrome.downloads supplies the download id
|
||||
// and filename routing hook; neither event is assumed to arrive first.
|
||||
|
||||
import type { CdpTarget } from "@/browser-driver/frame-graph";
|
||||
import type { ClickResult, RpcError, TransferEffectState } from "@/transport/types";
|
||||
import { transferError } from "./errors";
|
||||
import { type CdpRunner, isRpcError } from "./shared";
|
||||
|
||||
const CORRELATION_GRACE_MS = 750;
|
||||
const UNIQUE_SETTLE_MS = 50;
|
||||
const SIZE_POLL_MS = 250;
|
||||
|
||||
type DeterminingFilenameListener = (
|
||||
item: chrome.downloads.DownloadItem,
|
||||
suggest: (suggestion?: chrome.downloads.DownloadFilenameSuggestion) => void,
|
||||
) => void | true;
|
||||
|
||||
interface ListenerEvent<T> {
|
||||
addListener(listener: T): void;
|
||||
removeListener(listener: T): void;
|
||||
}
|
||||
|
||||
export interface DownloadsApi {
|
||||
onCreated: ListenerEvent<(item: chrome.downloads.DownloadItem) => void>;
|
||||
onChanged: ListenerEvent<(delta: chrome.downloads.DownloadDelta) => void>;
|
||||
onDeterminingFilename: ListenerEvent<DeterminingFilenameListener>;
|
||||
search(query: chrome.downloads.DownloadQuery): Promise<chrome.downloads.DownloadItem[]>;
|
||||
cancel(downloadId: number): Promise<void>;
|
||||
removeFile(downloadId: number): Promise<void>;
|
||||
}
|
||||
|
||||
export const chromeDownloadsApi: DownloadsApi = {
|
||||
get onCreated() {
|
||||
return chrome.downloads.onCreated;
|
||||
},
|
||||
get onChanged() {
|
||||
return chrome.downloads.onChanged;
|
||||
},
|
||||
get onDeterminingFilename() {
|
||||
return chrome.downloads.onDeterminingFilename;
|
||||
},
|
||||
search: (query) => chrome.downloads.search(query),
|
||||
cancel: (id) => chrome.downloads.cancel(id),
|
||||
removeFile: (id) => chrome.downloads.removeFile(id),
|
||||
};
|
||||
|
||||
export interface DownloadCaptureOptions {
|
||||
cdp: CdpRunner;
|
||||
target: CdpTarget;
|
||||
expectedFrameId?: string;
|
||||
downloads: DownloadsApi;
|
||||
browserRelativeDir: string;
|
||||
maxByteSize?: number;
|
||||
timeoutMs: number;
|
||||
signal?: AbortSignal;
|
||||
trigger(): Promise<ClickResult | RpcError>;
|
||||
}
|
||||
|
||||
export interface DownloadCaptureResult {
|
||||
click: ClickResult;
|
||||
item: chrome.downloads.DownloadItem;
|
||||
}
|
||||
|
||||
interface DownloadIntent {
|
||||
url: string;
|
||||
suggestedFilename: string;
|
||||
frameId?: string;
|
||||
}
|
||||
|
||||
interface DownloadCandidate {
|
||||
item: chrome.downloads.DownloadItem;
|
||||
suggest: (suggestion?: chrome.downloads.DownloadFilenameSuggestion) => void;
|
||||
suggested: boolean;
|
||||
graceTimer: ReturnType<typeof setTimeout>;
|
||||
}
|
||||
|
||||
function safeBasename(filename: string): string {
|
||||
const basename = filename.split(/[\\/]/).pop()?.trim();
|
||||
return basename && basename !== "." && basename !== ".." ? basename : "download";
|
||||
}
|
||||
|
||||
function sameTarget(source: { tabId?: number; sessionId?: string }, target: CdpTarget): boolean {
|
||||
return source.tabId === target.tabId && source.sessionId === target.sessionId;
|
||||
}
|
||||
|
||||
function matchesIntent(item: chrome.downloads.DownloadItem, intent: DownloadIntent): boolean {
|
||||
const urlMatches = item.url === intent.url || item.finalUrl === intent.url;
|
||||
return urlMatches && safeBasename(item.filename) === safeBasename(intent.suggestedFilename);
|
||||
}
|
||||
|
||||
function knownSize(item: chrome.downloads.DownloadItem): number | undefined {
|
||||
if (item.fileSize >= 0) return item.fileSize;
|
||||
if (item.totalBytes >= 0) return item.totalBytes;
|
||||
return undefined;
|
||||
}
|
||||
|
||||
function captureError(
|
||||
message: string,
|
||||
effectState: TransferEffectState,
|
||||
phase: string,
|
||||
cleanupFailed = false,
|
||||
): RpcError {
|
||||
return transferError("cdp_failed", "download_capture_failed", message, {
|
||||
effectState,
|
||||
phase,
|
||||
...(cleanupFailed ? { cleanupState: "failed" } : {}),
|
||||
});
|
||||
}
|
||||
|
||||
async function cleanupClaimedDownload(downloads: DownloadsApi, downloadId: number): Promise<void> {
|
||||
const lookup = async () => (await downloads.search({ id: downloadId }))[0];
|
||||
const item = await lookup();
|
||||
if (!item || item.state === "interrupted") return;
|
||||
if (item.state === "complete") {
|
||||
await downloads.removeFile(downloadId);
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
await downloads.cancel(downloadId);
|
||||
} catch (cancelError) {
|
||||
// Completion can win the race after the lookup but before cancellation.
|
||||
// Reconcile against Chrome's authoritative state before declaring cleanup
|
||||
// failed so every terminal state has one explicit cleanup path.
|
||||
const reconciled = await lookup();
|
||||
if (!reconciled || reconciled.state === "interrupted") return;
|
||||
if (reconciled.state === "complete") {
|
||||
await downloads.removeFile(downloadId);
|
||||
return;
|
||||
}
|
||||
throw cancelError;
|
||||
}
|
||||
}
|
||||
|
||||
export async function captureBrowserDownload(
|
||||
options: DownloadCaptureOptions,
|
||||
): Promise<DownloadCaptureResult | RpcError> {
|
||||
let click: ClickResult | undefined;
|
||||
let intent: DownloadIntent | undefined;
|
||||
let capturedId: number | undefined;
|
||||
let settled = false;
|
||||
let succeeded = false;
|
||||
let failureResult: RpcError | undefined;
|
||||
let uniquenessTimer: ReturnType<typeof setTimeout> | undefined;
|
||||
let operationTimer: ReturnType<typeof setTimeout> | undefined;
|
||||
let sizePoll: ReturnType<typeof setInterval> | undefined;
|
||||
const candidates = new Map<number, DownloadCandidate>();
|
||||
const createdItems = new Map<number, chrome.downloads.DownloadItem>();
|
||||
|
||||
let resolveCompletion!: (item: chrome.downloads.DownloadItem) => void;
|
||||
let rejectCompletion!: (error: Error) => void;
|
||||
const completion = new Promise<chrome.downloads.DownloadItem>((resolve, reject) => {
|
||||
resolveCompletion = resolve;
|
||||
rejectCompletion = reject;
|
||||
});
|
||||
const fail = (error: Error) => {
|
||||
if (settled) return;
|
||||
settled = true;
|
||||
rejectCompletion(error);
|
||||
};
|
||||
const complete = (item: chrome.downloads.DownloadItem) => {
|
||||
if (settled) return;
|
||||
const size = knownSize(item);
|
||||
if (size !== undefined && options.maxByteSize !== undefined && size > options.maxByteSize) {
|
||||
fail(new Error(`download exceeds transfer limit ${options.maxByteSize}`));
|
||||
return;
|
||||
}
|
||||
settled = true;
|
||||
resolveCompletion(item);
|
||||
};
|
||||
const suggestDefault = (candidate: DownloadCandidate) => {
|
||||
if (candidate.suggested) return;
|
||||
candidate.suggested = true;
|
||||
candidate.suggest();
|
||||
};
|
||||
const matchingCandidates = (): DownloadCandidate[] => {
|
||||
const currentIntent = intent;
|
||||
return currentIntent
|
||||
? [...candidates.values()].filter(
|
||||
(candidate) => !candidate.suggested && matchesIntent(candidate.item, currentIntent),
|
||||
)
|
||||
: [];
|
||||
};
|
||||
|
||||
const claimUnique = () => {
|
||||
uniquenessTimer = undefined;
|
||||
if (settled || capturedId !== undefined || !intent) return;
|
||||
const matches = matchingCandidates();
|
||||
if (matches.length !== 1) {
|
||||
if (matches.length > 1) {
|
||||
for (const candidate of matches) suggestDefault(candidate);
|
||||
fail(new Error("download attribution is ambiguous"));
|
||||
}
|
||||
return;
|
||||
}
|
||||
const candidate = matches[0];
|
||||
candidate.suggested = true;
|
||||
clearTimeout(candidate.graceTimer);
|
||||
capturedId = candidate.item.id;
|
||||
candidate.suggest({
|
||||
filename: `${options.browserRelativeDir}/${safeBasename(intent.suggestedFilename)}`,
|
||||
conflictAction: "overwrite",
|
||||
});
|
||||
const size = knownSize(candidate.item);
|
||||
if (size !== undefined && options.maxByteSize !== undefined && size > options.maxByteSize) {
|
||||
fail(new Error(`download exceeds transfer limit ${options.maxByteSize}`));
|
||||
return;
|
||||
}
|
||||
const created = createdItems.get(candidate.item.id);
|
||||
if (created?.state === "interrupted") {
|
||||
fail(new Error(created.error ?? "download interrupted"));
|
||||
} else if (created?.state === "complete") {
|
||||
complete(created);
|
||||
}
|
||||
};
|
||||
const reconcile = () => {
|
||||
if (settled || capturedId !== undefined || !intent) return;
|
||||
const matches = matchingCandidates();
|
||||
if (matches.length > 1) {
|
||||
for (const candidate of matches) suggestDefault(candidate);
|
||||
fail(new Error("download attribution is ambiguous"));
|
||||
return;
|
||||
}
|
||||
if (matches.length === 1 && !uniquenessTimer) {
|
||||
uniquenessTimer = setTimeout(claimUnique, UNIQUE_SETTLE_MS);
|
||||
}
|
||||
};
|
||||
|
||||
const determiningListener: DeterminingFilenameListener = (item, suggest) => {
|
||||
const candidate: DownloadCandidate = {
|
||||
item,
|
||||
suggest,
|
||||
suggested: false,
|
||||
graceTimer: setTimeout(() => {
|
||||
suggestDefault(candidate);
|
||||
candidates.delete(item.id);
|
||||
if (intent && matchesIntent(item, intent) && capturedId === undefined) {
|
||||
fail(new Error("download correlation grace elapsed before unique attribution"));
|
||||
}
|
||||
}, CORRELATION_GRACE_MS),
|
||||
};
|
||||
candidates.set(item.id, candidate);
|
||||
reconcile();
|
||||
return true;
|
||||
};
|
||||
const createdListener = (item: chrome.downloads.DownloadItem) => {
|
||||
createdItems.set(item.id, item);
|
||||
if (capturedId !== item.id) return;
|
||||
if (item.state === "interrupted") {
|
||||
fail(new Error(item.error ?? "download interrupted"));
|
||||
} else if (item.state === "complete") {
|
||||
complete(item);
|
||||
}
|
||||
};
|
||||
const changedListener = async (delta: chrome.downloads.DownloadDelta) => {
|
||||
if (capturedId === undefined || delta.id !== capturedId || settled) return;
|
||||
if (delta.state?.current === "interrupted" || delta.error?.current) {
|
||||
fail(new Error(delta.error?.current ?? "download interrupted"));
|
||||
return;
|
||||
}
|
||||
if (delta.state?.current === "complete") {
|
||||
try {
|
||||
const [item] = await options.downloads.search({ id: delta.id });
|
||||
if (item) complete(item);
|
||||
else fail(new Error("completed download disappeared"));
|
||||
} catch (err) {
|
||||
fail(err instanceof Error ? err : new Error(String(err)));
|
||||
}
|
||||
}
|
||||
};
|
||||
const onAbort = () => fail(new DOMException("aborted", "AbortError"));
|
||||
const cdpSubscription = options.cdp.onEvent?.((source, method, raw) => {
|
||||
if (method !== "Page.downloadWillBegin" || !sameTarget(source, options.target)) return;
|
||||
const event = raw as { url?: unknown; suggestedFilename?: unknown; frameId?: unknown };
|
||||
if (typeof event.url !== "string" || typeof event.suggestedFilename !== "string") return;
|
||||
if (options.expectedFrameId && event.frameId !== options.expectedFrameId) {
|
||||
fail(new Error("download originated from a different frame"));
|
||||
return;
|
||||
}
|
||||
if (intent) {
|
||||
fail(new Error("download trigger produced more than one browser download intent"));
|
||||
return;
|
||||
}
|
||||
intent = {
|
||||
url: event.url,
|
||||
suggestedFilename: event.suggestedFilename,
|
||||
...(typeof event.frameId === "string" ? { frameId: event.frameId } : {}),
|
||||
};
|
||||
reconcile();
|
||||
});
|
||||
if (!cdpSubscription) {
|
||||
return captureError("CDP download intent subscription unavailable", "none", "arm");
|
||||
}
|
||||
|
||||
options.downloads.onDeterminingFilename.addListener(determiningListener);
|
||||
options.downloads.onCreated.addListener(createdListener);
|
||||
options.downloads.onChanged.addListener(changedListener);
|
||||
options.signal?.addEventListener("abort", onAbort, { once: true });
|
||||
operationTimer = setTimeout(
|
||||
() => fail(new Error("download did not complete before timeout")),
|
||||
options.timeoutMs,
|
||||
);
|
||||
sizePoll = setInterval(() => {
|
||||
if (capturedId === undefined || settled || options.maxByteSize === undefined) return;
|
||||
void options.downloads
|
||||
.search({ id: capturedId })
|
||||
.then(([item]) => {
|
||||
if (!item || settled) return;
|
||||
if (item.bytesReceived > (options.maxByteSize as number)) {
|
||||
fail(new Error(`download exceeds transfer limit ${options.maxByteSize}`));
|
||||
}
|
||||
})
|
||||
.catch((err) => fail(err instanceof Error ? err : new Error(String(err))));
|
||||
}, SIZE_POLL_MS);
|
||||
|
||||
try {
|
||||
const triggered = await options.trigger();
|
||||
if (isRpcError(triggered)) {
|
||||
void completion.catch(() => undefined);
|
||||
const effect: TransferEffectState =
|
||||
capturedId !== undefined ? "committed" : intent ? "unknown" : "none";
|
||||
failureResult = {
|
||||
...triggered,
|
||||
data: { ...triggered.data, effect_state: effect, phase: "trigger" },
|
||||
};
|
||||
return failureResult;
|
||||
}
|
||||
click = triggered;
|
||||
const item = await completion;
|
||||
succeeded = true;
|
||||
return { click, item };
|
||||
} catch (err) {
|
||||
const effect: TransferEffectState =
|
||||
capturedId !== undefined ? "committed" : click ? "unknown" : "none";
|
||||
failureResult = captureError(
|
||||
err instanceof Error ? err.message : String(err),
|
||||
effect,
|
||||
capturedId !== undefined ? "download" : "attribution",
|
||||
);
|
||||
return failureResult;
|
||||
} finally {
|
||||
settled = true;
|
||||
if (operationTimer) clearTimeout(operationTimer);
|
||||
if (uniquenessTimer) clearTimeout(uniquenessTimer);
|
||||
if (sizePoll) clearInterval(sizePoll);
|
||||
options.signal?.removeEventListener("abort", onAbort);
|
||||
options.downloads.onDeterminingFilename.removeListener(determiningListener);
|
||||
options.downloads.onCreated.removeListener(createdListener);
|
||||
options.downloads.onChanged.removeListener(changedListener);
|
||||
cdpSubscription.dispose();
|
||||
for (const candidate of candidates.values()) {
|
||||
clearTimeout(candidate.graceTimer);
|
||||
if (candidate.item.id !== capturedId) suggestDefault(candidate);
|
||||
}
|
||||
if (!succeeded && capturedId !== undefined) {
|
||||
try {
|
||||
await cleanupClaimedDownload(options.downloads, capturedId);
|
||||
} catch {
|
||||
if (failureResult?.data) failureResult.data.cleanup_state = "failed";
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,64 @@
|
||||
// Download orchestration: validate session/tab ownership, then delegate one
|
||||
// browser-global chrome.downloads transaction to download-capture.ts.
|
||||
|
||||
import type { SessionManager } from "@/session-manager/manager";
|
||||
import type { DownloadParams, DownloadResult, RpcError } from "@/transport/types";
|
||||
import { captureBrowserDownload, chromeDownloadsApi, type DownloadsApi } from "./download-capture";
|
||||
import { clickResolvedTarget, type InteractionDeps, resolveActionTarget } from "./interaction";
|
||||
import { enforceAgentWindow, isRpcError, lookupSession, resolveTargetTab } from "./shared";
|
||||
|
||||
let downloadActive = false;
|
||||
|
||||
export type { DownloadsApi } from "./download-capture";
|
||||
|
||||
export interface DownloadDeps extends InteractionDeps {
|
||||
downloads?: DownloadsApi;
|
||||
}
|
||||
|
||||
export async function handleDownload(
|
||||
manager: SessionManager,
|
||||
params: DownloadParams,
|
||||
deps: DownloadDeps,
|
||||
): Promise<DownloadResult | RpcError> {
|
||||
if (downloadActive) return { code: "invalid_params", message: "another bsk download is active" };
|
||||
downloadActive = true;
|
||||
try {
|
||||
const ctx = lookupSession(manager, params, "download");
|
||||
if (isRpcError(ctx)) return ctx;
|
||||
const target = await resolveTargetTab(manager, ctx, params.tab_id, deps.tabsApi);
|
||||
if (isRpcError(target)) return target;
|
||||
const denied = enforceAgentWindow(ctx, target, "download");
|
||||
if (denied) return denied;
|
||||
if (!params.browser_relative_dir) {
|
||||
return { code: "invalid_params", message: "download requires a daemon capability directory" };
|
||||
}
|
||||
const address = await resolveActionTarget(deps.cdp, ctx, target, params, "download");
|
||||
if (isRpcError(address)) return address;
|
||||
|
||||
const capture = await captureBrowserDownload({
|
||||
cdp: deps.cdp,
|
||||
target: address.cdpTarget,
|
||||
downloads: deps.downloads ?? chromeDownloadsApi,
|
||||
browserRelativeDir: params.browser_relative_dir,
|
||||
maxByteSize: params.max_byte_size,
|
||||
timeoutMs: params.timeout_ms ?? 120_000,
|
||||
signal: deps.signal,
|
||||
expectedFrameId: address.frameId,
|
||||
trigger: () => clickResolvedTarget(ctx, address, {}, deps),
|
||||
});
|
||||
if (isRpcError(capture)) return capture;
|
||||
const { click, item } = capture;
|
||||
return {
|
||||
tab_id: target.tabId,
|
||||
used_ref: click.used_ref,
|
||||
used_selector: click.used_selector,
|
||||
suggested_filename: item.filename.split(/[\\/]/).pop() ?? "download",
|
||||
byte_size: item.fileSize >= 0 ? item.fileSize : item.totalBytes,
|
||||
mime: item.mime || undefined,
|
||||
danger: item.danger,
|
||||
browser_path: item.filename,
|
||||
};
|
||||
} finally {
|
||||
downloadActive = false;
|
||||
}
|
||||
}
|
||||
@@ -2,10 +2,23 @@
|
||||
// rendering. Extension handlers attach reasons here; human-facing copy
|
||||
// lives in bsk-cli `render_error.rs`.
|
||||
|
||||
import type { ErrorCode, RpcError, RpcErrorData, RpcErrorReason } from "@/transport/types";
|
||||
import type {
|
||||
ErrorCode,
|
||||
RpcError,
|
||||
RpcErrorData,
|
||||
RpcErrorReason,
|
||||
TransferCleanupState,
|
||||
TransferEffectState,
|
||||
} from "@/transport/types";
|
||||
|
||||
export type { RpcErrorData, RpcErrorReason };
|
||||
|
||||
export interface TransferErrorOptions {
|
||||
effectState: TransferEffectState;
|
||||
phase: string;
|
||||
cleanupState?: TransferCleanupState;
|
||||
}
|
||||
|
||||
export function rpcError(
|
||||
code: ErrorCode,
|
||||
reason: RpcErrorReason,
|
||||
@@ -15,3 +28,16 @@ export function rpcError(
|
||||
const data: RpcErrorData = { reason, ...extra };
|
||||
return { code, message, data };
|
||||
}
|
||||
|
||||
export function transferError(
|
||||
code: ErrorCode,
|
||||
reason: RpcErrorReason,
|
||||
message: string,
|
||||
options: TransferErrorOptions,
|
||||
): RpcError {
|
||||
return rpcError(code, reason, message, {
|
||||
effect_state: options.effectState,
|
||||
phase: options.phase,
|
||||
...(options.cleanupState ? { cleanup_state: options.cleanupState } : {}),
|
||||
});
|
||||
}
|
||||
|
||||
@@ -0,0 +1,484 @@
|
||||
// One upload transaction with an explicit browser-side commit boundary.
|
||||
// Chrome interception prevents a native chooser from escaping automation;
|
||||
// chooser events and a frame-scoped DOM probe are independent input-location
|
||||
// signals, so event delivery is not required for standard file inputs.
|
||||
|
||||
import type { ClickResult, RpcError, TransferEffectState } from "@/transport/types";
|
||||
import { transferError } from "./errors";
|
||||
import type { ResolvedActionTarget } from "./interaction";
|
||||
import { type CdpRunner, isRpcError, sendToCdpTarget } from "./shared";
|
||||
|
||||
const CLEANUP_TIMEOUT_MS = 1_000;
|
||||
const ACTIVATION_GRACE_MS = 1_000;
|
||||
const PROBE_INTERVAL_MS = 20;
|
||||
|
||||
type UploadPhase =
|
||||
| "arm_interception"
|
||||
| "arm_input_probe"
|
||||
| "trigger"
|
||||
| "resolve_input"
|
||||
| "set_files"
|
||||
| "cleanup";
|
||||
|
||||
interface RuntimeReply {
|
||||
result?: { value?: unknown; objectId?: string };
|
||||
exceptionDetails?: { text?: string; exception?: { description?: string } };
|
||||
}
|
||||
|
||||
interface ChooserEvent {
|
||||
frameId?: string;
|
||||
backendNodeId?: number;
|
||||
mode?: "selectSingle" | "selectMultiple";
|
||||
}
|
||||
|
||||
export interface FileInputTransactionOptions {
|
||||
cdp: CdpRunner;
|
||||
actionTarget: ResolvedActionTarget;
|
||||
files: string[];
|
||||
timeoutMs: number;
|
||||
signal?: AbortSignal;
|
||||
trigger(): Promise<ClickResult | RpcError>;
|
||||
}
|
||||
|
||||
export interface FileInputTransactionResult {
|
||||
click: ClickResult;
|
||||
multiple: boolean;
|
||||
}
|
||||
|
||||
class BoundedWaitError extends Error {
|
||||
constructor(
|
||||
readonly kind: "timeout" | "aborted",
|
||||
message: string,
|
||||
) {
|
||||
super(message);
|
||||
}
|
||||
}
|
||||
|
||||
function remainingMs(deadline: number): number {
|
||||
return Math.max(0, deadline - Date.now());
|
||||
}
|
||||
|
||||
async function waitBounded<T>(
|
||||
promise: Promise<T>,
|
||||
deadline: number,
|
||||
signal: AbortSignal | undefined,
|
||||
timeoutMessage: string,
|
||||
): Promise<T> {
|
||||
const remaining = remainingMs(deadline);
|
||||
if (signal?.aborted) throw new BoundedWaitError("aborted", "upload transaction aborted");
|
||||
if (remaining === 0) throw new BoundedWaitError("timeout", timeoutMessage);
|
||||
|
||||
let timer: ReturnType<typeof setTimeout> | undefined;
|
||||
let onAbort: (() => void) | undefined;
|
||||
const boundary = new Promise<never>((_resolve, reject) => {
|
||||
timer = setTimeout(() => reject(new BoundedWaitError("timeout", timeoutMessage)), remaining);
|
||||
if (signal) {
|
||||
onAbort = () => reject(new BoundedWaitError("aborted", "upload transaction aborted"));
|
||||
signal.addEventListener("abort", onAbort, { once: true });
|
||||
}
|
||||
});
|
||||
try {
|
||||
return await Promise.race([promise, boundary]);
|
||||
} finally {
|
||||
if (timer) clearTimeout(timer);
|
||||
if (signal && onAbort) signal.removeEventListener("abort", onAbort);
|
||||
}
|
||||
}
|
||||
|
||||
function runtimeError(reply: RuntimeReply, fallback: string): Error | null {
|
||||
if (!reply.exceptionDetails) return null;
|
||||
return new Error(
|
||||
reply.exceptionDetails.exception?.description ?? reply.exceptionDetails.text ?? fallback,
|
||||
);
|
||||
}
|
||||
|
||||
function transferFailure(
|
||||
code: RpcError["code"],
|
||||
reason: "file_input_probe_failed" | "file_input_not_activated" | "set_file_input_failed",
|
||||
message: string,
|
||||
effectState: TransferEffectState,
|
||||
phase: UploadPhase,
|
||||
): RpcError {
|
||||
return transferError(code, reason, message, { effectState, phase });
|
||||
}
|
||||
|
||||
function enrichFailure(
|
||||
error: RpcError,
|
||||
effectState: TransferEffectState,
|
||||
phase: UploadPhase,
|
||||
): RpcError {
|
||||
return {
|
||||
...error,
|
||||
data: { ...error.data, effect_state: effectState, phase },
|
||||
};
|
||||
}
|
||||
|
||||
function sameTarget(
|
||||
source: { tabId?: number; sessionId?: string },
|
||||
target: { tabId: number; sessionId?: string },
|
||||
): boolean {
|
||||
return source.tabId === target.tabId && source.sessionId === target.sessionId;
|
||||
}
|
||||
|
||||
async function delay(ms: number, signal?: AbortSignal): Promise<void> {
|
||||
await waitBounded(
|
||||
new Promise((resolve) => setTimeout(resolve, ms)),
|
||||
Date.now() + ms + 1,
|
||||
signal,
|
||||
"upload activation probe timed out",
|
||||
);
|
||||
}
|
||||
|
||||
async function callOnTrigger<T = RuntimeReply>(
|
||||
options: FileInputTransactionOptions,
|
||||
objectId: string,
|
||||
functionDeclaration: string,
|
||||
args: unknown[],
|
||||
returnByValue: boolean,
|
||||
): Promise<T> {
|
||||
return sendToCdpTarget<T>(options.cdp, options.actionTarget.cdpTarget, "Runtime.callFunctionOn", {
|
||||
objectId,
|
||||
functionDeclaration,
|
||||
arguments: args.map((value) => ({ value })),
|
||||
returnByValue,
|
||||
awaitPromise: false,
|
||||
});
|
||||
}
|
||||
|
||||
export async function uploadThroughActivatedFileInput(
|
||||
options: FileInputTransactionOptions,
|
||||
): Promise<FileInputTransactionResult | RpcError> {
|
||||
const deadline = Date.now() + options.timeoutMs;
|
||||
const target = options.actionTarget.cdpTarget;
|
||||
const objectGroup = `bsk-upload-${crypto.randomUUID()}`;
|
||||
const stateKey = `__bskUpload_${crypto.randomUUID().replaceAll("-", "")}`;
|
||||
const chooserEvents: ChooserEvent[] = [];
|
||||
let interceptionArmed = false;
|
||||
let probeArmed = false;
|
||||
let triggerObjectId: string | undefined;
|
||||
let outcome: FileInputTransactionResult | RpcError = transferFailure(
|
||||
"protocol_error",
|
||||
"file_input_probe_failed",
|
||||
"upload transaction ended without an outcome",
|
||||
"none",
|
||||
"cleanup",
|
||||
);
|
||||
|
||||
const chooserSubscription = options.cdp.onEvent?.((source, method, raw) => {
|
||||
if (method !== "Page.fileChooserOpened" || !sameTarget(source, target)) return;
|
||||
const event = raw as ChooserEvent;
|
||||
chooserEvents.push(event);
|
||||
});
|
||||
|
||||
try {
|
||||
try {
|
||||
await waitBounded(
|
||||
sendToCdpTarget(options.cdp, target, "Page.setInterceptFileChooserDialog", {
|
||||
enabled: true,
|
||||
}),
|
||||
deadline,
|
||||
options.signal,
|
||||
"arming native file chooser interception timed out",
|
||||
);
|
||||
interceptionArmed = true;
|
||||
} catch (err) {
|
||||
outcome = transferFailure(
|
||||
err instanceof BoundedWaitError ? "timeout" : "cdp_failed",
|
||||
"file_input_probe_failed",
|
||||
err instanceof Error ? err.message : String(err),
|
||||
"none",
|
||||
"arm_interception",
|
||||
);
|
||||
return outcome;
|
||||
}
|
||||
|
||||
try {
|
||||
const resolved = await waitBounded(
|
||||
sendToCdpTarget<{ object?: { objectId?: string } }>(
|
||||
options.cdp,
|
||||
target,
|
||||
"DOM.resolveNode",
|
||||
{
|
||||
backendNodeId: options.actionTarget.backendNodeId,
|
||||
objectGroup,
|
||||
},
|
||||
),
|
||||
deadline,
|
||||
options.signal,
|
||||
"resolving upload trigger timed out",
|
||||
);
|
||||
triggerObjectId = resolved.object?.objectId;
|
||||
if (!triggerObjectId) throw new Error("DOM.resolveNode returned no trigger objectId");
|
||||
|
||||
const armed = await waitBounded(
|
||||
callOnTrigger<RuntimeReply>(
|
||||
options,
|
||||
triggerObjectId,
|
||||
`function(key) {
|
||||
const doc = this.ownerDocument;
|
||||
const owner = doc.defaultView;
|
||||
if (!owner) return false;
|
||||
const state = { inputs: [], listener: null };
|
||||
Object.defineProperty(owner, key, { value: state, configurable: true });
|
||||
state.listener = event => {
|
||||
const path = typeof event.composedPath === "function" ? event.composedPath() : [];
|
||||
const candidate = path[0] || event.target;
|
||||
if (candidate && candidate.nodeType === 1 &&
|
||||
candidate.localName === "input" && candidate.type === "file") {
|
||||
if (!state.inputs.includes(candidate)) state.inputs.push(candidate);
|
||||
event.preventDefault();
|
||||
}
|
||||
};
|
||||
doc.addEventListener("click", state.listener, true);
|
||||
return true;
|
||||
}`,
|
||||
[stateKey],
|
||||
true,
|
||||
),
|
||||
deadline,
|
||||
options.signal,
|
||||
"arming frame-scoped file input probe timed out",
|
||||
);
|
||||
const armError = runtimeError(armed, "failed to arm file input probe");
|
||||
if (armError || armed.result?.value !== true) {
|
||||
throw armError ?? new Error("file input probe did not arm");
|
||||
}
|
||||
probeArmed = true;
|
||||
} catch (err) {
|
||||
outcome = transferFailure(
|
||||
err instanceof BoundedWaitError ? "timeout" : "cdp_failed",
|
||||
"file_input_probe_failed",
|
||||
err instanceof Error ? err.message : String(err),
|
||||
"none",
|
||||
"arm_input_probe",
|
||||
);
|
||||
return outcome;
|
||||
}
|
||||
|
||||
let click: ClickResult | RpcError;
|
||||
try {
|
||||
click = await waitBounded(
|
||||
options.trigger(),
|
||||
deadline,
|
||||
options.signal,
|
||||
"upload trigger timed out",
|
||||
);
|
||||
} catch (err) {
|
||||
outcome = transferFailure(
|
||||
err instanceof BoundedWaitError ? "timeout" : "cdp_failed",
|
||||
"file_input_probe_failed",
|
||||
err instanceof Error ? err.message : String(err),
|
||||
"none",
|
||||
"trigger",
|
||||
);
|
||||
return outcome;
|
||||
}
|
||||
if (isRpcError(click)) {
|
||||
outcome = enrichFailure(click, "none", "trigger");
|
||||
return outcome;
|
||||
}
|
||||
|
||||
try {
|
||||
const activationDeadline = Math.min(deadline, Date.now() + ACTIVATION_GRACE_MS);
|
||||
let summary: { count: number; multiple: boolean } = { count: 0, multiple: false };
|
||||
while (Date.now() < activationDeadline) {
|
||||
if (chooserEvents.length > 0) break;
|
||||
const reply = await callOnTrigger<RuntimeReply>(
|
||||
options,
|
||||
triggerObjectId,
|
||||
`function(key) {
|
||||
const state = this.ownerDocument.defaultView?.[key];
|
||||
return state
|
||||
? { count: state.inputs.length, multiple: state.inputs[0]?.multiple === true }
|
||||
: { count: 0, multiple: false };
|
||||
}`,
|
||||
[stateKey],
|
||||
true,
|
||||
);
|
||||
const summaryError = runtimeError(reply, "failed to inspect activated file input");
|
||||
if (summaryError) throw summaryError;
|
||||
const value = reply.result?.value as { count?: unknown; multiple?: unknown } | undefined;
|
||||
summary = {
|
||||
count: typeof value?.count === "number" ? value.count : 0,
|
||||
multiple: value?.multiple === true,
|
||||
};
|
||||
if (summary.count > 0) break;
|
||||
await delay(Math.min(PROBE_INTERVAL_MS, remainingMs(activationDeadline)), options.signal);
|
||||
}
|
||||
|
||||
if (chooserEvents.length > 1) {
|
||||
outcome = transferFailure(
|
||||
"unsupported",
|
||||
"file_input_not_activated",
|
||||
"upload trigger activated more than one file chooser",
|
||||
"none",
|
||||
"resolve_input",
|
||||
);
|
||||
return outcome;
|
||||
}
|
||||
|
||||
const chooser = chooserEvents[0];
|
||||
let backendNodeId: number | undefined;
|
||||
let multiple = summary.multiple;
|
||||
if (chooser) {
|
||||
if (options.actionTarget.frameId && chooser.frameId !== options.actionTarget.frameId) {
|
||||
outcome = transferFailure(
|
||||
"unsupported",
|
||||
"file_input_not_activated",
|
||||
"upload trigger activated a file chooser in a different frame",
|
||||
"none",
|
||||
"resolve_input",
|
||||
);
|
||||
return outcome;
|
||||
}
|
||||
if (typeof chooser.backendNodeId !== "number") {
|
||||
outcome = transferFailure(
|
||||
"unsupported",
|
||||
"file_input_not_activated",
|
||||
"upload trigger invoked a non-input file picker",
|
||||
"none",
|
||||
"resolve_input",
|
||||
);
|
||||
return outcome;
|
||||
}
|
||||
backendNodeId = chooser.backendNodeId;
|
||||
multiple = chooser.mode === "selectMultiple";
|
||||
} else {
|
||||
if (summary.count !== 1) {
|
||||
outcome = transferFailure(
|
||||
"unsupported",
|
||||
"file_input_not_activated",
|
||||
summary.count === 0
|
||||
? "upload trigger did not activate an input[type=file]"
|
||||
: "upload trigger activated more than one input[type=file]",
|
||||
"none",
|
||||
"resolve_input",
|
||||
);
|
||||
return outcome;
|
||||
}
|
||||
const input = await callOnTrigger<RuntimeReply>(
|
||||
options,
|
||||
triggerObjectId,
|
||||
`function(key) { return this.ownerDocument.defaultView?.[key]?.inputs[0]; }`,
|
||||
[stateKey],
|
||||
false,
|
||||
);
|
||||
const inputError = runtimeError(input, "failed to resolve activated file input object");
|
||||
if (inputError) throw inputError;
|
||||
if (!input.result?.objectId) throw new Error("activated file input returned no objectId");
|
||||
const described = await sendToCdpTarget<{ node?: { backendNodeId?: number } }>(
|
||||
options.cdp,
|
||||
target,
|
||||
"DOM.describeNode",
|
||||
{ objectId: input.result.objectId },
|
||||
);
|
||||
backendNodeId = described.node?.backendNodeId;
|
||||
if (typeof backendNodeId !== "number") {
|
||||
throw new Error("DOM.describeNode returned no file input backendNodeId");
|
||||
}
|
||||
}
|
||||
|
||||
if (!multiple && options.files.length !== 1) {
|
||||
outcome = enrichFailure(
|
||||
{ code: "invalid_params", message: "file input accepts exactly one file" },
|
||||
"none",
|
||||
"resolve_input",
|
||||
);
|
||||
return outcome;
|
||||
}
|
||||
|
||||
try {
|
||||
await waitBounded(
|
||||
sendToCdpTarget(options.cdp, target, "DOM.setFileInputFiles", {
|
||||
files: options.files,
|
||||
backendNodeId,
|
||||
}),
|
||||
deadline,
|
||||
options.signal,
|
||||
"setting file input files timed out",
|
||||
);
|
||||
} catch (err) {
|
||||
outcome = transferFailure(
|
||||
err instanceof BoundedWaitError ? "timeout" : "cdp_failed",
|
||||
"set_file_input_failed",
|
||||
err instanceof Error ? err.message : String(err),
|
||||
"unknown",
|
||||
"set_files",
|
||||
);
|
||||
return outcome;
|
||||
}
|
||||
outcome = { click, multiple };
|
||||
return outcome;
|
||||
} catch (err) {
|
||||
outcome = transferFailure(
|
||||
err instanceof BoundedWaitError ? "timeout" : "cdp_failed",
|
||||
"file_input_probe_failed",
|
||||
err instanceof Error ? err.message : String(err),
|
||||
"none",
|
||||
"resolve_input",
|
||||
);
|
||||
return outcome;
|
||||
}
|
||||
} finally {
|
||||
chooserSubscription?.dispose();
|
||||
let cleanupFailed = false;
|
||||
if (probeArmed && triggerObjectId) {
|
||||
try {
|
||||
await waitBounded(
|
||||
callOnTrigger(
|
||||
options,
|
||||
triggerObjectId,
|
||||
`function(key) {
|
||||
const owner = this.ownerDocument.defaultView;
|
||||
const state = owner?.[key];
|
||||
if (state?.listener) this.ownerDocument.removeEventListener("click", state.listener, true);
|
||||
if (owner) delete owner[key];
|
||||
}`,
|
||||
[stateKey],
|
||||
true,
|
||||
),
|
||||
Date.now() + CLEANUP_TIMEOUT_MS,
|
||||
undefined,
|
||||
"cleaning file input probe timed out",
|
||||
);
|
||||
} catch {
|
||||
cleanupFailed = true;
|
||||
}
|
||||
}
|
||||
if (interceptionArmed) {
|
||||
try {
|
||||
await waitBounded(
|
||||
sendToCdpTarget(options.cdp, target, "Page.setInterceptFileChooserDialog", {
|
||||
enabled: false,
|
||||
cancel: true,
|
||||
}),
|
||||
Date.now() + CLEANUP_TIMEOUT_MS,
|
||||
undefined,
|
||||
"disabling file chooser interception timed out",
|
||||
);
|
||||
} catch {
|
||||
cleanupFailed = true;
|
||||
}
|
||||
}
|
||||
try {
|
||||
await waitBounded(
|
||||
sendToCdpTarget(options.cdp, target, "Runtime.releaseObjectGroup", { objectGroup }),
|
||||
Date.now() + CLEANUP_TIMEOUT_MS,
|
||||
undefined,
|
||||
"releasing upload object group timed out",
|
||||
);
|
||||
} catch {
|
||||
cleanupFailed = true;
|
||||
}
|
||||
|
||||
const effect = isRpcError(outcome)
|
||||
? (outcome.data?.effect_state as TransferEffectState | undefined)
|
||||
: "committed";
|
||||
if (effect === "unknown" || cleanupFailed) {
|
||||
await options.cdp.detach?.(target.tabId);
|
||||
}
|
||||
if (cleanupFailed && isRpcError(outcome)) {
|
||||
outcome.data = { ...outcome.data, cleanup_state: "failed" };
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -40,6 +40,7 @@ import {
|
||||
enforceAgentWindow,
|
||||
isRpcError,
|
||||
lookupSession,
|
||||
type ResolvedTargetTab,
|
||||
resolveTargetTab,
|
||||
} from "./shared";
|
||||
import { resolveSnapshotRef } from "./snapshot-ref";
|
||||
@@ -56,6 +57,15 @@ export interface InteractionDeps {
|
||||
keepOverlayBypassAfterHover?: boolean;
|
||||
}
|
||||
|
||||
export interface ResolvedActionTarget {
|
||||
tab: ResolvedTargetTab;
|
||||
backendNodeId: number;
|
||||
cdpTarget: CdpTarget;
|
||||
frameId?: string;
|
||||
usedRef?: string;
|
||||
usedSelector?: string;
|
||||
}
|
||||
|
||||
const DEFAULT_TIMEOUT_MS = 30_000;
|
||||
const DEFAULT_HOVER_SETTLE_MS = 200;
|
||||
|
||||
@@ -128,7 +138,7 @@ async function wait(ms: number, signal?: AbortSignal): Promise<void> {
|
||||
* `RpcError` if the caller supplied neither (or both), or if neither
|
||||
* lookup matched.
|
||||
*/
|
||||
async function resolveBackendNode(
|
||||
export async function resolveBackendNode(
|
||||
cdp: CdpRunner,
|
||||
ctx: SessionContext,
|
||||
target: { tabId: number },
|
||||
@@ -215,6 +225,17 @@ async function resolveBackendNode(
|
||||
}
|
||||
}
|
||||
|
||||
export async function resolveActionTarget(
|
||||
cdp: CdpRunner,
|
||||
ctx: SessionContext,
|
||||
target: ResolvedTargetTab,
|
||||
params: { ref?: string; selector?: string },
|
||||
toolName: string,
|
||||
): Promise<ResolvedActionTarget | RpcError> {
|
||||
const node = await resolveBackendNode(cdp, ctx, target, params, toolName);
|
||||
return isRpcError(node) ? node : { tab: target, ...node };
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// tool.click
|
||||
// ---------------------------------------------------------------------------
|
||||
@@ -233,10 +254,19 @@ export async function handleClick(
|
||||
if (isRpcError(target)) return target;
|
||||
const denied = enforceAgentWindow(ctx, target, "click");
|
||||
if (denied) return denied;
|
||||
const dialogCursor = markDialogCursor(deps.cdp, target.tabId);
|
||||
const resolved = await resolveActionTarget(deps.cdp, ctx, target, params, "click");
|
||||
if (isRpcError(resolved)) return resolved;
|
||||
return clickResolvedTarget(ctx, resolved, params, deps);
|
||||
}
|
||||
|
||||
const node = await resolveBackendNode(deps.cdp, ctx, target, params, "click");
|
||||
if (isRpcError(node)) return node;
|
||||
export async function clickResolvedTarget(
|
||||
ctx: SessionContext,
|
||||
resolved: ResolvedActionTarget,
|
||||
params: Pick<ClickParams, "button" | "click_count" | "modifiers">,
|
||||
deps: InteractionDeps,
|
||||
): Promise<ClickResult | RpcError> {
|
||||
const { tab: target } = resolved;
|
||||
const dialogCursor = markDialogCursor(deps.cdp, target.tabId);
|
||||
|
||||
if (throwIfAborted(deps.signal)) {
|
||||
return { code: "cancelled", message: "click aborted" };
|
||||
@@ -247,9 +277,9 @@ export async function handleClick(
|
||||
deps.cdp,
|
||||
target.tabId,
|
||||
{
|
||||
target: node.cdpTarget,
|
||||
backendNodeId: node.backendNodeId,
|
||||
...(node.frameId ? { frameId: node.frameId } : {}),
|
||||
target: resolved.cdpTarget,
|
||||
backendNodeId: resolved.backendNodeId,
|
||||
...(resolved.frameId ? { frameId: resolved.frameId } : {}),
|
||||
},
|
||||
{ scrollIntoView: true },
|
||||
);
|
||||
@@ -337,8 +367,8 @@ export async function handleClick(
|
||||
|
||||
return attachDialogs(deps.cdp, target.tabId, dialogCursor, {
|
||||
tab_id: target.tabId,
|
||||
used_ref: node.usedRef,
|
||||
used_selector: node.usedSelector,
|
||||
used_ref: resolved.usedRef,
|
||||
used_selector: resolved.usedSelector,
|
||||
x: centre.x,
|
||||
y: centre.y,
|
||||
});
|
||||
|
||||
@@ -53,6 +53,7 @@ export type { DialogCursor };
|
||||
export interface CdpRunner {
|
||||
send<T = unknown>(tabId: number, method: string, params?: object): Promise<T>;
|
||||
sendToTarget?<T = unknown>(target: CdpTarget, method: string, params?: object): Promise<T>;
|
||||
detach?(tabId: number): Promise<void>;
|
||||
getFrameGraph?(tabId: number): Promise<CdpFrameGraph>;
|
||||
ensureAttachedToUrl?(tabId: number, expectedUrl: string | undefined): Promise<void>;
|
||||
trackSessionTab?(sessionId: string, tabId: number): void;
|
||||
|
||||
@@ -0,0 +1,59 @@
|
||||
// Upload orchestration: validate the session-scoped request, resolve its click
|
||||
// target, then delegate the browser protocol transaction to the file-input
|
||||
// transaction module.
|
||||
|
||||
import type { SessionManager } from "@/session-manager/manager";
|
||||
import type { RpcError, UploadParams, UploadResult } from "@/transport/types";
|
||||
import { uploadThroughActivatedFileInput } from "./file-input-transaction";
|
||||
import { clickResolvedTarget, type InteractionDeps, resolveActionTarget } from "./interaction";
|
||||
import {
|
||||
type CdpRunner,
|
||||
enforceAgentWindow,
|
||||
isRpcError,
|
||||
lookupSession,
|
||||
resolveTargetTab,
|
||||
} from "./shared";
|
||||
|
||||
const DEFAULT_TIMEOUT_MS = 120_000;
|
||||
|
||||
export interface UploadDeps extends InteractionDeps {
|
||||
cdp: CdpRunner;
|
||||
}
|
||||
|
||||
export async function handleUpload(
|
||||
manager: SessionManager,
|
||||
params: UploadParams,
|
||||
deps: UploadDeps,
|
||||
): Promise<UploadResult | RpcError> {
|
||||
const ctx = lookupSession(manager, params, "upload");
|
||||
if (isRpcError(ctx)) return ctx;
|
||||
const target = await resolveTargetTab(manager, ctx, params.tab_id, deps.tabsApi);
|
||||
if (isRpcError(target)) return target;
|
||||
const denied = enforceAgentWindow(ctx, target, "upload");
|
||||
if (denied) return denied;
|
||||
if (
|
||||
params.files.length === 0 ||
|
||||
params.files.length > 20 ||
|
||||
params.files.some((file) => !file.staged_path)
|
||||
) {
|
||||
return { code: "invalid_params", message: "upload requires daemon-staged files" };
|
||||
}
|
||||
const address = await resolveActionTarget(deps.cdp, ctx, target, params, "upload");
|
||||
if (isRpcError(address)) return address;
|
||||
const timeoutMs = params.timeout_ms ?? DEFAULT_TIMEOUT_MS;
|
||||
const transaction = await uploadThroughActivatedFileInput({
|
||||
cdp: deps.cdp,
|
||||
actionTarget: address,
|
||||
files: params.files.map((file) => file.staged_path as string),
|
||||
timeoutMs,
|
||||
signal: deps.signal,
|
||||
trigger: () => clickResolvedTarget(ctx, address, {}, deps),
|
||||
});
|
||||
if (isRpcError(transaction)) return transaction;
|
||||
return {
|
||||
tab_id: target.tabId,
|
||||
used_ref: transaction.click.used_ref,
|
||||
used_selector: transaction.click.used_selector,
|
||||
file_names: params.files.map((file) => file.name),
|
||||
};
|
||||
}
|
||||
@@ -33,10 +33,22 @@ export type RpcErrorReason =
|
||||
| "restricted_tab_url"
|
||||
| "borrow_conflict"
|
||||
| "screenshot_capture_failed"
|
||||
| "file_input_probe_failed"
|
||||
| "file_input_not_activated"
|
||||
| "set_file_input_failed"
|
||||
| "download_capture_failed"
|
||||
| "transfer_outcome_unknown"
|
||||
| "transfer_timeout"
|
||||
| "cleanup_failed";
|
||||
|
||||
export type TransferEffectState = "none" | "committed" | "unknown";
|
||||
export type TransferCleanupState = "complete" | "failed";
|
||||
|
||||
export interface RpcErrorData {
|
||||
reason?: RpcErrorReason;
|
||||
effect_state?: TransferEffectState;
|
||||
phase?: string;
|
||||
cleanup_state?: TransferCleanupState;
|
||||
[key: string]: unknown;
|
||||
}
|
||||
|
||||
@@ -500,6 +512,50 @@ export interface SelectResult {
|
||||
dialogs?: JavaScriptDialogInfo[];
|
||||
}
|
||||
|
||||
export interface UploadFile {
|
||||
transfer_id: string;
|
||||
name: string;
|
||||
staged_path?: string;
|
||||
}
|
||||
|
||||
export interface UploadParams {
|
||||
session_id: string;
|
||||
ref?: string;
|
||||
selector?: string;
|
||||
tab_id?: number;
|
||||
files: UploadFile[];
|
||||
timeout_ms?: number;
|
||||
}
|
||||
|
||||
export interface UploadResult {
|
||||
tab_id: number;
|
||||
used_ref?: string;
|
||||
used_selector?: string;
|
||||
file_names: string[];
|
||||
}
|
||||
|
||||
export interface DownloadParams {
|
||||
session_id: string;
|
||||
ref?: string;
|
||||
selector?: string;
|
||||
tab_id?: number;
|
||||
timeout_ms?: number;
|
||||
browser_relative_dir?: string;
|
||||
max_byte_size?: number;
|
||||
}
|
||||
|
||||
export interface DownloadResult {
|
||||
tab_id: number;
|
||||
used_ref?: string;
|
||||
used_selector?: string;
|
||||
suggested_filename: string;
|
||||
byte_size: number;
|
||||
mime?: string;
|
||||
danger?: string;
|
||||
browser_path?: string;
|
||||
transfer_id?: string;
|
||||
}
|
||||
|
||||
// --------------------------------------------------------------------------
|
||||
// M9 tool payloads — evaluate / wait_for_navigation / wait_ms
|
||||
// --------------------------------------------------------------------------
|
||||
|
||||
@@ -30,6 +30,7 @@ export default defineConfig({
|
||||
permissions: [
|
||||
"alarms",
|
||||
"debugger",
|
||||
"downloads",
|
||||
"idle",
|
||||
"notifications",
|
||||
"tabs",
|
||||
|
||||
@@ -53,6 +53,7 @@ flate2 = { workspace = true }
|
||||
tar = { workspace = true }
|
||||
zip = { workspace = true }
|
||||
sha2 = { workspace = true }
|
||||
base64 = { workspace = true }
|
||||
|
||||
[target.'cfg(unix)'.dependencies]
|
||||
nix = { workspace = true }
|
||||
@@ -63,6 +64,7 @@ windows-sys = { version = "0.59", features = [
|
||||
"Win32_Foundation",
|
||||
"Win32_System_Threading",
|
||||
"Win32_Security",
|
||||
"Win32_Storage_FileSystem",
|
||||
] }
|
||||
|
||||
[dev-dependencies]
|
||||
|
||||
@@ -29,7 +29,7 @@ Drive the user's **real Chromium browser** (with their logins and cookies) throu
|
||||
## Prerequisites
|
||||
|
||||
1. `bsk` on `PATH` (Rust CLI from browser-skill)
|
||||
2. browser-skill **extension** loaded in Chromium and connected (popup shows green, or an upgrade reminder — still connected; continue)
|
||||
2. browser-skill **extension** loaded in Chromium and connected (popup shows green)
|
||||
3. Any `bsk` command auto-starts background services as needed; use `bsk doctor` if anything fails
|
||||
|
||||
## Mandatory workflow
|
||||
@@ -206,6 +206,29 @@ Both capture from the moment the tab is attached and read a bounded per-tab buff
|
||||
| `bsk select <ref-or-selector> --value <v>` | Set `<select>` option(s) by `value` (repeat `--value` for multi-select) |
|
||||
| `bsk press <key>` | Key/combo (`Enter`, `Ctrl+A`, …; optional `--ref` to focus first) |
|
||||
|
||||
### File transfer (require `--session`)
|
||||
|
||||
| Command | Summary |
|
||||
|---------|---------|
|
||||
| `bsk upload <ref-or-selector> --file <path>` | Click one upload trigger and attach an agent-readable local file (`--file` is repeatable) |
|
||||
| `bsk download <ref-or-selector> --out <path>` | Click one download trigger and copy the single completed file to an exact local path (`--overwrite` is opt-in) |
|
||||
|
||||
The agent/harness decides whether a file transfer is appropriate and which local path belongs to the task. Treat upload as disclosure of that file to the current website, and download as accepting website-controlled bytes onto the local filesystem. Use only paths that are necessary for the user's bounded goal.
|
||||
|
||||
BrowserSkill enforces the mechanical boundary: files are staged under a session-scoped opaque transfer, only daemon-minted capabilities reach the extension, upload/download still obey Agent Window tab checks, and transfers are chunk/size bounded. Upload intercepts the native chooser for one transaction, locates the input activated in the resolved target's document, and assigns only the staged file paths. Download uniquely correlates one exact-target browser intent with one Chrome download in either event order, routes it through a daemon-minted relative directory, and lets the daemon validate and import it. Upload staging remains available for a later form submission and is removed when the session ends. Downloads cannot overwrite an existing destination unless `--overwrite` is explicit. BrowserSkill does not inspect file content or decide whether its meaning is sensitive.
|
||||
|
||||
Do not use `request-help` merely because a native file chooser or browser download is involved; try these commands first. For transfer failures, use the structured error instead of retrying blindly:
|
||||
|
||||
- `reason=file_input_not_activated` means the requested click did not activate exactly one `<input type="file">`; the page may use a non-input picker such as `window.showOpenFilePicker()`. Do not retry blindly. If human help is available, call `request-help` and tell the user the exact original local path to choose. The staged daemon path is internal and must not be shown to the user.
|
||||
- `reason=file_input_probe_failed` means BrowserSkill could not safely establish the browser-side upload transaction. Do not repeat the same action; use `request-help` when available.
|
||||
- `reason=set_file_input_failed` means BrowserSkill found the activated file input but Chrome rejected the staged path or assignment. Check the extension's file-URL access permission; otherwise use `request-help`.
|
||||
- `reason=download_capture_failed` means BrowserSkill could not attribute exactly one completed download to the requested target. Do not retry blindly or accept an unrelated browser download; use `request-help` when available.
|
||||
- `effect_state=none` means BrowserSkill confirmed that no file-transfer effect was committed. Follow the accompanying reason; a corrected target or explicit human fallback may be attempted.
|
||||
- `effect_state=unknown` means the browser may already have attached or created the file. Do not repeat the transfer. Observe the page if that can establish the result; otherwise stop and report the uncertainty.
|
||||
- `effect_state=committed` means the browser-side effect occurred even if later completion or cleanup failed. Do not repeat it; continue only after verifying the resulting page/download state.
|
||||
|
||||
If `request-help` returns `outcome="disabled"`, do not retry it. Stop gracefully and report the transfer mechanism that requires human intervention.
|
||||
|
||||
### Scripting & timing
|
||||
|
||||
| Command | Summary |
|
||||
@@ -299,7 +322,7 @@ bsk record stop [--output trace] # terminal fallback if the browser panel is u
|
||||
| `2` | Protocol / transport — service unreachable, IPC failure | `bsk doctor`; check extension connected; retry the command |
|
||||
| `3` | Browser / CDP execution failed | Retry; simplify selector; check tab still open |
|
||||
| `4` | Timeout | Increase `--timeout`; try `--wait-until domcontentloaded` |
|
||||
| `5` | Unknown RPC method | This command is not implemented in the current build. Continue the task with other commands. If a newly added command is missing, suggest `bsk update`. |
|
||||
| `5` | Version skew (CLI vs extension) | Upgrade/reinstall matching versions |
|
||||
|
||||
Human errors print `error:` + `hint:` on stderr; `--json` includes `code`, `message`, `hint`, `exit_code`.
|
||||
|
||||
|
||||
@@ -0,0 +1,75 @@
|
||||
//! Atomic visibility boundary for files received into a same-directory temp.
|
||||
|
||||
use std::path::Path;
|
||||
|
||||
pub fn commit(temp: &Path, out: &Path, overwrite: bool) -> std::io::Result<()> {
|
||||
if !overwrite {
|
||||
std::fs::hard_link(temp, out)?;
|
||||
std::fs::remove_file(temp)?;
|
||||
return Ok(());
|
||||
}
|
||||
replace(temp, out)
|
||||
}
|
||||
|
||||
#[cfg(unix)]
|
||||
fn replace(temp: &Path, out: &Path) -> std::io::Result<()> {
|
||||
// POSIX rename replaces an existing non-directory destination atomically.
|
||||
std::fs::rename(temp, out)
|
||||
}
|
||||
|
||||
#[cfg(windows)]
|
||||
fn replace(temp: &Path, out: &Path) -> std::io::Result<()> {
|
||||
use std::os::windows::ffi::OsStrExt;
|
||||
use windows_sys::Win32::Storage::FileSystem::{
|
||||
MOVEFILE_REPLACE_EXISTING, MOVEFILE_WRITE_THROUGH, MoveFileExW,
|
||||
};
|
||||
|
||||
let from: Vec<u16> = temp.as_os_str().encode_wide().chain(Some(0)).collect();
|
||||
let to: Vec<u16> = out.as_os_str().encode_wide().chain(Some(0)).collect();
|
||||
// SAFETY: both buffers are NUL-terminated and remain alive for the call.
|
||||
let ok = unsafe {
|
||||
MoveFileExW(
|
||||
from.as_ptr(),
|
||||
to.as_ptr(),
|
||||
MOVEFILE_REPLACE_EXISTING | MOVEFILE_WRITE_THROUGH,
|
||||
)
|
||||
};
|
||||
if ok == 0 {
|
||||
Err(std::io::Error::last_os_error())
|
||||
} else {
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(not(any(unix, windows)))]
|
||||
fn replace(temp: &Path, out: &Path) -> std::io::Result<()> {
|
||||
std::fs::rename(temp, out)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn no_replace_never_overwrites_existing_output() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let temp = dir.path().join("new.part");
|
||||
let out = dir.path().join("out.bin");
|
||||
std::fs::write(&temp, b"new").unwrap();
|
||||
std::fs::write(&out, b"old").unwrap();
|
||||
assert!(commit(&temp, &out, false).is_err());
|
||||
assert_eq!(std::fs::read(&out).unwrap(), b"old");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn overwrite_replaces_existing_output_without_predelete() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let temp = dir.path().join("new.part");
|
||||
let out = dir.path().join("out.bin");
|
||||
std::fs::write(&temp, b"new").unwrap();
|
||||
std::fs::write(&out, b"old").unwrap();
|
||||
commit(&temp, &out, true).unwrap();
|
||||
assert_eq!(std::fs::read(&out).unwrap(), b"new");
|
||||
assert!(!temp.exists());
|
||||
}
|
||||
}
|
||||
@@ -351,10 +351,10 @@ fn check_version_compatible(status: Option<&StatusResult>) -> CheckResult {
|
||||
}
|
||||
}
|
||||
|
||||
/// `bsk doctor` check: connected browsers should speak a protocol the
|
||||
/// daemon accepts. A different protocol string is still a live
|
||||
/// connection — report Ok so agents keep working. The detail is an
|
||||
/// upgrade reminder, not a blocker.
|
||||
/// `bsk doctor` check: every connected browser should use the same
|
||||
/// protocol version as the daemon. Minor protocol drift is accepted by
|
||||
/// the daemon but flagged here so the user can see who needs updating
|
||||
/// (M10.4).
|
||||
///
|
||||
/// Review M2 (round-1 minor): when no browsers are connected, the
|
||||
/// check has nothing to compare against, so it now reports
|
||||
@@ -396,12 +396,13 @@ fn check_browsers_protocol_compatible(status: Option<&StatusResult>) -> CheckRes
|
||||
})
|
||||
.collect::<Vec<_>>()
|
||||
.join(", ");
|
||||
CheckResult::ok(
|
||||
CheckResult::fail(
|
||||
name,
|
||||
format!(
|
||||
"{} browser(s) report a different protocol version (still usable — continue, and upgrade soon): {stale}",
|
||||
"{} browser(s) have protocol minor drift from the daemon: {stale}",
|
||||
status.version_skew_browsers.len()
|
||||
),
|
||||
"upgrade the browser-skill extension or bsk CLI so both sides use the same protocol version",
|
||||
)
|
||||
}
|
||||
|
||||
@@ -580,7 +581,7 @@ mod m2_tests {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn browsers_check_reports_ok_when_skew_present() {
|
||||
fn browsers_check_reports_fail_when_skew_present() {
|
||||
let status = fake_status(
|
||||
vec![BrowserStatusEntry {
|
||||
instance_id: "alpha".into(),
|
||||
@@ -604,17 +605,9 @@ mod m2_tests {
|
||||
}],
|
||||
);
|
||||
let check = check_browsers_protocol_compatible(Some(&status));
|
||||
assert_eq!(check.status, CheckStatus::Ok);
|
||||
assert_eq!(check.status, CheckStatus::Fail);
|
||||
assert!(check.detail.contains("alpha"));
|
||||
assert!(
|
||||
check
|
||||
.detail
|
||||
.contains("still usable — continue, and upgrade soon")
|
||||
);
|
||||
assert!(
|
||||
check.hint.is_none(),
|
||||
"a protocol-version note must not hint to stop"
|
||||
);
|
||||
assert!(check.hint.is_some());
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -642,7 +635,7 @@ mod m2_tests {
|
||||
}],
|
||||
);
|
||||
let check = check_browsers_protocol_compatible(Some(&status));
|
||||
assert_eq!(check.status, CheckStatus::Ok);
|
||||
assert_eq!(check.status, CheckStatus::Fail);
|
||||
assert!(
|
||||
check
|
||||
.detail
|
||||
|
||||
@@ -0,0 +1,147 @@
|
||||
//! `bsk download` — capture one browser download and atomically commit it.
|
||||
|
||||
use std::fs::OpenOptions;
|
||||
use std::io::Write;
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::time::Duration;
|
||||
|
||||
use anyhow::Context;
|
||||
use base64::Engine;
|
||||
use bsk_protocol::Method;
|
||||
use bsk_protocol::tools::{
|
||||
DownloadParams, DownloadResult, TransferChunkParams, TransferChunkResult, TransferIdParams,
|
||||
TransferReleaseResult,
|
||||
};
|
||||
use clap::Args;
|
||||
use uuid::Uuid;
|
||||
|
||||
use crate::cli::atomic_output;
|
||||
use crate::cli::ensure_daemon::ensure_daemon;
|
||||
use crate::cli::error::{CliError, Format};
|
||||
use crate::cli::interaction::split_target;
|
||||
use crate::cli::navigate::parse_timeout_ms;
|
||||
|
||||
#[derive(Debug, Clone, Args)]
|
||||
pub struct DownloadArgs {
|
||||
/// Snapshot ref (`@e3`) or CSS selector for the download trigger.
|
||||
pub target: Option<String>,
|
||||
#[arg(long = "ref")]
|
||||
pub ref_: Option<String>,
|
||||
#[arg(long = "selector")]
|
||||
pub selector: Option<String>,
|
||||
#[arg(long)]
|
||||
pub out: PathBuf,
|
||||
#[arg(long)]
|
||||
pub session: String,
|
||||
#[arg(long = "tab-id")]
|
||||
pub tab_id: Option<i64>,
|
||||
#[arg(long, default_value = "2m", value_parser = parse_timeout_ms)]
|
||||
pub timeout: u32,
|
||||
#[arg(long)]
|
||||
pub overwrite: bool,
|
||||
}
|
||||
|
||||
pub fn dispatch(args: DownloadArgs, format: Format) -> Result<(), CliError> {
|
||||
if args.out.exists() && !args.overwrite {
|
||||
return Err(CliError::Local(anyhow::anyhow!(
|
||||
"output already exists (pass --overwrite to replace it): {}",
|
||||
args.out.display()
|
||||
)));
|
||||
}
|
||||
let info = ensure_daemon().context("ensure daemon is running")?;
|
||||
let (ref_, selector) = split_target(args.target, args.ref_, args.selector)?;
|
||||
let params = DownloadParams {
|
||||
session_id: args.session,
|
||||
ref_,
|
||||
selector,
|
||||
tab_id: args.tab_id,
|
||||
timeout_ms: Some(args.timeout),
|
||||
browser_relative_dir: None,
|
||||
max_byte_size: None,
|
||||
};
|
||||
let reply: DownloadResult = crate::cli::business_rpc::call(
|
||||
info.sock_path.clone(),
|
||||
"download",
|
||||
Method::ToolDownload,
|
||||
Some(params),
|
||||
ipc_timeout(args.timeout),
|
||||
)?;
|
||||
let transfer_id = reply.transfer_id.clone().ok_or_else(|| {
|
||||
CliError::Local(anyhow::anyhow!("daemon returned no download transfer id"))
|
||||
})?;
|
||||
let write_result = write_transfer(&info.sock_path, &transfer_id, &args.out, args.overwrite);
|
||||
let _: Result<TransferReleaseResult, CliError> = crate::cli::business_rpc::call(
|
||||
info.sock_path,
|
||||
"transfer-release",
|
||||
Method::TransferRelease,
|
||||
Some(TransferIdParams { transfer_id }),
|
||||
Duration::from_secs(5),
|
||||
);
|
||||
write_result?;
|
||||
match format {
|
||||
Format::Json => {
|
||||
let mut value = serde_json::to_value(&reply).unwrap();
|
||||
value["path"] = serde_json::json!(args.out.to_string_lossy());
|
||||
if let Some(obj) = value.as_object_mut() {
|
||||
obj.remove("transfer_id");
|
||||
obj.remove("browser_path");
|
||||
}
|
||||
println!("{}", serde_json::to_string_pretty(&value).unwrap());
|
||||
}
|
||||
Format::Human => println!("{}", args.out.display()),
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn write_transfer(sock: &Path, id: &str, out: &Path, overwrite: bool) -> Result<(), CliError> {
|
||||
let parent = out
|
||||
.parent()
|
||||
.filter(|p| !p.as_os_str().is_empty())
|
||||
.unwrap_or(Path::new("."));
|
||||
let temp = parent.join(format!(".bsk-download-{}.part", Uuid::new_v4().simple()));
|
||||
let result = (|| {
|
||||
let mut file = OpenOptions::new()
|
||||
.write(true)
|
||||
.create_new(true)
|
||||
.open(&temp)
|
||||
.with_context(|| format!("create temporary download {}", temp.display()))
|
||||
.map_err(CliError::Local)?;
|
||||
let mut offset = 0u64;
|
||||
loop {
|
||||
let chunk: TransferChunkResult = crate::cli::business_rpc::call(
|
||||
sock.to_path_buf(),
|
||||
"transfer-read",
|
||||
Method::TransferRead,
|
||||
Some(TransferChunkParams {
|
||||
transfer_id: id.to_string(),
|
||||
offset,
|
||||
data_base64: String::new(),
|
||||
}),
|
||||
Duration::from_secs(30),
|
||||
)?;
|
||||
let encoded = chunk.data_base64.as_deref().unwrap_or("");
|
||||
let bytes = base64::engine::general_purpose::STANDARD
|
||||
.decode(encoded)
|
||||
.map_err(|e| CliError::Local(anyhow::anyhow!("decode download chunk: {e}")))?;
|
||||
file.write_all(&bytes)
|
||||
.map_err(|e| CliError::Local(e.into()))?;
|
||||
offset = chunk.next_offset;
|
||||
if chunk.eof {
|
||||
break;
|
||||
}
|
||||
}
|
||||
file.sync_all().map_err(|e| CliError::Local(e.into()))?;
|
||||
drop(file);
|
||||
atomic_output::commit(&temp, out, overwrite)
|
||||
.with_context(|| format!("atomically commit download to {}", out.display()))
|
||||
.map_err(CliError::Local)
|
||||
})();
|
||||
if result.is_err() {
|
||||
let _ = std::fs::remove_file(&temp);
|
||||
}
|
||||
result
|
||||
}
|
||||
|
||||
fn ipc_timeout(timeout_ms: u32) -> Duration {
|
||||
Duration::from_millis(u64::from(timeout_ms) + 5_000)
|
||||
}
|
||||
@@ -71,7 +71,7 @@ pub(crate) fn parse_modifiers(input: &str) -> Result<Vec<KeyModifier>, String> {
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
fn split_target(
|
||||
pub(crate) fn split_target(
|
||||
positional: Option<String>,
|
||||
explicit_ref: Option<String>,
|
||||
explicit_selector: Option<String>,
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
|
||||
use std::time::Duration;
|
||||
|
||||
mod atomic_output;
|
||||
pub mod browser_wait;
|
||||
pub mod browsers;
|
||||
pub mod business_rpc;
|
||||
@@ -9,6 +10,7 @@ pub mod console;
|
||||
pub mod daemon;
|
||||
pub mod dialogs;
|
||||
pub mod doctor;
|
||||
pub mod download;
|
||||
pub mod emulate;
|
||||
pub mod ensure_daemon;
|
||||
pub mod error;
|
||||
@@ -31,6 +33,7 @@ pub mod snapshot;
|
||||
pub mod status;
|
||||
pub mod tab;
|
||||
pub mod update;
|
||||
pub mod upload;
|
||||
pub mod waits;
|
||||
pub mod window;
|
||||
|
||||
@@ -38,6 +41,7 @@ use clap::{Args, Parser, Subcommand};
|
||||
|
||||
use crate::cli::console::ConsoleArgs;
|
||||
use crate::cli::daemon::DaemonCmd;
|
||||
use crate::cli::download::DownloadArgs;
|
||||
use crate::cli::emulate::EmulateArgs;
|
||||
use crate::cli::evaluate::EvaluateArgs;
|
||||
use crate::cli::get_html::GetHtmlArgs;
|
||||
@@ -53,6 +57,7 @@ use crate::cli::session::SessionCmd;
|
||||
use crate::cli::snapshot::SnapshotArgs;
|
||||
use crate::cli::tab::TabCmd;
|
||||
use crate::cli::update::UpdateArgs;
|
||||
use crate::cli::upload::UploadArgs;
|
||||
use crate::cli::waits::{WaitForNavigationArgs, WaitMsArgs};
|
||||
use crate::cli::window::WindowCmd;
|
||||
|
||||
@@ -177,6 +182,12 @@ pub enum Command {
|
||||
/// Set `<select>` option values by `value` attribute.
|
||||
Select(SelectArgs),
|
||||
|
||||
/// Upload one or more local files through a page file chooser.
|
||||
Upload(UploadArgs),
|
||||
|
||||
/// Capture one browser download and write it to a local path.
|
||||
Download(DownloadArgs),
|
||||
|
||||
/// Evaluate a JavaScript expression inside the Agent Window.
|
||||
Evaluate(EvaluateArgs),
|
||||
|
||||
|
||||
@@ -49,6 +49,12 @@ pub mod reason {
|
||||
pub const TAB_NOT_ACTIVE: &str = "tab_not_active";
|
||||
pub const BORROW_CONFLICT: &str = "borrow_conflict";
|
||||
pub const SCREENSHOT_CAPTURE_FAILED: &str = "screenshot_capture_failed";
|
||||
pub const FILE_INPUT_PROBE_FAILED: &str = "file_input_probe_failed";
|
||||
pub const FILE_INPUT_NOT_ACTIVATED: &str = "file_input_not_activated";
|
||||
pub const SET_FILE_INPUT_FAILED: &str = "set_file_input_failed";
|
||||
pub const DOWNLOAD_CAPTURE_FAILED: &str = "download_capture_failed";
|
||||
pub const TRANSFER_OUTCOME_UNKNOWN: &str = "transfer_outcome_unknown";
|
||||
pub const TRANSFER_TIMEOUT: &str = "transfer_timeout";
|
||||
pub const SESSION_BUSY: &str = crate::rpc_reason::SESSION_BUSY;
|
||||
pub const RECORD_START_PAGE_UNREACHABLE: &str = "record_start_page_unreachable";
|
||||
}
|
||||
@@ -197,6 +203,20 @@ pub fn info_for_error(code: ErrorCode, data: Option<&serde_json::Value>) -> Rend
|
||||
return base;
|
||||
};
|
||||
match (code, reason) {
|
||||
(_, reason::TRANSFER_OUTCOME_UNKNOWN) => RenderInfo {
|
||||
summary: "the file transfer outcome could not be confirmed",
|
||||
hint: Some(
|
||||
"do not retry the transfer: the browser may already have applied it; inspect the page or stop safely",
|
||||
),
|
||||
exit_code: base.exit_code,
|
||||
},
|
||||
(ErrorCode::Timeout, reason::TRANSFER_TIMEOUT) => RenderInfo {
|
||||
summary: "the file transfer timed out after browser dispatch",
|
||||
hint: Some(
|
||||
"do not retry when effect_state is unknown or committed; inspect the page before taking another action",
|
||||
),
|
||||
exit_code: base.exit_code,
|
||||
},
|
||||
(ErrorCode::PermissionDenied, reason::ELEMENT_NOT_VISIBLE) => RenderInfo {
|
||||
summary: "target element has no visible geometry",
|
||||
hint: Some(
|
||||
@@ -280,6 +300,36 @@ pub fn info_for_error(code: ErrorCode, data: Option<&serde_json::Value>) -> Rend
|
||||
),
|
||||
exit_code: base.exit_code,
|
||||
},
|
||||
(ErrorCode::Unsupported, reason::FILE_INPUT_NOT_ACTIVATED) => RenderInfo {
|
||||
summary: "the upload trigger did not activate a file input",
|
||||
hint: Some(
|
||||
"the page may use a non-input picker such as showOpenFilePicker(); do not retry blindly — use `bsk request-help` with the exact original local path, or stop if human help is disabled",
|
||||
),
|
||||
exit_code: base.exit_code,
|
||||
},
|
||||
(ErrorCode::Timeout | ErrorCode::CdpFailed, reason::FILE_INPUT_PROBE_FAILED) => {
|
||||
RenderInfo {
|
||||
summary: "the browser upload transaction could not be established",
|
||||
hint: Some(
|
||||
"do not retry the same upload blindly; use `bsk request-help` when human help is available",
|
||||
),
|
||||
exit_code: base.exit_code,
|
||||
}
|
||||
}
|
||||
(ErrorCode::Timeout | ErrorCode::CdpFailed, reason::SET_FILE_INPUT_FAILED) => RenderInfo {
|
||||
summary: "the browser could not attach the staged file to the input",
|
||||
hint: Some(
|
||||
"check that BrowserSkill has Chrome's 'Allow access to file URLs' permission; otherwise use `bsk request-help`",
|
||||
),
|
||||
exit_code: base.exit_code,
|
||||
},
|
||||
(ErrorCode::CdpFailed, reason::DOWNLOAD_CAPTURE_FAILED) => RenderInfo {
|
||||
summary: "the browser download could not be attributed or completed",
|
||||
hint: Some(
|
||||
"do not retry blindly; the target browser may not expose a reliable download intent signal",
|
||||
),
|
||||
exit_code: base.exit_code,
|
||||
},
|
||||
_ => base,
|
||||
}
|
||||
}
|
||||
@@ -419,6 +469,38 @@ mod tests {
|
||||
assert_eq!(info.exit_code, 3);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn file_transfer_reasons_render_actionable_fallbacks() {
|
||||
let unsupported = serde_json::json!({ "reason": reason::FILE_INPUT_NOT_ACTIVATED });
|
||||
let info = info_for_error(ErrorCode::Unsupported, Some(&unsupported));
|
||||
assert_eq!(
|
||||
info.summary,
|
||||
"the upload trigger did not activate a file input"
|
||||
);
|
||||
assert!(info.hint.unwrap().contains("exact original local path"));
|
||||
|
||||
let control = serde_json::json!({ "reason": reason::FILE_INPUT_PROBE_FAILED });
|
||||
let info = info_for_error(ErrorCode::Timeout, Some(&control));
|
||||
assert!(
|
||||
info.summary
|
||||
.contains("transaction could not be established")
|
||||
);
|
||||
assert!(info.hint.unwrap().contains("do not retry"));
|
||||
|
||||
let download = serde_json::json!({ "reason": reason::DOWNLOAD_CAPTURE_FAILED });
|
||||
let info = info_for_error(ErrorCode::CdpFailed, Some(&download));
|
||||
assert!(info.summary.contains("download could not be attributed"));
|
||||
|
||||
let unknown = serde_json::json!({ "reason": reason::TRANSFER_OUTCOME_UNKNOWN });
|
||||
let info = info_for_error(ErrorCode::ProtocolError, Some(&unknown));
|
||||
assert!(info.summary.contains("outcome could not be confirmed"));
|
||||
assert!(info.hint.unwrap().contains("do not retry"));
|
||||
|
||||
let timeout = serde_json::json!({ "reason": reason::TRANSFER_TIMEOUT });
|
||||
let info = info_for_error(ErrorCode::Timeout, Some(&timeout));
|
||||
assert!(info.summary.contains("timed out after browser dispatch"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ref_not_found_overrides_not_found_copy() {
|
||||
let data = serde_json::json!({ "reason": reason::REF_NOT_FOUND });
|
||||
|
||||
@@ -65,7 +65,7 @@ fn render_human(s: &StatusResult) {
|
||||
let bold = "\x1b[1m";
|
||||
let reset = "\x1b[0m";
|
||||
eprintln!(
|
||||
"{bold}{yellow}note:{reset} {} browser(s) report a different protocol version; still usable — continue, and run `bsk update` soon{reset}",
|
||||
"{bold}{yellow}warning:{reset} {} browser(s) have protocol version drift from the daemon{reset}",
|
||||
s.version_skew_browsers.len()
|
||||
);
|
||||
for skew in &s.version_skew_browsers {
|
||||
@@ -75,7 +75,7 @@ fn render_human(s: &StatusResult) {
|
||||
skew.label.clone()
|
||||
};
|
||||
eprintln!(
|
||||
" {} ({}) — protocol ext {} vs CLI {} (app ext v{}, CLI v{}) — still usable; please upgrade",
|
||||
" {} ({}) — protocol ext {} vs daemon {} (app ext v{}, daemon v{}) — please align protocol versions",
|
||||
skew.instance_id,
|
||||
label,
|
||||
display_protocol(&skew.client_protocol_version),
|
||||
|
||||
@@ -0,0 +1,185 @@
|
||||
//! `bsk upload` — stage caller-readable files and attach them to a page.
|
||||
|
||||
use std::fs::File;
|
||||
use std::io::Read;
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::time::Duration;
|
||||
|
||||
use anyhow::Context;
|
||||
use base64::Engine;
|
||||
use bsk_protocol::Method;
|
||||
use bsk_protocol::tools::{
|
||||
TransferBeginParams, TransferBeginResult, TransferChunkParams, TransferChunkResult,
|
||||
TransferIdParams, TransferReadyResult, TransferReleaseResult, UploadFile, UploadParams,
|
||||
UploadResult,
|
||||
};
|
||||
use clap::Args;
|
||||
|
||||
use crate::cli::ensure_daemon::ensure_daemon;
|
||||
use crate::cli::error::{CliError, Format};
|
||||
use crate::cli::interaction::split_target;
|
||||
use crate::cli::navigate::parse_timeout_ms;
|
||||
|
||||
#[derive(Debug, Clone, Args)]
|
||||
pub struct UploadArgs {
|
||||
/// Snapshot ref (`@e3`) or CSS selector for the file input / chooser trigger.
|
||||
pub target: Option<String>,
|
||||
#[arg(long = "ref")]
|
||||
pub ref_: Option<String>,
|
||||
#[arg(long = "selector")]
|
||||
pub selector: Option<String>,
|
||||
/// Local file to upload. Repeat for a multiple-file input.
|
||||
#[arg(long = "file", required = true)]
|
||||
pub files: Vec<PathBuf>,
|
||||
#[arg(long)]
|
||||
pub session: String,
|
||||
#[arg(long = "tab-id")]
|
||||
pub tab_id: Option<i64>,
|
||||
#[arg(long, default_value = "2m", value_parser = parse_timeout_ms)]
|
||||
pub timeout: u32,
|
||||
}
|
||||
|
||||
pub fn dispatch(args: UploadArgs, format: Format) -> Result<(), CliError> {
|
||||
if args.files.len() > crate::daemon::file_transfer::MAX_UPLOAD_FILES {
|
||||
return Err(CliError::Local(anyhow::anyhow!(
|
||||
"upload accepts at most {} files",
|
||||
crate::daemon::file_transfer::MAX_UPLOAD_FILES
|
||||
)));
|
||||
}
|
||||
let info = ensure_daemon().context("ensure daemon is running")?;
|
||||
let (ref_, selector) = split_target(args.target, args.ref_, args.selector)?;
|
||||
let mut staged = Vec::new();
|
||||
for path in &args.files {
|
||||
match stage_file(&info.sock_path, &args.session, path) {
|
||||
Ok(file) => staged.push(file),
|
||||
Err(err) => {
|
||||
for (id, _) in &staged {
|
||||
let _ = release(&info.sock_path, id);
|
||||
}
|
||||
return Err(err);
|
||||
}
|
||||
}
|
||||
}
|
||||
let params = UploadParams {
|
||||
session_id: args.session,
|
||||
ref_,
|
||||
selector,
|
||||
tab_id: args.tab_id,
|
||||
files: staged
|
||||
.iter()
|
||||
.map(|(id, name)| UploadFile {
|
||||
transfer_id: id.clone(),
|
||||
name: name.clone(),
|
||||
staged_path: None,
|
||||
})
|
||||
.collect(),
|
||||
timeout_ms: Some(args.timeout),
|
||||
};
|
||||
let result = crate::cli::business_rpc::call::<_, UploadResult>(
|
||||
info.sock_path.clone(),
|
||||
"upload",
|
||||
Method::ToolUpload,
|
||||
Some(params),
|
||||
ipc_timeout(args.timeout),
|
||||
);
|
||||
// Once tool.upload is dispatched, staging ownership belongs to the
|
||||
// session. A transport timeout cannot prove that Chrome did not attach
|
||||
// the file, so releasing here could invalidate a late successful attach.
|
||||
// Session teardown remains the single cleanup boundary after dispatch.
|
||||
let reply = result?;
|
||||
match format {
|
||||
Format::Json => println!("{}", serde_json::to_string_pretty(&reply).unwrap()),
|
||||
Format::Human => println!(
|
||||
"upload ok tab={} files={}",
|
||||
reply.tab_id,
|
||||
reply.file_names.join(", ")
|
||||
),
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn stage_file(sock: &Path, session: &str, path: &PathBuf) -> Result<(String, String), CliError> {
|
||||
let mut file = File::open(path)
|
||||
.with_context(|| format!("open upload file {}", path.display()))
|
||||
.map_err(CliError::Local)?;
|
||||
let meta = file
|
||||
.metadata()
|
||||
.with_context(|| format!("read upload file metadata {}", path.display()))
|
||||
.map_err(CliError::Local)?;
|
||||
if !meta.is_file() {
|
||||
return Err(CliError::Local(anyhow::anyhow!(
|
||||
"upload source is not a regular file: {}",
|
||||
path.display()
|
||||
)));
|
||||
}
|
||||
let name = path
|
||||
.file_name()
|
||||
.and_then(|v| v.to_str())
|
||||
.filter(|v| !v.is_empty())
|
||||
.ok_or_else(|| CliError::Local(anyhow::anyhow!("upload file has no valid basename")))?
|
||||
.to_string();
|
||||
let begin: TransferBeginResult = crate::cli::business_rpc::call(
|
||||
sock.to_path_buf(),
|
||||
"transfer-begin",
|
||||
Method::TransferBegin,
|
||||
Some(TransferBeginParams {
|
||||
session_id: session.to_string(),
|
||||
name: name.clone(),
|
||||
byte_size: meta.len(),
|
||||
}),
|
||||
Duration::from_secs(10),
|
||||
)?;
|
||||
let staged = (|| {
|
||||
let mut offset = 0u64;
|
||||
let mut buf = vec![0u8; begin.chunk_size as usize];
|
||||
loop {
|
||||
let n = file.read(&mut buf).map_err(|e| CliError::Local(e.into()))?;
|
||||
if n == 0 {
|
||||
break;
|
||||
}
|
||||
let reply: TransferChunkResult = crate::cli::business_rpc::call(
|
||||
sock.to_path_buf(),
|
||||
"transfer-chunk",
|
||||
Method::TransferChunk,
|
||||
Some(TransferChunkParams {
|
||||
transfer_id: begin.transfer_id.clone(),
|
||||
offset,
|
||||
data_base64: base64::engine::general_purpose::STANDARD.encode(&buf[..n]),
|
||||
}),
|
||||
Duration::from_secs(30),
|
||||
)?;
|
||||
offset = reply.next_offset;
|
||||
}
|
||||
let _: TransferReadyResult = crate::cli::business_rpc::call(
|
||||
sock.to_path_buf(),
|
||||
"transfer-finish",
|
||||
Method::TransferFinish,
|
||||
Some(TransferIdParams {
|
||||
transfer_id: begin.transfer_id.clone(),
|
||||
}),
|
||||
Duration::from_secs(10),
|
||||
)?;
|
||||
Ok::<_, CliError>(())
|
||||
})();
|
||||
if let Err(err) = staged {
|
||||
let _ = release(sock, &begin.transfer_id);
|
||||
return Err(err);
|
||||
}
|
||||
Ok((begin.transfer_id, name))
|
||||
}
|
||||
|
||||
fn release(sock: &Path, id: &str) -> Result<TransferReleaseResult, CliError> {
|
||||
crate::cli::business_rpc::call(
|
||||
sock.to_path_buf(),
|
||||
"transfer-release",
|
||||
Method::TransferRelease,
|
||||
Some(TransferIdParams {
|
||||
transfer_id: id.to_string(),
|
||||
}),
|
||||
Duration::from_secs(5),
|
||||
)
|
||||
}
|
||||
|
||||
fn ipc_timeout(timeout_ms: u32) -> Duration {
|
||||
Duration::from_millis(u64::from(timeout_ms) + 5_000)
|
||||
}
|
||||
@@ -0,0 +1,728 @@
|
||||
//! Session-scoped, content-transparent staging for upload/download tools.
|
||||
//!
|
||||
//! The registry deliberately exposes no arbitrary-path operations. Upload
|
||||
//! bytes arrive from the invoking CLI in bounded chunks; browser downloads
|
||||
//! land in a daemon-minted directory and are read back by the invoking CLI.
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::fs::{self, File, OpenOptions};
|
||||
use std::io::{Read, Seek, SeekFrom, Write};
|
||||
use std::path::{Component, Path, PathBuf};
|
||||
use std::sync::Mutex;
|
||||
|
||||
use base64::Engine;
|
||||
use bsk_protocol::tools::{
|
||||
TransferBeginParams, TransferBeginResult, TransferChunkParams, TransferChunkResult,
|
||||
TransferIdParams, TransferReadyResult, TransferReleaseResult,
|
||||
};
|
||||
use bsk_protocol::{ErrorCode, RpcError};
|
||||
use uuid::Uuid;
|
||||
|
||||
use super::paths;
|
||||
|
||||
pub const TRANSFER_CHUNK_SIZE: u32 = 512 * 1024;
|
||||
pub const MAX_TRANSFER_BYTES: u64 = 512 * 1024 * 1024;
|
||||
pub const MAX_UPLOAD_FILES: usize = 20;
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
enum Direction {
|
||||
Upload,
|
||||
Download,
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
struct Entry {
|
||||
session_id: String,
|
||||
direction: Direction,
|
||||
path: PathBuf,
|
||||
expected_size: Option<u64>,
|
||||
written: u64,
|
||||
ready: bool,
|
||||
}
|
||||
|
||||
/// Browser download path after the daemon has verified that it belongs to
|
||||
/// the capability minted for this transfer. From this point onward the daemon
|
||||
/// owns cleanup on every error path; unvalidated arbitrary paths are never
|
||||
/// removed.
|
||||
#[derive(Debug)]
|
||||
struct ValidatedBrowserFile {
|
||||
path: PathBuf,
|
||||
parent: PathBuf,
|
||||
cleanup_armed: bool,
|
||||
}
|
||||
|
||||
impl ValidatedBrowserFile {
|
||||
fn validate(reported_path: &Path, transfer_id: &str) -> Result<Self, RpcError> {
|
||||
if !reported_path.is_absolute() {
|
||||
return Err(permission(
|
||||
"download path is outside its browser capability",
|
||||
));
|
||||
}
|
||||
let expected_parent = Path::new("BrowserSkill").join(transfer_id);
|
||||
let parent = reported_path
|
||||
.parent()
|
||||
.ok_or_else(|| permission("download path has no parent directory"))?;
|
||||
if !parent.ends_with(&expected_parent) {
|
||||
return Err(permission(
|
||||
"download escaped its browser capability directory",
|
||||
));
|
||||
}
|
||||
for path in [reported_path, parent] {
|
||||
if fs::symlink_metadata(path)
|
||||
.map_err(io_error)?
|
||||
.file_type()
|
||||
.is_symlink()
|
||||
{
|
||||
return Err(permission("download capability path contains a symlink"));
|
||||
}
|
||||
}
|
||||
Ok(Self {
|
||||
path: reported_path.to_path_buf(),
|
||||
parent: parent.to_path_buf(),
|
||||
cleanup_armed: true,
|
||||
})
|
||||
}
|
||||
|
||||
fn remove_source(mut self) -> Result<(), RpcError> {
|
||||
remove_if_present(&self.path).map_err(io_error)?;
|
||||
remove_dir_if_present(&self.parent).map_err(io_error)?;
|
||||
self.cleanup_armed = false;
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for ValidatedBrowserFile {
|
||||
fn drop(&mut self) {
|
||||
if !self.cleanup_armed {
|
||||
return;
|
||||
}
|
||||
let _ = remove_if_present(&self.path);
|
||||
let _ = remove_dir_if_present(&self.parent);
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
pub struct DownloadStaging {
|
||||
pub transfer_id: String,
|
||||
pub browser_relative_dir: String,
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
pub struct TransferRegistry {
|
||||
root: PathBuf,
|
||||
initialized: Mutex<bool>,
|
||||
entries: Mutex<HashMap<String, Entry>>,
|
||||
}
|
||||
|
||||
impl TransferRegistry {
|
||||
pub fn new() -> anyhow::Result<Self> {
|
||||
Ok(Self {
|
||||
root: paths::bsk_home()?.join("run").join("transfers"),
|
||||
initialized: Mutex::new(false),
|
||||
entries: Mutex::new(HashMap::new()),
|
||||
})
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
fn at_root(root: PathBuf) -> anyhow::Result<Self> {
|
||||
let registry = Self {
|
||||
root,
|
||||
initialized: Mutex::new(false),
|
||||
entries: Mutex::new(HashMap::new()),
|
||||
};
|
||||
registry.ensure_root()?;
|
||||
Ok(registry)
|
||||
}
|
||||
|
||||
pub fn initialize(&self) -> anyhow::Result<()> {
|
||||
self.ensure_root()
|
||||
}
|
||||
|
||||
fn ensure_root(&self) -> anyhow::Result<()> {
|
||||
let mut initialized = self.initialized.lock().unwrap();
|
||||
if *initialized {
|
||||
return Ok(());
|
||||
}
|
||||
if self.root.exists() {
|
||||
// Transfers never survive a daemon; removing stale directories on
|
||||
// startup gives crash cleanup without a second persistence model.
|
||||
let _ = fs::remove_dir_all(&self.root);
|
||||
}
|
||||
fs::create_dir_all(&self.root)?;
|
||||
#[cfg(unix)]
|
||||
{
|
||||
use std::os::unix::fs::PermissionsExt;
|
||||
fs::set_permissions(&self.root, fs::Permissions::from_mode(0o700))?;
|
||||
}
|
||||
*initialized = true;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub fn begin_upload(&self, p: TransferBeginParams) -> Result<TransferBeginResult, RpcError> {
|
||||
self.ensure_root()
|
||||
.map_err(|err| protocol_error(err.to_string()))?;
|
||||
let basename = safe_upload_basename(&p.name)?;
|
||||
if p.byte_size > MAX_TRANSFER_BYTES {
|
||||
return Err(invalid(format!(
|
||||
"file size {} exceeds transfer limit {}",
|
||||
p.byte_size, MAX_TRANSFER_BYTES
|
||||
)));
|
||||
}
|
||||
let mut entries = self.entries.lock().unwrap();
|
||||
let session_bytes: u64 = entries
|
||||
.values()
|
||||
.filter(|entry| entry.session_id == p.session_id)
|
||||
.map(|entry| entry.expected_size.unwrap_or(entry.written))
|
||||
.sum();
|
||||
if session_bytes.saturating_add(p.byte_size) > MAX_TRANSFER_BYTES {
|
||||
return Err(invalid(format!(
|
||||
"session transfer staging exceeds limit {}",
|
||||
MAX_TRANSFER_BYTES
|
||||
)));
|
||||
}
|
||||
let id = new_id();
|
||||
let dir = self.root.join(&id);
|
||||
fs::create_dir(&dir).map_err(io_error)?;
|
||||
set_private_dir(&dir).map_err(io_error)?;
|
||||
let path = dir.join(basename);
|
||||
let file = OpenOptions::new()
|
||||
.write(true)
|
||||
.create_new(true)
|
||||
.open(&path)
|
||||
.map_err(io_error)?;
|
||||
set_private_file(&file).map_err(io_error)?;
|
||||
entries.insert(
|
||||
id.clone(),
|
||||
Entry {
|
||||
session_id: p.session_id,
|
||||
direction: Direction::Upload,
|
||||
path,
|
||||
expected_size: Some(p.byte_size),
|
||||
written: 0,
|
||||
ready: false,
|
||||
},
|
||||
);
|
||||
Ok(TransferBeginResult {
|
||||
transfer_id: id,
|
||||
chunk_size: TRANSFER_CHUNK_SIZE,
|
||||
})
|
||||
}
|
||||
|
||||
pub fn write_chunk(&self, p: TransferChunkParams) -> Result<TransferChunkResult, RpcError> {
|
||||
let bytes = base64::engine::general_purpose::STANDARD
|
||||
.decode(p.data_base64)
|
||||
.map_err(|e| invalid(format!("invalid transfer chunk: {e}")))?;
|
||||
if bytes.len() > TRANSFER_CHUNK_SIZE as usize {
|
||||
return Err(invalid("transfer chunk exceeds negotiated size"));
|
||||
}
|
||||
let mut entries = self.entries.lock().unwrap();
|
||||
let entry = entries
|
||||
.get_mut(&p.transfer_id)
|
||||
.ok_or_else(|| not_found("transfer not found"))?;
|
||||
if entry.direction != Direction::Upload || entry.ready {
|
||||
return Err(invalid("transfer is not writable"));
|
||||
}
|
||||
if p.offset != entry.written {
|
||||
return Err(invalid(format!(
|
||||
"non-sequential transfer offset: expected {}, got {}",
|
||||
entry.written, p.offset
|
||||
)));
|
||||
}
|
||||
let next = entry.written.saturating_add(bytes.len() as u64);
|
||||
if next > entry.expected_size.unwrap_or(MAX_TRANSFER_BYTES) {
|
||||
return Err(invalid("transfer exceeds declared byte size"));
|
||||
}
|
||||
let mut file = OpenOptions::new()
|
||||
.append(true)
|
||||
.open(&entry.path)
|
||||
.map_err(io_error)?;
|
||||
file.write_all(&bytes).map_err(io_error)?;
|
||||
entry.written = next;
|
||||
Ok(TransferChunkResult {
|
||||
next_offset: next,
|
||||
eof: false,
|
||||
data_base64: None,
|
||||
})
|
||||
}
|
||||
|
||||
pub fn finish_upload(&self, p: TransferIdParams) -> Result<TransferReadyResult, RpcError> {
|
||||
let mut entries = self.entries.lock().unwrap();
|
||||
let entry = entries
|
||||
.get_mut(&p.transfer_id)
|
||||
.ok_or_else(|| not_found("transfer not found"))?;
|
||||
if entry.direction != Direction::Upload {
|
||||
return Err(invalid("transfer is not an upload"));
|
||||
}
|
||||
if entry.written != entry.expected_size.unwrap_or(entry.written) {
|
||||
return Err(invalid(format!(
|
||||
"incomplete transfer: expected {} bytes, received {}",
|
||||
entry.expected_size.unwrap_or(0),
|
||||
entry.written
|
||||
)));
|
||||
}
|
||||
entry.ready = true;
|
||||
Ok(TransferReadyResult {
|
||||
transfer_id: p.transfer_id,
|
||||
byte_size: entry.written,
|
||||
})
|
||||
}
|
||||
|
||||
pub fn resolve_uploads(
|
||||
&self,
|
||||
session_id: &str,
|
||||
ids: &[String],
|
||||
) -> Result<Vec<PathBuf>, RpcError> {
|
||||
let entries = self.entries.lock().unwrap();
|
||||
ids.iter()
|
||||
.map(|id| {
|
||||
let entry = entries
|
||||
.get(id)
|
||||
.ok_or_else(|| not_found("upload transfer not found"))?;
|
||||
if entry.session_id != session_id
|
||||
|| entry.direction != Direction::Upload
|
||||
|| !entry.ready
|
||||
{
|
||||
return Err(permission(
|
||||
"upload transfer is outside this session or not ready",
|
||||
));
|
||||
}
|
||||
Ok(entry.path.clone())
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
pub fn begin_download(&self, session_id: &str) -> Result<DownloadStaging, RpcError> {
|
||||
self.ensure_root()
|
||||
.map_err(|err| protocol_error(err.to_string()))?;
|
||||
let id = new_id();
|
||||
let dir = self.root.join(&id);
|
||||
fs::create_dir(&dir).map_err(io_error)?;
|
||||
set_private_dir(&dir).map_err(io_error)?;
|
||||
self.entries.lock().unwrap().insert(
|
||||
id.clone(),
|
||||
Entry {
|
||||
session_id: session_id.to_string(),
|
||||
direction: Direction::Download,
|
||||
path: dir.clone(),
|
||||
expected_size: None,
|
||||
written: 0,
|
||||
ready: false,
|
||||
},
|
||||
);
|
||||
Ok(DownloadStaging {
|
||||
browser_relative_dir: format!("BrowserSkill/{id}"),
|
||||
transfer_id: id,
|
||||
})
|
||||
}
|
||||
|
||||
pub fn import_download(&self, id: &str, reported_path: &Path) -> Result<u64, RpcError> {
|
||||
let mut entries = self.entries.lock().unwrap();
|
||||
let entry = entries
|
||||
.get_mut(id)
|
||||
.ok_or_else(|| not_found("download transfer not found"))?;
|
||||
if entry.direction != Direction::Download {
|
||||
return Err(permission("transfer is not a download capability"));
|
||||
}
|
||||
let browser_file = ValidatedBrowserFile::validate(reported_path, id)?;
|
||||
let canonical = browser_file.path.canonicalize().map_err(io_error)?;
|
||||
let meta = fs::metadata(&canonical).map_err(io_error)?;
|
||||
if !meta.is_file() || meta.len() > MAX_TRANSFER_BYTES {
|
||||
return Err(invalid(
|
||||
"download is not a regular file or exceeds the transfer limit",
|
||||
));
|
||||
}
|
||||
|
||||
let destination = entry.path.join("payload");
|
||||
let source = File::open(&canonical).map_err(io_error)?;
|
||||
let mut destination_file = OpenOptions::new()
|
||||
.write(true)
|
||||
.create_new(true)
|
||||
.open(&destination)
|
||||
.map_err(io_error)?;
|
||||
let copied = std::io::copy(
|
||||
&mut source.take(MAX_TRANSFER_BYTES.saturating_add(1)),
|
||||
&mut destination_file,
|
||||
)
|
||||
.map_err(io_error)?;
|
||||
if copied > MAX_TRANSFER_BYTES {
|
||||
let _ = fs::remove_file(&destination);
|
||||
return Err(invalid("download exceeds the transfer limit"));
|
||||
}
|
||||
entry.path = destination;
|
||||
entry.written = copied;
|
||||
entry.expected_size = Some(copied);
|
||||
entry.ready = true;
|
||||
browser_file.remove_source()?;
|
||||
Ok(copied)
|
||||
}
|
||||
|
||||
pub fn read_chunk(&self, p: TransferChunkParams) -> Result<TransferChunkResult, RpcError> {
|
||||
let entries = self.entries.lock().unwrap();
|
||||
let entry = entries
|
||||
.get(&p.transfer_id)
|
||||
.ok_or_else(|| not_found("transfer not found"))?;
|
||||
if entry.direction != Direction::Download || !entry.ready {
|
||||
return Err(invalid("transfer is not readable"));
|
||||
}
|
||||
if p.offset > entry.written {
|
||||
return Err(invalid("read offset is beyond transfer length"));
|
||||
}
|
||||
let mut file = File::open(&entry.path).map_err(io_error)?;
|
||||
file.seek(SeekFrom::Start(p.offset)).map_err(io_error)?;
|
||||
let mut buf = vec![0u8; TRANSFER_CHUNK_SIZE as usize];
|
||||
let count = file.read(&mut buf).map_err(io_error)?;
|
||||
buf.truncate(count);
|
||||
let next = p.offset + count as u64;
|
||||
Ok(TransferChunkResult {
|
||||
next_offset: next,
|
||||
eof: next >= entry.written,
|
||||
data_base64: Some(base64::engine::general_purpose::STANDARD.encode(buf)),
|
||||
})
|
||||
}
|
||||
|
||||
pub fn release(&self, p: TransferIdParams) -> TransferReleaseResult {
|
||||
let entry = self.entries.lock().unwrap().remove(&p.transfer_id);
|
||||
if let Some(entry) = entry {
|
||||
let dir = if entry.path.is_dir() {
|
||||
entry.path
|
||||
} else {
|
||||
entry.path.parent().unwrap_or(&self.root).to_path_buf()
|
||||
};
|
||||
let _ = fs::remove_dir_all(dir);
|
||||
TransferReleaseResult { released: true }
|
||||
} else {
|
||||
TransferReleaseResult { released: false }
|
||||
}
|
||||
}
|
||||
|
||||
pub fn release_session(&self, session_id: &str) {
|
||||
let ids: Vec<String> = self
|
||||
.entries
|
||||
.lock()
|
||||
.unwrap()
|
||||
.iter()
|
||||
.filter(|(_, entry)| entry.session_id == session_id)
|
||||
.map(|(id, _)| id.clone())
|
||||
.collect();
|
||||
for id in ids {
|
||||
self.release(TransferIdParams { transfer_id: id });
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn new_id() -> String {
|
||||
format!("tr_{}", Uuid::new_v4().simple())
|
||||
}
|
||||
|
||||
fn set_private_dir(path: &Path) -> std::io::Result<()> {
|
||||
#[cfg(unix)]
|
||||
{
|
||||
use std::os::unix::fs::PermissionsExt;
|
||||
fs::set_permissions(path, fs::Permissions::from_mode(0o700))?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn set_private_file(file: &File) -> std::io::Result<()> {
|
||||
#[cfg(unix)]
|
||||
{
|
||||
use std::os::unix::fs::PermissionsExt;
|
||||
file.set_permissions(fs::Permissions::from_mode(0o600))?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn remove_if_present(path: &Path) -> std::io::Result<()> {
|
||||
match fs::remove_file(path) {
|
||||
Ok(()) => Ok(()),
|
||||
Err(err) if err.kind() == std::io::ErrorKind::NotFound => Ok(()),
|
||||
Err(err) => Err(err),
|
||||
}
|
||||
}
|
||||
|
||||
fn remove_dir_if_present(path: &Path) -> std::io::Result<()> {
|
||||
match fs::remove_dir(path) {
|
||||
Ok(()) => Ok(()),
|
||||
Err(err) if err.kind() == std::io::ErrorKind::NotFound => Ok(()),
|
||||
Err(err) => Err(err),
|
||||
}
|
||||
}
|
||||
|
||||
fn safe_upload_basename(name: &str) -> Result<&str, RpcError> {
|
||||
if name.is_empty() || name.contains(['/', '\\', '\0']) {
|
||||
return Err(invalid("upload name must be a safe basename"));
|
||||
}
|
||||
let mut components = Path::new(name).components();
|
||||
match (components.next(), components.next()) {
|
||||
(Some(Component::Normal(_)), None) => Ok(name),
|
||||
_ => Err(invalid("upload name must be a safe basename")),
|
||||
}
|
||||
}
|
||||
|
||||
fn invalid(message: impl Into<String>) -> RpcError {
|
||||
RpcError {
|
||||
code: ErrorCode::InvalidParams,
|
||||
message: message.into(),
|
||||
data: None,
|
||||
}
|
||||
}
|
||||
fn not_found(message: impl Into<String>) -> RpcError {
|
||||
RpcError {
|
||||
code: ErrorCode::NotFound,
|
||||
message: message.into(),
|
||||
data: None,
|
||||
}
|
||||
}
|
||||
fn permission(message: impl Into<String>) -> RpcError {
|
||||
RpcError {
|
||||
code: ErrorCode::PermissionDenied,
|
||||
message: message.into(),
|
||||
data: None,
|
||||
}
|
||||
}
|
||||
fn io_error(err: std::io::Error) -> RpcError {
|
||||
RpcError {
|
||||
code: ErrorCode::ProtocolError,
|
||||
message: err.to_string(),
|
||||
data: None,
|
||||
}
|
||||
}
|
||||
|
||||
fn protocol_error(message: impl Into<String>) -> RpcError {
|
||||
RpcError {
|
||||
code: ErrorCode::ProtocolError,
|
||||
message: message.into(),
|
||||
data: None,
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn registry() -> (tempfile::TempDir, TransferRegistry) {
|
||||
let temp = tempfile::tempdir().unwrap();
|
||||
let registry = TransferRegistry::at_root(temp.path().join("transfers")).unwrap();
|
||||
(temp, registry)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn upload_is_not_resolvable_until_complete_or_from_another_session() {
|
||||
let (_temp, registry) = registry();
|
||||
let begin = registry
|
||||
.begin_upload(TransferBeginParams {
|
||||
session_id: "s1".into(),
|
||||
name: "image.png".into(),
|
||||
byte_size: 3,
|
||||
})
|
||||
.unwrap();
|
||||
assert!(
|
||||
registry
|
||||
.resolve_uploads("s1", std::slice::from_ref(&begin.transfer_id))
|
||||
.is_err()
|
||||
);
|
||||
registry
|
||||
.write_chunk(TransferChunkParams {
|
||||
transfer_id: begin.transfer_id.clone(),
|
||||
offset: 0,
|
||||
data_base64: base64::engine::general_purpose::STANDARD.encode(b"abc"),
|
||||
})
|
||||
.unwrap();
|
||||
registry
|
||||
.finish_upload(TransferIdParams {
|
||||
transfer_id: begin.transfer_id.clone(),
|
||||
})
|
||||
.unwrap();
|
||||
assert!(
|
||||
registry
|
||||
.resolve_uploads("s2", std::slice::from_ref(&begin.transfer_id))
|
||||
.is_err()
|
||||
);
|
||||
let [path] = registry
|
||||
.resolve_uploads("s1", &[begin.transfer_id])
|
||||
.unwrap()
|
||||
.try_into()
|
||||
.unwrap();
|
||||
assert_eq!(path.file_name().unwrap(), "image.png");
|
||||
assert_eq!(fs::read(path).unwrap(), b"abc");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn upload_name_must_be_one_safe_path_component() {
|
||||
let (_temp, registry) = registry();
|
||||
for name in ["", ".", "..", "../secret", "folder/file", "folder\\file"] {
|
||||
let result = registry.begin_upload(TransferBeginParams {
|
||||
session_id: "s1".into(),
|
||||
name: name.into(),
|
||||
byte_size: 0,
|
||||
});
|
||||
assert!(result.is_err(), "accepted unsafe upload name");
|
||||
}
|
||||
assert!(registry.entries.lock().unwrap().is_empty());
|
||||
assert!(fs::read_dir(®istry.root).unwrap().next().is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn same_upload_names_are_isolated_by_transfer_directory() {
|
||||
let (_temp, registry) = registry();
|
||||
let first = registry
|
||||
.begin_upload(TransferBeginParams {
|
||||
session_id: "s1".into(),
|
||||
name: "image.png".into(),
|
||||
byte_size: 0,
|
||||
})
|
||||
.unwrap();
|
||||
let second = registry
|
||||
.begin_upload(TransferBeginParams {
|
||||
session_id: "s1".into(),
|
||||
name: "image.png".into(),
|
||||
byte_size: 0,
|
||||
})
|
||||
.unwrap();
|
||||
let entries = registry.entries.lock().unwrap();
|
||||
let first_path = &entries[&first.transfer_id].path;
|
||||
let second_path = &entries[&second.transfer_id].path;
|
||||
assert_ne!(first_path.parent(), second_path.parent());
|
||||
assert_eq!(first_path.file_name().unwrap(), "image.png");
|
||||
assert_eq!(second_path.file_name().unwrap(), "image.png");
|
||||
}
|
||||
|
||||
#[cfg(unix)]
|
||||
#[test]
|
||||
fn upload_staging_is_private() {
|
||||
use std::os::unix::fs::PermissionsExt;
|
||||
|
||||
let (_temp, registry) = registry();
|
||||
let begin = registry
|
||||
.begin_upload(TransferBeginParams {
|
||||
session_id: "s1".into(),
|
||||
name: "private.png".into(),
|
||||
byte_size: 0,
|
||||
})
|
||||
.unwrap();
|
||||
let entries = registry.entries.lock().unwrap();
|
||||
let path = &entries[&begin.transfer_id].path;
|
||||
assert_eq!(
|
||||
fs::metadata(path.parent().unwrap())
|
||||
.unwrap()
|
||||
.permissions()
|
||||
.mode()
|
||||
& 0o777,
|
||||
0o700
|
||||
);
|
||||
assert_eq!(
|
||||
fs::metadata(path).unwrap().permissions().mode() & 0o777,
|
||||
0o600
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn download_must_import_from_its_browser_capability_directory() {
|
||||
let (temp, registry) = registry();
|
||||
let staging = registry.begin_download("s1").unwrap();
|
||||
let outside = temp.path().join("outside.bin");
|
||||
fs::write(&outside, b"secret").unwrap();
|
||||
assert!(
|
||||
registry
|
||||
.import_download(&staging.transfer_id, &outside)
|
||||
.is_err()
|
||||
);
|
||||
assert_eq!(fs::read(&outside).unwrap(), b"secret");
|
||||
|
||||
let browser_dir = temp
|
||||
.path()
|
||||
.join("Downloads")
|
||||
.join("BrowserSkill")
|
||||
.join(&staging.transfer_id);
|
||||
fs::create_dir_all(&browser_dir).unwrap();
|
||||
let inside = browser_dir.join("result.bin");
|
||||
fs::write(&inside, b"result").unwrap();
|
||||
assert_eq!(
|
||||
registry
|
||||
.import_download(&staging.transfer_id, &inside)
|
||||
.unwrap(),
|
||||
6
|
||||
);
|
||||
assert!(!inside.exists());
|
||||
let chunk = registry
|
||||
.read_chunk(TransferChunkParams {
|
||||
transfer_id: staging.transfer_id,
|
||||
offset: 0,
|
||||
data_base64: String::new(),
|
||||
})
|
||||
.unwrap();
|
||||
assert_eq!(
|
||||
base64::engine::general_purpose::STANDARD
|
||||
.decode(chunk.data_base64.unwrap())
|
||||
.unwrap(),
|
||||
b"result"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn validated_oversized_browser_download_is_removed_on_rejection() {
|
||||
let (temp, registry) = registry();
|
||||
let staging = registry.begin_download("s1").unwrap();
|
||||
let browser_dir = temp
|
||||
.path()
|
||||
.join("Downloads")
|
||||
.join("BrowserSkill")
|
||||
.join(&staging.transfer_id);
|
||||
fs::create_dir_all(&browser_dir).unwrap();
|
||||
let inside = browser_dir.join("oversized.bin");
|
||||
File::create(&inside)
|
||||
.unwrap()
|
||||
.set_len(MAX_TRANSFER_BYTES + 1)
|
||||
.unwrap();
|
||||
|
||||
assert!(
|
||||
registry
|
||||
.import_download(&staging.transfer_id, &inside)
|
||||
.is_err()
|
||||
);
|
||||
assert!(!inside.exists());
|
||||
assert!(!browser_dir.exists());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn releasing_a_session_removes_all_staging() {
|
||||
let (_temp, registry) = registry();
|
||||
let first = registry.begin_download("s1").unwrap();
|
||||
let second = registry.begin_download("s2").unwrap();
|
||||
let first_directory = registry.root.join(&first.transfer_id);
|
||||
let second_directory = registry.root.join(&second.transfer_id);
|
||||
registry.release_session("s1");
|
||||
assert!(!first_directory.exists());
|
||||
assert!(second_directory.exists());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn upload_staging_is_bounded_across_a_session() {
|
||||
let (_temp, registry) = registry();
|
||||
registry
|
||||
.begin_upload(TransferBeginParams {
|
||||
session_id: "s1".into(),
|
||||
name: "large.bin".into(),
|
||||
byte_size: MAX_TRANSFER_BYTES,
|
||||
})
|
||||
.unwrap();
|
||||
assert!(
|
||||
registry
|
||||
.begin_upload(TransferBeginParams {
|
||||
session_id: "s1".into(),
|
||||
name: "one-more-byte.bin".into(),
|
||||
byte_size: 1,
|
||||
})
|
||||
.is_err()
|
||||
);
|
||||
assert!(
|
||||
registry
|
||||
.begin_upload(TransferBeginParams {
|
||||
session_id: "s2".into(),
|
||||
name: "other-session.bin".into(),
|
||||
byte_size: 1,
|
||||
})
|
||||
.is_ok()
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -27,7 +27,8 @@ use bsk_protocol::system::{
|
||||
VersionSkewEntry,
|
||||
};
|
||||
use bsk_protocol::tools::{
|
||||
HelpOutcome, RequestHelpResult, ReturnFailure, WaitMsParams, WaitMsResult,
|
||||
DownloadParams, DownloadResult, HelpOutcome, RequestHelpResult, ReturnFailure,
|
||||
TransferBeginParams, TransferIdParams, UploadParams, WaitMsParams, WaitMsResult,
|
||||
};
|
||||
use bsk_protocol::{
|
||||
CancelParams, CancelResult, ErrorCode, Method, PingResult, ResponseBody, RpcError, RpcId,
|
||||
@@ -61,6 +62,10 @@ pub type RpcHandler = Arc<
|
||||
>;
|
||||
|
||||
const DEFAULT_RPC_TIMEOUT: Duration = Duration::from_secs(15);
|
||||
// The upload transaction owns its operation deadline and may need a bounded
|
||||
// cleanup before it can return a useful structured error. Keep only that
|
||||
// transport alive slightly longer so it does not replace the result.
|
||||
const EXTENSION_RESPONSE_GRACE: Duration = Duration::from_secs(2);
|
||||
/// Upper bound on `wait_for_browser_ms` accepted over IPC.
|
||||
const MAX_BROWSER_WAIT: Duration = Duration::from_secs(60);
|
||||
// `session.stop` fast-fails while another tool is active; this budget
|
||||
@@ -223,6 +228,11 @@ pub fn full_handler(status: DaemonStatus, state: Arc<DaemonState>) -> RpcHandler
|
||||
Ok(v) => ResponseBody::Ok(v),
|
||||
Err(e) => ResponseBody::Err(e),
|
||||
},
|
||||
Method::TransferBegin => handle_transfer_begin(&state, params),
|
||||
Method::TransferChunk => handle_transfer_chunk(&state, params),
|
||||
Method::TransferFinish => handle_transfer_finish(&state, params),
|
||||
Method::TransferRead => handle_transfer_read(&state, params),
|
||||
Method::TransferRelease => handle_transfer_release(&state, params),
|
||||
Method::ToolTabList
|
||||
| Method::ToolTabCreate
|
||||
| Method::ToolTabClose
|
||||
@@ -246,6 +256,8 @@ pub fn full_handler(status: DaemonStatus, state: Arc<DaemonState>) -> RpcHandler
|
||||
| Method::ToolFill
|
||||
| Method::ToolPress
|
||||
| Method::ToolSelect
|
||||
| Method::ToolUpload
|
||||
| Method::ToolDownload
|
||||
| Method::ToolEvaluate
|
||||
| Method::ToolWaitForNavigation
|
||||
| Method::ToolRequestHelp
|
||||
@@ -324,7 +336,7 @@ async fn handle_tool_dispatch(
|
||||
data: None,
|
||||
});
|
||||
}
|
||||
let timeout = match tool_dispatch_timeout(¶ms) {
|
||||
let timeout = match tool_dispatch_transport_timeout(&method, ¶ms) {
|
||||
Ok(timeout) => timeout,
|
||||
Err(err) => return ResponseBody::Err(err),
|
||||
};
|
||||
@@ -338,24 +350,180 @@ async fn handle_tool_dispatch(
|
||||
});
|
||||
}
|
||||
};
|
||||
let mut params = params;
|
||||
let mut download_transfer_id: Option<String> = None;
|
||||
if method == Method::ToolUpload {
|
||||
let mut upload: UploadParams = match serde_json::from_value(params) {
|
||||
Ok(v) => v,
|
||||
Err(err) => return ResponseBody::Err(invalid_params(err.to_string())),
|
||||
};
|
||||
if upload.files.is_empty() || upload.files.len() > super::file_transfer::MAX_UPLOAD_FILES {
|
||||
return ResponseBody::Err(invalid_params(format!(
|
||||
"upload requires 1..={} files",
|
||||
super::file_transfer::MAX_UPLOAD_FILES
|
||||
)));
|
||||
}
|
||||
let ids: Vec<String> = upload.files.iter().map(|f| f.transfer_id.clone()).collect();
|
||||
let paths = match state.transfers.resolve_uploads(&session_id.0, &ids) {
|
||||
Ok(v) => v,
|
||||
Err(err) => return ResponseBody::Err(err),
|
||||
};
|
||||
for (file, path) in upload.files.iter_mut().zip(paths) {
|
||||
file.staged_path = Some(path.to_string_lossy().into_owned());
|
||||
}
|
||||
params = serde_json::to_value(upload).unwrap_or(Value::Null);
|
||||
} else if method == Method::ToolDownload {
|
||||
let mut download: DownloadParams = match serde_json::from_value(params) {
|
||||
Ok(v) => v,
|
||||
Err(err) => return ResponseBody::Err(invalid_params(err.to_string())),
|
||||
};
|
||||
let staging = match state.transfers.begin_download(&session_id.0) {
|
||||
Ok(v) => v,
|
||||
Err(err) => return ResponseBody::Err(err),
|
||||
};
|
||||
download.browser_relative_dir = Some(staging.browser_relative_dir);
|
||||
download.max_byte_size = Some(super::file_transfer::MAX_TRANSFER_BYTES);
|
||||
download_transfer_id = Some(staging.transfer_id);
|
||||
params = serde_json::to_value(download).unwrap_or(Value::Null);
|
||||
}
|
||||
let entry = inflight_guard.entry();
|
||||
// `record_stop` must reach the extension while `record_await` holds the
|
||||
// serial busy lock — finishing the recording unblocks await.
|
||||
let outcome = if method == Method::ToolRecordStop {
|
||||
state
|
||||
.tool_queues
|
||||
.dispatch_unlocked(&session_id, method, params, timeout, Some(entry))
|
||||
.dispatch_unlocked(&session_id, method.clone(), params, timeout, Some(entry))
|
||||
.await
|
||||
} else {
|
||||
state
|
||||
.tool_queues
|
||||
.dispatch(&session_id, method, params, timeout, Some(entry))
|
||||
.dispatch(&session_id, method.clone(), params, timeout, Some(entry))
|
||||
.await
|
||||
};
|
||||
drop(inflight_guard);
|
||||
match outcome {
|
||||
Ok(v) if method == Method::ToolDownload => {
|
||||
let id = download_transfer_id.expect("download transfer allocated");
|
||||
let mut result: DownloadResult = match serde_json::from_value(v) {
|
||||
Ok(v) => v,
|
||||
Err(err) => {
|
||||
state
|
||||
.transfers
|
||||
.release(TransferIdParams { transfer_id: id });
|
||||
return ResponseBody::Err(RpcError {
|
||||
code: ErrorCode::ProtocolError,
|
||||
message: format!("invalid tool.download result: {err}"),
|
||||
data: None,
|
||||
});
|
||||
}
|
||||
};
|
||||
let Some(path) = result.browser_path.take() else {
|
||||
state
|
||||
.transfers
|
||||
.release(TransferIdParams { transfer_id: id });
|
||||
return ResponseBody::Err(RpcError {
|
||||
code: ErrorCode::ProtocolError,
|
||||
message: "tool.download returned no browser_path".into(),
|
||||
data: None,
|
||||
});
|
||||
};
|
||||
match state
|
||||
.transfers
|
||||
.import_download(&id, std::path::Path::new(&path))
|
||||
{
|
||||
Ok(size) => {
|
||||
result.byte_size = size;
|
||||
result.transfer_id = Some(id);
|
||||
ResponseBody::Ok(serde_json::to_value(result).unwrap_or(Value::Null))
|
||||
}
|
||||
Err(err) => {
|
||||
state
|
||||
.transfers
|
||||
.release(TransferIdParams { transfer_id: id });
|
||||
ResponseBody::Err(err)
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(v) => ResponseBody::Ok(v),
|
||||
Err(err) => ResponseBody::Err(err.into_rpc()),
|
||||
Err(err) => {
|
||||
if let Some(id) = download_transfer_id {
|
||||
state
|
||||
.transfers
|
||||
.release(TransferIdParams { transfer_id: id });
|
||||
}
|
||||
ResponseBody::Err(err.into_rpc())
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn handle_transfer_begin(state: &Arc<DaemonState>, params: Value) -> ResponseBody {
|
||||
let p: TransferBeginParams = match serde_json::from_value(params) {
|
||||
Ok(v) => v,
|
||||
Err(err) => return ResponseBody::Err(invalid_params(err.to_string())),
|
||||
};
|
||||
if state
|
||||
.sessions
|
||||
.get(&SessionId(p.session_id.clone()))
|
||||
.is_none()
|
||||
{
|
||||
return ResponseBody::Err(RpcError {
|
||||
code: ErrorCode::NotFound,
|
||||
message: format!("session {} unknown", p.session_id),
|
||||
data: None,
|
||||
});
|
||||
}
|
||||
match state.transfers.begin_upload(p) {
|
||||
Ok(v) => ResponseBody::Ok(serde_json::to_value(v).unwrap_or(Value::Null)),
|
||||
Err(err) => ResponseBody::Err(err),
|
||||
}
|
||||
}
|
||||
|
||||
fn handle_transfer_chunk(state: &Arc<DaemonState>, params: Value) -> ResponseBody {
|
||||
let p = match serde_json::from_value(params) {
|
||||
Ok(v) => v,
|
||||
Err(err) => return ResponseBody::Err(invalid_params(err.to_string())),
|
||||
};
|
||||
match state.transfers.write_chunk(p) {
|
||||
Ok(v) => ResponseBody::Ok(serde_json::to_value(v).unwrap_or(Value::Null)),
|
||||
Err(err) => ResponseBody::Err(err),
|
||||
}
|
||||
}
|
||||
|
||||
fn handle_transfer_finish(state: &Arc<DaemonState>, params: Value) -> ResponseBody {
|
||||
let p = match serde_json::from_value(params) {
|
||||
Ok(v) => v,
|
||||
Err(err) => return ResponseBody::Err(invalid_params(err.to_string())),
|
||||
};
|
||||
match state.transfers.finish_upload(p) {
|
||||
Ok(v) => ResponseBody::Ok(serde_json::to_value(v).unwrap_or(Value::Null)),
|
||||
Err(err) => ResponseBody::Err(err),
|
||||
}
|
||||
}
|
||||
|
||||
fn handle_transfer_read(state: &Arc<DaemonState>, params: Value) -> ResponseBody {
|
||||
let p = match serde_json::from_value(params) {
|
||||
Ok(v) => v,
|
||||
Err(err) => return ResponseBody::Err(invalid_params(err.to_string())),
|
||||
};
|
||||
match state.transfers.read_chunk(p) {
|
||||
Ok(v) => ResponseBody::Ok(serde_json::to_value(v).unwrap_or(Value::Null)),
|
||||
Err(err) => ResponseBody::Err(err),
|
||||
}
|
||||
}
|
||||
|
||||
fn handle_transfer_release(state: &Arc<DaemonState>, params: Value) -> ResponseBody {
|
||||
let p: TransferIdParams = match serde_json::from_value(params) {
|
||||
Ok(v) => v,
|
||||
Err(err) => return ResponseBody::Err(invalid_params(err.to_string())),
|
||||
};
|
||||
ResponseBody::Ok(serde_json::to_value(state.transfers.release(p)).unwrap_or(Value::Null))
|
||||
}
|
||||
|
||||
fn invalid_params(message: impl Into<String>) -> RpcError {
|
||||
RpcError {
|
||||
code: ErrorCode::InvalidParams,
|
||||
message: message.into(),
|
||||
data: None,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -533,6 +701,16 @@ fn tool_dispatch_timeout(params: &Value) -> Result<Duration, RpcError> {
|
||||
Ok(Duration::from_millis(u64::from(ms)))
|
||||
}
|
||||
|
||||
fn tool_dispatch_transport_timeout(method: &Method, params: &Value) -> Result<Duration, RpcError> {
|
||||
tool_dispatch_timeout(params).map(|timeout| {
|
||||
if matches!(method, Method::ToolUpload | Method::ToolDownload) {
|
||||
timeout.saturating_add(EXTENSION_RESPONSE_GRACE)
|
||||
} else {
|
||||
timeout
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
// Local CLI-facing shapes. Intentionally distinct from
|
||||
// `bsk_protocol::tools::SessionStart*` (which describes the WS-facing
|
||||
// `tool.session_*` round-trip with the extension) so the two sides can
|
||||
@@ -798,6 +976,7 @@ async fn handle_session_stop(
|
||||
.await
|
||||
{
|
||||
Ok(stop) => {
|
||||
state.transfers.release_session(&session_id.0);
|
||||
let result = CliSessionStopResult {
|
||||
stopped: vec![session_id.0],
|
||||
failed: Vec::new(),
|
||||
@@ -897,6 +1076,7 @@ async fn handle_session_stop_all(
|
||||
.await
|
||||
{
|
||||
Ok(stop) => {
|
||||
state.transfers.release_session(&id.0);
|
||||
stopped.push(id.0);
|
||||
returned_tab_ids.extend(stop.returned_tab_ids);
|
||||
return_failures.extend(stop.return_failures);
|
||||
@@ -1329,6 +1509,30 @@ mod tests {
|
||||
assert_eq!(got, std::time::Duration::from_millis(300_000));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn extension_transport_outlives_the_operation_deadline() {
|
||||
let params = serde_json::json!({
|
||||
"session_id": "abcd",
|
||||
"timeout_ms": 60_000,
|
||||
});
|
||||
let dispatch_timeout =
|
||||
tool_dispatch_transport_timeout(&Method::ToolUpload, ¶ms).unwrap();
|
||||
|
||||
assert_eq!(dispatch_timeout, Duration::from_secs(62));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn extension_response_grace_does_not_change_other_tools() {
|
||||
let params = serde_json::json!({
|
||||
"session_id": "abcd",
|
||||
"timeout_ms": 60_000,
|
||||
});
|
||||
let dispatch_timeout =
|
||||
tool_dispatch_transport_timeout(&Method::ToolClick, ¶ms).unwrap();
|
||||
|
||||
assert_eq!(dispatch_timeout, Duration::from_secs(60));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn tool_dispatch_timeout_rejects_zero_timeout_ms() {
|
||||
let params = serde_json::json!({
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
pub mod abort;
|
||||
pub mod browsers;
|
||||
mod cancel_forward;
|
||||
pub mod file_transfer;
|
||||
pub mod inflight;
|
||||
pub mod info;
|
||||
pub mod ipc;
|
||||
|
||||
@@ -438,6 +438,7 @@ async fn dispatch_with_sender(
|
||||
lifecycle_cancel: Option<AbortToken>,
|
||||
cancel_cleanup_timeout: Duration,
|
||||
) -> Result<Value, DispatchError> {
|
||||
let effect_aware_transfer = is_effect_aware_transfer(&method);
|
||||
let (respond_tx, respond_rx) = oneshot::channel();
|
||||
let job = ToolJob {
|
||||
method,
|
||||
@@ -474,15 +475,21 @@ async fn dispatch_with_sender(
|
||||
mpsc::error::TrySendError::Closed(_) => DispatchError::QueueClosed,
|
||||
});
|
||||
}
|
||||
let mut response_timeout = timeout.saturating_add(Duration::from_secs(1));
|
||||
if lifecycle_cancellable {
|
||||
// A lifecycle cancel can arrive at the original RPC deadline and
|
||||
// intentionally keeps the worker alive while extension cleanup
|
||||
// settles. Do not let this outer waiter abandon that reconciliation
|
||||
// early, which could reopen a daemon session whose window is gone.
|
||||
response_timeout = response_timeout.saturating_add(cancel_cleanup_timeout);
|
||||
}
|
||||
let waited = tokio::time::timeout(response_timeout, respond_rx).await;
|
||||
// Lifecycle teardown and effect-aware transfers both keep the worker busy
|
||||
// for bounded compensation after their original deadline. Keep the outer
|
||||
// waiter alive for the same grace period so it cannot abandon reconciliation.
|
||||
let response_grace = if lifecycle_cancellable || effect_aware_transfer {
|
||||
cancel_cleanup_timeout
|
||||
} else {
|
||||
Duration::ZERO
|
||||
};
|
||||
let waited = tokio::time::timeout(
|
||||
timeout
|
||||
.saturating_add(response_grace)
|
||||
.saturating_add(Duration::from_secs(1)),
|
||||
respond_rx,
|
||||
)
|
||||
.await;
|
||||
match waited {
|
||||
Ok(Ok(Ok(v))) => Ok(v),
|
||||
Ok(Ok(Err(rpc))) => Err(DispatchError::Rpc(rpc)),
|
||||
@@ -636,44 +643,85 @@ async fn forward_one(
|
||||
.lifecycle_cancel
|
||||
.as_ref()
|
||||
.map(|_| &forward_lifecycle_cancel as &(dyn Fn() + Sync));
|
||||
let send_deadline_cancel = || {
|
||||
client
|
||||
.sink
|
||||
.send(Frame::Request(RequestFrame {
|
||||
id: format!("deadline-cancel-{rpc_id}"),
|
||||
method: Method::Cancel,
|
||||
params: Some(serde_json::json!({ "rpc_id": rpc_id })),
|
||||
}))
|
||||
.is_ok()
|
||||
};
|
||||
let deadline_cancel: Option<&(dyn Fn() -> bool + Sync)> =
|
||||
is_effect_aware_transfer(&job.method).then_some(&send_deadline_cancel);
|
||||
let waited = await_with_optional_cancel(
|
||||
job.timeout,
|
||||
job.cancel_cleanup_timeout,
|
||||
waiter,
|
||||
cancel_token.as_ref(),
|
||||
on_abort,
|
||||
deadline_cancel,
|
||||
)
|
||||
.await;
|
||||
let response = match waited {
|
||||
WaitOutcome::Response(resp) => resp,
|
||||
WaitOutcome::CancelledAfterResponse(resp) => {
|
||||
if job.method == Method::ToolSessionStop
|
||||
&& let ResponseBody::Ok(value) = resp.body
|
||||
&& let ResponseBody::Ok(value) = &resp.body
|
||||
{
|
||||
// Teardown crossed its final irreversible boundary before
|
||||
// the cancel landed. Commit the real extension result so the
|
||||
// daemon does not retain a session whose window is gone.
|
||||
return Ok(value);
|
||||
return Ok(value.clone());
|
||||
}
|
||||
// Cancellation wins the external verdict, but only after the
|
||||
// extension's original RPC has settled. Preserve any non-cancel
|
||||
// error so compensation failures remain explicit instead of
|
||||
// being hidden behind a generic cancelled result.
|
||||
if let ResponseBody::Err(err) = resp.body
|
||||
&& !matches!(err.code, ErrorCode::Cancelled | ErrorCode::UserAborted)
|
||||
{
|
||||
return Err(err);
|
||||
// File transfer commits are irreversible. A late cancel cannot
|
||||
// overwrite a confirmed success, and an unknown transfer effect
|
||||
// must remain explicit so callers do not retry or release upload
|
||||
// staging as though nothing happened.
|
||||
if is_effect_aware_transfer(&job.method) {
|
||||
match &resp.body {
|
||||
ResponseBody::Ok(_) => resp,
|
||||
ResponseBody::Err(err)
|
||||
if transfer_effect(err) == Some("unknown")
|
||||
|| transfer_effect(err) == Some("committed") =>
|
||||
{
|
||||
return Err(err.clone());
|
||||
}
|
||||
ResponseBody::Err(err)
|
||||
if !matches!(err.code, ErrorCode::Cancelled | ErrorCode::UserAborted) =>
|
||||
{
|
||||
return Err(err.clone());
|
||||
}
|
||||
ResponseBody::Err(_) => {
|
||||
return Err(cancelled_error(
|
||||
job.inflight.as_deref(),
|
||||
"tool dispatch cancelled after extension cleanup",
|
||||
));
|
||||
}
|
||||
}
|
||||
} else {
|
||||
// For ordinary tools cancellation keeps the existing verdict,
|
||||
// while non-cancel errors still expose compensation failures.
|
||||
if let ResponseBody::Err(err) = resp.body
|
||||
&& !matches!(err.code, ErrorCode::Cancelled | ErrorCode::UserAborted)
|
||||
{
|
||||
return Err(err);
|
||||
}
|
||||
return Err(cancelled_error(
|
||||
job.inflight.as_deref(),
|
||||
"tool dispatch cancelled after extension cleanup",
|
||||
));
|
||||
}
|
||||
return Err(cancelled_error(
|
||||
job.inflight.as_deref(),
|
||||
"tool dispatch cancelled after extension cleanup",
|
||||
));
|
||||
}
|
||||
WaitOutcome::TimedOutAfterResponse(resp) => match resp.body {
|
||||
ResponseBody::Ok(value) if job.method == Method::ToolSessionStop => {
|
||||
ResponseBody::Ok(value)
|
||||
if job.method == Method::ToolSessionStop
|
||||
|| is_effect_aware_transfer(&job.method) =>
|
||||
{
|
||||
// The close crossed its irreversible boundary during the
|
||||
// timeout cleanup grace. Reconcile daemon state instead of
|
||||
// reopening a session whose window is already gone.
|
||||
// timeout cleanup grace, or the transfer committed before its
|
||||
// deadline cancel settled. Preserve the irreversible result.
|
||||
return Ok(value);
|
||||
}
|
||||
ResponseBody::Err(err)
|
||||
@@ -681,6 +729,9 @@ async fn forward_one(
|
||||
{
|
||||
return Err(err);
|
||||
}
|
||||
ResponseBody::Err(err) if is_effect_aware_transfer(&job.method) => {
|
||||
return Err(timed_out_transfer_error(&err));
|
||||
}
|
||||
_ => {
|
||||
return Err(RpcError {
|
||||
code: ErrorCode::Timeout,
|
||||
@@ -691,6 +742,14 @@ async fn forward_one(
|
||||
},
|
||||
WaitOutcome::CleanupTimeout => {
|
||||
client.pending.lock().unwrap().cancel(&rpc_id);
|
||||
if is_effect_aware_transfer(&job.method) {
|
||||
return Err(unknown_transfer_error(
|
||||
ErrorCode::Timeout,
|
||||
"cancelled file transfer did not confirm its outcome before cleanup timed out",
|
||||
"cleanup",
|
||||
true,
|
||||
));
|
||||
}
|
||||
return Err(RpcError {
|
||||
code: ErrorCode::Timeout,
|
||||
message: format!(
|
||||
@@ -700,8 +759,25 @@ async fn forward_one(
|
||||
data: Some(serde_json::json!({ "reason": "cancel_cleanup_timeout" })),
|
||||
});
|
||||
}
|
||||
WaitOutcome::TimeoutCleanupFailed => {
|
||||
client.pending.lock().unwrap().cancel(&rpc_id);
|
||||
return Err(unknown_transfer_error(
|
||||
ErrorCode::Timeout,
|
||||
"file transfer timed out and cleanup could not be confirmed",
|
||||
"cleanup",
|
||||
true,
|
||||
));
|
||||
}
|
||||
WaitOutcome::WaiterClosed => {
|
||||
client.pending.lock().unwrap().cancel(&rpc_id);
|
||||
if is_effect_aware_transfer(&job.method) {
|
||||
return Err(unknown_transfer_error(
|
||||
ErrorCode::ProtocolError,
|
||||
"file transfer transport closed after dispatch; outcome is unknown",
|
||||
"transport",
|
||||
false,
|
||||
));
|
||||
}
|
||||
return Err(RpcError {
|
||||
code: ErrorCode::ProtocolError,
|
||||
message: "transport closed mid-call".into(),
|
||||
@@ -710,6 +786,14 @@ async fn forward_one(
|
||||
}
|
||||
WaitOutcome::Timeout => {
|
||||
client.pending.lock().unwrap().cancel(&rpc_id);
|
||||
if is_effect_aware_transfer(&job.method) {
|
||||
return Err(unknown_transfer_error(
|
||||
ErrorCode::Timeout,
|
||||
"file transfer timed out after dispatch; outcome is unknown",
|
||||
"transport",
|
||||
false,
|
||||
));
|
||||
}
|
||||
return Err(RpcError {
|
||||
code: ErrorCode::Timeout,
|
||||
message: format!("tool RPC timed out after {:?}", job.timeout),
|
||||
@@ -723,12 +807,65 @@ async fn forward_one(
|
||||
}
|
||||
}
|
||||
|
||||
fn is_effect_aware_transfer(method: &Method) -> bool {
|
||||
matches!(method, Method::ToolUpload | Method::ToolDownload)
|
||||
}
|
||||
|
||||
fn transfer_effect(err: &RpcError) -> Option<&str> {
|
||||
err.data.as_ref()?.get("effect_state")?.as_str()
|
||||
}
|
||||
|
||||
fn unknown_transfer_error(
|
||||
code: ErrorCode,
|
||||
message: impl Into<String>,
|
||||
phase: &str,
|
||||
cleanup_failed: bool,
|
||||
) -> RpcError {
|
||||
let mut data = serde_json::json!({
|
||||
"reason": "transfer_outcome_unknown",
|
||||
"effect_state": "unknown",
|
||||
"phase": phase,
|
||||
});
|
||||
if cleanup_failed {
|
||||
data["cleanup_state"] = serde_json::json!("failed");
|
||||
}
|
||||
RpcError {
|
||||
code,
|
||||
message: message.into(),
|
||||
data: Some(data),
|
||||
}
|
||||
}
|
||||
|
||||
fn timed_out_transfer_error(err: &RpcError) -> RpcError {
|
||||
let data = err.data.as_ref();
|
||||
RpcError {
|
||||
code: ErrorCode::Timeout,
|
||||
message: "file transfer timed out after dispatch".into(),
|
||||
data: Some(serde_json::json!({
|
||||
"reason": "transfer_timeout",
|
||||
"effect_state": data
|
||||
.and_then(|value| value.get("effect_state"))
|
||||
.and_then(Value::as_str)
|
||||
.unwrap_or("unknown"),
|
||||
"phase": data
|
||||
.and_then(|value| value.get("phase"))
|
||||
.and_then(Value::as_str)
|
||||
.unwrap_or("cleanup"),
|
||||
"cleanup_state": data
|
||||
.and_then(|value| value.get("cleanup_state"))
|
||||
.and_then(Value::as_str)
|
||||
.unwrap_or("complete"),
|
||||
})),
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
enum WaitOutcome {
|
||||
Response(bsk_protocol::ResponseFrame),
|
||||
CancelledAfterResponse(bsk_protocol::ResponseFrame),
|
||||
TimedOutAfterResponse(bsk_protocol::ResponseFrame),
|
||||
CleanupTimeout,
|
||||
TimeoutCleanupFailed,
|
||||
WaiterClosed,
|
||||
Timeout,
|
||||
}
|
||||
@@ -739,6 +876,7 @@ async fn await_with_optional_cancel(
|
||||
mut waiter: oneshot::Receiver<bsk_protocol::ResponseFrame>,
|
||||
cancel: Option<&super::abort::AbortToken>,
|
||||
on_abort: Option<&(dyn Fn() + Sync)>,
|
||||
on_deadline: Option<&(dyn Fn() -> bool + Sync)>,
|
||||
) -> WaitOutcome {
|
||||
match cancel {
|
||||
Some(token) => {
|
||||
@@ -764,6 +902,16 @@ async fn await_with_optional_cancel(
|
||||
Err(_) => WaitOutcome::WaiterClosed,
|
||||
},
|
||||
_ = &mut deadline => {
|
||||
if let Some(send_cancel) = on_deadline {
|
||||
return if send_cancel() {
|
||||
match tokio::time::timeout(cleanup_timeout, &mut waiter).await {
|
||||
Ok(Ok(resp)) => WaitOutcome::TimedOutAfterResponse(resp),
|
||||
Ok(Err(_)) | Err(_) => WaitOutcome::TimeoutCleanupFailed,
|
||||
}
|
||||
} else {
|
||||
WaitOutcome::TimeoutCleanupFailed
|
||||
};
|
||||
}
|
||||
let Some(on_abort) = on_abort else {
|
||||
return WaitOutcome::Timeout;
|
||||
};
|
||||
@@ -779,7 +927,16 @@ async fn await_with_optional_cancel(
|
||||
None => match tokio::time::timeout(timeout, &mut waiter).await {
|
||||
Ok(Ok(resp)) => WaitOutcome::Response(resp),
|
||||
Ok(Err(_)) => WaitOutcome::WaiterClosed,
|
||||
Err(_) => WaitOutcome::Timeout,
|
||||
Err(_) => match on_deadline {
|
||||
Some(send_cancel) if send_cancel() => {
|
||||
match tokio::time::timeout(cleanup_timeout, &mut waiter).await {
|
||||
Ok(Ok(resp)) => WaitOutcome::TimedOutAfterResponse(resp),
|
||||
Ok(Err(_)) | Err(_) => WaitOutcome::TimeoutCleanupFailed,
|
||||
}
|
||||
}
|
||||
Some(_) => WaitOutcome::TimeoutCleanupFailed,
|
||||
None => WaitOutcome::Timeout,
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
@@ -864,6 +1021,7 @@ mod await_with_optional_cancel_tests {
|
||||
rx,
|
||||
Some(&token),
|
||||
None,
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
assert!(
|
||||
@@ -887,6 +1045,7 @@ mod await_with_optional_cancel_tests {
|
||||
rx,
|
||||
Some(&token),
|
||||
None,
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
match outcome {
|
||||
@@ -907,6 +1066,7 @@ mod await_with_optional_cancel_tests {
|
||||
rx,
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
assert!(matches!(outcome, WaitOutcome::Response(_)));
|
||||
@@ -924,6 +1084,7 @@ mod await_with_optional_cancel_tests {
|
||||
rx,
|
||||
Some(&token),
|
||||
None,
|
||||
None,
|
||||
)
|
||||
.await
|
||||
});
|
||||
@@ -948,6 +1109,7 @@ mod await_with_optional_cancel_tests {
|
||||
rx,
|
||||
Some(&token),
|
||||
None,
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
assert!(matches!(outcome, WaitOutcome::CleanupTimeout));
|
||||
@@ -972,12 +1134,49 @@ mod await_with_optional_cancel_tests {
|
||||
rx,
|
||||
Some(&token),
|
||||
Some(&forward_abort),
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
|
||||
assert!(abort_forwarded.load(Ordering::SeqCst));
|
||||
assert!(matches!(outcome, WaitOutcome::TimedOutAfterResponse(_)));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn transfer_deadline_sends_cancel_and_waits_for_cleanup_response() {
|
||||
let (tx, rx) = oneshot::channel();
|
||||
tokio::spawn(async move {
|
||||
tokio::time::sleep(Duration::from_millis(5)).await;
|
||||
tx.send(dummy_response()).unwrap();
|
||||
});
|
||||
let send_cancel = || true;
|
||||
let outcome = await_with_optional_cancel(
|
||||
Duration::from_millis(1),
|
||||
Duration::from_secs(1),
|
||||
rx,
|
||||
None,
|
||||
None,
|
||||
Some(&send_cancel),
|
||||
)
|
||||
.await;
|
||||
assert!(matches!(outcome, WaitOutcome::TimedOutAfterResponse(_)));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn transfer_deadline_is_unknown_when_cancel_cannot_be_sent() {
|
||||
let (_tx, rx) = oneshot::channel();
|
||||
let send_cancel = || false;
|
||||
let outcome = await_with_optional_cancel(
|
||||
Duration::from_millis(1),
|
||||
Duration::from_secs(1),
|
||||
rx,
|
||||
None,
|
||||
None,
|
||||
Some(&send_cancel),
|
||||
)
|
||||
.await;
|
||||
assert!(matches!(outcome, WaitOutcome::TimeoutCleanupFailed));
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
|
||||
@@ -225,6 +225,10 @@ pub fn run_foreground(cfg: DaemonConfig) -> Result<()> {
|
||||
.with_context(|| format!("bind IPC endpoint {}", sock_path.display()))?;
|
||||
|
||||
let state = Arc::new(DaemonState::new(cfg.clone()));
|
||||
state
|
||||
.transfers
|
||||
.initialize()
|
||||
.context("initialize transfer staging")?;
|
||||
let session_idle_task = spawn_session_idle_reaper(Arc::clone(&state));
|
||||
let browser_liveness_task = spawn_browser_liveness_reaper(Arc::clone(&state));
|
||||
// Fired by the update check task after a successful auto-update:
|
||||
@@ -448,6 +452,7 @@ pub(crate) fn spawn_browser_liveness_reaper(
|
||||
for s in state.sessions.purge_browser(&client.id) {
|
||||
state.tool_queues.remove(&s.id);
|
||||
state.session_interrupts.drop_session(&s.id);
|
||||
state.transfers.release_session(&s.id.0);
|
||||
debug!(session = %s.id, "purged session on browser liveness timeout");
|
||||
}
|
||||
}
|
||||
@@ -485,7 +490,10 @@ pub(crate) fn spawn_session_idle_reaper(state: Arc<DaemonState>) -> tokio::task:
|
||||
)
|
||||
.await
|
||||
{
|
||||
Ok(_) => info!(session = %session_id, "idle session stopped"),
|
||||
Ok(_) => {
|
||||
state.transfers.release_session(&session_id.0);
|
||||
info!(session = %session_id, "idle session stopped");
|
||||
}
|
||||
Err(StopSessionError::SessionBusy | StopSessionError::Stopping) => {
|
||||
debug!(session = %session_id, "idle session still active; retrying later");
|
||||
}
|
||||
@@ -496,6 +504,7 @@ pub(crate) fn spawn_session_idle_reaper(state: Arc<DaemonState>) -> tokio::task:
|
||||
&state.session_interrupts,
|
||||
&session_id,
|
||||
);
|
||||
state.transfers.release_session(&session_id.0);
|
||||
}
|
||||
Err(err) => {
|
||||
warn!(session = %session_id, error = %err, "failed to stop idle session");
|
||||
|
||||
@@ -7,6 +7,7 @@ use tokio::task::JoinHandle;
|
||||
|
||||
use super::abort::AbortRegistry;
|
||||
use super::browsers::BrowserRegistry;
|
||||
use super::file_transfer::TransferRegistry;
|
||||
use super::inflight::ToolInflightRegistry;
|
||||
use super::ipc::IpcHandle;
|
||||
use super::queue::ToolQueueRegistry;
|
||||
@@ -51,6 +52,9 @@ pub struct DaemonState {
|
||||
/// `SessionRegistry` because the signal is a transient runtime
|
||||
/// control state.
|
||||
pub session_interrupts: Arc<SessionInterruptRegistry>,
|
||||
/// Operation-scoped local file staging. The extension only sees paths
|
||||
/// minted here; agent-facing RPCs use opaque transfer ids.
|
||||
pub transfers: Arc<TransferRegistry>,
|
||||
}
|
||||
|
||||
impl DaemonState {
|
||||
@@ -64,6 +68,7 @@ impl DaemonState {
|
||||
));
|
||||
let abort_registry = Arc::new(AbortRegistry::new());
|
||||
let session_interrupts = Arc::new(SessionInterruptRegistry::new());
|
||||
let transfers = Arc::new(TransferRegistry::new().expect("initialise transfer staging"));
|
||||
Self {
|
||||
config,
|
||||
browsers,
|
||||
@@ -72,6 +77,7 @@ impl DaemonState {
|
||||
abort_registry,
|
||||
tool_inflight,
|
||||
session_interrupts,
|
||||
transfers,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -295,6 +295,7 @@ async fn drive_connection(
|
||||
for session in state.sessions.purge_browser(&browser_id) {
|
||||
state.tool_queues.remove(&session.id);
|
||||
state.session_interrupts.drop_session(&session.id);
|
||||
state.transfers.release_session(&session.id.0);
|
||||
debug!(session = %session.id, "purged stale session before browser reconnect");
|
||||
}
|
||||
let (tx, mut rx) = mpsc::unbounded_channel::<Frame>();
|
||||
@@ -407,6 +408,7 @@ async fn drive_connection(
|
||||
for s in state.sessions.purge_browser(&browser_id) {
|
||||
state.tool_queues.remove(&s.id);
|
||||
state.session_interrupts.drop_session(&s.id);
|
||||
state.transfers.release_session(&s.id.0);
|
||||
debug!(session = %s.id, "purged session on browser disconnect");
|
||||
}
|
||||
} else {
|
||||
@@ -503,6 +505,7 @@ fn handle_session_window_closed(
|
||||
&state.session_interrupts,
|
||||
&session_id,
|
||||
) {
|
||||
state.transfers.release_session(&session_id.0);
|
||||
info!(session = %session_id, "session removed: user closed Agent Window");
|
||||
} else {
|
||||
debug!(session = %session_id, "session.window_closed for unknown session id");
|
||||
|
||||
@@ -97,6 +97,8 @@ fn dispatch(cli: Cli, format: Format) -> Result<(), CliError> {
|
||||
Command::Fill(args) => cli::interaction::dispatch_fill(args, format),
|
||||
Command::Press(args) => cli::interaction::dispatch_press(args, format),
|
||||
Command::Select(args) => cli::interaction::dispatch_select(args, format),
|
||||
Command::Upload(args) => cli::upload::dispatch(args, format),
|
||||
Command::Download(args) => cli::download::dispatch(args, format),
|
||||
Command::Evaluate(args) => cli::evaluate::dispatch(args, format),
|
||||
Command::WaitForNavigation(args) => cli::waits::dispatch_wait_for_navigation(args, format),
|
||||
Command::WaitMs(args) => cli::waits::dispatch_wait_ms(args, format),
|
||||
|
||||
@@ -213,6 +213,45 @@ fn rejects_zero_click_count() {
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parses_upload_with_repeated_files() {
|
||||
let cli = parse(&[
|
||||
"bsk",
|
||||
"upload",
|
||||
"@e3",
|
||||
"--file",
|
||||
"one.png",
|
||||
"--file",
|
||||
"two.png",
|
||||
"--session",
|
||||
"s1",
|
||||
]);
|
||||
let Command::Upload(args) = cli.command else {
|
||||
panic!("expected upload command");
|
||||
};
|
||||
assert_eq!(args.target.as_deref(), Some("@e3"));
|
||||
assert_eq!(args.files.len(), 2);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parses_download_with_exact_output_policy() {
|
||||
let cli = parse(&[
|
||||
"bsk",
|
||||
"download",
|
||||
"#export",
|
||||
"--out",
|
||||
"result.zip",
|
||||
"--session",
|
||||
"s1",
|
||||
"--overwrite",
|
||||
]);
|
||||
let Command::Download(args) = cli.command else {
|
||||
panic!("expected download command");
|
||||
};
|
||||
assert_eq!(args.target.as_deref(), Some("#export"));
|
||||
assert!(args.overwrite);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parses_record_start_with_browser_and_url() {
|
||||
let cli = parse(&[
|
||||
|
||||
@@ -0,0 +1,56 @@
|
||||
{
|
||||
"$schema": "http://json-schema.org/draft-07/schema#",
|
||||
"title": "DownloadParams",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"session_id"
|
||||
],
|
||||
"properties": {
|
||||
"browser_relative_dir": {
|
||||
"description": "Daemon-injected relative directory beneath Chrome's Downloads root.",
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"max_byte_size": {
|
||||
"description": "Daemon-injected authoritative transfer size limit. The extension uses it only for early cancellation; daemon import remains the final check.",
|
||||
"type": [
|
||||
"integer",
|
||||
"null"
|
||||
],
|
||||
"format": "uint64",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"ref": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"selector": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"session_id": {
|
||||
"type": "string"
|
||||
},
|
||||
"tab_id": {
|
||||
"type": [
|
||||
"integer",
|
||||
"null"
|
||||
],
|
||||
"format": "int64"
|
||||
},
|
||||
"timeout_ms": {
|
||||
"type": [
|
||||
"integer",
|
||||
"null"
|
||||
],
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,62 @@
|
||||
{
|
||||
"$schema": "http://json-schema.org/draft-07/schema#",
|
||||
"title": "DownloadResult",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"byte_size",
|
||||
"suggested_filename",
|
||||
"tab_id"
|
||||
],
|
||||
"properties": {
|
||||
"browser_path": {
|
||||
"description": "Extension-internal completed Chrome download path, stripped by the daemon.",
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"byte_size": {
|
||||
"type": "integer",
|
||||
"format": "uint64",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"danger": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"mime": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"suggested_filename": {
|
||||
"type": "string"
|
||||
},
|
||||
"tab_id": {
|
||||
"type": "integer",
|
||||
"format": "int64"
|
||||
},
|
||||
"transfer_id": {
|
||||
"description": "Opaque id returned by the daemon to the CLI.",
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"used_ref": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"used_selector": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,70 @@
|
||||
{
|
||||
"$schema": "http://json-schema.org/draft-07/schema#",
|
||||
"title": "UploadParams",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"files",
|
||||
"session_id"
|
||||
],
|
||||
"properties": {
|
||||
"files": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/UploadFile"
|
||||
}
|
||||
},
|
||||
"ref": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"selector": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"session_id": {
|
||||
"type": "string"
|
||||
},
|
||||
"tab_id": {
|
||||
"type": [
|
||||
"integer",
|
||||
"null"
|
||||
],
|
||||
"format": "int64"
|
||||
},
|
||||
"timeout_ms": {
|
||||
"type": [
|
||||
"integer",
|
||||
"null"
|
||||
],
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
}
|
||||
},
|
||||
"definitions": {
|
||||
"UploadFile": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"name",
|
||||
"transfer_id"
|
||||
],
|
||||
"properties": {
|
||||
"name": {
|
||||
"type": "string"
|
||||
},
|
||||
"staged_path": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"transfer_id": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,33 @@
|
||||
{
|
||||
"$schema": "http://json-schema.org/draft-07/schema#",
|
||||
"title": "UploadResult",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"file_names",
|
||||
"tab_id"
|
||||
],
|
||||
"properties": {
|
||||
"file_names": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"tab_id": {
|
||||
"type": "integer",
|
||||
"format": "int64"
|
||||
},
|
||||
"used_ref": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"used_selector": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -85,6 +85,10 @@ fn main() {
|
||||
dump!(PressResult, "tool_press_result");
|
||||
dump!(SelectParams, "tool_select_params");
|
||||
dump!(SelectResult, "tool_select_result");
|
||||
dump!(UploadParams, "tool_upload_params");
|
||||
dump!(UploadResult, "tool_upload_result");
|
||||
dump!(DownloadParams, "tool_download_params");
|
||||
dump!(DownloadResult, "tool_download_result");
|
||||
|
||||
dump!(SnapshotParams, "tool_snapshot_params");
|
||||
dump!(SnapshotResult, "tool_snapshot_result");
|
||||
|
||||
@@ -78,6 +78,10 @@ pub enum Method {
|
||||
ToolPress,
|
||||
#[serde(rename = "tool.select")]
|
||||
ToolSelect,
|
||||
#[serde(rename = "tool.upload")]
|
||||
ToolUpload,
|
||||
#[serde(rename = "tool.download")]
|
||||
ToolDownload,
|
||||
#[serde(rename = "tool.snapshot")]
|
||||
ToolSnapshot,
|
||||
#[serde(rename = "tool.observe")]
|
||||
@@ -105,6 +109,17 @@ pub enum Method {
|
||||
#[serde(rename = "tool.record_await")]
|
||||
ToolRecordAwait,
|
||||
|
||||
#[serde(rename = "transfer.begin")]
|
||||
TransferBegin,
|
||||
#[serde(rename = "transfer.chunk")]
|
||||
TransferChunk,
|
||||
#[serde(rename = "transfer.finish")]
|
||||
TransferFinish,
|
||||
#[serde(rename = "transfer.read")]
|
||||
TransferRead,
|
||||
#[serde(rename = "transfer.release")]
|
||||
TransferRelease,
|
||||
|
||||
#[serde(rename = "cancel")]
|
||||
Cancel,
|
||||
}
|
||||
@@ -156,6 +171,8 @@ impl Method {
|
||||
| Method::ToolFill
|
||||
| Method::ToolPress
|
||||
| Method::ToolSelect
|
||||
| Method::ToolUpload
|
||||
| Method::ToolDownload
|
||||
| Method::ToolEvaluate
|
||||
// May navigate via optional `url` and changes Agent Window
|
||||
// chrome; gate behind pending-interrupt like other writes.
|
||||
@@ -194,6 +211,11 @@ impl Method {
|
||||
| Method::SystemPing
|
||||
| Method::SystemStatus
|
||||
| Method::BrowserList
|
||||
| Method::TransferBegin
|
||||
| Method::TransferChunk
|
||||
| Method::TransferFinish
|
||||
| Method::TransferRead
|
||||
| Method::TransferRelease
|
||||
| Method::Cancel => MethodEffect::ControlPlane,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,176 @@
|
||||
//! Content-transparent file transfer contracts.
|
||||
//!
|
||||
//! Public tool calls carry opaque transfer ids. Filesystem paths are injected
|
||||
//! by the daemon only on the daemon -> extension hop and are never accepted as
|
||||
//! an agent-facing source or destination.
|
||||
|
||||
use schemars::JsonSchema;
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct UploadFile {
|
||||
pub transfer_id: String,
|
||||
pub name: String,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub staged_path: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct UploadParams {
|
||||
pub session_id: String,
|
||||
#[serde(
|
||||
rename = "ref",
|
||||
alias = "ref_",
|
||||
default,
|
||||
skip_serializing_if = "Option::is_none"
|
||||
)]
|
||||
pub ref_: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub selector: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub tab_id: Option<i64>,
|
||||
pub files: Vec<UploadFile>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub timeout_ms: Option<u32>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct UploadResult {
|
||||
pub tab_id: i64,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub used_ref: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub used_selector: Option<String>,
|
||||
pub file_names: Vec<String>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct DownloadParams {
|
||||
pub session_id: String,
|
||||
#[serde(
|
||||
rename = "ref",
|
||||
alias = "ref_",
|
||||
default,
|
||||
skip_serializing_if = "Option::is_none"
|
||||
)]
|
||||
pub ref_: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub selector: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub tab_id: Option<i64>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub timeout_ms: Option<u32>,
|
||||
/// Daemon-injected relative directory beneath Chrome's Downloads root.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub browser_relative_dir: Option<String>,
|
||||
/// Daemon-injected authoritative transfer size limit. The extension uses
|
||||
/// it only for early cancellation; daemon import remains the final check.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub max_byte_size: Option<u64>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct DownloadResult {
|
||||
pub tab_id: i64,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub used_ref: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub used_selector: Option<String>,
|
||||
pub suggested_filename: String,
|
||||
pub byte_size: u64,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub mime: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub danger: Option<String>,
|
||||
/// Extension-internal completed Chrome download path, stripped by the daemon.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub browser_path: Option<String>,
|
||||
/// Opaque id returned by the daemon to the CLI.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub transfer_id: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct TransferBeginParams {
|
||||
pub session_id: String,
|
||||
pub name: String,
|
||||
pub byte_size: u64,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct TransferBeginResult {
|
||||
pub transfer_id: String,
|
||||
pub chunk_size: u32,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct TransferChunkParams {
|
||||
pub transfer_id: String,
|
||||
pub offset: u64,
|
||||
pub data_base64: String,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct TransferChunkResult {
|
||||
pub next_offset: u64,
|
||||
#[serde(default)]
|
||||
pub eof: bool,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub data_base64: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct TransferIdParams {
|
||||
pub transfer_id: String,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct TransferReadyResult {
|
||||
pub transfer_id: String,
|
||||
pub byte_size: u64,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct TransferReleaseResult {
|
||||
pub released: bool,
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn upload_agent_shape_omits_daemon_path_and_uses_ref_wire_name() {
|
||||
let value = serde_json::to_value(UploadParams {
|
||||
session_id: "s1".into(),
|
||||
ref_: Some("@e3".into()),
|
||||
selector: None,
|
||||
tab_id: None,
|
||||
files: vec![UploadFile {
|
||||
transfer_id: "tr_1".into(),
|
||||
name: "image.png".into(),
|
||||
staged_path: None,
|
||||
}],
|
||||
timeout_ms: None,
|
||||
})
|
||||
.unwrap();
|
||||
assert_eq!(value["ref"], "@e3");
|
||||
assert!(value["files"][0].get("staged_path").is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn download_agent_shape_omits_daemon_path() {
|
||||
let value = serde_json::to_value(DownloadParams {
|
||||
session_id: "s1".into(),
|
||||
ref_: None,
|
||||
selector: Some("#export".into()),
|
||||
tab_id: None,
|
||||
timeout_ms: None,
|
||||
browser_relative_dir: None,
|
||||
max_byte_size: None,
|
||||
})
|
||||
.unwrap();
|
||||
assert!(value.get("browser_relative_dir").is_none());
|
||||
assert!(value.get("max_byte_size").is_none());
|
||||
}
|
||||
}
|
||||
@@ -3,6 +3,7 @@
|
||||
pub mod console;
|
||||
pub mod dialog;
|
||||
pub mod emulate;
|
||||
pub mod file_transfer;
|
||||
pub mod human_loop;
|
||||
pub mod interaction;
|
||||
pub mod navigation;
|
||||
@@ -21,6 +22,7 @@ pub mod window;
|
||||
pub use console::*;
|
||||
pub use dialog::*;
|
||||
pub use emulate::*;
|
||||
pub use file_transfer::*;
|
||||
pub use human_loop::*;
|
||||
pub use interaction::*;
|
||||
pub use navigation::*;
|
||||
|
||||
@@ -143,6 +143,15 @@ flowchart LR
|
||||
- No credential storage in bsk — cookies stay in the user's browser profile.
|
||||
- `evaluate` restricted to Agent Window tabs in sandbox mode.
|
||||
|
||||
### File-transfer boundary
|
||||
|
||||
- The invoking agent/harness decides whether a transfer is authorized and supplies the task-local source or destination path.
|
||||
- The CLI is the only component that reads an upload source or writes the final download destination. Before browser dispatch it owns rollback of partially staged uploads; after dispatch, ownership moves to the session because a transport timeout cannot prove that Chrome did not attach the file. Download output becomes visible through one atomic commit, and replacement is opt-in without a pre-delete window. The extension never receives either agent-facing path.
|
||||
- The daemon is the authority for storage capabilities and limits. It issues opaque session-scoped transfer IDs, stages bounded chunks in a private runtime directory, and injects only private staged upload paths. For download it mints one relative Chrome directory capability. Only after validating the reported path, file type, symlink boundary, and authoritative byte limit does it take ownership of browser-file cleanup and import the bytes.
|
||||
- The extension owns only the browser transaction. Both tools resolve one `ResolvedActionTarget` and use that same target for protocol setup and the triggering click. Upload arms Chrome's chooser interception before clicking, then accepts either an exact `Page.fileChooserOpened` input node or an independent probe anchored in the trigger node's own document; one verified input is committed with `DOM.setFileInputFiles`. A non-input picker is rejected immediately as unsupported. Download correlates exact-target CDP intent and `chrome.downloads` filename candidates in either arrival order, claims only one unique match, and never cancels an unclaimed candidate.
|
||||
- Browser-side operations report `effect_state` (`none`, `committed`, or `unknown`), `phase`, and `cleanup_state`. Confirmed success wins over a late cancel; an unknown effect is preserved across timeout or transport loss and must not be retried blindly. A transfer deadline sends cancellation to the extension and keeps the session queue occupied for bounded compensation rather than abandoning an in-flight browser effect.
|
||||
- Download staging is released after CLI commit. Upload staging remains until session teardown because the page may read an attached file only on a later form submission. Remaining staging is released on session stop/browser disconnect and on daemon startup after a crash. BrowserSkill does not inspect content or decide whether a transfer is appropriate.
|
||||
|
||||
## Repository layout
|
||||
|
||||
```
|
||||
|
||||
@@ -20,43 +20,22 @@ extension keep owning the actual browser control — this package is a thin, wel
|
||||
| `browser_snapshot` | `bsk snapshot` | Indented aria-tree snapshot with `@eN` refs for interaction tools. |
|
||||
| `browser_observe` | `bsk observe` | Semantic VOM observation (read-only) with `@eN` refs. |
|
||||
| `browser_click` | `bsk click` | Click a snapshot ref or CSS selector (button / click-count options). |
|
||||
| `browser_hover` | `bsk hover` | Hover a ref or selector and allow hover-triggered UI to settle. |
|
||||
| `browser_fill` | `bsk fill` | Fill an input / textarea / contenteditable (clears first by default). |
|
||||
| `browser_select` | `bsk select` | Select one or more option values on a `<select>`. |
|
||||
| `browser_press` | `bsk press` | Dispatch a key or combo, optionally focusing a target first. |
|
||||
| `browser_screenshot` | `bsk screenshot` | PNG capture of the tab or a ref-cropped element; inlines the image when the deployment supports image input, otherwise returns a file path. |
|
||||
| `browser_emulate` | `bsk emulate` | Apply or clear mobile device emulation on the active tab. |
|
||||
| `browser_tab_list` | `bsk tab list` | List user and/or Agent Window tabs visible to an owned session. |
|
||||
| `browser_tab_create` | `bsk tab create` | Create a foreground or background tab in the Agent Window. |
|
||||
| `browser_tab_select` | `bsk tab select` | Focus an Agent Window tab. |
|
||||
| `browser_tab_close` | `bsk tab close` | Close an Agent Window tab. |
|
||||
| `browser_tab_borrow` | `bsk tab borrow` | Move a user tab into the Agent Window for controlled interaction. |
|
||||
| `browser_tab_return` | `bsk tab return` | Return a borrowed tab to its original user window. |
|
||||
| `browser_navigate_back` / `browser_navigate_forward` | `bsk navigate-back` / `bsk navigate-forward` | Traverse tab history and wait for a lifecycle phase. |
|
||||
| `browser_reload` | `bsk reload` | Reload the active tab, optionally bypassing cache. |
|
||||
| `browser_wait_for_navigation` | `bsk wait-for-navigation` | Wait explicitly for a page lifecycle event. |
|
||||
| `browser_request_help` | `bsk request-help` | Pause for login, captcha, OTP, confirmation, or another human browser step. |
|
||||
| `browser_get_html` | `bsk get-html` | Read raw document or ref-scoped HTML after semantic observation is insufficient. |
|
||||
| `browser_console` | `bsk console` | Read buffered console/log/exception entries without evaluating JavaScript. |
|
||||
| `browser_network` | `bsk network` | Read buffered network response/failure metadata. |
|
||||
| `browser_window_resize` | `bsk window resize` | Resize the owned session's Agent Window. |
|
||||
|
||||
`bsk evaluate` and `bsk record` are intentionally not exposed by this plugin: arbitrary page
|
||||
evaluation is a higher-risk capability, while recording is long-running and needs a dedicated
|
||||
DeepSeek Harness lifecycle before it can be added safely.
|
||||
|
||||
## Agent skill (progressive disclosure)
|
||||
|
||||
Beyond the tools, the plugin publishes the **`browser-skill` agent skill** through the harness's
|
||||
official skill seam (`ctx.skills.register`): the catalog entry (name + routing description) is
|
||||
resident in `<available_skills>`, and the body is loaded only when the model invokes the `skill`
|
||||
tool. Its single source is the DSH-specific `skill/SKILL.md`, which documents only structured
|
||||
`browser_*` calls and their plugin semantics. The repository-root CLI skill is intentionally not
|
||||
concatenated: its command examples belong to a different execution interface and would bypass
|
||||
the plugin's ownership, live observation UI, cancellation, and cleanup path if followed directly.
|
||||
The build rejects internal CLI-name leakage, command-line code blocks, unknown browser tools, and
|
||||
missing supported browser tools before embedding the Markdown. Registration and every pre-step
|
||||
catalog snapshot are pure in-memory reads (no disk/process/daemon); compositions without the skill
|
||||
tool. The body is assembled at build time from two parts, so there is exactly one source of
|
||||
truth: a dsh-specific prelude (`skill/prelude.md` — tool↔CLI map, owned-session semantics,
|
||||
plugin-only overrides) followed verbatim by the canonical CLI skill (`skill/SKILL.md` at the
|
||||
repo root — the same file `crates/bsk-cli/build.rs` mirrors for the CLI package; workflows,
|
||||
stop-when-done rules, refs usage, sandbox rules). Registration and every pre-step catalog
|
||||
snapshot are pure in-memory reads (no disk/process/daemon); compositions without the skill
|
||||
seam degrade silently.
|
||||
|
||||
## Multi-session model
|
||||
@@ -109,7 +88,7 @@ All fields are optional and validated through the plugin's Schemastery `Config`:
|
||||
# lazyTools: true # reveal browser_* tools only after the skill is invoked
|
||||
```
|
||||
|
||||
- **`lazyTools` (default `true`)** — the final progressive-disclosure stage: the 28
|
||||
- **`lazyTools` (default `true`)** — the final progressive-disclosure stage: the eleven
|
||||
`browser_*` tool schemas stay OUT of the system prompt (zero schema tokens) until the
|
||||
`browser-skill` skill is actually invoked — the skill catalog entry is the only
|
||||
advertisement. One successful invocation (model tool call, or a `/browser-skill` user
|
||||
@@ -173,7 +152,7 @@ shell's theme cannot bleed back in.
|
||||
card). Screenshots additionally attach the image itself when the host mounts an attachment store
|
||||
and the active model route declares image input; otherwise the PNG path is returned.
|
||||
- **Web UI toolview (browser half)**: the package is dual-face. `dsh.client` (platform `web`) ships
|
||||
`lib/client.cjs`, which registers a keyed `tool.call.toolview` view for `browser_screenshot`. The
|
||||
`lib/client.js`, which registers a keyed `tool.call.toolview` view for `browser_screenshot`. The
|
||||
custom view keeps the terminal block (command + output) and, when the settled result carries an
|
||||
image block, resolves the durable attachment through the client session's authorized
|
||||
`readAttachment` RPC and renders it with the shared `MessageImage` thumbnail/lightbox atoms.
|
||||
@@ -204,8 +183,8 @@ as `@wxg-prc-cpg/browser-skill-dsh-plugin`. The Cordis plugin id stays
|
||||
Trigger it by pushing a tag that matches `package.json`'s `version`:
|
||||
|
||||
```sh
|
||||
git tag dsh-plugin-v0.1.2
|
||||
git push origin dsh-plugin-v0.1.2
|
||||
git tag dsh-plugin-v0.1.0
|
||||
git push origin dsh-plugin-v0.1.0
|
||||
```
|
||||
|
||||
Or run the workflow from the Actions tab (`workflow_dispatch`). The job reads the
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "@wxg-prc-cpg/browser-skill-dsh-plugin",
|
||||
"description": "DeepSeek Harness tool plugin that exposes BrowserSkill browser automation (browser_* tools) to the model",
|
||||
"version": "0.1.2",
|
||||
"description": "DeepSeek Harness tool plugin that exposes BrowserSkill (bsk) browser automation to the model",
|
||||
"version": "0.1.1",
|
||||
"license": "MIT",
|
||||
"publishConfig": {
|
||||
"access": "public",
|
||||
|
||||
@@ -1,15 +1,20 @@
|
||||
// Embed the DeepSeek Harness-specific browser skill at build time. The model-facing
|
||||
// skill deliberately documents only injected browser_* tools; the repository's CLI
|
||||
// skill is a separate interface and must never be concatenated here.
|
||||
// Assemble the plugin's skill content: a dsh-specific routing prelude, the
|
||||
// canonical CLI skill body verbatim (repo-root skill/SKILL.md — the same source
|
||||
// crates/bsk-cli's build.rs mirrors), then a dsh routing reminder postlude. The
|
||||
// original's YAML frontmatter is stripped: name/description are supplied by the
|
||||
// registration.
|
||||
// Output: src/skill-content.generated.ts (gitignored, built before tsdown and
|
||||
// before vitest), so registration is a pure in-memory read.
|
||||
// before vitest) so the markdown ships inlined in the bundle and registration
|
||||
// is a pure in-memory read — never a per-step disk access.
|
||||
import { readFileSync, writeFileSync } from "node:fs";
|
||||
import { dirname, join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
const here = dirname(fileURLToPath(import.meta.url));
|
||||
const pkg = join(here, "..");
|
||||
const source = readFileSync(join(pkg, "skill", "SKILL.md"), "utf8");
|
||||
const prelude = readFileSync(join(pkg, "skill", "prelude.md"), "utf8");
|
||||
const upstream = readFileSync(join(pkg, "..", "..", "skill", "SKILL.md"), "utf8");
|
||||
const postlude = readFileSync(join(pkg, "skill", "postlude.md"), "utf8");
|
||||
|
||||
function stripFrontmatter(markdown) {
|
||||
if (!markdown.startsWith("---\n")) return markdown;
|
||||
@@ -17,69 +22,18 @@ function stripFrontmatter(markdown) {
|
||||
return end === -1 ? markdown : markdown.slice(end + 5).replace(/^\s*\n/, "");
|
||||
}
|
||||
|
||||
const SUPPORTED_BROWSER_TOOLS = [
|
||||
"browser_session_start",
|
||||
"browser_session_stop",
|
||||
"browser_session_list",
|
||||
"browser_navigate",
|
||||
"browser_navigate_back",
|
||||
"browser_navigate_forward",
|
||||
"browser_reload",
|
||||
"browser_wait_for_navigation",
|
||||
"browser_snapshot",
|
||||
"browser_observe",
|
||||
"browser_get_html",
|
||||
"browser_screenshot",
|
||||
"browser_console",
|
||||
"browser_network",
|
||||
"browser_click",
|
||||
"browser_hover",
|
||||
"browser_fill",
|
||||
"browser_select",
|
||||
"browser_press",
|
||||
"browser_tab_list",
|
||||
"browser_tab_create",
|
||||
"browser_tab_select",
|
||||
"browser_tab_close",
|
||||
"browser_tab_borrow",
|
||||
"browser_tab_return",
|
||||
"browser_request_help",
|
||||
"browser_emulate",
|
||||
"browser_window_resize",
|
||||
];
|
||||
|
||||
function validateModelFacingSkill(markdown) {
|
||||
if (/\bbsk\b/i.test(markdown)) {
|
||||
throw new Error("DSH skill must not expose the internal CLI name");
|
||||
}
|
||||
if (/```(?:bash|sh|shell)\b/i.test(markdown)) {
|
||||
throw new Error("DSH skill must not contain command-line code blocks");
|
||||
}
|
||||
|
||||
const mentioned = new Set(markdown.match(/\bbrowser_[a-z][a-z_]*\b/g) ?? []);
|
||||
const supported = new Set(SUPPORTED_BROWSER_TOOLS);
|
||||
const unknown = [...mentioned].filter((name) => !supported.has(name));
|
||||
const missing = SUPPORTED_BROWSER_TOOLS.filter((name) => !mentioned.has(name));
|
||||
if (unknown.length > 0) {
|
||||
throw new Error(`DSH skill mentions unsupported browser tools: ${unknown.join(", ")}`);
|
||||
}
|
||||
if (missing.length > 0) {
|
||||
throw new Error(`DSH skill omits supported browser tools: ${missing.join(", ")}`);
|
||||
}
|
||||
}
|
||||
|
||||
const NAME = "browser-skill";
|
||||
const DESCRIPTION =
|
||||
"Browser automation against the user's logged-in Chromium through this plugin's injected " +
|
||||
"browser_* tools. Use to visit and read pages, fill forms, click through flows, inspect tabs, " +
|
||||
"debug page activity, or smoke-test UI changes in a managed Agent Window.";
|
||||
"Browser automation against the user's logged-in Chromium through this plugin's injected browser_* tools; never invoke bsk through bash or shell. " +
|
||||
"tools (bsk engine): visit and read pages, fill forms, click through flows, scrape data, " +
|
||||
"smoke-test UI changes. Use when a task needs a real browser; requires the bsk CLI and the " +
|
||||
"browser-skill extension connected.";
|
||||
|
||||
const content = `${stripFrontmatter(source).trim()}\n`;
|
||||
validateModelFacingSkill(content);
|
||||
const content = `${prelude.trim()}\n\n${stripFrontmatter(upstream).trim()}\n\n${postlude.trim()}\n`;
|
||||
|
||||
const banner =
|
||||
"// GENERATED by scripts/build-skill-content.mjs — do not edit.\n" +
|
||||
"// Source: skill/SKILL.md (DeepSeek Harness browser_* tool instructions).\n";
|
||||
"// Sources: skill/prelude.md + ../../../skill/SKILL.md + skill/postlude.md.\n";
|
||||
|
||||
writeFileSync(
|
||||
join(pkg, "src", "skill-content.generated.ts"),
|
||||
|
||||
@@ -1,271 +0,0 @@
|
||||
---
|
||||
name: browser-skill
|
||||
description: |
|
||||
Use the injected browser_* tools to automate the user's logged-in Chromium
|
||||
browser in a managed Agent Window.
|
||||
---
|
||||
|
||||
# browser-skill for DeepSeek Harness
|
||||
|
||||
Drive the user's real Chromium browser through the structured `browser_*` tools provided by this
|
||||
plugin. The browser keeps the user's existing logins and cookies while isolating automation in an
|
||||
Agent Window. Use the tool schemas exactly as exposed by DeepSeek Harness; arguments are structured
|
||||
fields, not command-line flags.
|
||||
|
||||
## Tool availability
|
||||
|
||||
Loading this skill reveals the browser tools for the rest of the conversation. If the tools were
|
||||
hidden before this skill was loaded, start browser work only after the skill invocation succeeds.
|
||||
|
||||
All browser operations must use the injected tools directly. Do not translate the examples in this
|
||||
document into terminal commands or invoke another process to control the browser. Direct tool calls
|
||||
preserve session ownership, the live observation window, tool cards, cancellation, and cleanup.
|
||||
|
||||
## When to use
|
||||
|
||||
- Visit and read pages that require the user's existing browser session.
|
||||
- Fill forms, select options, click through bounded flows, or smoke-test a web UI.
|
||||
- Inspect tabs, page structure, screenshots, console messages, or network metadata.
|
||||
- Work with a user tab after explicitly borrowing it into the Agent Window.
|
||||
|
||||
Do not use browser automation for tasks that need only local files, a purpose-built API, or an
|
||||
explanation. Never extract credentials, cookies, tokens, or other secrets. Do not keep control of a
|
||||
personal tab longer than the immediate task requires.
|
||||
|
||||
## Mandatory workflow
|
||||
|
||||
Every browser task follows this lifecycle:
|
||||
|
||||
1. Call `browser_session_start` and retain its returned `sessionId`.
|
||||
2. Use that session for every browser operation. Pass `session` explicitly when more than one
|
||||
session exists.
|
||||
3. Call `browser_session_stop` when the bounded goal is reached or an error ends the task. Treat
|
||||
cleanup like a `finally` path unless the user explicitly asks to keep the session open.
|
||||
|
||||
Conceptual tool-call sequence:
|
||||
|
||||
```text
|
||||
browser_session_start({ url: "https://example.com" })
|
||||
browser_observe({ session: sessionId })
|
||||
browser_click({ session: sessionId, target: "@e4" })
|
||||
browser_observe({ session: sessionId })
|
||||
browser_session_stop({ session: sessionId })
|
||||
```
|
||||
|
||||
The plugin owns only sessions created by `browser_session_start`. `browser_session_list` reports
|
||||
only those owned sessions. Never guess or reuse a session id from another program. When `session`
|
||||
is omitted, most tools use the most recently started or used owned session; explicit ids are safer
|
||||
for multi-session work.
|
||||
|
||||
## Stop when the goal is met
|
||||
|
||||
Browser work is a bounded task, not open-ended browsing.
|
||||
|
||||
1. Define one observable success condition from the user's request.
|
||||
2. Take the shortest path: observe, act, then perform at most one relevant check.
|
||||
3. Stop immediately when success is visible. Do not refresh, navigate, click, or perform extra
|
||||
verification after success.
|
||||
4. If a human-only step appears, use `browser_request_help`. If an action fails twice without
|
||||
progress, pause instead of brute-forcing.
|
||||
5. When uncertain, make at most one fresh observation. If the state is still ambiguous, ask the
|
||||
user rather than continuing to click.
|
||||
|
||||
## Core interaction loop
|
||||
|
||||
Use `browser_navigate` to open a destination in the active Agent Window tab, then inspect the page
|
||||
before interacting:
|
||||
|
||||
1. `browser_observe` is the primary semantic view. It returns roles, states, text, and `@eN` refs,
|
||||
including hints about conditional hover or focus surfaces.
|
||||
2. `browser_snapshot` is the static accessibility-tree fallback when the semantic view is
|
||||
insufficient or a stricter tree is useful.
|
||||
3. Prefer a fresh `@eN` ref over a raw CSS selector for `browser_click`, `browser_hover`,
|
||||
`browser_fill`, `browser_select`, and targeted `browser_press` calls.
|
||||
4. Observe again after navigation or a meaningful DOM change before using another ref.
|
||||
|
||||
Refs invalidate after navigation and may also become stale after large DOM changes. Never carry a
|
||||
ref to a new page. If an observation marks a control as hover-triggered, call `browser_hover` on the
|
||||
trigger, observe again, and then act on the newly visible ref.
|
||||
|
||||
Use interaction tools according to their schemas:
|
||||
|
||||
- `browser_click`: click a ref or selector; optionally choose mouse button and click count.
|
||||
- `browser_hover`: move to a ref or selector and allow hover UI to settle; then observe again.
|
||||
- `browser_fill`: clear and fill a text-capable element; set `noClear` only when appending is
|
||||
intentional.
|
||||
- `browser_select`: pass `values` as an array, including for a single option.
|
||||
- `browser_press`: send a key or combo, optionally focusing `target` first.
|
||||
|
||||
Actions may trigger a navigation after the action itself returns. Use
|
||||
`browser_wait_for_navigation` only when that lifecycle wait is actually needed, then observe the
|
||||
new page.
|
||||
|
||||
## Observation and page reading priority
|
||||
|
||||
Escalate page reading only as needed:
|
||||
|
||||
1. `browser_observe` for normal semantic understanding and interaction refs.
|
||||
2. `browser_snapshot` for a stricter static accessibility tree.
|
||||
3. `browser_get_html` for hidden DOM, metadata, or exact markup that the semantic views cannot
|
||||
provide. A `ref` must come from a fresh observation; keep `maxBytes` bounded.
|
||||
4. `browser_screenshot` for layout, styling, canvas, image content, or another genuinely visual
|
||||
question. Pass a fresh `ref` to crop to one element, or omit it for the visible tab.
|
||||
|
||||
Do not start with raw HTML or a screenshot merely to discover ordinary controls. Screenshots may
|
||||
return an inline image or a saved path depending on the deployment.
|
||||
|
||||
## Tabs and the Agent Window boundary
|
||||
|
||||
Write actions normally affect only Agent Window tabs.
|
||||
|
||||
- `browser_tab_list` lists `agent`, `user`, or `all` tabs visible to the owned session. Use
|
||||
`scope: "user"` to locate a user tab before borrowing it.
|
||||
- `browser_tab_create` creates an Agent Window tab. It is active by default; set `active: false`
|
||||
for a background tab.
|
||||
- `browser_tab_select` focuses an Agent Window tab.
|
||||
- `browser_tab_close` closes an Agent Window tab.
|
||||
- `browser_tab_borrow` moves a chosen user tab into the Agent Window before any write action.
|
||||
- `browser_tab_return` returns a borrowed tab to its original window and position. Return it as
|
||||
soon as the immediate task is finished; stopping the session is only the fallback cleanup.
|
||||
|
||||
Use only a `tabId` returned by `browser_tab_list` or `browser_tab_create`. Do not assume a tab id
|
||||
from page position or title.
|
||||
|
||||
## Navigation and lifecycle
|
||||
|
||||
- `browser_navigate` opens a URL and can wait for a requested lifecycle phase.
|
||||
- `browser_navigate_back` and `browser_navigate_forward` move one history entry.
|
||||
- `browser_reload` reloads the current tab; use `hard: true` only when bypassing cache is necessary.
|
||||
- `browser_wait_for_navigation` waits for a lifecycle event after an action that did not already
|
||||
wait. Avoid speculative waits when no navigation is expected.
|
||||
|
||||
After navigation, history traversal, or reload, discard old refs and call `browser_observe` or
|
||||
`browser_snapshot` again.
|
||||
|
||||
## Ask the user for browser help
|
||||
|
||||
Call `browser_request_help` for login, captcha, OTP, payment confirmation, consent, or another step
|
||||
that requires the user. Give a clear `prompt`; when the prompt refers to a concrete control, pass
|
||||
its fresh ref or selector in `targets` so the overlay can highlight it. Use `title` only when it
|
||||
adds clarity.
|
||||
|
||||
Optional `completionCriteria` can finish the pause automatically when the page has an explicit,
|
||||
stable success signal. It supports `any` or `all` conditions using `urlContains`, `urlMatches`,
|
||||
`selectorExists`, `selectorMissing`, `textExists`, or `textMissing`, plus `stableForMs`.
|
||||
|
||||
```text
|
||||
browser_request_help({
|
||||
session: sessionId,
|
||||
title: "Sign in required",
|
||||
prompt: "Complete sign-in, then return control after the account page appears.",
|
||||
targets: ["@e7"],
|
||||
timeoutMs: 300000,
|
||||
completionCriteria: {
|
||||
any: [{ urlContains: "/account" }, { selectorExists: "[data-testid='account-menu']" }],
|
||||
stableForMs: 1000
|
||||
}
|
||||
})
|
||||
```
|
||||
|
||||
Interpret outcomes deliberately:
|
||||
|
||||
- `continued` or `completed`: the user returned control or the explicit detector matched. Observe
|
||||
again before reasoning about the new state or using refs.
|
||||
- `cancelled`: stop the requested flow.
|
||||
- `timed_out`: report the timeout; do not repeatedly reopen the prompt.
|
||||
- `disabled`: unattended mode does not allow help; proceed only if safe or stop gracefully.
|
||||
- `navigated`: legacy outcome; do not treat navigation alone as proof of success.
|
||||
|
||||
## Debugging without page scripting
|
||||
|
||||
Use the read-only debugging tools only when relevant to the user's task:
|
||||
|
||||
- `browser_console` reads buffered console messages, browser logs, and JavaScript exceptions.
|
||||
Set `includeStack: true` when stack frames are useful.
|
||||
- `browser_network` reads buffered response and failure metadata. It does not return request or
|
||||
response headers or bodies and does not modify traffic.
|
||||
|
||||
Both tools accept `tabId`, `since`, `limit`, and `maxTextChars`. Their result returns `nextSince`;
|
||||
pass that value as the next `since` cursor instead of repeatedly requesting the same entries.
|
||||
|
||||
Arbitrary page script evaluation and interaction recording are intentionally unsupported in this
|
||||
DeepSeek Harness plugin. Do not invent tool names or route around this limitation. If the task
|
||||
strictly requires either capability, explain that it is not available in this phase.
|
||||
|
||||
## Window and device controls
|
||||
|
||||
- `browser_window_resize` changes the Agent Window's outer `width` and `height`; each dimension
|
||||
must be from 100 through 7680 CSS pixels.
|
||||
- `browser_emulate` applies a built-in device preset or explicit viewport settings to the active
|
||||
tab. Explicit `width` and `height` must be supplied together. `mobile: true` also requires both
|
||||
dimensions. Use `off: true` alone to clear emulation. Emulation is per-tab and new tabs do not
|
||||
inherit it.
|
||||
|
||||
## Tool reference
|
||||
|
||||
### Sessions
|
||||
|
||||
| Tool | Purpose |
|
||||
| --- | --- |
|
||||
| `browser_session_start` | Start an owned Agent Window session, optionally with URL, size, browser instance, focus behavior, or device preset. |
|
||||
| `browser_session_stop` | Stop an owned session and close its Agent Window. |
|
||||
| `browser_session_list` | List owned sessions and identify the current one. |
|
||||
|
||||
### Navigation and reading
|
||||
|
||||
| Tool | Purpose |
|
||||
| --- | --- |
|
||||
| `browser_navigate` | Navigate the active tab to a URL. |
|
||||
| `browser_navigate_back` | Go back one history entry. |
|
||||
| `browser_navigate_forward` | Go forward one history entry. |
|
||||
| `browser_reload` | Reload the active tab, optionally bypassing cache. |
|
||||
| `browser_wait_for_navigation` | Wait for an expected page lifecycle event. |
|
||||
| `browser_observe` | Read the primary semantic page representation with refs. |
|
||||
| `browser_snapshot` | Read the static accessibility-tree fallback with refs. |
|
||||
| `browser_get_html` | Read raw document or ref-scoped HTML as a last resort. |
|
||||
| `browser_screenshot` | Capture the visible tab or a ref-cropped element. |
|
||||
|
||||
### Interaction
|
||||
|
||||
| Tool | Purpose |
|
||||
| --- | --- |
|
||||
| `browser_click` | Click a ref or CSS selector. |
|
||||
| `browser_hover` | Hover a ref or selector to reveal conditional UI. |
|
||||
| `browser_fill` | Fill a text-capable element. |
|
||||
| `browser_select` | Select one or more option values. |
|
||||
| `browser_press` | Send a keyboard key or combo. |
|
||||
|
||||
### Tabs
|
||||
|
||||
| Tool | Purpose |
|
||||
| --- | --- |
|
||||
| `browser_tab_list` | List visible user and Agent Window tabs. |
|
||||
| `browser_tab_create` | Create an Agent Window tab. |
|
||||
| `browser_tab_select` | Focus an Agent Window tab. |
|
||||
| `browser_tab_close` | Close an Agent Window tab. |
|
||||
| `browser_tab_borrow` | Move a user tab into the Agent Window. |
|
||||
| `browser_tab_return` | Return a borrowed tab to its original window. |
|
||||
|
||||
### Assistance, diagnostics, and display
|
||||
|
||||
| Tool | Purpose |
|
||||
| --- | --- |
|
||||
| `browser_request_help` | Pause for an explicit user browser step. |
|
||||
| `browser_console` | Read buffered console and exception entries. |
|
||||
| `browser_network` | Read buffered response and failure metadata. |
|
||||
| `browser_window_resize` | Resize the Agent Window. |
|
||||
| `browser_emulate` | Apply or clear per-tab device emulation. |
|
||||
|
||||
## Error handling and final checklist
|
||||
|
||||
On a stale ref, observe again and retry once with a fresh ref. On a closed or unknown tab, list
|
||||
tabs instead of guessing. On an unknown session, list owned sessions or start a new one; never try
|
||||
foreign ids. On timeout, decide whether one longer, purposeful wait is justified. Avoid repeated
|
||||
actions with unchanged state.
|
||||
|
||||
Before finishing a browser task, verify that:
|
||||
|
||||
- The user's observable goal is met, or the blocker is clearly reported.
|
||||
- Any borrowed tab has been returned when practical.
|
||||
- The owned session has been stopped unless the user explicitly asked to keep it open.
|
||||
- No further page action is taken after success.
|
||||
@@ -0,0 +1,10 @@
|
||||
---
|
||||
|
||||
## DSH routing reminder
|
||||
|
||||
The CLI-oriented reference above describes behavior, not the execution mechanism in
|
||||
DeepSeek Harness. For every browser operation, call the injected `browser_*` model
|
||||
tool. Do not translate these examples into `bash` commands and do not execute `bsk`
|
||||
directly; the plugin invokes `bsk` internally. In particular, always begin with
|
||||
`browser_session_start`, pass its returned session id to subsequent `browser_*`
|
||||
tools, and finish with `browser_session_stop`.
|
||||
@@ -0,0 +1,71 @@
|
||||
# browser-skill (dsh edition)
|
||||
|
||||
## DSH tool-routing requirement (highest priority)
|
||||
|
||||
This skill is running inside DeepSeek Harness, where the plugin has injected
|
||||
structured `browser_*` model tools. **Always call those injected `browser_*` tools
|
||||
for browser work. Never use `bash`, `shell`, `exec`, or another tool to invoke the
|
||||
`bsk` CLI directly.** The CLI command names shown below are only conceptual
|
||||
mappings for understanding the tools; they are not instructions to run commands.
|
||||
If a `browser_*` tool is available, using the CLI directly is incorrect because it
|
||||
bypasses plugin session ownership, observation UI, tool cards, and cleanup.
|
||||
|
||||
Use the injected tool whose name matches the operation, for example:
|
||||
`browser_session_start`, `browser_navigate`, `browser_observe`, `browser_snapshot`,
|
||||
`browser_click`, `browser_fill`, `browser_press`, `browser_screenshot`, and
|
||||
`browser_session_stop`. Pass arguments as the tool schema specifies (not CLI flags).
|
||||
|
||||
Drive the user's **real Chromium browser** (logins, cookies, Agent Window isolation)
|
||||
through this plugin's `browser_*` tools. Each injected tool is implemented by the
|
||||
plugin using the `bsk` engine internally — the model must not invoke that engine
|
||||
itself. The workflow, judgment rules, and safety constraints remain identical to
|
||||
the canonical CLI skill.
|
||||
|
||||
> **Tool availability**: invoking this skill (which you just did) makes the plugin's
|
||||
> `browser_*` tools available for the rest of the session — they are injected on
|
||||
> successful skill load. Until then they are deliberately hidden, so do not attempt
|
||||
> a browser action before loading this skill.
|
||||
|
||||
## Tool ↔ CLI map
|
||||
|
||||
The tools below are thin, structured wrappers over the bsk CLI verbs of the same
|
||||
name; `session` parameters take the 4-letter session id (no `--session` flag
|
||||
spelling here):
|
||||
|
||||
| dsh tool | bsk equivalent |
|
||||
|---|---|
|
||||
| `browser_session_start` / `browser_session_stop` / `browser_session_list` | `bsk session start/stop/list` |
|
||||
| `browser_navigate` | `bsk navigate` |
|
||||
| `browser_snapshot` / `browser_observe` | `bsk snapshot` / `bsk observe` |
|
||||
| `browser_click` / `browser_fill` / `browser_press` | `bsk click` / `bsk fill` / `bsk press` |
|
||||
| `browser_screenshot` | `bsk screenshot` |
|
||||
| `browser_emulate` | `bsk emulate` |
|
||||
|
||||
(`bsk console` / `bsk network` have no tool counterpart in this plugin yet.)
|
||||
|
||||
Plugin-specific semantics that override or narrow the CLI rules below:
|
||||
|
||||
- **Owned sessions only**: this plugin sees and acts on just the sessions IT
|
||||
started via `browser_session_start`. `browser_session_list` shows only those
|
||||
(with the current marker); passing a session id created by another program
|
||||
(or person) sharing the bsk daemon is refused outright — never retry those
|
||||
with the same id.
|
||||
- **Current session**: omitted `session` parameters resolve to the most
|
||||
recently started or used OWNED session. With several sessions in flight,
|
||||
pass `session` explicitly to avoid surprises.
|
||||
- **Screenshots**: when the model route accepts images, `browser_screenshot`
|
||||
returns the PNG inline as an image attachment; otherwise it returns the
|
||||
saved file path.
|
||||
- **`browser_emulate mobile`**: requires `width`+`height` — the bsk daemon
|
||||
refuses `--mobile` without viewport dimensions.
|
||||
- **Live observation**: in the dsh Web UI, every owned session is watched by
|
||||
the floating overlay (thumbnails, action, interrupt) — observational side
|
||||
traffic is free and never steals your command turn; just keep acting.
|
||||
|
||||
The canonical CLI skill body follows verbatim (source of truth: `skill/SKILL.md`
|
||||
in the BrowserSkill repo, mirrored by build into `crates/bsk-cli/skill/SKILL.md`).
|
||||
Read `bsk <verb> --session <id>` as "the `browser_<verb>` tool with
|
||||
`session: <id>`"; everything else — when to use, the stop-when-done rules, the
|
||||
interaction loop, refs, sandbox rules — applies unchanged.
|
||||
|
||||
---
|
||||
@@ -1,108 +0,0 @@
|
||||
/**
|
||||
* Archive-triggered bsk session cleanup. DSH conversation archival writes
|
||||
* the workspace registry's durable global state, which the storage domain
|
||||
* broadcasts as `domain/changed` ({domain: 'workspace', table: ''}) with
|
||||
* the new `archivedSessionIds`. The watcher diffs that set and stops every
|
||||
* bsk session owned by a freshly archived conversation — archived sessions
|
||||
* are hidden from every surface, so their Agent Windows would otherwise
|
||||
* linger with no way back to them.
|
||||
*
|
||||
* Ownership lineage: browser_session_start records the calling agent's
|
||||
* session id PLUS every ancestor along `header.parentSession` (a subagent's
|
||||
* browsers are reaped when any ancestor is archived, the root conversation
|
||||
* included). Lineage is resolved through the host session store; an
|
||||
* unloaded ancestor simply ends the walk.
|
||||
*/
|
||||
|
||||
import type { Context } from "@deepseek-ai/cordis";
|
||||
import type { ObservationService } from "./observation";
|
||||
import type { SessionRegistry } from "./sessions";
|
||||
|
||||
/** The wire shape of a `domain/changed` frame (see dsh-storage-domain). */
|
||||
interface DomainChange {
|
||||
domain?: string;
|
||||
table?: string;
|
||||
value?: unknown;
|
||||
}
|
||||
|
||||
/** The workspace registry's durable global singleton slice we read. */
|
||||
interface WorkspaceGlobal {
|
||||
archivedSessionIds?: unknown;
|
||||
}
|
||||
|
||||
/** Structural host session store face: only the lineage read is needed. */
|
||||
interface SessionStoreLike {
|
||||
get(id: string): { header: { parentSession?: string } } | undefined;
|
||||
}
|
||||
|
||||
/** Cap on lineage walks — defensive against a malformed parent chain. */
|
||||
const MAX_LINEAGE_DEPTH = 16;
|
||||
|
||||
/**
|
||||
* The DSH session ids that own a tool call's browser sessions: the calling
|
||||
* agent's own session plus every ancestor along the seed lineage. Empty
|
||||
* when the call carried no agent identity (those sessions outlive any
|
||||
* archive cleanup by design — nothing can name their owner).
|
||||
*/
|
||||
export function ownerSessionIds(ctx: Context, agentId: string | undefined): string[] {
|
||||
if (agentId === undefined) return [];
|
||||
const store = ctx.get("sessions") as SessionStoreLike | undefined;
|
||||
const ids: string[] = [];
|
||||
let current: string | undefined = agentId;
|
||||
for (let depth = 0; current !== undefined && depth < MAX_LINEAGE_DEPTH; depth += 1) {
|
||||
if (ids.includes(current)) break; // a cycle in stored headers must not loop
|
||||
ids.push(current);
|
||||
current = store?.get(current)?.header.parentSession;
|
||||
}
|
||||
return ids;
|
||||
}
|
||||
|
||||
/**
|
||||
* Watch conversation archival and stop the bsk sessions it opened. Returns
|
||||
* the disposer (plugin unload). In compositions without the workspace
|
||||
* domain (headless), the event simply never fires — and a context without
|
||||
* the events mixin degrades to a no-op like the other optional seams.
|
||||
*/
|
||||
export function armArchiveCleanup(
|
||||
ctx: Context,
|
||||
registry: SessionRegistry,
|
||||
observation: ObservationService,
|
||||
): () => void {
|
||||
// 'domain/changed' lives outside the vendored Events type map, so the
|
||||
// listener goes through a structural view of the events mixin.
|
||||
const on = (
|
||||
ctx as { on?: (event: string, listener: (change: DomainChange) => void) => () => void }
|
||||
).on;
|
||||
if (typeof on !== "function") return () => {};
|
||||
|
||||
/** Archived ids already accounted for; lazily seeded from the registry. */
|
||||
let seen: Set<string> | undefined;
|
||||
const initialize = (): Set<string> => {
|
||||
if (seen === undefined) {
|
||||
const registryService = ctx.get("workspaceRegistry") as
|
||||
| { archivedSessionIds?: readonly string[] }
|
||||
| undefined;
|
||||
seen = new Set(registryService?.archivedSessionIds ?? []);
|
||||
}
|
||||
return seen;
|
||||
};
|
||||
|
||||
return on.call(ctx, "domain/changed", (change: DomainChange) => {
|
||||
if (change?.domain !== "workspace" || change?.table !== "") return;
|
||||
const archived = (change.value as WorkspaceGlobal | undefined)?.archivedSessionIds;
|
||||
if (!Array.isArray(archived)) return;
|
||||
const previous = initialize();
|
||||
const fresh = archived.filter(
|
||||
(id): id is string => typeof id === "string" && !previous.has(id),
|
||||
);
|
||||
seen = new Set(archived.filter((id): id is string => typeof id === "string"));
|
||||
for (const dshSessionId of fresh) {
|
||||
for (const sessionId of registry.ownedByDsh(dshSessionId)) {
|
||||
// stopSession owns the full teardown (kill in-flight tools, queue
|
||||
// the daemon stop, drop registry + observation entries); a failure
|
||||
// just leaves the session for idle timeout or unload cleanup.
|
||||
void observation.stopSession(sessionId).catch(() => {});
|
||||
}
|
||||
}
|
||||
});
|
||||
}
|
||||
@@ -30,51 +30,17 @@
|
||||
color: var(--card-foreground);
|
||||
}
|
||||
|
||||
/* Sidebar carrier: fill the better-sidebar tab area and let the body flex
|
||||
(the tab content container owns the outer sizing). */
|
||||
.sidebar-tab {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
height: 100%;
|
||||
min-height: 0;
|
||||
}
|
||||
|
||||
.sidebar-tab > * {
|
||||
flex: 1;
|
||||
min-height: 0;
|
||||
}
|
||||
|
||||
/* The BSK product mark in the sidebar tab strip (the source artwork already
|
||||
carries rounded corners — just soften the square to match). */
|
||||
.brand-icon {
|
||||
display: block;
|
||||
border-radius: 22%;
|
||||
}
|
||||
|
||||
.header {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 6px;
|
||||
padding: 6px 8px;
|
||||
cursor: move;
|
||||
user-select: none;
|
||||
touch-action: none;
|
||||
border-bottom: 1px solid var(--border);
|
||||
}
|
||||
|
||||
/* Only the floating card drags by its header — the sidebar tab does not. */
|
||||
.header[data-draggable] {
|
||||
cursor: move;
|
||||
}
|
||||
|
||||
/* In the sidebar the status row is chrome, not a card edge: drop its bottom
|
||||
border. A fixed-height bordered row can never track the shell header's
|
||||
hairline across the panel boundary (its height varies per window), and
|
||||
the BSK warm border tint clashes with the sidebar's neutral hairlines —
|
||||
borderless, the stage's own background provides the separation. */
|
||||
.sidebar-tab .header {
|
||||
border-bottom: none;
|
||||
}
|
||||
|
||||
.status-text {
|
||||
flex: 1;
|
||||
min-width: 0;
|
||||
@@ -165,12 +131,6 @@
|
||||
border-top: 1px solid var(--border);
|
||||
}
|
||||
|
||||
.actions-group {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
gap: 4px;
|
||||
}
|
||||
|
||||
.tool-wrap {
|
||||
position: relative;
|
||||
display: inline-flex;
|
||||
@@ -205,14 +165,8 @@
|
||||
color: var(--destructive);
|
||||
}
|
||||
|
||||
/* Stop session: danger-tinted like interrupt. */
|
||||
.tool-stop {
|
||||
color: var(--destructive);
|
||||
}
|
||||
|
||||
/* Hover the wrap (not just :hover on the button) so the tooltip target
|
||||
lights the icon up too. The corner resize handles sit above this bar, so
|
||||
the outer sliver of the corner-most buttons starts a resize instead. */
|
||||
/* Hover the wrap (not just :hover on the button) so the tooltip target and
|
||||
the left-corner resize overlay still light the icon up. */
|
||||
.tool-wrap:hover .tool-button:not(:disabled) {
|
||||
color: var(--foreground);
|
||||
background: var(--accent);
|
||||
@@ -227,15 +181,6 @@
|
||||
background: var(--accent);
|
||||
}
|
||||
|
||||
/* The armed (confirm) state of the stop button: solid destructive fill so
|
||||
the second click reads as final. Written at the wrap-hover specificity so
|
||||
the hover rules above never wash it out. */
|
||||
.tool-button.tool-stop-armed,
|
||||
.tool-wrap:hover .tool-button.tool-stop-armed {
|
||||
color: var(--destructive-foreground);
|
||||
background: var(--destructive);
|
||||
}
|
||||
|
||||
.hint {
|
||||
position: absolute;
|
||||
bottom: calc(100% + 4px);
|
||||
@@ -258,16 +203,13 @@
|
||||
right: 0;
|
||||
}
|
||||
|
||||
/* Invisible hit targets — no grip glyph; all four corners resize. Must sit
|
||||
above the bottom .actions bar (z-index 3): it spans the full card width
|
||||
and would otherwise swallow every pointer event aimed at the sw/se
|
||||
corners. The top corners already win because .header is not positioned. */
|
||||
/* Invisible hit targets — no grip glyph; all four corners resize. */
|
||||
.resize-handle {
|
||||
position: absolute;
|
||||
width: 16px;
|
||||
height: 16px;
|
||||
touch-action: none;
|
||||
z-index: 4;
|
||||
z-index: 2;
|
||||
}
|
||||
|
||||
.resize-handle[data-corner="nw"] {
|
||||
|
||||
@@ -3,18 +3,13 @@
|
||||
// composer's way); collapsed = a status capsule; "pop out" upgrades the same
|
||||
// content to a native Document PiP window (user gesture required by the
|
||||
// browser). Multi-session renders a meeting-style strip under the focus view.
|
||||
// When the dsh-better-sidebar plugin is installed the tracking view moves
|
||||
// into a sidebar tab instead (see observation-sidebar.tsx) and this floating
|
||||
// card hides itself via the sidebar-mode flag. Visuals follow the
|
||||
// BrowserSkill product family: @browser-skill/ui components and oklch tokens
|
||||
// on a .bsk-obs scope root, so the card reads as BSK's own surface without
|
||||
// leaking styles into (or inheriting themes from) the shell.
|
||||
// Visuals follow the BrowserSkill product family: @browser-skill/ui components
|
||||
// and oklch tokens on a .bsk-obs scope root, so the card reads as BSK's own
|
||||
// surface without leaking styles into (or inheriting themes from) the shell.
|
||||
|
||||
import { cn } from "@browser-skill/ui";
|
||||
import {
|
||||
RiArrowDownSLine,
|
||||
RiCheckLine,
|
||||
RiCloseCircleLine,
|
||||
RiErrorWarningLine,
|
||||
RiPictureInPicture2Line,
|
||||
RiPushpinFill,
|
||||
@@ -30,8 +25,6 @@ const IconPip = asIcon(RiPictureInPicture2Line);
|
||||
const IconDown = asIcon(RiArrowDownSLine);
|
||||
const IconWarn = asIcon(RiErrorWarningLine);
|
||||
const IconPin = asIcon(RiPushpinFill);
|
||||
const IconCloseSession = asIcon(RiCloseCircleLine);
|
||||
const IconCheck = asIcon(RiCheckLine);
|
||||
|
||||
import {
|
||||
type ReactNode,
|
||||
@@ -46,12 +39,11 @@ import { createPortal } from "react-dom";
|
||||
import type { SessionObservation } from "../observation";
|
||||
import css from "./ObservationOverlay.module.css";
|
||||
import type { ObservationClientStore } from "./observation-store";
|
||||
import { focusOf, statusOf, useObservationView, usePip } from "./observation-view";
|
||||
import { getSidebarMode, subscribeSidebarMode } from "./sidebar-mode";
|
||||
|
||||
// The pure view helpers live in observation-view (shared with the sidebar
|
||||
// tab); re-export so existing imports of this module keep working.
|
||||
export { focusOf, statusOf };
|
||||
/** Minimal Document PiP surface (TS lib.dom lacks it). */
|
||||
interface DocumentPip {
|
||||
requestWindow(options?: { width?: number; height?: number }): Promise<Window>;
|
||||
}
|
||||
|
||||
interface Point {
|
||||
x: number;
|
||||
@@ -69,6 +61,11 @@ const EDGE_MARGIN = 16;
|
||||
/** Default dock: top-right, just under the shell's top bar (no spacing tokens exist in dsh 0.1). */
|
||||
const TOP_OFFSET = 64;
|
||||
|
||||
function pipApi(): DocumentPip | undefined {
|
||||
if (typeof window === "undefined") return undefined;
|
||||
return (window as unknown as { documentPictureInPicture?: DocumentPip }).documentPictureInPicture;
|
||||
}
|
||||
|
||||
function clampSize(size: Size, viewport: Size): Size {
|
||||
const maxW = viewport.w * 0.8;
|
||||
const maxH = viewport.h * 0.8;
|
||||
@@ -117,6 +114,23 @@ function formatElapsed(sinceMs: number, nowMs: number): string {
|
||||
return `${mm}:${ss}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Auto-follow focus: the most recently touched session, but never yank focus
|
||||
* to a session whose latest action failed (errors flag the strip item, they
|
||||
* do not steal the stage) or one already reported dead.
|
||||
*/
|
||||
export function focusOf(sessions: readonly SessionObservation[]): SessionObservation | undefined {
|
||||
if (sessions.length === 0) return undefined;
|
||||
const byRecency = [...sessions].sort((a, b) => b.since - a.since);
|
||||
const healthy = byRecency.find((s) => s.lastError === undefined && s.dead !== true);
|
||||
return healthy ?? byRecency[0];
|
||||
}
|
||||
|
||||
export function statusOf(obs: SessionObservation): "active" | "idle" | "error" {
|
||||
if (obs.lastError !== undefined && obs.action === "idle") return "error";
|
||||
return obs.action === "idle" ? "idle" : "active";
|
||||
}
|
||||
|
||||
/** Compact toolbar icon: no label, hover bubble for the name / semantics. */
|
||||
function IconAction(props: {
|
||||
label: string;
|
||||
@@ -152,83 +166,6 @@ function IconAction(props: {
|
||||
);
|
||||
}
|
||||
|
||||
/** How long the stop button stays armed before the confirm click expires. */
|
||||
const STOP_ARM_MS = 3000;
|
||||
|
||||
/**
|
||||
* Stop-session button with a lightweight two-click confirm: the first click
|
||||
* arms the button (solid red, check icon, short expiry), the second executes
|
||||
* the stop. No dialog, no layout shift — and a stray single click can never
|
||||
* close an Agent Window.
|
||||
*/
|
||||
function StopSessionAction(props: {
|
||||
sessionId: string | undefined;
|
||||
onStop: (sessionId: string) => Promise<boolean>;
|
||||
}) {
|
||||
const { sessionId, onStop } = props;
|
||||
const [hover, setHover] = useState(false);
|
||||
const [armed, setArmed] = useState(false);
|
||||
const [stopping, setStopping] = useState(false);
|
||||
|
||||
useEffect(() => {
|
||||
if (!armed) return;
|
||||
const timer = setTimeout(() => setArmed(false), STOP_ARM_MS);
|
||||
return () => clearTimeout(timer);
|
||||
}, [armed]);
|
||||
|
||||
// The focused session can vanish mid-arm (stopped elsewhere): disarm.
|
||||
useEffect(() => {
|
||||
if (sessionId === undefined) setArmed(false);
|
||||
}, [sessionId]);
|
||||
|
||||
const disabled = sessionId === undefined || stopping;
|
||||
const label = stopping
|
||||
? "Stopping session…"
|
||||
: armed
|
||||
? `Confirm stop session ${sessionId ?? ""}`
|
||||
: `Stop session ${sessionId ?? ""}`;
|
||||
const hint = stopping
|
||||
? "Closing the Agent Window…"
|
||||
: armed
|
||||
? `Click again to stop session ${sessionId ?? ""} and close its Agent Window.`
|
||||
: `Stop session ${sessionId ?? ""} and close its Agent Window.`;
|
||||
|
||||
return (
|
||||
<span
|
||||
className={css["tool-wrap"]}
|
||||
onPointerEnter={() => setHover(true)}
|
||||
onPointerLeave={() => setHover(false)}
|
||||
>
|
||||
<button
|
||||
type="button"
|
||||
className={cn(css["tool-button"], css["tool-stop"], armed && css["tool-stop-armed"])}
|
||||
disabled={disabled}
|
||||
aria-label={label}
|
||||
aria-pressed={armed}
|
||||
data-armed={armed || undefined}
|
||||
onClick={() => {
|
||||
if (sessionId === undefined || stopping) return;
|
||||
if (!armed) {
|
||||
setArmed(true);
|
||||
return;
|
||||
}
|
||||
setArmed(false);
|
||||
setStopping(true);
|
||||
const target = sessionId;
|
||||
void onStop(target).finally(() => setStopping(false));
|
||||
}}
|
||||
>
|
||||
{armed ? <IconCheck size={16} /> : <IconCloseSession size={16} />}
|
||||
</button>
|
||||
{hover ? (
|
||||
<span className={css.hint} role="tooltip">
|
||||
{hint}
|
||||
</span>
|
||||
) : null}
|
||||
</span>
|
||||
);
|
||||
}
|
||||
|
||||
/** Flat status dot, specced after the BSK popup's ConnectionStatusIndicator. */
|
||||
function StatusDot({ state }: { state: "active" | "idle" | "error" | "dead" }) {
|
||||
const color =
|
||||
@@ -309,8 +246,8 @@ function StripItem(props: {
|
||||
);
|
||||
}
|
||||
|
||||
/** The floating card / PiP / sidebar-tab shared content. */
|
||||
export function OverlayBody(props: {
|
||||
/** The floating card / PiP shared content. */
|
||||
function OverlayBody(props: {
|
||||
store: ObservationClientStore;
|
||||
focus: SessionObservation | undefined;
|
||||
sessions: readonly SessionObservation[];
|
||||
@@ -373,7 +310,6 @@ export function OverlayBody(props: {
|
||||
<div
|
||||
className={css.header}
|
||||
data-testid="obs-header"
|
||||
data-draggable={onHeaderPointerDown !== undefined || undefined}
|
||||
onPointerDown={onHeaderPointerDown}
|
||||
role="presentation"
|
||||
>
|
||||
@@ -427,21 +363,15 @@ export function OverlayBody(props: {
|
||||
</div>
|
||||
) : null}
|
||||
<div className={css.actions}>
|
||||
<div className={css["actions-group"]}>
|
||||
<IconAction
|
||||
label={interrupting ? "Interrupting…" : "Interrupt the current browser action"}
|
||||
hint="Stop the current browser action."
|
||||
disabled={!canInterrupt}
|
||||
danger
|
||||
onClick={onInterrupt}
|
||||
>
|
||||
<IconStop size={16} />
|
||||
</IconAction>
|
||||
<StopSessionAction
|
||||
sessionId={focus?.sessionId}
|
||||
onStop={(sessionId) => store.stopSession(sessionId)}
|
||||
/>
|
||||
</div>
|
||||
<IconAction
|
||||
label={interrupting ? "Interrupting…" : "Interrupt the current browser action"}
|
||||
hint="Stop the current browser action."
|
||||
disabled={!canInterrupt}
|
||||
danger
|
||||
onClick={onInterrupt}
|
||||
>
|
||||
<IconStop size={16} />
|
||||
</IconAction>
|
||||
{onPopOut !== undefined ? (
|
||||
<IconAction
|
||||
label="Pop out into a mini window"
|
||||
@@ -457,17 +387,25 @@ export function OverlayBody(props: {
|
||||
);
|
||||
}
|
||||
|
||||
/** Clone the host document's style/link nodes into a PiP window. */
|
||||
function cloneStylesInto(pipWindow: Window): void {
|
||||
for (const node of document.querySelectorAll('link[rel="stylesheet"], style')) {
|
||||
pipWindow.document.head.appendChild(node.cloneNode(true));
|
||||
}
|
||||
}
|
||||
|
||||
export function ObservationOverlay({ store }: { store: ObservationClientStore }) {
|
||||
const { snapshot, focus, pinnedId, onTogglePin, now } = useObservationView(store);
|
||||
// While the better-sidebar plugin carries the tracking view, this floating
|
||||
// card (and its capsule) stays out of the way; an already-open PiP window
|
||||
// keeps its portal until the user closes it.
|
||||
const sidebarMode = useSyncExternalStore(subscribeSidebarMode, getSidebarMode);
|
||||
const snapshot = useSyncExternalStore(store.subscribe, store.getSnapshot);
|
||||
useEffect(() => {
|
||||
store.start();
|
||||
return () => store.stop();
|
||||
}, [store]);
|
||||
|
||||
const [collapsed, setCollapsed] = useState(false);
|
||||
const [pos, setPos] = useState<Point | null>(null);
|
||||
const [size, setSize] = useState<Size>(DEFAULT_SIZE);
|
||||
const { pipWindow, pipSupported, popOut } = usePip();
|
||||
const [pipWindow, setPipWindow] = useState<Window | null>(null);
|
||||
const [pinnedId, setPinnedId] = useState<string | null>(null);
|
||||
const dragRef = useRef<{
|
||||
kind: "move" | "resize";
|
||||
corner?: ResizeCorner;
|
||||
@@ -476,6 +414,23 @@ export function ObservationOverlay({ store }: { store: ObservationClientStore })
|
||||
base: Point & Size;
|
||||
} | null>(null);
|
||||
|
||||
// Elapsed-time ticker: 1s while anything is active.
|
||||
const anyActive = snapshot.sessions.some((s) => s.action !== "idle");
|
||||
const [now, setNow] = useState(() => Date.now());
|
||||
useEffect(() => {
|
||||
if (!anyActive) return;
|
||||
const timer = setInterval(() => setNow(Date.now()), 1000);
|
||||
return () => clearInterval(timer);
|
||||
}, [anyActive]);
|
||||
|
||||
// Focus: the pinned session wins while it still exists; otherwise auto-follow.
|
||||
const pinned =
|
||||
pinnedId !== null ? snapshot.sessions.find((s) => s.sessionId === pinnedId) : undefined;
|
||||
const focus = pinned ?? focusOf(snapshot.sessions);
|
||||
const onTogglePin = useCallback((sessionId: string) => {
|
||||
setPinnedId((current) => (current === sessionId ? null : sessionId));
|
||||
}, []);
|
||||
|
||||
const viewport = (): Size => ({ w: window.innerWidth, h: window.innerHeight });
|
||||
|
||||
const onPointerMove = useCallback((event: PointerEvent) => {
|
||||
@@ -556,21 +511,34 @@ export function ObservationOverlay({ store }: { store: ObservationClientStore })
|
||||
document.addEventListener("pointerup", onPointerUp);
|
||||
};
|
||||
|
||||
const popOut = async (): Promise<void> => {
|
||||
const pip = pipApi();
|
||||
if (pip === undefined) return;
|
||||
try {
|
||||
const win = await pip.requestWindow({ width: size.w, height: size.h });
|
||||
cloneStylesInto(win);
|
||||
win.addEventListener("pagehide", () => setPipWindow(null));
|
||||
setPipWindow(win);
|
||||
} catch {
|
||||
// requestWindow rejects without a user gesture (or when the window was
|
||||
// closed mid-request): stay on the in-page card, no state change.
|
||||
}
|
||||
};
|
||||
|
||||
// Hidden while no owned session exists (and no PiP is up).
|
||||
if (snapshot.sessions.length === 0 && pipWindow === null) return null;
|
||||
|
||||
const body = (
|
||||
<OverlayBody
|
||||
store={store}
|
||||
focus={focus}
|
||||
sessions={snapshot.sessions}
|
||||
available={snapshot.available}
|
||||
pinnedId={pinnedId}
|
||||
pinnedId={pinned !== undefined ? pinnedId : null}
|
||||
onTogglePin={onTogglePin}
|
||||
now={now}
|
||||
inPip={pipWindow !== null}
|
||||
onPopOut={
|
||||
pipWindow === null && pipSupported
|
||||
? () => popOut({ width: size.w, height: size.h })
|
||||
: undefined
|
||||
}
|
||||
onPopOut={pipWindow === null && pipApi() !== undefined ? () => void popOut() : undefined}
|
||||
onCollapse={pipWindow === null ? () => setCollapsed(true) : undefined}
|
||||
onHeaderPointerDown={pipWindow === null ? beginMove : undefined}
|
||||
/>
|
||||
@@ -580,12 +548,6 @@ export function ObservationOverlay({ store }: { store: ObservationClientStore })
|
||||
return createPortal(body, pipWindow.document.body);
|
||||
}
|
||||
|
||||
// The sidebar tab is the carrier now — no floating card, no capsule.
|
||||
if (sidebarMode) return null;
|
||||
|
||||
// Hidden while no owned session exists (and no PiP is up).
|
||||
if (snapshot.sessions.length === 0) return null;
|
||||
|
||||
if (collapsed) {
|
||||
const state = focus !== undefined ? statusOf(focus) : "idle";
|
||||
return (
|
||||
|
||||
@@ -1,8 +0,0 @@
|
||||
/**
|
||||
* The BrowserSkill product mark (apps/extension/assets/logo.png, downscaled
|
||||
* to 32px and inlined): the sidebar tab icon, so the tracking view reads as
|
||||
* BSK's own surface next to better-sidebar's built-in "browser" tab.
|
||||
* Regenerate with: resize the source to 32x32 PNG and replace the payload.
|
||||
*/
|
||||
export const BSK_LOGO_URL =
|
||||
"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAYAAABzenr0AAAJGklEQVR42m2X248c2V3HP79zTlV19/TcPOOZWY+vsdmQFdngTQJswgs3hQ15iCJslAey+8xDAIHgH+CBNxIpj0gkEXnAKyKQWITEJlKWXSssEMfJRmuMHa/W9njHMz3Tl+nprsv5/XiomlugpdN16lR1n9/1e75fAbAbeLlO3Pz2ly4vjO7+sRbF58oiP49FEREOPwaIYQjS3IvUy0fP66kAZoYJIM7SpPVY0uy14cyzXzn9u9+4c+MG/vp1otiNa16uvxr7X/+1L3bK3lcTyU9P80ilR/uKCGYHOxxd6s2kmdaGIVYb1qyY1e8HD63UUblstwirf9J95fW/sRvXvAD0v/6Z6/O68Xf5/pS8kigiTsTkuCfIwfzAxQN37chjaAyt594JIXE4gbKMhog6Nd/qthi7lVe6L3/nG7Jz4/fPJ9u3b4U4WSwiKog/EcsTLh+PSuO3GXYs/D44QnBgyniQs7OV0+8bOxPPM6c9l660VEyF0Nmrlp9/IYT+/S/PUJ4a5C46EW8HnjVhBKjXaHJfz9Xq95xzhMThHZR5xXBrwtPNnP6eRzvzhJWzrF1d5UI759//9iYra5lrt1ycDeXsaOf+n4Y4HX8+r9SciasLS/6P12LHoyF47wipYKpM9gqePpyyvVMxjBn+1DILH17l0rkZlhYcLetT3n/A5FGfTtvTCh4ibjpVI81/J0il62V1rNStLp6j4jowxwiJh2iMevtsbU3pjYQim6Nz9hJLH13i8mrGbJgSxjuUvQfEjT2GOw4Zddh0BakPtFPH/jRKqaCSrwetKi9mKIIYhyFuOg6TurCcF7ae5ty5t48/vcTiz3+Ei+szLC06WnGA7TykfNCnmObk5nAtj+53yDcTZteMwZOChbkMVa3/14DCJBBBFBxyopndUUBqwwxu/XjAJ778Bc4ul+R3fgyj9yg/GDMpFZxHgoe0hQ9QDRzFI48zqHzJeKhcvNKiLAyiQ1BwgsOcqNZ9Xo96rgaVgmqdiBACH740y/1/+j4b7w1x80tUkwLM4VptJAQMcN6o+p7iYQIqhBQmsUIrR7cd0NI12ODAwJnWTpvZoRGqdRrS1CECeR7Z2NzHRBjeecgbX/kut39UUlx5ATe3gFUlAM5Tb/4oHCJl0oadUcFsK8E5sFh7ZaqgEFBr3KyhxAySxDMplDv3BuztlmhhZC6QqONDrXkW28Kbf/1d7A9+i6vPr7N/e5vQDZQ7jvxxUqdPwFRwHaP3sGR1boZYGabWwHdd7MHMMBUMh5mRJp6tnZx33tllJbS53J1jZj6BKpKPpkhZ8e5mj3Nf/FV+8ZOnmNz6Pr6TUvUcxeOAc0cN6xwUoWIyUhZXMqqq6bKmrQ0hWBQzdXX+nGOwV3LnJ30+trjE8mxGUUVG/Qn7owkt7/hpb5fwmat84qXz5P95EwmOqh8oNkKTSkFqICTJYDgpCOppZYGyjIfgdoApQRScCtEgDZ5b93dxuWd3OuWD8T5pqbSLyEyWcHdrh+S3r/Kpz15i/z/eQhIh9hOKjQRTyBYgFhAndSpDW+gNC+ZbGR6jjM3BVsNsXQNqhqpiOPIicuXcLFvdKT7xtNXTG+X0Bopt9Wi9dJUXX7rI5O23kNQRB4F8I0CEzgr0/B7t2YB/2iJOQDrK8HHJ+dkZqqI+JfUAYRXMGc6pM1MBBa2Ubttz5ews51c6XFjr8OJHT5N0HPIbz/Ppz11h/+03IXVUg4T8ccAidFaNp7LHzdd2uXdvj866gQq5L8nHMD+TUJUG0aAyJBqidZQcUfFqeK2vlEo1rYh5iUXl6eMB5eoKv37tOUb/9q+4LBD7gXKj3ry7Ak/iHrdeH/ArC7P07ykPekMWzkFvkNMmkHkgagNoDciogYKzaGg0YgSNYLEJj0ndfiGj6o/Z3i5or5+j2BTKJwlawcwKbMQ9fvK9Ib+QtgitQMg8d384wVYLdocls1mCad1+Fg1TDgeV4UxFTB2ow44NjQ5rwcwZ+MjqmLe/dZOBLOHHCeUUZteED+Ie73xvyHNpmyx1TERozXieW1/mO/+yzWhTWVtsU5aCqKvRT4+GmRDEHGJ6yPPMBHNGeqbCdwqs02X9l14g/a8H3PzWm/zyp1ZZXgn8tDfgnbeGPN/qIEDSyhh7w2OcPZWRsESWCq3EoVFPksZDMuUIps7qLjDUQBxkZ0tcu8KSGdIXPo3OLbOSdfj4YMIPbg6ZXxux9wRmWimFKe0k0J7v8H5vQHchpSoiC536bKhKEDlCJzM7JLJqhqtxoEYoUcEFw/Yduh8Jl38OP7eAlBOm29ssziV87JlluqMOL15ZZWVlhu0yMrfYxQQmZUU3TbCmnog1mRE1xOp9xOoaEwVRI9QnUGOECLoPNlcRlucJ6xeIgx3K/3kX23pINWnRdnBxtYNWkbkkMJ5JSLKEwaSgRFlsJWh1mNBDfsEBzZGaYIiBIgRtImCmmArhlBFOR1Q9xZ3b6OOHWFnifEq1U/84TpXgHL4hUh7hv3t91pY6eCeUpTYs2o6RisYEO6B5hpoSRCWaanNAGP5UVZ+r+YB4dxcbZ4QVx/Q9wSYg/uDYNvKyoorwgyfbtDuBC6c65HmFa5Jsh56f3NgO4iGeYC68n8Ty2WkphAUVv2hoLuggoXgUcJkRh0rVF7wHi3Vw1YxuSMh1zMpcmw8tdynzCmmCL1JjzQG3lpNU37xAdNnjYC79xywUfzbJNboEX214iqeeuN/IrkKw6BAvNUiJ1a2K0HKOT144jQBFoTWRayrc9LhWa1TTUTNqJ038wCf/LO//xZfWWw9u/zCt8uWpSiTixZ3UeJzQPkerdkyq1RryZwVN87CJBALR0ExULMtG4/VnPy4AG3/4m19YGG/9fT7NmZpEhzg5lCNHOo//774u7UYp2WEt1XrSftYMTcF32ynbrfmXz3ztjW86u3bNn/nq69/eaZ/6PUlbm/PB+YSapUozMG2uB0M5fI7WSKoKasjBu6qHczEjwWQ2iJc03XmaLb5y5mtvfNOuNeLUruHlVeK7f/7yxbXBu39U5sXnq7I8L2pH6rwRgCZyUpeKnZSOB5K2EbT1l5hPk0c+zV7bXXzmry7/5T/cvXENf/1V4v8CSYIrURx3kHgAAAAASUVORK5CYII=";
|
||||
@@ -1,9 +1,7 @@
|
||||
/**
|
||||
* dsh-plugin-browserskill browser half: the `browser_screenshot` keyed
|
||||
* toolview plus the observation overlay (live thumbnails + interrupt). The
|
||||
* default carrier is a floating card on the `shell.overlay` seat; when the
|
||||
* dsh-better-sidebar plugin provides its `betterSidebar` service, the view
|
||||
* moves into a sidebar tab instead (see observation-sidebar.tsx).
|
||||
* toolview plus the observation overlay (live thumbnails + interrupt) floating
|
||||
* over the shell via the `shell.overlay` seat.
|
||||
*/
|
||||
|
||||
import type { ImageAttachmentRef } from "@deepseek-ai/dsh-attachment";
|
||||
@@ -17,7 +15,6 @@ import { createElement } from "react";
|
||||
import "./bsk-tokens.nomodule.css";
|
||||
import "./bsk-ui.nomodule.css";
|
||||
import { ObservationOverlay } from "./ObservationOverlay";
|
||||
import { type BetterSidebarLike, registerObservationSidebar } from "./observation-sidebar";
|
||||
import { type EventSourceLike, ObservationClientStore } from "./observation-store";
|
||||
import { ScreenshotToolView } from "./ScreenshotToolView";
|
||||
|
||||
@@ -84,14 +81,4 @@ export function apply(ctx: ClientContext): void {
|
||||
createElement(ObservationOverlay, { store }),
|
||||
),
|
||||
);
|
||||
// Optional carrier upgrade: when the dsh-better-sidebar plugin is installed,
|
||||
// its service moves the tracking view into a sidebar tab (the floating
|
||||
// overlay hides itself through the sidebar-mode flag). In profiles without
|
||||
// the sidebar plugin this fiber never runs and nothing changes.
|
||||
ctx.inject(["betterSidebar"], (injected) =>
|
||||
registerObservationSidebar(
|
||||
(injected as unknown as { betterSidebar: BetterSidebarLike }).betterSidebar,
|
||||
store,
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
@@ -1,256 +0,0 @@
|
||||
// better-sidebar carrier for the observation view. When the
|
||||
// dsh-better-sidebar plugin is installed its client publishes a
|
||||
// `betterSidebar` cordis service; the client entry then runs the
|
||||
// registration below and the tracking view moves from the floating card
|
||||
// into a single-instance sidebar tab (Document PiP pop-out stays available
|
||||
// from inside the tab). Detection is purely service-based — profiles
|
||||
// without the sidebar plugin never start this fiber and keep the floating
|
||||
// overlay.
|
||||
|
||||
import { createElement, type ReactNode } from "react";
|
||||
import { createPortal } from "react-dom";
|
||||
import { BSK_LOGO_URL } from "./brand-icon";
|
||||
import { OverlayBody } from "./ObservationOverlay";
|
||||
import css from "./ObservationOverlay.module.css";
|
||||
import type { ObservationClientStore } from "./observation-store";
|
||||
import { useObservationView, usePip, visibleToScope } from "./observation-view";
|
||||
import { setSidebarMode } from "./sidebar-mode";
|
||||
|
||||
/** The tab title — "Browser Skill", distinct from the sidebar's built-in "browser" tab. */
|
||||
const TAB_TITLE = "Browser Skill";
|
||||
|
||||
/** Tab strip icon: the BrowserSkill product mark at the requested size. */
|
||||
function TabIcon({ size }: { size: number }) {
|
||||
return (
|
||||
<img
|
||||
src={BSK_LOGO_URL}
|
||||
width={size}
|
||||
height={size}
|
||||
alt=""
|
||||
aria-hidden
|
||||
className={css["brand-icon"]}
|
||||
/>
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Structural mirrors of dsh-better-sidebar's client service surface (only
|
||||
* the slices this integration touches — see the upstream
|
||||
* lib/types/client/service.d.ts). Declared locally so the plugin carries no
|
||||
* dependency on the sidebar package; the service contract has been stable
|
||||
* since v0.4.0 and newer capabilities arrive behind its `features` list.
|
||||
*/
|
||||
export interface SidebarTabLike {
|
||||
id: string;
|
||||
type: string;
|
||||
title: string;
|
||||
}
|
||||
|
||||
export interface SidebarLeafLike {
|
||||
kind: "leaf";
|
||||
id: string;
|
||||
tabs: SidebarTabLike[];
|
||||
active: string | null;
|
||||
}
|
||||
|
||||
export interface SidebarSplitLike {
|
||||
kind: "split";
|
||||
id: string;
|
||||
dir: "row" | "col";
|
||||
sizes: number[];
|
||||
children: SidebarNodeLike[];
|
||||
}
|
||||
|
||||
export type SidebarNodeLike = SidebarLeafLike | SidebarSplitLike;
|
||||
|
||||
export interface SidebarStateLike {
|
||||
splits: SidebarNodeLike;
|
||||
bottomSplits: SidebarNodeLike;
|
||||
/** Whether the right panel is expanded (the merged drawer on narrow screens). */
|
||||
panelOpen?: boolean;
|
||||
}
|
||||
|
||||
export interface SidebarSnapshotLike {
|
||||
sessionId?: string;
|
||||
state?: SidebarStateLike;
|
||||
}
|
||||
|
||||
/** Props every tab component receives (the slices we read). */
|
||||
export interface TabComponentPropsLike {
|
||||
/** The conversation this sidebar instance belongs to. */
|
||||
scope: { sessionId: string };
|
||||
}
|
||||
|
||||
export interface TabDescriptorLike {
|
||||
id: string;
|
||||
title: string | (() => string);
|
||||
icon?: ReactNode | ((size: number) => ReactNode);
|
||||
order?: number;
|
||||
/** Single-instance: opening focuses the existing tab instead of duplicating. */
|
||||
single?: boolean;
|
||||
/** Small pill on the tab strip; null/undefined hides it. */
|
||||
badge?: (
|
||||
ctx: unknown,
|
||||
scope: { sessionId: string },
|
||||
state: unknown,
|
||||
) => string | number | null | undefined;
|
||||
component: (props: TabComponentPropsLike) => ReactNode;
|
||||
}
|
||||
|
||||
export interface BetterSidebarLike {
|
||||
registerTab(descriptor: TabDescriptorLike): () => void;
|
||||
openTab(seed: {
|
||||
type: string;
|
||||
title?: string;
|
||||
/** Content seed (lands on tab.path); content opens expand the panel. */
|
||||
path?: string;
|
||||
}): void;
|
||||
getSnapshot(): SidebarSnapshotLike;
|
||||
/** Sidebar state feed (v0.12+): session switches, state and prefs changes. */
|
||||
subscribeState?: (listener: () => void) => () => void;
|
||||
isTabEnabled(id: string): boolean;
|
||||
}
|
||||
|
||||
/** The registered tab type id (also the SidebarTab.type value). */
|
||||
export const OBSERVATION_TAB_TYPE = "browserskill:observation";
|
||||
|
||||
/**
|
||||
* Inert content seed carried on auto-opened tabs: its mere presence makes
|
||||
* the sidebar treat the open as a content open (expanding the hosting panel
|
||||
* so the tracking view lands in sight). Never read by our component.
|
||||
*/
|
||||
export const OBSERVATION_TAB_PATH = "browser-skill:observation";
|
||||
|
||||
/**
|
||||
* The observation tab body: the same OverlayBody the floating card renders,
|
||||
* minus the card chrome (no drag header, no collapse — the sidebar tab bar
|
||||
* owns those), plus the PiP pop-out upgrade. The view is scoped to the
|
||||
* sidebar's conversation: only browser sessions started by it (or its
|
||||
* descendants) show here — the floating card keeps the global view.
|
||||
*/
|
||||
export function ObservationSidebarTab({
|
||||
store,
|
||||
scopeId,
|
||||
}: {
|
||||
store: ObservationClientStore;
|
||||
scopeId: string;
|
||||
}) {
|
||||
const { snapshot, focus, pinnedId, onTogglePin, now } = useObservationView(store, scopeId);
|
||||
const { pipWindow, pipSupported, popOut } = usePip();
|
||||
|
||||
const body = (
|
||||
<OverlayBody
|
||||
store={store}
|
||||
focus={focus}
|
||||
sessions={snapshot.sessions}
|
||||
available={snapshot.available}
|
||||
pinnedId={pinnedId}
|
||||
onTogglePin={onTogglePin}
|
||||
now={now}
|
||||
inPip={pipWindow !== null}
|
||||
onPopOut={pipWindow === null && pipSupported ? () => popOut() : undefined}
|
||||
/>
|
||||
);
|
||||
|
||||
if (pipWindow !== null) {
|
||||
return createPortal(body, pipWindow.document.body);
|
||||
}
|
||||
return <div className={css["sidebar-tab"]}>{body}</div>;
|
||||
}
|
||||
|
||||
function* leafNodes(node: SidebarNodeLike): Generator<SidebarLeafLike> {
|
||||
if (node.kind === "leaf") {
|
||||
yield node;
|
||||
return;
|
||||
}
|
||||
for (const child of node.children) yield* leafNodes(child);
|
||||
}
|
||||
|
||||
/** Whether a tab of our type is already open in either sidebar workbench. */
|
||||
export function observationTabOpen(state: SidebarStateLike | undefined): boolean {
|
||||
if (state === undefined) return false;
|
||||
for (const root of [state.splits, state.bottomSplits]) {
|
||||
for (const leaf of leafNodes(root)) {
|
||||
if (leaf.tabs.some((tab) => tab.type === OBSERVATION_TAB_TYPE)) return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* Register the observation sidebar tab and flip the carrier flag. Returns
|
||||
* the disposer the cordis fiber invokes when the sidebar service goes away
|
||||
* (plugin unload/HMR): the floating overlay then resumes as the carrier.
|
||||
*/
|
||||
export function registerObservationSidebar(
|
||||
service: BetterSidebarLike,
|
||||
store: ObservationClientStore,
|
||||
): () => void {
|
||||
setSidebarMode(true);
|
||||
// Hold the feed for the whole sidebar lifetime so the auto-open watcher
|
||||
// sees new sessions even while the tab itself is closed.
|
||||
store.acquire();
|
||||
const disposeTab = service.registerTab({
|
||||
id: OBSERVATION_TAB_TYPE,
|
||||
title: TAB_TITLE,
|
||||
icon: (size: number) => createElement(TabIcon, { size }),
|
||||
single: true,
|
||||
// The badge counts only sessions visible to the tab strip's own
|
||||
// conversation (global sessions from other conversations stay hidden,
|
||||
// mirroring the tab body's scoped view).
|
||||
badge: (_ctx, scope) => {
|
||||
const count = store
|
||||
.getSnapshot()
|
||||
.sessions.filter((s) => visibleToScope(s, scope.sessionId)).length;
|
||||
return count > 0 ? count : null;
|
||||
},
|
||||
component: (props) =>
|
||||
createElement(ObservationSidebarTab, { store, scopeId: props.scope.sessionId }),
|
||||
});
|
||||
|
||||
// Auto-open the tab when a browser session VISIBLE TO the active
|
||||
// conversation appears (and right away when one is already live). The
|
||||
// evaluation runs on observation publishes AND on sidebar state changes:
|
||||
// the latter covers the sidebar store's async init (its state is
|
||||
// undefined for the first beats after page load) and conversation
|
||||
// switches. The open carries a content seed (`path`): only content opens
|
||||
// land in sight — the sidebar expands the hosting panel for them. While
|
||||
// the panel is OPEN an existing tab is never re-focused — the user may
|
||||
// be reading another page on purpose; while it is collapsed a NEW
|
||||
// session (0→N) nudges the tab back into sight, mirroring how the
|
||||
// floating card used to reappear.
|
||||
const activeVisibleCount = (): number => {
|
||||
const activeId = service.getSnapshot().sessionId;
|
||||
if (activeId === undefined) return 0;
|
||||
return store.getSnapshot().sessions.filter((s) => visibleToScope(s, activeId)).length;
|
||||
};
|
||||
let previousVisible = activeVisibleCount();
|
||||
const evaluate = (): void => {
|
||||
const count = activeVisibleCount();
|
||||
const { state, sessionId } = service.getSnapshot();
|
||||
if (
|
||||
state !== undefined &&
|
||||
sessionId !== undefined &&
|
||||
service.isTabEnabled(OBSERVATION_TAB_TYPE)
|
||||
) {
|
||||
const open = observationTabOpen(state);
|
||||
if (count > 0 && !open) {
|
||||
service.openTab({ type: OBSERVATION_TAB_TYPE, path: OBSERVATION_TAB_PATH });
|
||||
} else if (previousVisible === 0 && count > 0 && open && state.panelOpen === false) {
|
||||
service.openTab({ type: OBSERVATION_TAB_TYPE, path: OBSERVATION_TAB_PATH });
|
||||
}
|
||||
}
|
||||
previousVisible = count;
|
||||
};
|
||||
evaluate();
|
||||
const unsubscribe = store.subscribe(evaluate);
|
||||
const unsubscribeState = service.subscribeState?.(evaluate);
|
||||
|
||||
return () => {
|
||||
unsubscribe();
|
||||
unsubscribeState?.();
|
||||
disposeTab();
|
||||
store.release();
|
||||
setSidebarMode(false);
|
||||
};
|
||||
}
|
||||
@@ -53,7 +53,6 @@ export interface OverlaySnapshot {
|
||||
const STATE_URL = "/bsk-observation/state";
|
||||
const EVENTS_URL = "/bsk-observation/events";
|
||||
const INTERRUPT_URL = "/bsk-observation/interrupt";
|
||||
const STOP_URL = "/bsk-observation/stop";
|
||||
|
||||
function revoke(url: string | undefined): void {
|
||||
if (url !== undefined && typeof URL.revokeObjectURL === "function") {
|
||||
@@ -79,8 +78,6 @@ export class ObservationClientStore {
|
||||
};
|
||||
private available = true;
|
||||
private started = false;
|
||||
/** Refcount of mounted consumers (overlay card, sidebar tab, sidebar fiber). */
|
||||
private consumers = 0;
|
||||
|
||||
constructor(private readonly deps: ObservationClientDeps) {}
|
||||
|
||||
@@ -192,23 +189,6 @@ export class ObservationClientStore {
|
||||
this.publish();
|
||||
}
|
||||
|
||||
/**
|
||||
* Hold the feed for one consumer's lifetime: the stream starts with the
|
||||
* first holder and stops with the last release. Several carriers can share
|
||||
* the store (the floating card, the better-sidebar tab, and the sidebar
|
||||
* integration fiber) without one unmount killing the others' updates.
|
||||
*/
|
||||
acquire(): void {
|
||||
this.consumers += 1;
|
||||
if (this.consumers === 1) this.start();
|
||||
}
|
||||
|
||||
release(): void {
|
||||
if (this.consumers === 0) return;
|
||||
this.consumers -= 1;
|
||||
if (this.consumers === 0) this.stop();
|
||||
}
|
||||
|
||||
/** Forget one session's tracked + held frames, revoking their blob URLs. */
|
||||
private dropThumb(sessionId: string): void {
|
||||
const attachmentId = this.thumbBySession.get(sessionId);
|
||||
@@ -312,23 +292,4 @@ export class ObservationClientStore {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Stop one session and close its Agent Window. The session's removal
|
||||
* arrives through the SSE remove event — no local state is touched here.
|
||||
*/
|
||||
async stopSession(sessionId: string): Promise<boolean> {
|
||||
try {
|
||||
const res = await this.deps.fetchFn(STOP_URL, {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify({ sessionId }),
|
||||
});
|
||||
if (!res.ok) return false;
|
||||
const body = (await res.json()) as { stopped?: boolean };
|
||||
return body.stopped === true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,148 +0,0 @@
|
||||
/**
|
||||
* Shared view logic for the observation carriers (the floating overlay card
|
||||
* and the better-sidebar tab): the store-backed view model (snapshot, focus
|
||||
* pinning, elapsed ticker) and the Document PiP pop-out. Extracted from
|
||||
* ObservationOverlay so both carriers run the same focus/interrupt behavior
|
||||
* without duplicating hooks.
|
||||
*/
|
||||
|
||||
import { useCallback, useEffect, useState, useSyncExternalStore } from "react";
|
||||
import type { SessionObservation } from "../observation";
|
||||
import type { ObservationClientStore, OverlaySnapshot } from "./observation-store";
|
||||
|
||||
/**
|
||||
* Auto-follow focus: the most recently touched session, but never yank focus
|
||||
* to a session whose latest action failed (errors flag the strip item, they
|
||||
* do not steal the stage) or one already reported dead.
|
||||
*/
|
||||
export function focusOf(sessions: readonly SessionObservation[]): SessionObservation | undefined {
|
||||
if (sessions.length === 0) return undefined;
|
||||
const byRecency = [...sessions].sort((a, b) => b.since - a.since);
|
||||
const healthy = byRecency.find((s) => s.lastError === undefined && s.dead !== true);
|
||||
return healthy ?? byRecency[0];
|
||||
}
|
||||
|
||||
export function statusOf(obs: SessionObservation): "active" | "idle" | "error" {
|
||||
if (obs.lastError !== undefined && obs.action === "idle") return "error";
|
||||
return obs.action === "idle" ? "idle" : "active";
|
||||
}
|
||||
|
||||
/** Minimal Document PiP surface (TS lib.dom lacks it). */
|
||||
export interface DocumentPip {
|
||||
requestWindow(options?: { width?: number; height?: number }): Promise<Window>;
|
||||
}
|
||||
|
||||
export function pipApi(): DocumentPip | undefined {
|
||||
if (typeof window === "undefined") return undefined;
|
||||
return (window as unknown as { documentPictureInPicture?: DocumentPip }).documentPictureInPicture;
|
||||
}
|
||||
|
||||
/** Clone the host document's style/link nodes into a PiP window. */
|
||||
export function cloneStylesInto(pipWindow: Window): void {
|
||||
for (const node of document.querySelectorAll('link[rel="stylesheet"], style')) {
|
||||
pipWindow.document.head.appendChild(node.cloneNode(true));
|
||||
}
|
||||
}
|
||||
|
||||
export interface ObservationView {
|
||||
readonly snapshot: OverlaySnapshot;
|
||||
/** Pinned session wins while it still exists; otherwise auto-follow. */
|
||||
readonly focus: SessionObservation | undefined;
|
||||
readonly pinnedId: string | null;
|
||||
readonly onTogglePin: (sessionId: string) => void;
|
||||
/** 1s ticker while anything is active (drives the elapsed readout). */
|
||||
readonly now: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether one session is visible on a surface scoped to one DSH
|
||||
* conversation: sessions started by that conversation or its descendants
|
||||
* (the lineage ancestors ride along on the entry). Untracked sessions (no
|
||||
* owner recorded) are hidden from scoped surfaces but stay in global views.
|
||||
*/
|
||||
export function visibleToScope(obs: SessionObservation, scopeId: string): boolean {
|
||||
return obs.dshSessionIds?.includes(scopeId) === true;
|
||||
}
|
||||
|
||||
/**
|
||||
* The store-backed observation view model. Holds the feed for the component
|
||||
* lifetime (refcounted — overlapping carriers never kill each other's
|
||||
* stream). `scopeId` narrows the view to one DSH conversation's sessions
|
||||
* (the better-sidebar tab); undefined keeps the global view (floating
|
||||
* card, PiP).
|
||||
*/
|
||||
export function useObservationView(
|
||||
store: ObservationClientStore,
|
||||
scopeId?: string,
|
||||
): ObservationView {
|
||||
const snapshot = useSyncExternalStore(store.subscribe, store.getSnapshot);
|
||||
useEffect(() => {
|
||||
store.acquire();
|
||||
return () => store.release();
|
||||
}, [store]);
|
||||
|
||||
const scopedSnapshot: OverlaySnapshot =
|
||||
scopeId === undefined
|
||||
? snapshot
|
||||
: {
|
||||
...snapshot,
|
||||
sessions: snapshot.sessions.filter((s) => visibleToScope(s, scopeId)),
|
||||
};
|
||||
|
||||
const [pinnedId, setPinnedId] = useState<string | null>(null);
|
||||
const pinned =
|
||||
pinnedId !== null ? scopedSnapshot.sessions.find((s) => s.sessionId === pinnedId) : undefined;
|
||||
const focus = pinned ?? focusOf(scopedSnapshot.sessions);
|
||||
const onTogglePin = useCallback((sessionId: string) => {
|
||||
setPinnedId((current) => (current === sessionId ? null : sessionId));
|
||||
}, []);
|
||||
|
||||
const anyActive = scopedSnapshot.sessions.some((s) => s.action !== "idle");
|
||||
const [now, setNow] = useState(() => Date.now());
|
||||
useEffect(() => {
|
||||
if (!anyActive) return;
|
||||
const timer = setInterval(() => setNow(Date.now()), 1000);
|
||||
return () => clearInterval(timer);
|
||||
}, [anyActive]);
|
||||
|
||||
return {
|
||||
snapshot: scopedSnapshot,
|
||||
focus,
|
||||
pinnedId: pinned !== undefined ? pinnedId : null,
|
||||
onTogglePin,
|
||||
now,
|
||||
};
|
||||
}
|
||||
|
||||
export interface PipHandle {
|
||||
/** The open PiP window, or null while the content renders in its carrier. */
|
||||
readonly pipWindow: Window | null;
|
||||
/** Whether the browser exposes Document PiP (Chrome-only today). */
|
||||
readonly pipSupported: boolean;
|
||||
/**
|
||||
* Upgrade the content into a native PiP window. Rejects silently without a
|
||||
* user gesture (or when closed mid-request): the carrier keeps the content.
|
||||
*/
|
||||
readonly popOut: (size?: { width: number; height: number }) => void;
|
||||
}
|
||||
|
||||
export function usePip(): PipHandle {
|
||||
const [pipWindow, setPipWindow] = useState<Window | null>(null);
|
||||
const popOut = useCallback((size?: { width: number; height: number }): void => {
|
||||
const pip = pipApi();
|
||||
if (pip === undefined) return;
|
||||
void pip
|
||||
.requestWindow(size)
|
||||
.then((win) => {
|
||||
cloneStylesInto(win);
|
||||
win.addEventListener("pagehide", () => setPipWindow(null));
|
||||
setPipWindow(win);
|
||||
})
|
||||
.catch(() => {});
|
||||
}, []);
|
||||
// Unmounting the carrier (tab close, conversation switch, plugin HMR) does
|
||||
// not destroy the PiP browsing context: close it ourselves or it survives
|
||||
// as a blank window. close() on an already-closed window is a no-op.
|
||||
useEffect(() => () => pipWindow?.close(), [pipWindow]);
|
||||
return { pipWindow, pipSupported: pipApi() !== undefined, popOut };
|
||||
}
|
||||
@@ -1,26 +0,0 @@
|
||||
/**
|
||||
* Carrier switch for the observation view: while the better-sidebar
|
||||
* integration fiber is alive (the dsh-better-sidebar plugin provides its
|
||||
* `betterSidebar` service), the tracking view lives in a sidebar tab and the
|
||||
* floating overlay card/capsule hides itself. The flag is a tiny external
|
||||
* store so the overlay can read it through useSyncExternalStore and flip
|
||||
* without a remount when the sidebar plugin (un)loads.
|
||||
*/
|
||||
|
||||
let active = false;
|
||||
const listeners = new Set<() => void>();
|
||||
|
||||
export function getSidebarMode(): boolean {
|
||||
return active;
|
||||
}
|
||||
|
||||
export function subscribeSidebarMode(listener: () => void): () => void {
|
||||
listeners.add(listener);
|
||||
return () => listeners.delete(listener);
|
||||
}
|
||||
|
||||
export function setSidebarMode(next: boolean): void {
|
||||
if (active === next) return;
|
||||
active = next;
|
||||
for (const listener of [...listeners]) listener();
|
||||
}
|
||||
@@ -12,7 +12,6 @@
|
||||
|
||||
import type { Context } from "@deepseek-ai/cordis";
|
||||
import Schema from "@deepseek-ai/schemastery";
|
||||
import { armArchiveCleanup } from "./archive-cleanup";
|
||||
import { armLazyTools } from "./lazy-tools";
|
||||
import { ObservationService } from "./observation";
|
||||
import { registerObservationRoutes } from "./observation-http";
|
||||
@@ -108,10 +107,6 @@ export function apply(
|
||||
removeRoutes = registerObservationRoutes(injected, observation);
|
||||
return () => removeRoutes();
|
||||
});
|
||||
// Reap a conversation's browsers when the conversation itself is archived:
|
||||
// archived sessions are hidden from every surface, so their Agent Windows
|
||||
// would otherwise linger unreachable until idle timeout or unload.
|
||||
const disarmArchiveCleanup = armArchiveCleanup(ctx, registry, observation);
|
||||
|
||||
// Non-blocking install probe: warn early when bsk is missing instead of
|
||||
// failing the first tool call with a bare spawn error. Uses --version on
|
||||
@@ -137,7 +132,6 @@ export function apply(
|
||||
removeSuite();
|
||||
unregisterSkill();
|
||||
removeRoutes();
|
||||
disarmArchiveCleanup();
|
||||
observation.dispose();
|
||||
runner.killAll();
|
||||
const stops = registry
|
||||
@@ -150,7 +144,6 @@ export function apply(
|
||||
});
|
||||
}
|
||||
|
||||
export { armArchiveCleanup, ownerSessionIds } from "./archive-cleanup";
|
||||
export type { ObservationEvent, ObservationOptions, SessionObservation } from "./observation";
|
||||
export { ObservationService } from "./observation";
|
||||
export { registerObservationRoutes } from "./observation-http";
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
/**
|
||||
* Lazy tool-schema injection ("progressive disclosure", final stage): with
|
||||
* `lazyTools` on (the default), the 28 browser_* tool schemas stay OUT of the
|
||||
* `lazyTools` on (the default), the 11 browser_* tool schemas stay OUT of the
|
||||
* system prompt until the `browser-skill` skill has actually been invoked —
|
||||
* the skill catalog entry is the only advertisement. One successful
|
||||
* invocation (model tool call, or a user's `/browser-skill` gesture) reveals
|
||||
@@ -96,7 +96,7 @@ export function hasSuccessfulSkillInvocation(events: readonly SessionEventLike[]
|
||||
/**
|
||||
* Arm the lazy reveal. Returns a disposer tearing down listeners and — when
|
||||
* the reveal already happened — the tool suite itself.
|
||||
* @param registerSuite - registers the 28 browser_* tools, returns their disposer.
|
||||
* @param registerSuite - registers the 11 browser_* tools, returns their disposer.
|
||||
*/
|
||||
export function armLazyTools(ctx: Context, registerSuite: () => () => void): () => void {
|
||||
let suiteDisposer: (() => void) | undefined;
|
||||
|
||||
@@ -8,7 +8,6 @@
|
||||
* GET /bsk-observation/state → { sessions, available }
|
||||
* GET /bsk-observation/events → SSE stream of ObservationEvent
|
||||
* POST /bsk-observation/interrupt → body {sessionId?} → {interrupted: boolean}
|
||||
* POST /bsk-observation/stop → body {sessionId} → {stopped: boolean}
|
||||
* GET /bsk-observation/thumbnail/<attachmentId> → image bytes
|
||||
*
|
||||
* Routes exist only when a webServer service is mounted (web composition);
|
||||
@@ -167,42 +166,6 @@ export function registerObservationRoutes(
|
||||
});
|
||||
},
|
||||
}),
|
||||
webServer.register({
|
||||
kind: "exact",
|
||||
path: `${ROUTE_BASE}/stop`,
|
||||
handler: (req, res) => {
|
||||
if (req.method !== "POST") {
|
||||
sendJson(res, 405, { error: "method not allowed" });
|
||||
return;
|
||||
}
|
||||
if (fenceRejected(req, res)) return;
|
||||
let body = "";
|
||||
req.on("data", (chunk: Buffer | string) => {
|
||||
body += chunk;
|
||||
});
|
||||
req.on("end", () => {
|
||||
let sessionId: string | undefined;
|
||||
try {
|
||||
const parsed = JSON.parse(body || "{}") as { sessionId?: unknown };
|
||||
if (typeof parsed.sessionId === "string" && parsed.sessionId !== "") {
|
||||
sessionId = parsed.sessionId;
|
||||
}
|
||||
} catch {
|
||||
sendJson(res, 400, { error: "invalid JSON body" });
|
||||
return;
|
||||
}
|
||||
// A destructive call always names its target (never "current").
|
||||
if (sessionId === undefined) {
|
||||
sendJson(res, 400, { error: "sessionId required" });
|
||||
return;
|
||||
}
|
||||
void observation.stopSession(sessionId).then(
|
||||
(stopped) => sendJson(res, 200, { stopped }),
|
||||
() => sendJson(res, 500, { error: "stop failed" }),
|
||||
);
|
||||
});
|
||||
},
|
||||
}),
|
||||
webServer.register({
|
||||
kind: "prefix",
|
||||
path: `${ROUTE_BASE}/thumbnail`,
|
||||
|
||||
@@ -39,13 +39,6 @@ export interface SessionObservation {
|
||||
* greys it out until it is stopped/removed. No more frames are requested.
|
||||
*/
|
||||
dead?: boolean;
|
||||
/**
|
||||
* The DSH conversations this session belongs to (the starting agent's
|
||||
* session plus its seed-lineage ancestors), recorded at start. Scoped
|
||||
* surfaces (the better-sidebar tab) filter by it; absent means untracked
|
||||
* ownership — visible only in the global (unscoped) view.
|
||||
*/
|
||||
dshSessionIds?: string[];
|
||||
}
|
||||
|
||||
/** Incremental event carried to subscribers (SSE on the wire). */
|
||||
@@ -160,13 +153,11 @@ export class ObservationService {
|
||||
/** Register a fresh owned session (called from browser_session_start). */
|
||||
addSession(sessionId: string, url?: string): void {
|
||||
if (!this.deps.options.enabled || this.disposed) return;
|
||||
const dshSessionIds = this.deps.registry.dshOwnersOf(sessionId);
|
||||
this.put({
|
||||
sessionId,
|
||||
...(url !== undefined ? { url } : {}),
|
||||
action: "idle",
|
||||
since: this.scheduler.now(),
|
||||
...(dshSessionIds.length > 0 ? { dshSessionIds } : {}),
|
||||
});
|
||||
this.lastActivity.set(sessionId, this.scheduler.now());
|
||||
this.scheduleCapture(sessionId, 0);
|
||||
@@ -237,36 +228,6 @@ export class ObservationService {
|
||||
return this.deps.runner.killFor(target) > 0;
|
||||
}
|
||||
|
||||
/**
|
||||
* Stop one owned session and close its Agent Window (the overlay's stop
|
||||
* button — same end state as `browser_session_stop`). Never waits behind a
|
||||
* hung in-flight command: tool children are killed first so the session's
|
||||
* keyed queue drains immediately, and no further captures queue up. A
|
||||
* session the daemon already forgot (dead strip entries) stops
|
||||
* idempotently — the goal state (entry gone) is identical.
|
||||
* @returns whether the session ended up stopped/removed.
|
||||
*/
|
||||
async stopSession(sessionId: string): Promise<boolean> {
|
||||
if (!this.deps.registry.isOwned(sessionId)) return false;
|
||||
this.cancelCapture(sessionId);
|
||||
this.deps.runner.killFor(sessionId);
|
||||
const result = await this.deps.queue.run(sessionId, () =>
|
||||
this.deps.runner.run(["session", "stop", sessionId], { timeoutMs: 30_000 }),
|
||||
);
|
||||
if (result.code !== 0) {
|
||||
let code: string | undefined;
|
||||
try {
|
||||
code = (JSON.parse(result.stdout) as { code?: string }).code;
|
||||
} catch {
|
||||
code = undefined;
|
||||
}
|
||||
if (code !== "session_not_found") return false;
|
||||
}
|
||||
this.deps.registry.remove(sessionId);
|
||||
this.removeSession(sessionId);
|
||||
return true;
|
||||
}
|
||||
|
||||
/**
|
||||
* Read one captured thumbnail from the in-process ring. Powers the plugin's
|
||||
* own HTTP thumbnail route — frames are plugin-owned runtime data, never
|
||||
@@ -439,48 +400,14 @@ export function actionForLabel(label: string): string {
|
||||
return "observing";
|
||||
case "click":
|
||||
return "clicking";
|
||||
case "hover":
|
||||
return "hovering";
|
||||
case "fill":
|
||||
return "filling";
|
||||
case "select":
|
||||
return "selecting";
|
||||
case "press":
|
||||
return "pressing";
|
||||
case "screenshot":
|
||||
return "capturing";
|
||||
case "emulate":
|
||||
return "emulating";
|
||||
case "tab list":
|
||||
return "listing tabs";
|
||||
case "tab create":
|
||||
return "creating tab";
|
||||
case "tab close":
|
||||
return "closing tab";
|
||||
case "tab select":
|
||||
return "selecting tab";
|
||||
case "tab borrow":
|
||||
return "borrowing tab";
|
||||
case "tab return":
|
||||
return "returning tab";
|
||||
case "navigate-back":
|
||||
return "navigating back";
|
||||
case "navigate-forward":
|
||||
return "navigating forward";
|
||||
case "reload":
|
||||
return "reloading";
|
||||
case "wait-for-navigation":
|
||||
return "waiting for navigation";
|
||||
case "request-help":
|
||||
return "waiting for user";
|
||||
case "get-html":
|
||||
return "reading HTML";
|
||||
case "console":
|
||||
return "reading console";
|
||||
case "network":
|
||||
return "reading network";
|
||||
case "window resize":
|
||||
return "resizing window";
|
||||
default:
|
||||
return label;
|
||||
}
|
||||
|
||||
@@ -1,58 +0,0 @@
|
||||
import type { ToolDefinition, ToolResult, ToolRunContext } from "@deepseek-ai/dsh-tools";
|
||||
import type { ToolDeps } from "./tools";
|
||||
|
||||
export type ToolRegistrar = (definition: ToolDefinition) => () => void;
|
||||
|
||||
type TerminalPresentation = { card: "terminal"; output: string; exitCode: number } | undefined;
|
||||
|
||||
/** Existing tool runtime seams reused by the phase-one capability modules. */
|
||||
export interface PhaseOneRuntime {
|
||||
run(
|
||||
exec: ToolRunContext,
|
||||
args: string[],
|
||||
label: string,
|
||||
observeSession?: string,
|
||||
runnerTimeoutMs?: number,
|
||||
): Promise<unknown>;
|
||||
commandLine(args: string[]): string;
|
||||
presentTerminalResult: (_args: never, result: ToolResult) => TerminalPresentation;
|
||||
}
|
||||
|
||||
export function requireNonEmpty(value: string, name: string): void {
|
||||
if (value.trim().length === 0) throw new Error(`${name} must be a non-empty string`);
|
||||
}
|
||||
|
||||
export function requirePositive(value: number | undefined, name: string): void {
|
||||
if (value !== undefined && value <= 0) throw new Error(`${name} must be greater than zero`);
|
||||
}
|
||||
|
||||
/** CLI positional target detection is shared by hover/select/request-help. */
|
||||
export function isSnapshotRef(target: string): boolean {
|
||||
return /^@?e\d+$/.test(target);
|
||||
}
|
||||
|
||||
/**
|
||||
* Give the child enough time to honour a command-level timeout plus IPC
|
||||
* settlement slack, without shortening the plugin's configured default.
|
||||
*/
|
||||
export function runnerTimeout(deps: ToolDeps, commandTimeoutMs: number | undefined): number {
|
||||
if (commandTimeoutMs === undefined) return deps.config.defaultTimeoutMs;
|
||||
return Math.max(deps.config.defaultTimeoutMs, commandTimeoutMs + 15_000);
|
||||
}
|
||||
|
||||
export function appendTarget(args: string[], target: string): void {
|
||||
args.push(target);
|
||||
}
|
||||
|
||||
export function appendTabId(args: string[], tabId: number | undefined): void {
|
||||
if (tabId !== undefined) args.push("--tab-id", String(tabId));
|
||||
}
|
||||
|
||||
export function appendWaitOptions(
|
||||
args: string[],
|
||||
waitUntil: string | undefined,
|
||||
timeoutMs: number | undefined,
|
||||
): void {
|
||||
if (waitUntil !== undefined) args.push("--wait-until", waitUntil);
|
||||
if (timeoutMs !== undefined) args.push("--timeout", `${timeoutMs}ms`);
|
||||
}
|
||||
@@ -1,197 +0,0 @@
|
||||
import { defineTool } from "@deepseek-ai/dsh-tools";
|
||||
import {
|
||||
appendTabId,
|
||||
appendTarget,
|
||||
type PhaseOneRuntime,
|
||||
requireNonEmpty,
|
||||
requirePositive,
|
||||
runnerTimeout,
|
||||
type ToolRegistrar,
|
||||
} from "./phase-one-runtime";
|
||||
import { SESSION_PARAM, TAB_ID_PARAM, TIMEOUT_MS_PARAM } from "./tool-params";
|
||||
import type { ToolDeps } from "./tools";
|
||||
|
||||
const MODIFIERS = ["alt", "ctrl", "meta", "shift"] as const;
|
||||
|
||||
/** Add hover and select without bypassing session ownership or observation. */
|
||||
export function registerPhaseOneInteractionTools(
|
||||
deps: ToolDeps,
|
||||
register: ToolRegistrar,
|
||||
runtime: PhaseOneRuntime,
|
||||
): void {
|
||||
const { registry } = deps;
|
||||
|
||||
register(
|
||||
defineTool({
|
||||
name: "browser_hover",
|
||||
description:
|
||||
"Move the mouse over a snapshot ref or CSS selector to reveal hover-triggered UI. " +
|
||||
"Run browser_observe or browser_snapshot again afterwards to discover newly visible refs.",
|
||||
parameters: {
|
||||
target: {
|
||||
type: "string",
|
||||
required: true,
|
||||
description: "Snapshot ref (@e3 / e3) or CSS selector of the element to hover.",
|
||||
},
|
||||
session: SESSION_PARAM,
|
||||
tabId: TAB_ID_PARAM,
|
||||
modifiers: {
|
||||
type: "array",
|
||||
items: { type: "string", enum: MODIFIERS },
|
||||
description: "Keyboard modifiers held during the mouse move.",
|
||||
},
|
||||
settleMs: {
|
||||
type: "integer",
|
||||
description: "Milliseconds to wait for hover-triggered UI to settle (default: 200).",
|
||||
},
|
||||
timeoutMs: TIMEOUT_MS_PARAM,
|
||||
},
|
||||
output: {
|
||||
schema: {
|
||||
type: "object",
|
||||
additionalProperties: false,
|
||||
properties: {
|
||||
session: { type: "string", required: true },
|
||||
tabId: { type: "integer", required: true },
|
||||
x: { type: "number", required: true },
|
||||
y: { type: "number", required: true },
|
||||
},
|
||||
},
|
||||
render: (_args, value) => [
|
||||
{
|
||||
type: "text",
|
||||
text: `[session ${value.session}] hovered at (${value.x}, ${value.y}) on tab ${value.tabId}`,
|
||||
},
|
||||
],
|
||||
},
|
||||
async execute(args, exec) {
|
||||
requireNonEmpty(args.target, "target");
|
||||
requirePositive(args.timeoutMs, "timeoutMs");
|
||||
requirePositive(args.settleMs, "settleMs");
|
||||
const sessionId = registry.resolve(args.session, "browser_hover");
|
||||
const cmdArgs = ["hover", "--session", sessionId];
|
||||
appendTabId(cmdArgs, args.tabId);
|
||||
if (args.modifiers !== undefined && args.modifiers.length > 0) {
|
||||
cmdArgs.push("--modifiers", args.modifiers.join(","));
|
||||
}
|
||||
if (args.settleMs !== undefined) cmdArgs.push("--settle", `${args.settleMs}ms`);
|
||||
if (args.timeoutMs !== undefined) cmdArgs.push("--timeout", `${args.timeoutMs}ms`);
|
||||
appendTarget(cmdArgs, args.target);
|
||||
const reply = (await runtime.run(
|
||||
exec,
|
||||
cmdArgs,
|
||||
"hover",
|
||||
sessionId,
|
||||
runnerTimeout(deps, args.timeoutMs),
|
||||
)) as { tab_id: number; x: number; y: number };
|
||||
return { session: sessionId, tabId: reply.tab_id, x: reply.x, y: reply.y };
|
||||
},
|
||||
presentCall: (args) => ({
|
||||
card: "terminal",
|
||||
title: runtime.commandLine([
|
||||
"hover",
|
||||
args.target,
|
||||
"--session",
|
||||
args.session ?? "(current)",
|
||||
]),
|
||||
description: "Hover an element",
|
||||
}),
|
||||
presentResult: runtime.presentTerminalResult,
|
||||
}),
|
||||
);
|
||||
|
||||
register(
|
||||
defineTool({
|
||||
name: "browser_select",
|
||||
description:
|
||||
"Set a select element's option values. Pass one value for a normal select or multiple " +
|
||||
"values for a multi-select; values replace the current selection.",
|
||||
parameters: {
|
||||
target: {
|
||||
type: "string",
|
||||
required: true,
|
||||
description: "Snapshot ref (@e3 / e3) or CSS selector of the select element.",
|
||||
},
|
||||
values: {
|
||||
type: "array",
|
||||
required: true,
|
||||
items: { type: "string" },
|
||||
description: "Option value attributes to select; at least one value is required.",
|
||||
},
|
||||
session: SESSION_PARAM,
|
||||
tabId: TAB_ID_PARAM,
|
||||
timeoutMs: TIMEOUT_MS_PARAM,
|
||||
},
|
||||
output: {
|
||||
schema: {
|
||||
type: "object",
|
||||
additionalProperties: false,
|
||||
properties: {
|
||||
session: { type: "string", required: true },
|
||||
tabId: { type: "integer", required: true },
|
||||
multiple: { type: "boolean", required: true },
|
||||
selectedValues: {
|
||||
type: "array",
|
||||
required: true,
|
||||
items: { type: "string" },
|
||||
},
|
||||
selectedLabels: {
|
||||
type: "array",
|
||||
required: true,
|
||||
items: { type: "string" },
|
||||
},
|
||||
},
|
||||
},
|
||||
render: (_args, value) => [
|
||||
{
|
||||
type: "text",
|
||||
text:
|
||||
`[session ${value.session}] selected ` +
|
||||
`${value.selectedValues.join(", ") || "(none)"} on tab ${value.tabId}`,
|
||||
},
|
||||
],
|
||||
},
|
||||
async execute(args, exec) {
|
||||
requireNonEmpty(args.target, "target");
|
||||
requirePositive(args.timeoutMs, "timeoutMs");
|
||||
if (args.values.length === 0) throw new Error("values must contain at least one option");
|
||||
const sessionId = registry.resolve(args.session, "browser_select");
|
||||
const cmdArgs = ["select", "--session", sessionId];
|
||||
appendTabId(cmdArgs, args.tabId);
|
||||
for (const value of args.values) cmdArgs.push("--value", value);
|
||||
if (args.timeoutMs !== undefined) cmdArgs.push("--timeout", `${args.timeoutMs}ms`);
|
||||
appendTarget(cmdArgs, args.target);
|
||||
const reply = (await runtime.run(
|
||||
exec,
|
||||
cmdArgs,
|
||||
"select",
|
||||
sessionId,
|
||||
runnerTimeout(deps, args.timeoutMs),
|
||||
)) as {
|
||||
tab_id: number;
|
||||
multiple: boolean;
|
||||
selected_values: string[];
|
||||
selected_labels: string[];
|
||||
};
|
||||
return {
|
||||
session: sessionId,
|
||||
tabId: reply.tab_id,
|
||||
multiple: reply.multiple,
|
||||
selectedValues: reply.selected_values,
|
||||
selectedLabels: reply.selected_labels,
|
||||
};
|
||||
},
|
||||
presentCall: (args) => ({
|
||||
card: "terminal",
|
||||
title: runtime.commandLine([
|
||||
"select",
|
||||
args.target,
|
||||
"--session",
|
||||
args.session ?? "(current)",
|
||||
]),
|
||||
description: `Select ${args.values.length} option${args.values.length === 1 ? "" : "s"}`,
|
||||
}),
|
||||
presentResult: runtime.presentTerminalResult,
|
||||
}),
|
||||
);
|
||||
}
|
||||
@@ -1,230 +0,0 @@
|
||||
import { defineTool } from "@deepseek-ai/dsh-tools";
|
||||
import {
|
||||
appendTabId,
|
||||
appendWaitOptions,
|
||||
type PhaseOneRuntime,
|
||||
requirePositive,
|
||||
runnerTimeout,
|
||||
type ToolRegistrar,
|
||||
} from "./phase-one-runtime";
|
||||
import { SESSION_PARAM, TAB_ID_PARAM, TIMEOUT_MS_PARAM, WAIT_UNTIL_PARAM } from "./tool-params";
|
||||
import type { ToolDeps } from "./tools";
|
||||
|
||||
interface RawHistoryResult {
|
||||
tab_id: number;
|
||||
previous_url?: string;
|
||||
final_url?: string;
|
||||
reached: string;
|
||||
error_text?: string;
|
||||
}
|
||||
|
||||
const HISTORY_OUTPUT = {
|
||||
type: "object",
|
||||
additionalProperties: false,
|
||||
properties: {
|
||||
session: { type: "string", required: true },
|
||||
tabId: { type: "integer", required: true },
|
||||
previousUrl: { type: "string" },
|
||||
finalUrl: { type: "string" },
|
||||
reached: { type: "string", required: true },
|
||||
errorText: { type: "string" },
|
||||
},
|
||||
} as const;
|
||||
|
||||
function mapHistory(sessionId: string, reply: RawHistoryResult) {
|
||||
return {
|
||||
session: sessionId,
|
||||
tabId: reply.tab_id,
|
||||
...(reply.previous_url !== undefined ? { previousUrl: reply.previous_url } : {}),
|
||||
...(reply.final_url !== undefined ? { finalUrl: reply.final_url } : {}),
|
||||
reached: reply.reached,
|
||||
...(reply.error_text !== undefined ? { errorText: reply.error_text } : {}),
|
||||
};
|
||||
}
|
||||
|
||||
/** Add browser history, reload, and explicit lifecycle waiting. */
|
||||
export function registerPhaseOneNavigationTools(
|
||||
deps: ToolDeps,
|
||||
register: ToolRegistrar,
|
||||
runtime: PhaseOneRuntime,
|
||||
): void {
|
||||
const { registry } = deps;
|
||||
|
||||
const registerHistory = (direction: "back" | "forward") => {
|
||||
const toolName = `browser_navigate_${direction}` as const;
|
||||
const command = `navigate-${direction}`;
|
||||
register(
|
||||
defineTool({
|
||||
name: toolName,
|
||||
description: `Navigate the active Agent Window tab ${direction} by one history entry.`,
|
||||
parameters: {
|
||||
session: SESSION_PARAM,
|
||||
tabId: TAB_ID_PARAM,
|
||||
waitUntil: WAIT_UNTIL_PARAM,
|
||||
timeoutMs: TIMEOUT_MS_PARAM,
|
||||
},
|
||||
output: {
|
||||
schema: HISTORY_OUTPUT,
|
||||
render: (_args, value) => [
|
||||
{
|
||||
type: "text",
|
||||
text:
|
||||
`[session ${value.session}] navigated ${direction} on tab ${value.tabId}` +
|
||||
(value.finalUrl !== undefined ? ` to ${value.finalUrl}` : "") +
|
||||
` (reached: ${value.reached})` +
|
||||
(value.errorText !== undefined ? ` — ${value.errorText}` : ""),
|
||||
},
|
||||
],
|
||||
},
|
||||
async execute(args, exec) {
|
||||
requirePositive(args.timeoutMs, "timeoutMs");
|
||||
const sessionId = registry.resolve(args.session, toolName);
|
||||
const cmdArgs = [command, "--session", sessionId];
|
||||
appendTabId(cmdArgs, args.tabId);
|
||||
appendWaitOptions(cmdArgs, args.waitUntil, args.timeoutMs);
|
||||
const reply = (await runtime.run(
|
||||
exec,
|
||||
cmdArgs,
|
||||
command,
|
||||
sessionId,
|
||||
runnerTimeout(deps, args.timeoutMs),
|
||||
)) as RawHistoryResult;
|
||||
if (reply.final_url !== undefined) deps.observation.setUrl(sessionId, reply.final_url);
|
||||
return mapHistory(sessionId, reply);
|
||||
},
|
||||
presentCall: (args) => ({
|
||||
card: "terminal",
|
||||
title: runtime.commandLine([command, "--session", args.session ?? "(current)"]),
|
||||
description: `Navigate ${direction}`,
|
||||
}),
|
||||
presentResult: runtime.presentTerminalResult,
|
||||
}),
|
||||
);
|
||||
};
|
||||
registerHistory("back");
|
||||
registerHistory("forward");
|
||||
|
||||
register(
|
||||
defineTool({
|
||||
name: "browser_reload",
|
||||
description:
|
||||
"Reload the active Agent Window tab and wait for a lifecycle phase. Set hard=true to " +
|
||||
"bypass the HTTP cache.",
|
||||
parameters: {
|
||||
session: SESSION_PARAM,
|
||||
tabId: TAB_ID_PARAM,
|
||||
waitUntil: WAIT_UNTIL_PARAM,
|
||||
timeoutMs: TIMEOUT_MS_PARAM,
|
||||
hard: { type: "boolean", description: "Bypass the HTTP cache while reloading." },
|
||||
},
|
||||
output: {
|
||||
schema: HISTORY_OUTPUT,
|
||||
render: (_args, value) => [
|
||||
{
|
||||
type: "text",
|
||||
text:
|
||||
`[session ${value.session}] reloaded tab ${value.tabId}` +
|
||||
(value.finalUrl !== undefined ? ` at ${value.finalUrl}` : "") +
|
||||
` (reached: ${value.reached})` +
|
||||
(value.errorText !== undefined ? ` — ${value.errorText}` : ""),
|
||||
},
|
||||
],
|
||||
},
|
||||
async execute(args, exec) {
|
||||
requirePositive(args.timeoutMs, "timeoutMs");
|
||||
const sessionId = registry.resolve(args.session, "browser_reload");
|
||||
const cmdArgs = ["reload", "--session", sessionId];
|
||||
appendTabId(cmdArgs, args.tabId);
|
||||
appendWaitOptions(cmdArgs, args.waitUntil, args.timeoutMs);
|
||||
if (args.hard === true) cmdArgs.push("--hard");
|
||||
const reply = (await runtime.run(
|
||||
exec,
|
||||
cmdArgs,
|
||||
"reload",
|
||||
sessionId,
|
||||
runnerTimeout(deps, args.timeoutMs),
|
||||
)) as RawHistoryResult;
|
||||
if (reply.final_url !== undefined) deps.observation.setUrl(sessionId, reply.final_url);
|
||||
return mapHistory(sessionId, reply);
|
||||
},
|
||||
presentCall: (args) => ({
|
||||
card: "terminal",
|
||||
title: runtime.commandLine([
|
||||
"reload",
|
||||
"--session",
|
||||
args.session ?? "(current)",
|
||||
...(args.hard === true ? ["--hard"] : []),
|
||||
]),
|
||||
description: "Reload the active tab",
|
||||
}),
|
||||
presentResult: runtime.presentTerminalResult,
|
||||
}),
|
||||
);
|
||||
|
||||
register(
|
||||
defineTool({
|
||||
name: "browser_wait_for_navigation",
|
||||
description:
|
||||
"Wait for a page lifecycle event on the active Agent Window tab. Use after an action " +
|
||||
"that may navigate when the action result itself does not wait for navigation.",
|
||||
parameters: {
|
||||
session: SESSION_PARAM,
|
||||
tabId: TAB_ID_PARAM,
|
||||
waitUntil: WAIT_UNTIL_PARAM,
|
||||
timeoutMs: TIMEOUT_MS_PARAM,
|
||||
},
|
||||
output: {
|
||||
schema: {
|
||||
type: "object",
|
||||
additionalProperties: false,
|
||||
properties: {
|
||||
session: { type: "string", required: true },
|
||||
tabId: { type: "integer", required: true },
|
||||
reached: { type: "string", required: true },
|
||||
errorText: { type: "string" },
|
||||
},
|
||||
},
|
||||
render: (_args, value) => [
|
||||
{
|
||||
type: "text",
|
||||
text:
|
||||
`[session ${value.session}] navigation wait on tab ${value.tabId}: ${value.reached}` +
|
||||
(value.errorText !== undefined ? ` — ${value.errorText}` : ""),
|
||||
},
|
||||
],
|
||||
},
|
||||
async execute(args, exec) {
|
||||
requirePositive(args.timeoutMs, "timeoutMs");
|
||||
const sessionId = registry.resolve(args.session, "browser_wait_for_navigation");
|
||||
const timeoutMs = args.timeoutMs ?? 30_000;
|
||||
const cmdArgs = ["wait-for-navigation", "--session", sessionId];
|
||||
appendTabId(cmdArgs, args.tabId);
|
||||
if (args.waitUntil !== undefined) cmdArgs.push("--wait-until", args.waitUntil);
|
||||
cmdArgs.push("--timeout", `${timeoutMs}ms`);
|
||||
const reply = (await runtime.run(
|
||||
exec,
|
||||
cmdArgs,
|
||||
"wait-for-navigation",
|
||||
sessionId,
|
||||
runnerTimeout(deps, timeoutMs),
|
||||
)) as { tab_id: number; reached: string; error_text?: string };
|
||||
return {
|
||||
session: sessionId,
|
||||
tabId: reply.tab_id,
|
||||
reached: reply.reached,
|
||||
...(reply.error_text !== undefined ? { errorText: reply.error_text } : {}),
|
||||
};
|
||||
},
|
||||
presentCall: (args) => ({
|
||||
card: "terminal",
|
||||
title: runtime.commandLine([
|
||||
"wait-for-navigation",
|
||||
"--session",
|
||||
args.session ?? "(current)",
|
||||
]),
|
||||
description: "Wait for page navigation",
|
||||
}),
|
||||
presentResult: runtime.presentTerminalResult,
|
||||
}),
|
||||
);
|
||||
}
|
||||
@@ -1,672 +0,0 @@
|
||||
import { defineTool } from "@deepseek-ai/dsh-tools";
|
||||
import {
|
||||
appendTabId,
|
||||
isSnapshotRef,
|
||||
type PhaseOneRuntime,
|
||||
requireNonEmpty,
|
||||
requirePositive,
|
||||
runnerTimeout,
|
||||
type ToolRegistrar,
|
||||
} from "./phase-one-runtime";
|
||||
import { SESSION_PARAM, TAB_ID_PARAM, TIMEOUT_MS_PARAM } from "./tool-params";
|
||||
import type { ToolDeps } from "./tools";
|
||||
|
||||
interface HelpConditionInput {
|
||||
urlContains?: string;
|
||||
urlMatches?: string;
|
||||
selectorExists?: string;
|
||||
selectorMissing?: string;
|
||||
textExists?: string;
|
||||
textMissing?: string;
|
||||
}
|
||||
|
||||
interface CompletionCriteriaInput {
|
||||
any?: HelpConditionInput[];
|
||||
all?: HelpConditionInput[];
|
||||
stableForMs?: number;
|
||||
}
|
||||
|
||||
interface RawConsoleFrame {
|
||||
function_name?: string;
|
||||
url?: string;
|
||||
line?: number;
|
||||
column?: number;
|
||||
}
|
||||
|
||||
interface RawConsoleEntry {
|
||||
sequence: number;
|
||||
kind: string;
|
||||
level: string;
|
||||
text: string;
|
||||
url?: string;
|
||||
line?: number;
|
||||
column?: number;
|
||||
timestamp?: number;
|
||||
stack_trace?: RawConsoleFrame[];
|
||||
truncated?: boolean;
|
||||
}
|
||||
|
||||
interface RawNetworkEntry {
|
||||
sequence: number;
|
||||
kind: "response" | "failure";
|
||||
method?: string;
|
||||
url?: string;
|
||||
status?: number;
|
||||
status_text?: string;
|
||||
mime_type?: string;
|
||||
resource_type?: string;
|
||||
error_text?: string;
|
||||
timestamp?: number;
|
||||
truncated?: boolean;
|
||||
}
|
||||
|
||||
const HELP_CONDITION_SCHEMA = {
|
||||
type: "object",
|
||||
additionalProperties: false,
|
||||
properties: {
|
||||
urlContains: { type: "string" },
|
||||
urlMatches: { type: "string" },
|
||||
selectorExists: { type: "string" },
|
||||
selectorMissing: { type: "string" },
|
||||
textExists: { type: "string" },
|
||||
textMissing: { type: "string" },
|
||||
},
|
||||
} as const;
|
||||
|
||||
const DEBUG_PARAMETERS = {
|
||||
session: SESSION_PARAM,
|
||||
tabId: TAB_ID_PARAM,
|
||||
since: {
|
||||
type: "integer",
|
||||
description: "Return entries with a sequence strictly greater than this cursor.",
|
||||
},
|
||||
limit: {
|
||||
type: "integer",
|
||||
description: "Maximum entries to return (default: 50, daemon cap: 200).",
|
||||
},
|
||||
maxTextChars: {
|
||||
type: "integer",
|
||||
description: "Maximum characters per entry text/URL (default: 1000, daemon cap: 4096).",
|
||||
},
|
||||
} as const;
|
||||
|
||||
function mapCondition(condition: HelpConditionInput) {
|
||||
return {
|
||||
...(condition.urlContains !== undefined ? { url_contains: condition.urlContains } : {}),
|
||||
...(condition.urlMatches !== undefined ? { url_matches: condition.urlMatches } : {}),
|
||||
...(condition.selectorExists !== undefined
|
||||
? { selector_exists: condition.selectorExists }
|
||||
: {}),
|
||||
...(condition.selectorMissing !== undefined
|
||||
? { selector_missing: condition.selectorMissing }
|
||||
: {}),
|
||||
...(condition.textExists !== undefined ? { text_exists: condition.textExists } : {}),
|
||||
...(condition.textMissing !== undefined ? { text_missing: condition.textMissing } : {}),
|
||||
};
|
||||
}
|
||||
|
||||
function mapCompletionCriteria(criteria: CompletionCriteriaInput) {
|
||||
return {
|
||||
...(criteria.any !== undefined ? { any: criteria.any.map(mapCondition) } : {}),
|
||||
...(criteria.all !== undefined ? { all: criteria.all.map(mapCondition) } : {}),
|
||||
...(criteria.stableForMs !== undefined ? { stable_for_ms: criteria.stableForMs } : {}),
|
||||
};
|
||||
}
|
||||
|
||||
function validateDebugArgs(args: { since?: number; limit?: number; maxTextChars?: number }): void {
|
||||
if (args.since !== undefined && args.since < 0) {
|
||||
throw new Error("since must be zero or greater");
|
||||
}
|
||||
requirePositive(args.limit, "limit");
|
||||
requirePositive(args.maxTextChars, "maxTextChars");
|
||||
}
|
||||
|
||||
function appendDebugOptions(
|
||||
args: string[],
|
||||
options: { tabId?: number; since?: number; limit?: number; maxTextChars?: number },
|
||||
): void {
|
||||
appendTabId(args, options.tabId);
|
||||
if (options.since !== undefined) args.push("--since", String(options.since));
|
||||
if (options.limit !== undefined) args.push("--limit", String(options.limit));
|
||||
if (options.maxTextChars !== undefined) {
|
||||
args.push("--max-text-chars", String(options.maxTextChars));
|
||||
}
|
||||
}
|
||||
|
||||
/** Human help, raw HTML, diagnostics, and Agent Window sizing. */
|
||||
export function registerPhaseOneSupportTools(
|
||||
deps: ToolDeps,
|
||||
register: ToolRegistrar,
|
||||
runtime: PhaseOneRuntime,
|
||||
): void {
|
||||
const { registry } = deps;
|
||||
|
||||
register(
|
||||
defineTool({
|
||||
name: "browser_request_help",
|
||||
description:
|
||||
"Pause browser automation and ask the user to complete an in-page step such as login, " +
|
||||
"captcha, OTP, or confirmation. After continued/completed, observe again before using refs.",
|
||||
parameters: {
|
||||
prompt: {
|
||||
type: "string",
|
||||
required: true,
|
||||
description: "Clear instructions shown to the user in the browser help overlay.",
|
||||
},
|
||||
session: SESSION_PARAM,
|
||||
tabId: TAB_ID_PARAM,
|
||||
title: { type: "string", description: "Optional title for the help overlay." },
|
||||
targets: {
|
||||
type: "array",
|
||||
items: { type: "string" },
|
||||
description: "Snapshot refs or CSS selectors to scroll to and highlight for the user.",
|
||||
},
|
||||
timeoutMs: {
|
||||
...TIMEOUT_MS_PARAM,
|
||||
description: "Maximum wait for the user in milliseconds (default: 300000).",
|
||||
},
|
||||
completionCriteria: {
|
||||
type: "object",
|
||||
additionalProperties: false,
|
||||
description: "Optional explicit success detector for automatic completion.",
|
||||
properties: {
|
||||
any: { type: "array", items: HELP_CONDITION_SCHEMA },
|
||||
all: { type: "array", items: HELP_CONDITION_SCHEMA },
|
||||
stableForMs: {
|
||||
type: "integer",
|
||||
description: "How long criteria must remain true before completing.",
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
output: {
|
||||
schema: {
|
||||
type: "object",
|
||||
additionalProperties: false,
|
||||
properties: {
|
||||
session: { type: "string", required: true },
|
||||
tabId: { type: "integer", required: true },
|
||||
outcome: {
|
||||
type: "string",
|
||||
required: true,
|
||||
enum: ["continued", "cancelled", "timed_out", "completed", "navigated", "disabled"],
|
||||
},
|
||||
completedBy: { type: "string" },
|
||||
note: { type: "string" },
|
||||
resolvedTargets: {
|
||||
type: "array",
|
||||
items: {
|
||||
type: "object",
|
||||
additionalProperties: false,
|
||||
properties: {
|
||||
matched: { type: "boolean", required: true },
|
||||
ref: { type: "string" },
|
||||
selector: { type: "string" },
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
render: (_args, value) => [
|
||||
{
|
||||
type: "text",
|
||||
text:
|
||||
`[session ${value.session}] user help outcome on tab ${value.tabId}: ${value.outcome}` +
|
||||
(value.note !== undefined ? ` — ${value.note}` : "") +
|
||||
(value.resolvedTargets !== undefined
|
||||
? `\ntargets: ${value.resolvedTargets
|
||||
.map(
|
||||
(target) =>
|
||||
`${target.ref ?? target.selector ?? "(unknown)"}=${target.matched ? "matched" : "not found"}`,
|
||||
)
|
||||
.join(", ")}`
|
||||
: ""),
|
||||
},
|
||||
],
|
||||
},
|
||||
async execute(args, exec) {
|
||||
requireNonEmpty(args.prompt, "prompt");
|
||||
if (args.title !== undefined) requireNonEmpty(args.title, "title");
|
||||
requirePositive(args.timeoutMs, "timeoutMs");
|
||||
for (const target of args.targets ?? []) requireNonEmpty(target, "target");
|
||||
const criteria = args.completionCriteria as CompletionCriteriaInput | undefined;
|
||||
if (criteria?.stableForMs !== undefined && criteria.stableForMs < 0) {
|
||||
throw new Error("completionCriteria.stableForMs must be zero or greater");
|
||||
}
|
||||
const sessionId = registry.resolve(args.session, "browser_request_help");
|
||||
const timeoutMs = args.timeoutMs ?? 300_000;
|
||||
const cmdArgs = [
|
||||
"request-help",
|
||||
"--session",
|
||||
sessionId,
|
||||
"--prompt",
|
||||
args.prompt,
|
||||
"--timeout",
|
||||
`${timeoutMs}ms`,
|
||||
];
|
||||
appendTabId(cmdArgs, args.tabId);
|
||||
if (args.title !== undefined) cmdArgs.push("--title", args.title);
|
||||
for (const target of args.targets ?? []) cmdArgs.push("--target", target);
|
||||
if (criteria !== undefined) {
|
||||
cmdArgs.push("--completion-criteria", JSON.stringify(mapCompletionCriteria(criteria)));
|
||||
}
|
||||
const reply = (await runtime.run(
|
||||
exec,
|
||||
cmdArgs,
|
||||
"request-help",
|
||||
sessionId,
|
||||
runnerTimeout(deps, timeoutMs),
|
||||
)) as {
|
||||
outcome: "continued" | "cancelled" | "timed_out" | "completed" | "navigated" | "disabled";
|
||||
completed_by?: string;
|
||||
note?: string;
|
||||
tab_id: number;
|
||||
resolved_targets?: { matched: boolean; ref?: string; selector?: string }[];
|
||||
};
|
||||
return {
|
||||
session: sessionId,
|
||||
tabId: reply.tab_id,
|
||||
outcome: reply.outcome,
|
||||
...(reply.completed_by !== undefined ? { completedBy: reply.completed_by } : {}),
|
||||
...(reply.note !== undefined ? { note: reply.note } : {}),
|
||||
...(reply.resolved_targets !== undefined
|
||||
? { resolvedTargets: reply.resolved_targets }
|
||||
: {}),
|
||||
};
|
||||
},
|
||||
presentCall: (args) => ({
|
||||
card: "terminal",
|
||||
title: runtime.commandLine(["request-help", "--session", args.session ?? "(current)"]),
|
||||
description: "Ask the user for browser help",
|
||||
}),
|
||||
presentResult: runtime.presentTerminalResult,
|
||||
}),
|
||||
);
|
||||
|
||||
register(
|
||||
defineTool({
|
||||
name: "browser_get_html",
|
||||
description:
|
||||
"Read raw DOM HTML from the active Agent Window tab, optionally scoped to a fresh " +
|
||||
"snapshot ref. Use only when browser_observe/browser_snapshot cannot answer the question.",
|
||||
parameters: {
|
||||
session: SESSION_PARAM,
|
||||
tabId: TAB_ID_PARAM,
|
||||
ref: { type: "string", description: "Fresh snapshot ref that scopes the HTML subtree." },
|
||||
maxBytes: {
|
||||
type: "integer",
|
||||
description: "Maximum returned HTML bytes before truncation (default: 524288).",
|
||||
},
|
||||
},
|
||||
output: {
|
||||
schema: {
|
||||
type: "object",
|
||||
additionalProperties: false,
|
||||
properties: {
|
||||
session: { type: "string", required: true },
|
||||
tabId: { type: "integer", required: true },
|
||||
html: { type: "string", required: true },
|
||||
truncated: { type: "boolean", required: true },
|
||||
byteSize: { type: "integer", required: true },
|
||||
},
|
||||
},
|
||||
render: (_args, value) => [
|
||||
{
|
||||
type: "text",
|
||||
text:
|
||||
value.html +
|
||||
(value.truncated ? `\n(HTML truncated; original size ${value.byteSize} bytes)` : ""),
|
||||
},
|
||||
],
|
||||
},
|
||||
isConcurrencySafe: () => true,
|
||||
async execute(args, exec) {
|
||||
if (args.ref !== undefined) {
|
||||
requireNonEmpty(args.ref, "ref");
|
||||
if (!isSnapshotRef(args.ref)) throw new Error("ref must be a snapshot ref such as @e3");
|
||||
}
|
||||
requirePositive(args.maxBytes, "maxBytes");
|
||||
const sessionId = registry.resolve(args.session, "browser_get_html");
|
||||
const cmdArgs = ["get-html", "--session", sessionId];
|
||||
appendTabId(cmdArgs, args.tabId);
|
||||
if (args.ref !== undefined) cmdArgs.push("--ref", args.ref);
|
||||
if (args.maxBytes !== undefined) cmdArgs.push("--max-bytes", String(args.maxBytes));
|
||||
const reply = (await runtime.run(exec, cmdArgs, "get-html", sessionId)) as {
|
||||
tab_id: number;
|
||||
html: string;
|
||||
truncated?: boolean;
|
||||
byte_size: number;
|
||||
};
|
||||
return {
|
||||
session: sessionId,
|
||||
tabId: reply.tab_id,
|
||||
html: reply.html,
|
||||
truncated: reply.truncated ?? false,
|
||||
byteSize: reply.byte_size,
|
||||
};
|
||||
},
|
||||
presentCall: (args) => ({
|
||||
card: "terminal",
|
||||
title: runtime.commandLine([
|
||||
"get-html",
|
||||
"--session",
|
||||
args.session ?? "(current)",
|
||||
...(args.ref !== undefined ? ["--ref", args.ref] : []),
|
||||
]),
|
||||
description: "Read raw page HTML",
|
||||
}),
|
||||
presentResult: runtime.presentTerminalResult,
|
||||
}),
|
||||
);
|
||||
|
||||
register(
|
||||
defineTool({
|
||||
name: "browser_console",
|
||||
description:
|
||||
"Read buffered console messages, browser log entries, and JavaScript exceptions from a tab. " +
|
||||
"This is read-only and does not evaluate JavaScript.",
|
||||
parameters: {
|
||||
...DEBUG_PARAMETERS,
|
||||
includeStack: {
|
||||
type: "boolean",
|
||||
description: "Include structured stack frames in returned console entries.",
|
||||
},
|
||||
},
|
||||
output: {
|
||||
schema: {
|
||||
type: "object",
|
||||
additionalProperties: false,
|
||||
properties: {
|
||||
session: { type: "string", required: true },
|
||||
tabId: { type: "integer", required: true },
|
||||
entries: {
|
||||
type: "array",
|
||||
required: true,
|
||||
items: {
|
||||
type: "object",
|
||||
additionalProperties: false,
|
||||
properties: {
|
||||
sequence: { type: "integer", required: true },
|
||||
kind: { type: "string", required: true },
|
||||
level: { type: "string", required: true },
|
||||
text: { type: "string", required: true },
|
||||
url: { type: "string" },
|
||||
line: { type: "integer" },
|
||||
column: { type: "integer" },
|
||||
timestamp: { type: "number" },
|
||||
stackTrace: {
|
||||
type: "array",
|
||||
required: true,
|
||||
items: {
|
||||
type: "object",
|
||||
additionalProperties: false,
|
||||
properties: {
|
||||
functionName: { type: "string" },
|
||||
url: { type: "string" },
|
||||
line: { type: "integer" },
|
||||
column: { type: "integer" },
|
||||
},
|
||||
},
|
||||
},
|
||||
truncated: { type: "boolean", required: true },
|
||||
},
|
||||
},
|
||||
},
|
||||
nextSince: { type: "integer", required: true },
|
||||
truncated: { type: "boolean", required: true },
|
||||
},
|
||||
},
|
||||
render: (_args, value) => [
|
||||
{
|
||||
type: "text",
|
||||
text:
|
||||
value.entries.length === 0
|
||||
? "(no console messages captured)"
|
||||
: value.entries
|
||||
.map((entry) => {
|
||||
const location =
|
||||
entry.url === undefined
|
||||
? ""
|
||||
: ` ${entry.url}${entry.line !== undefined ? `:${entry.line}` : ""}${entry.column !== undefined ? `:${entry.column}` : ""}`;
|
||||
const stack = entry.stackTrace
|
||||
.map(
|
||||
(frame) =>
|
||||
`\n at ${frame.functionName ?? "<anonymous>"} ${frame.url ?? ""}` +
|
||||
`${frame.line !== undefined ? `:${frame.line}` : ""}` +
|
||||
`${frame.column !== undefined ? `:${frame.column}` : ""}`,
|
||||
)
|
||||
.join("");
|
||||
return `#${entry.sequence} ${entry.level} ${entry.kind}${location} ${entry.text}${stack}`;
|
||||
})
|
||||
.join("\n") +
|
||||
(value.truncated
|
||||
? `\n(output truncated; continue with since=${value.nextSince})`
|
||||
: ""),
|
||||
},
|
||||
],
|
||||
},
|
||||
isConcurrencySafe: () => true,
|
||||
async execute(args, exec) {
|
||||
validateDebugArgs(args);
|
||||
const sessionId = registry.resolve(args.session, "browser_console");
|
||||
const cmdArgs = ["console", "--session", sessionId];
|
||||
appendDebugOptions(cmdArgs, args);
|
||||
if (args.includeStack === true) cmdArgs.push("--include-stack");
|
||||
const reply = (await runtime.run(exec, cmdArgs, "console", sessionId)) as {
|
||||
tab_id: number;
|
||||
entries?: RawConsoleEntry[];
|
||||
next_since: number;
|
||||
truncated?: boolean;
|
||||
};
|
||||
return {
|
||||
session: sessionId,
|
||||
tabId: reply.tab_id,
|
||||
entries: (reply.entries ?? []).map((entry) => ({
|
||||
sequence: entry.sequence,
|
||||
kind: entry.kind,
|
||||
level: entry.level,
|
||||
text: entry.text,
|
||||
...(entry.url !== undefined ? { url: entry.url } : {}),
|
||||
...(entry.line !== undefined ? { line: entry.line } : {}),
|
||||
...(entry.column !== undefined ? { column: entry.column } : {}),
|
||||
...(entry.timestamp !== undefined ? { timestamp: entry.timestamp } : {}),
|
||||
stackTrace: (entry.stack_trace ?? []).map((frame) => ({
|
||||
...(frame.function_name !== undefined ? { functionName: frame.function_name } : {}),
|
||||
...(frame.url !== undefined ? { url: frame.url } : {}),
|
||||
...(frame.line !== undefined ? { line: frame.line } : {}),
|
||||
...(frame.column !== undefined ? { column: frame.column } : {}),
|
||||
})),
|
||||
truncated: entry.truncated ?? false,
|
||||
})),
|
||||
nextSince: reply.next_since,
|
||||
truncated: reply.truncated ?? false,
|
||||
};
|
||||
},
|
||||
presentCall: (args) => ({
|
||||
card: "terminal",
|
||||
title: runtime.commandLine(["console", "--session", args.session ?? "(current)"]),
|
||||
description: "Read browser console messages",
|
||||
}),
|
||||
presentResult: runtime.presentTerminalResult,
|
||||
}),
|
||||
);
|
||||
|
||||
register(
|
||||
defineTool({
|
||||
name: "browser_network",
|
||||
description:
|
||||
"Read buffered network responses and failures from a tab. Returns metadata only; request " +
|
||||
"and response headers and bodies are not captured.",
|
||||
parameters: DEBUG_PARAMETERS,
|
||||
output: {
|
||||
schema: {
|
||||
type: "object",
|
||||
additionalProperties: false,
|
||||
properties: {
|
||||
session: { type: "string", required: true },
|
||||
tabId: { type: "integer", required: true },
|
||||
entries: {
|
||||
type: "array",
|
||||
required: true,
|
||||
items: {
|
||||
type: "object",
|
||||
additionalProperties: false,
|
||||
properties: {
|
||||
sequence: { type: "integer", required: true },
|
||||
kind: { type: "string", required: true, enum: ["response", "failure"] },
|
||||
method: { type: "string" },
|
||||
url: { type: "string" },
|
||||
status: { type: "integer" },
|
||||
statusText: { type: "string" },
|
||||
mimeType: { type: "string" },
|
||||
resourceType: { type: "string" },
|
||||
errorText: { type: "string" },
|
||||
timestamp: { type: "number" },
|
||||
truncated: { type: "boolean", required: true },
|
||||
},
|
||||
},
|
||||
},
|
||||
nextSince: { type: "integer", required: true },
|
||||
truncated: { type: "boolean", required: true },
|
||||
},
|
||||
},
|
||||
render: (_args, value) => [
|
||||
{
|
||||
type: "text",
|
||||
text:
|
||||
value.entries.length === 0
|
||||
? "(no network activity captured)"
|
||||
: value.entries
|
||||
.map((entry) =>
|
||||
entry.kind === "failure"
|
||||
? `#${entry.sequence} FAILED ${entry.method ?? "?"} ${entry.url ?? "(unknown)"} — ${entry.errorText ?? "failed"}`
|
||||
: `#${entry.sequence} ${entry.status ?? "?"} ${entry.method ?? "?"} ${entry.url ?? "(unknown)"}`,
|
||||
)
|
||||
.join("\n") +
|
||||
(value.truncated
|
||||
? `\n(output truncated; continue with since=${value.nextSince})`
|
||||
: ""),
|
||||
},
|
||||
],
|
||||
},
|
||||
isConcurrencySafe: () => true,
|
||||
async execute(args, exec) {
|
||||
validateDebugArgs(args);
|
||||
const sessionId = registry.resolve(args.session, "browser_network");
|
||||
const cmdArgs = ["network", "--session", sessionId];
|
||||
appendDebugOptions(cmdArgs, args);
|
||||
const reply = (await runtime.run(exec, cmdArgs, "network", sessionId)) as {
|
||||
tab_id: number;
|
||||
entries?: RawNetworkEntry[];
|
||||
next_since: number;
|
||||
truncated?: boolean;
|
||||
};
|
||||
return {
|
||||
session: sessionId,
|
||||
tabId: reply.tab_id,
|
||||
entries: (reply.entries ?? []).map((entry) => ({
|
||||
sequence: entry.sequence,
|
||||
kind: entry.kind,
|
||||
...(entry.method !== undefined ? { method: entry.method } : {}),
|
||||
...(entry.url !== undefined ? { url: entry.url } : {}),
|
||||
...(entry.status !== undefined ? { status: entry.status } : {}),
|
||||
...(entry.status_text !== undefined ? { statusText: entry.status_text } : {}),
|
||||
...(entry.mime_type !== undefined ? { mimeType: entry.mime_type } : {}),
|
||||
...(entry.resource_type !== undefined ? { resourceType: entry.resource_type } : {}),
|
||||
...(entry.error_text !== undefined ? { errorText: entry.error_text } : {}),
|
||||
...(entry.timestamp !== undefined ? { timestamp: entry.timestamp } : {}),
|
||||
truncated: entry.truncated ?? false,
|
||||
})),
|
||||
nextSince: reply.next_since,
|
||||
truncated: reply.truncated ?? false,
|
||||
};
|
||||
},
|
||||
presentCall: (args) => ({
|
||||
card: "terminal",
|
||||
title: runtime.commandLine(["network", "--session", args.session ?? "(current)"]),
|
||||
description: "Read browser network activity",
|
||||
}),
|
||||
presentResult: runtime.presentTerminalResult,
|
||||
}),
|
||||
);
|
||||
|
||||
register(
|
||||
defineTool({
|
||||
name: "browser_window_resize",
|
||||
description: "Resize the owned session's Agent Window using outer CSS-pixel dimensions.",
|
||||
parameters: {
|
||||
width: {
|
||||
type: "integer",
|
||||
required: true,
|
||||
description: "Agent Window outer width in CSS pixels (100..=7680).",
|
||||
},
|
||||
height: {
|
||||
type: "integer",
|
||||
required: true,
|
||||
description: "Agent Window outer height in CSS pixels (100..=7680).",
|
||||
},
|
||||
session: SESSION_PARAM,
|
||||
},
|
||||
output: {
|
||||
schema: {
|
||||
type: "object",
|
||||
additionalProperties: false,
|
||||
properties: {
|
||||
session: { type: "string", required: true },
|
||||
windowId: { type: "integer", required: true },
|
||||
width: { type: "integer", required: true },
|
||||
height: { type: "integer", required: true },
|
||||
},
|
||||
},
|
||||
render: (_args, value) => [
|
||||
{
|
||||
type: "text",
|
||||
text: `[session ${value.session}] resized window ${value.windowId} to ${value.width}x${value.height}`,
|
||||
},
|
||||
],
|
||||
},
|
||||
async execute(args, exec) {
|
||||
if (args.width < 100 || args.width > 7680 || args.height < 100 || args.height > 7680) {
|
||||
throw new Error("width and height must each be in the range 100..=7680");
|
||||
}
|
||||
const sessionId = registry.resolve(args.session, "browser_window_resize");
|
||||
const reply = (await runtime.run(
|
||||
exec,
|
||||
[
|
||||
"window",
|
||||
"resize",
|
||||
"--session",
|
||||
sessionId,
|
||||
"--width",
|
||||
String(args.width),
|
||||
"--height",
|
||||
String(args.height),
|
||||
],
|
||||
"window resize",
|
||||
sessionId,
|
||||
)) as { window_id: number; width: number; height: number };
|
||||
return {
|
||||
session: sessionId,
|
||||
windowId: reply.window_id,
|
||||
width: reply.width,
|
||||
height: reply.height,
|
||||
};
|
||||
},
|
||||
presentCall: (args) => ({
|
||||
card: "terminal",
|
||||
title: runtime.commandLine([
|
||||
"window",
|
||||
"resize",
|
||||
"--session",
|
||||
args.session ?? "(current)",
|
||||
"--width",
|
||||
String(args.width),
|
||||
"--height",
|
||||
String(args.height),
|
||||
]),
|
||||
description: "Resize the Agent Window",
|
||||
}),
|
||||
presentResult: runtime.presentTerminalResult,
|
||||
}),
|
||||
);
|
||||
}
|
||||
@@ -1,402 +0,0 @@
|
||||
import { defineTool } from "@deepseek-ai/dsh-tools";
|
||||
import type { PhaseOneRuntime, ToolRegistrar } from "./phase-one-runtime";
|
||||
import { requireNonEmpty } from "./phase-one-runtime";
|
||||
import { SESSION_PARAM } from "./tool-params";
|
||||
import type { ToolDeps } from "./tools";
|
||||
|
||||
const TAB_ID_REQUIRED = {
|
||||
type: "integer",
|
||||
required: true,
|
||||
description: "Chrome tab id returned by browser_tab_list or browser_tab_create.",
|
||||
} as const;
|
||||
|
||||
interface RawTabInfo {
|
||||
tab_id: number;
|
||||
title?: string;
|
||||
url?: string;
|
||||
window_id?: number;
|
||||
active?: boolean;
|
||||
scope?: "user" | "agent";
|
||||
}
|
||||
|
||||
/** Add Agent Window tab management while preserving the owned-session boundary. */
|
||||
export function registerPhaseOneTabTools(
|
||||
deps: ToolDeps,
|
||||
register: ToolRegistrar,
|
||||
runtime: PhaseOneRuntime,
|
||||
): void {
|
||||
const { registry } = deps;
|
||||
|
||||
register(
|
||||
defineTool({
|
||||
name: "browser_tab_list",
|
||||
description:
|
||||
"List tabs visible to an owned browser session. Other sessions' Agent Windows remain " +
|
||||
"hidden; use scope=user to find a user tab before borrowing it.",
|
||||
parameters: {
|
||||
session: SESSION_PARAM,
|
||||
scope: {
|
||||
type: "string",
|
||||
enum: ["user", "agent", "all"],
|
||||
description: "Which tabs to list (default: all).",
|
||||
},
|
||||
},
|
||||
output: {
|
||||
schema: {
|
||||
type: "object",
|
||||
additionalProperties: false,
|
||||
properties: {
|
||||
session: { type: "string", required: true },
|
||||
tabs: {
|
||||
type: "array",
|
||||
required: true,
|
||||
items: {
|
||||
type: "object",
|
||||
additionalProperties: false,
|
||||
properties: {
|
||||
tabId: { type: "integer", required: true },
|
||||
title: { type: "string" },
|
||||
url: { type: "string" },
|
||||
windowId: { type: "integer" },
|
||||
active: { type: "boolean" },
|
||||
scope: { type: "string", enum: ["user", "agent"] },
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
render: (_args, value) => [
|
||||
{
|
||||
type: "text",
|
||||
text:
|
||||
value.tabs.length === 0
|
||||
? `[session ${value.session}] no matching tabs`
|
||||
: value.tabs
|
||||
.map(
|
||||
(tab) =>
|
||||
`${tab.tabId} [${tab.scope ?? "unknown"}]${tab.active === true ? " [active]" : ""} ` +
|
||||
`${tab.title ?? "(untitled)"} — ${tab.url ?? "(unknown URL)"}`,
|
||||
)
|
||||
.join("\n"),
|
||||
},
|
||||
],
|
||||
},
|
||||
isConcurrencySafe: () => true,
|
||||
async execute(args, exec) {
|
||||
const sessionId = registry.resolve(args.session, "browser_tab_list");
|
||||
const cmdArgs = ["tab", "list", "--session", sessionId];
|
||||
if (args.scope !== undefined) cmdArgs.push("--scope", args.scope);
|
||||
const reply = (await runtime.run(exec, cmdArgs, "tab list", sessionId)) as {
|
||||
tabs: RawTabInfo[];
|
||||
};
|
||||
return {
|
||||
session: sessionId,
|
||||
tabs: reply.tabs.map((tab) => ({
|
||||
tabId: tab.tab_id,
|
||||
...(tab.title !== undefined ? { title: tab.title } : {}),
|
||||
...(tab.url !== undefined ? { url: tab.url } : {}),
|
||||
...(tab.window_id !== undefined ? { windowId: tab.window_id } : {}),
|
||||
...(tab.active !== undefined ? { active: tab.active } : {}),
|
||||
...(tab.scope !== undefined ? { scope: tab.scope } : {}),
|
||||
})),
|
||||
};
|
||||
},
|
||||
presentCall: (args) => ({
|
||||
card: "terminal",
|
||||
title: runtime.commandLine([
|
||||
"tab",
|
||||
"list",
|
||||
"--session",
|
||||
args.session ?? "(current)",
|
||||
...(args.scope !== undefined ? ["--scope", args.scope] : []),
|
||||
]),
|
||||
description: "List browser tabs",
|
||||
}),
|
||||
presentResult: runtime.presentTerminalResult,
|
||||
}),
|
||||
);
|
||||
|
||||
register(
|
||||
defineTool({
|
||||
name: "browser_tab_create",
|
||||
description:
|
||||
"Create a tab inside the owned session's Agent Window. The tab is focused by default; " +
|
||||
"set active=false to open it in the background.",
|
||||
parameters: {
|
||||
session: SESSION_PARAM,
|
||||
url: { type: "string", description: "Initial URL (default: chrome://newtab/)." },
|
||||
active: { type: "boolean", description: "Whether to focus the new tab (default: true)." },
|
||||
index: { type: "integer", description: "Insertion index in the Agent Window tab strip." },
|
||||
},
|
||||
output: {
|
||||
schema: {
|
||||
type: "object",
|
||||
additionalProperties: false,
|
||||
properties: {
|
||||
session: { type: "string", required: true },
|
||||
tabId: { type: "integer", required: true },
|
||||
windowId: { type: "integer", required: true },
|
||||
url: { type: "string", required: true },
|
||||
},
|
||||
},
|
||||
render: (_args, value) => [
|
||||
{
|
||||
type: "text",
|
||||
text: `[session ${value.session}] created tab ${value.tabId} in window ${value.windowId} (${value.url || "pending URL"})`,
|
||||
},
|
||||
],
|
||||
},
|
||||
async execute(args, exec) {
|
||||
if (args.url !== undefined) requireNonEmpty(args.url, "url");
|
||||
const sessionId = registry.resolve(args.session, "browser_tab_create");
|
||||
const cmdArgs = ["tab", "create", "--session", sessionId];
|
||||
if (args.url !== undefined) cmdArgs.push("--url", args.url);
|
||||
if (args.active === false) cmdArgs.push("--no-active");
|
||||
if (args.index !== undefined) cmdArgs.push("--index", String(args.index));
|
||||
const reply = (await runtime.run(exec, cmdArgs, "tab create", sessionId)) as {
|
||||
tab_id: number;
|
||||
window_id: number;
|
||||
url: string;
|
||||
};
|
||||
if (args.active !== false && reply.url.length > 0) {
|
||||
deps.observation.setUrl(sessionId, reply.url);
|
||||
}
|
||||
return {
|
||||
session: sessionId,
|
||||
tabId: reply.tab_id,
|
||||
windowId: reply.window_id,
|
||||
url: reply.url,
|
||||
};
|
||||
},
|
||||
presentCall: (args) => ({
|
||||
card: "terminal",
|
||||
title: runtime.commandLine([
|
||||
"tab",
|
||||
"create",
|
||||
"--session",
|
||||
args.session ?? "(current)",
|
||||
...(args.url !== undefined ? ["--url", args.url] : []),
|
||||
]),
|
||||
description: "Create an Agent Window tab",
|
||||
}),
|
||||
presentResult: runtime.presentTerminalResult,
|
||||
}),
|
||||
);
|
||||
|
||||
register(
|
||||
defineTool({
|
||||
name: "browser_tab_close",
|
||||
description: "Close a tab in the owned session's Agent Window.",
|
||||
parameters: { tabId: TAB_ID_REQUIRED, session: SESSION_PARAM },
|
||||
output: {
|
||||
schema: {
|
||||
type: "object",
|
||||
additionalProperties: false,
|
||||
properties: {
|
||||
session: { type: "string", required: true },
|
||||
tabId: { type: "integer", required: true },
|
||||
},
|
||||
},
|
||||
render: (_args, value) => [
|
||||
{ type: "text", text: `[session ${value.session}] closed tab ${value.tabId}` },
|
||||
],
|
||||
},
|
||||
async execute(args, exec) {
|
||||
const sessionId = registry.resolve(args.session, "browser_tab_close");
|
||||
const reply = (await runtime.run(
|
||||
exec,
|
||||
["tab", "close", String(args.tabId), "--session", sessionId],
|
||||
"tab close",
|
||||
sessionId,
|
||||
)) as { tab_id: number };
|
||||
return { session: sessionId, tabId: reply.tab_id };
|
||||
},
|
||||
presentCall: (args) => ({
|
||||
card: "terminal",
|
||||
title: runtime.commandLine([
|
||||
"tab",
|
||||
"close",
|
||||
String(args.tabId),
|
||||
"--session",
|
||||
args.session ?? "(current)",
|
||||
]),
|
||||
description: "Close an Agent Window tab",
|
||||
}),
|
||||
presentResult: runtime.presentTerminalResult,
|
||||
}),
|
||||
);
|
||||
|
||||
register(
|
||||
defineTool({
|
||||
name: "browser_tab_select",
|
||||
description: "Focus a tab in the owned session's Agent Window.",
|
||||
parameters: { tabId: TAB_ID_REQUIRED, session: SESSION_PARAM },
|
||||
output: {
|
||||
schema: {
|
||||
type: "object",
|
||||
additionalProperties: false,
|
||||
properties: {
|
||||
session: { type: "string", required: true },
|
||||
tabId: { type: "integer", required: true },
|
||||
windowId: { type: "integer", required: true },
|
||||
},
|
||||
},
|
||||
render: (_args, value) => [
|
||||
{
|
||||
type: "text",
|
||||
text: `[session ${value.session}] selected tab ${value.tabId} in window ${value.windowId}`,
|
||||
},
|
||||
],
|
||||
},
|
||||
async execute(args, exec) {
|
||||
const sessionId = registry.resolve(args.session, "browser_tab_select");
|
||||
const reply = (await runtime.run(
|
||||
exec,
|
||||
["tab", "select", String(args.tabId), "--session", sessionId],
|
||||
"tab select",
|
||||
sessionId,
|
||||
)) as { tab_id: number; window_id: number };
|
||||
return { session: sessionId, tabId: reply.tab_id, windowId: reply.window_id };
|
||||
},
|
||||
presentCall: (args) => ({
|
||||
card: "terminal",
|
||||
title: runtime.commandLine([
|
||||
"tab",
|
||||
"select",
|
||||
String(args.tabId),
|
||||
"--session",
|
||||
args.session ?? "(current)",
|
||||
]),
|
||||
description: "Select an Agent Window tab",
|
||||
}),
|
||||
presentResult: runtime.presentTerminalResult,
|
||||
}),
|
||||
);
|
||||
|
||||
register(
|
||||
defineTool({
|
||||
name: "browser_tab_borrow",
|
||||
description:
|
||||
"Move a user-window tab into the owned session's Agent Window for controlled interaction. " +
|
||||
"Return it with browser_tab_return when finished; stopping the session also auto-returns it.",
|
||||
parameters: { tabId: TAB_ID_REQUIRED, session: SESSION_PARAM },
|
||||
output: {
|
||||
schema: {
|
||||
type: "object",
|
||||
additionalProperties: false,
|
||||
properties: {
|
||||
session: { type: "string", required: true },
|
||||
tabId: { type: "integer", required: true },
|
||||
originalWindowId: { type: "integer", required: true },
|
||||
originalIndex: { type: "integer", required: true },
|
||||
agentWindowId: { type: "integer", required: true },
|
||||
},
|
||||
},
|
||||
render: (_args, value) => [
|
||||
{
|
||||
type: "text",
|
||||
text:
|
||||
`[session ${value.session}] borrowed tab ${value.tabId} from window ` +
|
||||
`${value.originalWindowId} into Agent Window ${value.agentWindowId}`,
|
||||
},
|
||||
],
|
||||
},
|
||||
async execute(args, exec) {
|
||||
const sessionId = registry.resolve(args.session, "browser_tab_borrow");
|
||||
const reply = (await runtime.run(
|
||||
exec,
|
||||
["tab", "borrow", String(args.tabId), "--session", sessionId],
|
||||
"tab borrow",
|
||||
sessionId,
|
||||
)) as {
|
||||
tab_id: number;
|
||||
original_window_id: number;
|
||||
original_index: number;
|
||||
agent_window_id: number;
|
||||
};
|
||||
return {
|
||||
session: sessionId,
|
||||
tabId: reply.tab_id,
|
||||
originalWindowId: reply.original_window_id,
|
||||
originalIndex: reply.original_index,
|
||||
agentWindowId: reply.agent_window_id,
|
||||
};
|
||||
},
|
||||
presentCall: (args) => ({
|
||||
card: "terminal",
|
||||
title: runtime.commandLine([
|
||||
"tab",
|
||||
"borrow",
|
||||
String(args.tabId),
|
||||
"--session",
|
||||
args.session ?? "(current)",
|
||||
]),
|
||||
description: "Borrow a user tab",
|
||||
}),
|
||||
presentResult: runtime.presentTerminalResult,
|
||||
}),
|
||||
);
|
||||
|
||||
register(
|
||||
defineTool({
|
||||
name: "browser_tab_return",
|
||||
description: "Return a borrowed tab to its original user window and tab-strip position.",
|
||||
parameters: { tabId: TAB_ID_REQUIRED, session: SESSION_PARAM },
|
||||
output: {
|
||||
schema: {
|
||||
type: "object",
|
||||
additionalProperties: false,
|
||||
properties: {
|
||||
session: { type: "string", required: true },
|
||||
tabId: { type: "integer", required: true },
|
||||
returnedToWindowId: { type: "integer", required: true },
|
||||
returnedToIndex: { type: "integer", required: true },
|
||||
fallback: { type: "boolean", required: true },
|
||||
},
|
||||
},
|
||||
render: (_args, value) => [
|
||||
{
|
||||
type: "text",
|
||||
text:
|
||||
`[session ${value.session}] returned tab ${value.tabId} to window ` +
|
||||
`${value.returnedToWindowId} at index ${value.returnedToIndex}` +
|
||||
(value.fallback ? " (original window unavailable; used fallback)" : ""),
|
||||
},
|
||||
],
|
||||
},
|
||||
async execute(args, exec) {
|
||||
const sessionId = registry.resolve(args.session, "browser_tab_return");
|
||||
const reply = (await runtime.run(
|
||||
exec,
|
||||
["tab", "return", String(args.tabId), "--session", sessionId],
|
||||
"tab return",
|
||||
sessionId,
|
||||
)) as {
|
||||
tab_id: number;
|
||||
returned_to_window_id: number;
|
||||
returned_to_index: number;
|
||||
fallback?: boolean;
|
||||
};
|
||||
return {
|
||||
session: sessionId,
|
||||
tabId: reply.tab_id,
|
||||
returnedToWindowId: reply.returned_to_window_id,
|
||||
returnedToIndex: reply.returned_to_index,
|
||||
fallback: reply.fallback ?? false,
|
||||
};
|
||||
},
|
||||
presentCall: (args) => ({
|
||||
card: "terminal",
|
||||
title: runtime.commandLine([
|
||||
"tab",
|
||||
"return",
|
||||
String(args.tabId),
|
||||
"--session",
|
||||
args.session ?? "(current)",
|
||||
]),
|
||||
description: "Return a borrowed tab",
|
||||
}),
|
||||
presentResult: runtime.presentTerminalResult,
|
||||
}),
|
||||
);
|
||||
}
|
||||
@@ -1,18 +0,0 @@
|
||||
import type { PhaseOneRuntime, ToolRegistrar } from "./phase-one-runtime";
|
||||
import { registerPhaseOneInteractionTools } from "./phase-one-tools-interaction";
|
||||
import { registerPhaseOneNavigationTools } from "./phase-one-tools-navigation";
|
||||
import { registerPhaseOneSupportTools } from "./phase-one-tools-support";
|
||||
import { registerPhaseOneTabTools } from "./phase-one-tools-tabs";
|
||||
import type { ToolDeps } from "./tools";
|
||||
|
||||
/** Register the first DSH capability-parity tranche on the existing runtime. */
|
||||
export function registerPhaseOneTools(
|
||||
deps: ToolDeps,
|
||||
register: ToolRegistrar,
|
||||
runtime: PhaseOneRuntime,
|
||||
): void {
|
||||
registerPhaseOneInteractionTools(deps, register, runtime);
|
||||
registerPhaseOneTabTools(deps, register, runtime);
|
||||
registerPhaseOneNavigationTools(deps, register, runtime);
|
||||
registerPhaseOneSupportTools(deps, register, runtime);
|
||||
}
|
||||
@@ -24,13 +24,6 @@ export interface TrackedSession {
|
||||
export class SessionRegistry {
|
||||
private readonly sessions = new Map<string, TrackedSession>();
|
||||
private currentId: string | undefined;
|
||||
/**
|
||||
* bsk session id → the DSH session ids that may clean it up: the agent
|
||||
* session that started it plus every ancestor along the seed lineage, so
|
||||
* archiving a conversation at ANY level of the chain reaps its browsers
|
||||
* (see archive-cleanup.ts).
|
||||
*/
|
||||
private readonly dshOwners = new Map<string, ReadonlySet<string>>();
|
||||
/**
|
||||
* Slots reserved by in-flight starts. reserveStart/completeStart/abandonStart
|
||||
* run synchronously around the async spawn, so concurrent starts can never
|
||||
@@ -79,37 +72,12 @@ export class SessionRegistry {
|
||||
/** Forget a session; falls back to the most recent remaining one. */
|
||||
remove(sessionId: string): void {
|
||||
this.sessions.delete(sessionId);
|
||||
this.dshOwners.delete(sessionId);
|
||||
if (this.currentId === sessionId) {
|
||||
const rest = [...this.sessions.values()];
|
||||
this.currentId = rest.length > 0 ? rest[rest.length - 1].sessionId : undefined;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Record which DSH conversation(s) a freshly started bsk session belongs
|
||||
* to (the starting agent's session plus its ancestors). No-op without ids
|
||||
* — e.g. a start whose caller carried no agent identity.
|
||||
*/
|
||||
trackOwner(sessionId: string, dshSessionIds: readonly string[]): void {
|
||||
if (dshSessionIds.length === 0 || !this.sessions.has(sessionId)) return;
|
||||
this.dshOwners.set(sessionId, new Set(dshSessionIds));
|
||||
}
|
||||
|
||||
/** bsk session ids owned by the given DSH conversation (or its descendants). */
|
||||
ownedByDsh(dshSessionId: string): string[] {
|
||||
const owned: string[] = [];
|
||||
for (const [sessionId, owners] of this.dshOwners) {
|
||||
if (owners.has(dshSessionId)) owned.push(sessionId);
|
||||
}
|
||||
return owned;
|
||||
}
|
||||
|
||||
/** The DSH conversation ids owning one bsk session (empty when untracked). */
|
||||
dshOwnersOf(sessionId: string): string[] {
|
||||
return [...(this.dshOwners.get(sessionId) ?? [])];
|
||||
}
|
||||
|
||||
/** Mark an owned session as most recently used (recency order refresh). */
|
||||
private touch(sessionId: string): void {
|
||||
const existing = this.sessions.get(sessionId);
|
||||
|
||||
@@ -2,10 +2,10 @@
|
||||
* Progressive disclosure of the BrowserSkill agent skill through the harness's
|
||||
* official skill seam (`ctx.skills`). The catalog entry (name + description)
|
||||
* is resident; the body is loaded only when the model invokes the `skill`
|
||||
* tool. The dedicated DeepSeek Harness markdown is embedded into a static
|
||||
* module at build time, so registration and every pre-step catalog snapshot
|
||||
* are pure in-memory reads — no disk, no process, no daemon. The CLI-oriented
|
||||
* repository skill is intentionally a separate interface.
|
||||
* tool. The markdown is build-time assembled into a static module (dsh
|
||||
* prelude + the canonical CLI skill body verbatim), so registration and every
|
||||
* pre-step catalog snapshot are pure in-memory reads — no disk, no process,
|
||||
* no daemon.
|
||||
*/
|
||||
|
||||
import type { Context } from "@deepseek-ai/cordis";
|
||||
@@ -26,7 +26,7 @@ interface SkillsLike {
|
||||
}
|
||||
|
||||
/**
|
||||
* Publish the browser skill as an embedded runtime skill. Returns the unregistration
|
||||
* Publish the BSK skill as an embedded runtime skill. Returns the unregistration
|
||||
* disposer. A composition without the `skills` service (or with dsh-tool-skill
|
||||
* retired) leaves the rest of the plugin unaffected — the no-op is silent.
|
||||
*/
|
||||
|
||||
@@ -1,24 +0,0 @@
|
||||
/** Shared model-facing parameter schemas for browser tools. */
|
||||
|
||||
export const SESSION_PARAM = {
|
||||
type: "string",
|
||||
description:
|
||||
"bsk session id to act on; must be one created by this plugin's browser_session_start. " +
|
||||
"Omit to use the current session (the one most recently started or used).",
|
||||
} as const;
|
||||
|
||||
export const TAB_ID_PARAM = {
|
||||
type: "integer",
|
||||
description: "Target tab id. Omit to use the Agent Window's active tab.",
|
||||
} as const;
|
||||
|
||||
export const WAIT_UNTIL_PARAM = {
|
||||
type: "string",
|
||||
enum: ["load", "domcontentloaded", "networkidle", "commit"],
|
||||
description: "Page lifecycle phase to wait for (default: load).",
|
||||
} as const;
|
||||
|
||||
export const TIMEOUT_MS_PARAM = {
|
||||
type: "integer",
|
||||
description: "Command timeout in milliseconds; must be greater than zero.",
|
||||
} as const;
|
||||
@@ -17,14 +17,11 @@ import {
|
||||
type ToolResult,
|
||||
type ToolRunContext,
|
||||
} from "@deepseek-ai/dsh-tools";
|
||||
import { ownerSessionIds } from "./archive-cleanup";
|
||||
import { trySaveScreenshot } from "./image";
|
||||
import { actionForLabel, type ObservationService } from "./observation";
|
||||
import { registerPhaseOneTools } from "./phase-one-tools";
|
||||
import type { KeyedExecutor } from "./queue";
|
||||
import { type BskRunner, bskInstallMessage, isCommandNotFound, parseBskJson } from "./runner";
|
||||
import type { SessionRegistry } from "./sessions";
|
||||
import { SESSION_PARAM } from "./tool-params";
|
||||
|
||||
/** Plugin configuration resolved from the Schemastery schema in index.ts. */
|
||||
export interface PluginConfig {
|
||||
@@ -64,6 +61,13 @@ const DEVICE_PRESETS = [
|
||||
"galaxy-tab-s8",
|
||||
] as const;
|
||||
|
||||
const SESSION_PARAM = {
|
||||
type: "string",
|
||||
description:
|
||||
"bsk session id to act on; must be one created by this plugin's browser_session_start. " +
|
||||
"Omit to use the current session (the one most recently started or used).",
|
||||
} as const;
|
||||
|
||||
/**
|
||||
* Run one bsk command and return its parsed JSON payload, or throw.
|
||||
* `observeSession` marks the run as model-facing work on that session: it is
|
||||
@@ -77,7 +81,6 @@ async function runBsk(
|
||||
args: string[],
|
||||
label: string,
|
||||
observeSession?: string,
|
||||
runnerTimeoutMs?: number,
|
||||
): Promise<unknown> {
|
||||
// beginAction must fire only when the task actually starts inside the
|
||||
// per-session queue — marking it while still queued would overwrite the
|
||||
@@ -94,7 +97,7 @@ async function runBsk(
|
||||
}
|
||||
return deps.runner.run(args, {
|
||||
signal: exec.signal,
|
||||
timeoutMs: runnerTimeoutMs ?? deps.config.defaultTimeoutMs,
|
||||
timeoutMs: deps.config.defaultTimeoutMs,
|
||||
...(observeSession !== undefined ? { tag: observeSession } : {}),
|
||||
});
|
||||
};
|
||||
@@ -242,9 +245,6 @@ export function registerTools(deps: ToolDeps): () => void {
|
||||
browserInstanceId: reply.browser_instance_id,
|
||||
startedAtMs: Date.now(),
|
||||
});
|
||||
// Ownership for archive cleanup: the calling conversation and its
|
||||
// ancestors reap this session when any of them is archived.
|
||||
registry.trackOwner(reply.session_id, ownerSessionIds(deps.ctx, exec.agent?.id));
|
||||
deps.observation.addSession(reply.session_id, args.url);
|
||||
try {
|
||||
if (args.device !== undefined) {
|
||||
@@ -955,12 +955,6 @@ export function registerTools(deps: ToolDeps): () => void {
|
||||
presentResult: presentTerminalResult,
|
||||
}),
|
||||
);
|
||||
registerPhaseOneTools(deps, register, {
|
||||
run: (exec, args, label, observeSession, runnerTimeoutMs) =>
|
||||
runBsk(deps, exec, args, label, observeSession, runnerTimeoutMs),
|
||||
commandLine: (args) => cmdline(deps, args),
|
||||
presentTerminalResult,
|
||||
});
|
||||
return () => {
|
||||
for (const dispose of disposers.splice(0)) dispose();
|
||||
};
|
||||
|
||||
@@ -1,157 +0,0 @@
|
||||
// Archive-triggered cleanup: lineage resolution for browser-session
|
||||
// ownership, the registry's owner index, and the domain/changed watcher
|
||||
// that reaps a freshly archived conversation's bsk sessions.
|
||||
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import { armArchiveCleanup, ownerSessionIds } from "../src/archive-cleanup";
|
||||
import type { ObservationService } from "../src/observation";
|
||||
import { SessionRegistry } from "../src/sessions";
|
||||
|
||||
/** A ctx stub carrying a session store with the given lineage headers. */
|
||||
function ctxWithSessions(headers: Record<string, { parentSession?: string }>) {
|
||||
return {
|
||||
get: (key: string) =>
|
||||
key === "sessions"
|
||||
? {
|
||||
get: (id: string) => (headers[id] === undefined ? undefined : { header: headers[id] }),
|
||||
}
|
||||
: undefined,
|
||||
} as never;
|
||||
}
|
||||
|
||||
describe("ownerSessionIds", () => {
|
||||
it("walks the seed lineage to the root; empty without an agent identity", () => {
|
||||
expect(ownerSessionIds(ctxWithSessions({}), undefined)).toEqual([]);
|
||||
const ctx = ctxWithSessions({
|
||||
child: { parentSession: "parent" },
|
||||
parent: { parentSession: "root" },
|
||||
root: {},
|
||||
});
|
||||
expect(ownerSessionIds(ctx, "child")).toEqual(["child", "parent", "root"]);
|
||||
});
|
||||
|
||||
it("stops the walk at an unloaded ancestor", () => {
|
||||
const ctx = ctxWithSessions({ child: { parentSession: "gone" } });
|
||||
expect(ownerSessionIds(ctx, "child")).toEqual(["child", "gone"]);
|
||||
});
|
||||
|
||||
it("never loops on a malformed parent cycle", () => {
|
||||
const ctx = ctxWithSessions({
|
||||
a: { parentSession: "b" },
|
||||
b: { parentSession: "a" },
|
||||
});
|
||||
expect(ownerSessionIds(ctx, "a")).toEqual(["a", "b"]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("SessionRegistry owner tracking", () => {
|
||||
function start(registry: SessionRegistry, sessionId: string): void {
|
||||
registry.reserveStart();
|
||||
registry.completeStart({ sessionId, startedAtMs: 1 });
|
||||
}
|
||||
|
||||
it("indexes owners and forgets them on remove; ignores empty/unknown ownership", () => {
|
||||
const registry = new SessionRegistry(5);
|
||||
start(registry, "bsk1");
|
||||
start(registry, "bsk2");
|
||||
registry.trackOwner("bsk1", ["conv-a", "root"]);
|
||||
registry.trackOwner("bsk2", ["conv-b"]);
|
||||
expect(registry.ownedByDsh("root")).toEqual(["bsk1"]);
|
||||
expect(registry.ownedByDsh("conv-a")).toEqual(["bsk1"]);
|
||||
expect(registry.ownedByDsh("conv-b")).toEqual(["bsk2"]);
|
||||
expect(registry.ownedByDsh("nobody")).toEqual([]);
|
||||
registry.remove("bsk1");
|
||||
expect(registry.ownedByDsh("root")).toEqual([]);
|
||||
// Empty owner lists and unknown session ids record nothing.
|
||||
registry.trackOwner("bsk2", []);
|
||||
registry.trackOwner("ghost", ["conv-c"]);
|
||||
expect(registry.ownedByDsh("conv-c")).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("armArchiveCleanup", () => {
|
||||
function harness(opts: { archived?: string[] } = {}) {
|
||||
const registry = new SessionRegistry(5);
|
||||
const observation = { stopSession: vi.fn(async () => true) };
|
||||
const listeners = new Set<(change: unknown) => void>();
|
||||
const ctx = {
|
||||
get: (key: string) =>
|
||||
key === "workspaceRegistry" ? { archivedSessionIds: opts.archived ?? [] } : undefined,
|
||||
on: (event: string, listener: (change: unknown) => void) => {
|
||||
if (event === "domain/changed") listeners.add(listener);
|
||||
return () => listeners.delete(listener);
|
||||
},
|
||||
};
|
||||
const emit = (change: unknown) => {
|
||||
for (const listener of [...listeners]) listener(change);
|
||||
};
|
||||
return {
|
||||
registry,
|
||||
observation: observation as unknown as ObservationService,
|
||||
stopSession: observation.stopSession,
|
||||
emit,
|
||||
arm: () =>
|
||||
armArchiveCleanup(ctx as never, registry, observation as unknown as ObservationService),
|
||||
};
|
||||
}
|
||||
|
||||
function startOwned(registry: SessionRegistry, bskId: string, owners: string[]): void {
|
||||
registry.reserveStart();
|
||||
registry.completeStart({ sessionId: bskId, startedAtMs: 1 });
|
||||
registry.trackOwner(bskId, owners);
|
||||
}
|
||||
|
||||
it("stops every bsk session owned by a freshly archived conversation, until disarmed", () => {
|
||||
const h = harness();
|
||||
startOwned(h.registry, "bsk1", ["conv-a", "root"]);
|
||||
startOwned(h.registry, "bsk2", ["conv-b"]);
|
||||
const disarm = h.arm();
|
||||
h.emit({
|
||||
domain: "workspace",
|
||||
table: "",
|
||||
value: { archivedSessionIds: ["root"] },
|
||||
});
|
||||
expect(h.stopSession).toHaveBeenCalledTimes(1);
|
||||
expect(h.stopSession).toHaveBeenCalledWith("bsk1");
|
||||
// After the disposer runs the watcher is silent again.
|
||||
disarm();
|
||||
h.emit({
|
||||
domain: "workspace",
|
||||
table: "",
|
||||
value: { archivedSessionIds: ["root", "conv-b"] },
|
||||
});
|
||||
expect(h.stopSession).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("ignores pre-archived ids, foreign domains, and malformed frames", () => {
|
||||
const h = harness({ archived: ["old-conv"] });
|
||||
startOwned(h.registry, "bsk1", ["old-conv"]);
|
||||
startOwned(h.registry, "bsk2", ["conv-a"]);
|
||||
h.arm();
|
||||
// Seeded from the registry: the pre-archived id must not retro-fire.
|
||||
h.emit({ domain: "workspace", table: "", value: { archivedSessionIds: ["old-conv"] } });
|
||||
h.emit({ domain: "settings", table: "", value: { archivedSessionIds: ["conv-a"] } });
|
||||
h.emit({ domain: "workspace", table: "rows", value: { archivedSessionIds: ["conv-a"] } });
|
||||
h.emit({ domain: "workspace", table: "", value: {} });
|
||||
expect(h.stopSession).not.toHaveBeenCalled();
|
||||
// A genuinely new archive still fires.
|
||||
h.emit({
|
||||
domain: "workspace",
|
||||
table: "",
|
||||
value: { archivedSessionIds: ["old-conv", "conv-a"] },
|
||||
});
|
||||
expect(h.stopSession).toHaveBeenCalledWith("bsk2");
|
||||
});
|
||||
|
||||
it("treats a re-archived session as fresh again after unarchive", () => {
|
||||
const h = harness();
|
||||
startOwned(h.registry, "bsk1", ["conv-a"]);
|
||||
h.arm();
|
||||
h.emit({ domain: "workspace", table: "", value: { archivedSessionIds: ["conv-a"] } });
|
||||
expect(h.stopSession).toHaveBeenCalledTimes(1);
|
||||
// Unarchive, then re-archive: the second archival cleans up again.
|
||||
h.emit({ domain: "workspace", table: "", value: { archivedSessionIds: [] } });
|
||||
h.emit({ domain: "workspace", table: "", value: { archivedSessionIds: ["conv-a"] } });
|
||||
expect(h.stopSession).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
});
|
||||
@@ -4,7 +4,7 @@
|
||||
// upgrade/fallback (mocked documentPictureInPicture). NOTE: use RTL's waitFor
|
||||
// (act-flushing), not vi.waitFor, when asserting store-driven UI updates.
|
||||
|
||||
import { act, cleanup, fireEvent, render, screen, waitFor } from "@testing-library/react";
|
||||
import { cleanup, fireEvent, render, screen, waitFor } from "@testing-library/react";
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { applyResize, ObservationOverlay } from "../../src/client/ObservationOverlay";
|
||||
import { type EventSourceLike, ObservationClientStore } from "../../src/client/observation-store";
|
||||
@@ -157,45 +157,6 @@ describe("ObservationOverlay", () => {
|
||||
await screen.findByText(/s1 · clicking/);
|
||||
});
|
||||
|
||||
it("stop session: the first click only arms the confirm, the second posts", async () => {
|
||||
const h = makeHarness([{ sessionId: "s1", action: "idle", since: Date.now() }]);
|
||||
render(<ObservationOverlay store={h.store} />);
|
||||
await screen.findByText(/s1 · idle/);
|
||||
const stop = screen.getByRole("button", { name: "Stop session s1" });
|
||||
fireEvent.click(stop);
|
||||
// Armed: a plain click must never close an Agent Window by itself.
|
||||
expect(h.fetches.some((f) => f.url === "/bsk-observation/stop")).toBe(false);
|
||||
const confirm = await screen.findByRole("button", { name: "Confirm stop session s1" });
|
||||
expect(confirm.getAttribute("aria-pressed")).toBe("true");
|
||||
fireEvent.click(confirm);
|
||||
await waitFor(() =>
|
||||
expect(
|
||||
h.fetches.some(
|
||||
(f) => f.url === "/bsk-observation/stop" && f.init?.body === '{"sessionId":"s1"}',
|
||||
),
|
||||
).toBe(true),
|
||||
);
|
||||
});
|
||||
|
||||
it("disarms the stop confirm after the expiry window", async () => {
|
||||
const h = makeHarness([{ sessionId: "s1", action: "idle", since: Date.now() }]);
|
||||
render(<ObservationOverlay store={h.store} />);
|
||||
const stop = await screen.findByRole("button", { name: "Stop session s1" });
|
||||
vi.useFakeTimers();
|
||||
try {
|
||||
fireEvent.click(stop);
|
||||
// Synchronous queries only: waitFor-style finds hang under fake timers.
|
||||
expect(screen.getByRole("button", { name: "Confirm stop session s1" })).toBeTruthy();
|
||||
act(() => {
|
||||
vi.advanceTimersByTime(3100);
|
||||
});
|
||||
expect(screen.getByRole("button", { name: "Stop session s1" })).toBeTruthy();
|
||||
expect(h.fetches.some((f) => f.url === "/bsk-observation/stop")).toBe(false);
|
||||
} finally {
|
||||
vi.useRealTimers();
|
||||
}
|
||||
});
|
||||
|
||||
it("resizes within min/max clamps", async () => {
|
||||
const h = makeHarness([BUSY]);
|
||||
render(<ObservationOverlay store={h.store} />);
|
||||
@@ -261,7 +222,6 @@ describe("ObservationOverlay", () => {
|
||||
addEventListener: (name: string, fn: () => void) => {
|
||||
listeners.set(name, [...(listeners.get(name) ?? []), fn]);
|
||||
},
|
||||
close: vi.fn(),
|
||||
} as unknown as Window;
|
||||
});
|
||||
(window as unknown as Record<string, unknown>).documentPictureInPicture = { requestWindow };
|
||||
@@ -276,28 +236,6 @@ describe("ObservationOverlay", () => {
|
||||
await screen.findByTestId("obs-card");
|
||||
});
|
||||
|
||||
it("closes the PiP window when the carrier unmounts", async () => {
|
||||
const h = makeHarness([BUSY]);
|
||||
const pipDoc = document.implementation.createHTMLDocument("pip");
|
||||
const close = vi.fn();
|
||||
const requestWindow = vi.fn(async () => {
|
||||
return {
|
||||
document: pipDoc,
|
||||
addEventListener: () => {},
|
||||
close,
|
||||
} as unknown as Window;
|
||||
});
|
||||
(window as unknown as Record<string, unknown>).documentPictureInPicture = { requestWindow };
|
||||
const { unmount } = render(<ObservationOverlay store={h.store} />);
|
||||
const popout = await screen.findByRole("button", { name: /Pop out/ });
|
||||
fireEvent.click(popout);
|
||||
await waitFor(() => expect(pipDoc.body.textContent).toContain("s1"));
|
||||
// Unmounting (tab close, conversation switch, plugin HMR) closes the PiP
|
||||
// instead of leaving a blank window behind.
|
||||
unmount();
|
||||
expect(close).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("renders the strip for two sessions and pins focus on click", async () => {
|
||||
const older: SessionObservation = {
|
||||
sessionId: "s1",
|
||||
|
||||
@@ -1,331 +0,0 @@
|
||||
// @vitest-environment happy-dom
|
||||
// better-sidebar carrier: tab registration + auto-open, the sidebar-mode
|
||||
// flag hiding the floating overlay, PiP pop-out from the tab, and the
|
||||
// store's refcounted acquire/release across overlapping carriers.
|
||||
|
||||
import { cleanup, render, screen, waitFor } from "@testing-library/react";
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import { ObservationOverlay } from "../../src/client/ObservationOverlay";
|
||||
import {
|
||||
type BetterSidebarLike,
|
||||
OBSERVATION_TAB_PATH,
|
||||
OBSERVATION_TAB_TYPE,
|
||||
ObservationSidebarTab,
|
||||
observationTabOpen,
|
||||
registerObservationSidebar,
|
||||
type SidebarNodeLike,
|
||||
type SidebarStateLike,
|
||||
type TabDescriptorLike,
|
||||
} from "../../src/client/observation-sidebar";
|
||||
import { type EventSourceLike, ObservationClientStore } from "../../src/client/observation-store";
|
||||
import { getSidebarMode, setSidebarMode } from "../../src/client/sidebar-mode";
|
||||
import type { SessionObservation } from "../../src/observation";
|
||||
|
||||
// Owned by the sidebar mock's active conversation ("conv-1") so the scoped
|
||||
// visibility logic lets it through; FOREIGN belongs to another conversation.
|
||||
const BUSY: SessionObservation = {
|
||||
sessionId: "s1",
|
||||
action: "clicking",
|
||||
since: Date.now() - 7000,
|
||||
dshSessionIds: ["conv-1"],
|
||||
};
|
||||
const FOREIGN: SessionObservation = {
|
||||
sessionId: "s9",
|
||||
action: "clicking",
|
||||
since: Date.now() - 5000,
|
||||
dshSessionIds: ["other-conv"],
|
||||
};
|
||||
|
||||
interface Harness {
|
||||
store: ObservationClientStore;
|
||||
es: () => EventSourceLike | undefined;
|
||||
push: (sessions: SessionObservation[]) => void;
|
||||
}
|
||||
|
||||
function makeHarness(initial: SessionObservation[]): Harness {
|
||||
let es: EventSourceLike | undefined;
|
||||
let current = initial;
|
||||
const store = new ObservationClientStore({
|
||||
fetchFn: async (url: string) => {
|
||||
if (url === "/bsk-observation/state") {
|
||||
return { ok: true, json: async () => ({ sessions: current }) };
|
||||
}
|
||||
return { ok: true, json: async () => ({ interrupted: true }) };
|
||||
},
|
||||
eventSourceFactory: () => {
|
||||
es = { onmessage: null, close: vi.fn() };
|
||||
return es;
|
||||
},
|
||||
loadImage: async (id: string) => `blob:${id}`,
|
||||
});
|
||||
return {
|
||||
store,
|
||||
es: () => es,
|
||||
push: (sessions) => {
|
||||
current = sessions;
|
||||
for (const s of sessions) {
|
||||
es?.onmessage?.({ data: JSON.stringify({ type: "upsert", session: s }) });
|
||||
}
|
||||
if (sessions.length === 0) es?.onmessage?.({ data: JSON.stringify({ type: "reset" }) });
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
function leafWith(type: string, panelOpen = true): SidebarStateLike {
|
||||
return {
|
||||
panelOpen,
|
||||
splits: {
|
||||
kind: "split",
|
||||
id: "root",
|
||||
dir: "row",
|
||||
sizes: [1],
|
||||
children: [
|
||||
{
|
||||
kind: "leaf",
|
||||
id: "pane-1",
|
||||
active: type,
|
||||
tabs: [{ id: type, type, title: type }],
|
||||
},
|
||||
],
|
||||
},
|
||||
bottomSplits: { kind: "leaf", id: "pane-2", active: null, tabs: [] },
|
||||
};
|
||||
}
|
||||
|
||||
interface SidebarMock {
|
||||
service: BetterSidebarLike;
|
||||
descriptor: () => TabDescriptorLike;
|
||||
openTab: ReturnType<typeof vi.fn>;
|
||||
disposeTab: ReturnType<typeof vi.fn>;
|
||||
setState: (state: SidebarStateLike | undefined) => void;
|
||||
}
|
||||
|
||||
/** Insert a tab into the first leaf of the main tree (mirrors a real open). */
|
||||
function withTabOpened(state: SidebarStateLike, type: string): SidebarStateLike {
|
||||
let done = false;
|
||||
const walk = (node: SidebarNodeLike): SidebarNodeLike => {
|
||||
if (done) return node;
|
||||
if (node.kind === "leaf") {
|
||||
done = true;
|
||||
return { ...node, active: type, tabs: [...node.tabs, { id: type, type, title: type }] };
|
||||
}
|
||||
return { ...node, children: node.children.map(walk) };
|
||||
};
|
||||
return { ...state, splits: walk(state.splits) };
|
||||
}
|
||||
|
||||
function makeSidebar(
|
||||
initialState: SidebarStateLike | undefined = leafWith("explorer"),
|
||||
): SidebarMock {
|
||||
let descriptor: TabDescriptorLike | undefined;
|
||||
let state: SidebarStateLike | undefined = initialState;
|
||||
const openTab = vi.fn((seed: { type: string }) => {
|
||||
// Mirror the real feedback loop: a created tab lands in the state, so
|
||||
// later evaluations see it open (single-instance dedupe re-focuses).
|
||||
if (state !== undefined && !observationTabOpen(state)) {
|
||||
state = withTabOpened(state, seed.type);
|
||||
}
|
||||
});
|
||||
const disposeTab = vi.fn();
|
||||
const service: BetterSidebarLike = {
|
||||
registerTab: (next) => {
|
||||
descriptor = next;
|
||||
return disposeTab;
|
||||
},
|
||||
openTab,
|
||||
getSnapshot: () => ({ sessionId: "conv-1", state }),
|
||||
isTabEnabled: () => true,
|
||||
};
|
||||
return {
|
||||
service,
|
||||
descriptor: () => {
|
||||
if (descriptor === undefined) throw new Error("no tab registered");
|
||||
return descriptor;
|
||||
},
|
||||
openTab,
|
||||
disposeTab,
|
||||
setState: (next) => {
|
||||
state = next;
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
afterEach(() => {
|
||||
cleanup();
|
||||
setSidebarMode(false);
|
||||
});
|
||||
|
||||
describe("registerObservationSidebar", () => {
|
||||
it("registers a single-instance tab and holds the feed for its lifetime", () => {
|
||||
const h = makeHarness([]);
|
||||
const sidebar = makeSidebar();
|
||||
expect(getSidebarMode()).toBe(false);
|
||||
const dispose = registerObservationSidebar(sidebar.service, h.store);
|
||||
expect(getSidebarMode()).toBe(true);
|
||||
// The fiber holds the observation feed itself, so the auto-open watcher
|
||||
// sees new sessions even while the tab is closed.
|
||||
expect(h.es()).toBeDefined();
|
||||
const descriptor = sidebar.descriptor();
|
||||
expect(descriptor.id).toBe(OBSERVATION_TAB_TYPE);
|
||||
expect(descriptor.single).toBe(true);
|
||||
expect(descriptor.title).toBe("Browser Skill");
|
||||
dispose();
|
||||
expect(getSidebarMode()).toBe(false);
|
||||
expect(sidebar.disposeTab).toHaveBeenCalledTimes(1);
|
||||
expect(h.es()?.close).toHaveBeenCalled?.();
|
||||
});
|
||||
|
||||
it("opens the tab right away when a session is already live at activation", async () => {
|
||||
const h = makeHarness([BUSY]);
|
||||
const sidebar = makeSidebar();
|
||||
registerObservationSidebar(sidebar.service, h.store);
|
||||
// The initial state fetch lands asynchronously; the 0→N watcher then fires.
|
||||
await waitFor(() =>
|
||||
expect(sidebar.openTab).toHaveBeenCalledWith({
|
||||
type: OBSERVATION_TAB_TYPE,
|
||||
path: OBSERVATION_TAB_PATH,
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it("auto-opens on the first session but never re-focuses an open tab", async () => {
|
||||
const h = makeHarness([]);
|
||||
const sidebar = makeSidebar();
|
||||
registerObservationSidebar(sidebar.service, h.store);
|
||||
expect(sidebar.openTab).not.toHaveBeenCalled();
|
||||
// First session arrives: the tab lands in the sidebar.
|
||||
h.push([BUSY]);
|
||||
await waitFor(() => expect(sidebar.openTab).toHaveBeenCalledTimes(1));
|
||||
// The sidebar now reports the tab open (persisted/restored): further
|
||||
// session arrivals must not steal focus from whatever the user reads.
|
||||
sidebar.setState(leafWith(OBSERVATION_TAB_TYPE));
|
||||
h.push([]);
|
||||
h.push([{ ...BUSY, sessionId: "s2" }]);
|
||||
h.push([]);
|
||||
h.push([{ ...BUSY, sessionId: "s3" }]);
|
||||
await waitFor(() => expect(h.store.getSnapshot().sessions.length).toBe(1));
|
||||
expect(sidebar.openTab).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("does not open while the sidebar has no active session state", () => {
|
||||
const h = makeHarness([BUSY]);
|
||||
const sidebar = makeSidebar(undefined);
|
||||
registerObservationSidebar(sidebar.service, h.store);
|
||||
expect(sidebar.openTab).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("nudges an existing tab back into sight when the panel is collapsed", async () => {
|
||||
const h = makeHarness([]);
|
||||
// Tab already open (persisted), but the panel is collapsed: a new
|
||||
// session should surface the tracking view again (focus + expand via a
|
||||
// content open), like the floating card reappearing.
|
||||
const sidebar = makeSidebar(leafWith(OBSERVATION_TAB_TYPE, false));
|
||||
registerObservationSidebar(sidebar.service, h.store);
|
||||
h.push([BUSY]);
|
||||
await waitFor(() =>
|
||||
expect(sidebar.openTab).toHaveBeenCalledWith({
|
||||
type: OBSERVATION_TAB_TYPE,
|
||||
path: OBSERVATION_TAB_PATH,
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it("shows the visible session count as the tab badge", () => {
|
||||
const h = makeHarness([]);
|
||||
const sidebar = makeSidebar();
|
||||
registerObservationSidebar(sidebar.service, h.store);
|
||||
const badge = sidebar.descriptor().badge;
|
||||
const scope = { sessionId: "conv-1" };
|
||||
expect(badge?.(undefined, scope, undefined)).toBeNull();
|
||||
h.store.acquire();
|
||||
h.push([BUSY, FOREIGN]);
|
||||
expect(badge?.(undefined, scope, undefined)).toBe(1);
|
||||
expect(badge?.(undefined, { sessionId: "other-conv" }, undefined)).toBe(1);
|
||||
expect(badge?.(undefined, { sessionId: "nobody" }, undefined)).toBeNull();
|
||||
h.store.release();
|
||||
});
|
||||
|
||||
it("does not auto-open for sessions owned by other conversations", async () => {
|
||||
const h = makeHarness([]);
|
||||
const sidebar = makeSidebar();
|
||||
registerObservationSidebar(sidebar.service, h.store);
|
||||
h.push([FOREIGN]);
|
||||
await waitFor(() => expect(h.store.getSnapshot().sessions.length).toBe(1));
|
||||
expect(sidebar.openTab).not.toHaveBeenCalled();
|
||||
// A session visible to the active conversation does open the tab.
|
||||
h.push([BUSY]);
|
||||
await waitFor(() => expect(sidebar.openTab).toHaveBeenCalledTimes(1));
|
||||
});
|
||||
});
|
||||
|
||||
describe("observationTabOpen", () => {
|
||||
it("finds the tab in either workbench tree", () => {
|
||||
expect(observationTabOpen(undefined)).toBe(false);
|
||||
expect(observationTabOpen(leafWith("explorer"))).toBe(false);
|
||||
expect(observationTabOpen(leafWith(OBSERVATION_TAB_TYPE))).toBe(true);
|
||||
const inBottom: SidebarStateLike = {
|
||||
splits: { kind: "leaf", id: "p1", active: null, tabs: [] },
|
||||
bottomSplits: {
|
||||
kind: "leaf",
|
||||
id: "p2",
|
||||
active: OBSERVATION_TAB_TYPE,
|
||||
tabs: [{ id: OBSERVATION_TAB_TYPE, type: OBSERVATION_TAB_TYPE, title: "Browser" }],
|
||||
},
|
||||
};
|
||||
expect(observationTabOpen(inBottom)).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe("ObservationSidebarTab", () => {
|
||||
it("renders the tracking view without card chrome (no collapse, no drag header)", async () => {
|
||||
const h = makeHarness([BUSY]);
|
||||
render(<ObservationSidebarTab store={h.store} scopeId="conv-1" />);
|
||||
await screen.findByText(/s1 · clicking/);
|
||||
expect(screen.queryByRole("button", { name: "Collapse" })).toBeNull();
|
||||
expect(screen.getByTestId("obs-header").dataset.draggable).toBeUndefined();
|
||||
// No resize handles — the sidebar owns the geometry.
|
||||
expect(screen.queryByTestId("obs-resize-se")).toBeNull();
|
||||
// Session controls ride along: interrupt + stop-with-confirm.
|
||||
expect(screen.getByRole("button", { name: "Stop session s1" })).toBeTruthy();
|
||||
});
|
||||
|
||||
it("scopes the view to the sidebar's conversation", async () => {
|
||||
const h = makeHarness([BUSY, FOREIGN]);
|
||||
render(<ObservationSidebarTab store={h.store} scopeId="conv-1" />);
|
||||
// Focus and content belong to the owned session; the foreign one never
|
||||
// reaches the stage, the status line, or the strip (which needs 2+).
|
||||
await screen.findByText(/s1 · clicking/);
|
||||
expect(screen.queryByText(/s9/)).toBeNull();
|
||||
expect(screen.queryByTestId("obs-strip")).toBeNull();
|
||||
// Scoping to the other conversation swaps what's visible.
|
||||
render(<ObservationSidebarTab store={h.store} scopeId="other-conv" />);
|
||||
await screen.findByText(/s9 · clicking/);
|
||||
});
|
||||
});
|
||||
|
||||
describe("carrier switching", () => {
|
||||
it("hides the floating overlay while sidebar mode is active", async () => {
|
||||
const h = makeHarness([]);
|
||||
const { container } = render(<ObservationOverlay store={h.store} />);
|
||||
h.push([BUSY]);
|
||||
await screen.findByTestId("obs-card");
|
||||
setSidebarMode(true);
|
||||
await waitFor(() => expect(container.firstChild).toBeNull());
|
||||
setSidebarMode(false);
|
||||
await screen.findByTestId("obs-card");
|
||||
});
|
||||
|
||||
it("keeps the feed alive across overlapping carriers (refcount)", async () => {
|
||||
const h = makeHarness([]);
|
||||
const overlay = render(<ObservationOverlay store={h.store} />);
|
||||
await waitFor(() => expect(h.es()).toBeDefined());
|
||||
// A second carrier mounts (the sidebar tab), then the overlay unmounts:
|
||||
// the stream must survive for the remaining consumer.
|
||||
const tab = render(<ObservationSidebarTab store={h.store} scopeId="conv-1" />);
|
||||
overlay.unmount();
|
||||
h.push([BUSY]);
|
||||
await screen.findByText(/s1 · clicking/);
|
||||
expect(h.es()?.close ?? vi.fn()).not.toHaveBeenCalled();
|
||||
tab.unmount();
|
||||
});
|
||||
});
|
||||
@@ -137,9 +137,6 @@ describe("client plugin registration", () => {
|
||||
const sessions = { binding: () => undefined };
|
||||
const ctx = {
|
||||
get: (key: string) => (key === "sessions" ? sessions : undefined),
|
||||
// cordis ctx.inject: the betterSidebar carrier upgrade stays dormant in
|
||||
// this composition (the callback only runs once the service exists).
|
||||
inject: (_deps: string[], _fn: (injected: unknown) => unknown) => {},
|
||||
slots: {
|
||||
inject: (_name: string, fn: () => unknown) => fn(),
|
||||
register: (slot: { name: string; key?: string; id?: string }, view: unknown) => {
|
||||
|
||||
@@ -61,8 +61,6 @@ function fakeRunner(
|
||||
opts: {
|
||||
screenshotFails?: boolean;
|
||||
screenshotNotFound?: boolean;
|
||||
stopNotFound?: boolean;
|
||||
stopFails?: boolean;
|
||||
killCount?: number;
|
||||
screenshotBytes?: () => Uint8Array;
|
||||
} = {},
|
||||
@@ -74,21 +72,6 @@ function fakeRunner(
|
||||
killed,
|
||||
async run(args: string[], options: BskRunOptions = {}): Promise<BskRunResult> {
|
||||
calls.push({ args, options });
|
||||
if (args[0] === "session" && args[1] === "stop") {
|
||||
if (opts.stopNotFound) {
|
||||
return {
|
||||
code: 4,
|
||||
stdout: JSON.stringify({ code: "session_not_found", message: "no such session" }),
|
||||
stderr: "",
|
||||
timedOut: false,
|
||||
aborted: false,
|
||||
};
|
||||
}
|
||||
if (opts.stopFails) {
|
||||
return { code: 1, stdout: "", stderr: "boom", timedOut: false, aborted: false };
|
||||
}
|
||||
return { code: 0, stdout: "{}", stderr: "", timedOut: false, aborted: false };
|
||||
}
|
||||
if (args[0] === "screenshot") {
|
||||
if (opts.screenshotNotFound) {
|
||||
return {
|
||||
@@ -197,23 +180,6 @@ describe("state machine", () => {
|
||||
expect(events[events.length - 1].type).toBe("reset");
|
||||
});
|
||||
|
||||
it("stamps the registry-recorded DSH owners onto new entries", () => {
|
||||
const registry = new SessionRegistry(5);
|
||||
own(registry, "s1");
|
||||
own(registry, "s2");
|
||||
registry.trackOwner("s1", ["conv-a", "root"]);
|
||||
const { service } = setup({ registry });
|
||||
service.addSession("s1");
|
||||
service.addSession("s2");
|
||||
expect(service.getState()).toEqual([
|
||||
{ sessionId: "s1", action: "idle", since: 1_000_000, dshSessionIds: ["conv-a", "root"] },
|
||||
{ sessionId: "s2", action: "idle", since: 1_000_000 },
|
||||
]);
|
||||
// The owner stamp survives later upserts (action/url/frame churn).
|
||||
service.beginAction("s1", "clicking");
|
||||
expect(service.getState()[0].dshSessionIds).toEqual(["conv-a", "root"]);
|
||||
});
|
||||
|
||||
it("ignores instrumentation for unknown sessions (owned-only by construction)", () => {
|
||||
const { service } = setup({});
|
||||
service.beginAction("foreign", "clicking");
|
||||
@@ -341,57 +307,6 @@ describe("interrupt routing", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("stopSession", () => {
|
||||
it("stops an owned session: kills in-flight tools, runs session stop, removes the entry", async () => {
|
||||
const registry = new SessionRegistry(5);
|
||||
own(registry, "s1");
|
||||
const runner = fakeRunner();
|
||||
const { service, events } = setup({ registry, runner });
|
||||
service.addSession("s1");
|
||||
expect(registry.isOwned("s1")).toBe(true);
|
||||
|
||||
await expect(service.stopSession("s1")).resolves.toBe(true);
|
||||
expect(runner.killed).toEqual(["s1"]);
|
||||
const stop = runner.calls.find((c) => c.args[0] === "session" && c.args[1] === "stop");
|
||||
expect(stop?.args).toEqual(["session", "stop", "s1"]);
|
||||
expect(registry.isOwned("s1")).toBe(false);
|
||||
expect(service.getState()).toEqual([]);
|
||||
expect(events[events.length - 1]).toMatchObject({ type: "remove" });
|
||||
});
|
||||
|
||||
it("refuses foreign sessions without touching the runner", async () => {
|
||||
const registry = new SessionRegistry(5);
|
||||
own(registry, "s1");
|
||||
const runner = fakeRunner();
|
||||
const { service } = setup({ registry, runner });
|
||||
await expect(service.stopSession("foreign")).resolves.toBe(false);
|
||||
expect(runner.killed).toEqual([]);
|
||||
expect(runner.calls).toEqual([]);
|
||||
});
|
||||
|
||||
it("stops idempotently when the daemon already forgot the session (dead entries)", async () => {
|
||||
const registry = new SessionRegistry(5);
|
||||
own(registry, "s1");
|
||||
const runner = fakeRunner({ stopNotFound: true });
|
||||
const { service, events } = setup({ registry, runner });
|
||||
service.addSession("s1");
|
||||
await expect(service.stopSession("s1")).resolves.toBe(true);
|
||||
expect(registry.isOwned("s1")).toBe(false);
|
||||
expect(events[events.length - 1]).toMatchObject({ type: "remove" });
|
||||
});
|
||||
|
||||
it("returns false and keeps the entry when the stop itself fails", async () => {
|
||||
const registry = new SessionRegistry(5);
|
||||
own(registry, "s1");
|
||||
const runner = fakeRunner({ stopFails: true });
|
||||
const { service } = setup({ registry, runner });
|
||||
service.addSession("s1");
|
||||
await expect(service.stopSession("s1")).resolves.toBe(false);
|
||||
expect(registry.isOwned("s1")).toBe(true);
|
||||
expect(service.getState().map((s) => s.sessionId)).toEqual(["s1"]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("HTTP/SSE interface", () => {
|
||||
interface RecordedRoute {
|
||||
path: string;
|
||||
@@ -490,45 +405,11 @@ describe("HTTP/SSE interface", () => {
|
||||
await interruptRoute?.handler(postReq, interruptRes.res as never);
|
||||
expect(JSON.parse(interruptRes.res.body())).toEqual({ interrupted: true });
|
||||
|
||||
// stop (POST {sessionId}): owned session stops and leaves the state
|
||||
const stopRoute = routes.get("/bsk-observation/stop");
|
||||
const stopRes = fakeRes();
|
||||
await stopRoute?.handler(postReq, stopRes.res as never);
|
||||
// The stop is async behind the response-less handler — poll the state.
|
||||
await waitFor(() => service.getState().length === 0);
|
||||
expect(JSON.parse(stopRes.res.body())).toEqual({ stopped: true });
|
||||
|
||||
dispose();
|
||||
expect(routes.size).toBe(0);
|
||||
service.dispose();
|
||||
});
|
||||
|
||||
it("refuses a stop without a sessionId", async () => {
|
||||
const registry = new SessionRegistry(5);
|
||||
own(registry, "s1");
|
||||
const { service } = setup({ registry });
|
||||
const { routes, webServer } = routeHarness();
|
||||
const ctx = { get: (key: string) => (key === "webServer" ? webServer : undefined) } as never;
|
||||
const dispose = registerObservationRoutes(ctx, service);
|
||||
|
||||
const stopRoute = routes.get("/bsk-observation/stop");
|
||||
const res = fakeRes();
|
||||
const emptyReq = fakeReq({
|
||||
method: "POST",
|
||||
headers: { host: "127.0.0.1:3999", "content-type": "application/json" },
|
||||
on: (event: string, fn: (chunk?: string) => void) => {
|
||||
if (event === "data") fn("{}");
|
||||
if (event === "end") fn();
|
||||
},
|
||||
});
|
||||
await stopRoute?.handler(emptyReq, res.res as never);
|
||||
expect(res.res.status).toBe(400);
|
||||
expect(registry.isOwned("s1")).toBe(true);
|
||||
|
||||
dispose();
|
||||
service.dispose();
|
||||
});
|
||||
|
||||
it("fences off non-loopback, cross-site, and simple-request traffic", async () => {
|
||||
const registry = new SessionRegistry(5);
|
||||
own(registry, "s1");
|
||||
@@ -649,9 +530,6 @@ describe("actionForLabel", () => {
|
||||
expect(actionForLabel("navigate")).toBe("navigating");
|
||||
expect(actionForLabel("screenshot")).toBe("capturing");
|
||||
expect(actionForLabel("session start")).toBe("starting");
|
||||
expect(actionForLabel("tab borrow")).toBe("borrowing tab");
|
||||
expect(actionForLabel("request-help")).toBe("waiting for user");
|
||||
expect(actionForLabel("wait-for-navigation")).toBe("waiting for navigation");
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
// Browser skill injection: registration wiring, catalog content, progressive-load
|
||||
// BSK skill injection: registration wiring, catalog content, progressive-load
|
||||
// weight, and silent degradation without the skill seam.
|
||||
|
||||
import { describe, expect, it } from "vitest";
|
||||
@@ -9,7 +9,7 @@ function fakeCtx(skills?: unknown) {
|
||||
}
|
||||
|
||||
describe("registerBskSkill", () => {
|
||||
it("registers the catalog entry with the DSH browser-tool body", () => {
|
||||
it("registers the catalog entry with the assembled body", () => {
|
||||
const registrations: { skill: Record<string, unknown>; disposer: () => void }[] = [];
|
||||
const skills = {
|
||||
register(skill: Record<string, unknown>) {
|
||||
@@ -26,51 +26,23 @@ describe("registerBskSkill", () => {
|
||||
expect(skill.name).toBe("browser-skill");
|
||||
expect(typeof skill.description).toBe("string");
|
||||
expect(String(skill.description)).toContain("browser_*");
|
||||
// body = dsh prelude + canonical CLI skill body + dsh postlude
|
||||
const content = String(skill.content);
|
||||
const mentionedTools = [...new Set(content.match(/\bbrowser_[a-z][a-z_]*\b/g) ?? [])].sort();
|
||||
expect(mentionedTools).toEqual([
|
||||
"browser_click",
|
||||
"browser_console",
|
||||
"browser_emulate",
|
||||
"browser_fill",
|
||||
"browser_get_html",
|
||||
"browser_hover",
|
||||
"browser_navigate",
|
||||
"browser_navigate_back",
|
||||
"browser_navigate_forward",
|
||||
"browser_network",
|
||||
"browser_observe",
|
||||
"browser_press",
|
||||
"browser_reload",
|
||||
"browser_request_help",
|
||||
"browser_screenshot",
|
||||
"browser_select",
|
||||
"browser_session_list",
|
||||
"browser_session_start",
|
||||
"browser_session_stop",
|
||||
"browser_snapshot",
|
||||
"browser_tab_borrow",
|
||||
"browser_tab_close",
|
||||
"browser_tab_create",
|
||||
"browser_tab_list",
|
||||
"browser_tab_return",
|
||||
"browser_tab_select",
|
||||
"browser_wait_for_navigation",
|
||||
"browser_window_resize",
|
||||
]);
|
||||
expect(String(skill.description)).not.toMatch(/\bbsk\b/i);
|
||||
expect(content).not.toMatch(/\bbsk\b/i);
|
||||
expect(content).not.toMatch(/```(?:bash|sh|shell)\b/i);
|
||||
expect(content).not.toMatch(/--[a-z]/);
|
||||
expect(content).toContain("All browser operations must use the injected tools directly");
|
||||
expect(content).toContain("browser_observe");
|
||||
expect(String(skill.description)).toContain("never invoke bsk through bash or shell");
|
||||
expect(content).toContain("Always call those injected `browser_*` tools");
|
||||
expect(content).toContain("Never use `bash`, `shell`, `exec`");
|
||||
expect(content).toContain("## DSH routing reminder");
|
||||
expect(content.trimEnd()).toMatch(
|
||||
/always begin with[\s\S]*`browser_session_start`[\s\S]*`browser_session_stop`\.$/,
|
||||
);
|
||||
expect(content).toContain("Owned sessions only");
|
||||
expect(content).toContain("Mandatory workflow");
|
||||
expect(content).toContain("Refs invalidate after navigation");
|
||||
expect(content).toContain("evaluation and interaction recording are intentionally unsupported");
|
||||
expect(content.length).toBeGreaterThan(10_000);
|
||||
expect(skill.source).toBe("bundled");
|
||||
// Source frontmatter is registration metadata and must not leak into the body.
|
||||
expect(content.startsWith("---")).toBe(false);
|
||||
expect(content).not.toContain("name: browser-skill\ndescription:");
|
||||
// the original YAML frontmatter must not leak into the body
|
||||
expect(content.includes("name: browser-skill\ndescription:")).toBe(false);
|
||||
// disposer passthrough
|
||||
unregister();
|
||||
expect(skill.__disposed).toBe(true);
|
||||
|
||||
@@ -142,33 +142,16 @@ describe("tool registration", () => {
|
||||
const { tools } = setup({});
|
||||
expect([...tools.keys()].sort()).toEqual([
|
||||
"browser_click",
|
||||
"browser_console",
|
||||
"browser_emulate",
|
||||
"browser_fill",
|
||||
"browser_get_html",
|
||||
"browser_hover",
|
||||
"browser_navigate",
|
||||
"browser_navigate_back",
|
||||
"browser_navigate_forward",
|
||||
"browser_network",
|
||||
"browser_observe",
|
||||
"browser_press",
|
||||
"browser_reload",
|
||||
"browser_request_help",
|
||||
"browser_screenshot",
|
||||
"browser_select",
|
||||
"browser_session_list",
|
||||
"browser_session_start",
|
||||
"browser_session_stop",
|
||||
"browser_snapshot",
|
||||
"browser_tab_borrow",
|
||||
"browser_tab_close",
|
||||
"browser_tab_create",
|
||||
"browser_tab_list",
|
||||
"browser_tab_return",
|
||||
"browser_tab_select",
|
||||
"browser_wait_for_navigation",
|
||||
"browser_window_resize",
|
||||
]);
|
||||
});
|
||||
|
||||
@@ -453,336 +436,6 @@ describe("interaction tools", () => {
|
||||
/mutually exclusive/,
|
||||
);
|
||||
});
|
||||
|
||||
it("browser_hover and browser_select map their structured arguments", async () => {
|
||||
const { tools, calls } = setup({
|
||||
"session start": START_REPLY("s1"),
|
||||
hover: { tab_id: 7, used_ref: "@e3", x: 12, y: 24 },
|
||||
select: {
|
||||
tab_id: 7,
|
||||
used_ref: "@e4",
|
||||
multiple: true,
|
||||
selected_values: ["a", "b"],
|
||||
selected_labels: ["Alpha", "Beta"],
|
||||
},
|
||||
});
|
||||
await startSession(tools);
|
||||
const hovered = (await tools
|
||||
.get("browser_hover")
|
||||
?.execute(
|
||||
{ target: "@e3", modifiers: ["shift"], settleMs: 250, timeoutMs: 2000 },
|
||||
makeExec(),
|
||||
)) as { x: number; y: number };
|
||||
expect(hovered).toMatchObject({ x: 12, y: 24 });
|
||||
expect(calls[1].args).toEqual([
|
||||
"hover",
|
||||
"--session",
|
||||
"s1",
|
||||
"--modifiers",
|
||||
"shift",
|
||||
"--settle",
|
||||
"250ms",
|
||||
"--timeout",
|
||||
"2000ms",
|
||||
"@e3",
|
||||
]);
|
||||
|
||||
const selected = (await tools
|
||||
.get("browser_select")
|
||||
?.execute({ target: "@e4", values: ["a", "b"], tabId: 7 }, makeExec())) as {
|
||||
selectedValues: string[];
|
||||
selectedLabels: string[];
|
||||
};
|
||||
expect(selected).toMatchObject({
|
||||
selectedValues: ["a", "b"],
|
||||
selectedLabels: ["Alpha", "Beta"],
|
||||
});
|
||||
expect(calls[2].args).toEqual([
|
||||
"select",
|
||||
"--session",
|
||||
"s1",
|
||||
"--tab-id",
|
||||
"7",
|
||||
"--value",
|
||||
"a",
|
||||
"--value",
|
||||
"b",
|
||||
"@e4",
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("phase-one tab tools", () => {
|
||||
it("maps list/create/close/select/borrow/return through the owned session", async () => {
|
||||
const { tools, calls } = setup({
|
||||
"session start": START_REPLY("s1"),
|
||||
"tab list": {
|
||||
tabs: [
|
||||
{
|
||||
tab_id: 7,
|
||||
title: "Example",
|
||||
url: "https://example.test",
|
||||
window_id: 2,
|
||||
active: true,
|
||||
scope: "agent",
|
||||
},
|
||||
],
|
||||
},
|
||||
"tab create": { tab_id: 8, window_id: 2, url: "https://new.test" },
|
||||
"tab close": { tab_id: 8 },
|
||||
"tab select": { tab_id: 7, window_id: 2 },
|
||||
"tab borrow": {
|
||||
tab_id: 20,
|
||||
original_window_id: 9,
|
||||
original_index: 1,
|
||||
agent_window_id: 2,
|
||||
},
|
||||
"tab return": {
|
||||
tab_id: 20,
|
||||
returned_to_window_id: 9,
|
||||
returned_to_index: 1,
|
||||
fallback: false,
|
||||
},
|
||||
});
|
||||
await startSession(tools);
|
||||
|
||||
const listed = (await tools
|
||||
.get("browser_tab_list")
|
||||
?.execute({ scope: "agent" }, makeExec())) as { tabs: { tabId: number }[] };
|
||||
expect(listed.tabs[0].tabId).toBe(7);
|
||||
const created = (await tools
|
||||
.get("browser_tab_create")
|
||||
?.execute({ url: "https://new.test", active: false, index: 1 }, makeExec())) as {
|
||||
tabId: number;
|
||||
};
|
||||
expect(created.tabId).toBe(8);
|
||||
await tools.get("browser_tab_close")?.execute({ tabId: 8 }, makeExec());
|
||||
await tools.get("browser_tab_select")?.execute({ tabId: 7 }, makeExec());
|
||||
const borrowed = (await tools
|
||||
.get("browser_tab_borrow")
|
||||
?.execute({ tabId: 20 }, makeExec())) as { originalWindowId: number };
|
||||
expect(borrowed.originalWindowId).toBe(9);
|
||||
const returned = (await tools
|
||||
.get("browser_tab_return")
|
||||
?.execute({ tabId: 20 }, makeExec())) as { fallback: boolean };
|
||||
expect(returned.fallback).toBe(false);
|
||||
|
||||
expect(calls.slice(1).map((call) => call.args)).toEqual([
|
||||
["tab", "list", "--session", "s1", "--scope", "agent"],
|
||||
[
|
||||
"tab",
|
||||
"create",
|
||||
"--session",
|
||||
"s1",
|
||||
"--url",
|
||||
"https://new.test",
|
||||
"--no-active",
|
||||
"--index",
|
||||
"1",
|
||||
],
|
||||
["tab", "close", "8", "--session", "s1"],
|
||||
["tab", "select", "7", "--session", "s1"],
|
||||
["tab", "borrow", "20", "--session", "s1"],
|
||||
["tab", "return", "20", "--session", "s1"],
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("phase-one navigation tools", () => {
|
||||
it("maps back/forward/reload/wait and preserves timeout budgets", async () => {
|
||||
const { tools, calls } = setup({
|
||||
"session start": START_REPLY("s1"),
|
||||
"navigate-back": {
|
||||
tab_id: 7,
|
||||
previous_url: "https://b.test",
|
||||
final_url: "https://a.test",
|
||||
reached: "load",
|
||||
},
|
||||
"navigate-forward": {
|
||||
tab_id: 7,
|
||||
previous_url: "https://a.test",
|
||||
final_url: "https://b.test",
|
||||
reached: "load",
|
||||
},
|
||||
reload: {
|
||||
tab_id: 7,
|
||||
previous_url: "https://b.test",
|
||||
final_url: "https://b.test",
|
||||
reached: "networkidle",
|
||||
},
|
||||
"wait-for-navigation": { tab_id: 7, reached: "timeout", error_text: "no navigation" },
|
||||
});
|
||||
await startSession(tools);
|
||||
await tools
|
||||
.get("browser_navigate_back")
|
||||
?.execute({ tabId: 7, waitUntil: "commit", timeoutMs: 1000 }, makeExec());
|
||||
await tools.get("browser_navigate_forward")?.execute({}, makeExec());
|
||||
await tools
|
||||
.get("browser_reload")
|
||||
?.execute({ hard: true, waitUntil: "networkidle" }, makeExec());
|
||||
const waited = (await tools
|
||||
.get("browser_wait_for_navigation")
|
||||
?.execute({ timeoutMs: 2500 }, makeExec())) as { reached: string; errorText: string };
|
||||
expect(waited).toMatchObject({ reached: "timeout", errorText: "no navigation" });
|
||||
expect(calls[1].args).toEqual([
|
||||
"navigate-back",
|
||||
"--session",
|
||||
"s1",
|
||||
"--tab-id",
|
||||
"7",
|
||||
"--wait-until",
|
||||
"commit",
|
||||
"--timeout",
|
||||
"1000ms",
|
||||
]);
|
||||
expect(calls[2].args).toEqual(["navigate-forward", "--session", "s1"]);
|
||||
expect(calls[3].args).toEqual([
|
||||
"reload",
|
||||
"--session",
|
||||
"s1",
|
||||
"--wait-until",
|
||||
"networkidle",
|
||||
"--hard",
|
||||
]);
|
||||
expect(calls[4].args).toEqual([
|
||||
"wait-for-navigation",
|
||||
"--session",
|
||||
"s1",
|
||||
"--timeout",
|
||||
"2500ms",
|
||||
]);
|
||||
expect(calls[4].options.timeoutMs).toBe(120_000);
|
||||
});
|
||||
});
|
||||
|
||||
describe("phase-one support tools", () => {
|
||||
it("maps request-help targets and structured completion criteria", async () => {
|
||||
const { tools, calls } = setup({
|
||||
"session start": START_REPLY("s1"),
|
||||
"request-help": {
|
||||
outcome: "completed",
|
||||
completed_by: "system",
|
||||
note: "signed in",
|
||||
tab_id: 7,
|
||||
resolved_targets: [{ matched: true, ref: "@e3" }],
|
||||
},
|
||||
});
|
||||
await startSession(tools);
|
||||
const value = (await tools.get("browser_request_help")?.execute(
|
||||
{
|
||||
prompt: "Please sign in",
|
||||
title: "Login required",
|
||||
targets: ["@e3", "#submit"],
|
||||
timeoutMs: 300_000,
|
||||
completionCriteria: {
|
||||
any: [{ urlContains: "/dashboard" }, { selectorExists: "[data-user]" }],
|
||||
stableForMs: 1000,
|
||||
},
|
||||
},
|
||||
makeExec(),
|
||||
)) as { outcome: string; note: string };
|
||||
expect(value).toMatchObject({ outcome: "completed", note: "signed in" });
|
||||
expect(calls[1].options.timeoutMs).toBe(315_000);
|
||||
expect(calls[1].args.slice(0, 8)).toEqual([
|
||||
"request-help",
|
||||
"--session",
|
||||
"s1",
|
||||
"--prompt",
|
||||
"Please sign in",
|
||||
"--timeout",
|
||||
"300000ms",
|
||||
"--title",
|
||||
]);
|
||||
const criteriaIndex = calls[1].args.indexOf("--completion-criteria");
|
||||
expect(JSON.parse(calls[1].args[criteriaIndex + 1])).toEqual({
|
||||
any: [{ url_contains: "/dashboard" }, { selector_exists: "[data-user]" }],
|
||||
stable_for_ms: 1000,
|
||||
});
|
||||
});
|
||||
|
||||
it("maps get-html, console, and network results without raw shell access", async () => {
|
||||
const { tools, calls } = setup({
|
||||
"session start": START_REPLY("s1"),
|
||||
"get-html": { tab_id: 7, html: "<main>Hi</main>", truncated: false, byte_size: 15 },
|
||||
console: {
|
||||
tab_id: 7,
|
||||
entries: [
|
||||
{
|
||||
sequence: 3,
|
||||
kind: "exception",
|
||||
level: "error",
|
||||
text: "boom",
|
||||
stack_trace: [{ function_name: "run", url: "https://a.test/app.js", line: 4 }],
|
||||
},
|
||||
],
|
||||
next_since: 3,
|
||||
truncated: false,
|
||||
},
|
||||
network: {
|
||||
tab_id: 7,
|
||||
entries: [
|
||||
{
|
||||
sequence: 5,
|
||||
kind: "response",
|
||||
method: "GET",
|
||||
url: "https://a.test/api",
|
||||
status: 200,
|
||||
mime_type: "application/json",
|
||||
},
|
||||
],
|
||||
next_since: 5,
|
||||
truncated: false,
|
||||
},
|
||||
});
|
||||
await startSession(tools);
|
||||
const html = (await tools
|
||||
.get("browser_get_html")
|
||||
?.execute({ ref: "@e2", maxBytes: 2048 }, makeExec())) as { html: string };
|
||||
expect(html.html).toBe("<main>Hi</main>");
|
||||
const consoleValue = (await tools
|
||||
.get("browser_console")
|
||||
?.execute({ since: 1, limit: 10, includeStack: true }, makeExec())) as {
|
||||
entries: { stackTrace: { functionName: string }[] }[];
|
||||
};
|
||||
expect(consoleValue.entries[0].stackTrace[0].functionName).toBe("run");
|
||||
const networkValue = (await tools
|
||||
.get("browser_network")
|
||||
?.execute({ tabId: 7, maxTextChars: 512 }, makeExec())) as {
|
||||
entries: { mimeType: string }[];
|
||||
};
|
||||
expect(networkValue.entries[0].mimeType).toBe("application/json");
|
||||
expect(calls.slice(1).map((call) => call.args)).toEqual([
|
||||
["get-html", "--session", "s1", "--ref", "@e2", "--max-bytes", "2048"],
|
||||
["console", "--session", "s1", "--since", "1", "--limit", "10", "--include-stack"],
|
||||
["network", "--session", "s1", "--tab-id", "7", "--max-text-chars", "512"],
|
||||
]);
|
||||
});
|
||||
|
||||
it("resizes only within the CLI-supported range", async () => {
|
||||
const { tools, calls } = setup({
|
||||
"session start": START_REPLY("s1"),
|
||||
"window resize": { window_id: 2, width: 1280, height: 800 },
|
||||
});
|
||||
await startSession(tools);
|
||||
const resize = tools.get("browser_window_resize");
|
||||
const value = (await resize?.execute({ width: 1280, height: 800 }, makeExec())) as {
|
||||
windowId: number;
|
||||
};
|
||||
expect(value.windowId).toBe(2);
|
||||
expect(calls[1].args).toEqual([
|
||||
"window",
|
||||
"resize",
|
||||
"--session",
|
||||
"s1",
|
||||
"--width",
|
||||
"1280",
|
||||
"--height",
|
||||
"800",
|
||||
]);
|
||||
await expect(resize?.execute({ width: 99, height: 800 }, makeExec())).rejects.toThrow(
|
||||
/100\.\.=7680/,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe("browser_screenshot", () => {
|
||||
|
||||
@@ -6,27 +6,27 @@
|
||||
"disconnected": "Disconnected",
|
||||
"connecting": "Connecting…",
|
||||
"connected": "Connected",
|
||||
"version_skew": "Connected",
|
||||
"version_skew": "Protocol mismatch",
|
||||
"disabled": "Connection off"
|
||||
},
|
||||
"stateDetail": {
|
||||
"disconnected": "Open BrowserSkill to connect.",
|
||||
"connecting": "Connecting to BrowserSkill…",
|
||||
"connected": "Ready for automation tasks.",
|
||||
"version_skew": "Protocol versions differ. Please upgrade."
|
||||
"version_skew": "Some features may be unavailable."
|
||||
},
|
||||
"stateBadge": {
|
||||
"disconnected": "Standby",
|
||||
"connecting": "Igniting",
|
||||
"connected": "Ready",
|
||||
"version_skew": "Upgradable",
|
||||
"version_skew": "Action needed",
|
||||
"disabled": "Off"
|
||||
},
|
||||
"connectionToggleTitle": "BrowserSkill connection",
|
||||
"controlHintsToggleTitle": "Control hints",
|
||||
"controlHintsToggleHint": "Show the status pill and orange glow while the Agent controls a page.",
|
||||
"controlHintsInfoLabel": "About control hints",
|
||||
"versionSkewWarning": "Extension protocol v{{extensionProtocol}}, CLI protocol v{{cliProtocol}}. Protocol versions differ — please upgrade.",
|
||||
"versionSkewWarning": "Extension protocol v{{extensionProtocol}}, daemon v{{daemonProtocol}}.",
|
||||
"upgradeAvailable": "Upgradable",
|
||||
"launcher": {
|
||||
"title": "Quick actions"
|
||||
|
||||
@@ -6,27 +6,27 @@
|
||||
"disconnected": "未连接",
|
||||
"connecting": "连接中…",
|
||||
"connected": "已连接",
|
||||
"version_skew": "已连接",
|
||||
"version_skew": "协议不一致",
|
||||
"disabled": "连接已关闭"
|
||||
},
|
||||
"stateDetail": {
|
||||
"disconnected": "请先打开 BrowserSkill。",
|
||||
"connecting": "正在连接 BrowserSkill…",
|
||||
"connected": "可接收自动化任务。",
|
||||
"version_skew": "协议版本不同,请及时升级。"
|
||||
"version_skew": "部分能力可能不可用。"
|
||||
},
|
||||
"stateBadge": {
|
||||
"disconnected": "Standby",
|
||||
"connecting": "Igniting",
|
||||
"connected": "Ready",
|
||||
"version_skew": "可升级",
|
||||
"version_skew": "Action needed",
|
||||
"disabled": "Off"
|
||||
},
|
||||
"connectionToggleTitle": "BrowserSkill 连接",
|
||||
"controlHintsToggleTitle": "控制提示",
|
||||
"controlHintsToggleHint": "Agent 控制页面时显示提示条和橙色闪光。",
|
||||
"controlHintsInfoLabel": "控制提示说明",
|
||||
"versionSkewWarning": "扩展协议 v{{extensionProtocol}},CLI 协议 v{{cliProtocol}}。协议版本不同,请及时升级。",
|
||||
"versionSkewWarning": "扩展协议 v{{extensionProtocol}},daemon 协议 v{{daemonProtocol}}。",
|
||||
"upgradeAvailable": "可升级",
|
||||
"launcher": {
|
||||
"title": "快捷功能"
|
||||
|
||||
+25
-2
@@ -29,7 +29,7 @@ Drive the user's **real Chromium browser** (with their logins and cookies) throu
|
||||
## Prerequisites
|
||||
|
||||
1. `bsk` on `PATH` (Rust CLI from browser-skill)
|
||||
2. browser-skill **extension** loaded in Chromium and connected (popup shows green, or an upgrade reminder — still connected; continue)
|
||||
2. browser-skill **extension** loaded in Chromium and connected (popup shows green)
|
||||
3. Any `bsk` command auto-starts background services as needed; use `bsk doctor` if anything fails
|
||||
|
||||
## Mandatory workflow
|
||||
@@ -206,6 +206,29 @@ Both capture from the moment the tab is attached and read a bounded per-tab buff
|
||||
| `bsk select <ref-or-selector> --value <v>` | Set `<select>` option(s) by `value` (repeat `--value` for multi-select) |
|
||||
| `bsk press <key>` | Key/combo (`Enter`, `Ctrl+A`, …; optional `--ref` to focus first) |
|
||||
|
||||
### File transfer (require `--session`)
|
||||
|
||||
| Command | Summary |
|
||||
|---------|---------|
|
||||
| `bsk upload <ref-or-selector> --file <path>` | Click one upload trigger and attach an agent-readable local file (`--file` is repeatable) |
|
||||
| `bsk download <ref-or-selector> --out <path>` | Click one download trigger and copy the single completed file to an exact local path (`--overwrite` is opt-in) |
|
||||
|
||||
The agent/harness decides whether a file transfer is appropriate and which local path belongs to the task. Treat upload as disclosure of that file to the current website, and download as accepting website-controlled bytes onto the local filesystem. Use only paths that are necessary for the user's bounded goal.
|
||||
|
||||
BrowserSkill enforces the mechanical boundary: files are staged under a session-scoped opaque transfer, only daemon-minted capabilities reach the extension, upload/download still obey Agent Window tab checks, and transfers are chunk/size bounded. Upload intercepts the native chooser for one transaction, locates the input activated in the resolved target's document, and assigns only the staged file paths. Download uniquely correlates one exact-target browser intent with one Chrome download in either event order, routes it through a daemon-minted relative directory, and lets the daemon validate and import it. Upload staging remains available for a later form submission and is removed when the session ends. Downloads cannot overwrite an existing destination unless `--overwrite` is explicit. BrowserSkill does not inspect file content or decide whether its meaning is sensitive.
|
||||
|
||||
Do not use `request-help` merely because a native file chooser or browser download is involved; try these commands first. For transfer failures, use the structured error instead of retrying blindly:
|
||||
|
||||
- `reason=file_input_not_activated` means the requested click did not activate exactly one `<input type="file">`; the page may use a non-input picker such as `window.showOpenFilePicker()`. Do not retry blindly. If human help is available, call `request-help` and tell the user the exact original local path to choose. The staged daemon path is internal and must not be shown to the user.
|
||||
- `reason=file_input_probe_failed` means BrowserSkill could not safely establish the browser-side upload transaction. Do not repeat the same action; use `request-help` when available.
|
||||
- `reason=set_file_input_failed` means BrowserSkill found the activated file input but Chrome rejected the staged path or assignment. Check the extension's file-URL access permission; otherwise use `request-help`.
|
||||
- `reason=download_capture_failed` means BrowserSkill could not attribute exactly one completed download to the requested target. Do not retry blindly or accept an unrelated browser download; use `request-help` when available.
|
||||
- `effect_state=none` means BrowserSkill confirmed that no file-transfer effect was committed. Follow the accompanying reason; a corrected target or explicit human fallback may be attempted.
|
||||
- `effect_state=unknown` means the browser may already have attached or created the file. Do not repeat the transfer. Observe the page if that can establish the result; otherwise stop and report the uncertainty.
|
||||
- `effect_state=committed` means the browser-side effect occurred even if later completion or cleanup failed. Do not repeat it; continue only after verifying the resulting page/download state.
|
||||
|
||||
If `request-help` returns `outcome="disabled"`, do not retry it. Stop gracefully and report the transfer mechanism that requires human intervention.
|
||||
|
||||
### Scripting & timing
|
||||
|
||||
| Command | Summary |
|
||||
@@ -299,7 +322,7 @@ bsk record stop [--output trace] # terminal fallback if the browser panel is u
|
||||
| `2` | Protocol / transport — service unreachable, IPC failure | `bsk doctor`; check extension connected; retry the command |
|
||||
| `3` | Browser / CDP execution failed | Retry; simplify selector; check tab still open |
|
||||
| `4` | Timeout | Increase `--timeout`; try `--wait-until domcontentloaded` |
|
||||
| `5` | Unknown RPC method | This command is not implemented in the current build. Continue the task with other commands. If a newly added command is missing, suggest `bsk update`. |
|
||||
| `5` | Version skew (CLI vs extension) | Upgrade/reinstall matching versions |
|
||||
|
||||
Human errors print `error:` + `hint:` on stderr; `--json` includes `code`, `message`, `hint`, `exit_code`.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user