Compare commits

..
7 Commits
153 changed files with 5902 additions and 17694 deletions
Generated
+3 -2
View File
@@ -134,9 +134,10 @@ dependencies = [
[[package]]
name = "bsk"
version = "0.1.11"
version = "0.1.10"
dependencies = [
"anyhow",
"base64",
"bsk-protocol",
"clap",
"console",
@@ -168,7 +169,7 @@ dependencies = [
[[package]]
name = "bsk-protocol"
version = "0.1.11"
version = "0.1.10"
dependencies = [
"schemars",
"semver",
+2 -1
View File
@@ -7,7 +7,7 @@ members = [
default-members = ["crates/bsk-cli"]
[workspace.package]
version = "0.1.11"
version = "0.1.10"
edition = "2024"
rust-version = "1.85"
license = "MIT"
@@ -36,6 +36,7 @@ flate2 = "1.1.9"
tar = "0.4.46"
zip = { version = "7.2.0", default-features = false, features = ["deflate"] }
sha2 = "0.11.0"
base64 = "0.22"
tempfile = "3"
nix = { version = "0.29", features = ["signal", "process"] }
rand = "0.8"
+6 -3
View File
@@ -1,6 +1,6 @@
# BrowserSkill — Privacy Policy
**Last updated:** May 25, 2026
**Last updated:** August 20, 2026
This Privacy Policy describes how the **BrowserSkill** browser extension (the "Extension") handles information when you install and use it. BrowserSkill is published as part of the open-source [BrowserSkill](https://github.com/Tencent/BrowserSkill) project. The source code is publicly auditable.
@@ -14,7 +14,7 @@ BrowserSkill is a local automation bridge that lets AI coding agents (such as Cu
## 2. Single Purpose
The Extension's single purpose is to expose browser automation primitives (navigation, DOM observation, screenshots, clicks, form filling, tab management) to a locally running BrowserSkill daemon over a WebSocket connection on `127.0.0.1`, so that an AI agent invoked by the user can interact with web pages on the user's behalf.
The Extension's single purpose is to expose browser automation primitives (navigation, DOM observation, screenshots, clicks, form filling, task-scoped file transfer, and tab management) to a locally running BrowserSkill daemon over a WebSocket connection on `127.0.0.1`, so that an AI agent invoked by the user can interact with web pages on the user's behalf.
## 3. Data the Extension Accesses
@@ -26,6 +26,7 @@ Depending on the commands the user (via their AI agent) sends to the local daemo
| **User input simulated by the agent** | Mouse clicks, keystrokes, and form values that the AI agent dispatches through the Chrome DevTools Protocol (CDP). | Required to perform automation actions the user has asked the agent to do. |
| **Tab and window metadata** | Tab IDs, URLs, titles, window IDs of the Agent Window and any tabs the user explicitly authorizes. | Required to target automation commands at the correct tab/window. |
| **Local extension storage** | A randomly generated 8-character instance ID and an optional user-supplied label. | Used so the local daemon can recognize this browser instance across reconnects. No personal data is stored. |
| **File transfers requested by the agent** | Local files explicitly supplied to `bsk upload`, and the file created by a single `bsk download` action. | Required to attach a task file to a web page or return a browser-generated download to the invoking local agent. |
| **OS notifications** | Permission to display a system notification when the agent requests to "borrow" one of the user's existing tabs. | Required to obtain explicit, per-tab user consent before the agent touches any pre-existing tab. |
## 4. Data the Extension Does **Not** Collect
@@ -34,7 +35,7 @@ BrowserSkill does **not**:
- Send any data to remote servers, the Extension's authors, or any third party.
- Call any LLM, AI, or cloud API. The Extension contains no API keys, model identifiers, or remote endpoints.
- Read or transmit cookies, browsing history, bookmarks, downloads, saved passwords, or autofill data.
- Read or transmit cookies, browsing history, bookmarks, saved passwords, or autofill data. It observes only the download initiated by an active `bsk download` call, not download history generally.
- Use webcam, microphone, geolocation, or any device sensor.
- Include analytics, telemetry, crash reporting, advertising SDKs, or fingerprinting code.
- Track users across websites or across sessions.
@@ -50,6 +51,7 @@ The Extension requests the following Chrome permissions. Each is used solely for
- **`alarms`** — Periodically wake the service worker to keep the local WebSocket connection alive.
- **`idle`** — Detect when the device returns from idle/locked so the Extension can promptly re-establish the local WebSocket connection after the machine wakes. No idle data is stored or transmitted.
- **`notifications`** — Show a system notification to obtain user approval before the agent borrows a user-owned tab.
- **`downloads`** — Correlate and route the one browser download initiated by an active `bsk download` command. If that claimed transaction fails, BrowserSkill cancels an in-progress file or removes its completed temporary browser file. It is not used to enumerate download history or alter unclaimed downloads.
- **`storage`** — Persist a random instance ID and optional label in `chrome.storage.local`.
- **Host permission `<all_urls>`** — Inject a small status overlay (showing "Agent Active") on pages controlled by the agent, and enable automation across whatever sites the user directs the agent to. The Extension does **not** read or transmit page content from sites the agent is not actively driving.
@@ -61,6 +63,7 @@ All Extension activity stays on the user's local device. The only network traffi
- The instance ID and optional label persist in `chrome.storage.local` until the user uninstalls the Extension or clears extension storage.
- Page content, screenshots, DOM snapshots, and other observed data are returned to the local daemon in response to commands and are **not retained by the Extension**. They live only as long as the agent's tool call.
- Upload and download bytes are staged by the local daemon in a private, session-scoped directory. Download staging is removed after it is copied to the requested destination. Upload staging is retained until the session ends so a later form submission can still read the attached file. Remaining staging is removed when the session ends or disconnects, or when the daemon next starts after a crash.
## 8. User Control
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@browser-skill/extension",
"version": "0.1.7",
"version": "0.1.6",
"private": true,
"type": "module",
"scripts": {
@@ -1,12 +1,10 @@
import { beforeEach, describe, expect, it, vi } from "vitest";
import type { RecordStepPayload } from "@/lib/record-bridge";
import { startRecordCapture } from "../record-capture";
import { RECORD_STOP, type RecordStepPayload } from "@/lib/record-bridge";
import { handleRecordContentMessage, startRecordCapture } from "../record-capture";
vi.stubGlobal("chrome", {
runtime: {
sendMessage: vi.fn((message: { sequence?: number }) =>
Promise.resolve({ ok: true, sequence: message.sequence }),
),
sendMessage: vi.fn(() => Promise.resolve()),
},
});
@@ -48,6 +46,58 @@ function click(el: Element): void {
el.dispatchEvent(new MouseEvent("click", { bubbles: true, button: 0, detail: 1 }));
}
describe("handleRecordContentMessage stop/cancel", () => {
it("ignores STOP when no recording is active", () => {
const dispose = vi.fn();
const onStop = vi.fn();
const setActiveRequestId = vi.fn();
const setCapture = vi.fn();
const sendResponse = vi.fn();
const needsAsync = handleRecordContentMessage(
{ type: RECORD_STOP, requestId: "rec-stale" },
{
activeRequestId: null,
capture: { dispose },
setActiveRequestId,
setCapture,
onStart: vi.fn(),
onStop,
},
sendResponse,
);
expect(needsAsync).toBe(false);
expect(dispose).not.toHaveBeenCalled();
expect(onStop).not.toHaveBeenCalled();
expect(setActiveRequestId).not.toHaveBeenCalled();
expect(setCapture).not.toHaveBeenCalled();
expect(sendResponse).not.toHaveBeenCalled();
});
it("ignores STOP for a mismatched requestId", () => {
const dispose = vi.fn();
const onStop = vi.fn();
const needsAsync = handleRecordContentMessage(
{ type: RECORD_STOP, requestId: "rec-other" },
{
activeRequestId: "rec-1",
capture: { dispose },
setActiveRequestId: vi.fn(),
setCapture: vi.fn(),
onStart: vi.fn(),
onStop,
},
vi.fn(),
);
expect(needsAsync).toBe(false);
expect(dispose).not.toHaveBeenCalled();
expect(onStop).not.toHaveBeenCalled();
});
});
describe("record-capture semantic", () => {
let steps: RecordStepPayload[];
@@ -124,19 +174,6 @@ describe("record-capture semantic", () => {
]);
});
it("does not emit page navigation from a child Document capture", () => {
const originalUrl = location.href;
const capture = startRecordCapture("rec-child", (step) => steps.push(step), {
captureNavigation: false,
});
history.pushState({}, "", "#inside-frame");
expect(steps).toEqual([]);
capture.dispose();
history.replaceState({}, "", originalUrl);
});
it("does not record clicks on anonymous layout divs", () => {
document.body.innerHTML = `
<div id="chrome">page chrome</div>
@@ -191,7 +228,6 @@ describe("record-capture semantic", () => {
expect(steps[0]).toMatchObject({
op: "hover",
target: { role: "button", name: "Open user navigation menu" },
geometry: { tag: "button", rect: { x: 900, y: 8, w: 32, h: 32 } },
});
expect(steps[1]).toMatchObject({
op: "click",
@@ -1,102 +0,0 @@
import { afterEach, describe, expect, it, vi } from "vitest";
import type { RecordFramePortMessage } from "@/lib/recording/frame-bridge";
import { RECORD_FRAME_START } from "@/lib/recording/frame-bridge";
import { RECORD_DOCUMENT_ATTRIBUTE } from "@/shared/recording-document-identity";
import { RecordFrameAgent } from "../recording/frame-agent";
class PortListeners<T extends (...args: never[]) => unknown> {
readonly values = new Set<T>();
addListener = (listener: T) => this.values.add(listener);
removeListener = (listener: T) => this.values.delete(listener);
}
function portHarness() {
const onMessage = new PortListeners<(message: unknown) => void>();
const onDisconnect = new PortListeners<() => void>();
const outbound: RecordFramePortMessage[] = [];
const port = {
name: "bsk-record-frame",
onMessage,
onDisconnect,
postMessage(message: RecordFramePortMessage) {
outbound.push(message);
if (message.type === "ready") {
queueMicrotask(() => {
for (const listener of onMessage.values) {
listener({
type: "ready_ack",
requestId: message.requestId,
producerId: message.producerId,
});
}
});
}
},
disconnect: vi.fn(),
} as unknown as chrome.runtime.Port;
return {
port,
outbound,
receive(message: RecordFramePortMessage) {
for (const listener of onMessage.values) listener(message);
},
};
}
describe("RecordFrameAgent", () => {
afterEach(() => {
document.documentElement.removeAttribute(RECORD_DOCUMENT_ATTRIBUTE);
document.body.replaceChildren();
});
it("keeps a failed stop retryable and flushes the final dirty fill", async () => {
const harness = portHarness();
const sendMessage = vi
.fn<(message: { sequence: number }) => Promise<unknown>>()
.mockRejectedValueOnce(new Error("offline"))
.mockRejectedValueOnce(new Error("still offline"))
.mockImplementation(async (message) => ({ ok: true, sequence: message.sequence }));
vi.stubGlobal("chrome", {
runtime: {
connect: vi.fn(() => harness.port),
sendMessage,
},
});
document.body.innerHTML = `<label for="draft">Draft</label><input id="draft" />`;
const agent = new RecordFrameAgent();
await expect(
agent.start({ type: RECORD_FRAME_START, requestId: "rec-1", startedAtMs: 10 }),
).resolves.toEqual({ ok: true });
expect(document.documentElement.hasAttribute(RECORD_DOCUMENT_ATTRIBUTE)).toBe(true);
const input = document.querySelector("input")!;
input.dispatchEvent(new FocusEvent("focusin", { bubbles: true }));
input.value = "final value";
input.dispatchEvent(new Event("input", { bubbles: true }));
harness.receive({ type: "stop", requestId: "rec-1", commandId: "stop-1" });
await vi.waitFor(() =>
expect(harness.outbound).toContainEqual({
type: "stopped",
requestId: "rec-1",
commandId: "stop-1",
ok: false,
error: "failed to deliver one or more recorded steps",
}),
);
expect(document.documentElement.hasAttribute(RECORD_DOCUMENT_ATTRIBUTE)).toBe(true);
harness.receive({ type: "stop", requestId: "rec-1", commandId: "stop-2" });
await vi.waitFor(() =>
expect(harness.outbound).toContainEqual({
type: "stopped",
requestId: "rec-1",
commandId: "stop-2",
ok: true,
}),
);
expect(sendMessage).toHaveBeenCalledTimes(3);
expect(document.documentElement.hasAttribute(RECORD_DOCUMENT_ATTRIBUTE)).toBe(false);
});
});
@@ -1,36 +0,0 @@
import { describe, expect, it, vi } from "vitest";
import type { RecordStepMessage } from "@/lib/record-bridge";
import { RecordStepDelivery } from "../record-step-delivery";
describe("RecordStepDelivery", () => {
it("retries from the first unacknowledged sequence before sending later steps", async () => {
const sent: RecordStepMessage[] = [];
const send = vi.fn(async (message: RecordStepMessage) => {
sent.push(message);
if (sent.length === 1) throw new Error("service worker unavailable");
return { ok: true, sequence: message.sequence };
});
const delivery = new RecordStepDelivery("rec-ordered", send, "document-1");
delivery.enqueue({ op: "click", target: { tag: "button", name: "First" } });
delivery.enqueue({ op: "click", target: { tag: "button", name: "Second" } });
await expect(delivery.flush()).resolves.toBe(true);
expect(sent.map((message) => message.sequence)).toEqual([1, 1, 2]);
expect(sent.map((message) => message.step.target?.name)).toEqual(["First", "First", "Second"]);
});
it("keeps unacknowledged steps pending for a later flush", async () => {
const send = vi
.fn<(message: RecordStepMessage) => Promise<unknown>>()
.mockRejectedValueOnce(new Error("offline"))
.mockRejectedValueOnce(new Error("still offline"))
.mockImplementation(async (message) => ({ ok: true, sequence: message.sequence }));
const delivery = new RecordStepDelivery("rec-retry", send, "document-1");
delivery.enqueue({ op: "click", target: { tag: "button", name: "Save" } });
await expect(delivery.flush()).resolves.toBe(false);
await expect(delivery.flush()).resolves.toBe(true);
expect(send).toHaveBeenCalledTimes(3);
});
});
+150 -76
View File
@@ -1,9 +1,9 @@
import {
type CaptureTargetDescriptor,
describeEventTarget,
describeTarget,
resolveClickableElement,
resolveHoverElement,
type TargetDescriptor,
} from "@/lib/describe-target";
import {
evaluateHoverTrigger,
@@ -12,8 +12,22 @@ import {
hasDirectHoverInteractiveSignal,
hasStrongHoverExpansionSignal,
} from "@/lib/hover-trigger-policy";
import type { RecordStepPayload } from "@/lib/record-bridge";
import { shouldRecordPress } from "@/lib/recording/draft-policy";
import {
isRecordCancelMessage,
isRecordStartMessage,
isRecordStopMessage,
RECORD_CANCEL,
RECORD_START,
RECORD_STEP,
RECORD_STOP,
type RecordCancelMessage,
type RecordStartAck,
type RecordStartMessage,
type RecordStepPayload,
type RecordStopAck,
type RecordStopMessage,
} from "@/lib/record-bridge";
import { shouldRecordPress } from "@/lib/trace-reducer";
import {
closestHoverSurfaceCandidate,
collectHoverSurfaceStates,
@@ -23,21 +37,42 @@ import {
isLikelyHoverSurfaceOwner,
} from "./record-hover-surface";
const pendingStepSends = new Map<string, Set<Promise<boolean>>>();
const failedStepDeliveries = new Set<string>();
const knownRecordRequests = new Set<string>();
function sendRecordStep(requestId: string, step: RecordStepPayload): void {
const payload = { type: RECORD_STEP, requestId, step };
const pending = Promise.resolve(chrome.runtime.sendMessage(payload)).then(
() => true,
() => {
failedStepDeliveries.add(requestId);
return false;
},
);
const sends = pendingStepSends.get(requestId) ?? new Set<Promise<boolean>>();
sends.add(pending);
pendingStepSends.set(requestId, sends);
void pending.then(() => {
sends.delete(pending);
if (sends.size === 0) pendingStepSends.delete(requestId);
});
}
export interface RecordCaptureController {
dispose(): void;
}
interface FillSession {
element: FillableElement;
target: CaptureTargetDescriptor;
target: TargetDescriptor;
baselineValue: string;
lastValue: string;
pendingCommit?: "enter" | "suggestion" | "blur";
}
interface HoverCandidate {
element: Element;
target: CaptureTargetDescriptor;
target: TargetDescriptor;
recordedAt: number;
score: number;
eligible: boolean;
@@ -122,24 +157,6 @@ function nearbyFillableFromSearchChrome(target: Element): FillableElement | null
return null;
}
function captureGeometry(el: Element): RecordStepPayload["geometry"] {
const rect = el.getBoundingClientRect();
return {
rect: { x: rect.x, y: rect.y, w: rect.width, h: rect.height },
tag: el.tagName.toLowerCase(),
};
}
function geometryForEventTarget(
target: EventTarget | null,
): RecordStepPayload["geometry"] | undefined {
if (!(target instanceof Element)) return undefined;
const clickable = target.closest(
'a,button,input,select,textarea,[role="button"],[role="link"],[role="menuitem"],[contenteditable="true"]',
);
return captureGeometry(clickable ?? target);
}
function fillableValue(el: FillableElement): string {
if (el instanceof HTMLInputElement || el instanceof HTMLTextAreaElement) {
return el.value;
@@ -215,10 +232,7 @@ function collectHoverTriggerLabelText(root: Element): string {
return normalizeLabelText(text);
}
function compactHoverTargetName(
el: Element,
desc: CaptureTargetDescriptor,
): CaptureTargetDescriptor {
function compactHoverTargetName(el: Element, desc: TargetDescriptor): TargetDescriptor {
if (!desc.name) return desc;
const fullText = normalizeLabelText(el.textContent ?? "");
const compactName = collectHoverTriggerLabelText(el);
@@ -236,7 +250,7 @@ function compactHoverTargetName(
return desc;
}
function isWeakHoverTarget(target: CaptureTargetDescriptor): boolean {
function isWeakHoverTarget(target: TargetDescriptor): boolean {
return !target.role && !target.name && target.tag === "div";
}
@@ -247,9 +261,9 @@ function looksLikeAvatarElement(el: Element): boolean {
function normalizeHoverTarget(
el: Element,
desc: CaptureTargetDescriptor,
desc: TargetDescriptor,
decision: HoverTriggerDecision,
): CaptureTargetDescriptor {
): TargetDescriptor {
if (desc.role === "img" && !desc.name && looksLikeAvatarElement(el)) {
return { ...desc, name: "image" };
}
@@ -263,7 +277,7 @@ function normalizeHoverTarget(
return desc;
}
function hoverTriggerSignals(el: Element, desc: CaptureTargetDescriptor) {
function hoverTriggerSignals(el: Element, desc: TargetDescriptor) {
if (!(el instanceof HTMLElement)) return null;
const style = hoverTriggerStyle(el);
return {
@@ -359,7 +373,6 @@ function scheduleInputCompletionCommit(
export function startRecordCapture(
_requestId: string,
sendStep: (step: RecordStepPayload) => void,
options: { captureNavigation?: boolean } = {},
): RecordCaptureController {
const emitStep = (step: RecordStepPayload) => {
sendStep({ page_url: location.href, ...step });
@@ -398,17 +411,14 @@ export function startRecordCapture(
emitStep({
op: "fill",
target: session.target,
geometry: captureGeometry(session.element),
value,
commit: session.pendingCommit ?? "blur",
...(isPassword ? { redacted: true } : {}),
});
};
const commitFillSession = (commit: "enter" | "suggestion" | "blur" = "blur") => {
const commitFillSession = () => {
if (!fillSession || composing) return;
const session = fillSession;
session.pendingCommit = commit;
fillSession = null;
emitFill(session);
committedValues.set(session.element, session.lastValue);
@@ -614,7 +624,6 @@ export function startRecordCapture(
emitStep({
op: "hover",
target: hover.target,
geometry: captureGeometry(hover.element),
});
emittedHoverElements.add(hover.element);
};
@@ -669,7 +678,6 @@ export function startRecordCapture(
emitStep({
op: "click",
target,
geometry: geometryForEventTarget(eventTarget(event)),
expects_navigation: true,
});
};
@@ -771,7 +779,7 @@ export function startRecordCapture(
scheduleInputCompletionCommit(
sessionElement,
syncFillSessionValue,
() => commitFillSession("suggestion"),
commitFillSession,
(el) => fillSession?.element === el,
);
return;
@@ -831,7 +839,6 @@ export function startRecordCapture(
emitStep({
op: "select",
target: desc,
geometry: captureGeometry(target),
values,
labels,
expects_navigation: true,
@@ -872,7 +879,7 @@ export function startRecordCapture(
};
}
if (fillable) {
commitFillSession(event.key === "Enter" ? "enter" : "blur");
commitFillSession();
}
const desc = describeEventTarget(target);
if (!desc && !event.key) return;
@@ -880,7 +887,7 @@ export function startRecordCapture(
emitStep({
op: "press",
key: event.key,
...(desc ? { target: desc, geometry: geometryForEventTarget(target) } : {}),
...(desc ? { target: desc } : {}),
...(modifiers.length ? { modifiers } : {}),
expects_navigation: event.key === "Enter",
});
@@ -896,33 +903,23 @@ export function startRecordCapture(
document.addEventListener("change", onChange, true);
document.addEventListener("keydown", onKeyDown, true);
const captureNavigation = options.captureNavigation ?? true;
const urlObserver = captureNavigation
? new MutationObserver(() => emitNavigateIfChanged())
: null;
urlObserver?.observe(document, { subtree: true, childList: true });
const urlObserver = new MutationObserver(() => emitNavigateIfChanged());
urlObserver.observe(document, { subtree: true, childList: true });
const onUrlEvent = () => emitNavigateIfChanged();
// Wrap: passing commitFillSession directly would forward the DOM event as
// the `commit` argument and stamp it onto the recorded fill step.
const onPageHide = () => commitFillSession();
if (captureNavigation) {
window.addEventListener("hashchange", onUrlEvent);
window.addEventListener("popstate", onUrlEvent);
}
window.addEventListener("pagehide", onPageHide);
window.addEventListener("hashchange", onUrlEvent);
window.addEventListener("popstate", onUrlEvent);
window.addEventListener("pagehide", commitFillSession);
const originalPushState = history.pushState;
const originalReplaceState = history.replaceState;
if (captureNavigation) {
history.pushState = function (...args: Parameters<History["pushState"]>) {
originalPushState.apply(this, args);
emitNavigateIfChanged(navigationActionPending ? true : undefined);
};
history.replaceState = function (...args: Parameters<History["replaceState"]>) {
originalReplaceState.apply(this, args);
emitNavigateIfChanged(navigationActionPending ? true : undefined);
};
}
history.pushState = function (...args: Parameters<History["pushState"]>) {
originalPushState.apply(this, args);
emitNavigateIfChanged(navigationActionPending ? true : undefined);
};
history.replaceState = function (...args: Parameters<History["replaceState"]>) {
originalReplaceState.apply(this, args);
emitNavigateIfChanged(navigationActionPending ? true : undefined);
};
return {
dispose() {
@@ -936,16 +933,93 @@ export function startRecordCapture(
document.removeEventListener("compositionend", onCompositionEnd, true);
document.removeEventListener("change", onChange, true);
document.removeEventListener("keydown", onKeyDown, true);
urlObserver?.disconnect();
if (captureNavigation) {
window.removeEventListener("hashchange", onUrlEvent);
window.removeEventListener("popstate", onUrlEvent);
}
window.removeEventListener("pagehide", onPageHide);
if (captureNavigation) {
history.pushState = originalPushState;
history.replaceState = originalReplaceState;
}
urlObserver.disconnect();
window.removeEventListener("hashchange", onUrlEvent);
window.removeEventListener("popstate", onUrlEvent);
window.removeEventListener("pagehide", commitFillSession);
history.pushState = originalPushState;
history.replaceState = originalReplaceState;
},
};
}
export type RecordContentMessage = RecordStartMessage | RecordStopMessage | RecordCancelMessage;
export function isRecordContentMessage(msg: unknown): msg is RecordContentMessage {
return isRecordStartMessage(msg) || isRecordStopMessage(msg) || isRecordCancelMessage(msg);
}
export function handleRecordContentMessage(
message: RecordContentMessage,
state: {
activeRequestId: string | null;
capture: RecordCaptureController | null;
setActiveRequestId(id: string | null): void;
setCapture(capture: RecordCaptureController | null): void;
onStart(requestId: string, startedAtMs?: number): void;
onStop(): void;
},
sendResponse?: (response: RecordStartAck | RecordStopAck) => void,
): boolean {
if (isRecordStartMessage(message)) {
if (!knownRecordRequests.has(message.requestId)) {
knownRecordRequests.add(message.requestId);
failedStepDeliveries.delete(message.requestId);
}
state.capture?.dispose();
state.setCapture(
startRecordCapture(message.requestId, (step) => {
sendRecordStep(message.requestId, step);
}),
);
state.setActiveRequestId(message.requestId);
state.onStart(message.requestId, message.startedAtMs);
sendResponse?.({ ok: true });
return sendResponse !== undefined;
}
if (isRecordStopMessage(message) || isRecordCancelMessage(message)) {
// Require an active recording that matches this requestId — otherwise a
// stray STOP/CANCEL (e.g. after teardown) would still run finishStop and
// clear overlay state even though nothing was capturing.
if (!state.activeRequestId || state.activeRequestId !== message.requestId) {
return false;
}
state.capture?.dispose();
state.setCapture(null);
const finishStop = () => {
state.onStop();
state.setActiveRequestId(null);
};
if (isRecordStopMessage(message) && sendResponse) {
const pending = [...(pendingStepSends.get(message.requestId) ?? [])];
void Promise.all(pending).then((delivered) => {
finishStop();
const succeeded = delivered.every(Boolean) && !failedStepDeliveries.has(message.requestId);
if (succeeded) {
failedStepDeliveries.delete(message.requestId);
knownRecordRequests.delete(message.requestId);
}
sendResponse(
succeeded
? { ok: true }
: {
ok: false,
error: "failed to deliver one or more recorded steps",
},
);
});
return true;
}
if (isRecordCancelMessage(message)) {
failedStepDeliveries.delete(message.requestId);
knownRecordRequests.delete(message.requestId);
}
finishStop();
return false;
}
return false;
}
export { RECORD_CANCEL, RECORD_START, RECORD_STEP, RECORD_STOP };
@@ -1,65 +0,0 @@
import {
isAcceptedRecordStepAck,
RECORD_STEP,
type RecordStepMessage,
type RecordStepPayload,
} from "@/lib/record-bridge";
export type SendRecordStepMessage = (message: RecordStepMessage) => Promise<unknown>;
export class RecordStepDelivery {
readonly #requestId: string;
readonly #producerId: string;
readonly #send: SendRecordStepMessage;
readonly #pending = new Map<number, RecordStepPayload>();
#nextSequence = 1;
#tail = Promise.resolve();
constructor(
requestId: string,
send: SendRecordStepMessage = (message) => chrome.runtime.sendMessage(message),
producerId: string = crypto.randomUUID(),
) {
this.#requestId = requestId;
this.#producerId = producerId;
this.#send = send;
}
enqueue(step: RecordStepPayload): void {
const sequence = this.#nextSequence;
this.#nextSequence += 1;
this.#pending.set(sequence, step);
void this.#schedule();
}
async flush(): Promise<boolean> {
await this.#tail;
await this.#schedule();
return this.#pending.size === 0;
}
#schedule(): Promise<void> {
const drain = () => this.#drain();
this.#tail = this.#tail.then(drain, drain);
return this.#tail;
}
async #drain(): Promise<void> {
for (const [sequence, step] of this.#pending) {
const message: RecordStepMessage = {
type: RECORD_STEP,
requestId: this.#requestId,
producerId: this.#producerId,
sequence,
step,
};
try {
const response = await this.#send(message);
if (!isAcceptedRecordStepAck(response, sequence)) return;
this.#pending.delete(sequence);
} catch {
return;
}
}
}
}
@@ -1,209 +0,0 @@
import {
markRecordingDocument,
type RecordingDocumentMarker,
waitForDocumentElement,
} from "@/lib/recording/document-marker";
import {
isRecordFrameStartMessage,
RECORD_FRAME_PORT,
RECORD_FRAME_QUERY,
RECORD_FRAME_START,
type RecordFramePortMessage,
type RecordFrameQueryResponse,
type RecordFrameStartMessage,
} from "@/lib/recording/frame-bridge";
import { type RecordCaptureController, startRecordCapture } from "../record-capture";
import { RecordStepDelivery } from "../record-step-delivery";
interface ActiveFrameRecording {
requestId: string;
producerId: string;
port: chrome.runtime.Port;
delivery: RecordStepDelivery;
capture: RecordCaptureController;
marker: RecordingDocumentMarker;
}
const RECORD_FRAME_REGISTER_TIMEOUT_MS = 5_000;
export class RecordFrameAgent {
#active: ActiveFrameRecording | null = null;
#startPromise: Promise<{ ok: true } | { ok: false; error: string }> | null = null;
#disposed = false;
async start(
message: RecordFrameStartMessage,
): Promise<{ ok: true } | { ok: false; error: string }> {
if (this.#active?.requestId === message.requestId) return { ok: true };
if (this.#disposed) return { ok: false, error: "recording document is disposed" };
if (this.#startPromise) {
await this.#startPromise;
return this.start(message);
}
if (this.#active) this.cancel(this.#active.requestId);
const starting = this.#start(message);
this.#startPromise = starting;
try {
return await starting;
} finally {
if (this.#startPromise === starting) this.#startPromise = null;
}
}
cancel(requestId: string): void {
const active = this.#active;
if (!active || active.requestId !== requestId) return;
active.capture.dispose();
active.marker.restore();
active.port.disconnect();
this.#active = null;
}
dispose(): void {
this.#disposed = true;
if (this.#active) this.cancel(this.#active.requestId);
}
async #start(
message: RecordFrameStartMessage,
): Promise<{ ok: true } | { ok: false; error: string }> {
const producerId = crypto.randomUUID();
const root = await waitForDocumentElement();
if (this.#disposed) return { ok: false, error: "recording document is disposed" };
const marker = markRecordingDocument(producerId, root);
const port = chrome.runtime.connect({ name: RECORD_FRAME_PORT });
try {
await registerPort(port, message.requestId, producerId);
} catch {
marker.restore();
port.disconnect();
return { ok: false, error: "failed to register recording document" };
}
if (this.#disposed) {
marker.restore();
port.disconnect();
return { ok: false, error: "recording document is disposed" };
}
const delivery = new RecordStepDelivery(message.requestId, undefined, producerId);
const capture = startRecordCapture(
message.requestId,
(step) => {
marker.ensure();
delivery.enqueue(step);
},
{ captureNavigation: window.top === window },
);
const active: ActiveFrameRecording = {
requestId: message.requestId,
producerId,
port,
delivery,
capture,
marker,
};
this.#active = active;
port.onMessage.addListener((raw: unknown) => {
const command = raw as Partial<RecordFramePortMessage>;
if (command.type === "stop" && command.requestId === active.requestId) {
void this.#stop(active, command.commandId);
} else if (command.type === "cancel" && command.requestId === active.requestId) {
this.cancel(active.requestId);
}
});
port.onDisconnect.addListener(() => {
if (this.#active !== active) return;
active.capture.dispose();
active.marker.restore();
this.#active = null;
});
return { ok: true };
}
async #stop(active: ActiveFrameRecording, commandId: string | undefined): Promise<void> {
if (!commandId || this.#active !== active) return;
active.capture.dispose();
const succeeded = await active.delivery.flush();
try {
active.port.postMessage({
type: "stopped",
requestId: active.requestId,
commandId,
ok: succeeded,
...(succeeded ? {} : { error: "failed to deliver one or more recorded steps" }),
} satisfies RecordFramePortMessage);
} catch {
return;
}
if (!succeeded || this.#active !== active) return;
active.marker.restore();
this.#active = null;
active.port.disconnect();
}
}
function registerPort(
port: chrome.runtime.Port,
requestId: string,
producerId: string,
): Promise<void> {
return new Promise((resolve, reject) => {
const timer = setTimeout(() => {
cleanup();
reject(new Error("recording document registration timed out"));
}, RECORD_FRAME_REGISTER_TIMEOUT_MS);
const cleanup = () => {
clearTimeout(timer);
port.onMessage.removeListener(onMessage);
port.onDisconnect.removeListener(onDisconnect);
};
const onMessage = (raw: unknown) => {
const message = raw as Partial<RecordFramePortMessage>;
if (
message.type !== "ready_ack" ||
message.requestId !== requestId ||
message.producerId !== producerId
) {
return;
}
cleanup();
resolve();
};
const onDisconnect = () => {
cleanup();
reject(new Error("recording document port disconnected"));
};
port.onMessage.addListener(onMessage);
port.onDisconnect.addListener(onDisconnect);
port.postMessage({ type: "ready", requestId, producerId } satisfies RecordFramePortMessage);
});
}
export function attachRecordFrameAgent(): () => void {
const agent = new RecordFrameAgent();
const onMessage = (
message: unknown,
_sender: chrome.runtime.MessageSender,
sendResponse: (response: { ok: true } | { ok: false; error: string }) => void,
) => {
if (!isRecordFrameStartMessage(message)) return false;
void agent.start(message).then(sendResponse);
return true;
};
chrome.runtime.onMessage.addListener(onMessage);
void chrome.runtime
.sendMessage({ type: RECORD_FRAME_QUERY })
.then((response: RecordFrameQueryResponse | undefined) => {
if (!response?.active || !response.requestId || response.startedAtMs === undefined) return;
return agent.start({
type: RECORD_FRAME_START,
requestId: response.requestId,
startedAtMs: response.startedAtMs,
});
})
.catch(() => {});
return () => {
chrome.runtime.onMessage.removeListener(onMessage);
agent.dispose();
};
}
+21 -25
View File
@@ -21,7 +21,6 @@ import {
type OverlayMode,
} from "@/lib/overlay-bridge";
import { POPUP_PORT_NAME, type PopupInbound, type PopupOutbound } from "@/lib/popup-bridge";
import { recordFrameCoordinator } from "@/lib/recording/frame-coordinator";
import { attachSessionsLiveFlag } from "@/lib/sessions-live-flag";
import { createDisconnectCleanup } from "@/session-manager/disconnect-cleanup";
import { attachSessionEventHandler } from "@/session-manager/event-handler";
@@ -39,7 +38,6 @@ import {
attachRecordFinishListener,
attachRecordQueryListener,
attachRecordStepListener,
type RecordRuntimeDeps,
} from "@/tools/record";
import { detectBrowserMeta } from "@/transport/handshake";
import type { Transport } from "@/transport/transport";
@@ -147,33 +145,10 @@ export default defineBackground(() => {
void sessionsLive.syncFromManager();
},
});
const recordDeps = {
tabsApi: chrome.tabs,
cdp,
frameCoordinator: recordFrameCoordinator,
sendToTab: (tabId: number, msg: Parameters<typeof chrome.tabs.sendMessage>[1]) =>
chrome.tabs.sendMessage(tabId, msg),
bypassOverlay: async (tabId: number, enabled: boolean) => {
try {
await chrome.tabs.sendMessage(tabId, {
type: OVERLAY_AUTOMATION_BYPASS,
enabled,
});
} catch {
// Content script may be unavailable on restricted pages.
}
},
} satisfies RecordRuntimeDeps;
recordFrameCoordinator.attach();
attachRecordStepListener(recordDeps);
attachRecordFinishListener(recordDeps);
attachRecordQueryListener(recordDeps);
const dispatcher = new ToolDispatcher({
transport,
sessions,
cdp,
recording: recordDeps,
onSessionsChanged: onOverlaySessionStateChanged,
onBrowserControlResumed,
approveBorrow: (ctx) =>
@@ -195,6 +170,27 @@ export default defineBackground(() => {
}),
});
dispatcher.start();
const recordDeps = {
tabsApi: chrome.tabs,
sendToTab: (tabId: number, msg: Parameters<typeof chrome.tabs.sendMessage>[1]) =>
chrome.tabs.sendMessage(tabId, msg),
bypassOverlay: async (tabId: number, enabled: boolean) => {
try {
await chrome.tabs.sendMessage(tabId, {
type: OVERLAY_AUTOMATION_BYPASS,
enabled,
});
} catch {
// Content script may be unavailable on restricted pages.
}
},
};
// Message listeners stay up (cheap; fire only for record message types).
// Tab / webNavigation observation attaches lazily while a recording is
// active — see ensureBrowserObservationListeners in tools/record.ts.
attachRecordStepListener();
attachRecordFinishListener(recordDeps);
attachRecordQueryListener(recordDeps);
if (typeof chrome.notifications?.onClicked?.addListener === "function") {
attachBorrowNotificationClickHandler({
onClicked: chrome.notifications.onClicked,
+45 -34
View File
@@ -11,6 +11,11 @@ import { createHelpRequestData } from "@/content/help-request";
import overlayCss from "@/content/overlay.css?inline";
import { OverlayController, shouldShowAgentControlOverlay } from "@/content/overlay-controller";
import { RecordOverlay } from "@/content/RecordOverlay";
import {
handleRecordContentMessage,
isRecordContentMessage,
type RecordCaptureController,
} from "@/content/record-capture";
import {
type CaptureSuppressAck,
type CaptureSuppressMessage,
@@ -40,17 +45,11 @@ import {
} from "@/lib/overlay-bridge";
import { sendInterrupt } from "@/lib/overlay-interrupt-client";
import {
isRecordCancelMessage,
isRecordStartMessage,
isRecordStopMessage,
RECORD_FINISH,
RECORD_QUERY,
type RecordCancelMessage,
type RecordQueryResponse,
type RecordStartAck,
type RecordStartMessage,
type RecordStopAck,
type RecordStopMessage,
} from "@/lib/record-bridge";
import type {
BorrowCancelMessage,
@@ -70,6 +69,7 @@ export default defineContentScript({
if (window.top !== window) return;
const overlays = new OverlayController();
let recordCapture: RecordCaptureController | null = null;
let activeRecordRequestId: string | null = null;
let reactRoot: ReactDOM.Root | null = null;
let overlayHost: HTMLElement | null = null;
@@ -228,6 +228,8 @@ export default defineContentScript({
if (previousHelp) {
void sendHelpFinish(previousHelp.id, "cancelled");
}
recordCapture?.dispose();
recordCapture = null;
activeRecordRequestId = null;
renderAll();
}
@@ -257,9 +259,6 @@ export default defineContentScript({
| HelpRequestMessage
| HelpCancelMessage
| CaptureSuppressMessage
| RecordStartMessage
| RecordStopMessage
| RecordCancelMessage
| OverlayAgentOverlayResetMessage
| OverlayAgentStateMessage
| OverlayAutomationBypassMessage,
@@ -270,32 +269,41 @@ export default defineContentScript({
return captureSuppress.handleMessage(message, sendResponse);
}
if (isRecordStartMessage(message)) {
activeRecordRequestId = message.requestId;
overlays.setAgentRecordRequest({
id: message.requestId,
...(typeof message.startedAtMs === "number" ? { startedAtMs: message.startedAtMs } : {}),
onFinish: () => {
void chrome.runtime.sendMessage({
type: RECORD_FINISH,
requestId: message.requestId,
});
if (isRecordContentMessage(message)) {
const needsAsync = handleRecordContentMessage(
message,
{
activeRequestId: activeRecordRequestId,
capture: recordCapture,
setActiveRequestId: (id) => {
activeRecordRequestId = id;
},
setCapture: (capture) => {
recordCapture = capture;
},
onStart: (requestId, startedAtMs) => {
overlays.setAgentRecordRequest({
id: requestId,
...(typeof startedAtMs === "number" ? { startedAtMs } : {}),
onFinish: () => {
void chrome.runtime.sendMessage({
type: RECORD_FINISH,
requestId,
});
},
});
renderAll();
},
onStop: () => {
overlays.clearAgentRecordRequest(activeRecordRequestId ?? undefined);
renderAll();
},
},
});
renderAll();
(sendResponse as unknown as (response: RecordStartAck) => void)({ ok: true });
return false;
}
if (isRecordStopMessage(message) || isRecordCancelMessage(message)) {
if (activeRecordRequestId !== message.requestId) return false;
overlays.clearAgentRecordRequest(message.requestId);
activeRecordRequestId = null;
renderAll();
if (isRecordStopMessage(message)) {
(sendResponse as unknown as (response: RecordStopAck) => void)({ ok: true });
}
return false;
sendResponse as unknown as
| ((response: RecordStartAck | RecordStopAck) => void)
| undefined,
);
return needsAsync;
}
if (
@@ -518,6 +526,9 @@ export default defineContentScript({
chrome.runtime.onMessage.removeListener(onMessage);
chrome.storage.onChanged.removeListener(onStorageChange);
window.removeEventListener("pageshow", onPageShow);
// Restore history hooks / remove capture listeners before the CS unloads.
recordCapture?.dispose();
recordCapture = null;
activeRecordRequestId = null;
});
},
@@ -200,9 +200,6 @@ describe("App", () => {
expect(copied).toContain("--browser 03c3e47f");
expect(copied).toContain('--purpose "发布 wiki 文档"');
expect(copied).not.toMatch(/bsk record start[^\n]*--url/);
expect(copied).toContain("./trace");
expect(copied).toContain("trace.json");
expect(copied).toContain("states/");
// Button label stays static; a transient toast confirms the copy.
expect(copyButton.textContent).toContain("复制录制指令");
await waitFor(() => expect(screen.getByRole("status").textContent).toContain("已复制"));
@@ -1,14 +0,0 @@
import { attachRecordFrameAgent } from "@/content/recording/frame-agent";
export default defineContentScript({
matches: ["<all_urls>"],
runAt: "document_start",
allFrames: true,
matchAboutBlank: true,
matchOriginAsFallback: true,
main(ctx) {
const dispose = attachRecordFrameAgent();
ctx.onInvalidated(dispose);
},
});
@@ -26,20 +26,20 @@ function handshake(
}
describe("computeConnectedState (protocol-based compat)", () => {
it("returns connected when daemon protocol equals extension protocol", () => {
expect(computeConnectedState(handshake("1.1", "1.0"), MIN_COMPATIBLE_PROTOCOL)).toEqual({
it("returns connected when protocol strings match", () => {
expect(computeConnectedState(handshake("1.0", "1.0"), MIN_COMPATIBLE_PROTOCOL)).toEqual({
kind: "connected",
});
});
it("returns version_skew when daemon protocol minor is newer", () => {
expect(computeConnectedState(handshake("1.2", "1.0"))).toEqual({
expect(computeConnectedState(handshake("1.1", "1.0"))).toEqual({
kind: "version_skew",
});
});
it("returns version_skew when daemon protocol string differs but floor is satisfied", () => {
expect(computeConnectedState(handshake("1.1.0", "1.0"))).toEqual({
expect(computeConnectedState(handshake("1", "1.0"))).toEqual({
kind: "version_skew",
});
});
@@ -53,7 +53,7 @@ describe("computeConnectedState (protocol-based compat)", () => {
});
it("rejects when extension is below daemon min_compatible_protocol", () => {
const result = computeConnectedState(handshake("1.1", "1.5"));
const result = computeConnectedState(handshake("1.0", "1.5"));
expect(result.kind).toBe("rejected");
if (result.kind === "rejected") {
expect(result.reason).toContain("min_compatible_protocol");
@@ -65,7 +65,7 @@ describe("computeConnectedState (protocol-based compat)", () => {
const result = computeConnectedState({
server: "browser-skill-daemon",
version: "0.1.0",
protocol_version: "1.1",
protocol_version: "1.0",
min_compatible_peer: "0.1.0",
});
expect(result).toEqual({ kind: "connected" });
@@ -80,7 +80,7 @@ describe("computeConnectedState (protocol-based compat)", () => {
});
it("rejects malformed daemon min_compatible_protocol with a daemon-floor reason", () => {
const result = computeConnectedState(handshake("1.1", "not-a-protocol"));
const result = computeConnectedState(handshake("1.0", "not-a-protocol"));
expect(result.kind).toBe("rejected");
if (result.kind === "rejected") {
expect(result.reason).toContain("daemon min_compatible_protocol");
@@ -242,11 +242,11 @@ describe("ConnectionController connectionEnabled", () => {
const second = transport.send.mock.calls[1]?.[0] as { id: string };
expect(second.id).not.toBe(first.id);
transport.emitMessage({ id: first.id, result: handshake("1.1", "1.0") });
transport.emitMessage({ id: first.id, result: handshake("1.0", "1.0") });
await Promise.resolve();
expect(controller.snapshot().state).not.toBe("connected");
transport.emitMessage({ id: second.id, result: handshake("1.1", "1.0") });
transport.emitMessage({ id: second.id, result: handshake("1.0", "1.0") });
await vi.waitFor(() => expect(controller.snapshot().state).toBe("connected"));
});
});
@@ -1,101 +0,0 @@
import { afterEach, describe, expect, it, vi } from "vitest";
import type { CdpRunner } from "@/tools/shared";
import { waitForDocumentSettled } from "../recording/document-settle";
function quietCdp(): { cdp: CdpRunner; send: ReturnType<typeof vi.fn> } {
const send = vi.fn(async () => ({
result: { value: { idleMs: 1_000, readyState: "complete" } },
}));
return {
cdp: { send: send as unknown as CdpRunner["send"] },
send,
};
}
describe("waitForDocumentSettled", () => {
afterEach(() => {
vi.useRealTimers();
});
it("waits through the minimum observation floor before accepting a quiet page", async () => {
vi.useFakeTimers();
const { cdp, send } = quietCdp();
const settled = waitForDocumentSettled(cdp, { target: { tabId: 7 } });
await vi.advanceTimersByTimeAsync(180);
await expect(settled).resolves.toBe("quiet");
expect(send).toHaveBeenCalledTimes(3);
});
it("cancels before probing a superseded observation", async () => {
vi.useFakeTimers();
const { cdp, send } = quietCdp();
const controller = new AbortController();
const settled = waitForDocumentSettled(
cdp,
{ target: { tabId: 7 } },
{
signal: controller.signal,
},
);
controller.abort();
await vi.advanceTimersByTimeAsync(60);
await expect(settled).resolves.toBe("cancelled");
expect(send).not.toHaveBeenCalled();
});
it("probes a same-process iframe through its isolated execution context", async () => {
vi.useFakeTimers();
const send = vi.fn(async (_tabId: number, method: string) => {
if (method === "Page.createIsolatedWorld") return { executionContextId: 91 };
return { result: { value: { idleMs: 1_000, readyState: "complete" } } };
});
const cdp: CdpRunner = { send: send as unknown as CdpRunner["send"] };
const settled = waitForDocumentSettled(cdp, {
target: { tabId: 7 },
frameId: "child-frame",
});
await vi.advanceTimersByTimeAsync(180);
await expect(settled).resolves.toBe("quiet");
expect(send).toHaveBeenCalledWith(
7,
"Page.createIsolatedWorld",
expect.objectContaining({ frameId: "child-frame" }),
);
expect(send).toHaveBeenCalledWith(
7,
"Runtime.evaluate",
expect.objectContaining({ contextId: 91 }),
);
});
it("routes an OOPIF probe through its CDP target session", async () => {
vi.useFakeTimers();
const send = vi.fn();
const sendToTarget = vi.fn(async (_target, method: string) => {
if (method === "Page.createIsolatedWorld") return { executionContextId: 27 };
return { result: { value: { idleMs: 1_000, readyState: "complete" } } };
});
const cdp: CdpRunner = {
send: send as unknown as CdpRunner["send"],
sendToTarget: sendToTarget as unknown as NonNullable<CdpRunner["sendToTarget"]>,
};
const target = { tabId: 7, sessionId: "oopif-session" };
const settled = waitForDocumentSettled(cdp, { target, frameId: "oopif-frame" });
await vi.advanceTimersByTimeAsync(180);
await expect(settled).resolves.toBe("quiet");
expect(send).not.toHaveBeenCalled();
expect(sendToTarget).toHaveBeenCalledWith(
target,
"Runtime.evaluate",
expect.objectContaining({ contextId: 27 }),
);
});
});
@@ -1,32 +0,0 @@
import { describe, expect, it } from "vitest";
import {
isAcceptedRecordStepAck,
isRecordStepMessage,
RECORD_STEP,
type RecordStepMessage,
} from "../record-bridge";
const message: RecordStepMessage = {
type: RECORD_STEP,
requestId: "rec-1",
producerId: "document-1",
sequence: 1,
step: { op: "click", target: { tag: "button", name: "Submit" } },
};
describe("record step delivery protocol", () => {
it("requires a producer and a positive integer sequence", () => {
expect(isRecordStepMessage(message)).toBe(true);
expect(isRecordStepMessage({ ...message, producerId: "" })).toBe(false);
expect(isRecordStepMessage({ ...message, sequence: 0 })).toBe(false);
expect(isRecordStepMessage({ ...message, sequence: 1.5 })).toBe(false);
});
it("accepts only the acknowledgement for the sent sequence", () => {
expect(isAcceptedRecordStepAck({ ok: true, sequence: 2 }, 2)).toBe(true);
expect(isAcceptedRecordStepAck({ ok: true, sequence: 1 }, 2)).toBe(false);
expect(isAcceptedRecordStepAck({ ok: false, expectedSequence: 2, error: "gap" }, 2)).toBe(
false,
);
});
});
@@ -1,213 +0,0 @@
import { beforeEach, describe, expect, it, vi } from "vitest";
import { RECORD_FRAME_PORT, RECORD_FRAME_QUERY } from "../recording/frame-bridge";
import { RecordFrameCoordinator } from "../recording/frame-coordinator";
class ListenerSet<T extends (...args: never[]) => unknown> {
readonly listeners = new Set<T>();
addListener = (listener: T) => this.listeners.add(listener);
removeListener = (listener: T) => this.listeners.delete(listener);
}
function fakePort(sender: chrome.runtime.MessageSender) {
const messages: unknown[] = [];
const onMessage = new ListenerSet<(message: unknown) => void>();
const onDisconnect = new ListenerSet<() => void>();
return {
port: {
name: RECORD_FRAME_PORT,
sender,
onMessage,
onDisconnect,
postMessage: vi.fn((message: unknown) => messages.push(message)),
disconnect: vi.fn(),
} as unknown as chrome.runtime.Port,
messages,
receive(message: unknown) {
for (const listener of onMessage.listeners) listener(message);
},
};
}
describe("RecordFrameCoordinator", () => {
const onConnect = new ListenerSet<(port: chrome.runtime.Port) => void>();
const onMessage = new ListenerSet<
(
message: unknown,
sender: chrome.runtime.MessageSender,
sendResponse: (response: unknown) => void,
) => boolean
>();
beforeEach(() => {
onConnect.listeners.clear();
onMessage.listeners.clear();
vi.stubGlobal("chrome", {
runtime: { onConnect, onMessage },
});
});
it("answers child-document queries for the initial tab before it is armed", () => {
const coordinator = new RecordFrameCoordinator({
getAllFrames: async () => [],
sendToDocument: async () => ({ ok: true }),
});
coordinator.attach();
coordinator.begin("rec-1", 10, 3);
const query = (tabId: number) => {
let response: unknown;
for (const listener of onMessage.listeners) {
listener(
{ type: RECORD_FRAME_QUERY },
{
tab: { id: tabId },
frameId: 7,
documentId: "child-document",
} as chrome.runtime.MessageSender,
(value) => {
response = value;
},
);
}
return response;
};
expect(query(3)).toEqual({ active: true, requestId: "rec-1", startedAtMs: 10 });
expect(query(4)).toEqual({ active: false });
});
it("starts a child Document that appears after the initial frame snapshot", async () => {
let notifyFrameNavigation:
| ((frame: {
tabId: number;
frameId: number;
documentId?: string;
lifecycle: "committed" | "completed";
}) => void)
| undefined;
const sendToDocument = vi.fn(async () => ({ ok: true }));
const coordinator = new RecordFrameCoordinator({
getAllFrames: async () => [{ frameId: 0, documentId: "top-document" }],
sendToDocument,
subscribeFrameNavigation(listener) {
notifyFrameNavigation = listener;
return () => {};
},
});
coordinator.attach();
const onDocumentReady = vi.fn();
coordinator.begin("rec-1", 10, 3, onDocumentReady);
await coordinator.armTab("rec-1", 3);
sendToDocument.mockClear();
notifyFrameNavigation?.({
tabId: 3,
frameId: 7,
documentId: "late-child",
lifecycle: "completed",
});
await vi.waitFor(() => {
expect(sendToDocument).toHaveBeenCalledWith(
3,
{ type: "bsk-record-frame-start", requestId: "rec-1", startedAtMs: 10 },
{ documentId: "late-child" },
);
});
expect(onDocumentReady).not.toHaveBeenCalled();
const child = fakePort({
tab: { id: 3 },
frameId: 7,
documentId: "late-child",
} as chrome.runtime.MessageSender);
for (const listener of onConnect.listeners) listener(child.port);
child.receive({ type: "ready", requestId: "rec-1", producerId: "producer-1" });
notifyFrameNavigation?.({
tabId: 3,
frameId: 7,
documentId: "late-child",
lifecycle: "completed",
});
await vi.waitFor(() => {
expect(onDocumentReady).toHaveBeenCalledWith({
tabId: 3,
documentId: "late-child",
browserFrameId: 7,
producerId: "producer-1",
});
});
});
it("binds a producer to its sender Document and keeps final steps valid while stopping", async () => {
const sendToDocument = vi.fn(async () => ({ ok: true }));
const coordinator = new RecordFrameCoordinator({
getAllFrames: async () => [
{ frameId: 0, documentId: "top-document" },
{ frameId: 7, documentId: "child-document" },
],
sendToDocument,
});
coordinator.attach();
coordinator.begin("rec-1", 10, 3);
await expect(coordinator.armTab("rec-1", 3)).resolves.toBe(true);
const sender = {
tab: { id: 3 },
frameId: 7,
documentId: "child-document",
} as chrome.runtime.MessageSender;
const frame = fakePort(sender);
for (const listener of onConnect.listeners) listener(frame.port);
frame.receive({ type: "ready", requestId: "rec-1", producerId: "producer-1" });
expect(coordinator.sourceFor("rec-1", "producer-1", sender)).toEqual({
tabId: 3,
documentId: "child-document",
browserFrameId: 7,
producerId: "producer-1",
});
const stopping = coordinator.stop("rec-1");
expect(coordinator.sourceFor("rec-1", "producer-1", sender)).not.toBeNull();
const stop = frame.messages.find(
(message): message is { type: "stop"; commandId: string } =>
typeof message === "object" &&
message !== null &&
(message as { type?: string }).type === "stop",
);
expect(stop).toBeDefined();
frame.receive({
type: "stopped",
requestId: "rec-1",
commandId: stop!.commandId,
ok: true,
});
await expect(stopping).resolves.toBe(true);
expect(coordinator.sourceFor("rec-1", "producer-1", sender)).toBeNull();
});
it("rejects a producer used from another Document", async () => {
const coordinator = new RecordFrameCoordinator({
getAllFrames: async () => [{ frameId: 0, documentId: "top-document" }],
sendToDocument: async () => ({ ok: true }),
});
coordinator.attach();
coordinator.begin("rec-1", 10, 3);
await coordinator.armTab("rec-1", 3);
const sender = {
tab: { id: 3 },
frameId: 0,
documentId: "top-document",
} as chrome.runtime.MessageSender;
const frame = fakePort(sender);
for (const listener of onConnect.listeners) listener(frame.port);
frame.receive({ type: "ready", requestId: "rec-1", producerId: "producer-1" });
expect(
coordinator.sourceFor("rec-1", "producer-1", {
...sender,
documentId: "other-document",
}),
).toBeNull();
});
});
@@ -1,70 +0,0 @@
import { describe, expect, it } from "vitest";
import {
RECORD_DOCUMENT_ATTRIBUTE,
recordingDocumentMarkerValue,
} from "@/shared/recording-document-identity";
import { markRecordingDocument } from "../recording/document-marker";
import { ObservationNodeIndex } from "../recording/observation-capture";
describe("recording document marker", () => {
it("rejects a non-random Document identity", () => {
expect(() => markRecordingDocument("user@example.com")).toThrow(
"recording Document identity must be a random UUID",
);
});
it("restores the Document attribute exactly", () => {
document.documentElement.setAttribute(RECORD_DOCUMENT_ATTRIBUTE, "page-value");
const producerId = "123e4567-e89b-42d3-a456-426614174000";
const marker = markRecordingDocument(producerId);
expect(document.documentElement.getAttribute(RECORD_DOCUMENT_ATTRIBUTE)).toBe(
recordingDocumentMarkerValue(producerId),
);
marker.restore();
expect(document.documentElement.getAttribute(RECORD_DOCUMENT_ATTRIBUTE)).toBe("page-value");
document.documentElement.removeAttribute(RECORD_DOCUMENT_ATTRIBUTE);
});
it("indexes the CDP scope from the safe frame identity", () => {
const index = new ObservationNodeIndex({
rootFrameId: "root",
frames: [
{
frameId: "child",
target: { tabId: 3, sessionId: "oopif-session" },
recordingDocumentId: "producer-1",
},
],
matchNodes: [],
refs: [],
});
expect(index.documentScope("producer-1")).toEqual({
frameId: "child",
target: { tabId: 3, sessionId: "oopif-session" },
});
});
it("fails closed when a recording identity maps to multiple Documents", () => {
const index = new ObservationNodeIndex({
rootFrameId: "root",
frames: [
{
frameId: "left",
target: { tabId: 3, sessionId: "left-session" },
recordingDocumentId: "duplicate",
},
{
frameId: "right",
target: { tabId: 3, sessionId: "right-session" },
recordingDocumentId: "duplicate",
},
],
matchNodes: [],
refs: [],
});
expect(index.documentScope("duplicate")).toBeUndefined();
});
});
@@ -1,146 +0,0 @@
import { describe, expect, it } from "vitest";
import { ObservationNodeIndex } from "../recording/observation-capture";
import { RecordingObservationSession } from "../recording/observation-session";
import { RecordingStateRegistry } from "../recording/state-registry";
import { buildTraceV3 } from "../recording/trace-builder-v3";
import type { RecordingDraftStep } from "../recording/types";
const URL = "https://example.com/login";
function sessionWithInput(redactValues = false): RecordingObservationSession {
const session = new RecordingObservationSession({ redactValues });
const state = session.registry.register({
url: URL,
vomText: '@vom 1\ntextbox "Password" value="••••••" [ref=e1]',
});
session.cursor.lastSettled = {
stateId: state.id,
rootFrameId: "root",
index: new ObservationNodeIndex({
rootFrameId: "root",
matchNodes: [
{
frameId: "root",
backendNodeId: 42,
tag: "input",
rect: { x: 20, y: 40, w: 200, h: 30 },
localRect: { x: 20, y: 40, w: 200, h: 30 },
},
],
refs: [{ ref: "e1", backendNodeId: 42, role: "textbox", name: "Password", line: 1 }],
}),
url: URL,
};
return session;
}
function finalizedFillTrace(value: string, redactValues: boolean) {
const session = sessionWithInput(redactValues);
const draft: RecordingDraftStep = {
op: "fill",
captureTarget: { tag: "input", role: "textbox", name: "Password" },
value,
targetHint: {
geometry: { rect: { x: 20, y: 40, w: 200, h: 30 }, tag: "input" },
},
};
session.bindDraft(draft, 1);
draft.postStateId = draft.preStateId;
return buildTraceV3({
registry: session.registry,
drafts: [draft],
annotations: session.annotations,
startedAt: "2026-08-12T00:00:00.000Z",
stoppedBy: "user_finish",
bskVersion: "test",
redactValues,
});
}
function finalizedFillBody(value: string, redactValues: boolean): string {
return finalizedFillTrace(value, redactValues).states[0]!.body;
}
describe("record observation annotations", () => {
it("omits a fill literal when values are redacted", () => {
const secret = "hunter2-private";
const dumped = JSON.stringify(finalizedFillTrace(secret, true));
expect(dumped).toContain("step 1: fill");
expect(dumped).not.toContain(secret);
});
it("keeps ordinary fill details", () => {
expect(finalizedFillBody("ordinary text", false)).toContain('step 1: fill: "ordinary text"');
});
it("encodes title and URL so line breaks cannot corrupt state metadata", () => {
const registry = new RecordingStateRegistry();
const state = registry.register({
url: "https://example.com/a\nb",
title: "hello\nworld",
vomText: "@vom 1",
});
const trace = buildTraceV3({
registry,
drafts: [{ op: "scroll", preStateId: state.id, postStateId: state.id }],
startedAt: "2026-08-12T00:00:00.000Z",
stoppedBy: "user_finish",
bskVersion: "test",
});
expect(trace.states[0]?.body).toContain('url: "https://example.com/a\\nb"');
expect(trace.states[0]?.body).toContain('title: "hello\\nworld"');
});
});
describe("recording state ownership", () => {
it("deduplicates within one recording and isolates ids between recordings", () => {
const first = new RecordingStateRegistry();
const second = new RecordingStateRegistry();
expect(first.register({ url: URL, vomText: "same" }).id).toBe("s1");
expect(first.register({ url: URL, vomText: "same" }).id).toBe("s1");
expect(second.register({ url: URL, vomText: "other" }).id).toBe("s1");
});
it("enriches metadata when a deduplicated observation becomes more complete", () => {
const registry = new RecordingStateRegistry();
registry.register({ url: URL, vomText: "same" });
const state = registry.register({
url: URL,
title: "Login",
vomText: "same",
truncated: true,
});
expect(state).toMatchObject({ id: "s1", title: "Login", truncated: true });
});
});
describe("draft binding", () => {
it("keeps capture semantics when no observation exists", () => {
const session = new RecordingObservationSession();
const draft: RecordingDraftStep = {
op: "hover",
captureTarget: { tag: "button", role: "button", name: "新建" },
};
session.bindDraft(draft, 1);
expect(draft.matchedTarget).toEqual({ role: "button", name: "新建", unmatched: true });
expect(draft.preStateId).toBeUndefined();
});
it("does not bind an unmatched new action to a stale observation", () => {
const session = sessionWithInput();
const draft: RecordingDraftStep = {
op: "click",
captureTarget: { tag: "button", role: "button", name: "Confirm" },
targetHint: {
geometry: { rect: { x: 400, y: 300, w: 80, h: 30 }, tag: "button" },
},
};
session.bindDraft(draft, 2, true);
expect(draft.preStateId).toBeUndefined();
expect("matchedTarget" in draft ? draft.matchedTarget : undefined).toEqual({
role: "button",
name: "Confirm",
unmatched: true,
});
});
});
@@ -1,289 +0,0 @@
import { afterEach, describe, expect, it, vi } from "vitest";
import type { CdpRunner, ChromeTabsApi } from "@/tools/shared";
const captureRecordingObservation = vi.hoisted(() => vi.fn());
const waitForDocumentSettled = vi.hoisted(() => vi.fn(async () => "quiet" as const));
vi.mock("../recording/observation-capture", async (importOriginal) => {
const actual = await importOriginal<typeof import("../recording/observation-capture")>();
return { ...actual, captureRecordingObservation };
});
vi.mock("../recording/document-settle", async (importOriginal) => {
const actual = await importOriginal<typeof import("../recording/document-settle")>();
return { ...actual, waitForDocumentSettled };
});
import { ObservationNodeIndex } from "../recording/observation-capture";
import { RecordingObservationRuntime } from "../recording/recording-runtime";
import type { RecordingDraftStep } from "../recording/types";
function observation(url = "https://example.com/") {
return {
rootFrameId: "root",
index: new ObservationNodeIndex({ rootFrameId: "root", matchNodes: [], refs: [] }),
url,
title: "Example",
vomText: '@vom 1\nRootWebArea "Example"',
truncated: false,
};
}
function runtime(): RecordingObservationRuntime {
return new RecordingObservationRuntime({
cdp: { send: vi.fn() as unknown as CdpRunner["send"] },
tabsApi: { get: vi.fn(), query: vi.fn() } as unknown as ChromeTabsApi,
});
}
describe("RecordingObservationRuntime", () => {
afterEach(() => {
captureRecordingObservation.mockReset();
waitForDocumentSettled.mockClear();
});
it("shares one initial capture and includes it in flush", async () => {
let release!: (value: ReturnType<typeof observation>) => void;
captureRecordingObservation.mockReturnValueOnce(
new Promise((resolve) => {
release = resolve;
}),
);
const recording = runtime();
const first = recording.captureInitial(7);
const second = recording.captureInitial(7);
const flushed = recording.flush();
let flushCompleted = false;
void flushed.then(() => {
flushCompleted = true;
});
await Promise.resolve();
expect(captureRecordingObservation).toHaveBeenCalledTimes(1);
expect(flushCompleted).toBe(false);
release(observation());
await Promise.all([first, second, flushed]);
expect(flushCompleted).toBe(true);
});
it("cancels an in-flight initial capture", async () => {
captureRecordingObservation.mockImplementationOnce(
(input: { signal?: AbortSignal }) =>
new Promise((_, reject) => {
input.signal?.addEventListener(
"abort",
() => reject(new DOMException("observation aborted", "AbortError")),
{ once: true },
);
}),
);
const recording = runtime();
const pending = recording.captureInitial(7);
recording.cancel();
await expect(pending).rejects.toMatchObject({ name: "AbortError" });
});
it("retries a failed initial capture when the recording settles at stop", async () => {
captureRecordingObservation
.mockRejectedValueOnce(new Error("document swapped"))
.mockResolvedValueOnce(observation());
const recording = runtime();
await expect(recording.captureInitial(7)).rejects.toThrow("document swapped");
await recording.settleTrailing(7, []);
const trace = recording.buildTrace({
drafts: [],
startedAt: "2026-08-19T00:00:00.000Z",
stoppedBy: "user_finish",
bskVersion: "0.1.6",
});
expect(captureRecordingObservation).toHaveBeenCalledTimes(2);
expect(trace.states).toHaveLength(1);
});
it("keeps runtime matching data out of the serialized trace", async () => {
const captured = observation();
captured.index = new ObservationNodeIndex({
rootFrameId: "root",
matchNodes: [
{
frameId: "root",
backendNodeId: 42,
tag: "button",
rect: { x: 10, y: 20, w: 100, h: 30 },
localRect: { x: 10, y: 20, w: 100, h: 30 },
},
],
refs: [{ ref: "e1", backendNodeId: 42, role: "button", name: "Submit", line: 1 }],
});
captureRecordingObservation.mockResolvedValueOnce(captured);
const recording = runtime();
await recording.captureInitial(7);
const dumped = JSON.stringify(
recording.buildTrace({
drafts: [],
startedAt: "2026-08-19T00:00:00.000Z",
stoppedBy: "user_finish",
bskVersion: "0.1.6",
}),
);
expect(dumped).toContain("RootWebArea");
expect(dumped).not.toContain("matchNodes");
expect(dumped).not.toContain("localRect");
expect(dumped).not.toContain("backendNodeId");
expect(dumped).not.toContain("ObservationNodeIndex");
});
it("captures a tab transition without sharing observation cursors", async () => {
captureRecordingObservation
.mockResolvedValueOnce(observation("https://example.com/first"))
.mockResolvedValueOnce(observation("https://example.com/second"));
const recording = runtime();
await recording.captureInitial(4);
const transition = await recording.captureTabTransition(4, 5);
await recording.captureInitial(4);
await recording.captureInitial(5);
expect(transition).toEqual({
preStateId: "s1",
postStateId: "s2",
targetUrl: "https://example.com/second",
});
expect(captureRecordingObservation.mock.calls.map(([input]) => input.tabId)).toEqual([4, 5]);
});
it("binds an iframe draft to its marked CDP Document scope", async () => {
captureRecordingObservation.mockResolvedValueOnce({
...observation(),
index: new ObservationNodeIndex({
rootFrameId: "root",
frames: [
{
frameId: "child",
target: { tabId: 7, sessionId: "oopif-session" },
recordingDocumentId: "producer-1",
},
],
matchNodes: [
{
backendNodeId: 42,
frameId: "child",
tag: "button",
rect: { x: 410, y: 20, w: 100, h: 30 },
localRect: { x: 10, y: 20, w: 100, h: 30 },
},
],
refs: [{ ref: "e1", backendNodeId: 42, frameId: "child", line: 1 }],
}),
});
const recording = runtime();
const drafts: RecordingDraftStep[] = [
{
op: "click" as const,
captureTarget: { tag: "button", role: "button", name: "Save" },
targetHint: {
geometry: { rect: { x: 10, y: 20, w: 100, h: 30 }, tag: "button" },
},
},
];
await recording.captureInitial(7);
await recording.processDraft(7, drafts, 0, "producer-1");
recording.cancel();
const draft = drafts[0];
expect(draft?.op).toBe("click");
if (!draft || draft.op !== "click") throw new Error("expected click draft");
expect(draft.targetHint).toMatchObject({
frameId: "child",
geometrySpace: "local",
});
expect(draft.matchedTarget?.ref).toBe("e1");
});
it("refreshes a late iframe before matching its first action", async () => {
const iframeObservation = {
...observation(),
index: new ObservationNodeIndex({
rootFrameId: "root",
frames: [
{
frameId: "child",
target: { tabId: 7, sessionId: "oopif-session" },
recordingDocumentId: "producer-1",
},
],
matchNodes: [
{
backendNodeId: 42,
frameId: "child",
tag: "button",
rect: { x: 410, y: 20, w: 100, h: 30 },
localRect: { x: 10, y: 20, w: 100, h: 30 },
},
],
refs: [
{
ref: "e1",
backendNodeId: 42,
frameId: "child",
role: "button",
name: "表格视图",
line: 1,
},
],
}),
};
captureRecordingObservation
.mockResolvedValueOnce(observation())
.mockResolvedValueOnce(iframeObservation)
.mockResolvedValueOnce(iframeObservation);
const recording = runtime();
await recording.captureInitial(7);
await recording.refreshDocument(7, "producer-1");
const drafts: RecordingDraftStep[] = [
{
op: "click",
captureTarget: { tag: "button", role: "button", name: "表格视图" },
targetHint: {
geometry: { rect: { x: 10, y: 20, w: 100, h: 30 }, tag: "button" },
},
},
];
await recording.processDraft(7, drafts, 0, "producer-1");
recording.cancel();
expect(captureRecordingObservation).toHaveBeenCalledTimes(3);
expect(waitForDocumentSettled).toHaveBeenCalledWith(
expect.anything(),
{ frameId: "child", target: { tabId: 7, sessionId: "oopif-session" } },
{ signal: expect.any(AbortSignal) },
);
const draft = drafts[0];
expect(draft?.op).toBe("click");
if (!draft || draft.op !== "click") throw new Error("expected click draft");
expect(draft.matchedTarget).toMatchObject({ ref: "e1", name: "表格视图" });
});
it("does not poison final flush when a frame readiness refresh fails", async () => {
captureRecordingObservation
.mockResolvedValueOnce(observation())
.mockRejectedValueOnce(new Error("child Document replaced"));
const recording = runtime();
await recording.captureInitial(7);
await expect(recording.refreshDocument(7, "producer-1")).rejects.toThrow(
"child Document replaced",
);
await expect(recording.flush()).resolves.toBeUndefined();
});
});
@@ -0,0 +1,80 @@
import { describe, expect, it } from "vitest";
import { appendRecordedPayload, observeRecordedNavigation } from "../recording-step-buffer";
describe("recording-step-buffer", () => {
it("stores semantic click without summary", () => {
const buffer = { steps: [], pendingNavigation: false };
appendRecordedPayload(buffer, {
op: "click",
target: { tag: "button", role: "button", name: "发布" },
expects_navigation: true,
});
expect(buffer.steps).toEqual([
{
op: "click",
target: { tag: "button", role: "button", name: "发布" },
},
]);
expect(buffer.pendingNavigation).toBe(true);
});
it("annotates navigated_to on action-caused navigation instead of wait_for_navigation", () => {
const buffer = {
steps: [
{
op: "click" as const,
target: { tag: "button", role: "button", name: "发布" },
},
],
currentUrl: "https://example.com/a",
pendingNavigation: true,
pendingNavigationDeadline: Date.now() + 5_000,
};
observeRecordedNavigation(buffer, "https://example.com/b", true);
expect(buffer.steps).toEqual([
{
op: "click",
target: { tag: "button", role: "button", name: "发布" },
navigated_to: "https://example.com/b",
},
]);
expect(JSON.stringify(buffer.steps)).not.toContain("wait_for_navigation");
});
it("annotates navigated_to onto a select that auto-submits a navigation", () => {
const buffer = {
steps: [
{
op: "select" as const,
target: { tag: "select", role: "combobox", name: "分类" },
values: ["tech"],
},
],
currentUrl: "https://example.com/list",
pendingNavigation: true,
pendingNavigationDeadline: Date.now() + 5_000,
};
observeRecordedNavigation(buffer, "https://example.com/list?cat=tech", true);
expect(buffer.steps).toEqual([
{
op: "select",
target: { tag: "select", role: "combobox", name: "分类" },
values: ["tech"],
navigated_to: "https://example.com/list?cat=tech",
},
]);
});
it("emits navigate for uncaused URL changes", () => {
const buffer = {
steps: [],
currentUrl: "https://example.com/a",
pendingNavigation: false,
};
observeRecordedNavigation(buffer, "https://example.com/b", false);
expect(buffer.steps[0]).toMatchObject({
op: "navigate",
url: "https://example.com/b",
});
});
});
@@ -1,136 +0,0 @@
import { afterEach, describe, expect, it, vi } from "vitest";
import type { CdpRunner, ChromeTabsApi } from "@/tools/shared";
import { ObservationNodeIndex, type RegisteredObservation } from "../recording/observation-capture";
import { RecordingObservationSession } from "../recording/observation-session";
import { SettleController } from "../recording/settle-controller";
import type { RecordingDraftStep } from "../recording/types";
const OBSERVATION: RegisteredObservation = {
stateId: "s-next",
rootFrameId: "root",
index: new ObservationNodeIndex({ rootFrameId: "root", matchNodes: [], refs: [] }),
url: "https://example.com/next",
};
describe("SettleController", () => {
afterEach(() => vi.useRealTimers());
it("aborts superseded capture work and lands the prior action on the next origin", async () => {
vi.useFakeTimers();
const cdp: CdpRunner = {
send: vi.fn(async () => ({
result: { value: { idleMs: 1_000, readyState: "complete" } },
})) as unknown as CdpRunner["send"],
};
const tabsApi: ChromeTabsApi = {
get: vi.fn(async () => ({ id: 7, url: OBSERVATION.url }) as chrome.tabs.Tab),
query: vi.fn(async () => []),
};
const session = new RecordingObservationSession();
let firstSignal: AbortSignal | undefined;
vi.spyOn(session, "capture")
.mockImplementationOnce(async (_cdp, _tabs, _tabId, signal) => {
firstSignal = signal;
return new Promise<RegisteredObservation>((_resolve, reject) => {
signal?.addEventListener(
"abort",
() => reject(new DOMException("observation aborted", "AbortError")),
{ once: true },
);
});
})
.mockResolvedValueOnce(OBSERVATION);
const drafts: RecordingDraftStep[] = [
{ op: "click", captureTarget: { tag: "button", name: "First" }, preStateId: "s1" },
];
const controller = new SettleController({ session, cdp, tabsApi, tabId: 7 });
controller.schedule(drafts, 0);
await vi.advanceTimersByTimeAsync(180);
drafts.push({
op: "click",
captureTarget: { tag: "button", name: "Second" },
preStateId: "s-next",
});
controller.schedule(drafts, 1);
expect(firstSignal?.aborted).toBe(true);
expect(drafts[0]?.postStateId).toBe("s-next");
await vi.advanceTimersByTimeAsync(500);
await controller.flush();
expect(drafts[1]?.postStateId).toBe("s-next");
});
it("does not consume a newer redirect while reading the prior landing URL", async () => {
vi.useFakeTimers();
const cdp: CdpRunner = {
send: vi.fn(async () => ({
result: { value: { idleMs: 1_000, readyState: "complete" } },
})) as unknown as CdpRunner["send"],
};
let resolveFirstTab!: (tab: chrome.tabs.Tab) => void;
const firstTab = new Promise<chrome.tabs.Tab>((resolve) => {
resolveFirstTab = resolve;
});
const tabsApi: ChromeTabsApi = {
get: vi
.fn()
.mockImplementationOnce(() => firstTab)
.mockResolvedValue({ id: 7, url: OBSERVATION.url }),
query: vi.fn(async () => []),
};
const session = new RecordingObservationSession();
session.cursor.lastSettled = OBSERVATION;
const drafts: RecordingDraftStep[] = [];
const controller = new SettleController({ session, cdp, tabsApi, tabId: 7 });
controller.scheduleRedirect(drafts, "https://example.com/intermediate");
await vi.advanceTimersByTimeAsync(180);
expect(tabsApi.get).toHaveBeenCalledTimes(1);
controller.scheduleRedirect(drafts, OBSERVATION.url);
resolveFirstTab({ id: 7, url: "https://example.com/intermediate" } as chrome.tabs.Tab);
await vi.advanceTimersByTimeAsync(500);
await controller.flushRedirects();
expect(drafts).toEqual([]);
expect(controller.hasPending).toBe(false);
});
it("settles an action in its own OOPIF document scope", async () => {
vi.useFakeTimers();
const send = vi.fn();
const sendToTarget = vi.fn(async (_target, method: string) => {
if (method === "Page.createIsolatedWorld") return { executionContextId: 39 };
return { result: { value: { idleMs: 1_000, readyState: "complete" } } };
});
const cdp: CdpRunner = {
send: send as unknown as CdpRunner["send"],
sendToTarget: sendToTarget as unknown as NonNullable<CdpRunner["sendToTarget"]>,
};
const tabsApi: ChromeTabsApi = {
get: vi.fn(async () => ({ id: 7, url: OBSERVATION.url }) as chrome.tabs.Tab),
query: vi.fn(async () => []),
};
const session = new RecordingObservationSession();
vi.spyOn(session, "capture").mockResolvedValue(OBSERVATION);
const drafts: RecordingDraftStep[] = [
{ op: "click", captureTarget: { tag: "button", name: "Inside frame" } },
];
const controller = new SettleController({ session, cdp, tabsApi, tabId: 7 });
const target = { tabId: 7, sessionId: "oopif-session" };
controller.schedule(drafts, 0, { target, frameId: "oopif-frame" });
await vi.advanceTimersByTimeAsync(180);
await controller.flush();
expect(send).not.toHaveBeenCalled();
expect(sendToTarget).toHaveBeenCalledWith(
target,
"Runtime.evaluate",
expect.objectContaining({ contextId: 39 }),
);
expect(drafts[0]?.postStateId).toBe("s-next");
});
});
@@ -1,164 +0,0 @@
import { describe, expect, it } from "vitest";
import { appendRecordedPayload, observeRecordedNavigation } from "../recording/step-buffer";
describe("recording-step-buffer", () => {
it("stores semantic click without summary", () => {
const buffer = { steps: [], navigation: { pendingNavigation: false } };
appendRecordedPayload(buffer, {
op: "click",
target: { tag: "button", role: "button", name: "发布" },
expects_navigation: true,
});
expect(buffer.steps).toEqual([
{
op: "click",
captureTarget: { tag: "button", role: "button", name: "发布" },
},
]);
expect(buffer.navigation.pendingNavigation).toBe(true);
});
it("keeps the hovered element description as a capture fallback", () => {
const buffer = { steps: [], navigation: { pendingNavigation: false } };
appendRecordedPayload(
buffer,
{
op: "hover",
target: { tag: "button", role: "button", name: "新建" },
geometry: {
rect: { x: 900, y: 8, w: 60, h: 32 },
tag: "button",
},
},
{
geometry: {
rect: { x: 900, y: 8, w: 60, h: 32 },
tag: "button",
},
},
);
expect(buffer.steps).toEqual([
{
op: "hover",
captureTarget: { tag: "button", role: "button", name: "新建" },
targetHint: {
geometry: {
rect: { x: 900, y: 8, w: 60, h: 32 },
tag: "button",
},
},
},
]);
});
it("annotates navigated_to on action-caused navigation instead of wait_for_navigation", () => {
const buffer = {
steps: [
{
op: "click" as const,
captureTarget: { tag: "button", role: "button", name: "发布" },
},
],
navigation: {
currentUrl: "https://example.com/a",
pendingNavigation: true,
pendingNavigationDeadline: Date.now() + 5_000,
},
};
observeRecordedNavigation(buffer, "https://example.com/b", true);
expect(buffer.steps).toEqual([
{
op: "click",
captureTarget: { tag: "button", role: "button", name: "发布" },
navigatedTo: "https://example.com/b",
},
]);
expect(JSON.stringify(buffer.steps)).not.toContain("wait_for_navigation");
});
it("annotates navigated_to onto a select that auto-submits a navigation", () => {
const buffer = {
steps: [
{
op: "select" as const,
captureTarget: { tag: "select", role: "combobox", name: "分类" },
values: ["tech"],
},
],
navigation: {
currentUrl: "https://example.com/list",
pendingNavigation: true,
pendingNavigationDeadline: Date.now() + 5_000,
},
};
observeRecordedNavigation(buffer, "https://example.com/list?cat=tech", true);
expect(buffer.steps).toEqual([
{
op: "select",
captureTarget: { tag: "select", role: "combobox", name: "分类" },
values: ["tech"],
navigatedTo: "https://example.com/list?cat=tech",
},
]);
});
it("emits navigate for uncaused URL changes", () => {
const buffer = {
steps: [],
navigation: { currentUrl: "https://example.com/a", pendingNavigation: false },
};
const result = observeRecordedNavigation(buffer, "https://example.com/b", false);
expect(result).toEqual({ kind: "appended", index: 0 });
expect(buffer.steps[0]).toMatchObject({
op: "navigate",
url: "https://example.com/b",
});
});
it("asks the recorder to coalesce redirect hops instead of emitting each one", () => {
const buffer = {
steps: [],
navigation: {
currentUrl: "https://passport.example/login",
pendingNavigation: false,
},
};
const hop1 = observeRecordedNavigation(
buffer,
"https://passport.example/callback",
false,
"link",
["server_redirect"],
);
expect(hop1).toEqual({
kind: "coalesce_redirect",
url: "https://passport.example/callback",
});
expect(buffer.steps).toEqual([]);
const hop2 = observeRecordedNavigation(buffer, "https://app.example/dashboard", false, "link", [
"client_redirect",
]);
expect(hop2).toEqual({
kind: "coalesce_redirect",
url: "https://app.example/dashboard",
});
expect(buffer.steps).toEqual([]);
expect(buffer.navigation.currentUrl).toBe("https://app.example/dashboard");
});
it("does not reuse redirect metadata for a later content-observed URL change", () => {
const buffer = {
steps: [],
navigation: {
currentUrl: "https://example.com/redirect",
pendingNavigation: false,
},
};
const result = observeRecordedNavigation(buffer, "https://example.com/spa");
expect(result).toEqual({ kind: "appended", index: 0 });
expect(buffer.steps[0]).toMatchObject({ op: "navigate", url: "https://example.com/spa" });
});
});
@@ -1,39 +0,0 @@
import { describe, expect, it } from "vitest";
import { RecordingTabCoordinator } from "../recording/tab-coordinator";
describe("RecordingTabCoordinator", () => {
it("keeps navigation state isolated per tab", () => {
const tabs = new RecordingTabCoordinator(4, "https://example.com/first");
const first = tabs.navigation(4);
first.pendingNavigation = true;
const second = tabs.navigation(5, "https://example.com/second");
expect(second).toEqual({
currentUrl: "https://example.com/second",
pendingNavigation: false,
});
expect(tabs.navigation(4)).toBe(first);
expect(tabs.navigation(4).pendingNavigation).toBe(true);
});
it("invalidates an earlier activation when a newer tab becomes active", () => {
const tabs = new RecordingTabCoordinator(4);
const second = tabs.noteActivation(5);
const third = tabs.noteActivation(6);
expect(tabs.isLatest(second)).toBe(false);
expect(tabs.isLatest(third)).toBe(true);
tabs.commit(6);
expect(tabs.currentTabId).toBe(6);
});
it("commits the freshly observed URL when returning to an existing tab", () => {
const tabs = new RecordingTabCoordinator(4);
tabs.navigation(5, "https://example.com/old");
tabs.commit(5, "https://example.com/new");
expect(tabs.currentTabId).toBe(5);
expect(tabs.navigation(5).currentUrl).toBe("https://example.com/new");
});
});
@@ -1,159 +0,0 @@
import type { RenderedRef } from "@browser-skill/vom";
import { describe, expect, it } from "vitest";
import type { CaptureVomMatchNode } from "@/tools/capture-vom-observation";
import { ObservationNodeIndex, type RegisteredObservation } from "../recording/observation-capture";
import { matchObservationTarget } from "../recording/target-matcher";
function node(backendNodeId: number, frameId: string, x = 10): CaptureVomMatchNode {
return {
backendNodeId,
frameId,
tag: "button",
rect: { x, y: 20, w: 100, h: 30 },
localRect: { x, y: 20, w: 100, h: 30 },
};
}
function observation(
matchNodes: CaptureVomMatchNode[],
refs: RenderedRef[],
): RegisteredObservation {
return {
stateId: "s1",
rootFrameId: "root",
index: new ObservationNodeIndex({ rootFrameId: "root", matchNodes, refs }),
url: "https://example.com",
};
}
describe("matchObservationTarget", () => {
it("indexes the safe geometry DTO without discarding frame-local geometry", () => {
const geometry = node(42, "child");
geometry.localRect = { x: 5, y: 6, w: 100, h: 30 };
const ref: RenderedRef = {
ref: "e1",
backendNodeId: 42,
frameId: "child",
line: 1,
};
const candidate = observation([geometry], [ref]).index.candidates("child", "button")[0];
expect(candidate).toEqual({ frameId: "child", geometry, ref });
});
it("matches a unique node using canonical top-level viewport geometry", () => {
const target = matchObservationTarget({
observation: observation(
[node(42, "root")],
[{ ref: "e1", backendNodeId: 42, role: "button", name: "发布", line: 1 }],
),
hint: { geometry: { rect: { x: 10, y: 20, w: 100, h: 30 }, tag: "button" } },
});
expect(target).toEqual({ ref: "e1", role: "button", name: "发布" });
});
it("uses frame id with backend node id so sibling frames cannot collide", () => {
const target = matchObservationTarget({
observation: observation(
[node(42, "left"), node(42, "right")],
[
{ ref: "e1", backendNodeId: 42, frameId: "left", line: 1 },
{ ref: "e2", backendNodeId: 42, frameId: "right", line: 2 },
],
),
hint: {
frameId: "right",
geometry: { rect: { x: 10, y: 20, w: 100, h: 30 }, tag: "button" },
},
});
expect(target.ref).toBe("e2");
});
it("matches iframe capture geometry in the child viewport coordinate space", () => {
const child = node(42, "child", 410);
child.localRect = { x: 10, y: 20, w: 100, h: 30 };
const target = matchObservationTarget({
observation: observation(
[child],
[{ ref: "e1", backendNodeId: 42, frameId: "child", line: 1 }],
),
hint: {
frameId: "child",
geometrySpace: "local",
geometry: { rect: { x: 10, y: 20, w: 100, h: 30 }, tag: "button" },
},
});
expect(target.ref).toBe("e1");
});
it("restricts a missing frame hint to the root frame", () => {
const target = matchObservationTarget({
observation: observation(
[node(42, "child")],
[{ ref: "e1", backendNodeId: 42, frameId: "child", line: 1 }],
),
hint: { geometry: { rect: { x: 10, y: 20, w: 100, h: 30 }, tag: "button" } },
fallback: { tag: "button", name: "发布" },
});
expect(target).toEqual({ name: "发布", unmatched: true });
});
it("fails closed when the source Document has no VOM frame mapping", () => {
const target = matchObservationTarget({
observation: observation(
[node(42, "root")],
[{ ref: "e1", backendNodeId: 42, role: "button", name: "发布", line: 1 }],
),
hint: {
frameId: null,
geometry: { rect: { x: 10, y: 20, w: 100, h: 30 }, tag: "button" },
},
fallback: { tag: "button", role: "button", name: "发布" },
});
expect(target).toEqual({ role: "button", name: "发布", unmatched: true });
});
it("returns unmatched for ambiguous geometry", () => {
const target = matchObservationTarget({
observation: observation(
[node(42, "root"), node(43, "root")],
[
{ ref: "e1", backendNodeId: 42, line: 1 },
{ ref: "e2", backendNodeId: 43, line: 2 },
],
),
hint: { geometry: { rect: { x: 10, y: 20, w: 100, h: 30 }, tag: "button" } },
});
expect(target.unmatched).toBe(true);
});
it("uses semantics when geometry is unavailable without crossing frame boundaries", () => {
const target = matchObservationTarget({
observation: observation(
[node(41, "root"), node(42, "child")],
[
{ ref: "e1", backendNodeId: 41, role: "button", name: "保存", line: 1 },
{ ref: "e2", backendNodeId: 42, frameId: "child", role: "button", name: "保存", line: 2 },
],
),
hint: { frameId: "child" },
fallback: { tag: "button", role: "button", name: "保存" },
});
expect(target.ref).toBe("e2");
});
it("uses semantics to disambiguate equal geometry in one frame", () => {
const target = matchObservationTarget({
observation: observation(
[node(42, "root"), node(43, "root")],
[
{ ref: "e1", backendNodeId: 42, role: "button", name: "保存", line: 1 },
{ ref: "e2", backendNodeId: 43, role: "button", name: "取消", line: 2 },
],
),
hint: { geometry: { rect: { x: 10, y: 20, w: 100, h: 30 }, tag: "button" } },
fallback: { tag: "button", role: "button", name: "取消" },
});
expect(target.ref).toBe("e2");
});
});
@@ -1,132 +0,0 @@
import { describe, expect, it } from "vitest";
import { TRACE_VERSION_V3, VOM_FORMAT_VERSION } from "@/transport/types";
import { RecordingStateRegistry } from "../recording/state-registry";
import { buildTraceV3 } from "../recording/trace-builder-v3";
import { reduceTraceStepsV3 } from "../recording/trace-reducer-v3";
import type { RecordingDraftStep } from "../recording/types";
describe("trace reducer v3", () => {
it("removes form literals when value redaction is requested", () => {
const reduced = reduceTraceStepsV3(
[
{
op: "fill",
value: "private@example.com",
preStateId: "s1",
postStateId: "s1",
},
{
op: "select",
values: ["private-account-id"],
labels: ["Private account"],
preStateId: "s1",
postStateId: "s1",
},
],
{ redactValues: true },
);
expect(reduced.steps[0]).toMatchObject({ op: "fill", value: "***", redacted: true });
expect(reduced.steps[1]).toMatchObject({ op: "select" });
expect(reduced.steps[1]).not.toHaveProperty("selection");
expect(JSON.stringify(reduced.steps)).not.toContain("private@example.com");
expect(JSON.stringify(reduced.steps)).not.toContain("private-account-id");
});
it("emits tab transitions only for callers that advertised support", () => {
const drafts: RecordingDraftStep[] = [
{
op: "switch_tab",
preStateId: "s1",
postStateId: "s2",
},
];
expect(reduceTraceStepsV3(drafts).steps).toEqual([]);
expect(reduceTraceStepsV3(drafts, { includeTabSwitches: true }).steps).toEqual([
{ op: "switch_tab", id: 1, state: "s1", result: { state: "s2" } },
]);
});
it("keeps hover steps before menu clicks", () => {
const drafts: RecordingDraftStep[] = [
{
op: "hover",
captureTarget: { tag: "span", role: "button", name: "Account" },
preStateId: "s1",
postStateId: "s1",
},
{
op: "click",
captureTarget: { tag: "a", role: "link", name: "Profile" },
preStateId: "s1",
postStateId: "s1",
},
];
expect(reduceTraceStepsV3(drafts).steps.map((s) => s.op)).toEqual(["hover", "click"]);
});
it("does not export a tab transition without both observation endpoints", () => {
const sourceOnly: RecordingDraftStep = { op: "switch_tab", preStateId: "s1" };
const targetOnly: RecordingDraftStep = { op: "switch_tab", postStateId: "s2" };
expect(
reduceTraceStepsV3([sourceOnly, targetOnly], { includeTabSwitches: true }).steps,
).toEqual([]);
});
it("collapses redirect hops while retaining draft-to-step identity", () => {
const drafts: RecordingDraftStep[] = [
{ op: "navigate", url: "https://example.com/start", preStateId: "s1", postStateId: "s2" },
{
op: "navigate",
url: "https://example.com/final",
transitionQualifiers: ["server_redirect"],
preStateId: "s2",
postStateId: "s3",
},
];
const before = structuredClone(drafts);
const reduced = reduceTraceStepsV3(drafts);
expect(drafts).toEqual(before);
expect(reduced.steps).toEqual([
expect.objectContaining({
id: 1,
op: "navigate",
state: "s1",
to: "https://example.com/final",
result: { state: "s3" },
}),
]);
expect(reduced.stepIdByDraftId.get(1)).toBe(1);
expect(reduced.stepIdByDraftId.get(2)).toBe(1);
});
it("builds the wire model from protocol constants", () => {
const registry = new RecordingStateRegistry();
const state = registry.register({ url: "https://example.com", vomText: "@vom 1" });
const trace = buildTraceV3({
registry,
drafts: [
{
op: "click",
captureTarget: { tag: "button", role: "button", name: "Save" },
preStateId: state.id,
postStateId: state.id,
},
],
startedAt: "2026-08-12T00:00:00.000Z",
stoppedBy: "user_finish",
bskVersion: "test",
});
expect(trace.version).toBe(TRACE_VERSION_V3);
expect(trace.recorder.vom).toBe(VOM_FORMAT_VERSION);
expect(trace.steps[0]).toMatchObject({
op: "click",
target: { role: "button", name: "Save", unmatched: true },
});
});
});
@@ -1,15 +1,6 @@
import { describe, expect, it } from "vitest";
import type { RecordingDraftStep } from "@/lib/recording/types";
import { shouldRecordPress } from "../recording/draft-policy";
import { buildTraceV2 } from "../recording/trace-reducer-v2";
function reduceTraceSteps(steps: RecordingDraftStep[], startUrl?: string) {
return buildTraceV2({
steps,
startedAt: "2026-01-01T00:00:00.000Z",
...(startUrl ? { startUrl } : {}),
});
}
import type { DraftTraceStep } from "@/transport/types";
import { reduceTraceSteps, resolveTraceStartUrl, shouldRecordPress } from "../trace-reducer";
describe("shouldRecordPress", () => {
it("keeps Enter and Escape", () => {
@@ -27,47 +18,36 @@ describe("shouldRecordPress", () => {
});
describe("reduceTraceSteps", () => {
it("does not expose internal tab transitions in trace v2", () => {
const trace = reduceTraceSteps([
{
op: "switch_tab",
preStateId: "s1",
postStateId: "s2",
},
]);
expect(trace.steps).toEqual([]);
});
it("builds steps with pages dictionary and page id references", () => {
const drafts: RecordingDraftStep[] = [
const drafts: DraftTraceStep[] = [
{
op: "navigate",
url: "https://example.com/search?q=hello&utm_source=x",
pageUrl: "https://example.com/search?q=hello&utm_source=x",
page_url: "https://example.com/search?q=hello&utm_source=x",
},
{
op: "fill",
captureTarget: { tag: "input", role: "textbox", name: "搜索", name_attr: "q" },
target: { tag: "input", role: "textbox", name: "搜索", name_attr: "q" },
value: "browser skill",
pageUrl: "https://example.com/search?q=hello&utm_source=x",
page_url: "https://example.com/search?q=hello&utm_source=x",
},
{
op: "press",
key: "Enter",
captureTarget: { tag: "input", role: "textbox", name: "搜索", name_attr: "q" },
navigatedTo: "https://example.com/results/42",
pageUrl: "https://example.com/search?q=hello&utm_source=x",
target: { tag: "input", role: "textbox", name: "搜索", name_attr: "q" },
navigated_to: "https://example.com/results/42",
page_url: "https://example.com/search?q=hello&utm_source=x",
},
{
op: "click",
captureTarget: { tag: "button", role: "button", name: "发布" },
navigatedTo: "https://example.com/p/99",
pageUrl: "https://example.com/results/42",
target: { tag: "button", role: "button", name: "发布" },
navigated_to: "https://example.com/p/99",
page_url: "https://example.com/results/42",
},
{
op: "press",
key: "a",
pageUrl: "https://example.com/p/99",
page_url: "https://example.com/p/99",
},
];
@@ -129,9 +109,9 @@ describe("reduceTraceSteps", () => {
[
{
op: "fill",
captureTarget: { tag: "input", role: "textbox", name: "Search query" },
target: { tag: "input", role: "textbox", name: "Search query" },
value: "",
pageUrl: "https://example.com/search",
page_url: "https://example.com/search",
},
],
"https://example.com/search",
@@ -145,16 +125,16 @@ describe("reduceTraceSteps", () => {
]);
});
it("maps select navigatedTo onto effect.navigatedTo (page id)", () => {
it("maps select navigated_to onto effect.navigated_to (page id)", () => {
const { pages, steps } = reduceTraceSteps(
[
{
op: "select",
captureTarget: { tag: "select", role: "combobox", name: "分类" },
target: { tag: "select", role: "combobox", name: "分类" },
values: ["tech"],
labels: ["技术"],
navigatedTo: "https://example.com/list?cat=tech",
pageUrl: "https://example.com/list",
navigated_to: "https://example.com/list?cat=tech",
page_url: "https://example.com/list",
},
],
"https://example.com/list",
@@ -171,32 +151,36 @@ describe("reduceTraceSteps", () => {
});
});
it("drops hover steps, which only exist in trace v3", () => {
it("keeps hover steps before menu clicks", () => {
const { steps } = reduceTraceSteps(
[
{
op: "hover",
captureTarget: { tag: "span", role: "button", name: "Account" },
pageUrl: "https://example.com/app",
target: { tag: "span", role: "button", name: "Account" },
page_url: "https://example.com/app",
},
{
op: "click",
captureTarget: { tag: "a", role: "link", name: "Profile" },
pageUrl: "https://example.com/app",
target: { tag: "a", role: "link", name: "Profile" },
page_url: "https://example.com/app",
},
],
"https://example.com/app",
);
expect(steps.map((s) => s.op)).toEqual(["click"]);
expect(steps.map((s) => s.op)).toEqual(["hover", "click"]);
expect(steps[0]).toMatchObject({
op: "hover",
page: "p1",
target: { name: "Account" },
});
});
it("resolveTraceStartUrl prefers explicit start URL", () => {
expect(
buildTraceV2({
steps: [{ op: "navigate", url: "https://example.com/other" }],
startedAt: "2026-01-01T00:00:00.000Z",
startUrl: "https://example.com/start",
}).entry.start_url,
resolveTraceStartUrl(
[{ op: "navigate", url: "https://example.com/other" }],
"https://example.com/start",
),
).toBe("https://example.com/start");
});
});
+5 -6
View File
@@ -1,13 +1,12 @@
/**
* Build a semantic capture descriptor for an interacted element.
* Build a semantic TargetDescriptor for an interacted element.
*
* Trace steps are an LLM *textbook*: each click must say what to look for
* on screen (usually a short visible name). Tag-only noise like
* `{ "tag": "div" }` / “点击div” is useless and must not be recorded.
*/
/** Content-script capture descriptor before VOM geometric matching. */
export interface CaptureTargetDescriptor {
export interface TargetDescriptor {
role?: string;
name?: string;
tag: string;
@@ -248,7 +247,7 @@ export function resolveHoverElement(target: Element): Element | null {
* (visible name), or at least a form `name_attr` for checkbox/radio.
* Recording “点击div” with no name fails this bar.
*/
export function isMeaningfulClickTarget(target: CaptureTargetDescriptor): boolean {
export function isMeaningfulClickTarget(target: TargetDescriptor): boolean {
const name = target.name?.trim();
if (name && isActionableLabel(name)) return true;
if (
@@ -324,7 +323,7 @@ function nearbyLabelText(el: Element): string | undefined {
return undefined;
}
export function describeTarget(el: Element): CaptureTargetDescriptor {
export function describeTarget(el: Element): TargetDescriptor {
const tag = el.tagName.toLowerCase();
const role = inferRole(el);
const name = accessibleName(el);
@@ -351,7 +350,7 @@ export function describeTarget(el: Element): CaptureTargetDescriptor {
};
}
export function describeEventTarget(target: EventTarget | null): CaptureTargetDescriptor | null {
export function describeEventTarget(target: EventTarget | null): TargetDescriptor | null {
if (!(target instanceof Element)) return null;
const clickable = resolveClickableElement(target);
if (!clickable) return null;
+3 -39
View File
@@ -3,7 +3,7 @@
* service worker and a tab's content script.
*/
import type { CaptureTargetDescriptor } from "./describe-target";
import type { TargetDescriptor } from "./describe-target";
export const RECORD_START = "bsk-record-start";
export const RECORD_STEP = "bsk-record-step";
@@ -16,10 +16,6 @@ export interface RecordStartAck {
ok: true;
}
export type RecordStepAck =
| { ok: true; sequence: number }
| { ok: false; expectedSequence: number; error: string };
export type RecordStopAck = { ok: true } | { ok: false; error: string };
export interface RecordQueryMessage {
@@ -46,7 +42,7 @@ export interface RecordStartMessage {
export interface RecordStepPayload {
op: "click" | "hover" | "fill" | "press" | "select" | "navigate";
target?: CaptureTargetDescriptor;
target?: TargetDescriptor;
value?: string;
key?: string;
modifiers?: Array<"alt" | "ctrl" | "meta" | "shift">;
@@ -54,30 +50,17 @@ export interface RecordStepPayload {
labels?: string[];
url?: string;
redacted?: boolean;
commit?: "enter" | "suggestion" | "blur";
/** Page URL when the step was captured. */
page_url?: string;
/** Event-target bounds in the source frame's viewport coordinate space. */
geometry?: {
rect: { x: number; y: number; w: number; h: number };
tag: string;
};
/** Capture-only hint; never persisted unless converted to navigated_to. */
expects_navigation?: boolean;
/** Whether an observed URL change was synchronously caused by the action. */
navigation_caused_by_action?: boolean;
/** Raw webNavigation transition metadata for cause mapping. */
transitionType?: string;
transitionQualifiers?: string[];
}
export interface RecordStepMessage {
type: typeof RECORD_STEP;
requestId: string;
/** Stable for the lifetime of one content-script document. */
producerId: string;
/** Monotonic within one content-script producer and recording request. */
sequence: number;
step: RecordStepPayload;
}
@@ -129,27 +112,8 @@ export function isRecordQueryMessage(msg: unknown): msg is RecordQueryMessage {
export function isRecordStepMessage(msg: unknown): msg is RecordStepMessage {
if (typeof msg !== "object" || msg === null) return false;
const m = msg as Record<string, unknown>;
if (
m.type !== RECORD_STEP ||
typeof m.requestId !== "string" ||
typeof m.producerId !== "string" ||
m.producerId.length === 0 ||
typeof m.sequence !== "number" ||
!Number.isSafeInteger(m.sequence) ||
m.sequence < 1
) {
return false;
}
if (m.type !== RECORD_STEP || typeof m.requestId !== "string") return false;
const step = m.step;
if (typeof step !== "object" || step === null) return false;
return typeof (step as RecordStepPayload).op === "string";
}
export function isAcceptedRecordStepAck(
value: unknown,
sequence: number,
): value is Extract<RecordStepAck, { ok: true }> {
if (typeof value !== "object" || value === null) return false;
const ack = value as Partial<RecordStepAck>;
return ack.ok === true && ack.sequence === sequence;
}
@@ -0,0 +1,136 @@
import type { DraftTraceStep } from "@/transport/types";
import type { RecordStepPayload } from "./record-bridge";
export interface RecordingStepBuffer {
steps: DraftTraceStep[];
currentUrl?: string;
pendingNavigation: boolean;
pendingNavigationDeadline?: number;
}
const NAVIGATION_TRIGGER_WINDOW_MS = 3_000;
function toDraftStep(payload: RecordStepPayload): DraftTraceStep | null {
const pageUrl = payload.page_url;
switch (payload.op) {
case "click":
return payload.target
? {
op: "click",
target: payload.target,
...(pageUrl ? { page_url: pageUrl } : {}),
}
: null;
case "hover":
return payload.target
? {
op: "hover",
target: payload.target,
...(pageUrl ? { page_url: pageUrl } : {}),
}
: null;
case "fill":
return payload.target
? {
op: "fill",
target: payload.target,
value: payload.value ?? "",
...(payload.redacted ? { redacted: true } : {}),
...(pageUrl ? { page_url: pageUrl } : {}),
}
: null;
case "press":
return payload.key
? {
op: "press",
key: payload.key,
...(payload.target ? { target: payload.target } : {}),
...(payload.modifiers?.length ? { modifiers: payload.modifiers } : {}),
...(pageUrl ? { page_url: pageUrl } : {}),
}
: null;
case "select":
return payload.target && payload.values
? {
op: "select",
target: payload.target,
values: payload.values,
...(payload.labels?.length ? { labels: payload.labels } : {}),
...(pageUrl ? { page_url: pageUrl } : {}),
}
: null;
case "navigate":
return null;
}
}
function annotateLastStepNavigation(buffer: RecordingStepBuffer, url: string): boolean {
for (let i = buffer.steps.length - 1; i >= 0; i -= 1) {
const step = buffer.steps[i];
if (!step) continue;
if (step.op === "click" || step.op === "press" || step.op === "select") {
buffer.steps[i] = { ...step, navigated_to: url };
return true;
}
break;
}
return false;
}
export function observeRecordedNavigation(
buffer: RecordingStepBuffer,
url: string,
causedByAction?: boolean,
): void {
if (!url || url === buffer.currentUrl) return;
buffer.currentUrl = url;
const pendingIsCurrent =
buffer.pendingNavigation &&
(buffer.pendingNavigationDeadline === undefined ||
buffer.pendingNavigationDeadline >= Date.now());
if (causedByAction === true || (causedByAction === undefined && pendingIsCurrent)) {
buffer.pendingNavigation = false;
buffer.pendingNavigationDeadline = undefined;
if (!annotateLastStepNavigation(buffer, url)) {
buffer.steps.push({
op: "navigate",
url,
page_url: url,
});
}
return;
}
buffer.pendingNavigation = false;
buffer.pendingNavigationDeadline = undefined;
buffer.steps.push({
op: "navigate",
url,
page_url: url,
});
}
export function appendRecordedPayload(
buffer: RecordingStepBuffer,
payload: RecordStepPayload,
): void {
if (payload.op === "navigate") {
if (payload.url) {
observeRecordedNavigation(buffer, payload.url, payload.navigation_caused_by_action);
}
return;
}
const step = toDraftStep({
...payload,
page_url: payload.page_url ?? buffer.currentUrl,
});
if (!step) return;
buffer.steps.push(step);
if (step.op === "click" || step.op === "press" || step.op === "select" || step.op === "fill") {
buffer.pendingNavigation = payload.expects_navigation === true;
buffer.pendingNavigationDeadline = buffer.pendingNavigation
? Date.now() + NAVIGATION_TRIGGER_WINDOW_MS
: undefined;
}
}
@@ -1,39 +0,0 @@
import {
RECORD_DOCUMENT_ATTRIBUTE,
recordingDocumentMarkerValue,
} from "@/shared/recording-document-identity";
export interface RecordingDocumentMarker {
restore(): void;
ensure(): void;
}
export function waitForDocumentElement(): Promise<HTMLElement> {
if (document.documentElement) return Promise.resolve(document.documentElement);
return new Promise((resolve) => {
const observer = new MutationObserver(() => {
if (!document.documentElement) return;
observer.disconnect();
resolve(document.documentElement);
});
observer.observe(document, { childList: true });
});
}
export function markRecordingDocument(
producerId: string,
root: HTMLElement = document.documentElement,
): RecordingDocumentMarker {
const previous = root.getAttribute(RECORD_DOCUMENT_ATTRIBUTE);
const markerValue = recordingDocumentMarkerValue(producerId);
const ensure = () => root.setAttribute(RECORD_DOCUMENT_ATTRIBUTE, markerValue);
ensure();
return {
ensure,
restore() {
if (root.getAttribute(RECORD_DOCUMENT_ATTRIBUTE) !== markerValue) return;
if (previous === null) root.removeAttribute(RECORD_DOCUMENT_ATTRIBUTE);
else root.setAttribute(RECORD_DOCUMENT_ATTRIBUTE, previous);
},
};
}
@@ -1,134 +0,0 @@
import type { CdpTarget } from "@/browser-driver/frame-graph";
import { type CdpRunner, sendToCdpTarget } from "@/tools/shared";
const SETTLE_MIN_MS = 150;
const SETTLE_QUIET_MS = 250;
const SETTLE_MAX_MS = 2_000;
const SETTLE_POLL_MS = 60;
const SETTLE_WORLD_NAME = "__bsk_record_settle__";
const QUIET_PROBE = `(() => {
const scope = window;
let probe = scope.__bskRecordQuiet;
if (!probe) {
probe = { changedAt: Date.now() };
const observer = new MutationObserver(() => {
probe.changedAt = Date.now();
});
observer.observe(document.documentElement, {
subtree: true,
childList: true,
attributes: true,
characterData: true,
});
scope.__bskRecordQuiet = probe;
}
return { idleMs: Date.now() - probe.changedAt, readyState: document.readyState };
})()`;
export interface DocumentSettleScope {
target: CdpTarget;
/** Omit only for the target's root document. */
frameId?: string;
}
type SettleOutcome = "quiet" | "timeout" | "cancelled";
interface QuietProbe {
idleMs: number;
readyState: string;
}
interface ProbeContext {
executionContextId?: number;
}
function sleep(ms: number, signal?: AbortSignal): Promise<void> {
return new Promise((resolve) => {
if (signal?.aborted) return resolve();
const timer = setTimeout(resolve, ms);
signal?.addEventListener(
"abort",
() => {
clearTimeout(timer);
resolve();
},
{ once: true },
);
});
}
async function executionContextForScope(
cdp: CdpRunner,
scope: DocumentSettleScope,
): Promise<number | undefined> {
if (!scope.frameId) return undefined;
const result = await sendToCdpTarget<{ executionContextId?: number }>(
cdp,
scope.target,
"Page.createIsolatedWorld",
{
frameId: scope.frameId,
worldName: SETTLE_WORLD_NAME,
grantUniveralAccess: false,
},
);
return result.executionContextId;
}
async function readQuietProbe(
cdp: CdpRunner,
scope: DocumentSettleScope,
context: ProbeContext,
): Promise<QuietProbe | null> {
try {
if (scope.frameId && context.executionContextId === undefined) {
context.executionContextId = await executionContextForScope(cdp, scope);
if (context.executionContextId === undefined) return null;
}
const reply = await sendToCdpTarget<{ result?: { value?: unknown } }>(
cdp,
scope.target,
"Runtime.evaluate",
{
expression: QUIET_PROBE,
returnByValue: true,
...(context.executionContextId !== undefined
? { contextId: context.executionContextId }
: {}),
},
);
const value = reply.result?.value;
if (!value || typeof value !== "object") return null;
const { idleMs, readyState } = value as { idleMs?: unknown; readyState?: unknown };
if (typeof idleMs !== "number" || typeof readyState !== "string") return null;
return { idleMs, readyState };
} catch {
context.executionContextId = undefined;
return null;
}
}
export async function waitForDocumentSettled(
cdp: CdpRunner,
scope: DocumentSettleScope,
options: { signal?: AbortSignal } = {},
): Promise<SettleOutcome> {
const startedAt = Date.now();
const floor = startedAt + SETTLE_MIN_MS;
const deadline = startedAt + SETTLE_MAX_MS;
const context: ProbeContext = {};
for (;;) {
if (options.signal?.aborted) return "cancelled";
await sleep(SETTLE_POLL_MS, options.signal);
if (options.signal?.aborted) return "cancelled";
const probe = await readQuietProbe(cdp, scope, context);
const now = Date.now();
if (now < floor) continue;
if (now >= deadline) return "timeout";
if (!probe || probe.readyState === "loading") continue;
if (probe.idleMs >= SETTLE_QUIET_MS) return "quiet";
}
}
@@ -1,34 +0,0 @@
import type { RecordingDraftStep } from "./types";
const CLIPBOARD_KEYS = new Set(["a", "c", "v", "x", "A", "C", "V", "X"]);
const MODIFIER_ONLY_KEYS = new Set(["Meta", "Control", "Alt", "Shift", "OS", "Hyper", "Super"]);
export function shouldRecordPress(
key: string,
modifiers?: Array<"alt" | "ctrl" | "meta" | "shift">,
): boolean {
if (MODIFIER_ONLY_KEYS.has(key)) return false;
const mods = modifiers ?? [];
const hasCtrlOrMeta = mods.includes("ctrl") || mods.includes("meta");
if (hasCtrlOrMeta && CLIPBOARD_KEYS.has(key)) return false;
if (key === "Enter" || key === "Escape") return true;
if (key.length === 1 && !hasCtrlOrMeta && !mods.includes("alt")) return false;
return false;
}
export function shouldIncludeDraft(step: RecordingDraftStep): boolean {
return step.op !== "press" || shouldRecordPress(step.key, step.modifiers);
}
export function resolveDraftStartUrl(
drafts: RecordingDraftStep[],
explicitStartUrl?: string,
fallbackUrl?: string,
): string {
if (explicitStartUrl) return explicitStartUrl;
const navigation = drafts.find(
(step): step is Extract<RecordingDraftStep, { op: "navigate" }> => step.op === "navigate",
);
if (navigation) return navigation.url;
return drafts.find((step) => step.pageUrl)?.pageUrl ?? fallbackUrl ?? "about:blank";
}
@@ -1,65 +0,0 @@
export const RECORD_FRAME_PORT = "bsk-record-frame";
export const RECORD_FRAME_QUERY = "bsk-record-frame-query";
export const RECORD_FRAME_START = "bsk-record-frame-start";
export interface RecordFrameQueryMessage {
type: typeof RECORD_FRAME_QUERY;
}
export interface RecordFrameQueryResponse {
active: boolean;
requestId?: string;
startedAtMs?: number;
}
export interface RecordFrameStartMessage {
type: typeof RECORD_FRAME_START;
requestId: string;
startedAtMs: number;
}
export type RecordFramePortMessage =
| {
type: "ready";
requestId: string;
producerId: string;
}
| {
type: "ready_ack";
requestId: string;
producerId: string;
}
| {
type: "stop";
requestId: string;
commandId: string;
}
| {
type: "cancel";
requestId: string;
}
| {
type: "stopped";
requestId: string;
commandId: string;
ok: boolean;
error?: string;
};
export function isRecordFrameQueryMessage(value: unknown): value is RecordFrameQueryMessage {
return (
typeof value === "object" &&
value !== null &&
(value as { type?: unknown }).type === RECORD_FRAME_QUERY
);
}
export function isRecordFrameStartMessage(value: unknown): value is RecordFrameStartMessage {
if (typeof value !== "object" || value === null) return false;
const message = value as Partial<RecordFrameStartMessage>;
return (
message.type === RECORD_FRAME_START &&
typeof message.requestId === "string" &&
typeof message.startedAtMs === "number"
);
}
@@ -1,399 +0,0 @@
import {
isRecordFrameQueryMessage,
RECORD_FRAME_PORT,
RECORD_FRAME_START,
type RecordFramePortMessage,
type RecordFrameQueryResponse,
type RecordFrameStartMessage,
} from "./frame-bridge";
export interface RecordingCaptureScope {
tabId: number;
documentId: string;
browserFrameId: number;
producerId: string;
}
interface ArmedRecording {
requestId: string;
startedAtMs: number;
tabIds: Set<number>;
agents: Map<string, FrameAgent>;
onDocumentReady?: (scope: RecordingCaptureScope) => void | Promise<void>;
finishing: boolean;
}
interface FrameAgent extends RecordingCaptureScope {
requestId: string;
port: chrome.runtime.Port;
stopWaiters: Map<string, (ok: boolean) => void>;
}
interface BrowserFrame {
frameId: number;
documentId?: string;
}
interface BrowserFrameNavigation extends BrowserFrame {
tabId: number;
lifecycle: "committed" | "completed";
}
const RECORD_FRAME_STOP_TIMEOUT_MS = 5_000;
export interface RecordFrameCoordinatorDeps {
getAllFrames(tabId: number): Promise<BrowserFrame[]>;
sendToDocument(
tabId: number,
message: RecordFrameStartMessage,
target: { documentId?: string; frameId?: number },
): Promise<unknown>;
subscribeFrameNavigation?(listener: (frame: BrowserFrameNavigation) => void): () => void;
}
function documentKey(tabId: number, documentId: string): string {
return `${tabId}:${documentId}`;
}
function senderAddress(
sender: chrome.runtime.MessageSender,
): { tabId: number; documentId: string; browserFrameId: number } | null {
const tabId = sender.tab?.id;
const documentId = sender.documentId;
const browserFrameId = sender.frameId;
if (
typeof tabId !== "number" ||
typeof documentId !== "string" ||
typeof browserFrameId !== "number"
) {
return null;
}
return { tabId, documentId, browserFrameId };
}
function defaultDeps(): RecordFrameCoordinatorDeps {
return {
async getAllFrames(tabId) {
return (await chrome.webNavigation.getAllFrames({ tabId })) ?? [];
},
sendToDocument(tabId, message, target) {
return chrome.tabs.sendMessage(tabId, message, target);
},
subscribeFrameNavigation(listener) {
const onCommitted = (details: chrome.webNavigation.WebNavigationTransitionCallbackDetails) =>
listener({ ...details, lifecycle: "committed" });
const onCompleted = (details: chrome.webNavigation.WebNavigationFramedCallbackDetails) =>
listener({ ...details, lifecycle: "completed" });
chrome.webNavigation.onCommitted.addListener(onCommitted);
chrome.webNavigation.onCompleted.addListener(onCompleted);
return () => {
chrome.webNavigation.onCommitted.removeListener(onCommitted);
chrome.webNavigation.onCompleted.removeListener(onCompleted);
};
},
};
}
export class RecordFrameCoordinator {
readonly #deps: RecordFrameCoordinatorDeps;
readonly #recordings = new Map<string, ArmedRecording>();
#attached = false;
#detachFrameNavigation: (() => void) | null = null;
constructor(deps: RecordFrameCoordinatorDeps = defaultDeps()) {
this.#deps = deps;
}
attach(): () => void {
if (this.#attached) return () => {};
this.#attached = true;
chrome.runtime.onConnect.addListener(this.#onConnect);
chrome.runtime.onMessage.addListener(this.#onMessage);
this.#detachFrameNavigation =
this.#deps.subscribeFrameNavigation?.(this.#onFrameNavigation) ?? null;
return () => {
if (!this.#attached) return;
this.#attached = false;
chrome.runtime.onConnect.removeListener(this.#onConnect);
chrome.runtime.onMessage.removeListener(this.#onMessage);
this.#detachFrameNavigation?.();
this.#detachFrameNavigation = null;
for (const recording of this.#recordings.values()) this.#cancelAgents(recording);
this.#recordings.clear();
};
}
begin(
requestId: string,
startedAtMs: number,
initialTabId: number,
onDocumentReady?: (scope: RecordingCaptureScope) => void | Promise<void>,
): void {
const previous = this.#recordings.get(requestId);
if (previous) this.#cancelAgents(previous);
this.#recordings.set(requestId, {
requestId,
startedAtMs,
// Register the initial tab before navigation begins. A newly-created
// child document can query at document_start before armTab's frame
// snapshot sees it, especially when Chromium promotes it to an OOPIF.
tabIds: new Set([initialTabId]),
agents: new Map(),
...(onDocumentReady ? { onDocumentReady } : {}),
finishing: false,
});
}
async armTab(requestId: string, tabId: number): Promise<boolean> {
const recording = this.#recordings.get(requestId);
if (!recording || recording.finishing) return false;
recording.tabIds.add(tabId);
let frames: BrowserFrame[];
try {
frames = await this.#deps.getAllFrames(tabId);
} catch {
frames = [{ frameId: 0 }];
}
if (!frames.some((frame) => frame.frameId === 0)) frames.unshift({ frameId: 0 });
const results = await Promise.all(
frames.map(async (frame) => {
const started = await this.#startFrame(recording, tabId, frame);
return { frameId: frame.frameId, started };
}),
);
return results.some((result) => result.frameId === 0 && result.started);
}
sourceFor(
requestId: string,
producerId: string,
sender: chrome.runtime.MessageSender,
): RecordingCaptureScope | null {
const address = senderAddress(sender);
const recording = this.#recordings.get(requestId);
if (!address || !recording) return null;
const agent = recording.agents.get(documentKey(address.tabId, address.documentId));
if (!agent || agent.producerId !== producerId) return null;
return {
tabId: agent.tabId,
documentId: agent.documentId,
browserFrameId: agent.browserFrameId,
producerId: agent.producerId,
};
}
async stop(requestId: string): Promise<boolean> {
const recording = this.#recordings.get(requestId);
if (!recording) return true;
recording.finishing = true;
const commandId = crypto.randomUUID();
const results = await Promise.all(
[...recording.agents.values()].map((agent) => this.#stopAgent(agent, requestId, commandId)),
);
const succeeded = results.every(Boolean);
if (succeeded) this.#recordings.delete(requestId);
else recording.finishing = false;
return succeeded;
}
cancel(requestId: string): void {
const recording = this.#recordings.get(requestId);
if (!recording) return;
this.#cancelAgents(recording);
this.#recordings.delete(requestId);
}
readonly #onMessage = (
message: unknown,
sender: chrome.runtime.MessageSender,
sendResponse: (response: RecordFrameQueryResponse) => void,
): boolean => {
if (!isRecordFrameQueryMessage(message)) return false;
const tabId = sender.tab?.id;
const recording =
typeof tabId === "number"
? [...this.#recordings.values()].find(
(candidate) => !candidate.finishing && candidate.tabIds.has(tabId),
)
: undefined;
sendResponse(
recording
? {
active: true,
requestId: recording.requestId,
startedAtMs: recording.startedAtMs,
}
: { active: false },
);
return false;
};
readonly #onFrameNavigation = (frame: BrowserFrameNavigation): void => {
if (frame.frameId === 0) return;
for (const recording of this.#recordings.values()) {
if (recording.finishing || !recording.tabIds.has(frame.tabId)) continue;
// onCommitted can race document_start, while onCompleted runs after the
// content script is available. Starting twice is safe and closes both
// the early-query and late-SPA-frame gaps.
void (async () => {
const started = await this.#startFrame(recording, frame.tabId, frame);
if (!started || frame.lifecycle !== "completed" || recording.finishing) return;
const agent = frame.documentId
? recording.agents.get(documentKey(frame.tabId, frame.documentId))
: [...recording.agents.values()].find(
(candidate) =>
candidate.tabId === frame.tabId && candidate.browserFrameId === frame.frameId,
);
if (!agent) return;
await recording.onDocumentReady?.({
tabId: agent.tabId,
documentId: agent.documentId,
browserFrameId: agent.browserFrameId,
producerId: agent.producerId,
});
})();
}
};
readonly #onConnect = (port: chrome.runtime.Port): void => {
if (port.name !== RECORD_FRAME_PORT) return;
const address = senderAddress(port.sender ?? {});
if (!address) {
port.disconnect();
return;
}
let registered: FrameAgent | null = null;
port.onMessage.addListener((raw: unknown) => {
const message = raw as Partial<RecordFramePortMessage>;
if (message.type === "ready") {
if (typeof message.requestId !== "string" || typeof message.producerId !== "string") {
port.disconnect();
return;
}
const requestId = message.requestId;
const producerId = message.producerId;
if (registered) {
if (registered.requestId === requestId && registered.producerId === producerId) {
port.postMessage({
type: "ready_ack",
requestId,
producerId,
} satisfies RecordFramePortMessage);
} else {
port.disconnect();
}
return;
}
const recording = this.#recordings.get(requestId);
if (
!recording ||
recording.finishing ||
!recording.tabIds.has(address.tabId) ||
producerId.length === 0
) {
port.disconnect();
return;
}
const agent: FrameAgent = {
...address,
requestId,
producerId,
port,
stopWaiters: new Map(),
};
registered = agent;
const key = documentKey(address.tabId, address.documentId);
const previous = recording.agents.get(key);
if (previous && previous !== agent) previous.port.disconnect();
recording.agents.set(key, agent);
port.postMessage({
type: "ready_ack",
requestId,
producerId,
} satisfies RecordFramePortMessage);
return;
}
if (
message.type === "stopped" &&
registered &&
message.requestId === registered.requestId &&
typeof message.commandId === "string"
) {
registered.stopWaiters.get(message.commandId)?.(message.ok === true);
registered.stopWaiters.delete(message.commandId);
}
});
port.onDisconnect.addListener(() => {
if (!registered) return;
for (const resolve of registered.stopWaiters.values()) resolve(true);
const recording = this.#recordings.get(registered.requestId);
const key = documentKey(registered.tabId, registered.documentId);
if (recording?.agents.get(key) === registered) recording.agents.delete(key);
});
};
#stopAgent(agent: FrameAgent, requestId: string, commandId: string): Promise<boolean> {
return new Promise((resolve) => {
const timer = setTimeout(() => {
agent.stopWaiters.delete(commandId);
resolve(false);
}, RECORD_FRAME_STOP_TIMEOUT_MS);
agent.stopWaiters.set(commandId, (ok) => {
clearTimeout(timer);
resolve(ok);
});
try {
agent.port.postMessage({
type: "stop",
requestId,
commandId,
} satisfies RecordFramePortMessage);
} catch {
clearTimeout(timer);
agent.stopWaiters.delete(commandId);
resolve(true);
}
});
}
async #startFrame(
recording: ArmedRecording,
tabId: number,
frame: BrowserFrame,
): Promise<boolean> {
const message: RecordFrameStartMessage = {
type: RECORD_FRAME_START,
requestId: recording.requestId,
startedAtMs: recording.startedAtMs,
};
try {
const response = await this.#deps.sendToDocument(tabId, message, {
...(frame.documentId ? { documentId: frame.documentId } : { frameId: frame.frameId }),
});
return isStarted(response);
} catch {
return false;
}
}
#cancelAgents(recording: ArmedRecording): void {
for (const agent of recording.agents.values()) {
try {
agent.port.postMessage({
type: "cancel",
requestId: recording.requestId,
} satisfies RecordFramePortMessage);
} catch {
// A destroyed Document has no remaining capture state to cancel.
}
}
}
}
function isStarted(value: unknown): boolean {
return typeof value === "object" && value !== null && (value as { ok?: unknown }).ok === true;
}
export const recordFrameCoordinator = new RecordFrameCoordinator();
@@ -1,5 +0,0 @@
const REDIRECT_QUALIFIERS = new Set(["client_redirect", "server_redirect"]);
export function hasRedirectQualifier(qualifiers?: readonly string[]): boolean {
return (qualifiers ?? []).some((qualifier) => REDIRECT_QUALIFIERS.has(qualifier));
}
@@ -1,140 +0,0 @@
import type { RenderedRef } from "@browser-skill/vom";
import type { CdpTarget } from "@/browser-driver/frame-graph";
import {
type CaptureVomMatchNode,
type CaptureVomObservationResult,
captureVomObservation,
} from "@/tools/capture-vom-observation";
import type { CdpRunner, ChromeTabsApi } from "@/tools/shared";
export interface IndexedObservationNode {
frameId: string;
geometry: CaptureVomMatchNode;
ref?: RenderedRef;
}
export interface CapturedRecordingObservation {
rootFrameId: string;
index: ObservationNodeIndex;
url: string;
title?: string;
vomText: string;
truncated: boolean;
}
export interface RegisteredObservation {
stateId: string;
rootFrameId: string;
index: ObservationNodeIndex;
url: string;
}
export interface RecordingDocumentScope {
frameId: string;
target: CdpTarget;
}
function nodeKey(frameId: string, backendNodeId: number): string {
return `${frameId}:${backendNodeId}`;
}
function frameTagKey(frameId: string, tag: string): string {
return `${frameId}:${tag.toLowerCase()}`;
}
export class ObservationNodeIndex {
readonly #nodesByFrameTag = new Map<string, IndexedObservationNode[]>();
readonly #refById = new Map<string, RenderedRef>();
readonly #refsByFrame = new Map<string, RenderedRef[]>();
readonly #scopeByProducer = new Map<string, RecordingDocumentScope | null>();
constructor(
input: Pick<CaptureVomObservationResult, "rootFrameId" | "matchNodes" | "refs"> &
Partial<Pick<CaptureVomObservationResult, "frames">>,
) {
const refByNode = new Map<string, RenderedRef>();
for (const ref of input.refs) {
const frameId = ref.frameId ?? input.rootFrameId;
refByNode.set(nodeKey(frameId, ref.backendNodeId), ref);
this.#refById.set(ref.ref, ref);
const frameRefs = this.#refsByFrame.get(frameId) ?? [];
frameRefs.push(ref);
this.#refsByFrame.set(frameId, frameRefs);
}
for (const frame of input.frames ?? []) {
const documentId = frame.recordingDocumentId;
if (!documentId) continue;
this.#scopeByProducer.set(
documentId,
this.#scopeByProducer.has(documentId)
? null
: { frameId: frame.frameId, target: frame.target },
);
}
for (const geometry of input.matchNodes) {
const { frameId } = geometry;
const entry = {
frameId,
geometry,
ref: refByNode.get(nodeKey(frameId, geometry.backendNodeId)),
};
const key = frameTagKey(frameId, geometry.tag);
const bucket = this.#nodesByFrameTag.get(key) ?? [];
bucket.push(entry);
this.#nodesByFrameTag.set(key, bucket);
}
}
candidates(frameId: string, tag: string): readonly IndexedObservationNode[] {
return this.#nodesByFrameTag.get(frameTagKey(frameId, tag)) ?? [];
}
ref(refId: string): RenderedRef | undefined {
return this.#refById.get(refId);
}
refs(frameId: string): readonly RenderedRef[] {
return this.#refsByFrame.get(frameId) ?? [];
}
documentScope(producerId: string): RecordingDocumentScope | undefined {
return this.#scopeByProducer.get(producerId) ?? undefined;
}
}
async function readTabMeta(
tabsApi: ChromeTabsApi,
tabId: number,
): Promise<{ url: string; title?: string }> {
try {
const tab = await tabsApi.get(tabId);
return { url: tab.url ?? "about:blank", title: tab.title };
} catch {
return { url: "about:blank" };
}
}
export async function captureRecordingObservation(input: {
cdp: CdpRunner;
tabsApi: ChromeTabsApi;
tabId: number;
maxTokens: number;
redactValues: boolean;
signal?: AbortSignal;
}): Promise<CapturedRecordingObservation> {
const { url, title } = await readTabMeta(input.tabsApi, input.tabId);
const captured = await captureVomObservation(input.cdp, input.tabId, url, {
maxTokens: input.maxTokens,
redactValues: input.redactValues,
conditionalSurfaceProbe: false,
signal: input.signal,
});
return {
rootFrameId: captured.rootFrameId,
index: new ObservationNodeIndex(captured),
url,
title,
vomText: captured.text,
truncated: captured.truncated,
};
}
@@ -1,128 +0,0 @@
import type { CdpRunner, ChromeTabsApi } from "@/tools/shared";
import { captureRecordingObservation, type RegisteredObservation } from "./observation-capture";
import { RecordingStateRegistry } from "./state-registry";
import { matchObservationTarget, unmatchedTarget } from "./target-matcher";
import type { RecordingDraftStep, StepAnnotation, TargetedRecordingDraft } from "./types";
const DEFAULT_MAX_PAGE_TOKENS = 3_000;
const MIN_CAPTURE_INTERVAL_MS = 200;
export interface TabObservationCursor {
lastSettled: RegisteredObservation | null;
lastCaptureAt: number;
}
function abortableDelay(ms: number, signal?: AbortSignal): Promise<void> {
return new Promise((resolve) => {
if (signal?.aborted) return resolve();
const timer = setTimeout(resolve, ms);
signal?.addEventListener(
"abort",
() => {
clearTimeout(timer);
resolve();
},
{ once: true },
);
});
}
function isTargeted(draft: RecordingDraftStep): draft is TargetedRecordingDraft {
return (
draft.op === "click" ||
draft.op === "hover" ||
draft.op === "fill" ||
draft.op === "press" ||
draft.op === "select"
);
}
export class RecordingObservationSession {
readonly registry: RecordingStateRegistry;
readonly cursor: TabObservationCursor;
readonly annotations: StepAnnotation[];
readonly #maxTokens: number;
readonly #redactValues: boolean;
constructor(
options: {
registry?: RecordingStateRegistry;
cursor?: TabObservationCursor;
annotations?: StepAnnotation[];
maxTokens?: number;
redactValues?: boolean;
} = {},
) {
this.registry = options.registry ?? new RecordingStateRegistry();
this.cursor = options.cursor ?? { lastSettled: null, lastCaptureAt: 0 };
this.annotations = options.annotations ?? [];
this.#maxTokens = options.maxTokens ?? DEFAULT_MAX_PAGE_TOKENS;
this.#redactValues = options.redactValues ?? false;
}
async capture(
cdp: CdpRunner,
tabsApi: ChromeTabsApi,
tabId: number,
signal?: AbortSignal,
): Promise<RegisteredObservation> {
const waitMs = Math.max(0, MIN_CAPTURE_INTERVAL_MS - (Date.now() - this.cursor.lastCaptureAt));
if (waitMs > 0) await abortableDelay(waitMs, signal);
if (signal?.aborted) throw new DOMException("observation aborted", "AbortError");
const captured = await captureRecordingObservation({
cdp,
tabsApi,
tabId,
maxTokens: this.#maxTokens,
redactValues: this.#redactValues,
signal,
});
const state = this.registry.register({
url: captured.url,
title: captured.title,
vomText: captured.vomText,
truncated: captured.truncated,
});
const observation: RegisteredObservation = {
stateId: state.id,
rootFrameId: captured.rootFrameId,
index: captured.index,
url: captured.url,
};
this.cursor.lastSettled = observation;
this.cursor.lastCaptureAt = Date.now();
return observation;
}
bindDraft(draft: RecordingDraftStep, draftId: number, previousActionPending = false): void {
const observation = this.cursor.lastSettled;
if (isTargeted(draft)) {
draft.matchedTarget = observation
? matchObservationTarget({
observation,
hint: draft.targetHint,
fallback: draft.captureTarget,
})
: unmatchedTarget(draft.captureTarget);
}
if (!observation) return;
const unmatched = isTargeted(draft) && draft.matchedTarget?.unmatched === true;
if (previousActionPending && unmatched) return;
draft.preStateId = observation.stateId;
this.registry.markStep(observation.stateId, draftId);
if (!isTargeted(draft) || !draft.matchedTarget?.ref) return;
const ref = observation.index.ref(draft.matchedTarget.ref);
if (!ref) return;
this.annotations.push({
draftId,
op: draft.op,
line: ref.line,
stateId: observation.stateId,
...(draft.op === "fill" && !this.#redactValues
? { detail: JSON.stringify(draft.value) }
: {}),
});
}
}
@@ -1,277 +0,0 @@
import type { CdpRunner, ChromeTabsApi } from "@/tools/shared";
import type { StopReason, TraceV3 } from "@/transport/types";
import { type DocumentSettleScope, waitForDocumentSettled } from "./document-settle";
import type { RegisteredObservation } from "./observation-capture";
import { RecordingObservationSession } from "./observation-session";
import { inferMissingPostStates, SettleController } from "./settle-controller";
import { RecordingStateRegistry } from "./state-registry";
import { buildTraceV3 } from "./trace-builder-v3";
import type { RecordingDraftStep, StepAnnotation } from "./types";
interface TabRecordingContext {
session: RecordingObservationSession;
settle: SettleController;
pendingCapture: PendingCapture | null;
frameRefresh: Promise<void>;
frameRefreshAbort: AbortController | null;
}
interface PendingCapture {
promise: Promise<RegisteredObservation>;
abort: AbortController;
}
export class RecordingObservationRuntime {
readonly #cdp: CdpRunner;
readonly #tabsApi: ChromeTabsApi;
readonly #registry = new RecordingStateRegistry();
readonly #annotations: StepAnnotation[] = [];
readonly #contexts = new Map<number, TabRecordingContext>();
readonly #maxTokens?: number;
readonly #redactValues: boolean;
constructor(input: {
cdp: CdpRunner;
tabsApi: ChromeTabsApi;
maxTokens?: number;
redactValues?: boolean;
}) {
this.#cdp = input.cdp;
this.#tabsApi = input.tabsApi;
this.#maxTokens = input.maxTokens;
this.#redactValues = input.redactValues ?? false;
}
#context(tabId: number): TabRecordingContext {
const existing = this.#contexts.get(tabId);
if (existing) return existing;
const session = new RecordingObservationSession({
registry: this.#registry,
annotations: this.#annotations,
maxTokens: this.#maxTokens,
redactValues: this.#redactValues,
});
const context: TabRecordingContext = {
session,
settle: new SettleController({
session,
cdp: this.#cdp,
tabsApi: this.#tabsApi,
tabId,
}),
pendingCapture: null,
frameRefresh: Promise.resolve(),
frameRefreshAbort: null,
};
this.#contexts.set(tabId, context);
return context;
}
async #capture(tabId: number): Promise<RegisteredObservation> {
const context = this.#context(tabId);
if (context.pendingCapture) return context.pendingCapture.promise;
const abort = new AbortController();
let pending!: PendingCapture;
const promise = context.session.capture(this.#cdp, this.#tabsApi, tabId, abort.signal);
pending = {
abort,
promise: promise.finally(() => {
if (context.pendingCapture === pending) context.pendingCapture = null;
}),
};
context.pendingCapture = pending;
return pending.promise;
}
async captureInitial(tabId: number): Promise<void> {
const context = this.#context(tabId);
if (context.session.cursor.lastSettled) return;
await this.#capture(tabId);
}
/** Refresh the safe observation after a late child Document becomes usable. */
async refreshDocument(tabId: number, producerId: string): Promise<void> {
const context = this.#context(tabId);
context.frameRefreshAbort?.abort();
const abort = new AbortController();
context.frameRefreshAbort = abort;
const previous = context.frameRefresh;
const refresh = (async () => {
await previous;
if (abort.signal.aborted) return;
if (context.pendingCapture) await Promise.allSettled([context.pendingCapture.promise]);
if (abort.signal.aborted) return;
// The first capture discovers the safe producer -> CDP Document scope.
await this.#capture(tabId);
if (abort.signal.aborted) return;
const scope = context.session.cursor.lastSettled?.index.documentScope(producerId);
if (scope) {
await waitForDocumentSettled(this.#cdp, scope, { signal: abort.signal });
}
if (abort.signal.aborted) return;
// Capture again after the child SPA is quiet so the next action can be
// matched against refs that actually exist in its pre-action state.
await this.#capture(tabId);
})();
const tracked = refresh.catch(() => {});
context.frameRefresh = tracked;
try {
await refresh;
} finally {
if (context.frameRefresh === tracked) context.frameRefreshAbort = null;
}
}
async captureTabTransition(
fromTabId: number,
toTabId: number,
): Promise<{ preStateId?: string; postStateId?: string; targetUrl?: string }> {
const from = this.#context(fromTabId);
await this.#flushContext(from);
if (!from.session.cursor.lastSettled) {
try {
await this.captureInitial(fromTabId);
} catch {
// A target-tab observation can still preserve the forward transition.
}
}
const to = this.#context(toTabId);
let target = to.session.cursor.lastSettled;
try {
target = await this.#capture(toTabId);
} catch {
// Reuse the last settled target state when a refresh races navigation.
}
const source = from.session.cursor.lastSettled;
return {
...(source ? { preStateId: source.stateId } : {}),
...(target ? { postStateId: target.stateId, targetUrl: target.url } : {}),
};
}
bindTabTransition(
draft: Extract<RecordingDraftStep, { op: "switch_tab" }>,
draftId: number,
): void {
if (draft.preStateId) this.#registry.markStep(draft.preStateId, draftId);
}
async processDraft(
tabId: number,
drafts: RecordingDraftStep[],
draftIndex: number,
producerId?: string,
): Promise<void> {
const draft = drafts[draftIndex];
if (!draft) return;
const context = this.#context(tabId);
if (!context.session.cursor.lastSettled) {
try {
await this.captureInitial(tabId);
} catch {
// Post-action settle can still provide a usable state.
}
}
const scope = producerId
? context.session.cursor.lastSettled?.index.documentScope(producerId)
: undefined;
if (
producerId &&
(draft.op === "click" ||
draft.op === "hover" ||
draft.op === "fill" ||
draft.op === "press" ||
draft.op === "select")
) {
draft.targetHint = {
...(draft.targetHint ?? {}),
frameId: scope?.frameId ?? null,
geometrySpace: "local",
};
}
context.session.bindDraft(draft, draftIndex + 1, context.settle.hasPending);
context.settle.schedule(drafts, draftIndex, scope);
}
scheduleSettle(
tabId: number,
drafts: RecordingDraftStep[],
draftIndex: number,
scope?: DocumentSettleScope,
): void {
this.#context(tabId).settle.schedule(drafts, draftIndex, scope);
}
clearRedirect(tabId: number): void {
this.#contexts.get(tabId)?.settle.clearRedirect();
}
scheduleRedirect(tabId: number, drafts: RecordingDraftStep[], url: string): void {
this.#context(tabId).settle.scheduleRedirect(drafts, url);
}
async #flushContext(context: TabRecordingContext): Promise<void> {
await context.frameRefresh;
if (context.pendingCapture) await Promise.allSettled([context.pendingCapture.promise]);
await context.settle.flushRedirects();
await context.settle.flush();
}
async flushRedirects(tabId: number): Promise<void> {
await this.#contexts.get(tabId)?.settle.flushRedirects();
}
async flush(): Promise<void> {
for (const context of this.#contexts.values()) await this.#flushContext(context);
}
async settleTrailing(tabId: number, drafts: RecordingDraftStep[]): Promise<void> {
const context = this.#context(tabId);
if (context.pendingCapture) await Promise.allSettled([context.pendingCapture.promise]);
if (!context.session.cursor.lastSettled) {
try {
await this.captureInitial(tabId);
} catch {
// The trace may remain observation-free when CDP is unavailable.
}
}
await context.settle.settleTrailing(drafts);
inferMissingPostStates(drafts);
}
cancel(): void {
for (const context of this.#contexts.values()) {
context.frameRefreshAbort?.abort();
context.pendingCapture?.abort.abort();
context.settle.cancel();
}
}
buildTrace(input: {
drafts: RecordingDraftStep[];
startedAt: string;
purpose?: string;
startUrl?: string;
stoppedBy: StopReason;
bskVersion: string;
includeTabSwitches?: boolean;
}): TraceV3 {
return buildTraceV3({
registry: this.#registry,
drafts: input.drafts,
annotations: this.#annotations,
startedAt: input.startedAt,
purpose: input.purpose,
startUrl: input.startUrl,
stoppedBy: input.stoppedBy,
bskVersion: input.bskVersion,
redactValues: this.#redactValues,
includeTabSwitches: input.includeTabSwitches,
});
}
}
@@ -1,233 +0,0 @@
import type { CdpRunner, ChromeTabsApi } from "@/tools/shared";
import { type DocumentSettleScope, waitForDocumentSettled } from "./document-settle";
import { RecordingObservationSession } from "./observation-session";
import type { RecordingDraftStep } from "./types";
interface PendingSettle {
abort: AbortController;
scope: DocumentSettleScope;
}
interface PendingRedirect {
url: string;
abort: AbortController;
}
const CAPTURE_RETRY_DELAY_MS = 250;
function delay(ms: number, signal?: AbortSignal): Promise<void> {
return new Promise((resolve) => {
if (signal?.aborted) return resolve();
const timer = setTimeout(resolve, ms);
signal?.addEventListener(
"abort",
() => {
clearTimeout(timer);
resolve();
},
{ once: true },
);
});
}
function isAbortError(error: unknown): boolean {
return (
typeof error === "object" &&
error !== null &&
(error as { name?: string }).name === "AbortError"
);
}
export function inferMissingPostStates(drafts: RecordingDraftStep[]): void {
for (let index = 0; index < drafts.length - 1; index += 1) {
const draft = drafts[index];
const next = drafts[index + 1];
if (draft && next && !draft.postStateId && next.preStateId) {
draft.postStateId = next.preStateId;
}
}
}
export class SettleController {
readonly #session: RecordingObservationSession;
readonly #cdp: CdpRunner;
readonly #tabsApi: ChromeTabsApi;
readonly #tabId: number;
readonly #rootScope: DocumentSettleScope;
readonly #pending = new Map<number, PendingSettle>();
#queue = Promise.resolve();
#pendingRedirect: PendingRedirect | null = null;
#redirectQueue = Promise.resolve();
constructor(input: {
session: RecordingObservationSession;
cdp: CdpRunner;
tabsApi: ChromeTabsApi;
tabId: number;
}) {
this.#session = input.session;
this.#cdp = input.cdp;
this.#tabsApi = input.tabsApi;
this.#tabId = input.tabId;
this.#rootScope = { target: { tabId: input.tabId } };
}
get hasPending(): boolean {
return this.#pending.size > 0 || this.#pendingRedirect !== null;
}
async #captureWithRetry(signal?: AbortSignal) {
try {
return await this.#session.capture(this.#cdp, this.#tabsApi, this.#tabId, signal);
} catch (error) {
if (signal?.aborted) throw error;
await delay(CAPTURE_RETRY_DELAY_MS, signal);
return this.#session.capture(this.#cdp, this.#tabsApi, this.#tabId, signal);
}
}
schedule(
drafts: RecordingDraftStep[],
draftIndex: number,
scope: DocumentSettleScope = this.#rootScope,
): void {
if (scope.target.tabId !== this.#tabId) {
throw new Error(
`settle scope tab ${scope.target.tabId} does not belong to tab ${this.#tabId}`,
);
}
const landing = drafts[draftIndex]?.preStateId;
for (const [index, pending] of this.#pending) {
if (index >= draftIndex) continue;
pending.abort.abort();
this.#pending.delete(index);
if (landing && drafts[index] && !drafts[index].postStateId) {
drafts[index].postStateId = landing;
}
}
this.#pending.get(draftIndex)?.abort.abort();
const pending = { abort: new AbortController(), scope };
this.#pending.set(draftIndex, pending);
const task = async () => {
if (pending.abort.signal.aborted) return;
try {
const outcome = await waitForDocumentSettled(this.#cdp, pending.scope, {
signal: pending.abort.signal,
});
if (outcome === "cancelled") return;
const observation = await this.#captureWithRetry(pending.abort.signal);
if (!pending.abort.signal.aborted && drafts[draftIndex]) {
drafts[draftIndex].postStateId = observation.stateId;
}
} catch (error) {
if (!isAbortError(error)) {
console.warn(
`[bsk record] post-action observation failed for step ${draftIndex + 1}`,
error,
);
}
} finally {
if (this.#pending.get(draftIndex) === pending) this.#pending.delete(draftIndex);
}
};
this.#queue = this.#queue.then(task, task).catch(() => {});
}
cancel(): void {
for (const pending of this.#pending.values()) pending.abort.abort();
this.#pending.clear();
this.clearRedirect();
}
async flush(): Promise<void> {
for (let round = 0; round < 10; round += 1) {
const current = this.#queue;
await current;
if (current === this.#queue) return;
}
console.warn("[bsk record] settle queue kept growing; using completed observations");
}
async settleTrailing(drafts: RecordingDraftStep[]): Promise<void> {
const trailing: RecordingDraftStep[] = [];
for (let index = drafts.length - 1; index >= 0; index -= 1) {
const draft = drafts[index];
if (!draft || draft.postStateId) break;
trailing.push(draft);
}
if (trailing.length === 0) return;
try {
const observation = await this.#captureWithRetry();
for (const draft of trailing) draft.postStateId = observation.stateId;
} catch (error) {
console.warn("[bsk record] final observation at stop failed", error);
}
}
clearRedirect(): void {
this.#pendingRedirect?.abort.abort();
this.#pendingRedirect = null;
}
scheduleRedirect(drafts: RecordingDraftStep[], url: string): void {
this.#pendingRedirect?.abort.abort();
const pending: PendingRedirect = { url, abort: new AbortController() };
this.#pendingRedirect = pending;
const task = () => this.#settleRedirect(drafts, pending);
this.#redirectQueue = this.#redirectQueue.then(task, task).catch(() => {});
}
async #settleRedirect(drafts: RecordingDraftStep[], pending: PendingRedirect): Promise<void> {
const outcome = await waitForDocumentSettled(this.#cdp, this.#rootScope, {
signal: pending.abort.signal,
});
if (outcome === "cancelled" || this.#pendingRedirect !== pending) return;
let finalUrl = pending.url;
try {
finalUrl = (await this.#tabsApi.get(this.#tabId)).url || finalUrl;
} catch {
// The navigation event URL remains the best available destination.
}
if (this.#pendingRedirect !== pending) return;
this.#pendingRedirect = null;
if (
!finalUrl ||
finalUrl === "about:blank" ||
this.#session.cursor.lastSettled?.url === finalUrl
) {
return;
}
const last = drafts[drafts.length - 1];
if (last?.op === "navigate" && last.url === finalUrl) {
if (!last.postStateId) this.schedule(drafts, drafts.length - 1);
await this.flush();
return;
}
const draft: RecordingDraftStep = {
op: "navigate",
url: finalUrl,
pageUrl: finalUrl,
cause: "browser",
transitionQualifiers: ["server_redirect"],
preStateId: this.#session.cursor.lastSettled?.stateId,
};
drafts.push(draft);
const draftIndex = drafts.length - 1;
if (draft.preStateId) this.#session.registry.markStep(draft.preStateId, draftIndex + 1);
this.schedule(drafts, draftIndex);
await this.flush();
}
async flushRedirects(): Promise<void> {
for (let round = 0; round < 10; round += 1) {
const current = this.#redirectQueue;
await current;
if (current === this.#redirectQueue) return;
}
console.warn("[bsk record] redirect queue kept growing; using the final completed landing");
}
}
@@ -1,57 +0,0 @@
export interface RecordedStateEntry {
id: string;
url: string;
title?: string;
/** Rendered VOM text; capture indexes and source DOM/AX data never enter the registry. */
vomText: string;
truncated: boolean;
stepsHere: number[];
}
function stateIdentity(url: string, body: string): string {
return `${url}\0${body}`;
}
export class RecordingStateRegistry {
readonly #entriesById = new Map<string, RecordedStateEntry>();
readonly #idByIdentity = new Map<string, string>();
#nextId = 1;
register(input: {
url: string;
title?: string;
vomText: string;
truncated?: boolean;
}): RecordedStateEntry {
const identity = stateIdentity(input.url, input.vomText);
const existingId = this.#idByIdentity.get(identity);
if (existingId) {
const existing = this.#entriesById.get(existingId)!;
if (!existing.title && input.title) existing.title = input.title;
if (input.truncated) existing.truncated = true;
return existing;
}
const entry: RecordedStateEntry = {
id: `s${this.#nextId}`,
url: input.url,
...(input.title ? { title: input.title } : {}),
vomText: input.vomText,
truncated: input.truncated ?? false,
stepsHere: [],
};
this.#nextId += 1;
this.#entriesById.set(entry.id, entry);
this.#idByIdentity.set(identity, entry.id);
return entry;
}
values(): RecordedStateEntry[] {
return [...this.#entriesById.values()];
}
markStep(stateId: string, draftId: number): void {
const entry = this.#entriesById.get(stateId);
if (entry && !entry.stepsHere.includes(draftId)) entry.stepsHere.push(draftId);
}
}
@@ -1,155 +0,0 @@
import type { RecordStepPayload } from "../record-bridge";
import { hasRedirectQualifier } from "./navigation-policy";
import type { RecordingDraftStep, TargetMatchHint } from "./types";
export interface RecordingStepBuffer {
steps: RecordingDraftStep[];
navigation: RecordingNavigationCursor;
}
export interface RecordingNavigationCursor {
currentUrl?: string;
pendingNavigation: boolean;
pendingNavigationDeadline?: number;
}
const NAVIGATION_TRIGGER_WINDOW_MS = 3_000;
function toDraftStep(
payload: RecordStepPayload,
targetHint?: TargetMatchHint,
): RecordingDraftStep | null {
const pageUrl = payload.page_url;
const common = {
...(pageUrl ? { pageUrl } : {}),
...(targetHint ? { targetHint } : {}),
};
switch (payload.op) {
case "click":
return payload.target ? { op: "click", captureTarget: payload.target, ...common } : null;
case "hover":
return payload.target ? { op: "hover", captureTarget: payload.target, ...common } : null;
case "fill":
return payload.target
? {
op: "fill",
captureTarget: payload.target,
value: payload.value ?? "",
...(payload.commit ? { commit: payload.commit } : {}),
...(payload.redacted ? { redacted: true } : {}),
...common,
}
: null;
case "press":
return payload.key
? {
op: "press",
key: payload.key,
...(payload.target ? { captureTarget: payload.target } : {}),
...(payload.modifiers?.length ? { modifiers: payload.modifiers } : {}),
...common,
}
: null;
case "select":
return payload.target && payload.values
? {
op: "select",
captureTarget: payload.target,
values: payload.values,
...(payload.labels?.length ? { labels: payload.labels } : {}),
...common,
}
: null;
case "navigate":
return null;
}
}
function annotateLastStepNavigation(buffer: RecordingStepBuffer, url: string): number {
for (let index = buffer.steps.length - 1; index >= 0; index -= 1) {
const step = buffer.steps[index];
if (!step) continue;
if (step.op === "click" || step.op === "press" || step.op === "select" || step.op === "fill") {
step.navigatedTo = url;
return index;
}
break;
}
return -1;
}
export type NavigationObserveResult =
| { kind: "noop" }
| { kind: "annotated"; index: number }
| { kind: "appended"; index: number }
| { kind: "coalesce_redirect"; url: string };
export function observeRecordedNavigation(
buffer: RecordingStepBuffer,
url: string,
causedByAction?: boolean,
transitionType?: string,
transitionQualifiers?: string[],
): NavigationObserveResult {
const navigation = buffer.navigation;
if (!url || url === navigation.currentUrl) return { kind: "noop" };
navigation.currentUrl = url;
const pendingIsCurrent =
navigation.pendingNavigation &&
(navigation.pendingNavigationDeadline === undefined ||
navigation.pendingNavigationDeadline >= Date.now());
if (causedByAction === true || (causedByAction === undefined && pendingIsCurrent)) {
navigation.pendingNavigation = false;
navigation.pendingNavigationDeadline = undefined;
const annotatedIndex = annotateLastStepNavigation(buffer, url);
if (annotatedIndex >= 0) return { kind: "annotated", index: annotatedIndex };
} else {
navigation.pendingNavigation = false;
navigation.pendingNavigationDeadline = undefined;
}
if (hasRedirectQualifier(transitionQualifiers)) {
return { kind: "coalesce_redirect", url };
}
buffer.steps.push({
op: "navigate",
url,
pageUrl: url,
transitionType,
transitionQualifiers,
});
return { kind: "appended", index: buffer.steps.length - 1 };
}
export function appendRecordedPayload(
buffer: RecordingStepBuffer,
payload: RecordStepPayload,
targetHint?: TargetMatchHint,
): number | null {
if (payload.op === "navigate") {
if (!payload.url) return null;
const result = observeRecordedNavigation(
buffer,
payload.url,
payload.navigation_caused_by_action,
payload.transitionType,
payload.transitionQualifiers,
);
return result.kind === "appended" ? result.index : null;
}
const step = toDraftStep(
{ ...payload, page_url: payload.page_url ?? buffer.navigation.currentUrl },
targetHint,
);
if (!step) return null;
buffer.steps.push(step);
if (step.op === "click" || step.op === "press" || step.op === "select" || step.op === "fill") {
buffer.navigation.pendingNavigation = payload.expects_navigation === true;
buffer.navigation.pendingNavigationDeadline = buffer.navigation.pendingNavigation
? Date.now() + NAVIGATION_TRIGGER_WINDOW_MS
: undefined;
}
return buffer.steps.length - 1;
}
@@ -1,61 +0,0 @@
import type { RecordingNavigationCursor } from "./step-buffer";
export interface TabActivation {
tabId: number;
revision: number;
}
export class RecordingTabCoordinator {
readonly #navigationByTab = new Map<number, RecordingNavigationCursor>();
#activeTabId: number;
#currentTabId: number;
#activationRevision = 0;
constructor(initialTabId: number, initialUrl?: string) {
this.#activeTabId = initialTabId;
this.#currentTabId = initialTabId;
this.#navigationByTab.set(initialTabId, {
currentUrl: initialUrl,
pendingNavigation: false,
});
}
get activeTabId(): number {
return this.#activeTabId;
}
get currentTabId(): number {
return this.#currentTabId;
}
noteActivation(tabId: number): TabActivation {
this.#activeTabId = tabId;
this.#activationRevision += 1;
return { tabId, revision: this.#activationRevision };
}
isLatest(activation: TabActivation): boolean {
return (
activation.tabId === this.#activeTabId && activation.revision === this.#activationRevision
);
}
commit(tabId: number, currentUrl?: string): void {
if (currentUrl !== undefined) this.navigation(tabId).currentUrl = currentUrl;
this.#currentTabId = tabId;
}
navigation(tabId: number, fallbackUrl?: string): RecordingNavigationCursor {
const existing = this.#navigationByTab.get(tabId);
if (existing) {
if (!existing.currentUrl && fallbackUrl) existing.currentUrl = fallbackUrl;
return existing;
}
const created: RecordingNavigationCursor = {
currentUrl: fallbackUrl,
pendingNavigation: false,
};
this.#navigationByTab.set(tabId, created);
return created;
}
}
@@ -1,86 +0,0 @@
import type { RenderedRef } from "@browser-skill/vom";
import type { TargetDescriptorV3 } from "@/transport/types";
import type { CaptureTargetDescriptor } from "../describe-target";
import type { IndexedObservationNode, RegisteredObservation } from "./observation-capture";
import type { TargetGeometry, TargetMatchHint } from "./types";
const MATCH_TOLERANCE_PX = 2;
function close(a: number, b: number): boolean {
return Math.abs(a - b) <= MATCH_TOLERANCE_PX;
}
function rectMatches(a: TargetGeometry["rect"], b: TargetGeometry["rect"]): boolean {
return close(a.x, b.x) && close(a.y, b.y) && close(a.w, b.w) && close(a.h, b.h);
}
function candidateMatches(
candidate: IndexedObservationNode,
geometry: TargetGeometry,
geometrySpace: "top" | "local",
): boolean {
const rect = geometrySpace === "local" ? candidate.geometry.localRect : candidate.geometry.rect;
return rect != null && rectMatches(geometry.rect, rect);
}
export function unmatchedTarget(fallback?: CaptureTargetDescriptor): TargetDescriptorV3 {
return {
...(fallback?.role ? { role: fallback.role } : {}),
...(fallback?.name ? { name: fallback.name } : {}),
unmatched: true,
};
}
function normalized(value: string | undefined): string {
return value?.replace(/\s+/g, " ").trim().toLowerCase() ?? "";
}
function matchesSemantics(ref: RenderedRef, fallback?: CaptureTargetDescriptor): boolean {
const role = normalized(fallback?.role);
const name = normalized(fallback?.name);
if (!role && !name) return false;
if (role && normalized(ref.role) !== role) return false;
if (name && normalized(ref.name) !== name) return false;
return true;
}
function descriptor(ref: RenderedRef): TargetDescriptorV3 {
return {
ref: ref.ref,
...(ref.role ? { role: ref.role } : {}),
...(ref.name ? { name: ref.name } : {}),
...(ref.ctx ? { ctx: ref.ctx } : {}),
};
}
export function matchObservationTarget(input: {
observation: RegisteredObservation;
hint?: TargetMatchHint;
fallback?: CaptureTargetDescriptor;
}): TargetDescriptorV3 {
if (input.hint?.frameId === null) return unmatchedTarget(input.fallback);
const frameId = input.hint?.frameId ?? input.observation.rootFrameId;
const geometry = input.hint?.geometry;
if (geometry) {
const matches = input.observation.index
.candidates(frameId, geometry.tag)
.filter(
(candidate) =>
candidate.ref &&
candidateMatches(candidate, geometry, input.hint?.geometrySpace ?? "top"),
);
if (matches.length === 1) return descriptor(matches[0]!.ref!);
const semanticMatches = matches.filter(
(candidate) => candidate.ref && matchesSemantics(candidate.ref, input.fallback),
);
if (semanticMatches.length === 1) return descriptor(semanticMatches[0]!.ref!);
return unmatchedTarget(input.fallback);
}
const semanticMatches = input.observation.index
.refs(frameId)
.filter((ref) => matchesSemantics(ref, input.fallback));
return semanticMatches.length === 1
? descriptor(semanticMatches[0]!)
: unmatchedTarget(input.fallback);
}
@@ -1,92 +0,0 @@
import {
type StepV3,
type StopReason,
TRACE_VERSION_V3,
type TraceStateV3,
type TraceV3,
VOM_FORMAT_VERSION,
} from "@/transport/types";
import { resolveDraftStartUrl } from "./draft-policy";
import type { RecordedStateEntry, RecordingStateRegistry } from "./state-registry";
import { reduceTraceStepsV3 } from "./trace-reducer-v3";
import { formatTraceStateBody } from "./trace-state-body";
import type { RecordingDraftStep, StepAnnotation } from "./types";
function publishedEntries(registry: RecordingStateRegistry, steps: StepV3[]): RecordedStateEntry[] {
const entries = registry.values();
if (steps.length === 0) return entries.slice(0, 1);
const referenced = new Set(steps.flatMap((step) => [step.state, step.result.state]));
return entries.filter((entry) => referenced.has(entry.id));
}
function remapDraftIds(draftIds: number[], stepIdByDraftId: Map<number, number>): number[] {
return [
...new Set(
draftIds.flatMap((id) => {
const stepId = stepIdByDraftId.get(id);
return stepId === undefined ? [] : [stepId];
}),
),
].sort((a, b) => a - b);
}
export function buildTraceV3(input: {
registry: RecordingStateRegistry;
drafts: RecordingDraftStep[];
annotations?: StepAnnotation[];
startedAt: string;
purpose?: string;
startUrl?: string;
stoppedBy: StopReason;
bskVersion: string;
redactValues?: boolean;
includeTabSwitches?: boolean;
}): TraceV3 {
const reduced = reduceTraceStepsV3(input.drafts, {
includeTabSwitches: input.includeTabSwitches,
redactValues: input.redactValues,
});
const entries = publishedEntries(input.registry, reduced.steps);
const publishedId = new Map(entries.map((entry, index) => [entry.id, `s${index + 1}`]));
const annotationsByState = new Map<string, StepAnnotation[]>();
for (const annotation of input.annotations ?? []) {
const bucket = annotationsByState.get(annotation.stateId) ?? [];
bucket.push(annotation);
annotationsByState.set(annotation.stateId, bucket);
}
const steps = reduced.steps.map((step) => ({
...step,
state: publishedId.get(step.state) ?? step.state,
result: { state: publishedId.get(step.result.state) ?? step.result.state },
}));
const states: TraceStateV3[] = entries.map((entry) => {
const id = publishedId.get(entry.id) ?? entry.id;
return {
id,
url: entry.url,
...(entry.title ? { title: entry.title } : {}),
body: formatTraceStateBody({
stateId: id,
url: entry.url,
title: entry.title,
stepIds: remapDraftIds(entry.stepsHere, reduced.stepIdByDraftId),
vomText: entry.vomText,
annotations: annotationsByState.get(entry.id) ?? [],
stepIdByDraftId: reduced.stepIdByDraftId,
}),
...(entry.truncated ? { truncated: true } : {}),
};
});
return {
version: TRACE_VERSION_V3,
...(input.purpose ? { purpose: input.purpose } : {}),
recorded_at: new Date().toISOString(),
started_at: input.startedAt,
stopped_by: input.stoppedBy,
entry: { start_url: resolveDraftStartUrl(input.drafts, input.startUrl, states[0]?.url) },
recorder: { bsk: input.bskVersion, vom: VOM_FORMAT_VERSION },
states,
steps,
};
}
@@ -1,154 +0,0 @@
import type { NavigationCause, StepV3 } from "@/transport/types";
import { shouldIncludeDraft } from "./draft-policy";
import { hasRedirectQualifier } from "./navigation-policy";
import { unmatchedTarget } from "./target-matcher";
import type { RecordingDraftStep } from "./types";
interface CollapsedDraft {
draft: RecordingDraftStep;
draftIds: number[];
}
const TRANSITION_CAUSES: Record<string, NavigationCause> = {
typed: "user_typed",
generated: "user_typed",
keyword: "user_typed",
keyword_generated: "user_typed",
link: "link",
form_submit: "form_submit",
reload: "reload",
auto_bookmark: "browser",
start_page: "browser",
};
function isRedirect(step: Extract<RecordingDraftStep, { op: "navigate" }>): boolean {
return hasRedirectQualifier(step.transitionQualifiers);
}
function collapseRedirects(steps: RecordingDraftStep[]): CollapsedDraft[] {
const output: CollapsedDraft[] = [];
steps.forEach((step, index) => {
const previous = output[output.length - 1];
if (step.op === "navigate" && previous?.draft.op === "navigate" && isRedirect(step)) {
previous.draft = {
...previous.draft,
url: step.url,
postStateId: step.postStateId ?? previous.draft.postStateId,
};
previous.draftIds.push(index + 1);
return;
}
output.push({ draft: { ...step }, draftIds: [index + 1] });
});
return output;
}
function navigationCause(step: Extract<RecordingDraftStep, { op: "navigate" }>): NavigationCause {
if (step.cause) return step.cause;
const qualifiers = step.transitionQualifiers ?? [];
if (qualifiers.includes("forward_back")) return "history";
if (qualifiers.includes("from_address_bar")) return "user_typed";
return TRANSITION_CAUSES[step.transitionType ?? ""] ?? "browser";
}
function selection(values: string[], labels?: string[]): Array<{ value: string; label?: string }> {
return values.map((value, index) => ({
value,
...(labels?.[index] ? { label: labels[index] } : {}),
}));
}
interface ReduceDraftOptions {
includeTabSwitches: boolean;
redactValues: boolean;
}
function reduceDraft(
draft: RecordingDraftStep,
id: number,
options: ReduceDraftOptions,
): StepV3 | null {
if (!shouldIncludeDraft(draft)) return null;
if (draft.op === "switch_tab") {
if (!options.includeTabSwitches || !draft.preStateId || !draft.postStateId) return null;
return {
op: "switch_tab",
id,
state: draft.preStateId,
result: { state: draft.postStateId },
};
}
const state = draft.preStateId ?? draft.postStateId;
const resultState = draft.postStateId ?? draft.preStateId;
if (!state || !resultState) return null;
const common = { id, state, result: { state: resultState } };
switch (draft.op) {
case "navigate":
return { op: "navigate", ...common, to: draft.url, cause: navigationCause(draft) };
case "click":
return {
op: "click",
...common,
target: draft.matchedTarget ?? unmatchedTarget(draft.captureTarget),
};
case "hover":
return {
op: "hover",
...common,
target: draft.matchedTarget ?? unmatchedTarget(draft.captureTarget),
};
case "fill":
const fillIsRedacted = options.redactValues || draft.redacted === true;
return {
op: "fill",
...common,
target: draft.matchedTarget ?? unmatchedTarget(draft.captureTarget),
value: fillIsRedacted ? "***" : draft.value,
commit: draft.commit ?? "blur",
...(fillIsRedacted ? { redacted: true } : {}),
};
case "press":
return {
op: "press",
...common,
key: draft.key,
...(draft.captureTarget || draft.matchedTarget
? { target: draft.matchedTarget ?? unmatchedTarget(draft.captureTarget) }
: {}),
...(draft.modifiers?.length ? { modifiers: draft.modifiers } : {}),
};
case "select":
return {
op: "select",
...common,
target: draft.matchedTarget ?? unmatchedTarget(draft.captureTarget),
...(!options.redactValues ? { selection: selection(draft.values, draft.labels) } : {}),
};
case "scroll":
return { op: "scroll", ...common };
}
}
export interface ReducedTraceV3 {
steps: StepV3[];
stepIdByDraftId: Map<number, number>;
}
export function reduceTraceStepsV3(
steps: RecordingDraftStep[],
options: { includeTabSwitches?: boolean; redactValues?: boolean } = {},
): ReducedTraceV3 {
const output: StepV3[] = [];
const stepIdByDraftId = new Map<number, number>();
for (const { draft, draftIds } of collapseRedirects(steps)) {
const step = reduceDraft(draft, output.length + 1, {
includeTabSwitches: options.includeTabSwitches === true,
redactValues: options.redactValues === true,
});
if (!step) continue;
output.push(step);
for (const draftId of draftIds) stepIdByDraftId.set(draftId, step.id);
}
return { steps: output, stepIdByDraftId };
}
@@ -1,39 +0,0 @@
import type { StepAnnotation } from "./types";
function annotationText(annotation: StepAnnotation, stepId: number): string {
return ` ⟵ step ${stepId}: ${annotation.op}${annotation.detail ? `: ${annotation.detail}` : ""}`;
}
export function formatTraceStateBody(input: {
stateId: string;
url: string;
title?: string;
stepIds: number[];
vomText: string;
annotations: StepAnnotation[];
stepIdByDraftId: Map<number, number>;
}): string {
const lines = ["# bsk-observation 1", `state: ${JSON.stringify(input.stateId)}`];
lines.push(`url: ${JSON.stringify(input.url)}`);
if (input.title) lines.push(`title: ${JSON.stringify(input.title)}`);
if (input.stepIds.length > 0) lines.push(`steps_here: [${input.stepIds.join(", ")}]`);
lines.push("---");
const byLine = new Map<number, Array<{ annotation: StepAnnotation; stepId: number }>>();
for (const annotation of input.annotations) {
const stepId = input.stepIdByDraftId.get(annotation.draftId);
if (stepId === undefined) continue;
const bucket = byLine.get(annotation.line) ?? [];
bucket.push({ annotation, stepId });
byLine.set(annotation.line, bucket);
}
input.vomText.split("\n").forEach((bodyLine, lineIndex) => {
let line = bodyLine;
for (const item of byLine.get(lineIndex) ?? []) {
line += annotationText(item.annotation, item.stepId);
}
lines.push(line);
});
return `${lines.join("\n")}\n`;
}
-85
View File
@@ -1,85 +0,0 @@
import type { CaptureTargetDescriptor } from "@/lib/describe-target";
import type {
FillCommit,
KeyModifier,
NavigationCause,
StepV3,
TargetDescriptorV3,
} from "@/transport/types";
export interface TargetGeometry {
/** Top-level viewport-relative CSS pixels, as defined by the geometry module. */
rect: { x: number; y: number; w: number; h: number };
tag: string;
}
export interface TargetMatchHint {
geometry?: TargetGeometry;
/** Missing means top frame; null means the source Document could not be resolved. */
frameId?: string | null;
geometrySpace?: "top" | "local";
}
export interface StepAnnotation {
draftId: number;
op: StepV3["op"];
line: number;
stateId: string;
detail?: string;
}
interface DraftStateLink {
pageUrl?: string;
preStateId?: string;
postStateId?: string;
}
interface DraftTarget {
captureTarget?: CaptureTargetDescriptor;
targetHint?: TargetMatchHint;
matchedTarget?: TargetDescriptorV3;
}
interface DraftNavigationEffect {
navigatedTo?: string;
}
export type RecordingDraftStep =
| ({ op: "click" } & DraftStateLink & DraftTarget & DraftNavigationEffect)
| ({ op: "hover" } & DraftStateLink & DraftTarget)
| ({
op: "fill";
value: string;
commit?: FillCommit;
redacted?: boolean;
} & DraftStateLink &
DraftTarget &
DraftNavigationEffect)
| ({
op: "press";
key: string;
modifiers?: KeyModifier[];
} & DraftStateLink &
DraftTarget &
DraftNavigationEffect)
| ({
op: "select";
values: string[];
labels?: string[];
} & DraftStateLink &
DraftTarget &
DraftNavigationEffect)
| ({ op: "scroll" } & DraftStateLink)
| ({ op: "switch_tab" } & DraftStateLink)
| ({
op: "navigate";
url: string;
cause?: NavigationCause;
transitionType?: string;
transitionQualifiers?: string[];
} & DraftStateLink);
export type TargetedRecordingDraft = Extract<
RecordingDraftStep,
{ op: "click" | "hover" | "fill" | "press" | "select" }
>;
@@ -1,10 +1,30 @@
import type { PageRefV2, SelectedOptionV2, StepV2, TraceV2 } from "@/transport/types";
import { resolveDraftStartUrl, shouldIncludeDraft } from "./draft-policy";
import type { RecordingDraftStep } from "./types";
import type { DraftTraceStep, PageRef, SelectedOption, Step } from "@/transport/types";
const CLIPBOARD_KEYS = new Set(["a", "c", "v", "x", "A", "C", "V", "X"]);
const MODIFIER_ONLY_KEYS = new Set(["Meta", "Control", "Alt", "Shift", "OS", "Hyper", "Super"]);
export function shouldRecordPress(
key: string,
modifiers?: Array<"alt" | "ctrl" | "meta" | "shift">,
): boolean {
if (MODIFIER_ONLY_KEYS.has(key)) return false;
const mods = modifiers ?? [];
const hasCtrlOrMeta = mods.includes("ctrl") || mods.includes("meta");
if (hasCtrlOrMeta && CLIPBOARD_KEYS.has(key)) return false;
if (key === "Enter" || key === "Escape") return true;
// Drop bare character typing — FillSession already records the value.
if (key.length === 1 && !hasCtrlOrMeta && !mods.includes("alt")) return false;
return false;
}
function shouldIncludeDraft(step: DraftTraceStep): boolean {
if (step.op === "press" && !shouldRecordPress(step.key, step.modifiers)) return false;
return true;
}
/** Collapse consecutive navigations to the last hop. */
function collapseNavigations(steps: RecordingDraftStep[]): RecordingDraftStep[] {
const out: RecordingDraftStep[] = [];
function collapseNavigations(steps: DraftTraceStep[]): DraftTraceStep[] {
const out: DraftTraceStep[] = [];
for (const step of steps) {
const prev = out[out.length - 1];
if (step.op === "navigate" && prev?.op === "navigate") {
@@ -16,7 +36,7 @@ function collapseNavigations(steps: RecordingDraftStep[]): RecordingDraftStep[]
return out;
}
function collectUrls(steps: RecordingDraftStep[], startUrl?: string): string[] {
function collectUrls(steps: DraftTraceStep[], startUrl?: string): string[] {
const urls: string[] = [];
if (startUrl) urls.push(startUrl);
for (const step of steps) {
@@ -24,8 +44,8 @@ function collectUrls(steps: RecordingDraftStep[], startUrl?: string): string[] {
urls.push(step.url);
continue;
}
if (step.pageUrl) urls.push(step.pageUrl);
if ("navigatedTo" in step && step.navigatedTo) urls.push(step.navigatedTo);
if ("page_url" in step && step.page_url) urls.push(step.page_url);
if ("navigated_to" in step && step.navigated_to) urls.push(step.navigated_to);
}
const seen = new Set<string>();
const unique: string[] = [];
@@ -38,9 +58,9 @@ function collectUrls(steps: RecordingDraftStep[], startUrl?: string): string[] {
}
function buildPageRegistry(
steps: RecordingDraftStep[],
steps: DraftTraceStep[],
startUrl?: string,
): { pages: PageRefV2[]; urlToId: Map<string, string> } {
): { pages: PageRef[]; urlToId: Map<string, string> } {
const urls = collectUrls(steps, startUrl);
const urlToId = new Map<string, string>();
const pages = urls.map((url, index) => {
@@ -61,28 +81,28 @@ function pageIdFor(
return urlToId.values().next().value ?? "p1";
}
function pageUrlForDraft(step: RecordingDraftStep, fallbackUrl?: string): string | undefined {
if (step.op === "navigate") return step.pageUrl ?? step.url;
if (step.pageUrl) return step.pageUrl;
function pageUrlForDraft(step: DraftTraceStep, fallbackUrl?: string): string | undefined {
if (step.op === "navigate") return step.page_url ?? step.url;
if ("page_url" in step && step.page_url) return step.page_url;
return fallbackUrl;
}
function effectForNavigation(
navigatedTo: string | undefined,
urlToId: Map<string, string>,
): StepV2["effect"] {
): Step["effect"] {
if (!navigatedTo) return undefined;
const pageId = urlToId.get(navigatedTo);
if (!pageId) return undefined;
return { navigated_to: pageId };
}
function withEffect(step: StepV2, effect: StepV2["effect"]): StepV2 {
function withEffect(step: Step, effect: Step["effect"]): Step {
if (!effect) return step;
return { ...step, effect };
}
function toSelection(values: string[], labels?: string[]): SelectedOptionV2[] {
function toSelection(values: string[], labels?: string[]): SelectedOption[] {
return values.map((value, index) => ({
value,
...(labels?.[index] ? { label: labels[index] } : {}),
@@ -90,11 +110,11 @@ function toSelection(values: string[], labels?: string[]): SelectedOptionV2[] {
}
function toV2Step(
step: RecordingDraftStep,
step: DraftTraceStep,
id: number,
urlToId: Map<string, string>,
fallbackUrl?: string,
): StepV2 | null {
): Step | null {
if (!shouldIncludeDraft(step)) return null;
const pageUrl = pageUrlForDraft(step, fallbackUrl);
@@ -109,23 +129,28 @@ function toV2Step(
to: step.url,
};
case "click":
if (!step.captureTarget) return null;
return withEffect(
{
op: "click",
id,
page,
target: step.captureTarget,
target: step.target,
},
effectForNavigation(step.navigatedTo, urlToId),
effectForNavigation(step.navigated_to, urlToId),
);
case "hover":
return {
op: "hover",
id,
page,
target: step.target,
};
case "fill":
if (!step.captureTarget) return null;
return {
op: "fill",
id,
page,
target: step.captureTarget,
target: step.target,
value: step.value,
...(step.redacted ? { redacted: true } : {}),
};
@@ -136,51 +161,44 @@ function toV2Step(
id,
page,
key: step.key,
...(step.captureTarget ? { target: step.captureTarget } : {}),
...(step.target ? { target: step.target } : {}),
...(step.modifiers?.length ? { modifiers: step.modifiers } : {}),
},
effectForNavigation(step.navigatedTo, urlToId),
effectForNavigation(step.navigated_to, urlToId),
);
case "select":
if (!step.captureTarget) return null;
return withEffect(
{
op: "select",
id,
page,
target: step.captureTarget,
target: step.target,
selection: toSelection(step.values, step.labels),
},
effectForNavigation(step.navigatedTo, urlToId),
effectForNavigation(step.navigated_to, urlToId),
);
// `hover` only exists in Trace v3. Peers that negotiate v2 predate the
// step variant and fail to decode the whole result if we emit it.
case "hover":
case "scroll":
case "switch_tab":
return null;
}
}
interface ReducedTrace {
pages: PageRefV2[];
steps: StepV2[];
export interface ReducedTrace {
pages: PageRef[];
steps: Step[];
}
/**
* Compile capture drafts into record-only trace v2 steps.
* Variable inputs are NOT classified here — executing agents infer that at run time.
*/
function reduceTraceSteps(steps: RecordingDraftStep[], startUrl?: string): ReducedTrace {
export function reduceTraceSteps(steps: DraftTraceStep[], startUrl?: string): ReducedTrace {
const collapsed = collapseNavigations(steps);
const { pages, urlToId } = buildPageRegistry(collapsed, startUrl);
const out: StepV2[] = [];
const out: Step[] = [];
let id = 1;
let lastUrl = startUrl;
for (const draft of collapsed) {
if (draft.op === "navigate") lastUrl = draft.url;
else if ("navigatedTo" in draft && draft.navigatedTo) lastUrl = draft.navigatedTo;
else if (draft.pageUrl) lastUrl = draft.pageUrl;
else if ("navigated_to" in draft && draft.navigated_to) lastUrl = draft.navigated_to;
else if ("page_url" in draft && draft.page_url) lastUrl = draft.page_url;
const step = toV2Step(draft, id, urlToId, lastUrl);
if (!step) continue;
out.push(step);
@@ -189,27 +207,18 @@ function reduceTraceSteps(steps: RecordingDraftStep[], startUrl?: string): Reduc
return { pages, steps: out };
}
function resolveTraceStartUrl(
drafts: RecordingDraftStep[],
export function resolveTraceStartUrl(
drafts: DraftTraceStep[],
startUrl?: string,
pages?: PageRefV2[],
pages?: PageRef[],
): string {
return resolveDraftStartUrl(drafts, startUrl, pages?.[0]?.url);
}
export function buildTraceV2(input: {
steps: RecordingDraftStep[];
startedAt: string;
startUrl?: string;
purpose?: string;
}): TraceV2 {
const { pages, steps } = reduceTraceSteps(input.steps, input.startUrl);
return {
recorded_at: new Date().toISOString(),
started_at: input.startedAt,
...(input.purpose ? { purpose: input.purpose } : {}),
entry: { start_url: resolveTraceStartUrl(input.steps, input.startUrl, pages) },
pages,
steps,
};
if (startUrl) return startUrl;
const navigate = drafts.find((step): step is Extract<DraftTraceStep, { op: "navigate" }> => {
return step.op === "navigate";
});
if (navigate) return navigate.url;
for (const draft of drafts) {
if ("page_url" in draft && draft.page_url) return draft.page_url;
}
return pages?.[0]?.url ?? "about:blank";
}
@@ -1,22 +0,0 @@
export const RECORD_DOCUMENT_ATTRIBUTE = "data-bsk-record-document";
const RECORDING_DOCUMENT_ID_PATTERN =
/^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i;
const RECORDING_DOCUMENT_MARKER_PREFIX = "bsk:";
export function recordingDocumentMarkerValue(identity: string): string {
if (!RECORDING_DOCUMENT_ID_PATTERN.test(identity)) {
throw new TypeError("recording Document identity must be a random UUID");
}
return `${RECORDING_DOCUMENT_MARKER_PREFIX}${identity}`;
}
/** Read BrowserSkill's opaque per-Document recording identity from captured attributes. */
export function readRecordingDocumentIdentity(
attrs: Readonly<Record<string, string>>,
): string | undefined {
const marker = attrs[RECORD_DOCUMENT_ATTRIBUTE];
if (!marker?.startsWith(RECORDING_DOCUMENT_MARKER_PREFIX)) return undefined;
const identity = marker.slice(RECORDING_DOCUMENT_MARKER_PREFIX.length);
return identity && RECORDING_DOCUMENT_ID_PATTERN.test(identity) ? identity : undefined;
}
@@ -8,7 +8,6 @@ import type {
RequestFrame,
} from "@/transport/types";
import { ToolDispatcher } from "../dispatcher";
import { resetBrowserObservationForTests, setBrowserObservationAttachForTests } from "../record";
type TestDispatcherCdp = NonNullable<ConstructorParameters<typeof ToolDispatcher>[0]["cdp"]>;
@@ -45,72 +44,9 @@ function makeRequest(method: string, params: unknown): RequestFrame {
describe("ToolDispatcher", () => {
afterEach(() => {
resetBrowserObservationForTests();
vi.unstubAllGlobals();
});
it("uses the configured recording runtime for start and stop", async () => {
const { transport, sent, deliver } = fakeTransport();
const sessions = new SessionManager({
agentWindow: {
create: vi.fn(async () => 4242),
remove: vi.fn(),
ensureActiveTab: vi.fn(async () => {}),
},
});
await sessions.start("aa11");
const tab = {
id: 7,
windowId: 4242,
active: true,
status: "complete",
url: "https://example.com/start",
} as chrome.tabs.Tab;
const frameCoordinator = {
begin: vi.fn(),
armTab: vi.fn(async () => true),
sourceFor: vi.fn(() => null),
stop: vi.fn(async () => true),
cancel: vi.fn(),
};
const sendToTab = vi.fn(async () => ({ ok: true }));
setBrowserObservationAttachForTests(
() => () => {},
() => () => {},
);
const dispatcher = new ToolDispatcher({
transport,
sessions,
recording: {
tabsApi: {
get: vi.fn(async () => tab),
query: vi.fn(async () => [tab]),
},
frameCoordinator,
sendToTab,
},
});
dispatcher.start();
deliver(
makeRequest("tool.record_start", {
session_id: "aa11",
url: "https://example.com/start",
}),
);
await vi.waitFor(() => expect(sent).toHaveLength(1));
expect(sent[0]).toMatchObject({ result: { tab_id: 7, recording: true } });
expect(frameCoordinator.begin).toHaveBeenCalledOnce();
expect(frameCoordinator.armTab).toHaveBeenCalledWith(expect.any(String), 7);
deliver({ id: "r-2", method: "tool.record_stop", params: { session_id: "aa11" } });
await vi.waitFor(() => expect(sent).toHaveLength(2));
expect(frameCoordinator.stop).toHaveBeenCalledOnce();
expect(sent[1]).toMatchObject({ id: "r-2", result: { trace: { steps: [] } } });
});
it("routes tool.session_start to the SessionManager and replies with the window id", async () => {
const { transport, sent, deliver } = fakeTransport();
const sessions = new SessionManager({
@@ -232,6 +168,107 @@ describe("ToolDispatcher", () => {
});
});
it("bypasses and restores the control overlay for an upload trigger click", async () => {
const sendMessage = vi.fn(async () => undefined);
vi.stubGlobal("chrome", {
tabs: {
get: vi.fn(async () => ({ id: 7, windowId: 4242, active: true })),
query: vi.fn(async () => [{ id: 7, windowId: 4242, active: true }]),
sendMessage,
},
});
const { transport, sent, deliver } = fakeTransport();
const sessions = new SessionManager({
agentWindow: {
create: vi.fn(async () => 4242),
remove: vi.fn(async () => {}),
ensureActiveTab: vi.fn(async () => {}),
},
});
const ctx = await sessions.start("aa11");
ctx.refStore.set("e1", 123, { tabId: 7 });
const send = vi.fn(async <T>(_tabId: number, method: string, params?: object) => {
if (method === "Page.getLayoutMetrics") {
return { cssLayoutViewport: { clientWidth: 1280, clientHeight: 720 } } as T;
}
if (method === "DOM.getContentQuads") {
return { quads: [[0, 0, 20, 0, 20, 20, 0, 20]] } as T;
}
if (method === "DOM.resolveNode") {
return { object: { objectId: "trigger-object" } } as T;
}
if (method === "DOM.describeNode") return { node: { backendNodeId: 456 } } as T;
if (method === "Runtime.callFunctionOn") {
const declaration = (params as { functionDeclaration?: string }).functionDeclaration ?? "";
if (declaration.includes("count: state.inputs.length")) {
return { result: { value: { count: 1, multiple: false } } } as T;
}
if (declaration.includes("inputs[0]")) {
return { result: { objectId: "input-object" } } as T;
}
return { result: { value: true } } as T;
}
if (method === "Runtime.evaluate") {
const expression = (params as { expression?: string }).expression ?? "";
if (expression.includes("overlayDetails")) {
return { result: { value: { hitIndex: 0 } } } as T;
}
if (expression.includes("overlayHostPresent")) {
return {
result: {
value: { overlayHostPresent: true, overlayHostConnected: true },
},
} as T;
}
if (expression.includes("count:")) {
return { result: { value: { count: 1, multiple: false } } } as T;
}
if (expression.includes("?.inputs[0]")) {
return { result: { objectId: "input-object" } } as T;
}
return { result: { value: true } } as T;
}
return {} as T;
});
const cdp = {
send,
detachSession: vi.fn(async () => {}),
ensureNetworkCapture: vi.fn(async () => {}),
networkEntriesSince: vi.fn(() => ({
tab_id: 7,
entries: [],
next_since: 0,
truncated: false,
})),
setDeviceMetricsOverride: vi.fn(async () => {}),
clearDeviceMetricsOverride: vi.fn(async () => {}),
setUserAgentOverride: vi.fn(async () => {}),
setTouchEmulationEnabled: vi.fn(async () => {}),
};
const dispatcher = new ToolDispatcher({ transport, sessions, cdp: cdp as TestDispatcherCdp });
dispatcher.start();
deliver(
makeRequest("tool.upload", {
session_id: "aa11",
ref: "@e1",
files: [{ transfer_id: "tr_1", name: "test.png", staged_path: "/stage/test.png" }],
}),
);
await flushMicrotasks();
await vi.waitFor(() => expect(sent).toHaveLength(1));
expect(sent[0]).toMatchObject({ result: { tab_id: 7, file_names: ["test.png"] } });
expect(sendMessage).toHaveBeenNthCalledWith(1, 7, {
type: "bh-automation-bypass",
enabled: true,
});
expect(sendMessage).toHaveBeenNthCalledWith(2, 7, {
type: "bh-automation-bypass",
enabled: false,
});
});
it("detaches CDP state before stopping a session", async () => {
const { transport, sent, deliver } = fakeTransport();
const sessions = new SessionManager({
@@ -267,55 +304,6 @@ describe("ToolDispatcher", () => {
expect(sent[0]).toEqual({ id: "r-1", result: {} });
});
it("forwards daemon cancellation into an in-flight session stop", async () => {
const { transport, sent, deliver } = fakeTransport();
const remove = vi.fn(async () => {});
const sessions = new SessionManager({
agentWindow: {
create: vi.fn(async () => 4242),
remove,
ensureActiveTab: vi.fn(async () => {}),
},
});
await sessions.start("aa11");
let finishDetach: () => void = () => {};
const detach = new Promise<void>((resolve) => {
finishDetach = resolve;
});
const cdp = {
send: vi.fn(),
detachSession: vi.fn(() => detach),
ensureNetworkCapture: vi.fn(async () => {}),
networkEntriesSince: vi.fn(() => ({
tab_id: 7,
entries: [],
next_since: 0,
truncated: false,
})),
setDeviceMetricsOverride: vi.fn(async () => {}),
clearDeviceMetricsOverride: vi.fn(async () => {}),
setUserAgentOverride: vi.fn(async () => {}),
setTouchEmulationEnabled: vi.fn(async () => {}),
};
const dispatcher = new ToolDispatcher({ transport, sessions, cdp: cdp as TestDispatcherCdp });
dispatcher.start();
deliver(makeRequest("tool.session_stop", { session_id: "aa11" }));
await vi.waitFor(() => expect(cdp.detachSession).toHaveBeenCalledWith("aa11"));
deliver({ id: "cancel-stop", method: "cancel", params: { rpc_id: "r-1" } });
await flushMicrotasks();
finishDetach();
await flushMicrotasks();
expect(sent).toContainEqual({ id: "cancel-stop", result: { cancelled: true } });
expect(sent).toContainEqual({
id: "r-1",
error: expect.objectContaining({ code: "cancelled" }),
});
expect(sessions.has("aa11")).toBe(true);
expect(remove).not.toHaveBeenCalled();
});
it("returns not_found when stopping an unknown session", async () => {
const { transport, sent, deliver } = fakeTransport();
const sessions = new SessionManager({
@@ -0,0 +1,559 @@
import { describe, expect, it, vi } from "vitest";
import { SessionManager } from "@/session-manager/manager";
import { type DownloadsApi, handleDownload } from "../download";
import { captureBrowserDownload } from "../download-capture";
import { uploadThroughActivatedFileInput } from "../file-input-transaction";
import type { ResolvedActionTarget } from "../interaction";
import type { CdpRunner } from "../shared";
import { handleUpload } from "../upload";
function sessions() {
return new SessionManager({
agentWindow: {
create: vi.fn(async () => 100),
remove: vi.fn(async () => {}),
ensureActiveTab: vi.fn(async () => {}),
},
});
}
function tabsApi() {
return {
get: vi.fn(
async (tabId: number) => ({ id: tabId, windowId: 100, active: true }) as chrome.tabs.Tab,
),
query: vi.fn(async () => [{ id: 4, windowId: 100, active: true } as chrome.tabs.Tab]),
};
}
function fakeEvent<T extends (...args: never[]) => unknown>() {
const listeners = new Set<T>();
return {
addListener: (listener: T) => listeners.add(listener),
removeListener: (listener: T) => listeners.delete(listener),
emit: (...args: Parameters<T>) => {
for (const listener of listeners) listener(...args);
},
};
}
function actionTarget(frameId?: string): ResolvedActionTarget {
return {
tab: { tabId: 4, windowId: 100, active: true },
backendNodeId: 123,
cdpTarget: { tabId: 4 },
...(frameId ? { frameId } : {}),
usedRef: "e3",
};
}
function uploadCdp(
options: {
inputCount?: number;
multiple?: boolean;
chooser?: { frameId?: string; backendNodeId?: number; mode?: string };
pendingResolve?: boolean;
} = {},
) {
const calls: Array<{ method: string; params?: object }> = [];
let cdpEvent: Parameters<NonNullable<CdpRunner["onEvent"]>>[0] | undefined;
const send = vi.fn(async (_tabId: number, method: string, params?: object) => {
calls.push({ method, params });
if (method === "Page.setInterceptFileChooserDialog") return {};
if (method === "Page.getLayoutMetrics")
return { cssLayoutViewport: { clientWidth: 1280, clientHeight: 720 } };
if (method === "DOM.getContentQuads") return { quads: [[0, 0, 20, 0, 20, 20, 0, 20]] };
if (method === "DOM.resolveNode") {
if (options.pendingResolve) return new Promise<never>(() => {});
return { object: { objectId: "trigger-object" } };
}
if (method === "Runtime.callFunctionOn") {
const declaration = (params as { functionDeclaration?: string }).functionDeclaration ?? "";
if (declaration.includes("Object.defineProperty")) return { result: { value: true } };
if (declaration.includes("count: state.inputs.length")) {
return {
result: {
value: {
count: options.inputCount ?? 1,
multiple: options.multiple ?? true,
},
},
};
}
if (declaration.includes("inputs[0]")) {
return { result: { objectId: "input-object" } };
}
return { result: { value: true } };
}
if (method === "DOM.describeNode") return { node: { backendNodeId: 456 } };
if (
method === "Input.dispatchMouseEvent" &&
(params as { type?: string }).type === "mousePressed" &&
options.chooser
) {
cdpEvent?.({ tabId: 4 }, "Page.fileChooserOpened", options.chooser);
}
return {};
});
const cdp: CdpRunner = {
send: send as unknown as CdpRunner["send"],
onEvent: (handler) => {
cdpEvent = handler;
return { dispose: vi.fn() };
},
};
return {
calls,
emitChooser: (event: { frameId?: string; backendNodeId?: number; mode?: string }) =>
cdpEvent?.({ tabId: 4 }, "Page.fileChooserOpened", event),
cdp,
};
}
describe("file transfer tools", () => {
it("captures the file input activated by the requested click and injects only staged paths", async () => {
const manager = sessions();
const ctx = await manager.start("s1");
ctx.refStore.set("e3", 123, { tabId: 4 });
const { cdp, calls } = uploadCdp();
const result = await handleUpload(
manager,
{
session_id: "s1",
ref: "@e3",
files: [
{ transfer_id: "tr_1", name: "one.png", staged_path: "/private/stage/one" },
{ transfer_id: "tr_2", name: "two.png", staged_path: "/private/stage/two" },
],
},
{ cdp, tabsApi: tabsApi() },
);
expect(result).toMatchObject({ tab_id: 4, file_names: ["one.png", "two.png"] });
expect(calls[0]).toEqual({
method: "Page.setInterceptFileChooserDialog",
params: { enabled: true },
});
expect(calls).toContainEqual({
method: "Page.setInterceptFileChooserDialog",
params: { enabled: false, cancel: true },
});
expect(calls).toContainEqual({
method: "DOM.setFileInputFiles",
params: { files: ["/private/stage/one", "/private/stage/two"], backendNodeId: 456 },
});
});
it("fails immediately when the trigger does not activate a file input", async () => {
const { cdp } = uploadCdp({ inputCount: 0 });
const result = await uploadThroughActivatedFileInput({
cdp,
actionTarget: actionTarget(),
files: ["/private/stage/one"],
timeoutMs: 100,
trigger: async () => ({ tab_id: 4, x: 10, y: 10 }),
});
expect(result).toMatchObject({
code: "unsupported",
data: { reason: "file_input_not_activated", phase: "resolve_input" },
});
});
it("fails before clicking when chooser interception is unavailable", async () => {
const trigger = vi.fn(async () => ({ tab_id: 4, x: 10, y: 10 }));
const cdp: CdpRunner = {
send: vi.fn(async (_tabId: number, method: string) => {
if (method === "Page.setInterceptFileChooserDialog") {
throw new Error("method unavailable");
}
return {};
}) as CdpRunner["send"],
};
const result = await uploadThroughActivatedFileInput({
cdp,
actionTarget: actionTarget(),
files: ["/private/stage/one"],
timeoutMs: 100,
trigger,
});
expect(result).toMatchObject({
code: "cdp_failed",
data: { effect_state: "none", phase: "arm_interception" },
});
expect(trigger).not.toHaveBeenCalled();
});
it("uses an exact chooser event as an independent input-location signal", async () => {
const { cdp, calls, emitChooser } = uploadCdp({ inputCount: 0 });
const result = await uploadThroughActivatedFileInput({
cdp,
actionTarget: actionTarget("f1"),
files: ["/private/stage/one"],
timeoutMs: 100,
trigger: async () => {
emitChooser({ frameId: "f1", backendNodeId: 789, mode: "selectSingle" });
return { tab_id: 4, x: 10, y: 10 };
},
});
expect(result).toMatchObject({ multiple: false });
expect(calls).toContainEqual({
method: "DOM.setFileInputFiles",
params: { files: ["/private/stage/one"], backendNodeId: 789 },
});
});
it("reports a File System Access picker without waiting for a timeout", async () => {
const { cdp, emitChooser } = uploadCdp({
inputCount: 0,
});
const result = await uploadThroughActivatedFileInput({
cdp,
actionTarget: actionTarget("f1"),
files: ["/private/stage/one"],
timeoutMs: 100,
trigger: async () => {
emitChooser({ frameId: "f1", mode: "selectSingle" });
return { tab_id: 4, x: 10, y: 10 };
},
});
expect(result).toMatchObject({
code: "unsupported",
message: "upload trigger invoked a non-input file picker",
data: { reason: "file_input_not_activated", phase: "resolve_input" },
});
});
it("bounds a stuck file-input probe", async () => {
const { cdp } = uploadCdp({ pendingResolve: true });
const result = await uploadThroughActivatedFileInput({
cdp,
actionTarget: actionTarget(),
files: ["/private/stage/one"],
timeoutMs: 5,
trigger: vi.fn(),
});
expect(result).toMatchObject({
code: "timeout",
data: { reason: "file_input_probe_failed", phase: "arm_input_probe" },
});
});
it("marks a timed-out file assignment unknown and detaches browser state", async () => {
const fixture = uploadCdp();
const originalSend = fixture.cdp.send;
const detach = vi.fn(async () => {});
fixture.cdp.detach = detach;
fixture.cdp.send = vi.fn((tabId: number, method: string, params?: object) => {
if (method === "DOM.setFileInputFiles") return new Promise<never>(() => {});
return originalSend(tabId, method, params);
}) as CdpRunner["send"];
const result = await uploadThroughActivatedFileInput({
cdp: fixture.cdp,
actionTarget: actionTarget(),
files: ["/private/stage/one"],
timeoutMs: 10,
trigger: async () => ({ tab_id: 4, x: 10, y: 10 }),
});
expect(result).toMatchObject({
code: "timeout",
data: { effect_state: "unknown", phase: "set_files" },
});
expect(detach).toHaveBeenCalledWith(4);
});
it("routes one exact-target download through a browser-relative capability", async () => {
const manager = sessions();
const ctx = await manager.start("s1");
ctx.refStore.set("e3", 123, { tabId: 4 });
const onCreated = fakeEvent<(item: chrome.downloads.DownloadItem) => void>();
const onChanged = fakeEvent<(delta: chrome.downloads.DownloadDelta) => void>();
const onDeterminingFilename =
fakeEvent<
(
item: chrome.downloads.DownloadItem,
suggest: (suggestion?: chrome.downloads.DownloadFilenameSuggestion) => void,
) => void | true
>();
const initial = {
id: 9,
url: "https://example.test/result.zip",
finalUrl: "https://example.test/result.zip",
filename: "result.zip",
state: "in_progress",
fileSize: -1,
totalBytes: 12,
mime: "application/zip",
danger: "safe",
} as chrome.downloads.DownloadItem;
const completed = {
...initial,
filename: "/profile/Downloads/BrowserSkill/tr_1/result.zip",
state: "complete",
fileSize: 12,
} as chrome.downloads.DownloadItem;
const downloads: DownloadsApi = {
onCreated,
onChanged,
onDeterminingFilename,
search: vi.fn(async () => [completed]),
cancel: vi.fn(async () => {}),
removeFile: vi.fn(async () => {}),
};
let cdpEvent: Parameters<NonNullable<CdpRunner["onEvent"]>>[0] | undefined;
let suggested: chrome.downloads.DownloadFilenameSuggestion | undefined;
const send = vi.fn(async (_tabId: number, method: string, params?: object) => {
if (method === "Page.getLayoutMetrics")
return { cssLayoutViewport: { clientWidth: 1280, clientHeight: 720 } };
if (method === "DOM.getContentQuads") return { quads: [[0, 0, 20, 0, 20, 20, 0, 20]] };
if (
method === "Input.dispatchMouseEvent" &&
(params as { type?: string }).type === "mousePressed"
) {
cdpEvent?.({ tabId: 4 }, "Page.downloadWillBegin", {
url: initial.url,
suggestedFilename: "result.zip",
});
await new Promise<void>((resolve) => {
onDeterminingFilename.emit(initial, (value) => {
suggested = value;
resolve();
});
});
onCreated.emit(completed);
}
return {};
});
const cdp: CdpRunner = {
send: send as unknown as CdpRunner["send"],
onEvent: (handler) => {
cdpEvent = handler;
return { dispose: vi.fn() };
},
};
const result = await handleDownload(
manager,
{ session_id: "s1", ref: "@e3", browser_relative_dir: "BrowserSkill/tr_1" },
{ cdp, tabsApi: tabsApi(), downloads },
);
expect(suggested).toEqual({
filename: "BrowserSkill/tr_1/result.zip",
conflictAction: "overwrite",
});
expect(result).toMatchObject({
tab_id: 4,
suggested_filename: "result.zip",
byte_size: 12,
browser_path: completed.filename,
});
});
it("does not claim a download without an intent from the exact target", async () => {
const onCreated = fakeEvent<(item: chrome.downloads.DownloadItem) => void>();
const onChanged = fakeEvent<(delta: chrome.downloads.DownloadDelta) => void>();
const onDeterminingFilename =
fakeEvent<
(
item: chrome.downloads.DownloadItem,
suggest: (suggestion?: chrome.downloads.DownloadFilenameSuggestion) => void,
) => void | true
>();
const downloads: DownloadsApi = {
onCreated,
onChanged,
onDeterminingFilename,
search: vi.fn(async () => []),
cancel: vi.fn(async () => {}),
removeFile: vi.fn(async () => {}),
};
let cdpEvent: Parameters<NonNullable<CdpRunner["onEvent"]>>[0] | undefined;
const cdp: CdpRunner = {
send: vi.fn(async () => ({})) as unknown as CdpRunner["send"],
onEvent: (handler) => {
cdpEvent = handler;
return { dispose: vi.fn() };
},
};
const unrelated = {
id: 17,
url: "https://example.test/unrelated.zip",
finalUrl: "https://example.test/unrelated.zip",
filename: "unrelated.zip",
state: "in_progress",
} as chrome.downloads.DownloadItem;
let defaultSuggestionCalled = false;
const result = await captureBrowserDownload({
cdp,
target: { tabId: 4, sessionId: "expected-child" },
downloads,
browserRelativeDir: "BrowserSkill/tr_1",
timeoutMs: 5,
trigger: async () => {
cdpEvent?.({ tabId: 4, sessionId: "other-child" }, "Page.downloadWillBegin", {
url: unrelated.url,
suggestedFilename: unrelated.filename,
});
onDeterminingFilename.emit(unrelated, (suggestion) => {
defaultSuggestionCalled = suggestion === undefined;
});
return { tab_id: 4, x: 10, y: 10 };
},
});
expect(defaultSuggestionCalled).toBe(true);
expect(downloads.cancel).not.toHaveBeenCalled();
expect(result).toMatchObject({
code: "cdp_failed",
data: { reason: "download_capture_failed" },
});
});
it("correlates a filename candidate that arrives before the CDP intent", async () => {
const onCreated = fakeEvent<(item: chrome.downloads.DownloadItem) => void>();
const onChanged = fakeEvent<(delta: chrome.downloads.DownloadDelta) => void>();
const onDeterminingFilename =
fakeEvent<
(
item: chrome.downloads.DownloadItem,
suggest: (suggestion?: chrome.downloads.DownloadFilenameSuggestion) => void,
) => void | true
>();
const initial = {
id: 21,
url: "https://example.test/candidate-first.bin",
finalUrl: "https://example.test/candidate-first.bin",
filename: "candidate-first.bin",
state: "in_progress",
fileSize: -1,
totalBytes: 4,
bytesReceived: 0,
} as chrome.downloads.DownloadItem;
const complete = {
...initial,
state: "complete",
fileSize: 4,
} as chrome.downloads.DownloadItem;
const downloads: DownloadsApi = {
onCreated,
onChanged,
onDeterminingFilename,
search: vi.fn(async () => [complete]),
cancel: vi.fn(async () => {}),
removeFile: vi.fn(async () => {}),
};
let cdpEvent: Parameters<NonNullable<CdpRunner["onEvent"]>>[0] | undefined;
const cdp: CdpRunner = {
send: vi.fn(async () => ({})) as CdpRunner["send"],
onEvent: (handler) => {
cdpEvent = handler;
return { dispose: vi.fn() };
},
};
let suggestion: chrome.downloads.DownloadFilenameSuggestion | undefined;
const result = await captureBrowserDownload({
cdp,
target: { tabId: 4 },
downloads,
browserRelativeDir: "BrowserSkill/tr_21",
timeoutMs: 1_000,
trigger: async () => {
const suggested = new Promise<void>((resolve) => {
onDeterminingFilename.emit(initial, (value) => {
suggestion = value;
resolve();
});
});
cdpEvent?.({ tabId: 4 }, "Page.downloadWillBegin", {
url: initial.url,
suggestedFilename: initial.filename,
});
await suggested;
onCreated.emit(complete);
return { tab_id: 4, x: 10, y: 10 };
},
});
expect(suggestion).toEqual({
filename: "BrowserSkill/tr_21/candidate-first.bin",
conflictAction: "overwrite",
});
expect(result).toMatchObject({ item: { id: 21, state: "complete" } });
});
it("rejects ambiguous attribution without cancelling either unclaimed download", async () => {
const onCreated = fakeEvent<(item: chrome.downloads.DownloadItem) => void>();
const onChanged = fakeEvent<(delta: chrome.downloads.DownloadDelta) => void>();
const onDeterminingFilename =
fakeEvent<
(
item: chrome.downloads.DownloadItem,
suggest: (suggestion?: chrome.downloads.DownloadFilenameSuggestion) => void,
) => void | true
>();
const downloads: DownloadsApi = {
onCreated,
onChanged,
onDeterminingFilename,
search: vi.fn(async () => []),
cancel: vi.fn(async () => {}),
removeFile: vi.fn(async () => {}),
};
let cdpEvent: Parameters<NonNullable<CdpRunner["onEvent"]>>[0] | undefined;
const cdp: CdpRunner = {
send: vi.fn(async () => ({})) as CdpRunner["send"],
onEvent: (handler) => {
cdpEvent = handler;
return { dispose: vi.fn() };
},
};
const defaults: number[] = [];
const candidate = (id: number) =>
({
id,
url: "https://example.test/same.bin",
finalUrl: "https://example.test/same.bin",
filename: "same.bin",
state: "in_progress",
}) as chrome.downloads.DownloadItem;
const result = await captureBrowserDownload({
cdp,
target: { tabId: 4 },
downloads,
browserRelativeDir: "BrowserSkill/tr_ambiguous",
timeoutMs: 100,
trigger: async () => {
cdpEvent?.({ tabId: 4 }, "Page.downloadWillBegin", {
url: "https://example.test/same.bin",
suggestedFilename: "same.bin",
});
for (const id of [31, 32]) {
onDeterminingFilename.emit(candidate(id), (value) => {
if (value === undefined) defaults.push(id);
});
}
return { tab_id: 4, x: 10, y: 10 };
},
});
expect(defaults.sort()).toEqual([31, 32]);
expect(downloads.cancel).not.toHaveBeenCalled();
expect(result).toMatchObject({
code: "cdp_failed",
data: { effect_state: "unknown", phase: "attribution" },
});
});
});
@@ -8,7 +8,6 @@ import {
buildFrameVomScene,
buildVomScene,
type CdpAxNode,
captureVomObservation,
handleGetHtml,
handleObserve,
handleScreenshot,
@@ -1330,7 +1329,7 @@ describe("buildVomScene", () => {
expect(rendered).toContain(' @e2 textbox "验证码"');
});
it("marks current-password autocomplete as sensitive even for text inputs", () => {
it("does not mark non-password inputs sensitive from password-like labels", () => {
const axNodes: CdpAxNode[] = [
{
nodeId: "1",
@@ -1365,7 +1364,7 @@ describe("buildVomScene", () => {
expect(buildVomScene(axNodes, captured).nodes[0]).toEqual(
expect.objectContaining({
id: 10,
sensitive: true,
sensitive: false,
}),
);
});
@@ -1660,9 +1659,7 @@ describe("buildVomScene", () => {
);
const rendered = renderVom(scene);
expect(rendered.text).toContain('@e1 button "close"');
expect(rendered.refs.map(({ ref, backendNodeId }) => ({ ref, backendNodeId }))).toEqual([
{ ref: "e1", backendNodeId: 20 },
]);
expect(rendered.refs).toEqual([{ ref: "e1", backendNodeId: 20 }]);
});
it("does not promote a clickable container that wraps a real interactive control", () => {
@@ -1806,9 +1803,7 @@ describe("buildVomScene", () => {
expect(scene.nodes.find((n) => n.id === 30)?.role).toBe("generic");
const rendered = renderVom(scene);
expect(rendered.text).toContain('@e1 button "收藏"');
expect(rendered.refs.map(({ ref, backendNodeId }) => ({ ref, backendNodeId }))).toEqual([
{ ref: "e1", backendNodeId: 20 },
]);
expect(rendered.refs).toEqual([{ ref: "e1", backendNodeId: 20 }]);
});
it("builds active scope blocks from active aria-controls relationships", () => {
@@ -2909,360 +2904,10 @@ describe("handleSnapshot", () => {
],
};
}
function secretsSnapshotReply(secrets: {
email: string;
password: string;
otp: string;
card: string;
}) {
const S = [
"html",
"body",
"input",
"position",
"static",
"pointer-events",
"auto",
"type",
"text",
"password",
"autocomplete",
"one-time-code",
"cc-number",
"value",
secrets.email,
secrets.password,
secrets.otp,
secrets.card,
];
const i = (s: string) => S.indexOf(s);
const style = [i("static"), i("auto"), i("auto")];
return {
strings: S,
documents: [
{
nodes: {
parentIndex: [-1, 0, 1, 1, 1, 1],
nodeName: [i("html"), i("body"), i("input"), i("input"), i("input"), i("input")],
backendNodeId: [10, 11, 12, 13, 14, 15],
attributes: [
[],
[],
[i("type"), i("text"), i("value"), i(secrets.email)],
[i("type"), i("password"), i("value"), i(secrets.password)],
[
i("type"),
i("text"),
i("autocomplete"),
i("one-time-code"),
i("value"),
i(secrets.otp),
],
[
i("type"),
i("text"),
i("autocomplete"),
i("cc-number"),
i("value"),
i(secrets.card),
],
],
inputValue: {
index: [2, 3, 4, 5],
value: [i(secrets.email), i(secrets.password), i(secrets.otp), i(secrets.card)],
},
},
layout: {
nodeIndex: [1, 2, 3, 4, 5],
styles: [style, style, style, style, style],
bounds: [
[0, 0, 1000, 800],
[10, 10, 200, 32],
[10, 50, 200, 32],
[10, 90, 200, 32],
[10, 130, 200, 32],
],
paintOrders: [0, 1, 2, 3, 4],
},
},
],
};
}
const VP_METRICS = {
cssLayoutViewport: { clientWidth: 1000, clientHeight: 800, pageX: 0, pageY: 0 },
};
function makeFrameAwareDeps() {
const strings = ["html", "body", "iframe", "button", "title", "Remote", "static", "auto"];
const i = (value: string) => strings.indexOf(value);
const styles = [i("static"), i("auto"), i("auto")];
const snapshot = {
strings,
documents: [
{
frameId: "main",
nodes: {
parentIndex: [-1, 0, 1],
nodeName: [i("html"), i("body"), i("iframe")],
backendNodeId: [10, 11, 12],
attributes: [[], [], [i("title"), i("Remote")]],
contentDocumentIndex: { index: [2], value: [1] },
},
layout: {
nodeIndex: [0, 1, 2],
styles: [styles, styles, styles],
bounds: [
[0, 0, 1000, 800],
[0, 0, 1000, 800],
[100, 100, 400, 300],
],
paintOrders: [0, 0, 1],
},
},
{
frameId: "child",
nodes: {
parentIndex: [-1, 0, 1],
nodeName: [i("html"), i("body"), i("button")],
backendNodeId: [20, 21, 22],
attributes: [[], [], []],
},
layout: {
nodeIndex: [0, 1, 2],
styles: [styles, styles, styles],
bounds: [
[0, 0, 400, 300],
[0, 0, 400, 300],
[20, 30, 120, 40],
],
paintOrders: [0, 0, 1],
},
},
],
};
const mainAx: CdpAxNode[] = [
{
nodeId: "main-root",
role: { type: "role", value: "RootWebArea" },
backendDOMNodeId: 11,
childIds: ["frame-owner"],
},
{
nodeId: "frame-owner",
parentId: "main-root",
role: { type: "role", value: "Iframe" },
name: { type: "computedString", value: "Remote" },
backendDOMNodeId: 12,
},
];
const childAx: CdpAxNode[] = [
{
nodeId: "child-root",
role: { type: "role", value: "RootWebArea" },
backendDOMNodeId: 21,
childIds: ["child-button"],
},
{
nodeId: "child-button",
parentId: "child-root",
role: { type: "role", value: "button" },
name: { type: "computedString", value: "Frame action" },
backendDOMNodeId: 22,
},
];
const send = vi.fn(async (_tabId: number, method: string) => {
if (method === "Page.getLayoutMetrics") return VP_METRICS;
if (method === "DOMSnapshot.enable" || method === "Accessibility.enable") return {};
if (method === "DOMSnapshot.captureSnapshot") return snapshot;
if (method === "Accessibility.getFullAXTree") return { nodes: mainAx };
throw new Error(`unexpected root CDP method ${method}`);
});
const sendToTarget = vi.fn(async (_target, method: string) => {
if (method === "Accessibility.enable") return {};
if (method === "Accessibility.getFullAXTree") return { nodes: childAx };
throw new Error(`unexpected child CDP method ${method}`);
});
const cdp = {
send: send as unknown as CdpRunner["send"],
sendToTarget: sendToTarget as unknown as NonNullable<CdpRunner["sendToTarget"]>,
getFrameGraph: vi.fn(async () => ({
rootFrameId: "main",
frames: [
{ frameId: "main", target: { tabId: 4 } },
{
frameId: "child",
parentFrameId: "main",
ownerBackendNodeId: 12,
target: { tabId: 4, sessionId: "child-session" },
},
],
})),
trackSessionTab: vi.fn(),
} satisfies CdpRunner;
return {
cdp,
tabsApi: {
get: vi.fn(
async (tabId: number) => ({ id: tabId, windowId: 100, active: true }) as chrome.tabs.Tab,
),
query: vi.fn(async () => [{ id: 4, windowId: 100, active: true } as chrome.tabs.Tab]),
},
};
}
it("exposes the frame-aware observation result for recording", async () => {
const ax: CdpAxNode[] = [
{
nodeId: "root",
role: { type: "role", value: "RootWebArea" },
backendDOMNodeId: 11,
childIds: ["password"],
},
{
nodeId: "password",
parentId: "root",
role: { type: "role", value: "textbox" },
name: { type: "computedString", value: "Password" },
value: { value: "hunter2" },
backendDOMNodeId: 13,
},
];
const deps = makeOverlayDeps(ax, loginSnapshotReply(), VP_METRICS);
const result = await captureVomObservation(deps.cdp, 4, "https://example.com", {
redactValues: true,
});
expect(result.text).not.toContain("hunter2");
expect(result.text).toContain("•••");
expect(result.refs[0]).toMatchObject({
backendNodeId: 13,
role: "textbox",
name: "Password",
line: expect.any(Number),
});
expect(result.rootFrameId).toBe("root");
expect(result).not.toHaveProperty("frameDocuments");
expect(result.frames).toEqual([{ frameId: "root", target: { tabId: 4 } }]);
expect(result.matchNodes.find((node) => node.backendNodeId === 13)).toMatchObject({
frameId: "root",
backendNodeId: 13,
tag: "input",
rect: { x: 400, y: 300, w: 200, h: 40 },
localRect: { x: 400, y: 300, w: 200, h: 40 },
});
});
it("does not leak form secrets through the record-safe observation payload", async () => {
const secrets = {
email: "user@example.com",
password: "hunter2",
otp: "847291",
card: "4111111111111111",
};
const ax: CdpAxNode[] = [
{
nodeId: "root",
role: { type: "role", value: "RootWebArea" },
backendDOMNodeId: 11,
childIds: ["email", "password", "otp", "card"],
},
{
nodeId: "email",
parentId: "root",
role: { type: "role", value: "textbox" },
name: { type: "computedString", value: "Email" },
value: { value: secrets.email },
backendDOMNodeId: 12,
},
{
nodeId: "password",
parentId: "root",
role: { type: "role", value: "textbox" },
name: { type: "computedString", value: "Password" },
value: { value: secrets.password },
backendDOMNodeId: 13,
},
{
nodeId: "otp",
parentId: "root",
role: { type: "role", value: "textbox" },
name: { type: "computedString", value: "Code" },
value: { value: secrets.otp },
backendDOMNodeId: 14,
},
{
nodeId: "card",
parentId: "root",
role: { type: "role", value: "textbox" },
name: { type: "computedString", value: "Card" },
value: { value: secrets.card },
backendDOMNodeId: 15,
},
];
const deps = makeOverlayDeps(ax, secretsSnapshotReply(secrets), VP_METRICS);
const result = await captureVomObservation(deps.cdp, 4, "https://example.com", {
redactValues: true,
});
const dumped = JSON.stringify(result);
expect(result).not.toHaveProperty("frameDocuments");
expect(dumped).not.toContain(secrets.email);
expect(dumped).not.toContain(secrets.password);
expect(dumped).not.toContain(secrets.otp);
expect(dumped).not.toContain(secrets.card);
expect(dumped).not.toContain("formValue");
expect(dumped).not.toContain("formDefaultValue");
expect(dumped).not.toContain("axNodes");
expect(dumped).not.toContain("domNodes");
expect(dumped).not.toContain("attrs");
expect(result.text).toContain("•••");
});
it("keeps frames and rendered refs on the same frame identity", async () => {
const deps = makeFrameAwareDeps();
const result = await captureVomObservation(deps.cdp, 4, "https://example.com");
expect(result.rootFrameId).toBe("main");
expect(result.frames).toHaveLength(2);
expect(result.frames.find((frame) => frame.frameId === "child")).toEqual({
frameId: "child",
parentFrameId: "main",
ownerBackendNodeId: 12,
target: { tabId: 4, sessionId: "child-session" },
});
expect(result.matchNodes.find((node) => node.backendNodeId === 22)).toMatchObject({
frameId: "child",
tag: "button",
localRect: { x: 20, y: 30, w: 120, h: 40 },
});
expect(result.refs.find((ref) => ref.backendNodeId === 22)).toMatchObject({
frameId: "child",
name: "Frame action",
});
});
it("stores iframe refs with their owning CDP session", async () => {
const sm = new SessionManager({ agentWindow: fakeAgentWindow([100]) });
const ctx = await sm.start("aa11");
const deps = makeFrameAwareDeps();
const result = await handleSnapshot(sm, { session_id: "aa11" }, deps);
if ("code" in result) throw new Error(`unexpected error: ${JSON.stringify(result)}`);
expect(result.text).toContain('@e1 button "Frame action"');
const frameRef = [...ctx.refStore.entries()].find(([, entry]) => entry.backendNodeId === 22);
expect(frameRef?.[1]).toMatchObject({
tabId: 4,
frameId: "child",
cdpSessionId: "child-session",
});
});
it("renders a blocking login overlay as the focused top layer", async () => {
const sm = new SessionManager({ agentWindow: fakeAgentWindow([100]) });
await sm.start("aa11");
File diff suppressed because it is too large Load Diff
@@ -66,52 +66,6 @@ function makeApis(
}
describe("handleSessionStop with auto-return", () => {
it("leaves the session untouched when cancellation arrived before teardown", async () => {
const aw = fakeAgentWindow([100]);
const sm = new SessionManager({ agentWindow: aw });
await sm.start("aa11");
const cdp = { detachSession: vi.fn(async () => {}) };
const controller = new AbortController();
controller.abort();
const res = await handleSessionStop(
sm,
{ session_id: "aa11" },
{ cdp, signal: controller.signal },
);
expect(res).toMatchObject({ code: "cancelled" });
expect(sm.has("aa11")).toBe(true);
expect(cdp.detachSession).not.toHaveBeenCalled();
expect(aw.remove).not.toHaveBeenCalled();
});
it("does not close the Agent Window when cancellation lands during CDP detach", async () => {
const aw = fakeAgentWindow([100]);
const sm = new SessionManager({ agentWindow: aw });
await sm.start("aa11");
const controller = new AbortController();
let finishDetach: () => void = () => {};
const detach = new Promise<void>((resolve) => {
finishDetach = resolve;
});
const cdp = { detachSession: vi.fn(() => detach) };
const stopping = handleSessionStop(
sm,
{ session_id: "aa11" },
{ cdp, signal: controller.signal },
);
await vi.waitFor(() => expect(cdp.detachSession).toHaveBeenCalledWith("aa11"));
controller.abort();
finishDetach();
const res = await stopping;
expect(res).toMatchObject({ code: "cancelled" });
expect(sm.has("aa11")).toBe(true);
expect(aw.remove).not.toHaveBeenCalled();
});
it("returns every borrowed tab and closes the Agent Window in the right order", async () => {
const aw = fakeAgentWindow([100]);
const sm = new SessionManager({ agentWindow: aw });
@@ -1,9 +0,0 @@
export {
type CaptureVomObservationOptions,
captureVomObservation,
} from "./observation";
export {
type CaptureVomFrame,
type CaptureVomMatchNode,
type CaptureVomObservationResult,
} from "./vom/record-safe-observation";
+77 -35
View File
@@ -4,6 +4,7 @@ import type { Transport } from "@/transport/transport";
import type {
ClickParams,
ConsoleParams,
DownloadParams,
EmulateParams,
EvaluateParams,
FillParams,
@@ -28,10 +29,12 @@ import type {
ScreenshotParams,
SelectParams,
SnapshotParams,
UploadParams,
WaitForNavigationParams,
} from "@/transport/types";
import { isRequestFrame } from "@/transport/types";
import { handleConsole } from "./console";
import { handleDownload } from "./download";
import { type EmulateCdpRunner, handleEmulate } from "./emulate";
import { handleEvaluate } from "./evaluate";
import { handleRequestHelp } from "./human-loop";
@@ -51,12 +54,7 @@ import {
handleScreenshot,
handleSnapshot,
} from "./observation";
import {
handleRecordAwait,
handleRecordStart,
handleRecordStop,
type RecordRuntimeDeps,
} from "./record";
import { handleRecordAwait, handleRecordStart, handleRecordStop } from "./record";
import {
handleSessionStart,
handleSessionStop,
@@ -79,6 +77,7 @@ import {
type TabReturnParams,
type TabSelectParams,
} from "./tabs";
import { handleUpload } from "./upload";
import { handleWaitForNavigation } from "./waits";
import { handleWindowResize, type WindowResizeParams } from "./window";
@@ -104,7 +103,6 @@ export interface DispatcherDeps {
transport: Transport;
sessions: SessionManager;
cdp?: DispatcherCdpRunner;
recording?: RecordRuntimeDeps;
/**
* Invoked whenever a dispatched RPC may have changed the live
* session set (currently `tool.session_start` and
@@ -140,7 +138,6 @@ export class ToolDispatcher {
private readonly transport: Transport;
private readonly sessions: SessionManager;
private readonly cdp?: DispatcherCdpRunner;
private readonly recording?: RecordRuntimeDeps;
private readonly onSessionsChanged?: () => void;
private readonly onBrowserControlResumed?: (sessionId: string) => void;
private readonly approveBorrow?: BorrowConfirmationApprover;
@@ -160,7 +157,6 @@ export class ToolDispatcher {
this.transport = deps.transport;
this.sessions = deps.sessions;
this.cdp = deps.cdp;
this.recording = deps.recording;
this.onSessionsChanged = deps.onSessionsChanged;
this.onBrowserControlResumed = deps.onBrowserControlResumed;
this.approveBorrow = deps.approveBorrow;
@@ -298,7 +294,6 @@ export class ToolDispatcher {
await this.releaseHoverLatch((req.params as SessionStopParams).session_id);
return handleSessionStop(this.sessions, req.params as SessionStopParams, {
cdp: this.cdp,
signal,
});
}
case "tool.tab_list":
@@ -513,6 +508,40 @@ export class ToolDispatcher {
),
signal,
);
case "tool.upload":
return this.withHoverReleaseForRequest(
req.params as UploadParams,
() =>
this.cdp
? handleUpload(this.sessions, req.params as UploadParams, {
cdp: this.cdp,
tabsApi: chromeTabsApi,
signal,
bypassOverlay,
})
: Promise.resolve({
code: "unsupported",
message: "upload requires CDP",
} satisfies RpcError),
signal,
);
case "tool.download":
return this.withHoverReleaseForRequest(
req.params as DownloadParams,
() =>
this.cdp
? handleDownload(this.sessions, req.params as DownloadParams, {
cdp: this.cdp,
tabsApi: chromeTabsApi,
signal,
bypassOverlay,
})
: Promise.resolve({
code: "unsupported",
message: "download requires CDP",
} satisfies RpcError),
signal,
);
case "tool.evaluate":
return handleEvaluate(
this.sessions,
@@ -539,26 +568,44 @@ export class ToolDispatcher {
signal,
});
case "tool.record_start":
return this.recording
? handleRecordStart(this.sessions, req.params as RecordStartParams, {
...this.recording,
signal,
})
: recordingRuntimeUnavailable();
return handleRecordStart(this.sessions, req.params as RecordStartParams, {
tabsApi: chromeTabsApi,
sendToTab: (tabId, msg) => chrome.tabs.sendMessage(tabId, msg),
bypassOverlay: async (tabId, enabled) => {
try {
await chrome.tabs.sendMessage(tabId, {
type: OVERLAY_AUTOMATION_BYPASS,
enabled,
});
} catch {
// Content script may be unavailable on restricted pages.
}
},
...(this.cdp ? { cdp: this.cdp } : {}),
signal,
});
case "tool.record_stop":
return this.recording
? handleRecordStop(this.sessions, req.params as RecordStopParams, {
...this.recording,
signal,
})
: recordingRuntimeUnavailable();
return handleRecordStop(this.sessions, req.params as RecordStopParams, {
tabsApi: chromeTabsApi,
sendToTab: (tabId, msg) => chrome.tabs.sendMessage(tabId, msg),
bypassOverlay: async (tabId, enabled) => {
try {
await chrome.tabs.sendMessage(tabId, {
type: OVERLAY_AUTOMATION_BYPASS,
enabled,
});
} catch {
// Content script may be unavailable on restricted pages.
}
},
signal,
});
case "tool.record_await":
return this.recording
? handleRecordAwait(this.sessions, req.params as RecordAwaitParams, {
...this.recording,
signal,
})
: recordingRuntimeUnavailable();
return handleRecordAwait(this.sessions, req.params as RecordAwaitParams, {
tabsApi: chromeTabsApi,
sendToTab: (tabId, msg) => chrome.tabs.sendMessage(tabId, msg),
signal,
});
default:
return {
code: "unknown_method",
@@ -694,13 +741,6 @@ function isRpcError(v: unknown): v is RpcError {
);
}
function recordingRuntimeUnavailable(): RpcError {
return {
code: "protocol_error",
message: "recording runtime is unavailable",
};
}
function sessionIdForBrowserControlMethod(req: RequestFrame): string | null {
switch (req.method) {
case "tool.tab_create":
@@ -719,6 +759,8 @@ function sessionIdForBrowserControlMethod(req: RequestFrame): string | null {
case "tool.fill":
case "tool.press":
case "tool.select":
case "tool.upload":
case "tool.download":
case "tool.evaluate":
case "tool.observe":
case "tool.request_help":
@@ -0,0 +1,349 @@
// Order-independent coordinator for one browser download. CDP supplies the
// exact target/frame intent while chrome.downloads supplies the download id
// and filename routing hook; neither event is assumed to arrive first.
import type { CdpTarget } from "@/browser-driver/frame-graph";
import type { ClickResult, RpcError, TransferEffectState } from "@/transport/types";
import { transferError } from "./errors";
import { type CdpRunner, isRpcError } from "./shared";
const CORRELATION_GRACE_MS = 750;
const UNIQUE_SETTLE_MS = 50;
const SIZE_POLL_MS = 250;
type DeterminingFilenameListener = (
item: chrome.downloads.DownloadItem,
suggest: (suggestion?: chrome.downloads.DownloadFilenameSuggestion) => void,
) => void | true;
interface ListenerEvent<T> {
addListener(listener: T): void;
removeListener(listener: T): void;
}
export interface DownloadsApi {
onCreated: ListenerEvent<(item: chrome.downloads.DownloadItem) => void>;
onChanged: ListenerEvent<(delta: chrome.downloads.DownloadDelta) => void>;
onDeterminingFilename: ListenerEvent<DeterminingFilenameListener>;
search(query: chrome.downloads.DownloadQuery): Promise<chrome.downloads.DownloadItem[]>;
cancel(downloadId: number): Promise<void>;
removeFile(downloadId: number): Promise<void>;
}
export const chromeDownloadsApi: DownloadsApi = {
get onCreated() {
return chrome.downloads.onCreated;
},
get onChanged() {
return chrome.downloads.onChanged;
},
get onDeterminingFilename() {
return chrome.downloads.onDeterminingFilename;
},
search: (query) => chrome.downloads.search(query),
cancel: (id) => chrome.downloads.cancel(id),
removeFile: (id) => chrome.downloads.removeFile(id),
};
export interface DownloadCaptureOptions {
cdp: CdpRunner;
target: CdpTarget;
expectedFrameId?: string;
downloads: DownloadsApi;
browserRelativeDir: string;
maxByteSize?: number;
timeoutMs: number;
signal?: AbortSignal;
trigger(): Promise<ClickResult | RpcError>;
}
export interface DownloadCaptureResult {
click: ClickResult;
item: chrome.downloads.DownloadItem;
}
interface DownloadIntent {
url: string;
suggestedFilename: string;
frameId?: string;
}
interface DownloadCandidate {
item: chrome.downloads.DownloadItem;
suggest: (suggestion?: chrome.downloads.DownloadFilenameSuggestion) => void;
suggested: boolean;
graceTimer: ReturnType<typeof setTimeout>;
}
function safeBasename(filename: string): string {
const basename = filename.split(/[\\/]/).pop()?.trim();
return basename && basename !== "." && basename !== ".." ? basename : "download";
}
function sameTarget(source: { tabId?: number; sessionId?: string }, target: CdpTarget): boolean {
return source.tabId === target.tabId && source.sessionId === target.sessionId;
}
function matchesIntent(item: chrome.downloads.DownloadItem, intent: DownloadIntent): boolean {
const urlMatches = item.url === intent.url || item.finalUrl === intent.url;
return urlMatches && safeBasename(item.filename) === safeBasename(intent.suggestedFilename);
}
function knownSize(item: chrome.downloads.DownloadItem): number | undefined {
if (item.fileSize >= 0) return item.fileSize;
if (item.totalBytes >= 0) return item.totalBytes;
return undefined;
}
function captureError(
message: string,
effectState: TransferEffectState,
phase: string,
cleanupFailed = false,
): RpcError {
return transferError("cdp_failed", "download_capture_failed", message, {
effectState,
phase,
...(cleanupFailed ? { cleanupState: "failed" } : {}),
});
}
export async function captureBrowserDownload(
options: DownloadCaptureOptions,
): Promise<DownloadCaptureResult | RpcError> {
let click: ClickResult | undefined;
let intent: DownloadIntent | undefined;
let capturedId: number | undefined;
let capturedItem: chrome.downloads.DownloadItem | undefined;
let settled = false;
let succeeded = false;
let failureResult: RpcError | undefined;
let uniquenessTimer: ReturnType<typeof setTimeout> | undefined;
let operationTimer: ReturnType<typeof setTimeout> | undefined;
let sizePoll: ReturnType<typeof setInterval> | undefined;
const candidates = new Map<number, DownloadCandidate>();
const createdItems = new Map<number, chrome.downloads.DownloadItem>();
let resolveCompletion!: (item: chrome.downloads.DownloadItem) => void;
let rejectCompletion!: (error: Error) => void;
const completion = new Promise<chrome.downloads.DownloadItem>((resolve, reject) => {
resolveCompletion = resolve;
rejectCompletion = reject;
});
const fail = (error: Error) => {
if (settled) return;
settled = true;
rejectCompletion(error);
};
const complete = (item: chrome.downloads.DownloadItem) => {
if (settled) return;
const size = knownSize(item);
if (size !== undefined && options.maxByteSize !== undefined && size > options.maxByteSize) {
fail(new Error(`download exceeds transfer limit ${options.maxByteSize}`));
return;
}
settled = true;
capturedItem = item;
resolveCompletion(item);
};
const suggestDefault = (candidate: DownloadCandidate) => {
if (candidate.suggested) return;
candidate.suggested = true;
candidate.suggest();
};
const matchingCandidates = (): DownloadCandidate[] => {
const currentIntent = intent;
return currentIntent
? [...candidates.values()].filter(
(candidate) => !candidate.suggested && matchesIntent(candidate.item, currentIntent),
)
: [];
};
const claimUnique = () => {
uniquenessTimer = undefined;
if (settled || capturedId !== undefined || !intent) return;
const matches = matchingCandidates();
if (matches.length !== 1) {
if (matches.length > 1) {
for (const candidate of matches) suggestDefault(candidate);
fail(new Error("download attribution is ambiguous"));
}
return;
}
const candidate = matches[0];
candidate.suggested = true;
clearTimeout(candidate.graceTimer);
capturedId = candidate.item.id;
capturedItem = candidate.item;
candidate.suggest({
filename: `${options.browserRelativeDir}/${safeBasename(intent.suggestedFilename)}`,
conflictAction: "overwrite",
});
const size = knownSize(candidate.item);
if (size !== undefined && options.maxByteSize !== undefined && size > options.maxByteSize) {
fail(new Error(`download exceeds transfer limit ${options.maxByteSize}`));
return;
}
const created = createdItems.get(candidate.item.id);
if (created?.state === "interrupted") {
fail(new Error(created.error ?? "download interrupted"));
} else if (created?.state === "complete") {
complete(created);
}
};
const reconcile = () => {
if (settled || capturedId !== undefined || !intent) return;
const matches = matchingCandidates();
if (matches.length > 1) {
for (const candidate of matches) suggestDefault(candidate);
fail(new Error("download attribution is ambiguous"));
return;
}
if (matches.length === 1 && !uniquenessTimer) {
uniquenessTimer = setTimeout(claimUnique, UNIQUE_SETTLE_MS);
}
};
const determiningListener: DeterminingFilenameListener = (item, suggest) => {
const candidate: DownloadCandidate = {
item,
suggest,
suggested: false,
graceTimer: setTimeout(() => {
suggestDefault(candidate);
candidates.delete(item.id);
if (intent && matchesIntent(item, intent) && capturedId === undefined) {
fail(new Error("download correlation grace elapsed before unique attribution"));
}
}, CORRELATION_GRACE_MS),
};
candidates.set(item.id, candidate);
reconcile();
return true;
};
const createdListener = (item: chrome.downloads.DownloadItem) => {
createdItems.set(item.id, item);
if (capturedId !== item.id) return;
capturedItem = item;
if (item.state === "interrupted") {
fail(new Error(item.error ?? "download interrupted"));
} else if (item.state === "complete") {
complete(item);
}
};
const changedListener = async (delta: chrome.downloads.DownloadDelta) => {
if (capturedId === undefined || delta.id !== capturedId || settled) return;
if (delta.state?.current === "interrupted" || delta.error?.current) {
fail(new Error(delta.error?.current ?? "download interrupted"));
return;
}
if (delta.state?.current === "complete") {
try {
const [item] = await options.downloads.search({ id: delta.id });
if (item) complete(item);
else fail(new Error("completed download disappeared"));
} catch (err) {
fail(err instanceof Error ? err : new Error(String(err)));
}
}
};
const onAbort = () => fail(new DOMException("aborted", "AbortError"));
const cdpSubscription = options.cdp.onEvent?.((source, method, raw) => {
if (method !== "Page.downloadWillBegin" || !sameTarget(source, options.target)) return;
const event = raw as { url?: unknown; suggestedFilename?: unknown; frameId?: unknown };
if (typeof event.url !== "string" || typeof event.suggestedFilename !== "string") return;
if (options.expectedFrameId && event.frameId !== options.expectedFrameId) {
fail(new Error("download originated from a different frame"));
return;
}
if (intent) {
fail(new Error("download trigger produced more than one browser download intent"));
return;
}
intent = {
url: event.url,
suggestedFilename: event.suggestedFilename,
...(typeof event.frameId === "string" ? { frameId: event.frameId } : {}),
};
reconcile();
});
if (!cdpSubscription) {
return captureError("CDP download intent subscription unavailable", "none", "arm");
}
options.downloads.onDeterminingFilename.addListener(determiningListener);
options.downloads.onCreated.addListener(createdListener);
options.downloads.onChanged.addListener(changedListener);
options.signal?.addEventListener("abort", onAbort, { once: true });
operationTimer = setTimeout(
() => fail(new Error("download did not complete before timeout")),
options.timeoutMs,
);
sizePoll = setInterval(() => {
if (capturedId === undefined || settled || options.maxByteSize === undefined) return;
void options.downloads
.search({ id: capturedId })
.then(([item]) => {
if (!item || settled) return;
capturedItem = item;
if (item.bytesReceived > (options.maxByteSize as number)) {
fail(new Error(`download exceeds transfer limit ${options.maxByteSize}`));
}
})
.catch((err) => fail(err instanceof Error ? err : new Error(String(err))));
}, SIZE_POLL_MS);
try {
const triggered = await options.trigger();
if (isRpcError(triggered)) {
void completion.catch(() => undefined);
const effect: TransferEffectState =
capturedId !== undefined ? "committed" : intent ? "unknown" : "none";
failureResult = {
...triggered,
data: { ...triggered.data, effect_state: effect, phase: "trigger" },
};
return failureResult;
}
click = triggered;
const item = await completion;
succeeded = true;
return { click, item };
} catch (err) {
const effect: TransferEffectState =
capturedId !== undefined ? "committed" : click ? "unknown" : "none";
failureResult = captureError(
err instanceof Error ? err.message : String(err),
effect,
capturedId !== undefined ? "download" : "attribution",
);
return failureResult;
} finally {
settled = true;
if (operationTimer) clearTimeout(operationTimer);
if (uniquenessTimer) clearTimeout(uniquenessTimer);
if (sizePoll) clearInterval(sizePoll);
options.signal?.removeEventListener("abort", onAbort);
options.downloads.onDeterminingFilename.removeListener(determiningListener);
options.downloads.onCreated.removeListener(createdListener);
options.downloads.onChanged.removeListener(changedListener);
cdpSubscription.dispose();
for (const candidate of candidates.values()) {
clearTimeout(candidate.graceTimer);
if (candidate.item.id !== capturedId) suggestDefault(candidate);
}
if (!succeeded && capturedId !== undefined) {
const item = capturedItem ?? createdItems.get(capturedId);
try {
if (item?.state === "complete") {
await options.downloads.removeFile(capturedId);
} else {
await options.downloads.cancel(capturedId);
}
} catch {
if (failureResult?.data) failureResult.data.cleanup_state = "failed";
}
}
}
}
+64
View File
@@ -0,0 +1,64 @@
// Download orchestration: validate session/tab ownership, then delegate one
// browser-global chrome.downloads transaction to download-capture.ts.
import type { SessionManager } from "@/session-manager/manager";
import type { DownloadParams, DownloadResult, RpcError } from "@/transport/types";
import { captureBrowserDownload, chromeDownloadsApi, type DownloadsApi } from "./download-capture";
import { clickResolvedTarget, type InteractionDeps, resolveActionTarget } from "./interaction";
import { enforceAgentWindow, isRpcError, lookupSession, resolveTargetTab } from "./shared";
let downloadActive = false;
export type { DownloadsApi } from "./download-capture";
export interface DownloadDeps extends InteractionDeps {
downloads?: DownloadsApi;
}
export async function handleDownload(
manager: SessionManager,
params: DownloadParams,
deps: DownloadDeps,
): Promise<DownloadResult | RpcError> {
if (downloadActive) return { code: "invalid_params", message: "another bsk download is active" };
downloadActive = true;
try {
const ctx = lookupSession(manager, params, "download");
if (isRpcError(ctx)) return ctx;
const target = await resolveTargetTab(manager, ctx, params.tab_id, deps.tabsApi);
if (isRpcError(target)) return target;
const denied = enforceAgentWindow(ctx, target, "download");
if (denied) return denied;
if (!params.browser_relative_dir) {
return { code: "invalid_params", message: "download requires a daemon capability directory" };
}
const address = await resolveActionTarget(deps.cdp, ctx, target, params, "download");
if (isRpcError(address)) return address;
const capture = await captureBrowserDownload({
cdp: deps.cdp,
target: address.cdpTarget,
downloads: deps.downloads ?? chromeDownloadsApi,
browserRelativeDir: params.browser_relative_dir,
maxByteSize: params.max_byte_size,
timeoutMs: params.timeout_ms ?? 120_000,
signal: deps.signal,
expectedFrameId: address.frameId,
trigger: () => clickResolvedTarget(ctx, address, {}, deps),
});
if (isRpcError(capture)) return capture;
const { click, item } = capture;
return {
tab_id: target.tabId,
used_ref: click.used_ref,
used_selector: click.used_selector,
suggested_filename: item.filename.split(/[\\/]/).pop() ?? "download",
byte_size: item.fileSize >= 0 ? item.fileSize : item.totalBytes,
mime: item.mime || undefined,
danger: item.danger,
browser_path: item.filename,
};
} finally {
downloadActive = false;
}
}
+27 -1
View File
@@ -2,10 +2,23 @@
// rendering. Extension handlers attach reasons here; human-facing copy
// lives in bsk-cli `render_error.rs`.
import type { ErrorCode, RpcError, RpcErrorData, RpcErrorReason } from "@/transport/types";
import type {
ErrorCode,
RpcError,
RpcErrorData,
RpcErrorReason,
TransferCleanupState,
TransferEffectState,
} from "@/transport/types";
export type { RpcErrorData, RpcErrorReason };
export interface TransferErrorOptions {
effectState: TransferEffectState;
phase: string;
cleanupState?: TransferCleanupState;
}
export function rpcError(
code: ErrorCode,
reason: RpcErrorReason,
@@ -15,3 +28,16 @@ export function rpcError(
const data: RpcErrorData = { reason, ...extra };
return { code, message, data };
}
export function transferError(
code: ErrorCode,
reason: RpcErrorReason,
message: string,
options: TransferErrorOptions,
): RpcError {
return rpcError(code, reason, message, {
effect_state: options.effectState,
phase: options.phase,
...(options.cleanupState ? { cleanup_state: options.cleanupState } : {}),
});
}
@@ -0,0 +1,484 @@
// One upload transaction with an explicit browser-side commit boundary.
// Chrome interception prevents a native chooser from escaping automation;
// chooser events and a frame-scoped DOM probe are independent input-location
// signals, so event delivery is not required for standard file inputs.
import type { ClickResult, RpcError, TransferEffectState } from "@/transport/types";
import { transferError } from "./errors";
import type { ResolvedActionTarget } from "./interaction";
import { type CdpRunner, isRpcError, sendToCdpTarget } from "./shared";
const CLEANUP_TIMEOUT_MS = 1_000;
const ACTIVATION_GRACE_MS = 1_000;
const PROBE_INTERVAL_MS = 20;
type UploadPhase =
| "arm_interception"
| "arm_input_probe"
| "trigger"
| "resolve_input"
| "set_files"
| "cleanup";
interface RuntimeReply {
result?: { value?: unknown; objectId?: string };
exceptionDetails?: { text?: string; exception?: { description?: string } };
}
interface ChooserEvent {
frameId?: string;
backendNodeId?: number;
mode?: "selectSingle" | "selectMultiple";
}
export interface FileInputTransactionOptions {
cdp: CdpRunner;
actionTarget: ResolvedActionTarget;
files: string[];
timeoutMs: number;
signal?: AbortSignal;
trigger(): Promise<ClickResult | RpcError>;
}
export interface FileInputTransactionResult {
click: ClickResult;
multiple: boolean;
}
class BoundedWaitError extends Error {
constructor(
readonly kind: "timeout" | "aborted",
message: string,
) {
super(message);
}
}
function remainingMs(deadline: number): number {
return Math.max(0, deadline - Date.now());
}
async function waitBounded<T>(
promise: Promise<T>,
deadline: number,
signal: AbortSignal | undefined,
timeoutMessage: string,
): Promise<T> {
const remaining = remainingMs(deadline);
if (signal?.aborted) throw new BoundedWaitError("aborted", "upload transaction aborted");
if (remaining === 0) throw new BoundedWaitError("timeout", timeoutMessage);
let timer: ReturnType<typeof setTimeout> | undefined;
let onAbort: (() => void) | undefined;
const boundary = new Promise<never>((_resolve, reject) => {
timer = setTimeout(() => reject(new BoundedWaitError("timeout", timeoutMessage)), remaining);
if (signal) {
onAbort = () => reject(new BoundedWaitError("aborted", "upload transaction aborted"));
signal.addEventListener("abort", onAbort, { once: true });
}
});
try {
return await Promise.race([promise, boundary]);
} finally {
if (timer) clearTimeout(timer);
if (signal && onAbort) signal.removeEventListener("abort", onAbort);
}
}
function runtimeError(reply: RuntimeReply, fallback: string): Error | null {
if (!reply.exceptionDetails) return null;
return new Error(
reply.exceptionDetails.exception?.description ?? reply.exceptionDetails.text ?? fallback,
);
}
function transferFailure(
code: RpcError["code"],
reason: "file_input_probe_failed" | "file_input_not_activated" | "set_file_input_failed",
message: string,
effectState: TransferEffectState,
phase: UploadPhase,
): RpcError {
return transferError(code, reason, message, { effectState, phase });
}
function enrichFailure(
error: RpcError,
effectState: TransferEffectState,
phase: UploadPhase,
): RpcError {
return {
...error,
data: { ...error.data, effect_state: effectState, phase },
};
}
function sameTarget(
source: { tabId?: number; sessionId?: string },
target: { tabId: number; sessionId?: string },
): boolean {
return source.tabId === target.tabId && source.sessionId === target.sessionId;
}
async function delay(ms: number, signal?: AbortSignal): Promise<void> {
await waitBounded(
new Promise((resolve) => setTimeout(resolve, ms)),
Date.now() + ms + 1,
signal,
"upload activation probe timed out",
);
}
async function callOnTrigger<T = RuntimeReply>(
options: FileInputTransactionOptions,
objectId: string,
functionDeclaration: string,
args: unknown[],
returnByValue: boolean,
): Promise<T> {
return sendToCdpTarget<T>(options.cdp, options.actionTarget.cdpTarget, "Runtime.callFunctionOn", {
objectId,
functionDeclaration,
arguments: args.map((value) => ({ value })),
returnByValue,
awaitPromise: false,
});
}
export async function uploadThroughActivatedFileInput(
options: FileInputTransactionOptions,
): Promise<FileInputTransactionResult | RpcError> {
const deadline = Date.now() + options.timeoutMs;
const target = options.actionTarget.cdpTarget;
const objectGroup = `bsk-upload-${crypto.randomUUID()}`;
const stateKey = `__bskUpload_${crypto.randomUUID().replaceAll("-", "")}`;
const chooserEvents: ChooserEvent[] = [];
let interceptionArmed = false;
let probeArmed = false;
let triggerObjectId: string | undefined;
let outcome: FileInputTransactionResult | RpcError = transferFailure(
"protocol_error",
"file_input_probe_failed",
"upload transaction ended without an outcome",
"none",
"cleanup",
);
const chooserSubscription = options.cdp.onEvent?.((source, method, raw) => {
if (method !== "Page.fileChooserOpened" || !sameTarget(source, target)) return;
const event = raw as ChooserEvent;
chooserEvents.push(event);
});
try {
try {
await waitBounded(
sendToCdpTarget(options.cdp, target, "Page.setInterceptFileChooserDialog", {
enabled: true,
}),
deadline,
options.signal,
"arming native file chooser interception timed out",
);
interceptionArmed = true;
} catch (err) {
outcome = transferFailure(
err instanceof BoundedWaitError ? "timeout" : "cdp_failed",
"file_input_probe_failed",
err instanceof Error ? err.message : String(err),
"none",
"arm_interception",
);
return outcome;
}
try {
const resolved = await waitBounded(
sendToCdpTarget<{ object?: { objectId?: string } }>(
options.cdp,
target,
"DOM.resolveNode",
{
backendNodeId: options.actionTarget.backendNodeId,
objectGroup,
},
),
deadline,
options.signal,
"resolving upload trigger timed out",
);
triggerObjectId = resolved.object?.objectId;
if (!triggerObjectId) throw new Error("DOM.resolveNode returned no trigger objectId");
const armed = await waitBounded(
callOnTrigger<RuntimeReply>(
options,
triggerObjectId,
`function(key) {
const doc = this.ownerDocument;
const owner = doc.defaultView;
if (!owner) return false;
const state = { inputs: [], listener: null };
Object.defineProperty(owner, key, { value: state, configurable: true });
state.listener = event => {
const path = typeof event.composedPath === "function" ? event.composedPath() : [];
const candidate = path[0] || event.target;
if (candidate && candidate.nodeType === 1 &&
candidate.localName === "input" && candidate.type === "file") {
if (!state.inputs.includes(candidate)) state.inputs.push(candidate);
event.preventDefault();
}
};
doc.addEventListener("click", state.listener, true);
return true;
}`,
[stateKey],
true,
),
deadline,
options.signal,
"arming frame-scoped file input probe timed out",
);
const armError = runtimeError(armed, "failed to arm file input probe");
if (armError || armed.result?.value !== true) {
throw armError ?? new Error("file input probe did not arm");
}
probeArmed = true;
} catch (err) {
outcome = transferFailure(
err instanceof BoundedWaitError ? "timeout" : "cdp_failed",
"file_input_probe_failed",
err instanceof Error ? err.message : String(err),
"none",
"arm_input_probe",
);
return outcome;
}
let click: ClickResult | RpcError;
try {
click = await waitBounded(
options.trigger(),
deadline,
options.signal,
"upload trigger timed out",
);
} catch (err) {
outcome = transferFailure(
err instanceof BoundedWaitError ? "timeout" : "cdp_failed",
"file_input_probe_failed",
err instanceof Error ? err.message : String(err),
"none",
"trigger",
);
return outcome;
}
if (isRpcError(click)) {
outcome = enrichFailure(click, "none", "trigger");
return outcome;
}
try {
const activationDeadline = Math.min(deadline, Date.now() + ACTIVATION_GRACE_MS);
let summary: { count: number; multiple: boolean } = { count: 0, multiple: false };
while (Date.now() < activationDeadline) {
if (chooserEvents.length > 0) break;
const reply = await callOnTrigger<RuntimeReply>(
options,
triggerObjectId,
`function(key) {
const state = this.ownerDocument.defaultView?.[key];
return state
? { count: state.inputs.length, multiple: state.inputs[0]?.multiple === true }
: { count: 0, multiple: false };
}`,
[stateKey],
true,
);
const summaryError = runtimeError(reply, "failed to inspect activated file input");
if (summaryError) throw summaryError;
const value = reply.result?.value as { count?: unknown; multiple?: unknown } | undefined;
summary = {
count: typeof value?.count === "number" ? value.count : 0,
multiple: value?.multiple === true,
};
if (summary.count > 0) break;
await delay(Math.min(PROBE_INTERVAL_MS, remainingMs(activationDeadline)), options.signal);
}
if (chooserEvents.length > 1) {
outcome = transferFailure(
"unsupported",
"file_input_not_activated",
"upload trigger activated more than one file chooser",
"none",
"resolve_input",
);
return outcome;
}
const chooser = chooserEvents[0];
let backendNodeId: number | undefined;
let multiple = summary.multiple;
if (chooser) {
if (options.actionTarget.frameId && chooser.frameId !== options.actionTarget.frameId) {
outcome = transferFailure(
"unsupported",
"file_input_not_activated",
"upload trigger activated a file chooser in a different frame",
"none",
"resolve_input",
);
return outcome;
}
if (typeof chooser.backendNodeId !== "number") {
outcome = transferFailure(
"unsupported",
"file_input_not_activated",
"upload trigger invoked a non-input file picker",
"none",
"resolve_input",
);
return outcome;
}
backendNodeId = chooser.backendNodeId;
multiple = chooser.mode === "selectMultiple";
} else {
if (summary.count !== 1) {
outcome = transferFailure(
"unsupported",
"file_input_not_activated",
summary.count === 0
? "upload trigger did not activate an input[type=file]"
: "upload trigger activated more than one input[type=file]",
"none",
"resolve_input",
);
return outcome;
}
const input = await callOnTrigger<RuntimeReply>(
options,
triggerObjectId,
`function(key) { return this.ownerDocument.defaultView?.[key]?.inputs[0]; }`,
[stateKey],
false,
);
const inputError = runtimeError(input, "failed to resolve activated file input object");
if (inputError) throw inputError;
if (!input.result?.objectId) throw new Error("activated file input returned no objectId");
const described = await sendToCdpTarget<{ node?: { backendNodeId?: number } }>(
options.cdp,
target,
"DOM.describeNode",
{ objectId: input.result.objectId },
);
backendNodeId = described.node?.backendNodeId;
if (typeof backendNodeId !== "number") {
throw new Error("DOM.describeNode returned no file input backendNodeId");
}
}
if (!multiple && options.files.length !== 1) {
outcome = enrichFailure(
{ code: "invalid_params", message: "file input accepts exactly one file" },
"none",
"resolve_input",
);
return outcome;
}
try {
await waitBounded(
sendToCdpTarget(options.cdp, target, "DOM.setFileInputFiles", {
files: options.files,
backendNodeId,
}),
deadline,
options.signal,
"setting file input files timed out",
);
} catch (err) {
outcome = transferFailure(
err instanceof BoundedWaitError ? "timeout" : "cdp_failed",
"set_file_input_failed",
err instanceof Error ? err.message : String(err),
"unknown",
"set_files",
);
return outcome;
}
outcome = { click, multiple };
return outcome;
} catch (err) {
outcome = transferFailure(
err instanceof BoundedWaitError ? "timeout" : "cdp_failed",
"file_input_probe_failed",
err instanceof Error ? err.message : String(err),
"none",
"resolve_input",
);
return outcome;
}
} finally {
chooserSubscription?.dispose();
let cleanupFailed = false;
if (probeArmed && triggerObjectId) {
try {
await waitBounded(
callOnTrigger(
options,
triggerObjectId,
`function(key) {
const owner = this.ownerDocument.defaultView;
const state = owner?.[key];
if (state?.listener) this.ownerDocument.removeEventListener("click", state.listener, true);
if (owner) delete owner[key];
}`,
[stateKey],
true,
),
Date.now() + CLEANUP_TIMEOUT_MS,
undefined,
"cleaning file input probe timed out",
);
} catch {
cleanupFailed = true;
}
}
if (interceptionArmed) {
try {
await waitBounded(
sendToCdpTarget(options.cdp, target, "Page.setInterceptFileChooserDialog", {
enabled: false,
cancel: true,
}),
Date.now() + CLEANUP_TIMEOUT_MS,
undefined,
"disabling file chooser interception timed out",
);
} catch {
cleanupFailed = true;
}
}
try {
await waitBounded(
sendToCdpTarget(options.cdp, target, "Runtime.releaseObjectGroup", { objectGroup }),
Date.now() + CLEANUP_TIMEOUT_MS,
undefined,
"releasing upload object group timed out",
);
} catch {
cleanupFailed = true;
}
const effect = isRpcError(outcome)
? (outcome.data?.effect_state as TransferEffectState | undefined)
: "committed";
if (effect === "unknown" || cleanupFailed) {
await options.cdp.detach?.(target.tabId);
}
if (cleanupFailed && isRpcError(outcome)) {
outcome.data = { ...outcome.data, cleanup_state: "failed" };
}
}
}
+39 -9
View File
@@ -40,6 +40,7 @@ import {
enforceAgentWindow,
isRpcError,
lookupSession,
type ResolvedTargetTab,
resolveTargetTab,
} from "./shared";
import { resolveSnapshotRef } from "./snapshot-ref";
@@ -56,6 +57,15 @@ export interface InteractionDeps {
keepOverlayBypassAfterHover?: boolean;
}
export interface ResolvedActionTarget {
tab: ResolvedTargetTab;
backendNodeId: number;
cdpTarget: CdpTarget;
frameId?: string;
usedRef?: string;
usedSelector?: string;
}
const DEFAULT_TIMEOUT_MS = 30_000;
const DEFAULT_HOVER_SETTLE_MS = 200;
@@ -128,7 +138,7 @@ async function wait(ms: number, signal?: AbortSignal): Promise<void> {
* `RpcError` if the caller supplied neither (or both), or if neither
* lookup matched.
*/
async function resolveBackendNode(
export async function resolveBackendNode(
cdp: CdpRunner,
ctx: SessionContext,
target: { tabId: number },
@@ -215,6 +225,17 @@ async function resolveBackendNode(
}
}
export async function resolveActionTarget(
cdp: CdpRunner,
ctx: SessionContext,
target: ResolvedTargetTab,
params: { ref?: string; selector?: string },
toolName: string,
): Promise<ResolvedActionTarget | RpcError> {
const node = await resolveBackendNode(cdp, ctx, target, params, toolName);
return isRpcError(node) ? node : { tab: target, ...node };
}
// ---------------------------------------------------------------------------
// tool.click
// ---------------------------------------------------------------------------
@@ -233,10 +254,19 @@ export async function handleClick(
if (isRpcError(target)) return target;
const denied = enforceAgentWindow(ctx, target, "click");
if (denied) return denied;
const dialogCursor = markDialogCursor(deps.cdp, target.tabId);
const resolved = await resolveActionTarget(deps.cdp, ctx, target, params, "click");
if (isRpcError(resolved)) return resolved;
return clickResolvedTarget(ctx, resolved, params, deps);
}
const node = await resolveBackendNode(deps.cdp, ctx, target, params, "click");
if (isRpcError(node)) return node;
export async function clickResolvedTarget(
ctx: SessionContext,
resolved: ResolvedActionTarget,
params: Pick<ClickParams, "button" | "click_count" | "modifiers">,
deps: InteractionDeps,
): Promise<ClickResult | RpcError> {
const { tab: target } = resolved;
const dialogCursor = markDialogCursor(deps.cdp, target.tabId);
if (throwIfAborted(deps.signal)) {
return { code: "cancelled", message: "click aborted" };
@@ -247,9 +277,9 @@ export async function handleClick(
deps.cdp,
target.tabId,
{
target: node.cdpTarget,
backendNodeId: node.backendNodeId,
...(node.frameId ? { frameId: node.frameId } : {}),
target: resolved.cdpTarget,
backendNodeId: resolved.backendNodeId,
...(resolved.frameId ? { frameId: resolved.frameId } : {}),
},
{ scrollIntoView: true },
);
@@ -337,8 +367,8 @@ export async function handleClick(
return attachDialogs(deps.cdp, target.tabId, dialogCursor, {
tab_id: target.tabId,
used_ref: node.usedRef,
used_selector: node.usedSelector,
used_ref: resolved.usedRef,
used_selector: resolved.usedSelector,
x: centre.x,
y: centre.y,
});
+45 -81
View File
@@ -10,7 +10,6 @@ import {
isVomReferenceNode,
renderVom,
type VomNode,
type VomOptions,
type VomScene,
} from "@browser-skill/vom";
import { ChromiumCdp } from "@/browser-driver/chromium-cdp";
@@ -54,10 +53,6 @@ import {
collectOverlayExcludedBackendIds,
} from "./vom/capture";
import { type CapturedFrameDocument, captureFrameData } from "./vom/frame-capture";
import {
type CaptureVomObservationResult,
projectRecordSafeObservation,
} from "./vom/record-safe-observation";
// ---------------------------------------------------------------------------
// Shared helpers (legacy aliases — observation.ts kept exporting these
@@ -416,15 +411,7 @@ function isModalSignal(
function isSensitive(capturedNode: CapturedNode | undefined): boolean {
const attrs = capturedNode?.attrs ?? {};
const type = (attrs.type ?? "").toLowerCase();
if (type === "password") return true;
const autocomplete = (attrs.autocomplete ?? "").toLowerCase();
return (
autocomplete.startsWith("cc-") ||
autocomplete === "one-time-code" ||
autocomplete === "current-password" ||
autocomplete === "new-password"
);
return (attrs.type ?? "").toLowerCase() === "password";
}
interface AxNodeSignals {
@@ -474,13 +461,7 @@ const AX_TEXT_STOP_ROLES = new Set([
"columnheader",
"rowheader",
]);
const SENSITIVE_AX_INPUT_TYPES = new Set([
"password",
"credit-card",
"one-time-code",
"current-password",
"new-password",
]);
const SENSITIVE_AX_INPUT_TYPES = new Set(["password", "credit-card"]);
function axPropertyString(axNode: CdpAxNode, name: string): string | undefined {
const prop = axNode.properties?.find((item) => item.name === name);
@@ -1556,13 +1537,14 @@ async function fallbackCapturedViewModel(
try {
const metrics = await cdp.send<LayoutMetricsViewportReply>(tabId, "Page.getLayoutMetrics", {});
throwIfAborted(signal, "observation");
const source = metrics.cssLayoutViewport ?? metrics.layoutViewport ?? {};
const vpSrc = metrics.cssLayoutViewport ?? metrics.layoutViewport ?? {};
viewport = {
width: source.clientWidth ?? 0,
height: source.clientHeight ?? 0,
width: vpSrc.clientWidth ?? 0,
height: vpSrc.clientHeight ?? 0,
};
} catch (error) {
if (isAbortError(error)) throw error;
} catch (err) {
if (isAbortError(err)) throw err;
// viewport stays zero-sized
}
const excludedBackendNodeIds = await collectOverlayExcludedBackendIds(cdp, tabId, signal);
throwIfAborted(signal, "observation");
@@ -1572,58 +1554,22 @@ async function fallbackCapturedViewModel(
async function captureForVom(
cdp: CdpRunner,
tabId: number,
options: CaptureVomObservationOptions,
conditionalSurfaceProbe: boolean,
hoverProbeBypassOverlay?: (tabId: number, enabled: boolean) => Promise<void>,
signal?: AbortSignal,
): Promise<CapturedViewModel> {
try {
return await captureViewModel(cdp, tabId, {
conditionalSurfaceProbe: options.conditionalSurfaceProbe ?? false,
hoverProbeBypassOverlay: options.hoverProbeBypassOverlay,
signal: options.signal,
conditionalSurfaceProbe,
hoverProbeBypassOverlay,
signal,
});
} catch (error) {
if (isAbortError(error)) throw error;
return fallbackCapturedViewModel(cdp, tabId, options.signal);
} catch (err) {
if (isAbortError(err)) throw err;
return fallbackCapturedViewModel(cdp, tabId, signal);
}
}
export interface CaptureVomObservationOptions extends VomOptions {
conditionalSurfaceProbe?: boolean;
hoverProbeBypassOverlay?: (tabId: number, enabled: boolean) => Promise<void>;
signal?: AbortSignal;
}
export async function captureVomObservation(
cdp: CdpRunner,
tabId: number,
url: string | undefined,
options: CaptureVomObservationOptions = {},
): Promise<CaptureVomObservationResult> {
throwIfAborted(options.signal, "observation");
await cdp.ensureAttachedToUrl?.(tabId, url);
throwIfAborted(options.signal, "observation");
const captured = await captureForVom(cdp, tabId, options);
throwIfAborted(options.signal, "observation");
const documents = await captureFrameData<CdpAxNode>(cdp, tabId, captured, options.signal);
throwIfAborted(options.signal, "observation");
const scene =
captured.rootFrameId && captured.frameNodes?.size
? buildFrameVomScene(documents, captured, { pageUrl: url })
: buildVomScene(documents[0]?.axNodes ?? [], captured, { pageUrl: url });
const rendered = renderVom(scene, {
maxDepth: options.maxDepth,
maxTokens: options.maxTokens,
redactValues: options.redactValues,
activeRegionPolicy: options.activeRegionPolicy,
});
throwIfAborted(options.signal, "observation");
return projectRecordSafeObservation({
rootFrameId: captured.rootFrameId ?? documents[0]?.frameId ?? "root",
frameDocuments: documents,
rendered,
surfaceProbes: captured.surfaceProbes,
});
}
async function handleVomObservation(
manager: SessionManager,
params: SnapshotParams | ObserveParams,
@@ -1653,22 +1599,40 @@ async function handleVomObservation(
try {
throwIfAborted(signal, toolName);
deps.cdp.trackSessionTab?.(ctx.sessionId, target.tabId);
await deps.cdp.ensureAttachedToUrl?.(target.tabId, target.url);
throwIfAborted(signal, toolName);
const effectiveConditionalSurfaceProbe =
deps.conditionalSurfaceProbe ?? conditionalSurfaceProbe;
const observation = await captureVomObservation(deps.cdp, target.tabId, target.url, {
const captured = await captureForVom(
deps.cdp,
target.tabId,
effectiveConditionalSurfaceProbe,
deps.hoverProbeBypassOverlay,
signal,
);
throwIfAborted(signal, toolName);
const frameDocuments = await captureFrameData<CdpAxNode>(
deps.cdp,
target.tabId,
captured,
signal,
);
throwIfAborted(signal, toolName);
const scene =
captured.rootFrameId && captured.frameNodes?.size
? buildFrameVomScene(frameDocuments, captured, { pageUrl: target.url })
: buildVomScene(frameDocuments[0]?.axNodes ?? [], captured, { pageUrl: target.url });
const rendered = renderVom(scene, {
maxDepth: params.max_depth,
maxTokens: params.max_tokens,
activeRegionPolicy: true,
conditionalSurfaceProbe: effectiveConditionalSurfaceProbe,
hoverProbeBypassOverlay: deps.hoverProbeBypassOverlay,
signal,
});
throwIfAborted(signal, toolName);
const targetByFrameId = new Map(
observation.frames.map((frame) => [frame.frameId, frame.target]),
frameDocuments.map((document) => [document.frameId, document.target]),
);
ctx.refStore.replace(
observation.refs.map((ref) => {
rendered.refs.map((ref) => {
const refTarget = ref.frameId ? targetByFrameId.get(ref.frameId) : undefined;
return [
ref.ref,
@@ -1682,14 +1646,14 @@ async function handleVomObservation(
}),
);
return attachDialogs(deps.cdp, target.tabId, dialogCursor, {
text: observation.text,
ref_count: observation.refs.length,
text: rendered.text,
ref_count: rendered.refs.length,
tab_id: target.tabId,
truncated: observation.truncated,
truncated: rendered.truncated,
...(toolName === "observe" && (params as ObserveParams).debug_surfaces
? {
debug: {
surface_probes: (observation.surfaceProbes ?? []).map((probe) => ({
surface_probes: (captured.surfaceProbes ?? []).map((probe) => ({
trigger_backend_node_id: probe.triggerBackendNodeId,
...(probe.triggerPoint ? { trigger_point: probe.triggerPoint } : {}),
trigger_action: probe.triggerAction,
+73 -390
View File
@@ -15,37 +15,22 @@ import {
type RecordQueryResponse,
type RecordStartAck,
type RecordStartMessage,
type RecordStepAck,
type RecordStopMessage,
} from "@/lib/record-bridge";
import {
type RecordFrameCoordinator,
type RecordingCaptureScope,
recordFrameCoordinator,
} from "@/lib/recording/frame-coordinator";
import { RecordingObservationRuntime } from "@/lib/recording/recording-runtime";
import {
appendRecordedPayload,
observeRecordedNavigation,
type RecordingStepBuffer,
} from "@/lib/recording/step-buffer";
import { RecordingTabCoordinator, type TabActivation } from "@/lib/recording/tab-coordinator";
import { buildTraceV2 } from "@/lib/recording/trace-reducer-v2";
import type { RecordingDraftStep } from "@/lib/recording/types";
import { appendRecordedPayload, observeRecordedNavigation } from "@/lib/recording-step-buffer";
import { reduceTraceSteps, resolveTraceStartUrl } from "@/lib/trace-reducer";
import type { SessionManager } from "@/session-manager/manager";
import { EXTENSION_VERSION } from "@/transport/handshake";
import type {
DraftTraceStep,
RecordAwaitParams,
RecordAwaitResult,
RecordedTrace,
RecordStartParams,
RecordStartResult,
RecordStopParams,
RecordStopResult,
RpcError,
StopReason,
Trace,
} from "@/transport/types";
import { TRACE_VERSION_V3 } from "@/transport/types";
import { handleNavigate } from "./navigation";
import {
type CdpRunner,
@@ -58,46 +43,21 @@ import {
interface ActiveRecording {
requestId: string;
tabs: RecordingTabCoordinator;
tabId: number;
agentWindowId: number;
startUrl?: string;
purpose?: string;
steps: RecordingDraftStep[];
steps: DraftTraceStep[];
startedAt: string;
startedAtMs: number;
traceVersion: 2 | 3;
supportsTabSwitchSteps: boolean;
finishPromise: Promise<RecordedTrace>;
resolveFinish: (trace: RecordedTrace) => void;
finishPromise: Promise<Trace>;
resolveFinish: (trace: Trace) => void;
rejectFinish: (err: Error) => void;
settled: boolean;
finishAttempt: Promise<RecordedTrace | null> | null;
observation: RecordingObservationRuntime | null;
stoppedBy: StopReason;
/** Navigation callbacks tracked from event receipt through action enqueue. */
navigationCallbacks: Set<Promise<void>>;
/** Synchronous intake gate closed only after finish drains to stability. */
acceptingNavigation: boolean;
/**
* Serializes step appends with navigation observation so a click is always
* in `steps` before a same-turn `webNavigation` tries to annotate it.
*/
actionQueue: Promise<void>;
/** Last accepted sequence for each content-script document producer. */
lastStepSequenceByProducer: Map<string, number>;
}
function enqueueRecordingAction(
recording: ActiveRecording,
task: () => Promise<void>,
): Promise<void> {
const queued = recording.actionQueue.then(task, task);
recording.actionQueue = queued.catch(() => {});
return queued;
}
function isRecordingFinishing(recording: ActiveRecording): boolean {
return recording.settled || recording.finishAttempt !== null;
finishing: boolean;
currentUrl?: string;
pendingNavigation: boolean;
pendingNavigationDeadline?: number;
}
const recordings = new Map<string, ActiveRecording>();
@@ -118,9 +78,6 @@ function sleep(ms: number): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, ms));
}
/** Recording producer version mirrored into trace.recorder.bsk. */
export const BSK_TRACE_VERSION = EXTENSION_VERSION;
/** Injectable http(s) landing page when `tool.record_start` omits `url`. */
export const RECORD_DEFAULT_START_URL = "https://example.com/";
@@ -180,11 +137,9 @@ async function sendRecordStartWithAck(
tabId: number,
msg: RecordStartMessage,
sendToTab: RecordDeps["sendToTab"],
cancelled?: () => boolean,
): Promise<void> {
let lastError: unknown;
for (let attempt = 0; attempt < RECORD_START_RETRIES; attempt += 1) {
if (cancelled?.()) throw new Error("record start cancelled");
try {
const response = await sendToTab(tabId, msg);
if (isRecordStartAck(response)) return;
@@ -192,7 +147,6 @@ async function sendRecordStartWithAck(
} catch (err) {
lastError = err;
}
if (cancelled?.()) throw new Error("record start cancelled");
if (attempt + 1 < RECORD_START_RETRIES) {
await sleep(RECORD_START_RETRY_DELAY_MS);
}
@@ -200,83 +154,19 @@ async function sendRecordStartWithAck(
throw lastError ?? new Error("failed to start recording in content script");
}
function negotiatedTraceVersion(params: RecordStartParams): 2 | 3 | RpcError {
if (params.trace_version === undefined) return 2;
if (params.trace_version === TRACE_VERSION_V3) return 3;
function buildTrace(recording: ActiveRecording): Trace {
const { pages, steps } = reduceTraceSteps(recording.steps, recording.startUrl);
const startUrl = resolveTraceStartUrl(recording.steps, recording.startUrl, pages);
return {
code: "invalid_params",
message: `unsupported trace_version ${params.trace_version}; supported values are omitted (v2) or ${TRACE_VERSION_V3} (v3)`,
};
}
function buildTrace(recording: ActiveRecording): RecordedTrace {
if (recording.traceVersion === 3 && recording.observation) {
return recording.observation.buildTrace({
drafts: recording.steps,
startedAt: recording.startedAt,
purpose: recording.purpose,
startUrl: recording.startUrl,
stoppedBy: recording.stoppedBy,
bskVersion: BSK_TRACE_VERSION,
includeTabSwitches: recording.supportsTabSwitchSteps,
});
}
if (recording.traceVersion === 3) {
throw new Error("trace v3 observation runtime is unavailable");
}
return buildTraceV2({
steps: recording.steps,
startedAt: recording.startedAt,
...(recording.startUrl ? { startUrl: recording.startUrl } : {}),
...(recording.purpose ? { purpose: recording.purpose } : {}),
});
}
function stepBufferFor(
recording: ActiveRecording,
tabId: number,
fallbackUrl?: string,
): RecordingStepBuffer {
return {
steps: recording.steps,
navigation: recording.tabs.navigation(tabId, fallbackUrl),
recorded_at: new Date().toISOString(),
started_at: recording.startedAt,
entry: { start_url: startUrl },
pages,
steps,
};
}
async function activateRecordingTab(
recording: ActiveRecording,
targetTabId: number,
): Promise<void> {
const previousTabId = recording.tabs.currentTabId;
if (targetTabId === previousTabId) return;
const transition = recording.observation
? await recording.observation.captureTabTransition(previousTabId, targetTabId)
: {};
const { targetUrl, ...stateLinks } = transition;
const draft: Extract<RecordingDraftStep, { op: "switch_tab" }> = {
op: "switch_tab",
...stateLinks,
};
recording.steps.push(draft);
recording.observation?.bindTabTransition(draft, recording.steps.length);
recording.tabs.commit(targetTabId, targetUrl);
}
async function processRecordedStep(
recording: ActiveRecording,
draftIndex: number,
tabId: number,
producerId?: string,
): Promise<void> {
if (!recording.observation) return;
try {
await recording.observation.processDraft(tabId, recording.steps, draftIndex, producerId);
} catch (err) {
console.warn(`[bsk record] observation failed for step ${draftIndex + 1}`, err);
}
}
export interface RecordDeps {
tabsApi: ChromeTabsApi;
sendToTab(
@@ -284,25 +174,16 @@ export interface RecordDeps {
msg: RecordStartMessage | RecordStopMessage | RecordCancelMessage,
): Promise<unknown>;
bypassOverlay?: (tabId: number, enabled: boolean) => Promise<void>;
frameCoordinator?: Pick<
RecordFrameCoordinator,
"begin" | "armTab" | "sourceFor" | "stop" | "cancel"
>;
cdp?: CdpRunner;
signal?: AbortSignal;
}
export type RecordRuntimeDeps = Omit<RecordDeps, "frameCoordinator" | "signal"> & {
frameCoordinator: NonNullable<RecordDeps["frameCoordinator"]>;
};
let defaultDeps: RecordDeps | null = null;
function getDefaultDeps(): RecordDeps {
if (!defaultDeps) {
defaultDeps = {
tabsApi: chromeTabsApi,
sendToTab: (tabId, msg) => chrome.tabs.sendMessage(tabId, msg),
frameCoordinator: recordFrameCoordinator,
};
}
return defaultDeps;
@@ -360,58 +241,16 @@ export function releaseBrowserObservationListenersIfIdle(): void {
detachBrowserObservation = null;
}
export function attachRecordStepListener(deps: RecordDeps = getDefaultDeps()): () => void {
export function attachRecordStepListener(): () => void {
const listener = (
message: unknown,
sender: chrome.runtime.MessageSender,
sendResponse: (response: RecordStepAck) => void,
_sender: chrome.runtime.MessageSender,
_sendResponse: () => void,
) => {
if (!isRecordStepMessage(message)) return false;
for (const recording of recordings.values()) {
if (recording.requestId !== message.requestId) continue;
const source: RecordingCaptureScope | null | undefined = deps.frameCoordinator
? deps.frameCoordinator.sourceFor(message.requestId, message.producerId, sender)
: undefined;
if (deps.frameCoordinator && !source) return false;
const sourceTabId = source?.tabId ?? sender.tab?.id ?? recording.tabs.currentTabId;
const sourceWasActive = sender.tab?.active ?? sourceTabId === recording.tabs.activeTabId;
const producerKey = source
? `${source.tabId}:${source.documentId}:${source.producerId}`
: `${sourceTabId}:${message.producerId}`;
const expectedSequence = (recording.lastStepSequenceByProducer.get(producerKey) ?? 0) + 1;
if (message.sequence < expectedSequence) {
sendResponse({ ok: true, sequence: message.sequence });
return false;
}
if (message.sequence > expectedSequence) {
sendResponse({
ok: false,
expectedSequence,
error: `out-of-order recorded step ${message.sequence}`,
});
return false;
}
recording.lastStepSequenceByProducer.set(producerKey, message.sequence);
enqueueRecordingAction(recording, async () => {
if (sourceTabId !== recording.tabs.currentTabId) {
if (!sourceWasActive) return;
if (sourceTabId !== recording.tabs.activeTabId)
recording.tabs.noteActivation(sourceTabId);
await activateRecordingTab(recording, sourceTabId);
}
await recording.observation?.flushRedirects(sourceTabId);
const targetHint = message.step.geometry ? { geometry: message.step.geometry } : undefined;
const draftIndex = appendRecordedPayload(
stepBufferFor(recording, sourceTabId, message.step.page_url),
message.step,
targetHint,
);
if (draftIndex !== null) {
await processRecordedStep(recording, draftIndex, sourceTabId, source?.producerId);
}
});
sendResponse({ ok: true, sequence: message.sequence });
appendRecordedPayload(recording, message.step);
return false;
}
return false;
@@ -438,7 +277,7 @@ export function attachRecordFinishListener(deps: RecordDeps = getDefaultDeps()):
function findRecordingByTabId(tabId: number): ActiveRecording | null {
for (const recording of recordings.values()) {
if (recording.tabs.currentTabId === tabId && !recording.settled) return recording;
if (recording.tabId === tabId && !recording.settled) return recording;
}
return null;
}
@@ -475,7 +314,7 @@ async function clearRearmTimersForRecording(
recording: ActiveRecording,
deps: RecordDeps,
): Promise<void> {
clearRearmTimer(recording.tabs.currentTabId);
clearRearmTimer(recording.tabId);
try {
const tabs = await deps.tabsApi.query({ windowId: recording.agentWindowId });
for (const tab of tabs) {
@@ -490,16 +329,13 @@ async function stopRecordingOnAllAgentTabs(
recording: ActiveRecording,
deps: RecordDeps,
): Promise<void> {
if (deps.frameCoordinator && !(await deps.frameCoordinator.stop(recording.requestId))) {
throw new Error("failed to flush one or more recording documents");
}
const stopMsg: RecordStopMessage = { type: RECORD_STOP, requestId: recording.requestId };
let tabIds = [recording.tabs.currentTabId];
let tabIds = [recording.tabId];
try {
const tabs = await deps.tabsApi.query({ windowId: recording.agentWindowId });
tabIds = [
...new Set([
recording.tabs.currentTabId,
recording.tabId,
...tabs.flatMap((tab) => (typeof tab.id === "number" ? [tab.id] : [])),
]),
];
@@ -510,11 +346,11 @@ async function stopRecordingOnAllAgentTabs(
for (const tabId of tabIds) {
try {
const response = await deps.sendToTab(tabId, stopMsg);
if (tabId === recording.tabs.currentTabId && !isRecordStartAck(response)) {
if (tabId === recording.tabId && !isRecordStartAck(response)) {
throw new Error("content script did not confirm recorded steps");
}
} catch {
if (tabId === recording.tabs.currentTabId) {
if (tabId === recording.tabId) {
throw new Error("failed to flush recorded steps");
}
}
@@ -532,39 +368,22 @@ async function rearmRecording(
recording: ActiveRecording,
targetTabId: number,
deps: RecordDeps,
activation?: TabActivation,
): Promise<boolean> {
// Do NOT toggle automation-bypass here: each retry used to increment the
// content-script counter, and a single stop decrement left the ControlOverlay
// stuck with pointer-events:none (page usable, Interrupt dead). RecordOverlay
// already hides the control chrome while activeRecord is set.
const isFinishing = () => isRecordingFinishing(recording);
for (let attempt = 0; attempt < RECORD_REARM_MAX_ATTEMPTS; attempt += 1) {
if (isFinishing()) return false;
const startMsg: RecordStartMessage = {
type: RECORD_START,
requestId: recording.requestId,
startedAtMs: recording.startedAtMs,
};
try {
if (deps.frameCoordinator) {
const frameStarted = await deps.frameCoordinator.armTab(recording.requestId, targetTabId);
if (!frameStarted) throw new Error("recording document did not start");
}
await sendRecordStartWithAck(targetTabId, startMsg, deps.sendToTab, isFinishing);
if (isFinishing()) return false;
if (activation) {
if (!recording.tabs.isLatest(activation)) return true;
await enqueueRecordingAction(recording, async () => {
if (isFinishing() || !recording.tabs.isLatest(activation)) {
return;
}
await activateRecordingTab(recording, targetTabId);
});
}
await sendRecordStartWithAck(targetTabId, startMsg, deps.sendToTab);
recording.tabId = targetTabId;
return true;
} catch {
if (isFinishing()) return false;
if (attempt + 1 < RECORD_REARM_MAX_ATTEMPTS) {
await sleep(RECORD_REARM_RETRY_DELAY_MS);
}
@@ -573,7 +392,7 @@ async function rearmRecording(
return false;
}
function scheduleRearmForTab(tabId: number, deps: RecordDeps, activation?: TabActivation): void {
function scheduleRearmForTab(tabId: number, deps: RecordDeps): void {
const existing = rearmTimers.get(tabId);
if (existing) clearTimeout(existing);
rearmTimers.set(
@@ -582,7 +401,7 @@ function scheduleRearmForTab(tabId: number, deps: RecordDeps, activation?: TabAc
rearmTimers.delete(tabId);
void (async () => {
const current = await findRecordingForTab(tabId, deps);
if (current) await rearmRecording(current, tabId, deps, activation);
if (current) await rearmRecording(current, tabId, deps);
})();
}, RECORD_REARM_DEBOUNCE_MS),
);
@@ -594,20 +413,14 @@ export function attachRecordTabListener(deps: RecordDeps = getDefaultDeps()): ()
const windowId = tab.windowId;
if (tabId === undefined || windowId === undefined) return;
for (const recording of recordings.values()) {
if (isRecordingFinishing(recording) || recording.agentWindowId !== windowId) continue;
if (recording.settled || recording.agentWindowId !== windowId) continue;
scheduleRearmForTab(tabId, deps);
return;
}
};
const onActivated = (activeInfo: chrome.tabs.TabActiveInfo) => {
for (const recording of recordings.values()) {
if (isRecordingFinishing(recording) || recording.agentWindowId !== activeInfo.windowId)
continue;
const activation = recording.tabs.noteActivation(activeInfo.tabId);
scheduleRearmForTab(activeInfo.tabId, deps, activation);
return;
}
scheduleRearmForTab(activeInfo.tabId, deps);
};
chrome.tabs.onCreated.addListener(onCreated);
@@ -619,47 +432,13 @@ export function attachRecordTabListener(deps: RecordDeps = getDefaultDeps()): ()
}
export function attachRecordNavigationListener(deps: RecordDeps = getDefaultDeps()): () => void {
const observeMainFrame = (
tabId: number,
url?: string,
causedByAction?: boolean,
transitionType?: string,
transitionQualifiers?: string[],
) => {
if (!url) return;
const candidates = [...recordings.values()].filter(
(recording) =>
!recording.settled && recording.acceptingNavigation && recording.tabs.activeTabId === tabId,
);
for (const recording of candidates) {
const queued = enqueueRecordingAction(recording, async () => {
if (tabId !== recording.tabs.currentTabId) {
await activateRecordingTab(recording, tabId);
}
const result = observeRecordedNavigation(
stepBufferFor(recording, tabId, url),
url,
causedByAction,
transitionType,
transitionQualifiers,
);
if (result.kind === "coalesce_redirect") {
recording.observation?.scheduleRedirect(tabId, recording.steps, result.url);
return;
}
if (result.kind === "noop") return;
recording.observation?.clearRedirect(tabId);
if (result.kind === "appended") {
await processRecordedStep(recording, result.index, tabId);
} else {
recording.observation?.scheduleSettle(tabId, recording.steps, result.index);
}
});
const tracked = queued.catch(() => {});
recording.navigationCallbacks.add(tracked);
void tracked.finally(() => recording.navigationCallbacks.delete(tracked));
}
const observeMainFrame = (tabId: number, url?: string, causedByAction?: boolean) => {
void (async () => {
const recording = await findRecordingForTab(tabId, deps);
if (recording && url) {
observeRecordedNavigation(recording, url, causedByAction);
}
})();
};
const onMainFrameComplete = (tabId: number, url?: string) => {
void (async () => {
@@ -679,13 +458,18 @@ export function attachRecordNavigationListener(deps: RecordDeps = getDefaultDeps
details: chrome.webNavigation.WebNavigationTransitionCallbackDetails,
) => {
if (details.frameId !== 0) return;
observeMainFrame(
details.tabId,
details.url,
undefined,
details.transitionType,
details.transitionQualifiers,
);
const causedByAction =
details.transitionType === "link" || details.transitionType === "form_submit"
? true
: details.transitionType === "typed" ||
details.transitionType === "auto_bookmark" ||
details.transitionType === "generated" ||
details.transitionType === "keyword" ||
details.transitionType === "keyword_generated" ||
details.transitionType === "reload"
? false
: undefined;
observeMainFrame(details.tabId, details.url, causedByAction);
};
chrome.webNavigation.onCompleted.addListener(completedListener);
chrome.webNavigation.onCommitted?.addListener(committedListener);
@@ -703,26 +487,6 @@ export function attachRecordNavigationListener(deps: RecordDeps = getDefaultDeps
return () => chrome.tabs.onUpdated.removeListener(listener);
}
const MAX_FINISH_DRAIN_ROUNDS = 10;
async function drainRecordingToStability(recording: ActiveRecording): Promise<boolean> {
for (let round = 0; round < MAX_FINISH_DRAIN_ROUNDS; round += 1) {
await Promise.all([...recording.navigationCallbacks]);
const actionTail = recording.actionQueue;
await actionTail;
await recording.observation?.flush();
if (recording.navigationCallbacks.size === 0 && recording.actionQueue === actionTail) {
// No event or promise continuation can interleave with this synchronous
// check-and-close, so work accepted before the cutoff is fully drained.
recording.acceptingNavigation = false;
return true;
}
}
console.warn("[bsk record] navigation/action queues did not stabilize at stop");
return false;
}
export function attachRecordQueryListener(deps: RecordDeps = getDefaultDeps()): () => void {
const listener = (
message: unknown,
@@ -741,8 +505,7 @@ export function attachRecordQueryListener(deps: RecordDeps = getDefaultDeps()):
sendResponse({ active: false });
return;
}
const activation = sender.tab?.active ? recording.tabs.noteActivation(tabId) : undefined;
await rearmRecording(recording, tabId, deps, activation);
await rearmRecording(recording, tabId, deps);
sendResponse({
active: true,
requestId: recording.requestId,
@@ -764,50 +527,23 @@ async function finishRecordingByRequest(
if (recording.requestId !== requestId || recording.settled) continue;
const match = await findRecordingForTab(tabId, deps);
if (match !== recording) continue;
await finishRecording(sessionId, deps, "user_finish");
await finishRecording(sessionId, deps);
return;
}
}
async function finishRecording(
sessionId: string,
deps: RecordDeps,
stoppedBy: StopReason,
): Promise<RecordedTrace | null> {
async function finishRecording(sessionId: string, deps: RecordDeps): Promise<Trace | null> {
const recording = recordings.get(sessionId);
if (!recording || recording.settled) return null;
if (recording.finishAttempt) return recording.finishAttempt;
recording.stoppedBy = stoppedBy;
if (!recording || recording.settled || recording.finishing) return null;
recording.finishing = true;
const attempt = finishRecordingAttempt(sessionId, recording, deps);
recording.finishAttempt = attempt;
try {
return await attempt;
} finally {
if (!recording.settled) {
recording.finishAttempt = null;
}
}
}
async function finishRecordingAttempt(
sessionId: string,
recording: ActiveRecording,
deps: RecordDeps,
): Promise<RecordedTrace | null> {
await clearRearmTimersForRecording(recording, deps);
try {
// Disposing content capture may commit a final dirty fill. Flush content
// first so all resulting RECORD_STEP messages enter actionQueue before it
// and the observation queues are drained.
await stopRecordingOnAllAgentTabs(recording, deps);
} catch {
recording.finishing = false;
return null;
}
if (!(await drainRecordingToStability(recording))) {
return null;
}
await recording.observation?.settleTrailing(recording.tabs.currentTabId, recording.steps);
recording.settled = true;
recordings.delete(sessionId);
@@ -822,15 +558,6 @@ export async function handleRecordStart(
params: RecordStartParams,
deps: RecordDeps = getDefaultDeps(),
): Promise<RecordStartResult | RpcError> {
const traceVersionOrErr = negotiatedTraceVersion(params);
if (typeof traceVersionOrErr !== "number") return traceVersionOrErr;
if (traceVersionOrErr === 3 && !deps.cdp) {
return {
code: "protocol_error",
message: "trace v3 recording requires an active CDP connection",
};
}
const ctxOrErr = lookupSession(manager, params, "record_start");
if (isRpcError(ctxOrErr)) return ctxOrErr;
const ctx = ctxOrErr;
@@ -847,64 +574,37 @@ export async function handleRecordStart(
// on the destination page can RECORD_QUERY → rearm → show RecordOverlay
// instead of flashing ControlOverlay ("Agent 正在控制").
const requestId = makeRequestId(target.tabId);
let resolveFinish!: (trace: RecordedTrace) => void;
let resolveFinish!: (trace: Trace) => void;
let rejectFinish!: (err: Error) => void;
const finishPromise = new Promise<RecordedTrace>((resolve, reject) => {
const finishPromise = new Promise<Trace>((resolve, reject) => {
resolveFinish = resolve;
rejectFinish = reject;
});
const navigateUrl = params.url ?? RECORD_DEFAULT_START_URL;
const startedAtMs = Date.now();
const maxPageTokens = params.max_page_tokens;
const redactValues = params.redact_values ?? false;
recordings.set(params.session_id, {
requestId,
tabs: new RecordingTabCoordinator(target.tabId, navigateUrl),
tabId: target.tabId,
agentWindowId: ctx.agentWindowId,
startUrl: navigateUrl,
...(params.purpose ? { purpose: params.purpose } : {}),
steps: [],
startedAt: new Date(startedAtMs).toISOString(),
startedAtMs,
traceVersion: traceVersionOrErr,
supportsTabSwitchSteps: params.supports_tab_switch_steps === true,
finishPromise,
resolveFinish,
rejectFinish,
settled: false,
finishAttempt: null,
observation:
traceVersionOrErr === 3 && deps.cdp
? new RecordingObservationRuntime({
cdp: deps.cdp,
tabsApi: deps.tabsApi,
maxTokens: maxPageTokens,
redactValues,
})
: null,
stoppedBy: "user_finish",
navigationCallbacks: new Set(),
acceptingNavigation: true,
actionQueue: Promise.resolve(),
lastStepSequenceByProducer: new Map(),
});
deps.frameCoordinator?.begin(requestId, startedAtMs, target.tabId, async (scope) => {
const recording = recordings.get(params.session_id);
if (!recording || recording.requestId !== requestId || recording.settled) return;
try {
await recording.observation?.refreshDocument(scope.tabId, scope.producerId);
} catch {
// The normal action-time capture remains available if a child Document
// is replaced while its readiness refresh is in flight.
}
finishing: false,
currentUrl: navigateUrl,
pendingNavigation: false,
pendingNavigationDeadline: undefined,
});
// Observe navigations for the whole recording lifetime; attach before
// optional navigate so the destination load can rearm capture.
ensureBrowserObservationListeners(deps);
const abortPending = async (notifyContent: boolean) => {
recordings.get(params.session_id)?.observation?.cancel();
deps.frameCoordinator?.cancel(requestId);
recordings.delete(params.session_id);
releaseBrowserObservationListenersIfIdle();
if (notifyContent) {
@@ -986,7 +686,7 @@ export async function handleRecordStart(
return cancelledError();
}
active.startUrl = startUrl;
active.tabs.navigation(target.tabId, startUrl).currentUrl = startUrl;
active.currentUrl = startUrl;
if (isContentScriptRestrictedUrl(startUrl)) {
await abortPending(false);
@@ -994,7 +694,7 @@ export async function handleRecordStart(
code: "invalid_params",
message: params.url
? `cannot record on restricted URL (${startUrl}); use an http(s) page`
: `cannot record on restricted URL (${startUrl}); default start page https://example.com/ did not load — pass --url with a page you can open`,
: `cannot record on restricted URL (${startUrl}); default start page must be injectable http(s)`,
};
}
@@ -1021,10 +721,6 @@ export async function handleRecordStart(
const startMsg: RecordStartMessage = { type: RECORD_START, requestId, startedAtMs };
try {
if (deps.frameCoordinator) {
const frameStarted = await deps.frameCoordinator.armTab(requestId, target.tabId);
if (!frameStarted) throw new Error("top recording document did not start");
}
await sendRecordStartWithAck(target.tabId, startMsg, deps.sendToTab);
} catch {
await abortPending(true);
@@ -1035,17 +731,6 @@ export async function handleRecordStart(
};
}
if (active.observation) {
const activeRecording = recordings.get(params.session_id);
if (activeRecording) {
try {
await activeRecording.observation?.captureInitial(target.tabId);
} catch {
// Proceed without initial observation; steps may be dropped by reducer.
}
}
}
{
const cancelled = await abortIfCancelled(true);
if (cancelled) return cancelled;
@@ -1070,7 +755,7 @@ export async function handleRecordStop(
};
}
const trace = await finishRecording(params.session_id, deps, "cli_stop");
const trace = await finishRecording(params.session_id, deps);
if (!trace) {
return {
code: "protocol_error",
@@ -1100,9 +785,9 @@ export async function handleRecordAwait(
return { code: "cancelled", message: "record_await aborted" };
}
const outcome = await new Promise<{ trace: RecordedTrace } | { error: RpcError }>((resolve) => {
const outcome = await new Promise<{ trace: Trace } | { error: RpcError }>((resolve) => {
let settled = false;
const finish = (result: { trace: RecordedTrace } | { error: RpcError }) => {
const finish = (result: { trace: Trace } | { error: RpcError }) => {
if (settled) return;
settled = true;
if (timer) clearTimeout(timer);
@@ -1144,8 +829,6 @@ export function clearRecordingForSession(sessionId: string): void {
}
void clearRearmTimersForRecording(recording, getDefaultDeps());
if (!recording.settled) {
getDefaultDeps().frameCoordinator?.cancel(recording.requestId);
recording.observation?.cancel();
recording.settled = true;
recording.rejectFinish(new Error("recording cleared"));
}
-20
View File
@@ -76,7 +76,6 @@ export interface SessionStopResult {
}
export interface SessionStopDeps {
signal?: AbortSignal;
cdp?: {
detachSession(sessionId: string): Promise<void>;
};
@@ -178,9 +177,6 @@ export async function handleSessionStop(
message: `session ${params.session_id} unknown`,
};
}
if (deps.signal?.aborted) {
return { code: "cancelled", message: "session_stop aborted before teardown" };
}
// Step 1: auto-return borrowed tabs. Iterate over a snapshot of the
// ids so deletions during iteration do not break the Map iterator.
@@ -191,7 +187,6 @@ export async function handleSessionStop(
try {
const tabManagement = {
...(deps.tabManagement ?? {}),
signal: deps.signal,
isAgentWindowId:
deps.tabManagement?.isAgentWindowId ??
((windowId: number) => manager.findByWindowId(windowId) !== null),
@@ -219,17 +214,6 @@ export async function handleSessionStop(
}
}
if (deps.signal?.aborted) {
const nonCancelFailures = returnFailures?.filter((failure) => failure.code !== "cancelled");
if (nonCancelFailures && nonCancelFailures.length > 0) {
return {
...(returnedTabIds.length > 0 ? { returned_tab_ids: returnedTabIds } : {}),
return_failures: nonCancelFailures,
};
}
return { code: "cancelled", message: "session_stop aborted during tab return" };
}
const result: SessionStopResult = {};
if (returnedTabIds.length > 0) result.returned_tab_ids = returnedTabIds;
if (returnFailures && returnFailures.length > 0) {
@@ -249,10 +233,6 @@ export async function handleSessionStop(
// Step 3: detach CDP sessions this session opened (no-op if none).
await deps.cdp?.detachSession(params.session_id);
if (deps.signal?.aborted) {
return { code: "cancelled", message: "session_stop aborted before window close" };
}
// Step 4: close the Agent Window and drop the context.
await manager.stop(params.session_id);
+1
View File
@@ -53,6 +53,7 @@ export type { DialogCursor };
export interface CdpRunner {
send<T = unknown>(tabId: number, method: string, params?: object): Promise<T>;
sendToTarget?<T = unknown>(target: CdpTarget, method: string, params?: object): Promise<T>;
detach?(tabId: number): Promise<void>;
getFrameGraph?(tabId: number): Promise<CdpFrameGraph>;
ensureAttachedToUrl?(tabId: number, expectedUrl: string | undefined): Promise<void>;
trackSessionTab?(sessionId: string, tabId: number): void;
+59
View File
@@ -0,0 +1,59 @@
// Upload orchestration: validate the session-scoped request, resolve its click
// target, then delegate the browser protocol transaction to the file-input
// transaction module.
import type { SessionManager } from "@/session-manager/manager";
import type { RpcError, UploadParams, UploadResult } from "@/transport/types";
import { uploadThroughActivatedFileInput } from "./file-input-transaction";
import { clickResolvedTarget, type InteractionDeps, resolveActionTarget } from "./interaction";
import {
type CdpRunner,
enforceAgentWindow,
isRpcError,
lookupSession,
resolveTargetTab,
} from "./shared";
const DEFAULT_TIMEOUT_MS = 120_000;
export interface UploadDeps extends InteractionDeps {
cdp: CdpRunner;
}
export async function handleUpload(
manager: SessionManager,
params: UploadParams,
deps: UploadDeps,
): Promise<UploadResult | RpcError> {
const ctx = lookupSession(manager, params, "upload");
if (isRpcError(ctx)) return ctx;
const target = await resolveTargetTab(manager, ctx, params.tab_id, deps.tabsApi);
if (isRpcError(target)) return target;
const denied = enforceAgentWindow(ctx, target, "upload");
if (denied) return denied;
if (
params.files.length === 0 ||
params.files.length > 20 ||
params.files.some((file) => !file.staged_path)
) {
return { code: "invalid_params", message: "upload requires daemon-staged files" };
}
const address = await resolveActionTarget(deps.cdp, ctx, target, params, "upload");
if (isRpcError(address)) return address;
const timeoutMs = params.timeout_ms ?? DEFAULT_TIMEOUT_MS;
const transaction = await uploadThroughActivatedFileInput({
cdp: deps.cdp,
actionTarget: address,
files: params.files.map((file) => file.staged_path as string),
timeoutMs,
signal: deps.signal,
trigger: () => clickResolvedTarget(ctx, address, {}, deps),
});
if (isRpcError(transaction)) return transaction;
return {
tab_id: target.tabId,
used_ref: transaction.click.used_ref,
used_selector: transaction.click.used_selector,
file_names: params.files.map((file) => file.name),
};
}
@@ -1,82 +0,0 @@
import { describe, expect, it } from "vitest";
import {
RECORD_DOCUMENT_ATTRIBUTE,
recordingDocumentMarkerValue,
} from "@/shared/recording-document-identity";
import { projectRecordSafeObservation } from "../record-safe-observation";
describe("projectRecordSafeObservation", () => {
it("projects the opaque recording identity without exposing captured attributes", () => {
const recordingDocumentId = "123e4567-e89b-42d3-a456-426614174000";
const result = projectRecordSafeObservation({
rootFrameId: "child",
frameDocuments: [
{
frameId: "child",
target: { tabId: 4, sessionId: "oopif-session" },
contextScopeId: "scope-1",
axNodes: [],
domNodes: [
{
backendNodeId: 1,
parentBackendNodeId: null,
tag: "html",
attrs: {
[RECORD_DOCUMENT_ATTRIBUTE]: recordingDocumentMarkerValue(recordingDocumentId),
value: "user@example.com",
},
rect: null,
paintOrder: 0,
position: "static",
pointerEvents: "auto",
},
],
},
],
rendered: { text: '@vom 1\nRootWebArea "Example"', refs: [], truncated: false },
});
expect(result.frames).toEqual([
{
frameId: "child",
target: { tabId: 4, sessionId: "oopif-session" },
recordingDocumentId,
},
]);
const dumped = JSON.stringify(result);
expect(dumped).not.toContain("user@example.com");
expect(dumped).not.toContain("attrs");
expect(dumped).not.toContain("domNodes");
expect(dumped).not.toContain("axNodes");
});
it("does not project a page-controlled marker value as recording identity", () => {
const result = projectRecordSafeObservation({
rootFrameId: "root",
frameDocuments: [
{
frameId: "root",
target: { tabId: 4 },
contextScopeId: "scope-1",
axNodes: [],
domNodes: [
{
backendNodeId: 1,
parentBackendNodeId: null,
tag: "html",
attrs: { [RECORD_DOCUMENT_ATTRIBUTE]: "bsk:user@example.com" },
rect: null,
paintOrder: 0,
position: "static",
pointerEvents: "auto",
},
],
},
],
rendered: { text: '@vom 1\nRootWebArea "Example"', refs: [], truncated: false },
});
expect(result.frames).toEqual([{ frameId: "root", target: { tabId: 4 } }]);
expect(JSON.stringify(result)).not.toContain("user@example.com");
});
});
@@ -77,12 +77,9 @@ export function buildFrameDocuments<T extends FrameOwnedAxNode>(
): FrameDocument<T>[] {
const frames = frameList(graph, batches, captured);
const frameById = new Map(frames.map((frame) => [frame.frameId, frame]));
const rootFrameId = graph?.rootFrameId ?? captured.rootFrameId ?? frames[0]?.frameId;
const domNodesForFrame = (frameId: string): CapturedNode[] =>
captured.frameNodes?.get(frameId) ?? (frameId === rootFrameId ? captured.nodes : []);
const backendOwner = new Map<string, string>();
for (const frame of frames) {
for (const node of domNodesForFrame(frame.frameId)) {
for (const node of captured.frameNodes?.get(frame.frameId) ?? []) {
backendOwner.set(targetBackendKey(frame.target, node.backendNodeId), frame.frameId);
}
}
@@ -168,6 +165,6 @@ export function buildFrameDocuments<T extends FrameOwnedAxNode>(
...frame,
contextScopeId: frame.frameId,
axNodes: axNodesByFrame.get(frame.frameId) ?? [],
domNodes: domNodesForFrame(frame.frameId),
domNodes: captured.frameNodes?.get(frame.frameId) ?? [],
}));
}
@@ -1,88 +0,0 @@
import type { Rect, RenderedRef, VomResult } from "@browser-skill/vom";
import type { CdpTarget } from "@/browser-driver/frame-graph";
import { readRecordingDocumentIdentity } from "@/shared/recording-document-identity";
import type { CapturedSurfaceProbe } from "./capture";
import type { CapturedFrameDocument, FrameAxNode } from "./frame-capture";
/** Frame identity needed to resolve an `@eN` that lives in an iframe. */
export interface CaptureVomFrame {
frameId: string;
target: CdpTarget;
parentFrameId?: string;
ownerBackendNodeId?: number;
/** BrowserSkill-generated identity for the recording agent in this Document. */
recordingDocumentId?: string;
}
/** Allowlisted geometry used to match a recorded action to a rendered ref. */
export interface CaptureVomMatchNode {
frameId: string;
backendNodeId: number;
tag: string;
/** Top-level viewport-relative geometry. */
rect: Rect | null;
/** Frame-local viewport-relative geometry, when available. */
localRect?: Rect | null;
}
/**
* Raw AX/DOM trees are never returned. When `redactValues` is enabled by the
* caller, form values in `text` are masked as well.
*/
export interface CaptureVomObservationResult {
text: string;
refs: RenderedRef[];
truncated: boolean;
rootFrameId: string;
frames: CaptureVomFrame[];
matchNodes: CaptureVomMatchNode[];
surfaceProbes?: CapturedSurfaceProbe[];
}
function projectFrames(documents: CapturedFrameDocument<FrameAxNode>[]): CaptureVomFrame[] {
return documents.map((document) => {
let recordingDocumentId: string | undefined;
for (const node of document.domNodes) {
recordingDocumentId = readRecordingDocumentIdentity(node.attrs);
if (recordingDocumentId) break;
}
return {
frameId: document.frameId,
target: document.target,
...(document.parentFrameId ? { parentFrameId: document.parentFrameId } : {}),
...(document.ownerBackendNodeId !== undefined
? { ownerBackendNodeId: document.ownerBackendNodeId }
: {}),
...(recordingDocumentId ? { recordingDocumentId } : {}),
};
});
}
function projectMatchNodes(documents: CapturedFrameDocument<FrameAxNode>[]): CaptureVomMatchNode[] {
return documents.flatMap((document) =>
document.domNodes.map((node) => ({
frameId: document.frameId,
backendNodeId: node.backendNodeId,
tag: node.tag,
rect: node.rect,
...(node.localRect !== undefined ? { localRect: node.localRect } : {}),
})),
);
}
export function projectRecordSafeObservation(input: {
rootFrameId: string;
frameDocuments: CapturedFrameDocument<FrameAxNode>[];
rendered: VomResult;
surfaceProbes?: CapturedSurfaceProbe[];
}): CaptureVomObservationResult {
return {
text: input.rendered.text,
refs: input.rendered.refs,
truncated: input.rendered.truncated,
rootFrameId: input.rootFrameId,
frames: projectFrames(input.frameDocuments),
matchNodes: projectMatchNodes(input.frameDocuments),
surfaceProbes: input.surfaceProbes,
};
}
@@ -65,11 +65,6 @@ function deferredFakeTransport(): { transport: Transport; emit: (frame: Protocol
}
describe("performHandshake", () => {
it("advertises the protocol compatibility boundary", () => {
expect(PROTOCOL_VERSION).toBe("1.1");
expect(MIN_COMPATIBLE_PROTOCOL).toBe("1.0");
});
it("sends system.handshake with identity and both compat fields", async () => {
let sentFrame: ProtocolFrame | null = null;
const transport = fakeTransport((req) => {
@@ -79,7 +74,7 @@ describe("performHandshake", () => {
result: {
server: "browser-skill-daemon",
version: "0.1.0",
protocol_version: "1.1",
protocol_version: "1.0",
min_compatible_peer: "0.0.0",
min_compatible_protocol: "1.0",
},
@@ -134,8 +129,8 @@ describe("performHandshake", () => {
const response = {
server: "browser-skill-daemon",
version: "0.1.0",
protocol_version: "1.1",
min_compatible_protocol: "1.1",
protocol_version: "1.0",
min_compatible_protocol: "1.0",
} satisfies HandshakeResult;
const transport = fakeTransport((req) => ({
id: (req as { id: string }).id,
@@ -150,7 +145,7 @@ describe("performHandshake", () => {
});
expect(outcome.result.min_compatible_peer).toBeUndefined();
expect(outcome.result.min_compatible_protocol).toBe("1.1");
expect(outcome.result.min_compatible_protocol).toBe("1.0");
});
it("rejects when the daemon responds with an error", async () => {
@@ -183,9 +178,9 @@ describe("performHandshake", () => {
result: {
server: "browser-skill-daemon",
version: "0.1.0",
protocol_version: "1.1",
protocol_version: "1.0",
min_compatible_peer: "0.0.0",
min_compatible_protocol: "1.1",
min_compatible_protocol: "1.0",
},
});
emit({
@@ -193,14 +188,14 @@ describe("performHandshake", () => {
result: {
server: "browser-skill-daemon",
version: "0.1.0",
protocol_version: "1.1",
protocol_version: "1.0",
min_compatible_peer: "0.0.0",
min_compatible_protocol: "1.1",
min_compatible_protocol: "1.0",
},
});
await expect(pending).resolves.toMatchObject({
result: { server: "browser-skill-daemon", protocol_version: "1.1" },
result: { server: "browser-skill-daemon", protocol_version: "1.0" },
});
});
+1 -1
View File
@@ -7,7 +7,7 @@ import type {
ResponseFrame,
} from "./types";
export const PROTOCOL_VERSION = "1.1";
export const PROTOCOL_VERSION = "1.0";
/**
* Extension semver, injected at build time from `package.json` via
* Vite's `define` (see `wxt.config.ts` and `vitest.config.ts`).
+121 -121
View File
@@ -33,10 +33,22 @@ export type RpcErrorReason =
| "restricted_tab_url"
| "borrow_conflict"
| "screenshot_capture_failed"
| "file_input_probe_failed"
| "file_input_not_activated"
| "set_file_input_failed"
| "download_capture_failed"
| "transfer_outcome_unknown"
| "transfer_timeout"
| "cleanup_failed";
export type TransferEffectState = "none" | "committed" | "unknown";
export type TransferCleanupState = "complete" | "failed";
export interface RpcErrorData {
reason?: RpcErrorReason;
effect_state?: TransferEffectState;
phase?: string;
cleanup_state?: TransferCleanupState;
[key: string]: unknown;
}
@@ -500,6 +512,50 @@ export interface SelectResult {
dialogs?: JavaScriptDialogInfo[];
}
export interface UploadFile {
transfer_id: string;
name: string;
staged_path?: string;
}
export interface UploadParams {
session_id: string;
ref?: string;
selector?: string;
tab_id?: number;
files: UploadFile[];
timeout_ms?: number;
}
export interface UploadResult {
tab_id: number;
used_ref?: string;
used_selector?: string;
file_names: string[];
}
export interface DownloadParams {
session_id: string;
ref?: string;
selector?: string;
tab_id?: number;
timeout_ms?: number;
browser_relative_dir?: string;
max_byte_size?: number;
}
export interface DownloadResult {
tab_id: number;
used_ref?: string;
used_selector?: string;
suggested_filename: string;
byte_size: number;
mime?: string;
danger?: string;
browser_path?: string;
transfer_id?: string;
}
// --------------------------------------------------------------------------
// M9 tool payloads — evaluate / wait_for_navigation / wait_ms
// --------------------------------------------------------------------------
@@ -657,60 +713,10 @@ export interface EmulateResult {
}
// --------------------------------------------------------------------------
// Semantic record payloads mirror the versioned Rust protocol models.
// Semantic record payloads — mirror bsk-protocol record.rs (Trace v2)
// --------------------------------------------------------------------------
export const TRACE_VERSION_V3 = 3;
/** Logical v2 identifier. Not a wire field — v2 envelopes omit `version`. */
export const TRACE_VERSION_V2 = 2;
export const VOM_FORMAT_VERSION = 1;
export interface TargetDescriptorV3 {
ref?: string;
role?: string;
name?: string;
ctx?: string;
unmatched?: boolean;
}
export interface RecorderInfo {
bsk: string;
vom: number;
}
export type StopReason = "user_finish" | "cli_stop";
export interface TraceStateV3 {
id: string;
url: string;
title?: string;
body: string;
truncated?: boolean;
}
export interface StepResultV3 {
state: string;
}
export interface StepCommonV3 {
id: number;
state: string;
result: StepResultV3;
}
export type NavigationCause =
| "user_typed"
| "link"
| "form_submit"
| "reload"
| "history"
| "script"
| "browser";
export type FillCommit = "enter" | "suggestion" | "blur";
/** Legacy v2 target shape retained for existing record producers. */
export interface TargetDescriptorV2 {
export interface TargetDescriptor {
role?: string;
name?: string;
tag: string;
@@ -723,109 +729,103 @@ export interface TraceEntry {
start_url: string;
}
export interface PageRefV2 {
export interface PageRef {
id: string;
url: string;
title?: string;
}
export interface SelectedOptionV2 {
export interface SelectedOption {
value: string;
label?: string;
}
export interface StepEffectV2 {
export interface StepEffect {
navigated_to: string;
}
export interface StepCommonV2 {
export interface StepCommon {
id: number;
page: string;
effect?: StepEffectV2;
effect?: StepEffect;
}
/** Capture/buffer draft before v2 reduction. */
export type DraftTraceStep =
| {
op: "click";
target: TargetDescriptor;
navigated_to?: string;
page_url?: string;
}
| {
op: "hover";
target: TargetDescriptor;
page_url?: string;
}
| {
op: "fill";
target: TargetDescriptor;
value: string;
redacted?: boolean;
page_url?: string;
}
| {
op: "press";
key: string;
target?: TargetDescriptor;
modifiers?: KeyModifier[];
navigated_to?: string;
page_url?: string;
}
| {
op: "select";
target: TargetDescriptor;
values: string[];
labels?: string[];
navigated_to?: string;
page_url?: string;
}
| {
op: "navigate";
url: string;
page_url?: string;
};
/** Exported record-only step (trace v2). */
export type StepV2 =
| ({ op: "navigate" } & StepCommonV2 & { to: string })
| ({ op: "click" } & StepCommonV2 & { target: TargetDescriptorV2 })
| ({ op: "hover" } & StepCommonV2 & { target: TargetDescriptorV2 })
| ({ op: "fill" } & StepCommonV2 & {
target: TargetDescriptorV2;
export type Step =
| ({ op: "navigate" } & StepCommon & { to: string })
| ({ op: "click" } & StepCommon & { target: TargetDescriptor })
| ({ op: "hover" } & StepCommon & { target: TargetDescriptor })
| ({ op: "fill" } & StepCommon & {
target: TargetDescriptor;
value: string;
redacted?: boolean;
})
| ({ op: "select" } & StepCommonV2 & {
target: TargetDescriptorV2;
selection: SelectedOptionV2[];
| ({ op: "select" } & StepCommon & {
target: TargetDescriptor;
selection: SelectedOption[];
})
| ({ op: "press" } & StepCommonV2 & {
| ({ op: "press" } & StepCommon & {
key: string;
modifiers?: KeyModifier[];
target?: TargetDescriptorV2;
target?: TargetDescriptor;
});
export interface TraceV2 {
export interface Trace {
recorded_at: string;
started_at?: string;
purpose?: string;
entry: TraceEntry;
pages: PageRefV2[];
steps: StepV2[];
pages: PageRef[];
steps: Step[];
}
export interface SelectedOptionV3 {
value: string;
label?: string;
}
export type StepV3 =
| ({ op: "navigate" } & StepCommonV3 & { to: string; cause: NavigationCause })
| ({ op: "switch_tab" } & StepCommonV3)
| ({ op: "click" } & StepCommonV3 & { target: TargetDescriptorV3 })
| ({ op: "hover" } & StepCommonV3 & { target: TargetDescriptorV3 })
| ({ op: "fill" } & StepCommonV3 & {
target: TargetDescriptorV3;
value: string;
commit: FillCommit;
redacted?: boolean;
})
| ({ op: "select" } & StepCommonV3 & {
target: TargetDescriptorV3;
selection?: SelectedOptionV3[];
})
| ({ op: "press" } & StepCommonV3 & {
key: string;
modifiers?: KeyModifier[];
target?: TargetDescriptorV3;
})
| ({ op: "scroll" } & StepCommonV3);
export interface TraceV3 {
version: typeof TRACE_VERSION_V3;
recorded_at: string;
started_at?: string;
purpose?: string;
stopped_by: StopReason;
entry: TraceEntry;
recorder: RecorderInfo;
states: TraceStateV3[];
steps: StepV3[];
}
export type RecordedTrace = TraceV2 | TraceV3;
export type RecordedStep = StepV2 | StepV3;
export interface RecordStartParams {
session_id: string;
tab_id?: number;
url?: string;
purpose?: string;
max_page_tokens?: number;
redact_values?: boolean;
/** Omitted means v2; `3` requests a state-linked v3 trace. */
trace_version?: number;
/** Client can decode the v3 `switch_tab` step variant. */
supports_tab_switch_steps?: boolean;
}
export interface RecordStartResult {
@@ -838,7 +838,7 @@ export interface RecordStopParams {
}
export interface RecordStopResult {
trace: RecordedTrace;
trace: Trace;
}
export interface RecordAwaitParams {
@@ -847,5 +847,5 @@ export interface RecordAwaitParams {
}
export interface RecordAwaitResult {
trace: RecordedTrace;
trace: Trace;
}
+1
View File
@@ -30,6 +30,7 @@ export default defineConfig({
permissions: [
"alarms",
"debugger",
"downloads",
"idle",
"notifications",
"tabs",
+2
View File
@@ -53,6 +53,7 @@ flate2 = { workspace = true }
tar = { workspace = true }
zip = { workspace = true }
sha2 = { workspace = true }
base64 = { workspace = true }
[target.'cfg(unix)'.dependencies]
nix = { workspace = true }
@@ -63,6 +64,7 @@ windows-sys = { version = "0.59", features = [
"Win32_Foundation",
"Win32_System_Threading",
"Win32_Security",
"Win32_Storage_FileSystem",
] }
[dev-dependencies]
+29 -14
View File
@@ -206,6 +206,29 @@ Both capture from the moment the tab is attached and read a bounded per-tab buff
| `bsk select <ref-or-selector> --value <v>` | Set `<select>` option(s) by `value` (repeat `--value` for multi-select) |
| `bsk press <key>` | Key/combo (`Enter`, `Ctrl+A`, …; optional `--ref` to focus first) |
### File transfer (require `--session`)
| Command | Summary |
|---------|---------|
| `bsk upload <ref-or-selector> --file <path>` | Click one upload trigger and attach an agent-readable local file (`--file` is repeatable) |
| `bsk download <ref-or-selector> --out <path>` | Click one download trigger and copy the single completed file to an exact local path (`--overwrite` is opt-in) |
The agent/harness decides whether a file transfer is appropriate and which local path belongs to the task. Treat upload as disclosure of that file to the current website, and download as accepting website-controlled bytes onto the local filesystem. Use only paths that are necessary for the user's bounded goal.
BrowserSkill enforces the mechanical boundary: files are staged under a session-scoped opaque transfer, only daemon-minted capabilities reach the extension, upload/download still obey Agent Window tab checks, and transfers are chunk/size bounded. Upload intercepts the native chooser for one transaction, locates the input activated in the resolved target's document, and assigns only the staged file paths. Download uniquely correlates one exact-target browser intent with one Chrome download in either event order, routes it through a daemon-minted relative directory, and lets the daemon validate and import it. Upload staging remains available for a later form submission and is removed when the session ends. Downloads cannot overwrite an existing destination unless `--overwrite` is explicit. BrowserSkill does not inspect file content or decide whether its meaning is sensitive.
Do not use `request-help` merely because a native file chooser or browser download is involved; try these commands first. For transfer failures, use the structured error instead of retrying blindly:
- `reason=file_input_not_activated` means the requested click did not activate exactly one `<input type="file">`; the page may use a non-input picker such as `window.showOpenFilePicker()`. Do not retry blindly. If human help is available, call `request-help` and tell the user the exact original local path to choose. The staged daemon path is internal and must not be shown to the user.
- `reason=file_input_probe_failed` means BrowserSkill could not safely establish the browser-side upload transaction. Do not repeat the same action; use `request-help` when available.
- `reason=set_file_input_failed` means BrowserSkill found the activated file input but Chrome rejected the staged path or assignment. Check the extension's file-URL access permission; otherwise use `request-help`.
- `reason=download_capture_failed` means BrowserSkill could not attribute exactly one completed download to the requested target. Do not retry blindly or accept an unrelated browser download; use `request-help` when available.
- `effect_state=none` means BrowserSkill confirmed that no file-transfer effect was committed. Follow the accompanying reason; a corrected target or explicit human fallback may be attempted.
- `effect_state=unknown` means the browser may already have attached or created the file. Do not repeat the transfer. Observe the page if that can establish the result; otherwise stop and report the uncertainty.
- `effect_state=committed` means the browser-side effect occurred even if later completion or cleanup failed. Do not repeat it; continue only after verifying the resulting page/download state.
If `request-help` returns `outcome="disabled"`, do not retry it. Stop gracefully and report the transfer mechanism that requires human intervention.
### Scripting & timing
| Command | Summary |
@@ -265,27 +288,19 @@ retry — complete the task autonomously or stop gracefully.
### Recording — `bsk record`
Capture the user's own actions in the Agent Window to a **trace bundle**, for later LLM-driven automation. New CLI builds request **trace v3** (page observations + action chain); older extensions may still return **trace v2** (actions only), which the CLI exports as a single `trace.json` without `states/`.
Capture the user's own actions in the Agent Window to a `trace.json`, for later LLM-driven automation:
```bash
bsk record start --browser <instance-id-or-label> \
[--url https://…] [--purpose "publish a wiki doc"] \
[--max-page-tokens 3000] [--redact-values] \
[--output trace]
bsk record start --browser <instance-id-or-label> [--url https://…] [--purpose "publish a wiki doc"] [--output trace.json]
# `--url` is optional; default https://example.com/ when omitted (must be http(s)).
# Blocks until the user clicks Finish in the recording panel, then writes:
# trace/trace.json — action chain (+ state index when v3)
# trace/states/ — v3 only: one `sN.txt` observe snapshot per settled page state
# Blocks until the user clicks Finish in the recording panel, then writes ./trace.json and closes the window.
bsk record stop [--output trace] # terminal fallback if the browser panel is unavailable
bsk record stop [--output trace.json] # terminal fallback if the browser panel is unavailable
```
- **v3 bundle (preferred):** `--output` is a directory (default `./trace`) containing `trace.json` and `states/`. `trace.json` lists `states[]` (page observation ids) and `steps[]` (each step binds `state` = observe snapshot *before* the action and `result.state` = snapshot *after* settle). Page bodies live in `states/sN.txt` using the same VOM format as `bsk observe`.
- **v2 fallback:** when the connected extension is older, export may contain only legacy `trace.json` with `pages[]` action context and no `states/` observation files. Update the extension for full v3 bundles.
- Each `states[]` entry is one **settled page observation** (captured after recording start, navigation landing, or action settle — not on a timer).
- `target.ref` values like `@e12` exist **only inside** the bundle for disambiguation; do **not** copy `@eN` refs into SKILL.md or agent runbooks — use visible names from the observation text instead.
- The trace is a **record-only action log** (a `pages[]` dictionary + `navigate`/`click`/`fill`/`select`/`press` steps with `target` descriptors). It records *what the user did*; deciding which inputs are variable is left to the executing agent.
- `--purpose` is optional context metadata; it does **not** change what gets captured.
- `--redact-values` masks all form values in page files as `[filled]` / `[empty]`.
- There is **no** `bsk replay` — to redo a flow, read the trace and reuse the existing `session` / `snapshot` / `@eN` / `click` / `fill` tools. Follow **Stop when the goal is met**.
- Do **not** record on banking/SSO/password-manager pages; passwords are redacted but traces may still contain sensitive text.
## Error handling
+75
View File
@@ -0,0 +1,75 @@
//! Atomic visibility boundary for files received into a same-directory temp.
use std::path::Path;
pub fn commit(temp: &Path, out: &Path, overwrite: bool) -> std::io::Result<()> {
if !overwrite {
std::fs::hard_link(temp, out)?;
std::fs::remove_file(temp)?;
return Ok(());
}
replace(temp, out)
}
#[cfg(unix)]
fn replace(temp: &Path, out: &Path) -> std::io::Result<()> {
// POSIX rename replaces an existing non-directory destination atomically.
std::fs::rename(temp, out)
}
#[cfg(windows)]
fn replace(temp: &Path, out: &Path) -> std::io::Result<()> {
use std::os::windows::ffi::OsStrExt;
use windows_sys::Win32::Storage::FileSystem::{
MOVEFILE_REPLACE_EXISTING, MOVEFILE_WRITE_THROUGH, MoveFileExW,
};
let from: Vec<u16> = temp.as_os_str().encode_wide().chain(Some(0)).collect();
let to: Vec<u16> = out.as_os_str().encode_wide().chain(Some(0)).collect();
// SAFETY: both buffers are NUL-terminated and remain alive for the call.
let ok = unsafe {
MoveFileExW(
from.as_ptr(),
to.as_ptr(),
MOVEFILE_REPLACE_EXISTING | MOVEFILE_WRITE_THROUGH,
)
};
if ok == 0 {
Err(std::io::Error::last_os_error())
} else {
Ok(())
}
}
#[cfg(not(any(unix, windows)))]
fn replace(temp: &Path, out: &Path) -> std::io::Result<()> {
std::fs::rename(temp, out)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn no_replace_never_overwrites_existing_output() {
let dir = tempfile::tempdir().unwrap();
let temp = dir.path().join("new.part");
let out = dir.path().join("out.bin");
std::fs::write(&temp, b"new").unwrap();
std::fs::write(&out, b"old").unwrap();
assert!(commit(&temp, &out, false).is_err());
assert_eq!(std::fs::read(&out).unwrap(), b"old");
}
#[test]
fn overwrite_replaces_existing_output_without_predelete() {
let dir = tempfile::tempdir().unwrap();
let temp = dir.path().join("new.part");
let out = dir.path().join("out.bin");
std::fs::write(&temp, b"new").unwrap();
std::fs::write(&out, b"old").unwrap();
commit(&temp, &out, true).unwrap();
assert_eq!(std::fs::read(&out).unwrap(), b"new");
assert!(!temp.exists());
}
}
+147
View File
@@ -0,0 +1,147 @@
//! `bsk download` — capture one browser download and atomically commit it.
use std::fs::OpenOptions;
use std::io::Write;
use std::path::{Path, PathBuf};
use std::time::Duration;
use anyhow::Context;
use base64::Engine;
use bsk_protocol::Method;
use bsk_protocol::tools::{
DownloadParams, DownloadResult, TransferChunkParams, TransferChunkResult, TransferIdParams,
TransferReleaseResult,
};
use clap::Args;
use uuid::Uuid;
use crate::cli::atomic_output;
use crate::cli::ensure_daemon::ensure_daemon;
use crate::cli::error::{CliError, Format};
use crate::cli::interaction::split_target;
use crate::cli::navigate::parse_timeout_ms;
#[derive(Debug, Clone, Args)]
pub struct DownloadArgs {
/// Snapshot ref (`@e3`) or CSS selector for the download trigger.
pub target: Option<String>,
#[arg(long = "ref")]
pub ref_: Option<String>,
#[arg(long = "selector")]
pub selector: Option<String>,
#[arg(long)]
pub out: PathBuf,
#[arg(long)]
pub session: String,
#[arg(long = "tab-id")]
pub tab_id: Option<i64>,
#[arg(long, default_value = "2m", value_parser = parse_timeout_ms)]
pub timeout: u32,
#[arg(long)]
pub overwrite: bool,
}
pub fn dispatch(args: DownloadArgs, format: Format) -> Result<(), CliError> {
if args.out.exists() && !args.overwrite {
return Err(CliError::Local(anyhow::anyhow!(
"output already exists (pass --overwrite to replace it): {}",
args.out.display()
)));
}
let info = ensure_daemon().context("ensure daemon is running")?;
let (ref_, selector) = split_target(args.target, args.ref_, args.selector)?;
let params = DownloadParams {
session_id: args.session,
ref_,
selector,
tab_id: args.tab_id,
timeout_ms: Some(args.timeout),
browser_relative_dir: None,
max_byte_size: None,
};
let reply: DownloadResult = crate::cli::business_rpc::call(
info.sock_path.clone(),
"download",
Method::ToolDownload,
Some(params),
ipc_timeout(args.timeout),
)?;
let transfer_id = reply.transfer_id.clone().ok_or_else(|| {
CliError::Local(anyhow::anyhow!("daemon returned no download transfer id"))
})?;
let write_result = write_transfer(&info.sock_path, &transfer_id, &args.out, args.overwrite);
let _: Result<TransferReleaseResult, CliError> = crate::cli::business_rpc::call(
info.sock_path,
"transfer-release",
Method::TransferRelease,
Some(TransferIdParams { transfer_id }),
Duration::from_secs(5),
);
write_result?;
match format {
Format::Json => {
let mut value = serde_json::to_value(&reply).unwrap();
value["path"] = serde_json::json!(args.out.to_string_lossy());
if let Some(obj) = value.as_object_mut() {
obj.remove("transfer_id");
obj.remove("browser_path");
}
println!("{}", serde_json::to_string_pretty(&value).unwrap());
}
Format::Human => println!("{}", args.out.display()),
}
Ok(())
}
fn write_transfer(sock: &Path, id: &str, out: &Path, overwrite: bool) -> Result<(), CliError> {
let parent = out
.parent()
.filter(|p| !p.as_os_str().is_empty())
.unwrap_or(Path::new("."));
let temp = parent.join(format!(".bsk-download-{}.part", Uuid::new_v4().simple()));
let result = (|| {
let mut file = OpenOptions::new()
.write(true)
.create_new(true)
.open(&temp)
.with_context(|| format!("create temporary download {}", temp.display()))
.map_err(CliError::Local)?;
let mut offset = 0u64;
loop {
let chunk: TransferChunkResult = crate::cli::business_rpc::call(
sock.to_path_buf(),
"transfer-read",
Method::TransferRead,
Some(TransferChunkParams {
transfer_id: id.to_string(),
offset,
data_base64: String::new(),
}),
Duration::from_secs(30),
)?;
let encoded = chunk.data_base64.as_deref().unwrap_or("");
let bytes = base64::engine::general_purpose::STANDARD
.decode(encoded)
.map_err(|e| CliError::Local(anyhow::anyhow!("decode download chunk: {e}")))?;
file.write_all(&bytes)
.map_err(|e| CliError::Local(e.into()))?;
offset = chunk.next_offset;
if chunk.eof {
break;
}
}
file.sync_all().map_err(|e| CliError::Local(e.into()))?;
drop(file);
atomic_output::commit(&temp, out, overwrite)
.with_context(|| format!("atomically commit download to {}", out.display()))
.map_err(CliError::Local)
})();
if result.is_err() {
let _ = std::fs::remove_file(&temp);
}
result
}
fn ipc_timeout(timeout_ms: u32) -> Duration {
Duration::from_millis(u64::from(timeout_ms) + 5_000)
}
+1 -1
View File
@@ -71,7 +71,7 @@ pub(crate) fn parse_modifiers(input: &str) -> Result<Vec<KeyModifier>, String> {
Ok(out)
}
fn split_target(
pub(crate) fn split_target(
positional: Option<String>,
explicit_ref: Option<String>,
explicit_selector: Option<String>,
+11 -1
View File
@@ -2,6 +2,7 @@
use std::time::Duration;
mod atomic_output;
pub mod browser_wait;
pub mod browsers;
pub mod business_rpc;
@@ -9,6 +10,7 @@ pub mod console;
pub mod daemon;
pub mod dialogs;
pub mod doctor;
pub mod download;
pub mod emulate;
pub mod ensure_daemon;
pub mod error;
@@ -22,7 +24,6 @@ pub mod navigate;
pub mod network;
pub mod observe;
pub mod record;
pub mod record_recovery;
pub mod record_state;
pub mod render_error;
pub mod screenshot;
@@ -31,6 +32,7 @@ pub mod snapshot;
pub mod status;
pub mod tab;
pub mod update;
pub mod upload;
pub mod waits;
pub mod window;
@@ -38,6 +40,7 @@ use clap::{Args, Parser, Subcommand};
use crate::cli::console::ConsoleArgs;
use crate::cli::daemon::DaemonCmd;
use crate::cli::download::DownloadArgs;
use crate::cli::emulate::EmulateArgs;
use crate::cli::evaluate::EvaluateArgs;
use crate::cli::get_html::GetHtmlArgs;
@@ -53,6 +56,7 @@ use crate::cli::session::SessionCmd;
use crate::cli::snapshot::SnapshotArgs;
use crate::cli::tab::TabCmd;
use crate::cli::update::UpdateArgs;
use crate::cli::upload::UploadArgs;
use crate::cli::waits::{WaitForNavigationArgs, WaitMsArgs};
use crate::cli::window::WindowCmd;
@@ -177,6 +181,12 @@ pub enum Command {
/// Set `<select>` option values by `value` attribute.
Select(SelectArgs),
/// Upload one or more local files through a page file chooser.
Upload(UploadArgs),
/// Capture one browser download and write it to a local path.
Download(DownloadArgs),
/// Evaluate a JavaScript expression inside the Agent Window.
Evaluate(EvaluateArgs),
+64 -266
View File
@@ -1,30 +1,23 @@
//! `bsk record start|stop` — capture user actions in the Agent Window.
use std::path::{Path, PathBuf};
use std::fs;
use std::path::PathBuf;
use std::time::Duration;
use anyhow::Context;
use bsk_protocol::Method;
use bsk_protocol::tools::{
RecordAwaitParams, RecordAwaitResult, RecordStartParams, RecordStartResult, RecordStopParams,
RecordStopResult, RecordedTrace, TRACE_VERSION_V3,
RecordStopResult, Trace,
};
use bsk_protocol::{ErrorCode, Method};
use clap::{Args, Subcommand};
use crate::cli::TOOL_IPC_TIMEOUT;
mod export;
use crate::cli::business_rpc;
use crate::cli::ensure_daemon::ensure_daemon;
use crate::cli::error::{CliError, Format};
use crate::cli::record_recovery;
use crate::cli::record_state;
use crate::cli::session::{SessionStartOptions, start_session, stop_session};
use export::{
ExportMeta, export_with_recovery, states_dir_for_output, trace_json_path,
validate_record_output,
};
/// Max wait for the user to click 结束 in the browser (24 hours).
const RECORD_AWAIT_TIMEOUT_MS: u32 = 86_400_000;
@@ -57,8 +50,7 @@ pub struct RecordStartArgs {
pub tab_id: Option<i64>,
/// Navigate to this http(s) URL before recording. When omitted, defaults
/// to `https://example.com/`. If that page does not load, pass `--url`
/// with a site you can open in this browser.
/// to `https://example.com/`.
#[arg(long)]
pub url: Option<String>,
@@ -66,25 +58,15 @@ pub struct RecordStartArgs {
#[arg(long)]
pub purpose: Option<String>,
/// Max VOM tokens per page observation file (default 3000).
#[arg(long = "max-page-tokens")]
pub max_page_tokens: Option<u32>,
/// Redact all form values in page observations (`[filled]` only).
#[arg(long = "redact-values")]
pub redact_values: bool,
/// Output directory for the trace bundle (default `./trace`).
/// Writes `<dir>/trace.json` and `<dir>/states/*.txt`.
#[arg(long, default_value = "trace")]
/// Output path for the trace JSON (default `./trace.json`).
#[arg(long, default_value = "trace.json")]
pub output: PathBuf,
}
#[derive(Debug, Clone, Args)]
pub struct RecordStopArgs {
/// Output directory for the trace bundle (default `./trace`).
/// Writes `<dir>/trace.json` and `<dir>/states/*.txt`.
#[arg(long, default_value = "trace")]
/// Output path for the trace JSON (default `./trace.json`).
#[arg(long, default_value = "trace.json")]
pub output: PathBuf,
}
@@ -96,7 +78,11 @@ pub fn dispatch(cmd: RecordCmd, format: Format) -> Result<(), CliError> {
}
fn dispatch_start(args: RecordStartArgs, format: Format) -> Result<(), CliError> {
prepare_record_start(&args.output)?;
if record_state::read().is_ok() {
return Err(CliError::Local(anyhow::anyhow!(
"a recording is already in progress; run `bsk record stop` first"
)));
}
let info = ensure_daemon().context("ensure daemon is running")?;
let session = start_session(
@@ -110,12 +96,8 @@ fn dispatch_start(args: RecordStartArgs, format: Format) -> Result<(), CliError>
let start_params = RecordStartParams {
session_id: session.session_id.clone(),
tab_id: args.tab_id,
url: args.url.clone(),
url: args.url,
purpose: args.purpose.clone(),
max_page_tokens: args.max_page_tokens,
redact_values: Some(args.redact_values),
trace_version: Some(TRACE_VERSION_V3),
supports_tab_switch_steps: Some(true),
};
let start_result = business_rpc::call::<RecordStartParams, RecordStartResult>(
info.sock_path.clone(),
@@ -129,7 +111,7 @@ fn dispatch_start(args: RecordStartArgs, format: Format) -> Result<(), CliError>
Ok(result) => result,
Err(err) => {
let _ = stop_session(info.sock_path, &session.session_id);
return Err(annotate_default_start_page_error(err, args.url.as_deref()));
return Err(err);
}
};
@@ -160,8 +142,8 @@ fn dispatch_start(args: RecordStartArgs, format: Format) -> Result<(), CliError>
// Keep write/render inside a Result so `?` cannot skip session teardown.
let run_result: Result<(), CliError> = match await_result {
Ok(result) => (|| {
let exported = export_with_recovery(&args.output, &result.trace)?;
render_finish(&result.trace, &args.output, &exported, format)
write_trace_file(&args.output, &result.trace)?;
render_finish(&result.trace, &args.output, format)
})(),
Err(err) => Err(err),
};
@@ -175,12 +157,11 @@ fn dispatch_start(args: RecordStartArgs, format: Format) -> Result<(), CliError>
}
fn dispatch_stop(args: RecordStopArgs, format: Format) -> Result<(), CliError> {
validate_record_output(&args.output)?;
if let Ok(state) = record_state::read() {
let info = ensure_daemon().context("ensure daemon is running")?;
let session_id = state.session_id.clone();
let state = record_state::read().map_err(CliError::Local)?;
let info = ensure_daemon().context("ensure daemon is running")?;
let session_id = state.session_id.clone();
let run_result: Result<(), CliError> = (|| {
let params = RecordStopParams {
session_id: session_id.clone(),
};
@@ -191,85 +172,16 @@ fn dispatch_stop(args: RecordStopArgs, format: Format) -> Result<(), CliError> {
Some(params),
TOOL_IPC_TIMEOUT,
)?;
write_trace_file(&args.output, &result.trace)?;
render_stop(&result, &args.output, format)
})();
let run_result: Result<(), CliError> = (|| {
let exported = export_with_recovery(&args.output, &result.trace)?;
render_stop(&result, &args.output, &exported, format)
})();
let session_stop_result = stop_session(info.sock_path, &session_id);
record_state::clear();
let session_stop_result = stop_session(info.sock_path, &session_id);
record_state::clear();
run_result?;
session_stop_result?;
return Ok(());
}
let Some(trace) = record_recovery::load().map_err(CliError::Local)? else {
return Err(CliError::Local(anyhow::anyhow!(
"no recording in progress; run `bsk record start` first"
)));
};
let exported = export_with_recovery(&args.output, &trace)?;
render_finish(&trace, &args.output, &exported, format)
}
/// When `record start` omitted `--url` and the default example.com page
/// never loaded, rewrite the RPC error so the CLI hint points at `--url`.
fn annotate_default_start_page_error(err: CliError, user_url: Option<&str>) -> CliError {
if user_url.is_some() {
return err;
}
let CliError::Rpc {
code,
message,
data,
source,
} = err
else {
return err;
};
if !is_default_start_page_failure(code, &message) {
return CliError::Rpc {
code,
message,
data,
source,
};
}
CliError::Rpc {
code,
message,
data: Some(serde_json::json!({
"reason": crate::cli::render_error::reason::RECORD_START_PAGE_UNREACHABLE
})),
source,
}
}
fn is_default_start_page_failure(code: ErrorCode, message: &str) -> bool {
match code {
ErrorCode::CdpFailed | ErrorCode::Timeout => true,
ErrorCode::InvalidParams => {
message.contains("restricted URL") || message.contains("about:blank")
}
_ => false,
}
}
fn prepare_record_start(output: &Path) -> Result<(), CliError> {
if record_state::read().is_ok() {
return Err(CliError::Local(anyhow::anyhow!(
"a recording is already in progress; run `bsk record stop` first"
)));
}
if record_recovery::exists() {
return Err(CliError::Local(anyhow::anyhow!(
"a previous recording was not exported; run `bsk record stop --output <dir>` to recover it first"
)));
}
validate_record_output(output)
run_result?;
session_stop_result?;
Ok(())
}
fn record_await_ipc_timeout(timeout_ms: u32) -> Duration {
@@ -278,66 +190,39 @@ fn record_await_ipc_timeout(timeout_ms: u32) -> Duration {
.unwrap_or(Duration::from_secs(u64::from(timeout_ms / 1_000) + 15))
}
fn render_finish(
trace: &RecordedTrace,
output_dir: &PathBuf,
exported: &ExportMeta,
format: Format,
) -> Result<(), CliError> {
let trace_path = trace_json_path(output_dir);
fn write_trace_file(output: &PathBuf, trace: &Trace) -> Result<(), CliError> {
let json = serde_json::to_string_pretty(trace)
.context("serialize trace JSON")
.map_err(CliError::Local)?;
if let Some(parent) = output.parent() {
if !parent.as_os_str().is_empty() {
fs::create_dir_all(parent)
.with_context(|| format!("create output directory {}", parent.display()))
.map_err(CliError::Local)?;
}
}
fs::write(output, format!("{json}\n"))
.with_context(|| format!("write trace to {}", output.display()))
.map_err(CliError::Local)?;
Ok(())
}
fn render_finish(trace: &Trace, output: &PathBuf, format: Format) -> Result<(), CliError> {
match format {
Format::Json => {
let payload = match trace {
RecordedTrace::V3(t) => serde_json::json!({
"output": output_dir,
"trace_json": trace_path,
"trace_version": exported.trace_version,
"states_dir": exported.states_dir,
"trace": t,
"window_closed": true,
}),
RecordedTrace::V2(t) => serde_json::json!({
"output": output_dir,
"trace_json": trace_path,
"trace_version": exported.trace_version,
"states_dir": null,
"trace": t,
"window_closed": true,
}),
};
println!(
"{}",
serde_json::to_string(&payload).map_err(|e| CliError::Local(anyhow::anyhow!(e)))?
serde_json::to_string(&serde_json::json!({
"output": output,
"trace": trace,
"window_closed": true,
}))
.map_err(|e| CliError::Local(anyhow::anyhow!(e)))?
);
}
Format::Human => match trace {
RecordedTrace::V3(t) => {
let states_dir = exported
.states_dir
.as_ref()
.map(|p| p.display().to_string())
.unwrap_or_else(|| states_dir_for_output(output_dir).display().to_string());
println!(
"saved {} steps to {} and {} states to {}",
t.steps.len(),
trace_path.display(),
t.states.len(),
states_dir
);
}
RecordedTrace::V2(t) => {
if exported.v2_fallback {
eprintln!(
"note: extension returned trace v2 (no page observations); update the BrowserSkill extension for v3 bundles"
);
}
println!(
"saved {} steps to {} (trace v2)",
t.steps.len(),
trace_path.display()
);
}
},
Format::Human => {
println!("saved {} steps to {}", trace.steps.len(), output.display());
}
}
Ok(())
}
@@ -345,89 +230,33 @@ fn render_finish(
fn render_stop(
result: &RecordStopResult,
output: &PathBuf,
exported: &ExportMeta,
format: Format,
) -> Result<(), CliError> {
render_finish(&result.trace, output, exported, format)
render_finish(&result.trace, output, format)
}
#[cfg(test)]
mod tests {
use super::*;
fn with_temp_home<F: FnOnce()>(f: F) {
let _lock = record_recovery::test_env_lock();
let tmp = tempfile::tempdir().unwrap();
unsafe {
std::env::set_var(crate::daemon::paths::BSK_HOME_ENV, tmp.path());
}
f();
record_recovery::clear();
record_state::clear();
unsafe {
std::env::remove_var(crate::daemon::paths::BSK_HOME_ENV);
}
}
#[test]
fn prepare_record_start_rejects_existing_trace_json_file() {
with_temp_home(|| {
let dir = tempfile::tempdir().unwrap();
let output = dir.path().join("trace.json");
std::fs::write(&output, "{}\n").unwrap();
let err = prepare_record_start(&output).unwrap_err();
let msg = err.to_string();
assert!(
msg.contains("JSON file") || msg.contains("json file"),
"{msg}"
);
assert!(
msg.contains("states/") || msg.contains("--output trace"),
"{msg}"
);
});
}
#[test]
fn prepare_record_start_rejects_unexported_recovery() {
with_temp_home(|| {
crate::daemon::paths::ensure_bsk_home().unwrap();
std::fs::write(
crate::daemon::paths::record_recovery_path().unwrap(),
"{}\n",
)
.unwrap();
let err = prepare_record_start(Path::new("trace")).unwrap_err();
let msg = err.to_string();
assert!(
msg.contains("not exported") || msg.contains("recover"),
"{msg}"
);
});
}
#[test]
fn default_output_is_trace_dir() {
fn default_output_is_trace_json() {
let args = RecordStopArgs {
output: PathBuf::from("trace"),
output: PathBuf::from("trace.json"),
};
assert_eq!(args.output, PathBuf::from("trace"));
assert_eq!(args.output, PathBuf::from("trace.json"));
}
#[test]
fn start_args_default_output_is_trace_dir() {
fn start_args_default_output_is_trace_json() {
let args = RecordStartArgs {
browser: None,
tab_id: None,
url: None,
purpose: None,
max_page_tokens: None,
redact_values: false,
output: PathBuf::from("trace"),
output: PathBuf::from("trace.json"),
};
assert_eq!(args.output, PathBuf::from("trace"));
assert_eq!(args.output, PathBuf::from("trace.json"));
}
#[test]
@@ -435,35 +264,4 @@ mod tests {
let got = record_await_ipc_timeout(RECORD_AWAIT_TIMEOUT_MS);
assert!(got >= Duration::from_secs(86_400));
}
#[test]
fn annotate_default_start_page_error_adds_url_reason() {
let err = CliError::from_rpc(bsk_protocol::RpcError {
code: ErrorCode::CdpFailed,
message: "Page.navigate rejected: net::ERR_ABORTED".into(),
data: None,
});
let annotated = annotate_default_start_page_error(err, None);
assert_eq!(
crate::cli::render_error::reason_for_data(annotated.data()),
Some(crate::cli::render_error::reason::RECORD_START_PAGE_UNREACHABLE)
);
let info =
crate::cli::render_error::info_for_error(annotated.code().unwrap(), annotated.data());
assert!(info.hint.unwrap().contains("--url"));
}
#[test]
fn annotate_default_start_page_error_skips_when_user_passed_url() {
let err = CliError::from_rpc(bsk_protocol::RpcError {
code: ErrorCode::CdpFailed,
message: "Page.navigate rejected: net::ERR_ABORTED".into(),
data: None,
});
let annotated = annotate_default_start_page_error(err, Some("https://www.example.org/"));
assert_eq!(
crate::cli::render_error::reason_for_data(annotated.data()),
None
);
}
}
-976
View File
@@ -1,976 +0,0 @@
use std::collections::HashSet;
use std::fs::{self, OpenOptions};
use std::io;
use std::path::{Path, PathBuf};
use anyhow::Context;
use bsk_protocol::tools::{RecordedTrace, TRACE_VERSION_V2, TRACE_VERSION_V3, TraceV2, TraceV3};
use serde::Serialize;
use crate::cli::error::CliError;
#[derive(Debug)]
pub(super) struct ExportMeta {
pub(super) states_dir: Option<PathBuf>,
pub(super) trace_version: u32,
/// True when the CLI requested v3 but the extension returned legacy v2.
pub(super) v2_fallback: bool,
}
pub(super) fn export_recorded_trace(
output_dir: &Path,
trace: &RecordedTrace,
) -> Result<ExportMeta, CliError> {
match trace {
RecordedTrace::V3(trace) => {
let states_dir = write_trace_bundle(output_dir, trace)?;
Ok(ExportMeta {
states_dir: Some(states_dir),
trace_version: TRACE_VERSION_V3,
v2_fallback: false,
})
}
RecordedTrace::V2(trace) => {
write_trace_v2(output_dir, trace)?;
Ok(ExportMeta {
states_dir: None,
trace_version: TRACE_VERSION_V2,
v2_fallback: true,
})
}
}
}
/// Save the completed Trace first, then write the bundle. Recovery is
/// removed only after the export commits so a bad `--output` cannot drop
/// a recording the extension already returned.
pub(super) fn export_with_recovery(
output_dir: &Path,
trace: &RecordedTrace,
) -> Result<ExportMeta, CliError> {
let save_result = crate::cli::record_recovery::save(trace);
match export_recorded_trace(output_dir, trace) {
Ok(meta) => {
crate::cli::record_recovery::clear();
Ok(meta)
}
Err(export_err) => {
let export_err = annotate_recovery_hint(export_err, save_result.is_ok());
if let Err(save_err) = save_result {
return Err(match export_err {
CliError::Local(inner) => CliError::Local(
inner.context(format!("also failed to save recovery data: {save_err}")),
),
other => other,
});
}
Err(export_err)
}
}
}
fn annotate_recovery_hint(err: CliError, recovery_saved: bool) -> CliError {
if !recovery_saved {
return err;
}
match err {
CliError::Local(inner) => CliError::Local(inner.context(
"export failed; the recorded trace was saved and can be recovered with `bsk record stop --output <dir>`",
)),
other => other,
}
}
fn looks_like_json_output(path: &Path) -> bool {
path.extension()
.and_then(|ext| ext.to_str())
.is_some_and(|ext| ext.eq_ignore_ascii_case("json"))
}
fn legacy_json_output_error(path: &Path) -> CliError {
CliError::Local(anyhow::anyhow!(
"--output {} is a JSON file path; Trace v3 writes a bundle directory \
(`<dir>/trace.json` and `<dir>/states/`), not a single file. \
Use `--output trace` or another directory.",
path.display()
))
}
/// Reject legacy `--output trace.json` (and any existing non-directory)
/// before a recording starts, so the user is not blocked only after Finish.
pub(super) fn validate_record_output(path: &Path) -> Result<(), CliError> {
match fs::symlink_metadata(path) {
Ok(metadata) if metadata.file_type().is_symlink() => Err(CliError::Local(anyhow::anyhow!(
"--output {} must not be a symlink",
path.display()
))),
Ok(metadata) if metadata.is_dir() => Ok(()),
Ok(_) if looks_like_json_output(path) => Err(legacy_json_output_error(path)),
Ok(_) => Err(CliError::Local(anyhow::anyhow!(
"--output {} is not a directory; Trace v3 writes a bundle directory \
(`<dir>/trace.json` and `<dir>/states/`). Use `--output trace`.",
path.display()
))),
Err(err) if err.kind() == io::ErrorKind::NotFound => {
if looks_like_json_output(path) {
Err(legacy_json_output_error(path))
} else {
Ok(())
}
}
Err(err) => Err(CliError::Local(
anyhow::Error::new(err).context(format!("inspect --output {}", path.display())),
)),
}
}
fn acquire_export_lock(output_dir: &Path) -> Result<std::fs::File, CliError> {
validate_bundle_directory(output_dir, "output bundle directory")?;
fs::create_dir_all(output_dir)
.with_context(|| format!("create output dir {}", output_dir.display()))
.map_err(CliError::Local)?;
let lock_path = output_dir.join(".bsk-record-export.lock");
let lock_file = OpenOptions::new()
.read(true)
.write(true)
.create(true)
.truncate(false)
.open(&lock_path)
.with_context(|| format!("open export lock {}", lock_path.display()))
.map_err(CliError::Local)?;
fs2::FileExt::lock_exclusive(&lock_file)
.with_context(|| format!("lock export bundle {}", output_dir.display()))
.map_err(CliError::Local)?;
Ok(lock_file)
}
pub(super) fn write_trace_v2(output_dir: &Path, trace: &TraceV2) -> Result<(), CliError> {
let trace_path = trace_json_path(output_dir);
let states_dir = states_dir_for_output(output_dir);
let json = serde_json::to_string_pretty(trace)
.context("serialize trace JSON")
.map_err(CliError::Local)?;
let _lock = acquire_export_lock(output_dir)?;
validate_replaceable_file(&trace_path, "trace JSON")?;
let transaction_id = uuid::Uuid::new_v4();
let staging_dir = output_dir.join(format!(".bsk-record-stage-{transaction_id}"));
fs::create_dir_all(&staging_dir)
.with_context(|| format!("create staging dir {}", staging_dir.display()))
.map_err(CliError::Local)?;
let staged_trace = staging_dir.join("trace.json");
let stage_result: Result<(), CliError> = (|| {
fs::write(&staged_trace, format!("{json}\n"))
.with_context(|| format!("write staged trace to {}", staged_trace.display()))
.map_err(CliError::Local)?;
Ok(())
})();
if let Err(err) = stage_result {
let _ = fs::remove_dir_all(&staging_dir);
return Err(err);
}
commit_export_transaction(
&staging_dir,
&states_dir,
&HashSet::new(),
transaction_id,
vec![(
staged_trace,
trace_path,
output_dir.join(format!(".trace.json.{transaction_id}.bak")),
)],
)?;
Ok(())
}
pub(super) fn states_dir_for_output(output_dir: &Path) -> PathBuf {
output_dir.join("states")
}
pub(super) fn trace_json_path(output_dir: &Path) -> PathBuf {
output_dir.join("trace.json")
}
fn is_canonical_state_id(id: &str) -> bool {
let Some(number) = id.strip_prefix('s') else {
return false;
};
let mut digits = number.bytes();
matches!(digits.next(), Some(b'1'..=b'9')) && digits.all(|byte| byte.is_ascii_digit())
}
fn validate_bundle_directory(path: &Path, description: &str) -> Result<(), CliError> {
match fs::symlink_metadata(path) {
Ok(metadata) if metadata.file_type().is_symlink() => Err(CliError::Local(anyhow::anyhow!(
"{description} {} must not be a symlink",
path.display()
))),
Ok(metadata) if metadata.is_dir() => Ok(()),
Ok(_) if looks_like_json_output(path) => Err(legacy_json_output_error(path)),
Ok(_) => Err(CliError::Local(anyhow::anyhow!(
"{description} {} is not a directory; Trace v3 writes `<dir>/trace.json` and `<dir>/states/`. Use `--output trace`.",
path.display()
))),
Err(err) if err.kind() == io::ErrorKind::NotFound => Ok(()),
Err(err) => Err(CliError::Local(
anyhow::Error::new(err).context(format!("inspect {description} {}", path.display())),
)),
}
}
fn validate_replaceable_file(path: &Path, description: &str) -> Result<(), CliError> {
match fs::symlink_metadata(path) {
Ok(metadata) if metadata.file_type().is_file() => Ok(()),
Ok(_) => Err(CliError::Local(anyhow::anyhow!(
"{description} {} is not a regular file",
path.display()
))),
Err(err) if err.kind() == io::ErrorKind::NotFound => Ok(()),
Err(err) => Err(CliError::Local(
anyhow::Error::new(err).context(format!("inspect {description} {}", path.display())),
)),
}
}
fn commit_staged_file(staged: &Path, target: &Path, backup: &Path) -> io::Result<Option<PathBuf>> {
let old_file = match fs::symlink_metadata(target) {
Ok(_) => {
fs::rename(target, backup)?;
Some(backup.to_path_buf())
}
Err(err) if err.kind() == io::ErrorKind::NotFound => None,
Err(err) => return Err(err),
};
if let Err(err) = fs::rename(staged, target) {
if let Some(old_file) = old_file.as_deref() {
if let Err(restore_err) = fs::rename(old_file, target) {
return Err(io::Error::new(
restore_err.kind(),
format!(
"{err}; additionally failed to restore {}: {restore_err}",
target.display()
),
));
}
}
return Err(err);
}
Ok(old_file)
}
fn rollback_installed_files(installed: &[(PathBuf, Option<PathBuf>)]) -> io::Result<()> {
let mut rollback_error = None;
for (target, backup) in installed.iter().rev() {
match fs::remove_file(target) {
Ok(()) => {}
Err(err) if err.kind() == io::ErrorKind::NotFound => {}
Err(err) => {
rollback_error.get_or_insert(err);
continue;
}
}
if let Some(backup) = backup {
if let Err(err) = fs::rename(backup, target) {
rollback_error.get_or_insert(err);
}
}
}
match rollback_error {
Some(err) => Err(err),
None => Ok(()),
}
}
fn is_generated_state_name(name: &str) -> bool {
name.strip_suffix(".txt").is_some_and(is_canonical_state_id)
}
fn restore_staged_stale_states(staged: &[(PathBuf, PathBuf)]) -> io::Result<()> {
let mut restore_error = None;
for (original, backup) in staged.iter().rev() {
if let Err(err) = fs::rename(backup, original) {
restore_error.get_or_insert(err);
}
}
match restore_error {
Some(err) => Err(err),
None => Ok(()),
}
}
/// Move stale generated states aside so replacement and cleanup form one
/// rollback boundary. User-owned files, including arbitrary non-`sN.txt`, stay.
fn stage_stale_states(
states_dir: &Path,
keep: &HashSet<String>,
transaction_id: uuid::Uuid,
) -> Result<Vec<(PathBuf, PathBuf)>, CliError> {
let entries = match fs::read_dir(states_dir) {
Ok(entries) => entries,
Err(err) if err.kind() == io::ErrorKind::NotFound => return Ok(Vec::new()),
Err(err) => {
return Err(CliError::Local(
anyhow::Error::new(err)
.context(format!("read states dir {}", states_dir.display())),
));
}
};
let mut staged = Vec::new();
for entry in entries {
let entry = match entry {
Ok(entry) => entry,
Err(err) => {
let restore_result = restore_staged_stale_states(&staged);
let message = match restore_result {
Ok(()) => format!("read stale state entry: {err}"),
Err(restore_err) => {
format!("read stale state entry: {err}; restore also failed: {restore_err}")
}
};
return Err(CliError::Local(anyhow::anyhow!(message)));
}
};
let name = entry.file_name();
let Some(name) = name.to_str() else { continue };
if !is_generated_state_name(name) || keep.contains(name) {
continue;
}
let original = entry.path();
let file_type = match entry.file_type() {
Ok(file_type) => file_type,
Err(err) => {
let restore_result = restore_staged_stale_states(&staged);
let message = match restore_result {
Ok(()) => format!("inspect stale state {}: {err}", original.display()),
Err(restore_err) => format!(
"inspect stale state {}: {err}; restore also failed: {restore_err}",
original.display()
),
};
return Err(CliError::Local(anyhow::anyhow!(message)));
}
};
if file_type.is_dir() {
let restore_result = restore_staged_stale_states(&staged);
let message = match restore_result {
Ok(()) => format!("stale state {} is a directory", original.display()),
Err(restore_err) => format!(
"stale state {} is a directory; restore also failed: {restore_err}",
original.display()
),
};
return Err(CliError::Local(anyhow::anyhow!(message)));
}
let backup = states_dir.join(format!(".{name}.{transaction_id}.stale"));
if let Err(err) = fs::rename(&original, &backup) {
let restore_result = restore_staged_stale_states(&staged);
let message = match restore_result {
Ok(()) => format!("stage stale state {}: {err}", original.display()),
Err(restore_err) => format!(
"stage stale state {}: {err}; restore also failed: {restore_err}",
original.display()
),
};
return Err(CliError::Local(anyhow::anyhow!(message)));
}
staged.push((original, backup));
}
Ok(staged)
}
fn commit_export_transaction(
staging_dir: &Path,
states_dir: &Path,
keep_states: &HashSet<String>,
transaction_id: uuid::Uuid,
replacements: Vec<(PathBuf, PathBuf, PathBuf)>,
) -> Result<(), CliError> {
let mut installed = Vec::with_capacity(replacements.len());
for (staged, target, backup) in replacements {
match commit_staged_file(&staged, &target, &backup) {
Ok(old_file) => installed.push((target, old_file)),
Err(err) => {
let rollback_result = rollback_installed_files(&installed);
let _ = fs::remove_dir_all(staging_dir);
let message = match rollback_result {
Ok(()) => format!("commit replacement {}: {err}", target.display()),
Err(rollback_err) => format!(
"commit replacement {}: {err}; rollback also failed: {rollback_err}",
target.display()
),
};
return Err(CliError::Local(anyhow::anyhow!(message)));
}
}
}
let stale_states = match stage_stale_states(states_dir, keep_states, transaction_id) {
Ok(stale) => stale,
Err(err) => {
let rollback_result = rollback_installed_files(&installed);
let _ = fs::remove_dir_all(staging_dir);
let message = match rollback_result {
Ok(()) => err.to_string(),
Err(rollback_err) => {
format!("{err}; replacement rollback also failed: {rollback_err}")
}
};
return Err(CliError::Local(anyhow::anyhow!(message)));
}
};
for (_, backup) in &installed {
if let Some(backup) = backup {
let _ = fs::remove_file(backup);
}
}
for (_, backup) in stale_states {
let _ = fs::remove_file(backup);
}
let _ = fs::remove_dir_all(staging_dir);
Ok(())
}
#[derive(Serialize)]
struct BundleState<'a> {
id: &'a str,
url: &'a str,
#[serde(skip_serializing_if = "Option::is_none")]
title: Option<&'a str>,
page: String,
#[serde(skip_serializing_if = "std::ops::Not::not")]
truncated: bool,
}
#[derive(Serialize)]
struct BundleTrace<'a> {
version: u32,
#[serde(skip_serializing_if = "Option::is_none")]
purpose: Option<&'a str>,
#[serde(skip_serializing_if = "Option::is_none")]
started_at: Option<&'a str>,
recorded_at: &'a str,
stopped_by: &'a bsk_protocol::tools::StopReason,
entry: &'a bsk_protocol::tools::TraceEntry,
recorder: &'a bsk_protocol::tools::RecorderInfo,
states: Vec<BundleState<'a>>,
steps: &'a [bsk_protocol::tools::StepV3],
}
pub(super) fn write_trace_bundle(output_dir: &Path, trace: &TraceV3) -> Result<PathBuf, CliError> {
let states_dir = states_dir_for_output(output_dir);
let trace_path = trace_json_path(output_dir);
if trace.version != TRACE_VERSION_V3 {
return Err(CliError::Local(anyhow::anyhow!(
"trace version {} is not supported (expected {})",
trace.version,
TRACE_VERSION_V3
)));
}
let mut state_ids = HashSet::with_capacity(trace.states.len());
for state in &trace.states {
if !is_canonical_state_id(&state.id) {
return Err(CliError::Local(anyhow::anyhow!(
"state id {:?} is not canonical (expected s1, s2, ...)",
state.id
)));
}
if !state_ids.insert(state.id.as_str()) {
return Err(CliError::Local(anyhow::anyhow!(
"duplicate state id {:?}",
state.id
)));
}
}
let mut disk_states = Vec::with_capacity(trace.states.len());
let mut state_writes = Vec::new();
let mut keep = HashSet::new();
for state in &trace.states {
let filename = format!("{}.txt", state.id);
let state_path = states_dir.join(&filename);
if state_path.parent() != Some(states_dir.as_path()) {
return Err(CliError::Local(anyhow::anyhow!(
"state path {} escapes states directory {}",
state_path.display(),
states_dir.display()
)));
}
keep.insert(filename.clone());
state_writes.push((filename.clone(), state.body.as_str()));
disk_states.push(BundleState {
id: &state.id,
url: &state.url,
title: state.title.as_deref(),
page: filename,
truncated: state.truncated,
});
}
let disk_trace = BundleTrace {
version: trace.version,
purpose: trace.purpose.as_deref(),
started_at: trace.started_at.as_deref(),
recorded_at: &trace.recorded_at,
stopped_by: &trace.stopped_by,
entry: &trace.entry,
recorder: &trace.recorder,
states: disk_states,
steps: &trace.steps,
};
let json = serde_json::to_string_pretty(&disk_trace)
.context("serialize trace JSON")
.map_err(CliError::Local)?;
validate_bundle_directory(output_dir, "output bundle directory")?;
fs::create_dir_all(output_dir)
.with_context(|| format!("create output dir {}", output_dir.display()))
.map_err(CliError::Local)?;
validate_bundle_directory(&states_dir, "states directory")?;
let _lock = acquire_export_lock(output_dir)?;
fs::create_dir_all(&states_dir)
.with_context(|| format!("create states dir {}", states_dir.display()))
.map_err(CliError::Local)?;
for (filename, _) in &state_writes {
validate_replaceable_file(&states_dir.join(filename), "state observation")?;
}
validate_replaceable_file(&trace_path, "trace JSON")?;
let transaction_id = uuid::Uuid::new_v4();
let staging_dir = output_dir.join(format!(".bsk-record-stage-{transaction_id}"));
let staging_states = staging_dir.join("states");
fs::create_dir_all(&staging_states)
.with_context(|| format!("create staging dir {}", staging_states.display()))
.map_err(CliError::Local)?;
let stage_result: Result<(), CliError> = (|| {
for (filename, body) in &state_writes {
let staged_path = staging_states.join(filename);
fs::write(&staged_path, body)
.with_context(|| {
format!("write staged state observation {}", staged_path.display())
})
.map_err(CliError::Local)?;
}
let staged_trace = staging_dir.join("trace.json");
fs::write(&staged_trace, format!("{json}\n"))
.with_context(|| format!("write staged trace to {}", staged_trace.display()))
.map_err(CliError::Local)?;
Ok(())
})();
if let Err(err) = stage_result {
let _ = fs::remove_dir_all(&staging_dir);
return Err(err);
}
let mut replacements: Vec<(PathBuf, PathBuf, PathBuf)> =
Vec::with_capacity(state_writes.len() + 1);
for (filename, _) in &state_writes {
replacements.push((
staging_states.join(filename),
states_dir.join(filename),
states_dir.join(format!(".{filename}.{transaction_id}.bak")),
));
}
replacements.push((
staging_dir.join("trace.json"),
trace_path.clone(),
output_dir.join(format!(".trace.json.{transaction_id}.bak")),
));
commit_export_transaction(
&staging_dir,
&states_dir,
&keep,
transaction_id,
replacements,
)?;
Ok(states_dir)
}
#[cfg(test)]
mod tests {
use std::fs;
use super::*;
use bsk_protocol::tools::{
NavigationCause, PageRefV2 as PageRef, RecorderInfo, StepCommonV3 as StepCommon,
StepResultV3 as StepResult, StepV3 as Step, StopReason, TRACE_VERSION_V2, TRACE_VERSION_V3,
TraceEntry, TraceStateV3, TraceV2, TraceV3, VOM_FORMAT_VERSION,
};
fn sample_trace(state_id: &str, body: &str) -> TraceV3 {
TraceV3 {
version: TRACE_VERSION_V3,
recorded_at: "2026-08-10T02:12:55.360Z".into(),
stopped_by: StopReason::UserFinish,
entry: TraceEntry {
start_url: "https://example.com/".into(),
},
recorder: RecorderInfo {
bsk: "0.1.10".into(),
vom: VOM_FORMAT_VERSION,
},
states: vec![TraceStateV3 {
id: state_id.into(),
url: "https://example.com/".into(),
title: Some("Example".into()),
body: body.into(),
truncated: false,
}],
steps: vec![Step::Navigate {
common: StepCommon {
id: 1,
state: state_id.into(),
result: StepResult {
state: state_id.into(),
},
},
to: "https://example.com/".into(),
cause: NavigationCause::UserTyped,
}],
purpose: None,
started_at: None,
}
}
#[test]
fn write_trace_bundle_writes_dir_layout() {
let dir = tempfile::tempdir().unwrap();
let output = dir.path().join("flow");
let trace = sample_trace("s1", "@vom 1\nL1 page");
let states_dir = write_trace_bundle(&output, &trace).unwrap();
let trace_path = output.join("trace.json");
assert!(trace_path.exists());
assert_eq!(states_dir, output.join("states"));
assert!(states_dir.join("s1.txt").exists());
let disk: serde_json::Value =
serde_json::from_str(&fs::read_to_string(&trace_path).unwrap()).unwrap();
assert_eq!(disk["states"][0]["page"], "s1.txt");
assert!(disk["states"][0].get("body").is_none());
assert_eq!(
fs::read_to_string(states_dir.join("s1.txt")).unwrap(),
"@vom 1\nL1 page"
);
// Re-exporting drops states from the previous run but keeps the
// directory in place for the concurrent writer.
fs::write(states_dir.join("s9.txt"), "stale").unwrap();
fs::write(states_dir.join("notes.md"), "keep me").unwrap();
fs::write(states_dir.join("reference.txt"), "user file").unwrap();
write_trace_bundle(&output, &trace).unwrap();
assert!(!states_dir.join("s9.txt").exists());
assert!(states_dir.join("s1.txt").exists());
assert!(states_dir.join("notes.md").exists());
assert_eq!(
fs::read_to_string(states_dir.join("reference.txt")).unwrap(),
"user file"
);
}
#[test]
fn write_trace_bundle_rejects_traversal_state_id() {
let dir = tempfile::tempdir().unwrap();
let output = dir.path().join("flow");
let trace = sample_trace("../escaped", "@vom 1\nL1 page");
let err = write_trace_bundle(&output, &trace).unwrap_err();
assert!(err.to_string().contains("state id"));
assert!(!output.join("escaped.txt").exists());
assert!(!output.join("trace.json").exists());
}
#[test]
fn write_trace_bundle_rejects_duplicate_state_id() {
let dir = tempfile::tempdir().unwrap();
let output = dir.path().join("flow");
let mut trace = sample_trace("s1", "@vom 1\nL1 first");
trace.states.push(TraceStateV3 {
body: "@vom 1\nL1 duplicate".into(),
..trace.states[0].clone()
});
let err = write_trace_bundle(&output, &trace).unwrap_err();
assert!(err.to_string().contains("duplicate state id"));
assert!(!output.exists());
}
#[test]
fn unsupported_version_preserves_existing_bundle() {
let dir = tempfile::tempdir().unwrap();
let output = dir.path().join("flow");
write_trace_bundle(&output, &sample_trace("s1", "original")).unwrap();
let original_json = fs::read(output.join("trace.json")).unwrap();
let mut replacement = sample_trace("s2", "replacement");
replacement.version += 1;
let err = write_trace_bundle(&output, &replacement).unwrap_err();
assert!(err.to_string().contains("not supported"));
assert_eq!(fs::read(output.join("trace.json")).unwrap(), original_json);
assert_eq!(
fs::read_to_string(output.join("states/s1.txt")).unwrap(),
"original"
);
assert!(!output.join("states/s2.txt").exists());
}
#[test]
fn state_replacement_failure_preserves_existing_bundle() {
let dir = tempfile::tempdir().unwrap();
let output = dir.path().join("flow");
write_trace_bundle(&output, &sample_trace("s1", "original")).unwrap();
let original_json = fs::read(output.join("trace.json")).unwrap();
fs::write(output.join("states/notes.md"), "keep me").unwrap();
fs::create_dir(output.join("states/s2.txt")).unwrap();
let err = write_trace_bundle(&output, &sample_trace("s2", "replacement")).unwrap_err();
assert!(err.to_string().contains("state"));
assert_eq!(fs::read(output.join("trace.json")).unwrap(), original_json);
assert_eq!(
fs::read_to_string(output.join("states/s1.txt")).unwrap(),
"original"
);
assert_eq!(
fs::read_to_string(output.join("states/notes.md")).unwrap(),
"keep me"
);
}
#[test]
fn stale_state_cleanup_failure_rolls_back_replacement() {
let dir = tempfile::tempdir().unwrap();
let output = dir.path().join("flow");
write_trace_bundle(&output, &sample_trace("s1", "original")).unwrap();
let original_json = fs::read(output.join("trace.json")).unwrap();
fs::create_dir(output.join("states/s9.txt")).unwrap();
let err = write_trace_bundle(&output, &sample_trace("s2", "replacement")).unwrap_err();
assert!(err.to_string().contains("stale state"));
assert_eq!(fs::read(output.join("trace.json")).unwrap(), original_json);
assert_eq!(
fs::read_to_string(output.join("states/s1.txt")).unwrap(),
"original"
);
assert!(!output.join("states/s2.txt").exists());
assert!(output.join("states/s9.txt").is_dir());
}
#[cfg(unix)]
#[test]
fn write_trace_bundle_rejects_symlinked_states_directory() {
use std::os::unix::fs::symlink;
let dir = tempfile::tempdir().unwrap();
let output = dir.path().join("flow");
let external_states = dir.path().join("external-states");
fs::create_dir_all(&output).unwrap();
fs::create_dir_all(&external_states).unwrap();
fs::write(external_states.join("s9.txt"), "external").unwrap();
symlink(&external_states, output.join("states")).unwrap();
let err = write_trace_bundle(&output, &sample_trace("s1", "replacement")).unwrap_err();
assert!(err.to_string().contains("symlink"));
assert_eq!(
fs::read_to_string(external_states.join("s9.txt")).unwrap(),
"external"
);
assert!(!external_states.join("s1.txt").exists());
assert!(!output.join("trace.json").exists());
}
#[cfg(unix)]
#[test]
fn write_trace_bundle_rejects_symlinked_output_directory() {
use std::os::unix::fs::symlink;
let dir = tempfile::tempdir().unwrap();
let external_output = dir.path().join("external-output");
let output = dir.path().join("flow");
fs::create_dir_all(&external_output).unwrap();
fs::write(external_output.join("notes.md"), "external").unwrap();
symlink(&external_output, &output).unwrap();
let err = write_trace_bundle(&output, &sample_trace("s1", "replacement")).unwrap_err();
assert!(err.to_string().contains("symlink"));
assert_eq!(
fs::read_to_string(external_output.join("notes.md")).unwrap(),
"external"
);
assert!(!external_output.join("trace.json").exists());
assert!(!external_output.join("states").exists());
}
#[test]
fn write_trace_v2_writes_single_json_and_prunes_stale_states() {
let dir = tempfile::tempdir().unwrap();
let output = dir.path().join("flow");
fs::create_dir_all(output.join("states")).unwrap();
fs::write(output.join("states/s1.txt"), "stale").unwrap();
let trace = TraceV2 {
recorded_at: "2026-08-10T02:12:55.360Z".into(),
started_at: None,
purpose: None,
entry: bsk_protocol::tools::TraceEntry {
start_url: "https://example.com/".into(),
},
pages: vec![PageRef {
id: "p1".into(),
url: "https://example.com/".into(),
title: None,
}],
steps: vec![],
};
write_trace_v2(&output, &trace).unwrap();
assert!(output.join("trace.json").exists());
assert!(!output.join("states/s1.txt").exists());
}
#[test]
fn write_trace_v2_stale_cleanup_failure_rolls_back_trace() {
let dir = tempfile::tempdir().unwrap();
let output = dir.path().join("flow");
write_trace_bundle(&output, &sample_trace("s1", "original")).unwrap();
let original_json = fs::read(output.join("trace.json")).unwrap();
fs::create_dir(output.join("states/s9.txt")).unwrap();
let trace = TraceV2 {
recorded_at: "2026-08-10T02:12:55.360Z".into(),
started_at: None,
purpose: None,
entry: bsk_protocol::tools::TraceEntry {
start_url: "https://example.com/".into(),
},
pages: vec![],
steps: vec![],
};
let err = write_trace_v2(&output, &trace).unwrap_err();
assert!(err.to_string().contains("stale state"));
assert_eq!(fs::read(output.join("trace.json")).unwrap(), original_json);
assert_eq!(
fs::read_to_string(output.join("states/s1.txt")).unwrap(),
"original"
);
assert!(output.join("states/s9.txt").is_dir());
}
#[test]
fn export_recorded_trace_accepts_v2() {
let dir = tempfile::tempdir().unwrap();
let output = dir.path().join("flow");
let trace = RecordedTrace::V2(TraceV2 {
recorded_at: "2026-08-10T02:12:55.360Z".into(),
started_at: None,
purpose: None,
entry: bsk_protocol::tools::TraceEntry {
start_url: "https://example.com/".into(),
},
pages: vec![],
steps: vec![],
});
let exported = export_recorded_trace(&output, &trace).unwrap();
assert_eq!(exported.trace_version, TRACE_VERSION_V2);
assert!(exported.states_dir.is_none());
assert!(output.join("trace.json").exists());
}
#[test]
fn existing_trace_json_file_is_rejected_with_bundle_usage() {
let dir = tempfile::tempdir().unwrap();
let output = dir.path().join("trace.json");
fs::write(&output, "{}\n").unwrap();
let err = validate_record_output(&output).unwrap_err();
let msg = err.to_string();
assert!(
msg.contains("JSON file") || msg.contains("json file"),
"should name the legacy JSON file usage: {msg}"
);
assert!(
msg.contains("states/") || msg.contains("--output trace"),
"should hint at the new bundle directory usage: {msg}"
);
assert_eq!(fs::read_to_string(&output).unwrap(), "{}\n");
}
#[test]
fn missing_trace_json_path_is_rejected_as_legacy_output() {
let err = validate_record_output(Path::new("trace.json")).unwrap_err();
let msg = err.to_string();
assert!(
msg.contains("JSON file") || msg.contains("json file"),
"{msg}"
);
assert!(
msg.contains("states/") || msg.contains("--output trace"),
"{msg}"
);
}
#[test]
fn export_failure_keeps_recovery_for_retry() {
let _lock = crate::cli::record_recovery::test_env_lock();
let tmp = tempfile::tempdir().unwrap();
unsafe {
std::env::set_var(crate::daemon::paths::BSK_HOME_ENV, tmp.path());
}
let dir = tempfile::tempdir().unwrap();
let bad_output = dir.path().join("trace.json");
fs::write(&bad_output, "{}\n").unwrap();
let trace = RecordedTrace::V3(sample_trace("s1", "@vom 1\nL1 page"));
let err = export_with_recovery(&bad_output, &trace).unwrap_err();
let msg = err.to_string();
assert!(
msg.contains("recover") || msg.contains("record stop"),
"export failure should tell the user how to recover: {msg}"
);
assert!(
crate::cli::record_recovery::exists(),
"recovery data must survive a failed bundle export"
);
let recovered = crate::cli::record_recovery::load()
.unwrap()
.expect("recovery file");
assert_eq!(recovered, trace);
assert_eq!(fs::read_to_string(&bad_output).unwrap(), "{}\n");
let good_output = dir.path().join("bundle");
export_with_recovery(&good_output, &recovered).unwrap();
assert!(good_output.join("trace.json").exists());
assert!(good_output.join("states/s1.txt").exists());
assert!(
!crate::cli::record_recovery::exists(),
"successful export must delete recovery data"
);
unsafe {
std::env::remove_var(crate::daemon::paths::BSK_HOME_ENV);
}
}
}
-112
View File
@@ -1,112 +0,0 @@
//! Persist a completed Trace so a failed bundle export can be retried.
use std::fs;
use anyhow::{Context, Result};
use bsk_protocol::tools::RecordedTrace;
use crate::daemon::paths;
pub fn save(trace: &RecordedTrace) -> Result<()> {
let path = paths::record_recovery_path()?;
paths::ensure_bsk_home()?;
let json = serde_json::to_string_pretty(trace).context("serialize record recovery")?;
let tmp = path.with_extension("tmp");
fs::write(&tmp, format!("{json}\n")).with_context(|| format!("write {}", tmp.display()))?;
fs::rename(&tmp, &path).with_context(|| format!("commit {}", path.display()))?;
Ok(())
}
pub fn load() -> Result<Option<RecordedTrace>> {
let path = paths::record_recovery_path()?;
if !path.exists() {
return Ok(None);
}
let raw = fs::read_to_string(&path).with_context(|| format!("read {}", path.display()))?;
let trace = serde_json::from_str(&raw).with_context(|| format!("parse {}", path.display()))?;
Ok(Some(trace))
}
pub fn exists() -> bool {
paths::record_recovery_path()
.map(|path| path.exists())
.unwrap_or(false)
}
pub fn clear() {
if let Ok(path) = paths::record_recovery_path() {
let _ = fs::remove_file(path);
}
}
#[cfg(test)]
pub(crate) fn test_env_lock() -> std::sync::MutexGuard<'static, ()> {
crate::daemon::paths::test_env_lock()
}
#[cfg(test)]
mod tests {
use super::*;
use bsk_protocol::tools::{
RecorderInfo, StopReason, TRACE_VERSION_V3, TraceEntry, TraceStateV3, TraceV3,
VOM_FORMAT_VERSION,
};
fn sample_trace() -> RecordedTrace {
RecordedTrace::V3(TraceV3 {
version: TRACE_VERSION_V3,
recorded_at: "2026-08-10T02:12:55.360Z".into(),
stopped_by: StopReason::UserFinish,
entry: TraceEntry {
start_url: "https://example.com/".into(),
},
recorder: RecorderInfo {
bsk: "0.1.10".into(),
vom: VOM_FORMAT_VERSION,
},
states: vec![TraceStateV3 {
id: "s1".into(),
url: "https://example.com/".into(),
title: Some("Example".into()),
body: "@vom 1\nL1 page".into(),
truncated: false,
}],
steps: vec![],
purpose: None,
started_at: None,
})
}
fn with_temp_home<F: FnOnce()>(f: F) {
let _lock = test_env_lock();
let tmp = tempfile::tempdir().unwrap();
unsafe {
std::env::set_var(crate::daemon::paths::BSK_HOME_ENV, tmp.path());
}
f();
clear();
unsafe {
std::env::remove_var(crate::daemon::paths::BSK_HOME_ENV);
}
}
#[test]
fn save_load_round_trips() {
with_temp_home(|| {
let trace = sample_trace();
save(&trace).unwrap();
assert!(exists());
assert_eq!(load().unwrap(), Some(trace));
});
}
#[test]
fn clear_removes_recovery_file() {
with_temp_home(|| {
save(&sample_trace()).unwrap();
clear();
assert!(!exists());
assert_eq!(load().unwrap(), None);
});
}
}
+7 -1
View File
@@ -59,9 +59,15 @@ fn started_at_unix_ms() -> String {
#[cfg(test)]
mod tests {
use super::*;
use std::sync::Mutex;
fn env_guard() -> &'static Mutex<()> {
static GUARD: Mutex<()> = Mutex::new(());
&GUARD
}
fn with_temp_home<F: FnOnce()>(f: F) {
let _lock = crate::daemon::paths::test_env_lock();
let _lock = env_guard().lock().unwrap_or_else(|e| e.into_inner());
let tmp = tempfile::tempdir().unwrap();
unsafe {
std::env::set_var(crate::daemon::paths::BSK_HOME_ENV, tmp.path());
+78 -23
View File
@@ -49,8 +49,13 @@ pub mod reason {
pub const TAB_NOT_ACTIVE: &str = "tab_not_active";
pub const BORROW_CONFLICT: &str = "borrow_conflict";
pub const SCREENSHOT_CAPTURE_FAILED: &str = "screenshot_capture_failed";
pub const FILE_INPUT_PROBE_FAILED: &str = "file_input_probe_failed";
pub const FILE_INPUT_NOT_ACTIVATED: &str = "file_input_not_activated";
pub const SET_FILE_INPUT_FAILED: &str = "set_file_input_failed";
pub const DOWNLOAD_CAPTURE_FAILED: &str = "download_capture_failed";
pub const TRANSFER_OUTCOME_UNKNOWN: &str = "transfer_outcome_unknown";
pub const TRANSFER_TIMEOUT: &str = "transfer_timeout";
pub const SESSION_BUSY: &str = crate::rpc_reason::SESSION_BUSY;
pub const RECORD_START_PAGE_UNREACHABLE: &str = "record_start_page_unreachable";
}
/// Per-code rendering metadata. The `summary` is what the CLI prints
@@ -197,6 +202,20 @@ pub fn info_for_error(code: ErrorCode, data: Option<&serde_json::Value>) -> Rend
return base;
};
match (code, reason) {
(_, reason::TRANSFER_OUTCOME_UNKNOWN) => RenderInfo {
summary: "the file transfer outcome could not be confirmed",
hint: Some(
"do not retry the transfer: the browser may already have applied it; inspect the page or stop safely",
),
exit_code: base.exit_code,
},
(ErrorCode::Timeout, reason::TRANSFER_TIMEOUT) => RenderInfo {
summary: "the file transfer timed out after browser dispatch",
hint: Some(
"do not retry when effect_state is unknown or committed; inspect the page before taking another action",
),
exit_code: base.exit_code,
},
(ErrorCode::PermissionDenied, reason::ELEMENT_NOT_VISIBLE) => RenderInfo {
summary: "target element has no visible geometry",
hint: Some(
@@ -270,13 +289,33 @@ pub fn info_for_error(code: ErrorCode, data: Option<&serde_json::Value>) -> Rend
),
exit_code: base.exit_code,
},
(
ErrorCode::CdpFailed | ErrorCode::Timeout | ErrorCode::InvalidParams,
reason::RECORD_START_PAGE_UNREACHABLE,
) => RenderInfo {
summary: "default start page https://example.com/ could not be opened",
(ErrorCode::Unsupported, reason::FILE_INPUT_NOT_ACTIVATED) => RenderInfo {
summary: "the upload trigger did not activate a file input",
hint: Some(
"that site may be unreachable from your network. Pass --url with a page you can open, e.g. `bsk record start --url https://<site>/`",
"the page may use a non-input picker such as showOpenFilePicker(); do not retry blindly — use `bsk request-help` with the exact original local path, or stop if human help is disabled",
),
exit_code: base.exit_code,
},
(ErrorCode::Timeout | ErrorCode::CdpFailed, reason::FILE_INPUT_PROBE_FAILED) => {
RenderInfo {
summary: "the browser upload transaction could not be established",
hint: Some(
"do not retry the same upload blindly; use `bsk request-help` when human help is available",
),
exit_code: base.exit_code,
}
}
(ErrorCode::Timeout | ErrorCode::CdpFailed, reason::SET_FILE_INPUT_FAILED) => RenderInfo {
summary: "the browser could not attach the staged file to the input",
hint: Some(
"check that BrowserSkill has Chrome's 'Allow access to file URLs' permission; otherwise use `bsk request-help`",
),
exit_code: base.exit_code,
},
(ErrorCode::CdpFailed, reason::DOWNLOAD_CAPTURE_FAILED) => RenderInfo {
summary: "the browser download could not be attributed or completed",
hint: Some(
"do not retry blindly; the target browser may not expose a reliable download intent signal",
),
exit_code: base.exit_code,
},
@@ -389,22 +428,6 @@ mod tests {
assert_eq!(info.exit_code, 4);
}
#[test]
fn record_start_page_unreachable_overrides_cdp_failed_copy() {
let data = serde_json::json!({ "reason": reason::RECORD_START_PAGE_UNREACHABLE });
let info = info_for_error(ErrorCode::CdpFailed, Some(&data));
assert_eq!(
info.summary,
"default start page https://example.com/ could not be opened"
);
assert!(
info.hint.unwrap().contains("--url"),
"expected --url hint, got {:?}",
info.hint
);
assert_eq!(info.exit_code, 3);
}
#[test]
fn screenshot_capture_failed_overrides_cdp_failed_copy() {
let data = serde_json::json!({ "reason": reason::SCREENSHOT_CAPTURE_FAILED });
@@ -419,6 +442,38 @@ mod tests {
assert_eq!(info.exit_code, 3);
}
#[test]
fn file_transfer_reasons_render_actionable_fallbacks() {
let unsupported = serde_json::json!({ "reason": reason::FILE_INPUT_NOT_ACTIVATED });
let info = info_for_error(ErrorCode::Unsupported, Some(&unsupported));
assert_eq!(
info.summary,
"the upload trigger did not activate a file input"
);
assert!(info.hint.unwrap().contains("exact original local path"));
let control = serde_json::json!({ "reason": reason::FILE_INPUT_PROBE_FAILED });
let info = info_for_error(ErrorCode::Timeout, Some(&control));
assert!(
info.summary
.contains("transaction could not be established")
);
assert!(info.hint.unwrap().contains("do not retry"));
let download = serde_json::json!({ "reason": reason::DOWNLOAD_CAPTURE_FAILED });
let info = info_for_error(ErrorCode::CdpFailed, Some(&download));
assert!(info.summary.contains("download could not be attributed"));
let unknown = serde_json::json!({ "reason": reason::TRANSFER_OUTCOME_UNKNOWN });
let info = info_for_error(ErrorCode::ProtocolError, Some(&unknown));
assert!(info.summary.contains("outcome could not be confirmed"));
assert!(info.hint.unwrap().contains("do not retry"));
let timeout = serde_json::json!({ "reason": reason::TRANSFER_TIMEOUT });
let info = info_for_error(ErrorCode::Timeout, Some(&timeout));
assert!(info.summary.contains("timed out after browser dispatch"));
}
#[test]
fn ref_not_found_overrides_not_found_copy() {
let data = serde_json::json!({ "reason": reason::REF_NOT_FOUND });
+185
View File
@@ -0,0 +1,185 @@
//! `bsk upload` — stage caller-readable files and attach them to a page.
use std::fs::File;
use std::io::Read;
use std::path::{Path, PathBuf};
use std::time::Duration;
use anyhow::Context;
use base64::Engine;
use bsk_protocol::Method;
use bsk_protocol::tools::{
TransferBeginParams, TransferBeginResult, TransferChunkParams, TransferChunkResult,
TransferIdParams, TransferReadyResult, TransferReleaseResult, UploadFile, UploadParams,
UploadResult,
};
use clap::Args;
use crate::cli::ensure_daemon::ensure_daemon;
use crate::cli::error::{CliError, Format};
use crate::cli::interaction::split_target;
use crate::cli::navigate::parse_timeout_ms;
#[derive(Debug, Clone, Args)]
pub struct UploadArgs {
/// Snapshot ref (`@e3`) or CSS selector for the file input / chooser trigger.
pub target: Option<String>,
#[arg(long = "ref")]
pub ref_: Option<String>,
#[arg(long = "selector")]
pub selector: Option<String>,
/// Local file to upload. Repeat for a multiple-file input.
#[arg(long = "file", required = true)]
pub files: Vec<PathBuf>,
#[arg(long)]
pub session: String,
#[arg(long = "tab-id")]
pub tab_id: Option<i64>,
#[arg(long, default_value = "2m", value_parser = parse_timeout_ms)]
pub timeout: u32,
}
pub fn dispatch(args: UploadArgs, format: Format) -> Result<(), CliError> {
if args.files.len() > crate::daemon::file_transfer::MAX_UPLOAD_FILES {
return Err(CliError::Local(anyhow::anyhow!(
"upload accepts at most {} files",
crate::daemon::file_transfer::MAX_UPLOAD_FILES
)));
}
let info = ensure_daemon().context("ensure daemon is running")?;
let (ref_, selector) = split_target(args.target, args.ref_, args.selector)?;
let mut staged = Vec::new();
for path in &args.files {
match stage_file(&info.sock_path, &args.session, path) {
Ok(file) => staged.push(file),
Err(err) => {
for (id, _) in &staged {
let _ = release(&info.sock_path, id);
}
return Err(err);
}
}
}
let params = UploadParams {
session_id: args.session,
ref_,
selector,
tab_id: args.tab_id,
files: staged
.iter()
.map(|(id, name)| UploadFile {
transfer_id: id.clone(),
name: name.clone(),
staged_path: None,
})
.collect(),
timeout_ms: Some(args.timeout),
};
let result = crate::cli::business_rpc::call::<_, UploadResult>(
info.sock_path.clone(),
"upload",
Method::ToolUpload,
Some(params),
ipc_timeout(args.timeout),
);
// Once tool.upload is dispatched, staging ownership belongs to the
// session. A transport timeout cannot prove that Chrome did not attach
// the file, so releasing here could invalidate a late successful attach.
// Session teardown remains the single cleanup boundary after dispatch.
let reply = result?;
match format {
Format::Json => println!("{}", serde_json::to_string_pretty(&reply).unwrap()),
Format::Human => println!(
"upload ok tab={} files={}",
reply.tab_id,
reply.file_names.join(", ")
),
}
Ok(())
}
fn stage_file(sock: &Path, session: &str, path: &PathBuf) -> Result<(String, String), CliError> {
let mut file = File::open(path)
.with_context(|| format!("open upload file {}", path.display()))
.map_err(CliError::Local)?;
let meta = file
.metadata()
.with_context(|| format!("read upload file metadata {}", path.display()))
.map_err(CliError::Local)?;
if !meta.is_file() {
return Err(CliError::Local(anyhow::anyhow!(
"upload source is not a regular file: {}",
path.display()
)));
}
let name = path
.file_name()
.and_then(|v| v.to_str())
.filter(|v| !v.is_empty())
.ok_or_else(|| CliError::Local(anyhow::anyhow!("upload file has no valid basename")))?
.to_string();
let begin: TransferBeginResult = crate::cli::business_rpc::call(
sock.to_path_buf(),
"transfer-begin",
Method::TransferBegin,
Some(TransferBeginParams {
session_id: session.to_string(),
name: name.clone(),
byte_size: meta.len(),
}),
Duration::from_secs(10),
)?;
let staged = (|| {
let mut offset = 0u64;
let mut buf = vec![0u8; begin.chunk_size as usize];
loop {
let n = file.read(&mut buf).map_err(|e| CliError::Local(e.into()))?;
if n == 0 {
break;
}
let reply: TransferChunkResult = crate::cli::business_rpc::call(
sock.to_path_buf(),
"transfer-chunk",
Method::TransferChunk,
Some(TransferChunkParams {
transfer_id: begin.transfer_id.clone(),
offset,
data_base64: base64::engine::general_purpose::STANDARD.encode(&buf[..n]),
}),
Duration::from_secs(30),
)?;
offset = reply.next_offset;
}
let _: TransferReadyResult = crate::cli::business_rpc::call(
sock.to_path_buf(),
"transfer-finish",
Method::TransferFinish,
Some(TransferIdParams {
transfer_id: begin.transfer_id.clone(),
}),
Duration::from_secs(10),
)?;
Ok::<_, CliError>(())
})();
if let Err(err) = staged {
let _ = release(sock, &begin.transfer_id);
return Err(err);
}
Ok((begin.transfer_id, name))
}
fn release(sock: &Path, id: &str) -> Result<TransferReleaseResult, CliError> {
crate::cli::business_rpc::call(
sock.to_path_buf(),
"transfer-release",
Method::TransferRelease,
Some(TransferIdParams {
transfer_id: id.to_string(),
}),
Duration::from_secs(5),
)
}
fn ipc_timeout(timeout_ms: u32) -> Duration {
Duration::from_millis(u64::from(timeout_ms) + 5_000)
}
+728
View File
@@ -0,0 +1,728 @@
//! Session-scoped, content-transparent staging for upload/download tools.
//!
//! The registry deliberately exposes no arbitrary-path operations. Upload
//! bytes arrive from the invoking CLI in bounded chunks; browser downloads
//! land in a daemon-minted directory and are read back by the invoking CLI.
use std::collections::HashMap;
use std::fs::{self, File, OpenOptions};
use std::io::{Read, Seek, SeekFrom, Write};
use std::path::{Component, Path, PathBuf};
use std::sync::Mutex;
use base64::Engine;
use bsk_protocol::tools::{
TransferBeginParams, TransferBeginResult, TransferChunkParams, TransferChunkResult,
TransferIdParams, TransferReadyResult, TransferReleaseResult,
};
use bsk_protocol::{ErrorCode, RpcError};
use uuid::Uuid;
use super::paths;
pub const TRANSFER_CHUNK_SIZE: u32 = 512 * 1024;
pub const MAX_TRANSFER_BYTES: u64 = 512 * 1024 * 1024;
pub const MAX_UPLOAD_FILES: usize = 20;
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum Direction {
Upload,
Download,
}
#[derive(Debug)]
struct Entry {
session_id: String,
direction: Direction,
path: PathBuf,
expected_size: Option<u64>,
written: u64,
ready: bool,
}
/// Browser download path after the daemon has verified that it belongs to
/// the capability minted for this transfer. From this point onward the daemon
/// owns cleanup on every error path; unvalidated arbitrary paths are never
/// removed.
#[derive(Debug)]
struct ValidatedBrowserFile {
path: PathBuf,
parent: PathBuf,
cleanup_armed: bool,
}
impl ValidatedBrowserFile {
fn validate(reported_path: &Path, transfer_id: &str) -> Result<Self, RpcError> {
if !reported_path.is_absolute() {
return Err(permission(
"download path is outside its browser capability",
));
}
let expected_parent = Path::new("BrowserSkill").join(transfer_id);
let parent = reported_path
.parent()
.ok_or_else(|| permission("download path has no parent directory"))?;
if !parent.ends_with(&expected_parent) {
return Err(permission(
"download escaped its browser capability directory",
));
}
for path in [reported_path, parent] {
if fs::symlink_metadata(path)
.map_err(io_error)?
.file_type()
.is_symlink()
{
return Err(permission("download capability path contains a symlink"));
}
}
Ok(Self {
path: reported_path.to_path_buf(),
parent: parent.to_path_buf(),
cleanup_armed: true,
})
}
fn remove_source(mut self) -> Result<(), RpcError> {
remove_if_present(&self.path).map_err(io_error)?;
remove_dir_if_present(&self.parent).map_err(io_error)?;
self.cleanup_armed = false;
Ok(())
}
}
impl Drop for ValidatedBrowserFile {
fn drop(&mut self) {
if !self.cleanup_armed {
return;
}
let _ = remove_if_present(&self.path);
let _ = remove_dir_if_present(&self.parent);
}
}
#[derive(Debug)]
pub struct DownloadStaging {
pub transfer_id: String,
pub browser_relative_dir: String,
}
#[derive(Debug)]
pub struct TransferRegistry {
root: PathBuf,
initialized: Mutex<bool>,
entries: Mutex<HashMap<String, Entry>>,
}
impl TransferRegistry {
pub fn new() -> anyhow::Result<Self> {
Ok(Self {
root: paths::bsk_home()?.join("run").join("transfers"),
initialized: Mutex::new(false),
entries: Mutex::new(HashMap::new()),
})
}
#[cfg(test)]
fn at_root(root: PathBuf) -> anyhow::Result<Self> {
let registry = Self {
root,
initialized: Mutex::new(false),
entries: Mutex::new(HashMap::new()),
};
registry.ensure_root()?;
Ok(registry)
}
pub fn initialize(&self) -> anyhow::Result<()> {
self.ensure_root()
}
fn ensure_root(&self) -> anyhow::Result<()> {
let mut initialized = self.initialized.lock().unwrap();
if *initialized {
return Ok(());
}
if self.root.exists() {
// Transfers never survive a daemon; removing stale directories on
// startup gives crash cleanup without a second persistence model.
let _ = fs::remove_dir_all(&self.root);
}
fs::create_dir_all(&self.root)?;
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
fs::set_permissions(&self.root, fs::Permissions::from_mode(0o700))?;
}
*initialized = true;
Ok(())
}
pub fn begin_upload(&self, p: TransferBeginParams) -> Result<TransferBeginResult, RpcError> {
self.ensure_root()
.map_err(|err| protocol_error(err.to_string()))?;
let basename = safe_upload_basename(&p.name)?;
if p.byte_size > MAX_TRANSFER_BYTES {
return Err(invalid(format!(
"file size {} exceeds transfer limit {}",
p.byte_size, MAX_TRANSFER_BYTES
)));
}
let mut entries = self.entries.lock().unwrap();
let session_bytes: u64 = entries
.values()
.filter(|entry| entry.session_id == p.session_id)
.map(|entry| entry.expected_size.unwrap_or(entry.written))
.sum();
if session_bytes.saturating_add(p.byte_size) > MAX_TRANSFER_BYTES {
return Err(invalid(format!(
"session transfer staging exceeds limit {}",
MAX_TRANSFER_BYTES
)));
}
let id = new_id();
let dir = self.root.join(&id);
fs::create_dir(&dir).map_err(io_error)?;
set_private_dir(&dir).map_err(io_error)?;
let path = dir.join(basename);
let file = OpenOptions::new()
.write(true)
.create_new(true)
.open(&path)
.map_err(io_error)?;
set_private_file(&file).map_err(io_error)?;
entries.insert(
id.clone(),
Entry {
session_id: p.session_id,
direction: Direction::Upload,
path,
expected_size: Some(p.byte_size),
written: 0,
ready: false,
},
);
Ok(TransferBeginResult {
transfer_id: id,
chunk_size: TRANSFER_CHUNK_SIZE,
})
}
pub fn write_chunk(&self, p: TransferChunkParams) -> Result<TransferChunkResult, RpcError> {
let bytes = base64::engine::general_purpose::STANDARD
.decode(p.data_base64)
.map_err(|e| invalid(format!("invalid transfer chunk: {e}")))?;
if bytes.len() > TRANSFER_CHUNK_SIZE as usize {
return Err(invalid("transfer chunk exceeds negotiated size"));
}
let mut entries = self.entries.lock().unwrap();
let entry = entries
.get_mut(&p.transfer_id)
.ok_or_else(|| not_found("transfer not found"))?;
if entry.direction != Direction::Upload || entry.ready {
return Err(invalid("transfer is not writable"));
}
if p.offset != entry.written {
return Err(invalid(format!(
"non-sequential transfer offset: expected {}, got {}",
entry.written, p.offset
)));
}
let next = entry.written.saturating_add(bytes.len() as u64);
if next > entry.expected_size.unwrap_or(MAX_TRANSFER_BYTES) {
return Err(invalid("transfer exceeds declared byte size"));
}
let mut file = OpenOptions::new()
.append(true)
.open(&entry.path)
.map_err(io_error)?;
file.write_all(&bytes).map_err(io_error)?;
entry.written = next;
Ok(TransferChunkResult {
next_offset: next,
eof: false,
data_base64: None,
})
}
pub fn finish_upload(&self, p: TransferIdParams) -> Result<TransferReadyResult, RpcError> {
let mut entries = self.entries.lock().unwrap();
let entry = entries
.get_mut(&p.transfer_id)
.ok_or_else(|| not_found("transfer not found"))?;
if entry.direction != Direction::Upload {
return Err(invalid("transfer is not an upload"));
}
if entry.written != entry.expected_size.unwrap_or(entry.written) {
return Err(invalid(format!(
"incomplete transfer: expected {} bytes, received {}",
entry.expected_size.unwrap_or(0),
entry.written
)));
}
entry.ready = true;
Ok(TransferReadyResult {
transfer_id: p.transfer_id,
byte_size: entry.written,
})
}
pub fn resolve_uploads(
&self,
session_id: &str,
ids: &[String],
) -> Result<Vec<PathBuf>, RpcError> {
let entries = self.entries.lock().unwrap();
ids.iter()
.map(|id| {
let entry = entries
.get(id)
.ok_or_else(|| not_found("upload transfer not found"))?;
if entry.session_id != session_id
|| entry.direction != Direction::Upload
|| !entry.ready
{
return Err(permission(
"upload transfer is outside this session or not ready",
));
}
Ok(entry.path.clone())
})
.collect()
}
pub fn begin_download(&self, session_id: &str) -> Result<DownloadStaging, RpcError> {
self.ensure_root()
.map_err(|err| protocol_error(err.to_string()))?;
let id = new_id();
let dir = self.root.join(&id);
fs::create_dir(&dir).map_err(io_error)?;
set_private_dir(&dir).map_err(io_error)?;
self.entries.lock().unwrap().insert(
id.clone(),
Entry {
session_id: session_id.to_string(),
direction: Direction::Download,
path: dir.clone(),
expected_size: None,
written: 0,
ready: false,
},
);
Ok(DownloadStaging {
browser_relative_dir: format!("BrowserSkill/{id}"),
transfer_id: id,
})
}
pub fn import_download(&self, id: &str, reported_path: &Path) -> Result<u64, RpcError> {
let mut entries = self.entries.lock().unwrap();
let entry = entries
.get_mut(id)
.ok_or_else(|| not_found("download transfer not found"))?;
if entry.direction != Direction::Download {
return Err(permission("transfer is not a download capability"));
}
let browser_file = ValidatedBrowserFile::validate(reported_path, id)?;
let canonical = browser_file.path.canonicalize().map_err(io_error)?;
let meta = fs::metadata(&canonical).map_err(io_error)?;
if !meta.is_file() || meta.len() > MAX_TRANSFER_BYTES {
return Err(invalid(
"download is not a regular file or exceeds the transfer limit",
));
}
let destination = entry.path.join("payload");
let source = File::open(&canonical).map_err(io_error)?;
let mut destination_file = OpenOptions::new()
.write(true)
.create_new(true)
.open(&destination)
.map_err(io_error)?;
let copied = std::io::copy(
&mut source.take(MAX_TRANSFER_BYTES.saturating_add(1)),
&mut destination_file,
)
.map_err(io_error)?;
if copied > MAX_TRANSFER_BYTES {
let _ = fs::remove_file(&destination);
return Err(invalid("download exceeds the transfer limit"));
}
entry.path = destination;
entry.written = copied;
entry.expected_size = Some(copied);
entry.ready = true;
browser_file.remove_source()?;
Ok(copied)
}
pub fn read_chunk(&self, p: TransferChunkParams) -> Result<TransferChunkResult, RpcError> {
let entries = self.entries.lock().unwrap();
let entry = entries
.get(&p.transfer_id)
.ok_or_else(|| not_found("transfer not found"))?;
if entry.direction != Direction::Download || !entry.ready {
return Err(invalid("transfer is not readable"));
}
if p.offset > entry.written {
return Err(invalid("read offset is beyond transfer length"));
}
let mut file = File::open(&entry.path).map_err(io_error)?;
file.seek(SeekFrom::Start(p.offset)).map_err(io_error)?;
let mut buf = vec![0u8; TRANSFER_CHUNK_SIZE as usize];
let count = file.read(&mut buf).map_err(io_error)?;
buf.truncate(count);
let next = p.offset + count as u64;
Ok(TransferChunkResult {
next_offset: next,
eof: next >= entry.written,
data_base64: Some(base64::engine::general_purpose::STANDARD.encode(buf)),
})
}
pub fn release(&self, p: TransferIdParams) -> TransferReleaseResult {
let entry = self.entries.lock().unwrap().remove(&p.transfer_id);
if let Some(entry) = entry {
let dir = if entry.path.is_dir() {
entry.path
} else {
entry.path.parent().unwrap_or(&self.root).to_path_buf()
};
let _ = fs::remove_dir_all(dir);
TransferReleaseResult { released: true }
} else {
TransferReleaseResult { released: false }
}
}
pub fn release_session(&self, session_id: &str) {
let ids: Vec<String> = self
.entries
.lock()
.unwrap()
.iter()
.filter(|(_, entry)| entry.session_id == session_id)
.map(|(id, _)| id.clone())
.collect();
for id in ids {
self.release(TransferIdParams { transfer_id: id });
}
}
}
fn new_id() -> String {
format!("tr_{}", Uuid::new_v4().simple())
}
fn set_private_dir(path: &Path) -> std::io::Result<()> {
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
fs::set_permissions(path, fs::Permissions::from_mode(0o700))?;
}
Ok(())
}
fn set_private_file(file: &File) -> std::io::Result<()> {
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
file.set_permissions(fs::Permissions::from_mode(0o600))?;
}
Ok(())
}
fn remove_if_present(path: &Path) -> std::io::Result<()> {
match fs::remove_file(path) {
Ok(()) => Ok(()),
Err(err) if err.kind() == std::io::ErrorKind::NotFound => Ok(()),
Err(err) => Err(err),
}
}
fn remove_dir_if_present(path: &Path) -> std::io::Result<()> {
match fs::remove_dir(path) {
Ok(()) => Ok(()),
Err(err) if err.kind() == std::io::ErrorKind::NotFound => Ok(()),
Err(err) => Err(err),
}
}
fn safe_upload_basename(name: &str) -> Result<&str, RpcError> {
if name.is_empty() || name.contains(['/', '\\', '\0']) {
return Err(invalid("upload name must be a safe basename"));
}
let mut components = Path::new(name).components();
match (components.next(), components.next()) {
(Some(Component::Normal(_)), None) => Ok(name),
_ => Err(invalid("upload name must be a safe basename")),
}
}
fn invalid(message: impl Into<String>) -> RpcError {
RpcError {
code: ErrorCode::InvalidParams,
message: message.into(),
data: None,
}
}
fn not_found(message: impl Into<String>) -> RpcError {
RpcError {
code: ErrorCode::NotFound,
message: message.into(),
data: None,
}
}
fn permission(message: impl Into<String>) -> RpcError {
RpcError {
code: ErrorCode::PermissionDenied,
message: message.into(),
data: None,
}
}
fn io_error(err: std::io::Error) -> RpcError {
RpcError {
code: ErrorCode::ProtocolError,
message: err.to_string(),
data: None,
}
}
fn protocol_error(message: impl Into<String>) -> RpcError {
RpcError {
code: ErrorCode::ProtocolError,
message: message.into(),
data: None,
}
}
#[cfg(test)]
mod tests {
use super::*;
fn registry() -> (tempfile::TempDir, TransferRegistry) {
let temp = tempfile::tempdir().unwrap();
let registry = TransferRegistry::at_root(temp.path().join("transfers")).unwrap();
(temp, registry)
}
#[test]
fn upload_is_not_resolvable_until_complete_or_from_another_session() {
let (_temp, registry) = registry();
let begin = registry
.begin_upload(TransferBeginParams {
session_id: "s1".into(),
name: "image.png".into(),
byte_size: 3,
})
.unwrap();
assert!(
registry
.resolve_uploads("s1", std::slice::from_ref(&begin.transfer_id))
.is_err()
);
registry
.write_chunk(TransferChunkParams {
transfer_id: begin.transfer_id.clone(),
offset: 0,
data_base64: base64::engine::general_purpose::STANDARD.encode(b"abc"),
})
.unwrap();
registry
.finish_upload(TransferIdParams {
transfer_id: begin.transfer_id.clone(),
})
.unwrap();
assert!(
registry
.resolve_uploads("s2", std::slice::from_ref(&begin.transfer_id))
.is_err()
);
let [path] = registry
.resolve_uploads("s1", &[begin.transfer_id])
.unwrap()
.try_into()
.unwrap();
assert_eq!(path.file_name().unwrap(), "image.png");
assert_eq!(fs::read(path).unwrap(), b"abc");
}
#[test]
fn upload_name_must_be_one_safe_path_component() {
let (_temp, registry) = registry();
for name in ["", ".", "..", "../secret", "folder/file", "folder\\file"] {
let result = registry.begin_upload(TransferBeginParams {
session_id: "s1".into(),
name: name.into(),
byte_size: 0,
});
assert!(result.is_err(), "accepted unsafe upload name");
}
assert!(registry.entries.lock().unwrap().is_empty());
assert!(fs::read_dir(&registry.root).unwrap().next().is_none());
}
#[test]
fn same_upload_names_are_isolated_by_transfer_directory() {
let (_temp, registry) = registry();
let first = registry
.begin_upload(TransferBeginParams {
session_id: "s1".into(),
name: "image.png".into(),
byte_size: 0,
})
.unwrap();
let second = registry
.begin_upload(TransferBeginParams {
session_id: "s1".into(),
name: "image.png".into(),
byte_size: 0,
})
.unwrap();
let entries = registry.entries.lock().unwrap();
let first_path = &entries[&first.transfer_id].path;
let second_path = &entries[&second.transfer_id].path;
assert_ne!(first_path.parent(), second_path.parent());
assert_eq!(first_path.file_name().unwrap(), "image.png");
assert_eq!(second_path.file_name().unwrap(), "image.png");
}
#[cfg(unix)]
#[test]
fn upload_staging_is_private() {
use std::os::unix::fs::PermissionsExt;
let (_temp, registry) = registry();
let begin = registry
.begin_upload(TransferBeginParams {
session_id: "s1".into(),
name: "private.png".into(),
byte_size: 0,
})
.unwrap();
let entries = registry.entries.lock().unwrap();
let path = &entries[&begin.transfer_id].path;
assert_eq!(
fs::metadata(path.parent().unwrap())
.unwrap()
.permissions()
.mode()
& 0o777,
0o700
);
assert_eq!(
fs::metadata(path).unwrap().permissions().mode() & 0o777,
0o600
);
}
#[test]
fn download_must_import_from_its_browser_capability_directory() {
let (temp, registry) = registry();
let staging = registry.begin_download("s1").unwrap();
let outside = temp.path().join("outside.bin");
fs::write(&outside, b"secret").unwrap();
assert!(
registry
.import_download(&staging.transfer_id, &outside)
.is_err()
);
assert_eq!(fs::read(&outside).unwrap(), b"secret");
let browser_dir = temp
.path()
.join("Downloads")
.join("BrowserSkill")
.join(&staging.transfer_id);
fs::create_dir_all(&browser_dir).unwrap();
let inside = browser_dir.join("result.bin");
fs::write(&inside, b"result").unwrap();
assert_eq!(
registry
.import_download(&staging.transfer_id, &inside)
.unwrap(),
6
);
assert!(!inside.exists());
let chunk = registry
.read_chunk(TransferChunkParams {
transfer_id: staging.transfer_id,
offset: 0,
data_base64: String::new(),
})
.unwrap();
assert_eq!(
base64::engine::general_purpose::STANDARD
.decode(chunk.data_base64.unwrap())
.unwrap(),
b"result"
);
}
#[test]
fn validated_oversized_browser_download_is_removed_on_rejection() {
let (temp, registry) = registry();
let staging = registry.begin_download("s1").unwrap();
let browser_dir = temp
.path()
.join("Downloads")
.join("BrowserSkill")
.join(&staging.transfer_id);
fs::create_dir_all(&browser_dir).unwrap();
let inside = browser_dir.join("oversized.bin");
File::create(&inside)
.unwrap()
.set_len(MAX_TRANSFER_BYTES + 1)
.unwrap();
assert!(
registry
.import_download(&staging.transfer_id, &inside)
.is_err()
);
assert!(!inside.exists());
assert!(!browser_dir.exists());
}
#[test]
fn releasing_a_session_removes_all_staging() {
let (_temp, registry) = registry();
let first = registry.begin_download("s1").unwrap();
let second = registry.begin_download("s2").unwrap();
let first_directory = registry.root.join(&first.transfer_id);
let second_directory = registry.root.join(&second.transfer_id);
registry.release_session("s1");
assert!(!first_directory.exists());
assert!(second_directory.exists());
}
#[test]
fn upload_staging_is_bounded_across_a_session() {
let (_temp, registry) = registry();
registry
.begin_upload(TransferBeginParams {
session_id: "s1".into(),
name: "large.bin".into(),
byte_size: MAX_TRANSFER_BYTES,
})
.unwrap();
assert!(
registry
.begin_upload(TransferBeginParams {
session_id: "s1".into(),
name: "one-more-byte.bin".into(),
byte_size: 1,
})
.is_err()
);
assert!(
registry
.begin_upload(TransferBeginParams {
session_id: "s2".into(),
name: "other-session.bin".into(),
byte_size: 1,
})
.is_ok()
);
}
}
+218 -93
View File
@@ -27,7 +27,8 @@ use bsk_protocol::system::{
VersionSkewEntry,
};
use bsk_protocol::tools::{
HelpOutcome, RequestHelpResult, ReturnFailure, WaitMsParams, WaitMsResult,
DownloadParams, DownloadResult, HelpOutcome, RequestHelpResult, ReturnFailure,
TransferBeginParams, TransferIdParams, UploadParams, WaitMsParams, WaitMsResult,
};
use bsk_protocol::{
CancelParams, CancelResult, ErrorCode, Method, PingResult, ResponseBody, RpcError, RpcId,
@@ -61,6 +62,10 @@ pub type RpcHandler = Arc<
>;
const DEFAULT_RPC_TIMEOUT: Duration = Duration::from_secs(15);
// The upload transaction owns its operation deadline and may need a bounded
// cleanup before it can return a useful structured error. Keep only that
// transport alive slightly longer so it does not replace the result.
const EXTENSION_RESPONSE_GRACE: Duration = Duration::from_secs(2);
/// Upper bound on `wait_for_browser_ms` accepted over IPC.
const MAX_BROWSER_WAIT: Duration = Duration::from_secs(60);
// `session.stop` fast-fails while another tool is active; this budget
@@ -206,15 +211,15 @@ pub fn full_handler(status: DaemonStatus, state: Arc<DaemonState>) -> RpcHandler
Ok(v) => ResponseBody::Ok(v),
Err(e) => ResponseBody::Err(e),
},
Method::SessionStart => match handle_session_start(&state, rpc_id, params).await {
Method::SessionStart => match handle_session_start(&state, params).await {
Ok(v) => ResponseBody::Ok(v),
Err(e) => ResponseBody::Err(e),
},
Method::SessionStop => match handle_session_stop(&state, rpc_id, params).await {
Method::SessionStop => match handle_session_stop(&state, params).await {
Ok(v) => ResponseBody::Ok(v),
Err(e) => ResponseBody::Err(e),
},
Method::SessionStopAll => match handle_session_stop_all_rpc(&state, rpc_id).await {
Method::SessionStopAll => match handle_session_stop_all(&state).await {
Ok(v) => ResponseBody::Ok(v),
Err(e) => ResponseBody::Err(e),
},
@@ -223,6 +228,11 @@ pub fn full_handler(status: DaemonStatus, state: Arc<DaemonState>) -> RpcHandler
Ok(v) => ResponseBody::Ok(v),
Err(e) => ResponseBody::Err(e),
},
Method::TransferBegin => handle_transfer_begin(&state, params),
Method::TransferChunk => handle_transfer_chunk(&state, params),
Method::TransferFinish => handle_transfer_finish(&state, params),
Method::TransferRead => handle_transfer_read(&state, params),
Method::TransferRelease => handle_transfer_release(&state, params),
Method::ToolTabList
| Method::ToolTabCreate
| Method::ToolTabClose
@@ -246,6 +256,8 @@ pub fn full_handler(status: DaemonStatus, state: Arc<DaemonState>) -> RpcHandler
| Method::ToolFill
| Method::ToolPress
| Method::ToolSelect
| Method::ToolUpload
| Method::ToolDownload
| Method::ToolEvaluate
| Method::ToolWaitForNavigation
| Method::ToolRequestHelp
@@ -324,7 +336,7 @@ async fn handle_tool_dispatch(
data: None,
});
}
let timeout = match tool_dispatch_timeout(&params) {
let timeout = match tool_dispatch_transport_timeout(&method, &params) {
Ok(timeout) => timeout,
Err(err) => return ResponseBody::Err(err),
};
@@ -338,24 +350,180 @@ async fn handle_tool_dispatch(
});
}
};
let mut params = params;
let mut download_transfer_id: Option<String> = None;
if method == Method::ToolUpload {
let mut upload: UploadParams = match serde_json::from_value(params) {
Ok(v) => v,
Err(err) => return ResponseBody::Err(invalid_params(err.to_string())),
};
if upload.files.is_empty() || upload.files.len() > super::file_transfer::MAX_UPLOAD_FILES {
return ResponseBody::Err(invalid_params(format!(
"upload requires 1..={} files",
super::file_transfer::MAX_UPLOAD_FILES
)));
}
let ids: Vec<String> = upload.files.iter().map(|f| f.transfer_id.clone()).collect();
let paths = match state.transfers.resolve_uploads(&session_id.0, &ids) {
Ok(v) => v,
Err(err) => return ResponseBody::Err(err),
};
for (file, path) in upload.files.iter_mut().zip(paths) {
file.staged_path = Some(path.to_string_lossy().into_owned());
}
params = serde_json::to_value(upload).unwrap_or(Value::Null);
} else if method == Method::ToolDownload {
let mut download: DownloadParams = match serde_json::from_value(params) {
Ok(v) => v,
Err(err) => return ResponseBody::Err(invalid_params(err.to_string())),
};
let staging = match state.transfers.begin_download(&session_id.0) {
Ok(v) => v,
Err(err) => return ResponseBody::Err(err),
};
download.browser_relative_dir = Some(staging.browser_relative_dir);
download.max_byte_size = Some(super::file_transfer::MAX_TRANSFER_BYTES);
download_transfer_id = Some(staging.transfer_id);
params = serde_json::to_value(download).unwrap_or(Value::Null);
}
let entry = inflight_guard.entry();
// `record_stop` must reach the extension while `record_await` holds the
// serial busy lock — finishing the recording unblocks await.
let outcome = if method == Method::ToolRecordStop {
state
.tool_queues
.dispatch_unlocked(&session_id, method, params, timeout, Some(entry))
.dispatch_unlocked(&session_id, method.clone(), params, timeout, Some(entry))
.await
} else {
state
.tool_queues
.dispatch(&session_id, method, params, timeout, Some(entry))
.dispatch(&session_id, method.clone(), params, timeout, Some(entry))
.await
};
drop(inflight_guard);
match outcome {
Ok(v) if method == Method::ToolDownload => {
let id = download_transfer_id.expect("download transfer allocated");
let mut result: DownloadResult = match serde_json::from_value(v) {
Ok(v) => v,
Err(err) => {
state
.transfers
.release(TransferIdParams { transfer_id: id });
return ResponseBody::Err(RpcError {
code: ErrorCode::ProtocolError,
message: format!("invalid tool.download result: {err}"),
data: None,
});
}
};
let Some(path) = result.browser_path.take() else {
state
.transfers
.release(TransferIdParams { transfer_id: id });
return ResponseBody::Err(RpcError {
code: ErrorCode::ProtocolError,
message: "tool.download returned no browser_path".into(),
data: None,
});
};
match state
.transfers
.import_download(&id, std::path::Path::new(&path))
{
Ok(size) => {
result.byte_size = size;
result.transfer_id = Some(id);
ResponseBody::Ok(serde_json::to_value(result).unwrap_or(Value::Null))
}
Err(err) => {
state
.transfers
.release(TransferIdParams { transfer_id: id });
ResponseBody::Err(err)
}
}
}
Ok(v) => ResponseBody::Ok(v),
Err(err) => ResponseBody::Err(err.into_rpc()),
Err(err) => {
if let Some(id) = download_transfer_id {
state
.transfers
.release(TransferIdParams { transfer_id: id });
}
ResponseBody::Err(err.into_rpc())
}
}
}
fn handle_transfer_begin(state: &Arc<DaemonState>, params: Value) -> ResponseBody {
let p: TransferBeginParams = match serde_json::from_value(params) {
Ok(v) => v,
Err(err) => return ResponseBody::Err(invalid_params(err.to_string())),
};
if state
.sessions
.get(&SessionId(p.session_id.clone()))
.is_none()
{
return ResponseBody::Err(RpcError {
code: ErrorCode::NotFound,
message: format!("session {} unknown", p.session_id),
data: None,
});
}
match state.transfers.begin_upload(p) {
Ok(v) => ResponseBody::Ok(serde_json::to_value(v).unwrap_or(Value::Null)),
Err(err) => ResponseBody::Err(err),
}
}
fn handle_transfer_chunk(state: &Arc<DaemonState>, params: Value) -> ResponseBody {
let p = match serde_json::from_value(params) {
Ok(v) => v,
Err(err) => return ResponseBody::Err(invalid_params(err.to_string())),
};
match state.transfers.write_chunk(p) {
Ok(v) => ResponseBody::Ok(serde_json::to_value(v).unwrap_or(Value::Null)),
Err(err) => ResponseBody::Err(err),
}
}
fn handle_transfer_finish(state: &Arc<DaemonState>, params: Value) -> ResponseBody {
let p = match serde_json::from_value(params) {
Ok(v) => v,
Err(err) => return ResponseBody::Err(invalid_params(err.to_string())),
};
match state.transfers.finish_upload(p) {
Ok(v) => ResponseBody::Ok(serde_json::to_value(v).unwrap_or(Value::Null)),
Err(err) => ResponseBody::Err(err),
}
}
fn handle_transfer_read(state: &Arc<DaemonState>, params: Value) -> ResponseBody {
let p = match serde_json::from_value(params) {
Ok(v) => v,
Err(err) => return ResponseBody::Err(invalid_params(err.to_string())),
};
match state.transfers.read_chunk(p) {
Ok(v) => ResponseBody::Ok(serde_json::to_value(v).unwrap_or(Value::Null)),
Err(err) => ResponseBody::Err(err),
}
}
fn handle_transfer_release(state: &Arc<DaemonState>, params: Value) -> ResponseBody {
let p: TransferIdParams = match serde_json::from_value(params) {
Ok(v) => v,
Err(err) => return ResponseBody::Err(invalid_params(err.to_string())),
};
ResponseBody::Ok(serde_json::to_value(state.transfers.release(p)).unwrap_or(Value::Null))
}
fn invalid_params(message: impl Into<String>) -> RpcError {
RpcError {
code: ErrorCode::InvalidParams,
message: message.into(),
data: None,
}
}
@@ -434,9 +602,8 @@ async fn handle_wait_ms(
/// cancellation surfaces (M10.2 + review C2):
///
/// 1. [`AbortRegistry`] — answers daemon-local cancellable runners
/// (`tool.wait_ms` and `session.*` lifecycle calls). If a token is
/// registered we trip it and stop. Lifecycle handlers forward their
/// own WS cancel after preserving request-before-cancel ordering.
/// (currently `tool.wait_ms`). If a token is registered we trip it
/// and stop.
/// 2. [`super::inflight::ToolInflightRegistry`] — every IPC-tracked
/// `tool.*` RPC, registered the moment the IPC handler accepts the
/// request. Trips the entry's cancel token regardless of whether
@@ -533,6 +700,16 @@ fn tool_dispatch_timeout(params: &Value) -> Result<Duration, RpcError> {
Ok(Duration::from_millis(u64::from(ms)))
}
fn tool_dispatch_transport_timeout(method: &Method, params: &Value) -> Result<Duration, RpcError> {
tool_dispatch_timeout(params).map(|timeout| {
if matches!(method, Method::ToolUpload | Method::ToolDownload) {
timeout.saturating_add(EXTENSION_RESPONSE_GRACE)
} else {
timeout
}
})
}
// Local CLI-facing shapes. Intentionally distinct from
// `bsk_protocol::tools::SessionStart*` (which describes the WS-facing
// `tool.session_*` round-trip with the extension) so the two sides can
@@ -635,20 +812,7 @@ fn clamp_browser_wait(wait_ms: Option<u64>) -> Option<Duration> {
))
}
async fn handle_session_start(
state: &Arc<DaemonState>,
rpc_id: RpcId,
params: Value,
) -> Result<Value, RpcError> {
let abort_guard = state
.abort_registry
.register(rpc_id)
.map_err(|err| RpcError {
code: ErrorCode::ProtocolError,
message: format!("session.start cancellation registration failed: {err:?}"),
data: None,
})?;
let cancel = abort_guard.token().clone();
async fn handle_session_start(state: &Arc<DaemonState>, params: Value) -> Result<Value, RpcError> {
let params: CliSessionStartParams = if params.is_null() {
CliSessionStartParams {
browser_instance_id: None,
@@ -687,7 +851,6 @@ async fn handle_session_start(
},
state.config.extension_connect_wait,
DEFAULT_RPC_TIMEOUT,
Some(cancel),
)
.await
{
@@ -711,8 +874,6 @@ fn map_start_error(err: StartSessionError) -> RpcError {
StartSessionError::AmbiguousBrowserLabel { .. } => ErrorCode::InvalidParams,
StartSessionError::IdExhausted => ErrorCode::ProtocolError,
StartSessionError::Timeout => ErrorCode::Timeout,
StartSessionError::Cancelled => ErrorCode::Cancelled,
StartSessionError::CleanupFailed { .. } => ErrorCode::ProtocolError,
StartSessionError::TransportClosed => ErrorCode::ProtocolError,
StartSessionError::ExtensionError(inner) => inner.code,
};
@@ -728,16 +889,6 @@ fn map_start_error(err: StartSessionError) -> RpcError {
"label": label,
"instance_ids": instance_ids,
})),
StartSessionError::CleanupFailed {
session_id,
agent_window_id,
..
} => Some(serde_json::json!({
"reason": "cleanup_failed",
"session_id": session_id.0,
"agent_window_id": agent_window_id,
})),
StartSessionError::ExtensionError(inner) => inner.data.clone(),
_ => None,
};
RpcError {
@@ -747,20 +898,7 @@ fn map_start_error(err: StartSessionError) -> RpcError {
}
}
async fn handle_session_stop(
state: &Arc<DaemonState>,
rpc_id: RpcId,
params: Value,
) -> Result<Value, RpcError> {
let abort_guard = state
.abort_registry
.register(rpc_id)
.map_err(|err| RpcError {
code: ErrorCode::ProtocolError,
message: format!("session.stop cancellation registration failed: {err:?}"),
data: None,
})?;
let cancel = abort_guard.token().clone();
async fn handle_session_stop(state: &Arc<DaemonState>, params: Value) -> Result<Value, RpcError> {
let params: CliSessionStopParams = if params.is_null() {
CliSessionStopParams {
session_id: None,
@@ -774,7 +912,7 @@ async fn handle_session_stop(
})?
};
if params.all {
return handle_session_stop_all(state, Some(cancel)).await;
return handle_session_stop_all(state).await;
}
let session_id = match params.session_id {
Some(s) => SessionId(s),
@@ -793,11 +931,11 @@ async fn handle_session_stop(
&state.session_interrupts,
&session_id,
DEFAULT_SESSION_STOP_TIMEOUT,
Some(cancel),
)
.await
{
Ok(stop) => {
state.transfers.release_session(&session_id.0);
let result = CliSessionStopResult {
stopped: vec![session_id.0],
failed: Vec::new(),
@@ -845,25 +983,7 @@ fn map_stop_error(err: StopSessionError) -> RpcError {
}
}
async fn handle_session_stop_all_rpc(
state: &Arc<DaemonState>,
rpc_id: RpcId,
) -> Result<Value, RpcError> {
let abort_guard = state
.abort_registry
.register(rpc_id)
.map_err(|err| RpcError {
code: ErrorCode::ProtocolError,
message: format!("session.stop_all cancellation registration failed: {err:?}"),
data: None,
})?;
handle_session_stop_all(state, Some(abort_guard.token().clone())).await
}
async fn handle_session_stop_all(
state: &Arc<DaemonState>,
cancel: Option<super::abort::AbortToken>,
) -> Result<Value, RpcError> {
async fn handle_session_stop_all(state: &Arc<DaemonState>) -> Result<Value, RpcError> {
let ids: Vec<SessionId> = state
.sessions
.snapshot()
@@ -875,16 +995,6 @@ async fn handle_session_stop_all(
let mut returned_tab_ids = Vec::new();
let mut return_failures = Vec::new();
for id in ids {
if cancel
.as_ref()
.is_some_and(super::abort::AbortToken::is_cancelled)
{
return Err(RpcError {
code: ErrorCode::Cancelled,
message: "session.stop_all was cancelled".into(),
data: None,
});
}
match stop_session(
&state.browsers,
&state.sessions,
@@ -892,11 +1002,11 @@ async fn handle_session_stop_all(
&state.session_interrupts,
&id,
DEFAULT_SESSION_STOP_TIMEOUT,
cancel.clone(),
)
.await
{
Ok(stop) => {
state.transfers.release_session(&id.0);
stopped.push(id.0);
returned_tab_ids.extend(stop.returned_tab_ids);
return_failures.extend(stop.return_failures);
@@ -918,15 +1028,6 @@ async fn handle_session_stop_all(
returned_tab_ids.extend(stop.returned_tab_ids);
return_failures.extend(stop.return_failures);
}
Err(err)
if matches!(
&err,
StopSessionError::ExtensionError(inner)
if inner.code == ErrorCode::Cancelled
) =>
{
return Err(map_stop_error(err));
}
Err(err) => {
debug!(session = %id, ?err, "session.stop_all: failure (continuing)");
let rpc = map_stop_error(err);
@@ -1329,6 +1430,30 @@ mod tests {
assert_eq!(got, std::time::Duration::from_millis(300_000));
}
#[test]
fn extension_transport_outlives_the_operation_deadline() {
let params = serde_json::json!({
"session_id": "abcd",
"timeout_ms": 60_000,
});
let dispatch_timeout =
tool_dispatch_transport_timeout(&Method::ToolUpload, &params).unwrap();
assert_eq!(dispatch_timeout, Duration::from_secs(62));
}
#[test]
fn extension_response_grace_does_not_change_other_tools() {
let params = serde_json::json!({
"session_id": "abcd",
"timeout_ms": 60_000,
});
let dispatch_timeout =
tool_dispatch_transport_timeout(&Method::ToolClick, &params).unwrap();
assert_eq!(dispatch_timeout, Duration::from_secs(60));
}
#[test]
fn tool_dispatch_timeout_rejects_zero_timeout_ms() {
let params = serde_json::json!({
+1
View File
@@ -3,6 +3,7 @@
pub mod abort;
pub mod browsers;
mod cancel_forward;
pub mod file_transfer;
pub mod inflight;
pub mod info;
pub mod ipc;
+9 -21
View File
@@ -100,14 +100,6 @@ pub fn record_session_path() -> Result<PathBuf> {
Ok(bsk_home()?.join("record-session.json"))
}
/// Path to a completed Trace saved before bundle export (`record-recovery.json`).
///
/// Kept until export succeeds so a failed `--output` write cannot drop the
/// recording the extension already returned.
pub fn record_recovery_path() -> Result<PathBuf> {
Ok(bsk_home()?.join("record-recovery.json"))
}
/// Windows named-pipe name. Include the resolved `BSK_HOME` path in the
/// token so test homes and custom installs do not share a predictable
/// per-username pipe.
@@ -127,13 +119,6 @@ pub fn pipe_name() -> String {
render_pipe_name(&user, &home)
}
/// Process-wide lock for tests that mutate `BSK_HOME`.
#[cfg(test)]
pub(crate) fn test_env_lock() -> std::sync::MutexGuard<'static, ()> {
static GUARD: std::sync::Mutex<()> = std::sync::Mutex::new(());
GUARD.lock().unwrap_or_else(|e| e.into_inner())
}
/// Render the Windows named-pipe name for a user/home pair. Kept
/// platform-agnostic so unit tests on any host can pin the output
/// character set.
@@ -151,12 +136,19 @@ fn render_pipe_name(user: &str, home: &str) -> String {
#[cfg(test)]
mod tests {
use super::*;
use std::sync::Mutex;
use tempfile::TempDir;
/// Serialise tests that mutate the global `BSK_HOME` env var.
fn env_guard() -> &'static Mutex<()> {
static GUARD: Mutex<()> = Mutex::new(());
&GUARD
}
fn with_temp_home<F: FnOnce(&Path)>(f: F) {
let _lock = test_env_lock();
let _lock = env_guard().lock().unwrap_or_else(|e| e.into_inner());
let tmp = TempDir::new().unwrap();
// SAFETY: serialised by test_env_lock above.
// SAFETY: serialised by env_guard above.
unsafe {
std::env::set_var(BSK_HOME_ENV, tmp.path().join("bsk"));
}
@@ -195,10 +187,6 @@ mod tests {
record_session_path().unwrap(),
home.join("record-session.json")
);
assert_eq!(
record_recovery_path().unwrap(),
home.join("record-recovery.json")
);
});
}
+210 -143
View File
@@ -43,7 +43,6 @@ use serde_json::Value;
use tokio::sync::{mpsc, oneshot};
use tracing::{debug, warn};
use super::abort::AbortToken;
use super::browsers::BrowserRegistry;
use super::inflight::{PromoteOutcome, ToolInflightEntry};
use super::sessions::{SessionId, SessionRegistry};
@@ -84,14 +83,6 @@ pub struct ToolJob {
/// IPC request id (e.g. `session.stop`'s queued teardown call —
/// those already carry their own bespoke retry / abort path).
pub inflight: Option<Arc<ToolInflightEntry>>,
/// Cancellation token for session-lifecycle work registered in the
/// daemon-local abort registry. Unlike `inflight`, lifecycle callers
/// forward their own WS cancel after the request has been queued.
pub lifecycle_cancel: Option<AbortToken>,
/// How long cancellation keeps the worker busy while the extension
/// finishes compensation. Session teardown uses its full RPC budget
/// so daemon state is reconciled even after the CLI exits.
pub cancel_cleanup_timeout: Duration,
/// Owning session — used by `ToolInflightRegistry::cancel_session`
/// to drain queued jobs that have not yet been promoted to
/// "forwarded". Worker callers also have access to it via the
@@ -331,8 +322,6 @@ impl ToolQueueRegistry {
timeout,
false,
inflight,
None,
CANCEL_CLEANUP_TIMEOUT,
)
.await;
// A long-running request may outlive the idle threshold. Touching
@@ -371,8 +360,6 @@ impl ToolQueueRegistry {
timeout,
respond: respond_tx,
inflight,
lifecycle_cancel: None,
cancel_cleanup_timeout: CANCEL_CLEANUP_TIMEOUT,
session_id: sid.clone(),
};
match forward_one(sid, &self.browsers, &self.sessions, &job).await {
@@ -391,7 +378,6 @@ impl ToolQueueRegistry {
method: Method,
params: Value,
timeout: Duration,
cancel: Option<AbortToken>,
) -> Result<Value, DispatchError> {
let (sender, state) = {
let mut guard = self.queues.lock().expect("tool queue registry poisoned");
@@ -418,8 +404,6 @@ impl ToolQueueRegistry {
timeout,
true,
None,
cancel,
timeout,
)
.await
}
@@ -435,9 +419,12 @@ async fn dispatch_with_sender(
timeout: Duration,
wait_for_capacity: bool,
inflight: Option<Arc<ToolInflightEntry>>,
lifecycle_cancel: Option<AbortToken>,
cancel_cleanup_timeout: Duration,
) -> Result<Value, DispatchError> {
let response_grace = if is_effect_aware_transfer(&method) {
CANCEL_CLEANUP_TIMEOUT
} else {
Duration::ZERO
};
let (respond_tx, respond_rx) = oneshot::channel();
let job = ToolJob {
method,
@@ -445,11 +432,8 @@ async fn dispatch_with_sender(
timeout,
respond: respond_tx,
inflight,
lifecycle_cancel,
cancel_cleanup_timeout,
session_id,
};
let lifecycle_cancellable = job.lifecycle_cancel.is_some();
if wait_for_capacity {
let waited = tokio::time::timeout(
timeout.saturating_add(Duration::from_secs(1)),
@@ -474,15 +458,13 @@ async fn dispatch_with_sender(
mpsc::error::TrySendError::Closed(_) => DispatchError::QueueClosed,
});
}
let mut response_timeout = timeout.saturating_add(Duration::from_secs(1));
if lifecycle_cancellable {
// A lifecycle cancel can arrive at the original RPC deadline and
// intentionally keeps the worker alive while extension cleanup
// settles. Do not let this outer waiter abandon that reconciliation
// early, which could reopen a daemon session whose window is gone.
response_timeout = response_timeout.saturating_add(cancel_cleanup_timeout);
}
let waited = tokio::time::timeout(response_timeout, respond_rx).await;
let waited = tokio::time::timeout(
timeout
.saturating_add(response_grace)
.saturating_add(Duration::from_secs(1)),
respond_rx,
)
.await;
match waited {
Ok(Ok(Ok(v))) => Ok(v),
Ok(Ok(Err(rpc))) => Err(DispatchError::Rpc(rpc)),
@@ -520,17 +502,11 @@ async fn forward_one(
// / browser resolution work, and before any WS frame leaves the
// daemon (review C2). We keep the `tokio::select!` later so
// cancels arriving mid-WS-hop also unblock the worker.
if job
.inflight
.as_ref()
.is_some_and(|entry| entry.is_cancelled())
|| job
.lifecycle_cancel
.as_ref()
.is_some_and(AbortToken::is_cancelled)
if let Some(entry) = job.inflight.as_ref()
&& entry.is_cancelled()
{
return Err(cancelled_error(
job.inflight.as_deref(),
Some(entry),
"tool dispatch cancelled before forwarding",
));
}
@@ -594,20 +570,11 @@ async fn forward_one(
}
}
None => {
// Daemon-internal callers do not carry a tool-inflight entry.
// Session lifecycle jobs instead use the abort-registry token
// below; other internal callers keep the plain send path.
if job
.lifecycle_cancel
.as_ref()
.is_some_and(AbortToken::is_cancelled)
{
client.pending.lock().unwrap().cancel(&rpc_id);
return Err(cancelled_error(
None,
"session lifecycle cancelled before forwarding",
));
}
// Daemon-internal callers (e.g. queued session.stop drain)
// don't carry an inflight entry; they cannot be cancelled
// through the IPC `cancel` surface, so a plain sink send
// is sufficient and keeps the existing transport-error
// behaviour.
if client.sink.send(request).is_err() {
client.pending.lock().unwrap().cancel(&rpc_id);
return Err(RpcError {
@@ -616,92 +583,120 @@ async fn forward_one(
data: None,
});
}
job.lifecycle_cancel.clone()
None
}
};
let forward_lifecycle_cancel = || {
if job.lifecycle_cancel.is_none() {
return;
}
let cancel = Frame::Request(RequestFrame {
id: format!("cancel-{rpc_id}"),
method: Method::Cancel,
params: Some(serde_json::json!({ "rpc_id": rpc_id })),
});
if let Err(err) = client.sink.send(cancel) {
warn!(%rpc_id, ?err, "failed to forward session lifecycle cancel");
}
let send_deadline_cancel = || {
client
.sink
.send(Frame::Request(RequestFrame {
id: format!("deadline-cancel-{rpc_id}"),
method: Method::Cancel,
params: Some(serde_json::json!({ "rpc_id": rpc_id })),
}))
.is_ok()
};
let on_abort: Option<&(dyn Fn() + Sync)> = job
.lifecycle_cancel
.as_ref()
.map(|_| &forward_lifecycle_cancel as &(dyn Fn() + Sync));
let deadline_cancel: Option<&(dyn Fn() -> bool + Sync)> =
is_effect_aware_transfer(&job.method).then_some(&send_deadline_cancel);
let waited = await_with_optional_cancel(
job.timeout,
job.cancel_cleanup_timeout,
CANCEL_CLEANUP_TIMEOUT,
waiter,
cancel_token.as_ref(),
on_abort,
deadline_cancel,
)
.await;
let response = match waited {
WaitOutcome::Response(resp) => resp,
WaitOutcome::CancelledAfterResponse(resp) => {
if job.method == Method::ToolSessionStop
&& let ResponseBody::Ok(value) = resp.body
{
// Teardown crossed its final irreversible boundary before
// the cancel landed. Commit the real extension result so the
// daemon does not retain a session whose window is gone.
return Ok(value);
// File transfer commits are irreversible. A late cancel cannot
// overwrite a confirmed success, and an unknown transfer effect
// must remain explicit so callers do not retry or release upload
// staging as though nothing happened.
if is_effect_aware_transfer(&job.method) {
match &resp.body {
ResponseBody::Ok(_) => resp,
ResponseBody::Err(err)
if transfer_effect(err) == Some("unknown")
|| transfer_effect(err) == Some("committed") =>
{
return Err(err.clone());
}
ResponseBody::Err(err)
if !matches!(err.code, ErrorCode::Cancelled | ErrorCode::UserAborted) =>
{
return Err(err.clone());
}
ResponseBody::Err(_) => {
return Err(cancelled_error(
job.inflight.as_deref(),
"tool dispatch cancelled after extension cleanup",
));
}
}
} else {
// For ordinary tools cancellation keeps the existing verdict,
// while non-cancel errors still expose compensation failures.
if let ResponseBody::Err(err) = resp.body
&& !matches!(err.code, ErrorCode::Cancelled | ErrorCode::UserAborted)
{
return Err(err);
}
return Err(cancelled_error(
job.inflight.as_deref(),
"tool dispatch cancelled after extension cleanup",
));
}
// Cancellation wins the external verdict, but only after the
// extension's original RPC has settled. Preserve any non-cancel
// error so compensation failures remain explicit instead of
// being hidden behind a generic cancelled result.
if let ResponseBody::Err(err) = resp.body
&& !matches!(err.code, ErrorCode::Cancelled | ErrorCode::UserAborted)
{
return Err(err);
}
WaitOutcome::CleanupTimeout => {
client.pending.lock().unwrap().cancel(&rpc_id);
if is_effect_aware_transfer(&job.method) {
return Err(unknown_transfer_error(
ErrorCode::Timeout,
"cancelled file transfer did not confirm its outcome before cleanup timed out",
"cleanup",
true,
));
}
return Err(cancelled_error(
job.inflight.as_deref(),
"tool dispatch cancelled after extension cleanup",
));
return Err(RpcError {
code: ErrorCode::Timeout,
message: format!(
"cancelled tool did not finish cleanup within {:?}",
CANCEL_CLEANUP_TIMEOUT
),
data: Some(serde_json::json!({ "reason": "cancel_cleanup_timeout" })),
});
}
WaitOutcome::TimedOutAfterResponse(resp) => match resp.body {
ResponseBody::Ok(value) if job.method == Method::ToolSessionStop => {
// The close crossed its irreversible boundary during the
// timeout cleanup grace. Reconcile daemon state instead of
// reopening a session whose window is already gone.
return Ok(value);
}
ResponseBody::Ok(value) => return Ok(value),
ResponseBody::Err(err)
if !matches!(err.code, ErrorCode::Cancelled | ErrorCode::UserAborted) =>
{
return Err(err);
}
_ => {
return Err(RpcError {
code: ErrorCode::Timeout,
message: format!("tool RPC timed out after {:?}", job.timeout),
data: None,
});
ResponseBody::Err(err) => {
return Err(timed_out_transfer_error(&err));
}
},
WaitOutcome::CleanupTimeout => {
WaitOutcome::TimeoutCleanupFailed => {
client.pending.lock().unwrap().cancel(&rpc_id);
return Err(RpcError {
code: ErrorCode::Timeout,
message: format!(
"cancelled tool did not finish cleanup within {:?}",
job.cancel_cleanup_timeout
),
data: Some(serde_json::json!({ "reason": "cancel_cleanup_timeout" })),
});
return Err(unknown_transfer_error(
ErrorCode::Timeout,
"file transfer timed out and cleanup could not be confirmed",
"cleanup",
true,
));
}
WaitOutcome::WaiterClosed => {
client.pending.lock().unwrap().cancel(&rpc_id);
if is_effect_aware_transfer(&job.method) {
return Err(unknown_transfer_error(
ErrorCode::ProtocolError,
"file transfer transport closed after dispatch; outcome is unknown",
"transport",
false,
));
}
return Err(RpcError {
code: ErrorCode::ProtocolError,
message: "transport closed mid-call".into(),
@@ -710,6 +705,14 @@ async fn forward_one(
}
WaitOutcome::Timeout => {
client.pending.lock().unwrap().cancel(&rpc_id);
if is_effect_aware_transfer(&job.method) {
return Err(unknown_transfer_error(
ErrorCode::Timeout,
"file transfer timed out after dispatch; outcome is unknown",
"transport",
false,
));
}
return Err(RpcError {
code: ErrorCode::Timeout,
message: format!("tool RPC timed out after {:?}", job.timeout),
@@ -723,12 +726,65 @@ async fn forward_one(
}
}
fn is_effect_aware_transfer(method: &Method) -> bool {
matches!(method, Method::ToolUpload | Method::ToolDownload)
}
fn transfer_effect(err: &RpcError) -> Option<&str> {
err.data.as_ref()?.get("effect_state")?.as_str()
}
fn unknown_transfer_error(
code: ErrorCode,
message: impl Into<String>,
phase: &str,
cleanup_failed: bool,
) -> RpcError {
let mut data = serde_json::json!({
"reason": "transfer_outcome_unknown",
"effect_state": "unknown",
"phase": phase,
});
if cleanup_failed {
data["cleanup_state"] = serde_json::json!("failed");
}
RpcError {
code,
message: message.into(),
data: Some(data),
}
}
fn timed_out_transfer_error(err: &RpcError) -> RpcError {
let data = err.data.as_ref();
RpcError {
code: ErrorCode::Timeout,
message: "file transfer timed out after dispatch".into(),
data: Some(serde_json::json!({
"reason": "transfer_timeout",
"effect_state": data
.and_then(|value| value.get("effect_state"))
.and_then(Value::as_str)
.unwrap_or("unknown"),
"phase": data
.and_then(|value| value.get("phase"))
.and_then(Value::as_str)
.unwrap_or("cleanup"),
"cleanup_state": data
.and_then(|value| value.get("cleanup_state"))
.and_then(Value::as_str)
.unwrap_or("complete"),
})),
}
}
#[derive(Debug)]
enum WaitOutcome {
Response(bsk_protocol::ResponseFrame),
CancelledAfterResponse(bsk_protocol::ResponseFrame),
TimedOutAfterResponse(bsk_protocol::ResponseFrame),
CleanupTimeout,
TimedOutAfterResponse(bsk_protocol::ResponseFrame),
TimeoutCleanupFailed,
WaiterClosed,
Timeout,
}
@@ -738,7 +794,7 @@ async fn await_with_optional_cancel(
cleanup_timeout: Duration,
mut waiter: oneshot::Receiver<bsk_protocol::ResponseFrame>,
cancel: Option<&super::abort::AbortToken>,
on_abort: Option<&(dyn Fn() + Sync)>,
on_deadline: Option<&(dyn Fn() -> bool + Sync)>,
) -> WaitOutcome {
match cancel {
Some(token) => {
@@ -750,9 +806,6 @@ async fn await_with_optional_cancel(
// after compensation or the cleanup deadline expires.
biased;
_ = token.cancelled() => {
if let Some(on_abort) = on_abort {
on_abort();
}
match tokio::time::timeout(cleanup_timeout, &mut waiter).await {
Ok(Ok(resp)) => WaitOutcome::CancelledAfterResponse(resp),
Ok(Err(_)) => WaitOutcome::WaiterClosed,
@@ -763,23 +816,31 @@ async fn await_with_optional_cancel(
Ok(resp) => WaitOutcome::Response(resp),
Err(_) => WaitOutcome::WaiterClosed,
},
_ = &mut deadline => {
let Some(on_abort) = on_abort else {
return WaitOutcome::Timeout;
};
on_abort();
match tokio::time::timeout(cleanup_timeout, &mut waiter).await {
Ok(Ok(resp)) => WaitOutcome::TimedOutAfterResponse(resp),
Ok(Err(_)) => WaitOutcome::WaiterClosed,
Err(_) => WaitOutcome::CleanupTimeout,
_ = &mut deadline => match on_deadline {
Some(send_cancel) if send_cancel() => {
match tokio::time::timeout(cleanup_timeout, &mut waiter).await {
Ok(Ok(resp)) => WaitOutcome::TimedOutAfterResponse(resp),
Ok(Err(_)) | Err(_) => WaitOutcome::TimeoutCleanupFailed,
}
}
Some(_) => WaitOutcome::TimeoutCleanupFailed,
None => WaitOutcome::Timeout,
},
}
}
None => match tokio::time::timeout(timeout, &mut waiter).await {
Ok(Ok(resp)) => WaitOutcome::Response(resp),
Ok(Err(_)) => WaitOutcome::WaiterClosed,
Err(_) => WaitOutcome::Timeout,
Err(_) => match on_deadline {
Some(send_cancel) if send_cancel() => {
match tokio::time::timeout(cleanup_timeout, &mut waiter).await {
Ok(Ok(resp)) => WaitOutcome::TimedOutAfterResponse(resp),
Ok(Err(_)) | Err(_) => WaitOutcome::TimeoutCleanupFailed,
}
}
Some(_) => WaitOutcome::TimeoutCleanupFailed,
None => WaitOutcome::Timeout,
},
},
}
}
@@ -954,30 +1015,38 @@ mod await_with_optional_cancel_tests {
}
#[tokio::test]
async fn lifecycle_timeout_forwards_abort_and_waits_for_cleanup_response() {
use std::sync::atomic::{AtomicBool, Ordering};
let token = AbortToken::new();
async fn transfer_deadline_sends_cancel_and_waits_for_cleanup_response() {
let (tx, rx) = oneshot::channel();
let abort_forwarded = AtomicBool::new(false);
let forward_abort = || abort_forwarded.store(true, Ordering::SeqCst);
tokio::spawn(async move {
tokio::time::sleep(Duration::from_millis(20)).await;
tokio::time::sleep(Duration::from_millis(5)).await;
tx.send(dummy_response()).unwrap();
});
let send_cancel = || true;
let outcome = await_with_optional_cancel(
Duration::from_millis(5),
Duration::from_millis(100),
Duration::from_millis(1),
Duration::from_secs(1),
rx,
Some(&token),
Some(&forward_abort),
None,
Some(&send_cancel),
)
.await;
assert!(abort_forwarded.load(Ordering::SeqCst));
assert!(matches!(outcome, WaitOutcome::TimedOutAfterResponse(_)));
}
#[tokio::test]
async fn transfer_deadline_is_unknown_when_cancel_cannot_be_sent() {
let (_tx, rx) = oneshot::channel();
let send_cancel = || false;
let outcome = await_with_optional_cancel(
Duration::from_millis(1),
Duration::from_secs(1),
rx,
None,
Some(&send_cancel),
)
.await;
assert!(matches!(outcome, WaitOutcome::TimeoutCleanupFailed));
}
}
#[cfg(test)]
@@ -1040,8 +1109,6 @@ mod tool_job_session_id_tests {
timeout: Duration::from_secs(1),
respond: tx,
inflight: None,
lifecycle_cancel: None,
cancel_cleanup_timeout: CANCEL_CLEANUP_TIMEOUT,
session_id: SessionId("sess-A".into()),
};
assert_eq!(job.session_id.0, "sess-A");
+26 -223
View File
@@ -12,14 +12,13 @@ use bsk_protocol::system::{BrowserStatusEntry, SessionStatusEntry};
use bsk_protocol::tools::{
SessionStartParams, SessionStartResult, SessionStopParams, SessionStopResult,
};
use bsk_protocol::{ErrorCode, Frame, Method, RequestFrame, ResponseBody, RpcError, RpcId};
use bsk_protocol::{Frame, RequestFrame, ResponseBody, RpcError, RpcId};
use rand::Rng;
use serde::{Deserialize, Serialize};
use tokio::time::{Duration, timeout};
use super::abort::AbortToken;
use super::browsers::{BrowserClient, BrowserId, BrowserRegistry, SelectError};
use super::queue::{CANCEL_CLEANUP_TIMEOUT, DispatchError, ToolQueueRegistry};
use super::queue::{DispatchError, ToolQueueRegistry};
use super::session_interrupt::SessionInterruptRegistry;
#[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize)]
@@ -340,14 +339,6 @@ pub enum StartSessionError {
IdExhausted,
#[error("session creation timed out waiting for extension")]
Timeout,
#[error("session creation was cancelled")]
Cancelled,
#[error("session creation cancellation cleanup failed for {session_id}: {message}")]
CleanupFailed {
session_id: SessionId,
agent_window_id: Option<i64>,
message: String,
},
#[error("extension rejected tool.session_start: {0:?}")]
ExtensionError(RpcError),
#[error("transport closed while waiting for extension response")]
@@ -363,8 +354,6 @@ impl StartSessionError {
StartSessionError::AmbiguousBrowserLabel { .. } => "invalid_params",
StartSessionError::IdExhausted => "protocol_error",
StartSessionError::Timeout => "timeout",
StartSessionError::Cancelled => "cancelled",
StartSessionError::CleanupFailed { .. } => "protocol_error",
StartSessionError::TransportClosed => "protocol_error",
StartSessionError::ExtensionError(err) => match err.code {
bsk_protocol::ErrorCode::Timeout => "timeout",
@@ -418,7 +407,6 @@ pub struct AgentWindowOptions {
/// On success the matching per-session dispatch queue (M6.5,
/// design §5) is also spawned so that subsequent `tool.*` RPCs serialise
/// against this session's Agent Window / ref-store.
#[allow(clippy::too_many_arguments)]
pub async fn start_session(
registry: &Arc<BrowserRegistry>,
sessions: &Arc<SessionRegistry>,
@@ -427,32 +415,24 @@ pub async fn start_session(
window: AgentWindowOptions,
connect_wait: Duration,
timeout_dur: Duration,
cancel: Option<AbortToken>,
) -> Result<Session, StartSessionError> {
let selection = registry.select_with_connect_wait(requested, connect_wait);
tokio::pin!(selection);
let selected = match cancel.as_ref() {
Some(token) => tokio::select! {
biased;
_ = token.cancelled() => return Err(StartSessionError::Cancelled),
selected = &mut selection => selected,
},
None => selection.await,
};
let client: Arc<BrowserClient> = selected.map_err(|e| match e {
SelectError::NoBrowserConnected => StartSessionError::NoBrowserConnected,
SelectError::MultipleBrowsersOnline => StartSessionError::MultipleBrowsersOnline {
browsers: snapshot_status_entries(registry, sessions),
},
SelectError::NotFound => StartSessionError::BrowserNotFound,
SelectError::AmbiguousLabel {
label,
instance_ids,
} => StartSessionError::AmbiguousBrowserLabel {
label,
instance_ids,
},
})?;
let client: Arc<BrowserClient> = registry
.select_with_connect_wait(requested, connect_wait)
.await
.map_err(|e| match e {
SelectError::NoBrowserConnected => StartSessionError::NoBrowserConnected,
SelectError::MultipleBrowsersOnline => StartSessionError::MultipleBrowsersOnline {
browsers: snapshot_status_entries(registry, sessions),
},
SelectError::NotFound => StartSessionError::BrowserNotFound,
SelectError::AmbiguousLabel {
label,
instance_ids,
} => StartSessionError::AmbiguousBrowserLabel {
label,
instance_ids,
},
})?;
let session_id = sessions
.reserve_id(client.id.clone(), SESSION_ID_MAX_RESERVE_ATTEMPTS, now_ms)
.ok_or(StartSessionError::IdExhausted)?;
@@ -473,71 +453,22 @@ pub async fn start_session(
let mut pending = client.pending.lock().unwrap();
pending.register(rpc_id.clone())
};
if cancel.as_ref().is_some_and(AbortToken::is_cancelled) {
sessions.cancel_reservation(&session_id);
client.pending.lock().unwrap().cancel(&rpc_id);
return Err(StartSessionError::Cancelled);
}
if client.sink.send(Frame::Request(request)).is_err() {
sessions.cancel_reservation(&session_id);
client.pending.lock().unwrap().cancel(&rpc_id);
return Err(StartSessionError::TransportClosed);
}
let mut waiter = waiter;
enum StartWaitOutcome {
Response(bsk_protocol::ResponseFrame),
WaiterClosed,
Cancelled,
Timeout,
}
let wait_outcome = match cancel.as_ref() {
Some(token) => {
let deadline = tokio::time::sleep(timeout_dur);
tokio::pin!(deadline);
tokio::select! {
biased;
_ = token.cancelled() => StartWaitOutcome::Cancelled,
response = &mut waiter => match response {
Ok(response) => StartWaitOutcome::Response(response),
Err(_) => StartWaitOutcome::WaiterClosed,
},
_ = &mut deadline => StartWaitOutcome::Timeout,
}
}
None => match timeout(timeout_dur, &mut waiter).await {
Ok(Ok(response)) => StartWaitOutcome::Response(response),
Ok(Err(_)) => StartWaitOutcome::WaiterClosed,
Err(_) => StartWaitOutcome::Timeout,
},
};
let response = match wait_outcome {
StartWaitOutcome::Response(response) => response,
StartWaitOutcome::WaiterClosed => {
let response = match timeout(timeout_dur, waiter).await {
Ok(Ok(resp)) => resp,
Ok(Err(_)) => {
sessions.cancel_reservation(&session_id);
client.pending.lock().unwrap().cancel(&rpc_id);
return Err(StartSessionError::TransportClosed);
}
StartWaitOutcome::Cancelled => {
return Err(finish_aborted_start(
&client,
sessions,
&session_id,
&rpc_id,
waiter,
StartAbortReason::Cancelled,
)
.await);
}
StartWaitOutcome::Timeout => {
return Err(finish_aborted_start(
&client,
sessions,
&session_id,
&rpc_id,
waiter,
StartAbortReason::Timeout,
)
.await);
Err(_) => {
sessions.cancel_reservation(&session_id);
client.pending.lock().unwrap().cancel(&rpc_id);
return Err(StartSessionError::Timeout);
}
};
let agent_window_id = match response.body {
@@ -573,124 +504,6 @@ pub async fn start_session(
Ok(session)
}
#[derive(Clone, Copy)]
enum StartAbortReason {
Cancelled,
Timeout,
}
impl StartAbortReason {
fn error(self) -> StartSessionError {
match self {
Self::Cancelled => StartSessionError::Cancelled,
Self::Timeout => StartSessionError::Timeout,
}
}
}
async fn finish_aborted_start(
client: &Arc<BrowserClient>,
sessions: &Arc<SessionRegistry>,
session_id: &SessionId,
rpc_id: &RpcId,
mut waiter: tokio::sync::oneshot::Receiver<bsk_protocol::ResponseFrame>,
reason: StartAbortReason,
) -> StartSessionError {
sessions.cancel_reservation(session_id);
let cancel = Frame::Request(RequestFrame {
id: format!("cancel-{rpc_id}"),
method: Method::Cancel,
params: Some(serde_json::json!({ "rpc_id": rpc_id })),
});
if client.sink.send(cancel).is_err() {
client.pending.lock().unwrap().cancel(rpc_id);
return StartSessionError::TransportClosed;
}
match timeout(CANCEL_CLEANUP_TIMEOUT, &mut waiter).await {
Ok(Ok(response)) => match response.body {
ResponseBody::Err(err)
if matches!(err.code, ErrorCode::Cancelled | ErrorCode::UserAborted) =>
{
reason.error()
}
ResponseBody::Err(err) => StartSessionError::ExtensionError(err),
ResponseBody::Ok(value) => {
let agent_window_id = serde_json::from_value::<SessionStartResult>(value)
.ok()
.and_then(|result| result.agent_window_id);
match rollback_extension_session(client, session_id).await {
Ok(()) => reason.error(),
Err(message) => StartSessionError::CleanupFailed {
session_id: session_id.clone(),
agent_window_id,
message,
},
}
}
},
Ok(Err(_)) => {
client.pending.lock().unwrap().cancel(rpc_id);
StartSessionError::TransportClosed
}
Err(_) => {
// The cancel frame is already queued behind session_start. Even
// if chrome.windows.create is still blocked, the extension will
// observe the aborted signal and compensate before replying.
client.pending.lock().unwrap().cancel(rpc_id);
reason.error()
}
}
}
async fn rollback_extension_session(
client: &Arc<BrowserClient>,
session_id: &SessionId,
) -> Result<(), String> {
let rollback_id = next_rpc_id("sess-start-rollback");
let waiter = {
let mut pending = client.pending.lock().unwrap();
pending.register(rollback_id.clone())
};
let request = RequestFrame {
id: rollback_id.clone(),
method: Method::ToolSessionStop,
params: Some(
serde_json::to_value(SessionStopParams {
session_id: session_id.0.clone(),
})
.expect("serialize session_start rollback params"),
),
};
if client.sink.send(Frame::Request(request)).is_err() {
client.pending.lock().unwrap().cancel(&rollback_id);
return Err("browser disconnected before rollback was queued".into());
}
let response = match timeout(CANCEL_CLEANUP_TIMEOUT, waiter).await {
Ok(Ok(response)) => response,
Ok(Err(_)) => return Err("browser disconnected during rollback".into()),
Err(_) => {
client.pending.lock().unwrap().cancel(&rollback_id);
return Err(format!(
"extension did not finish rollback within {CANCEL_CLEANUP_TIMEOUT:?}"
));
}
};
match response.body {
ResponseBody::Ok(value) => {
let result: SessionStopResult = serde_json::from_value(value)
.map_err(|err| format!("invalid rollback response: {err}"))?;
if result.return_failures.is_empty() {
Ok(())
} else {
Err("rollback reported borrowed-tab return failures".into())
}
}
ResponseBody::Err(err) if err.code == ErrorCode::NotFound => Ok(()),
ResponseBody::Err(err) => Err(format!("rollback failed: {} ({:?})", err.message, err.code)),
}
}
/// Tear down a single session id (round-trips `tool.session_stop` to the
/// owning extension and unregisters on success). Also tears down the
/// per-session dispatch queue so the worker task exits.
@@ -701,7 +514,6 @@ pub async fn stop_session(
interrupts: &Arc<SessionInterruptRegistry>,
session_id: &SessionId,
timeout_dur: Duration,
cancel: Option<AbortToken>,
) -> Result<SessionStopResult, StopSessionError> {
let session = sessions.get(session_id).ok_or(StopSessionError::NotFound)?;
if registry.get(&session.browser_id).is_none() {
@@ -712,17 +524,12 @@ pub async fn stop_session(
session_id: session_id.0.clone(),
})
.unwrap();
// Internal callers (idle reaping and shutdown) do not have a CLI-owned
// token, but still need timeout-driven WS cancellation so a late window
// close cannot leave daemon state pointing at a vanished session.
let lifecycle_cancel = Some(cancel.unwrap_or_default());
match queues
.dispatch_after_closing(
session_id,
bsk_protocol::Method::ToolSessionStop,
params,
timeout_dur,
lifecycle_cancel,
)
.await
{
@@ -760,10 +567,6 @@ pub async fn stop_session(
drop_session_local(queues, interrupts, session_id);
return Ok(SessionStopResult::default());
}
if err.code == bsk_protocol::ErrorCode::Timeout {
queues.reopen(session_id);
return Err(StopSessionError::Timeout);
}
queues.reopen(session_id);
Err(StopSessionError::ExtensionError(err))
}
+39 -99
View File
@@ -15,6 +15,7 @@ use std::net::{IpAddr, Ipv4Addr, SocketAddr};
use std::path::Path;
use std::sync::Arc;
use std::sync::Mutex;
use std::sync::atomic::{AtomicUsize, Ordering};
use std::time::{Duration, Instant};
use anyhow::{Context, Result};
@@ -26,7 +27,7 @@ use crate::daemon::{
browsers::{BROWSER_LIVENESS_TICK, BROWSER_LIVENESS_TIMEOUT, EXTENSION_CONNECT_WAIT},
info as daemon_info, ipc, lockfile, paths,
sessions::{StopSessionError, forget_session, stop_session},
state::{DaemonState, PROTOCOL_VERSION},
state::DaemonState,
ws,
};
@@ -225,6 +226,10 @@ pub fn run_foreground(cfg: DaemonConfig) -> Result<()> {
.with_context(|| format!("bind IPC endpoint {}", sock_path.display()))?;
let state = Arc::new(DaemonState::new(cfg.clone()));
state
.transfers
.initialize()
.context("initialize transfer staging")?;
let session_idle_task = spawn_session_idle_reaper(Arc::clone(&state));
let browser_liveness_task = spawn_browser_liveness_reaper(Arc::clone(&state));
// Fired by the update check task after a successful auto-update:
@@ -290,14 +295,15 @@ pub fn run_foreground(cfg: DaemonConfig) -> Result<()> {
});
let started_at = Instant::now();
let activity = Arc::new(Mutex::new(IpcActivityState::new(started_at)));
let activity: Arc<Mutex<Instant>> = Arc::new(Mutex::new(started_at));
let active_ipc_connections = Arc::new(AtomicUsize::new(0));
let (ipc_shutdown_tx, ipc_shutdown_rx) = tokio::sync::oneshot::channel::<()>();
let status = ipc::DaemonStatus {
started_at,
ws_port,
sock_path: sock_path.clone(),
daemon_version: env!("CARGO_PKG_VERSION"),
protocol_version: PROTOCOL_VERSION,
protocol_version: "1.0",
};
let handler = ipc::full_handler(status, Arc::clone(&state));
@@ -312,14 +318,18 @@ pub fn run_foreground(cfg: DaemonConfig) -> Result<()> {
let handler = handler.clone();
let ipc_open = {
let activity = activity.clone();
let active = active_ipc_connections.clone();
move || {
record_ipc_open(&activity);
active.fetch_add(1, Ordering::SeqCst);
record_activity(&activity);
}
};
let ipc_close = {
let activity = activity.clone();
let active = active_ipc_connections.clone();
move || {
record_ipc_close(&activity);
active.fetch_sub(1, Ordering::SeqCst);
record_activity(&activity);
}
};
tokio::spawn(ipc::serve(
@@ -338,6 +348,7 @@ pub fn run_foreground(cfg: DaemonConfig) -> Result<()> {
let idle_task = {
let activity = activity.clone();
let state = Arc::clone(&state);
let active_ipc_connections = active_ipc_connections.clone();
let daemon_idle = cfg.daemon_idle;
tokio::spawn(async move {
let tick = (daemon_idle / 4).max(Duration::from_millis(250));
@@ -345,19 +356,22 @@ pub fn run_foreground(cfg: DaemonConfig) -> Result<()> {
ticker.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Delay);
loop {
ticker.tick().await;
let ipc_is_idle =
lock_activity(&activity).should_exit_for_idle(Instant::now(), daemon_idle);
if !ipc_is_idle {
let last = match activity.lock() {
Ok(a) => *a,
Err(p) => *p.into_inner(),
};
if last.elapsed() < daemon_idle {
continue;
}
// IPC liveness (open-connection count + idle window) is already
// enforced above by `should_exit_for_idle`, which reads
// `IpcActivityState` under one lock and returns true only when
// there are zero connections AND the idle interval has elapsed.
// Here we additionally hold the daemon alive while any browser
// is paired or any session is live (design §3.2, M4/M5
// registries).
if !state.browsers.is_empty() || !state.sessions.is_empty() {
// Bridge from M2/M3 (which only knew about connection
// counts) to M4/M5 (browser + session registries):
// hold the daemon alive while any IPC client is
// connected, any browser is paired, or any session
// is live (design §3.2).
if active_ipc_connections.load(Ordering::SeqCst) > 0
|| !state.browsers.is_empty()
|| !state.sessions.is_empty()
{
continue;
}
info!(
@@ -448,6 +462,7 @@ pub(crate) fn spawn_browser_liveness_reaper(
for s in state.sessions.purge_browser(&client.id) {
state.tool_queues.remove(&s.id);
state.session_interrupts.drop_session(&s.id);
state.transfers.release_session(&s.id.0);
debug!(session = %s.id, "purged session on browser liveness timeout");
}
}
@@ -481,11 +496,13 @@ pub(crate) fn spawn_session_idle_reaper(state: Arc<DaemonState>) -> tokio::task:
&state.session_interrupts,
&session_id,
Duration::from_secs(10),
None,
)
.await
{
Ok(_) => info!(session = %session_id, "idle session stopped"),
Ok(_) => {
state.transfers.release_session(&session_id.0);
info!(session = %session_id, "idle session stopped");
}
Err(StopSessionError::SessionBusy | StopSessionError::Stopping) => {
debug!(session = %session_id, "idle session still active; retrying later");
}
@@ -496,6 +513,7 @@ pub(crate) fn spawn_session_idle_reaper(state: Arc<DaemonState>) -> tokio::task:
&state.session_interrupts,
&session_id,
);
state.transfers.release_session(&session_id.0);
}
Err(err) => {
warn!(session = %session_id, error = %err, "failed to stop idle session");
@@ -672,55 +690,10 @@ fn restart_start_args(cfg: &DaemonConfig) -> StartArgs {
}
}
#[derive(Debug)]
struct IpcActivityState {
last_activity: Instant,
active_connections: usize,
}
impl IpcActivityState {
fn new(now: Instant) -> Self {
Self {
last_activity: now,
active_connections: 0,
}
fn record_activity(activity: &Arc<Mutex<Instant>>) {
if let Ok(mut a) = activity.lock() {
*a = Instant::now();
}
fn touch(&mut self, now: Instant) {
self.last_activity = now;
}
fn opened(&mut self, now: Instant) {
self.active_connections = self.active_connections.saturating_add(1);
self.last_activity = now;
}
fn closed(&mut self, now: Instant) {
self.active_connections = self.active_connections.saturating_sub(1);
self.last_activity = now;
}
fn should_exit_for_idle(&self, now: Instant, idle: Duration) -> bool {
self.active_connections == 0 && now.saturating_duration_since(self.last_activity) >= idle
}
}
fn lock_activity(
activity: &Arc<Mutex<IpcActivityState>>,
) -> std::sync::MutexGuard<'_, IpcActivityState> {
activity.lock().unwrap_or_else(|poison| poison.into_inner())
}
fn record_activity(activity: &Arc<Mutex<IpcActivityState>>) {
lock_activity(activity).touch(Instant::now());
}
fn record_ipc_open(activity: &Arc<Mutex<IpcActivityState>>) {
lock_activity(activity).opened(Instant::now());
}
fn record_ipc_close(activity: &Arc<Mutex<IpcActivityState>>) {
lock_activity(activity).closed(Instant::now());
}
/// Initialise tracing for the daemon: write a daily-rotated file in
@@ -1103,37 +1076,4 @@ mod tests {
assert_eq!(args.session_idle, Some(Duration::from_secs(11)));
assert_eq!(args.daemon_idle, Some(Duration::from_secs(22)));
}
#[test]
fn ipc_close_restarts_the_idle_window_atomically() {
let started = Instant::now();
let idle = Duration::from_secs(10);
let mut activity = IpcActivityState::new(started);
activity.opened(started);
let closed_at = started + Duration::from_secs(30);
assert!(
!activity.should_exit_for_idle(closed_at, idle),
"an open IPC connection must hold the daemon alive"
);
activity.closed(closed_at);
assert_eq!(activity.active_connections, 0);
assert_eq!(activity.last_activity, closed_at);
assert!(!activity.should_exit_for_idle(closed_at, idle));
assert!(!activity.should_exit_for_idle(closed_at + idle - Duration::from_millis(1), idle));
assert!(activity.should_exit_for_idle(closed_at + idle, idle));
}
#[test]
fn ipc_activity_keeps_the_daemon_alive_until_a_full_idle_interval_passes() {
let started = Instant::now();
let idle = Duration::from_secs(10);
let mut activity = IpcActivityState::new(started);
let request_at = started + Duration::from_secs(20);
activity.touch(request_at);
assert!(!activity.should_exit_for_idle(request_at + Duration::from_secs(9), idle));
assert!(activity.should_exit_for_idle(request_at + idle, idle));
}
}
+9 -3
View File
@@ -7,6 +7,7 @@ use tokio::task::JoinHandle;
use super::abort::AbortRegistry;
use super::browsers::BrowserRegistry;
use super::file_transfer::TransferRegistry;
use super::inflight::ToolInflightRegistry;
use super::ipc::IpcHandle;
use super::queue::ToolQueueRegistry;
@@ -16,7 +17,7 @@ use super::start::DaemonConfig;
use super::ws::WsHandle;
pub const DAEMON_VERSION: &str = env!("CARGO_PKG_VERSION");
pub const PROTOCOL_VERSION: &str = "1.1";
pub const PROTOCOL_VERSION: &str = "1.0";
/// Lowest **protocol** version peers must speak (e.g. `"1.0"`).
pub const MIN_COMPATIBLE_PROTOCOL: &str = "1.0";
/// Legacy app-semver floor used only when `HandshakeResult.min_compatible_peer`
@@ -34,8 +35,8 @@ pub struct DaemonState {
/// `stop_session` / browser disconnect.
pub tool_queues: Arc<ToolQueueRegistry>,
/// Per-rpc-id cancellation tokens for daemon-side long-runners
/// (`tool.wait_ms` plus `session.*` lifecycle calls). The CLI's
/// `cancel { rpc_id }` consults this registry first.
/// (M9.3 — currently only `tool.wait_ms`). The CLI's `cancel
/// { rpc_id }` consults this registry first.
pub abort_registry: Arc<AbortRegistry>,
/// Tracks `tool.*` RPCs that have been forwarded to an extension
/// over WS but have not yet received a response. Indexed by the
@@ -51,6 +52,9 @@ pub struct DaemonState {
/// `SessionRegistry` because the signal is a transient runtime
/// control state.
pub session_interrupts: Arc<SessionInterruptRegistry>,
/// Operation-scoped local file staging. The extension only sees paths
/// minted here; agent-facing RPCs use opaque transfer ids.
pub transfers: Arc<TransferRegistry>,
}
impl DaemonState {
@@ -64,6 +68,7 @@ impl DaemonState {
));
let abort_registry = Arc::new(AbortRegistry::new());
let session_interrupts = Arc::new(SessionInterruptRegistry::new());
let transfers = Arc::new(TransferRegistry::new().expect("initialise transfer staging"));
Self {
config,
browsers,
@@ -72,6 +77,7 @@ impl DaemonState {
abort_registry,
tool_inflight,
session_interrupts,
transfers,
}
}
}
+3
View File
@@ -295,6 +295,7 @@ async fn drive_connection(
for session in state.sessions.purge_browser(&browser_id) {
state.tool_queues.remove(&session.id);
state.session_interrupts.drop_session(&session.id);
state.transfers.release_session(&session.id.0);
debug!(session = %session.id, "purged stale session before browser reconnect");
}
let (tx, mut rx) = mpsc::unbounded_channel::<Frame>();
@@ -407,6 +408,7 @@ async fn drive_connection(
for s in state.sessions.purge_browser(&browser_id) {
state.tool_queues.remove(&s.id);
state.session_interrupts.drop_session(&s.id);
state.transfers.release_session(&s.id.0);
debug!(session = %s.id, "purged session on browser disconnect");
}
} else {
@@ -503,6 +505,7 @@ fn handle_session_window_closed(
&state.session_interrupts,
&session_id,
) {
state.transfers.release_session(&session_id.0);
info!(session = %session_id, "session removed: user closed Agent Window");
} else {
debug!(session = %session_id, "session.window_closed for unknown session id");
+2
View File
@@ -97,6 +97,8 @@ fn dispatch(cli: Cli, format: Format) -> Result<(), CliError> {
Command::Fill(args) => cli::interaction::dispatch_fill(args, format),
Command::Press(args) => cli::interaction::dispatch_press(args, format),
Command::Select(args) => cli::interaction::dispatch_select(args, format),
Command::Upload(args) => cli::upload::dispatch(args, format),
Command::Download(args) => cli::download::dispatch(args, format),
Command::Evaluate(args) => cli::evaluate::dispatch(args, format),
Command::WaitForNavigation(args) => cli::waits::dispatch_wait_for_navigation(args, format),
Command::WaitMs(args) => cli::waits::dispatch_wait_ms(args, format),
+39
View File
@@ -213,6 +213,45 @@ fn rejects_zero_click_count() {
);
}
#[test]
fn parses_upload_with_repeated_files() {
let cli = parse(&[
"bsk",
"upload",
"@e3",
"--file",
"one.png",
"--file",
"two.png",
"--session",
"s1",
]);
let Command::Upload(args) = cli.command else {
panic!("expected upload command");
};
assert_eq!(args.target.as_deref(), Some("@e3"));
assert_eq!(args.files.len(), 2);
}
#[test]
fn parses_download_with_exact_output_policy() {
let cli = parse(&[
"bsk",
"download",
"#export",
"--out",
"result.zip",
"--session",
"s1",
"--overwrite",
]);
let Command::Download(args) = cli.command else {
panic!("expected download command");
};
assert_eq!(args.target.as_deref(), Some("#export"));
assert!(args.overwrite);
}
#[test]
fn parses_record_start_with_browser_and_url() {
let cli = parse(&[
+10 -10
View File
@@ -108,12 +108,12 @@ async fn send_handshake_with_floors(
async fn handshake_ok_when_protocol_matches() {
let (handle, _sock) = spawn_daemon().await;
let mut ws = open_ws(handle.ws_addr()).await;
let resp = send_handshake(&mut ws, "1.1", env!("CARGO_PKG_VERSION")).await;
let resp = send_handshake(&mut ws, "1.0", env!("CARGO_PKG_VERSION")).await;
let result: HandshakeResult = match resp.body {
ResponseBody::Ok(v) => serde_json::from_value(v).unwrap(),
ResponseBody::Err(e) => panic!("expected ok handshake, got {e:?}"),
};
assert_eq!(result.protocol_version, "1.1");
assert_eq!(result.protocol_version, "1.0");
assert_eq!(
result
.min_compatible_peer
@@ -135,7 +135,7 @@ async fn handshake_ok_when_app_versions_differ_but_protocol_matches() {
let (handle, _sock) = spawn_daemon().await;
let mut ws = open_ws(handle.ws_addr()).await;
let resp =
send_handshake_with_floors(&mut ws, "1.1", "9.9.9", Some("0.0.0"), Some("1.1")).await;
send_handshake_with_floors(&mut ws, "1.0", "9.9.9", Some("0.0.0"), Some("1.0")).await;
match resp.body {
ResponseBody::Ok(_) => {}
other => panic!("expected ok when protocol matches, got {other:?}"),
@@ -149,10 +149,10 @@ async fn handshake_skew_when_protocol_minor_differs() {
let mut ws = open_ws(handle.ws_addr()).await;
let resp = send_handshake_with_floors(
&mut ws,
"1.2",
"1.1",
env!("CARGO_PKG_VERSION"),
Some("0.0.0"),
Some("1.1"),
Some("1.0"),
)
.await;
match resp.body {
@@ -210,7 +210,7 @@ async fn handshake_legacy_ext_without_protocol_floor_still_ok() {
let mut ws = open_ws(handle.ws_addr()).await;
let resp = send_handshake_with_floors(
&mut ws,
"1.1",
"1.0",
env!("CARGO_PKG_VERSION"),
Some("0.1.0"),
None,
@@ -235,7 +235,7 @@ async fn status_surfaces_version_skew_for_skewed_browser() {
browser_name: "chrome".into(),
browser_version: "131.0".into(),
extension_version: "9.9.9".into(),
extension_protocol_version: "1.2".into(),
extension_protocol_version: "1.1".into(),
label: "Older".into(),
sink: bsk::daemon::browsers::BrowserSink { tx },
pending: Mutex::new(bsk::daemon::browsers::Pending::default()),
@@ -263,8 +263,8 @@ async fn status_surfaces_version_skew_for_skewed_browser() {
.iter()
.find(|s| s.instance_id == "skew-only-test")
.expect("status must list our skew client");
assert_eq!(skew.client_protocol_version, "1.2");
assert_eq!(skew.server_protocol_version, "1.1");
assert_eq!(skew.client_protocol_version, "1.1");
assert_eq!(skew.server_protocol_version, "1.0");
assert_eq!(skew.client_version, "9.9.9");
let entry = status
.browsers
@@ -281,7 +281,7 @@ async fn handshake_rejects_when_local_below_peer_min_compatible_protocol() {
let mut ws = open_ws(handle.ws_addr()).await;
let resp = send_handshake_with_floors(
&mut ws,
"1.1",
"1.0",
env!("CARGO_PKG_VERSION"),
Some("0.0.0"),
Some("99.0.0"),
@@ -325,7 +325,6 @@ async fn session_stop_fast_fails_while_tool_is_in_flight() {
&state.session_interrupts,
&sid,
Duration::from_secs(5),
None,
)
.await;
assert!(
@@ -367,7 +366,6 @@ async fn session_stop_fast_fails_while_tool_is_in_flight() {
&state.session_interrupts,
&sid,
Duration::from_secs(5),
None,
)
.await
.expect("session.stop ok after tool completes");

Some files were not shown because too many files have changed in this diff Show More