mirror of
https://github.com/Tencent/BrowserSkill.git
synced 2026-09-10 13:01:23 +08:00
Compare commits
7
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
ec62b8f0f2 | ||
|
|
9393f0e58b | ||
|
|
f9d395ee41 | ||
|
|
8f81c15656 | ||
|
|
3cfa9eceef | ||
|
|
34f47d7387 | ||
|
|
0a184e33d1 |
Generated
+1
@@ -137,6 +137,7 @@ name = "bsk"
|
||||
version = "0.1.10"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"base64",
|
||||
"bsk-protocol",
|
||||
"clap",
|
||||
"console",
|
||||
|
||||
@@ -36,6 +36,7 @@ flate2 = "1.1.9"
|
||||
tar = "0.4.46"
|
||||
zip = { version = "7.2.0", default-features = false, features = ["deflate"] }
|
||||
sha2 = "0.11.0"
|
||||
base64 = "0.22"
|
||||
tempfile = "3"
|
||||
nix = { version = "0.29", features = ["signal", "process"] }
|
||||
rand = "0.8"
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# BrowserSkill — Privacy Policy
|
||||
|
||||
**Last updated:** May 25, 2026
|
||||
**Last updated:** August 20, 2026
|
||||
|
||||
This Privacy Policy describes how the **BrowserSkill** browser extension (the "Extension") handles information when you install and use it. BrowserSkill is published as part of the open-source [BrowserSkill](https://github.com/Tencent/BrowserSkill) project. The source code is publicly auditable.
|
||||
|
||||
@@ -14,7 +14,7 @@ BrowserSkill is a local automation bridge that lets AI coding agents (such as Cu
|
||||
|
||||
## 2. Single Purpose
|
||||
|
||||
The Extension's single purpose is to expose browser automation primitives (navigation, DOM observation, screenshots, clicks, form filling, tab management) to a locally running BrowserSkill daemon over a WebSocket connection on `127.0.0.1`, so that an AI agent invoked by the user can interact with web pages on the user's behalf.
|
||||
The Extension's single purpose is to expose browser automation primitives (navigation, DOM observation, screenshots, clicks, form filling, task-scoped file transfer, and tab management) to a locally running BrowserSkill daemon over a WebSocket connection on `127.0.0.1`, so that an AI agent invoked by the user can interact with web pages on the user's behalf.
|
||||
|
||||
## 3. Data the Extension Accesses
|
||||
|
||||
@@ -26,6 +26,7 @@ Depending on the commands the user (via their AI agent) sends to the local daemo
|
||||
| **User input simulated by the agent** | Mouse clicks, keystrokes, and form values that the AI agent dispatches through the Chrome DevTools Protocol (CDP). | Required to perform automation actions the user has asked the agent to do. |
|
||||
| **Tab and window metadata** | Tab IDs, URLs, titles, window IDs of the Agent Window and any tabs the user explicitly authorizes. | Required to target automation commands at the correct tab/window. |
|
||||
| **Local extension storage** | A randomly generated 8-character instance ID and an optional user-supplied label. | Used so the local daemon can recognize this browser instance across reconnects. No personal data is stored. |
|
||||
| **File transfers requested by the agent** | Local files explicitly supplied to `bsk upload`, and the file created by a single `bsk download` action. | Required to attach a task file to a web page or return a browser-generated download to the invoking local agent. |
|
||||
| **OS notifications** | Permission to display a system notification when the agent requests to "borrow" one of the user's existing tabs. | Required to obtain explicit, per-tab user consent before the agent touches any pre-existing tab. |
|
||||
|
||||
## 4. Data the Extension Does **Not** Collect
|
||||
@@ -34,7 +35,7 @@ BrowserSkill does **not**:
|
||||
|
||||
- Send any data to remote servers, the Extension's authors, or any third party.
|
||||
- Call any LLM, AI, or cloud API. The Extension contains no API keys, model identifiers, or remote endpoints.
|
||||
- Read or transmit cookies, browsing history, bookmarks, downloads, saved passwords, or autofill data.
|
||||
- Read or transmit cookies, browsing history, bookmarks, saved passwords, or autofill data. It observes only the download initiated by an active `bsk download` call, not download history generally.
|
||||
- Use webcam, microphone, geolocation, or any device sensor.
|
||||
- Include analytics, telemetry, crash reporting, advertising SDKs, or fingerprinting code.
|
||||
- Track users across websites or across sessions.
|
||||
@@ -50,6 +51,7 @@ The Extension requests the following Chrome permissions. Each is used solely for
|
||||
- **`alarms`** — Periodically wake the service worker to keep the local WebSocket connection alive.
|
||||
- **`idle`** — Detect when the device returns from idle/locked so the Extension can promptly re-establish the local WebSocket connection after the machine wakes. No idle data is stored or transmitted.
|
||||
- **`notifications`** — Show a system notification to obtain user approval before the agent borrows a user-owned tab.
|
||||
- **`downloads`** — Correlate and route the one browser download initiated by an active `bsk download` command. If that claimed transaction fails, BrowserSkill cancels an in-progress file or removes its completed temporary browser file. It is not used to enumerate download history or alter unclaimed downloads.
|
||||
- **`storage`** — Persist a random instance ID and optional label in `chrome.storage.local`.
|
||||
- **Host permission `<all_urls>`** — Inject a small status overlay (showing "Agent Active") on pages controlled by the agent, and enable automation across whatever sites the user directs the agent to. The Extension does **not** read or transmit page content from sites the agent is not actively driving.
|
||||
|
||||
@@ -61,6 +63,7 @@ All Extension activity stays on the user's local device. The only network traffi
|
||||
|
||||
- The instance ID and optional label persist in `chrome.storage.local` until the user uninstalls the Extension or clears extension storage.
|
||||
- Page content, screenshots, DOM snapshots, and other observed data are returned to the local daemon in response to commands and are **not retained by the Extension**. They live only as long as the agent's tool call.
|
||||
- Upload and download bytes are staged by the local daemon in a private, session-scoped directory. Download staging is removed after it is copied to the requested destination. Upload staging is retained until the session ends so a later form submission can still read the attached file. Remaining staging is removed when the session ends or disconnects, or when the daemon next starts after a crash.
|
||||
|
||||
## 8. User Control
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import type { DraftTraceStep, PageRefV2, SelectedOptionV2, StepV2 } from "@/transport/types";
|
||||
import type { DraftTraceStep, PageRef, SelectedOption, Step } from "@/transport/types";
|
||||
|
||||
const CLIPBOARD_KEYS = new Set(["a", "c", "v", "x", "A", "C", "V", "X"]);
|
||||
const MODIFIER_ONLY_KEYS = new Set(["Meta", "Control", "Alt", "Shift", "OS", "Hyper", "Super"]);
|
||||
@@ -60,7 +60,7 @@ function collectUrls(steps: DraftTraceStep[], startUrl?: string): string[] {
|
||||
function buildPageRegistry(
|
||||
steps: DraftTraceStep[],
|
||||
startUrl?: string,
|
||||
): { pages: PageRefV2[]; urlToId: Map<string, string> } {
|
||||
): { pages: PageRef[]; urlToId: Map<string, string> } {
|
||||
const urls = collectUrls(steps, startUrl);
|
||||
const urlToId = new Map<string, string>();
|
||||
const pages = urls.map((url, index) => {
|
||||
@@ -90,19 +90,19 @@ function pageUrlForDraft(step: DraftTraceStep, fallbackUrl?: string): string | u
|
||||
function effectForNavigation(
|
||||
navigatedTo: string | undefined,
|
||||
urlToId: Map<string, string>,
|
||||
): StepV2["effect"] {
|
||||
): Step["effect"] {
|
||||
if (!navigatedTo) return undefined;
|
||||
const pageId = urlToId.get(navigatedTo);
|
||||
if (!pageId) return undefined;
|
||||
return { navigated_to: pageId };
|
||||
}
|
||||
|
||||
function withEffect(step: StepV2, effect: StepV2["effect"]): StepV2 {
|
||||
function withEffect(step: Step, effect: Step["effect"]): Step {
|
||||
if (!effect) return step;
|
||||
return { ...step, effect };
|
||||
}
|
||||
|
||||
function toSelection(values: string[], labels?: string[]): SelectedOptionV2[] {
|
||||
function toSelection(values: string[], labels?: string[]): SelectedOption[] {
|
||||
return values.map((value, index) => ({
|
||||
value,
|
||||
...(labels?.[index] ? { label: labels[index] } : {}),
|
||||
@@ -114,7 +114,7 @@ function toV2Step(
|
||||
id: number,
|
||||
urlToId: Map<string, string>,
|
||||
fallbackUrl?: string,
|
||||
): StepV2 | null {
|
||||
): Step | null {
|
||||
if (!shouldIncludeDraft(step)) return null;
|
||||
|
||||
const pageUrl = pageUrlForDraft(step, fallbackUrl);
|
||||
@@ -181,8 +181,8 @@ function toV2Step(
|
||||
}
|
||||
|
||||
export interface ReducedTrace {
|
||||
pages: PageRefV2[];
|
||||
steps: StepV2[];
|
||||
pages: PageRef[];
|
||||
steps: Step[];
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -192,7 +192,7 @@ export interface ReducedTrace {
|
||||
export function reduceTraceSteps(steps: DraftTraceStep[], startUrl?: string): ReducedTrace {
|
||||
const collapsed = collapseNavigations(steps);
|
||||
const { pages, urlToId } = buildPageRegistry(collapsed, startUrl);
|
||||
const out: StepV2[] = [];
|
||||
const out: Step[] = [];
|
||||
let id = 1;
|
||||
let lastUrl = startUrl;
|
||||
for (const draft of collapsed) {
|
||||
@@ -210,7 +210,7 @@ export function reduceTraceSteps(steps: DraftTraceStep[], startUrl?: string): Re
|
||||
export function resolveTraceStartUrl(
|
||||
drafts: DraftTraceStep[],
|
||||
startUrl?: string,
|
||||
pages?: PageRefV2[],
|
||||
pages?: PageRef[],
|
||||
): string {
|
||||
if (startUrl) return startUrl;
|
||||
const navigate = drafts.find((step): step is Extract<DraftTraceStep, { op: "navigate" }> => {
|
||||
|
||||
@@ -168,6 +168,107 @@ describe("ToolDispatcher", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("bypasses and restores the control overlay for an upload trigger click", async () => {
|
||||
const sendMessage = vi.fn(async () => undefined);
|
||||
vi.stubGlobal("chrome", {
|
||||
tabs: {
|
||||
get: vi.fn(async () => ({ id: 7, windowId: 4242, active: true })),
|
||||
query: vi.fn(async () => [{ id: 7, windowId: 4242, active: true }]),
|
||||
sendMessage,
|
||||
},
|
||||
});
|
||||
const { transport, sent, deliver } = fakeTransport();
|
||||
const sessions = new SessionManager({
|
||||
agentWindow: {
|
||||
create: vi.fn(async () => 4242),
|
||||
remove: vi.fn(async () => {}),
|
||||
ensureActiveTab: vi.fn(async () => {}),
|
||||
},
|
||||
});
|
||||
const ctx = await sessions.start("aa11");
|
||||
ctx.refStore.set("e1", 123, { tabId: 7 });
|
||||
const send = vi.fn(async <T>(_tabId: number, method: string, params?: object) => {
|
||||
if (method === "Page.getLayoutMetrics") {
|
||||
return { cssLayoutViewport: { clientWidth: 1280, clientHeight: 720 } } as T;
|
||||
}
|
||||
if (method === "DOM.getContentQuads") {
|
||||
return { quads: [[0, 0, 20, 0, 20, 20, 0, 20]] } as T;
|
||||
}
|
||||
if (method === "DOM.resolveNode") {
|
||||
return { object: { objectId: "trigger-object" } } as T;
|
||||
}
|
||||
if (method === "DOM.describeNode") return { node: { backendNodeId: 456 } } as T;
|
||||
if (method === "Runtime.callFunctionOn") {
|
||||
const declaration = (params as { functionDeclaration?: string }).functionDeclaration ?? "";
|
||||
if (declaration.includes("count: state.inputs.length")) {
|
||||
return { result: { value: { count: 1, multiple: false } } } as T;
|
||||
}
|
||||
if (declaration.includes("inputs[0]")) {
|
||||
return { result: { objectId: "input-object" } } as T;
|
||||
}
|
||||
return { result: { value: true } } as T;
|
||||
}
|
||||
if (method === "Runtime.evaluate") {
|
||||
const expression = (params as { expression?: string }).expression ?? "";
|
||||
if (expression.includes("overlayDetails")) {
|
||||
return { result: { value: { hitIndex: 0 } } } as T;
|
||||
}
|
||||
if (expression.includes("overlayHostPresent")) {
|
||||
return {
|
||||
result: {
|
||||
value: { overlayHostPresent: true, overlayHostConnected: true },
|
||||
},
|
||||
} as T;
|
||||
}
|
||||
if (expression.includes("count:")) {
|
||||
return { result: { value: { count: 1, multiple: false } } } as T;
|
||||
}
|
||||
if (expression.includes("?.inputs[0]")) {
|
||||
return { result: { objectId: "input-object" } } as T;
|
||||
}
|
||||
return { result: { value: true } } as T;
|
||||
}
|
||||
return {} as T;
|
||||
});
|
||||
const cdp = {
|
||||
send,
|
||||
detachSession: vi.fn(async () => {}),
|
||||
ensureNetworkCapture: vi.fn(async () => {}),
|
||||
networkEntriesSince: vi.fn(() => ({
|
||||
tab_id: 7,
|
||||
entries: [],
|
||||
next_since: 0,
|
||||
truncated: false,
|
||||
})),
|
||||
setDeviceMetricsOverride: vi.fn(async () => {}),
|
||||
clearDeviceMetricsOverride: vi.fn(async () => {}),
|
||||
setUserAgentOverride: vi.fn(async () => {}),
|
||||
setTouchEmulationEnabled: vi.fn(async () => {}),
|
||||
};
|
||||
const dispatcher = new ToolDispatcher({ transport, sessions, cdp: cdp as TestDispatcherCdp });
|
||||
dispatcher.start();
|
||||
|
||||
deliver(
|
||||
makeRequest("tool.upload", {
|
||||
session_id: "aa11",
|
||||
ref: "@e1",
|
||||
files: [{ transfer_id: "tr_1", name: "test.png", staged_path: "/stage/test.png" }],
|
||||
}),
|
||||
);
|
||||
await flushMicrotasks();
|
||||
await vi.waitFor(() => expect(sent).toHaveLength(1));
|
||||
|
||||
expect(sent[0]).toMatchObject({ result: { tab_id: 7, file_names: ["test.png"] } });
|
||||
expect(sendMessage).toHaveBeenNthCalledWith(1, 7, {
|
||||
type: "bh-automation-bypass",
|
||||
enabled: true,
|
||||
});
|
||||
expect(sendMessage).toHaveBeenNthCalledWith(2, 7, {
|
||||
type: "bh-automation-bypass",
|
||||
enabled: false,
|
||||
});
|
||||
});
|
||||
|
||||
it("detaches CDP state before stopping a session", async () => {
|
||||
const { transport, sent, deliver } = fakeTransport();
|
||||
const sessions = new SessionManager({
|
||||
@@ -203,55 +304,6 @@ describe("ToolDispatcher", () => {
|
||||
expect(sent[0]).toEqual({ id: "r-1", result: {} });
|
||||
});
|
||||
|
||||
it("forwards daemon cancellation into an in-flight session stop", async () => {
|
||||
const { transport, sent, deliver } = fakeTransport();
|
||||
const remove = vi.fn(async () => {});
|
||||
const sessions = new SessionManager({
|
||||
agentWindow: {
|
||||
create: vi.fn(async () => 4242),
|
||||
remove,
|
||||
ensureActiveTab: vi.fn(async () => {}),
|
||||
},
|
||||
});
|
||||
await sessions.start("aa11");
|
||||
let finishDetach: () => void = () => {};
|
||||
const detach = new Promise<void>((resolve) => {
|
||||
finishDetach = resolve;
|
||||
});
|
||||
const cdp = {
|
||||
send: vi.fn(),
|
||||
detachSession: vi.fn(() => detach),
|
||||
ensureNetworkCapture: vi.fn(async () => {}),
|
||||
networkEntriesSince: vi.fn(() => ({
|
||||
tab_id: 7,
|
||||
entries: [],
|
||||
next_since: 0,
|
||||
truncated: false,
|
||||
})),
|
||||
setDeviceMetricsOverride: vi.fn(async () => {}),
|
||||
clearDeviceMetricsOverride: vi.fn(async () => {}),
|
||||
setUserAgentOverride: vi.fn(async () => {}),
|
||||
setTouchEmulationEnabled: vi.fn(async () => {}),
|
||||
};
|
||||
const dispatcher = new ToolDispatcher({ transport, sessions, cdp: cdp as TestDispatcherCdp });
|
||||
dispatcher.start();
|
||||
|
||||
deliver(makeRequest("tool.session_stop", { session_id: "aa11" }));
|
||||
await vi.waitFor(() => expect(cdp.detachSession).toHaveBeenCalledWith("aa11"));
|
||||
deliver({ id: "cancel-stop", method: "cancel", params: { rpc_id: "r-1" } });
|
||||
await flushMicrotasks();
|
||||
finishDetach();
|
||||
await flushMicrotasks();
|
||||
|
||||
expect(sent).toContainEqual({ id: "cancel-stop", result: { cancelled: true } });
|
||||
expect(sent).toContainEqual({
|
||||
id: "r-1",
|
||||
error: expect.objectContaining({ code: "cancelled" }),
|
||||
});
|
||||
expect(sessions.has("aa11")).toBe(true);
|
||||
expect(remove).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("returns not_found when stopping an unknown session", async () => {
|
||||
const { transport, sent, deliver } = fakeTransport();
|
||||
const sessions = new SessionManager({
|
||||
|
||||
@@ -0,0 +1,559 @@
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import { SessionManager } from "@/session-manager/manager";
|
||||
import { type DownloadsApi, handleDownload } from "../download";
|
||||
import { captureBrowserDownload } from "../download-capture";
|
||||
import { uploadThroughActivatedFileInput } from "../file-input-transaction";
|
||||
import type { ResolvedActionTarget } from "../interaction";
|
||||
import type { CdpRunner } from "../shared";
|
||||
import { handleUpload } from "../upload";
|
||||
|
||||
function sessions() {
|
||||
return new SessionManager({
|
||||
agentWindow: {
|
||||
create: vi.fn(async () => 100),
|
||||
remove: vi.fn(async () => {}),
|
||||
ensureActiveTab: vi.fn(async () => {}),
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
function tabsApi() {
|
||||
return {
|
||||
get: vi.fn(
|
||||
async (tabId: number) => ({ id: tabId, windowId: 100, active: true }) as chrome.tabs.Tab,
|
||||
),
|
||||
query: vi.fn(async () => [{ id: 4, windowId: 100, active: true } as chrome.tabs.Tab]),
|
||||
};
|
||||
}
|
||||
|
||||
function fakeEvent<T extends (...args: never[]) => unknown>() {
|
||||
const listeners = new Set<T>();
|
||||
return {
|
||||
addListener: (listener: T) => listeners.add(listener),
|
||||
removeListener: (listener: T) => listeners.delete(listener),
|
||||
emit: (...args: Parameters<T>) => {
|
||||
for (const listener of listeners) listener(...args);
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
function actionTarget(frameId?: string): ResolvedActionTarget {
|
||||
return {
|
||||
tab: { tabId: 4, windowId: 100, active: true },
|
||||
backendNodeId: 123,
|
||||
cdpTarget: { tabId: 4 },
|
||||
...(frameId ? { frameId } : {}),
|
||||
usedRef: "e3",
|
||||
};
|
||||
}
|
||||
|
||||
function uploadCdp(
|
||||
options: {
|
||||
inputCount?: number;
|
||||
multiple?: boolean;
|
||||
chooser?: { frameId?: string; backendNodeId?: number; mode?: string };
|
||||
pendingResolve?: boolean;
|
||||
} = {},
|
||||
) {
|
||||
const calls: Array<{ method: string; params?: object }> = [];
|
||||
let cdpEvent: Parameters<NonNullable<CdpRunner["onEvent"]>>[0] | undefined;
|
||||
const send = vi.fn(async (_tabId: number, method: string, params?: object) => {
|
||||
calls.push({ method, params });
|
||||
if (method === "Page.setInterceptFileChooserDialog") return {};
|
||||
if (method === "Page.getLayoutMetrics")
|
||||
return { cssLayoutViewport: { clientWidth: 1280, clientHeight: 720 } };
|
||||
if (method === "DOM.getContentQuads") return { quads: [[0, 0, 20, 0, 20, 20, 0, 20]] };
|
||||
if (method === "DOM.resolveNode") {
|
||||
if (options.pendingResolve) return new Promise<never>(() => {});
|
||||
return { object: { objectId: "trigger-object" } };
|
||||
}
|
||||
if (method === "Runtime.callFunctionOn") {
|
||||
const declaration = (params as { functionDeclaration?: string }).functionDeclaration ?? "";
|
||||
if (declaration.includes("Object.defineProperty")) return { result: { value: true } };
|
||||
if (declaration.includes("count: state.inputs.length")) {
|
||||
return {
|
||||
result: {
|
||||
value: {
|
||||
count: options.inputCount ?? 1,
|
||||
multiple: options.multiple ?? true,
|
||||
},
|
||||
},
|
||||
};
|
||||
}
|
||||
if (declaration.includes("inputs[0]")) {
|
||||
return { result: { objectId: "input-object" } };
|
||||
}
|
||||
return { result: { value: true } };
|
||||
}
|
||||
if (method === "DOM.describeNode") return { node: { backendNodeId: 456 } };
|
||||
if (
|
||||
method === "Input.dispatchMouseEvent" &&
|
||||
(params as { type?: string }).type === "mousePressed" &&
|
||||
options.chooser
|
||||
) {
|
||||
cdpEvent?.({ tabId: 4 }, "Page.fileChooserOpened", options.chooser);
|
||||
}
|
||||
return {};
|
||||
});
|
||||
const cdp: CdpRunner = {
|
||||
send: send as unknown as CdpRunner["send"],
|
||||
onEvent: (handler) => {
|
||||
cdpEvent = handler;
|
||||
return { dispose: vi.fn() };
|
||||
},
|
||||
};
|
||||
return {
|
||||
calls,
|
||||
emitChooser: (event: { frameId?: string; backendNodeId?: number; mode?: string }) =>
|
||||
cdpEvent?.({ tabId: 4 }, "Page.fileChooserOpened", event),
|
||||
cdp,
|
||||
};
|
||||
}
|
||||
|
||||
describe("file transfer tools", () => {
|
||||
it("captures the file input activated by the requested click and injects only staged paths", async () => {
|
||||
const manager = sessions();
|
||||
const ctx = await manager.start("s1");
|
||||
ctx.refStore.set("e3", 123, { tabId: 4 });
|
||||
const { cdp, calls } = uploadCdp();
|
||||
|
||||
const result = await handleUpload(
|
||||
manager,
|
||||
{
|
||||
session_id: "s1",
|
||||
ref: "@e3",
|
||||
files: [
|
||||
{ transfer_id: "tr_1", name: "one.png", staged_path: "/private/stage/one" },
|
||||
{ transfer_id: "tr_2", name: "two.png", staged_path: "/private/stage/two" },
|
||||
],
|
||||
},
|
||||
{ cdp, tabsApi: tabsApi() },
|
||||
);
|
||||
|
||||
expect(result).toMatchObject({ tab_id: 4, file_names: ["one.png", "two.png"] });
|
||||
expect(calls[0]).toEqual({
|
||||
method: "Page.setInterceptFileChooserDialog",
|
||||
params: { enabled: true },
|
||||
});
|
||||
expect(calls).toContainEqual({
|
||||
method: "Page.setInterceptFileChooserDialog",
|
||||
params: { enabled: false, cancel: true },
|
||||
});
|
||||
expect(calls).toContainEqual({
|
||||
method: "DOM.setFileInputFiles",
|
||||
params: { files: ["/private/stage/one", "/private/stage/two"], backendNodeId: 456 },
|
||||
});
|
||||
});
|
||||
|
||||
it("fails immediately when the trigger does not activate a file input", async () => {
|
||||
const { cdp } = uploadCdp({ inputCount: 0 });
|
||||
const result = await uploadThroughActivatedFileInput({
|
||||
cdp,
|
||||
actionTarget: actionTarget(),
|
||||
files: ["/private/stage/one"],
|
||||
timeoutMs: 100,
|
||||
trigger: async () => ({ tab_id: 4, x: 10, y: 10 }),
|
||||
});
|
||||
|
||||
expect(result).toMatchObject({
|
||||
code: "unsupported",
|
||||
data: { reason: "file_input_not_activated", phase: "resolve_input" },
|
||||
});
|
||||
});
|
||||
|
||||
it("fails before clicking when chooser interception is unavailable", async () => {
|
||||
const trigger = vi.fn(async () => ({ tab_id: 4, x: 10, y: 10 }));
|
||||
const cdp: CdpRunner = {
|
||||
send: vi.fn(async (_tabId: number, method: string) => {
|
||||
if (method === "Page.setInterceptFileChooserDialog") {
|
||||
throw new Error("method unavailable");
|
||||
}
|
||||
return {};
|
||||
}) as CdpRunner["send"],
|
||||
};
|
||||
|
||||
const result = await uploadThroughActivatedFileInput({
|
||||
cdp,
|
||||
actionTarget: actionTarget(),
|
||||
files: ["/private/stage/one"],
|
||||
timeoutMs: 100,
|
||||
trigger,
|
||||
});
|
||||
|
||||
expect(result).toMatchObject({
|
||||
code: "cdp_failed",
|
||||
data: { effect_state: "none", phase: "arm_interception" },
|
||||
});
|
||||
expect(trigger).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("uses an exact chooser event as an independent input-location signal", async () => {
|
||||
const { cdp, calls, emitChooser } = uploadCdp({ inputCount: 0 });
|
||||
const result = await uploadThroughActivatedFileInput({
|
||||
cdp,
|
||||
actionTarget: actionTarget("f1"),
|
||||
files: ["/private/stage/one"],
|
||||
timeoutMs: 100,
|
||||
trigger: async () => {
|
||||
emitChooser({ frameId: "f1", backendNodeId: 789, mode: "selectSingle" });
|
||||
return { tab_id: 4, x: 10, y: 10 };
|
||||
},
|
||||
});
|
||||
|
||||
expect(result).toMatchObject({ multiple: false });
|
||||
expect(calls).toContainEqual({
|
||||
method: "DOM.setFileInputFiles",
|
||||
params: { files: ["/private/stage/one"], backendNodeId: 789 },
|
||||
});
|
||||
});
|
||||
|
||||
it("reports a File System Access picker without waiting for a timeout", async () => {
|
||||
const { cdp, emitChooser } = uploadCdp({
|
||||
inputCount: 0,
|
||||
});
|
||||
const result = await uploadThroughActivatedFileInput({
|
||||
cdp,
|
||||
actionTarget: actionTarget("f1"),
|
||||
files: ["/private/stage/one"],
|
||||
timeoutMs: 100,
|
||||
trigger: async () => {
|
||||
emitChooser({ frameId: "f1", mode: "selectSingle" });
|
||||
return { tab_id: 4, x: 10, y: 10 };
|
||||
},
|
||||
});
|
||||
|
||||
expect(result).toMatchObject({
|
||||
code: "unsupported",
|
||||
message: "upload trigger invoked a non-input file picker",
|
||||
data: { reason: "file_input_not_activated", phase: "resolve_input" },
|
||||
});
|
||||
});
|
||||
|
||||
it("bounds a stuck file-input probe", async () => {
|
||||
const { cdp } = uploadCdp({ pendingResolve: true });
|
||||
|
||||
const result = await uploadThroughActivatedFileInput({
|
||||
cdp,
|
||||
actionTarget: actionTarget(),
|
||||
files: ["/private/stage/one"],
|
||||
timeoutMs: 5,
|
||||
trigger: vi.fn(),
|
||||
});
|
||||
|
||||
expect(result).toMatchObject({
|
||||
code: "timeout",
|
||||
data: { reason: "file_input_probe_failed", phase: "arm_input_probe" },
|
||||
});
|
||||
});
|
||||
|
||||
it("marks a timed-out file assignment unknown and detaches browser state", async () => {
|
||||
const fixture = uploadCdp();
|
||||
const originalSend = fixture.cdp.send;
|
||||
const detach = vi.fn(async () => {});
|
||||
fixture.cdp.detach = detach;
|
||||
fixture.cdp.send = vi.fn((tabId: number, method: string, params?: object) => {
|
||||
if (method === "DOM.setFileInputFiles") return new Promise<never>(() => {});
|
||||
return originalSend(tabId, method, params);
|
||||
}) as CdpRunner["send"];
|
||||
|
||||
const result = await uploadThroughActivatedFileInput({
|
||||
cdp: fixture.cdp,
|
||||
actionTarget: actionTarget(),
|
||||
files: ["/private/stage/one"],
|
||||
timeoutMs: 10,
|
||||
trigger: async () => ({ tab_id: 4, x: 10, y: 10 }),
|
||||
});
|
||||
|
||||
expect(result).toMatchObject({
|
||||
code: "timeout",
|
||||
data: { effect_state: "unknown", phase: "set_files" },
|
||||
});
|
||||
expect(detach).toHaveBeenCalledWith(4);
|
||||
});
|
||||
|
||||
it("routes one exact-target download through a browser-relative capability", async () => {
|
||||
const manager = sessions();
|
||||
const ctx = await manager.start("s1");
|
||||
ctx.refStore.set("e3", 123, { tabId: 4 });
|
||||
const onCreated = fakeEvent<(item: chrome.downloads.DownloadItem) => void>();
|
||||
const onChanged = fakeEvent<(delta: chrome.downloads.DownloadDelta) => void>();
|
||||
const onDeterminingFilename =
|
||||
fakeEvent<
|
||||
(
|
||||
item: chrome.downloads.DownloadItem,
|
||||
suggest: (suggestion?: chrome.downloads.DownloadFilenameSuggestion) => void,
|
||||
) => void | true
|
||||
>();
|
||||
const initial = {
|
||||
id: 9,
|
||||
url: "https://example.test/result.zip",
|
||||
finalUrl: "https://example.test/result.zip",
|
||||
filename: "result.zip",
|
||||
state: "in_progress",
|
||||
fileSize: -1,
|
||||
totalBytes: 12,
|
||||
mime: "application/zip",
|
||||
danger: "safe",
|
||||
} as chrome.downloads.DownloadItem;
|
||||
const completed = {
|
||||
...initial,
|
||||
filename: "/profile/Downloads/BrowserSkill/tr_1/result.zip",
|
||||
state: "complete",
|
||||
fileSize: 12,
|
||||
} as chrome.downloads.DownloadItem;
|
||||
const downloads: DownloadsApi = {
|
||||
onCreated,
|
||||
onChanged,
|
||||
onDeterminingFilename,
|
||||
search: vi.fn(async () => [completed]),
|
||||
cancel: vi.fn(async () => {}),
|
||||
removeFile: vi.fn(async () => {}),
|
||||
};
|
||||
let cdpEvent: Parameters<NonNullable<CdpRunner["onEvent"]>>[0] | undefined;
|
||||
let suggested: chrome.downloads.DownloadFilenameSuggestion | undefined;
|
||||
const send = vi.fn(async (_tabId: number, method: string, params?: object) => {
|
||||
if (method === "Page.getLayoutMetrics")
|
||||
return { cssLayoutViewport: { clientWidth: 1280, clientHeight: 720 } };
|
||||
if (method === "DOM.getContentQuads") return { quads: [[0, 0, 20, 0, 20, 20, 0, 20]] };
|
||||
if (
|
||||
method === "Input.dispatchMouseEvent" &&
|
||||
(params as { type?: string }).type === "mousePressed"
|
||||
) {
|
||||
cdpEvent?.({ tabId: 4 }, "Page.downloadWillBegin", {
|
||||
url: initial.url,
|
||||
suggestedFilename: "result.zip",
|
||||
});
|
||||
await new Promise<void>((resolve) => {
|
||||
onDeterminingFilename.emit(initial, (value) => {
|
||||
suggested = value;
|
||||
resolve();
|
||||
});
|
||||
});
|
||||
onCreated.emit(completed);
|
||||
}
|
||||
return {};
|
||||
});
|
||||
const cdp: CdpRunner = {
|
||||
send: send as unknown as CdpRunner["send"],
|
||||
onEvent: (handler) => {
|
||||
cdpEvent = handler;
|
||||
return { dispose: vi.fn() };
|
||||
},
|
||||
};
|
||||
|
||||
const result = await handleDownload(
|
||||
manager,
|
||||
{ session_id: "s1", ref: "@e3", browser_relative_dir: "BrowserSkill/tr_1" },
|
||||
{ cdp, tabsApi: tabsApi(), downloads },
|
||||
);
|
||||
|
||||
expect(suggested).toEqual({
|
||||
filename: "BrowserSkill/tr_1/result.zip",
|
||||
conflictAction: "overwrite",
|
||||
});
|
||||
expect(result).toMatchObject({
|
||||
tab_id: 4,
|
||||
suggested_filename: "result.zip",
|
||||
byte_size: 12,
|
||||
browser_path: completed.filename,
|
||||
});
|
||||
});
|
||||
|
||||
it("does not claim a download without an intent from the exact target", async () => {
|
||||
const onCreated = fakeEvent<(item: chrome.downloads.DownloadItem) => void>();
|
||||
const onChanged = fakeEvent<(delta: chrome.downloads.DownloadDelta) => void>();
|
||||
const onDeterminingFilename =
|
||||
fakeEvent<
|
||||
(
|
||||
item: chrome.downloads.DownloadItem,
|
||||
suggest: (suggestion?: chrome.downloads.DownloadFilenameSuggestion) => void,
|
||||
) => void | true
|
||||
>();
|
||||
const downloads: DownloadsApi = {
|
||||
onCreated,
|
||||
onChanged,
|
||||
onDeterminingFilename,
|
||||
search: vi.fn(async () => []),
|
||||
cancel: vi.fn(async () => {}),
|
||||
removeFile: vi.fn(async () => {}),
|
||||
};
|
||||
let cdpEvent: Parameters<NonNullable<CdpRunner["onEvent"]>>[0] | undefined;
|
||||
const cdp: CdpRunner = {
|
||||
send: vi.fn(async () => ({})) as unknown as CdpRunner["send"],
|
||||
onEvent: (handler) => {
|
||||
cdpEvent = handler;
|
||||
return { dispose: vi.fn() };
|
||||
},
|
||||
};
|
||||
const unrelated = {
|
||||
id: 17,
|
||||
url: "https://example.test/unrelated.zip",
|
||||
finalUrl: "https://example.test/unrelated.zip",
|
||||
filename: "unrelated.zip",
|
||||
state: "in_progress",
|
||||
} as chrome.downloads.DownloadItem;
|
||||
let defaultSuggestionCalled = false;
|
||||
|
||||
const result = await captureBrowserDownload({
|
||||
cdp,
|
||||
target: { tabId: 4, sessionId: "expected-child" },
|
||||
downloads,
|
||||
browserRelativeDir: "BrowserSkill/tr_1",
|
||||
timeoutMs: 5,
|
||||
trigger: async () => {
|
||||
cdpEvent?.({ tabId: 4, sessionId: "other-child" }, "Page.downloadWillBegin", {
|
||||
url: unrelated.url,
|
||||
suggestedFilename: unrelated.filename,
|
||||
});
|
||||
onDeterminingFilename.emit(unrelated, (suggestion) => {
|
||||
defaultSuggestionCalled = suggestion === undefined;
|
||||
});
|
||||
return { tab_id: 4, x: 10, y: 10 };
|
||||
},
|
||||
});
|
||||
|
||||
expect(defaultSuggestionCalled).toBe(true);
|
||||
expect(downloads.cancel).not.toHaveBeenCalled();
|
||||
expect(result).toMatchObject({
|
||||
code: "cdp_failed",
|
||||
data: { reason: "download_capture_failed" },
|
||||
});
|
||||
});
|
||||
|
||||
it("correlates a filename candidate that arrives before the CDP intent", async () => {
|
||||
const onCreated = fakeEvent<(item: chrome.downloads.DownloadItem) => void>();
|
||||
const onChanged = fakeEvent<(delta: chrome.downloads.DownloadDelta) => void>();
|
||||
const onDeterminingFilename =
|
||||
fakeEvent<
|
||||
(
|
||||
item: chrome.downloads.DownloadItem,
|
||||
suggest: (suggestion?: chrome.downloads.DownloadFilenameSuggestion) => void,
|
||||
) => void | true
|
||||
>();
|
||||
const initial = {
|
||||
id: 21,
|
||||
url: "https://example.test/candidate-first.bin",
|
||||
finalUrl: "https://example.test/candidate-first.bin",
|
||||
filename: "candidate-first.bin",
|
||||
state: "in_progress",
|
||||
fileSize: -1,
|
||||
totalBytes: 4,
|
||||
bytesReceived: 0,
|
||||
} as chrome.downloads.DownloadItem;
|
||||
const complete = {
|
||||
...initial,
|
||||
state: "complete",
|
||||
fileSize: 4,
|
||||
} as chrome.downloads.DownloadItem;
|
||||
const downloads: DownloadsApi = {
|
||||
onCreated,
|
||||
onChanged,
|
||||
onDeterminingFilename,
|
||||
search: vi.fn(async () => [complete]),
|
||||
cancel: vi.fn(async () => {}),
|
||||
removeFile: vi.fn(async () => {}),
|
||||
};
|
||||
let cdpEvent: Parameters<NonNullable<CdpRunner["onEvent"]>>[0] | undefined;
|
||||
const cdp: CdpRunner = {
|
||||
send: vi.fn(async () => ({})) as CdpRunner["send"],
|
||||
onEvent: (handler) => {
|
||||
cdpEvent = handler;
|
||||
return { dispose: vi.fn() };
|
||||
},
|
||||
};
|
||||
let suggestion: chrome.downloads.DownloadFilenameSuggestion | undefined;
|
||||
|
||||
const result = await captureBrowserDownload({
|
||||
cdp,
|
||||
target: { tabId: 4 },
|
||||
downloads,
|
||||
browserRelativeDir: "BrowserSkill/tr_21",
|
||||
timeoutMs: 1_000,
|
||||
trigger: async () => {
|
||||
const suggested = new Promise<void>((resolve) => {
|
||||
onDeterminingFilename.emit(initial, (value) => {
|
||||
suggestion = value;
|
||||
resolve();
|
||||
});
|
||||
});
|
||||
cdpEvent?.({ tabId: 4 }, "Page.downloadWillBegin", {
|
||||
url: initial.url,
|
||||
suggestedFilename: initial.filename,
|
||||
});
|
||||
await suggested;
|
||||
onCreated.emit(complete);
|
||||
return { tab_id: 4, x: 10, y: 10 };
|
||||
},
|
||||
});
|
||||
|
||||
expect(suggestion).toEqual({
|
||||
filename: "BrowserSkill/tr_21/candidate-first.bin",
|
||||
conflictAction: "overwrite",
|
||||
});
|
||||
expect(result).toMatchObject({ item: { id: 21, state: "complete" } });
|
||||
});
|
||||
|
||||
it("rejects ambiguous attribution without cancelling either unclaimed download", async () => {
|
||||
const onCreated = fakeEvent<(item: chrome.downloads.DownloadItem) => void>();
|
||||
const onChanged = fakeEvent<(delta: chrome.downloads.DownloadDelta) => void>();
|
||||
const onDeterminingFilename =
|
||||
fakeEvent<
|
||||
(
|
||||
item: chrome.downloads.DownloadItem,
|
||||
suggest: (suggestion?: chrome.downloads.DownloadFilenameSuggestion) => void,
|
||||
) => void | true
|
||||
>();
|
||||
const downloads: DownloadsApi = {
|
||||
onCreated,
|
||||
onChanged,
|
||||
onDeterminingFilename,
|
||||
search: vi.fn(async () => []),
|
||||
cancel: vi.fn(async () => {}),
|
||||
removeFile: vi.fn(async () => {}),
|
||||
};
|
||||
let cdpEvent: Parameters<NonNullable<CdpRunner["onEvent"]>>[0] | undefined;
|
||||
const cdp: CdpRunner = {
|
||||
send: vi.fn(async () => ({})) as CdpRunner["send"],
|
||||
onEvent: (handler) => {
|
||||
cdpEvent = handler;
|
||||
return { dispose: vi.fn() };
|
||||
},
|
||||
};
|
||||
const defaults: number[] = [];
|
||||
const candidate = (id: number) =>
|
||||
({
|
||||
id,
|
||||
url: "https://example.test/same.bin",
|
||||
finalUrl: "https://example.test/same.bin",
|
||||
filename: "same.bin",
|
||||
state: "in_progress",
|
||||
}) as chrome.downloads.DownloadItem;
|
||||
|
||||
const result = await captureBrowserDownload({
|
||||
cdp,
|
||||
target: { tabId: 4 },
|
||||
downloads,
|
||||
browserRelativeDir: "BrowserSkill/tr_ambiguous",
|
||||
timeoutMs: 100,
|
||||
trigger: async () => {
|
||||
cdpEvent?.({ tabId: 4 }, "Page.downloadWillBegin", {
|
||||
url: "https://example.test/same.bin",
|
||||
suggestedFilename: "same.bin",
|
||||
});
|
||||
for (const id of [31, 32]) {
|
||||
onDeterminingFilename.emit(candidate(id), (value) => {
|
||||
if (value === undefined) defaults.push(id);
|
||||
});
|
||||
}
|
||||
return { tab_id: 4, x: 10, y: 10 };
|
||||
},
|
||||
});
|
||||
|
||||
expect(defaults.sort()).toEqual([31, 32]);
|
||||
expect(downloads.cancel).not.toHaveBeenCalled();
|
||||
expect(result).toMatchObject({
|
||||
code: "cdp_failed",
|
||||
data: { effect_state: "unknown", phase: "attribution" },
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -8,7 +8,6 @@ import {
|
||||
buildFrameVomScene,
|
||||
buildVomScene,
|
||||
type CdpAxNode,
|
||||
captureVomObservation,
|
||||
handleGetHtml,
|
||||
handleObserve,
|
||||
handleScreenshot,
|
||||
@@ -1330,7 +1329,7 @@ describe("buildVomScene", () => {
|
||||
expect(rendered).toContain(' @e2 textbox "验证码"');
|
||||
});
|
||||
|
||||
it("marks current-password autocomplete as sensitive even for text inputs", () => {
|
||||
it("does not mark non-password inputs sensitive from password-like labels", () => {
|
||||
const axNodes: CdpAxNode[] = [
|
||||
{
|
||||
nodeId: "1",
|
||||
@@ -1365,7 +1364,7 @@ describe("buildVomScene", () => {
|
||||
expect(buildVomScene(axNodes, captured).nodes[0]).toEqual(
|
||||
expect.objectContaining({
|
||||
id: 10,
|
||||
sensitive: true,
|
||||
sensitive: false,
|
||||
}),
|
||||
);
|
||||
});
|
||||
@@ -1660,9 +1659,7 @@ describe("buildVomScene", () => {
|
||||
);
|
||||
const rendered = renderVom(scene);
|
||||
expect(rendered.text).toContain('@e1 button "close"');
|
||||
expect(rendered.refs.map(({ ref, backendNodeId }) => ({ ref, backendNodeId }))).toEqual([
|
||||
{ ref: "e1", backendNodeId: 20 },
|
||||
]);
|
||||
expect(rendered.refs).toEqual([{ ref: "e1", backendNodeId: 20 }]);
|
||||
});
|
||||
|
||||
it("does not promote a clickable container that wraps a real interactive control", () => {
|
||||
@@ -1806,9 +1803,7 @@ describe("buildVomScene", () => {
|
||||
expect(scene.nodes.find((n) => n.id === 30)?.role).toBe("generic");
|
||||
const rendered = renderVom(scene);
|
||||
expect(rendered.text).toContain('@e1 button "收藏"');
|
||||
expect(rendered.refs.map(({ ref, backendNodeId }) => ({ ref, backendNodeId }))).toEqual([
|
||||
{ ref: "e1", backendNodeId: 20 },
|
||||
]);
|
||||
expect(rendered.refs).toEqual([{ ref: "e1", backendNodeId: 20 }]);
|
||||
});
|
||||
|
||||
it("builds active scope blocks from active aria-controls relationships", () => {
|
||||
@@ -2909,360 +2904,10 @@ describe("handleSnapshot", () => {
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
function secretsSnapshotReply(secrets: {
|
||||
email: string;
|
||||
password: string;
|
||||
otp: string;
|
||||
card: string;
|
||||
}) {
|
||||
const S = [
|
||||
"html",
|
||||
"body",
|
||||
"input",
|
||||
"position",
|
||||
"static",
|
||||
"pointer-events",
|
||||
"auto",
|
||||
"type",
|
||||
"text",
|
||||
"password",
|
||||
"autocomplete",
|
||||
"one-time-code",
|
||||
"cc-number",
|
||||
"value",
|
||||
secrets.email,
|
||||
secrets.password,
|
||||
secrets.otp,
|
||||
secrets.card,
|
||||
];
|
||||
const i = (s: string) => S.indexOf(s);
|
||||
const style = [i("static"), i("auto"), i("auto")];
|
||||
return {
|
||||
strings: S,
|
||||
documents: [
|
||||
{
|
||||
nodes: {
|
||||
parentIndex: [-1, 0, 1, 1, 1, 1],
|
||||
nodeName: [i("html"), i("body"), i("input"), i("input"), i("input"), i("input")],
|
||||
backendNodeId: [10, 11, 12, 13, 14, 15],
|
||||
attributes: [
|
||||
[],
|
||||
[],
|
||||
[i("type"), i("text"), i("value"), i(secrets.email)],
|
||||
[i("type"), i("password"), i("value"), i(secrets.password)],
|
||||
[
|
||||
i("type"),
|
||||
i("text"),
|
||||
i("autocomplete"),
|
||||
i("one-time-code"),
|
||||
i("value"),
|
||||
i(secrets.otp),
|
||||
],
|
||||
[
|
||||
i("type"),
|
||||
i("text"),
|
||||
i("autocomplete"),
|
||||
i("cc-number"),
|
||||
i("value"),
|
||||
i(secrets.card),
|
||||
],
|
||||
],
|
||||
inputValue: {
|
||||
index: [2, 3, 4, 5],
|
||||
value: [i(secrets.email), i(secrets.password), i(secrets.otp), i(secrets.card)],
|
||||
},
|
||||
},
|
||||
layout: {
|
||||
nodeIndex: [1, 2, 3, 4, 5],
|
||||
styles: [style, style, style, style, style],
|
||||
bounds: [
|
||||
[0, 0, 1000, 800],
|
||||
[10, 10, 200, 32],
|
||||
[10, 50, 200, 32],
|
||||
[10, 90, 200, 32],
|
||||
[10, 130, 200, 32],
|
||||
],
|
||||
paintOrders: [0, 1, 2, 3, 4],
|
||||
},
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
const VP_METRICS = {
|
||||
cssLayoutViewport: { clientWidth: 1000, clientHeight: 800, pageX: 0, pageY: 0 },
|
||||
};
|
||||
|
||||
function makeFrameAwareDeps() {
|
||||
const strings = ["html", "body", "iframe", "button", "title", "Remote", "static", "auto"];
|
||||
const i = (value: string) => strings.indexOf(value);
|
||||
const styles = [i("static"), i("auto"), i("auto")];
|
||||
const snapshot = {
|
||||
strings,
|
||||
documents: [
|
||||
{
|
||||
frameId: "main",
|
||||
nodes: {
|
||||
parentIndex: [-1, 0, 1],
|
||||
nodeName: [i("html"), i("body"), i("iframe")],
|
||||
backendNodeId: [10, 11, 12],
|
||||
attributes: [[], [], [i("title"), i("Remote")]],
|
||||
contentDocumentIndex: { index: [2], value: [1] },
|
||||
},
|
||||
layout: {
|
||||
nodeIndex: [0, 1, 2],
|
||||
styles: [styles, styles, styles],
|
||||
bounds: [
|
||||
[0, 0, 1000, 800],
|
||||
[0, 0, 1000, 800],
|
||||
[100, 100, 400, 300],
|
||||
],
|
||||
paintOrders: [0, 0, 1],
|
||||
},
|
||||
},
|
||||
{
|
||||
frameId: "child",
|
||||
nodes: {
|
||||
parentIndex: [-1, 0, 1],
|
||||
nodeName: [i("html"), i("body"), i("button")],
|
||||
backendNodeId: [20, 21, 22],
|
||||
attributes: [[], [], []],
|
||||
},
|
||||
layout: {
|
||||
nodeIndex: [0, 1, 2],
|
||||
styles: [styles, styles, styles],
|
||||
bounds: [
|
||||
[0, 0, 400, 300],
|
||||
[0, 0, 400, 300],
|
||||
[20, 30, 120, 40],
|
||||
],
|
||||
paintOrders: [0, 0, 1],
|
||||
},
|
||||
},
|
||||
],
|
||||
};
|
||||
const mainAx: CdpAxNode[] = [
|
||||
{
|
||||
nodeId: "main-root",
|
||||
role: { type: "role", value: "RootWebArea" },
|
||||
backendDOMNodeId: 11,
|
||||
childIds: ["frame-owner"],
|
||||
},
|
||||
{
|
||||
nodeId: "frame-owner",
|
||||
parentId: "main-root",
|
||||
role: { type: "role", value: "Iframe" },
|
||||
name: { type: "computedString", value: "Remote" },
|
||||
backendDOMNodeId: 12,
|
||||
},
|
||||
];
|
||||
const childAx: CdpAxNode[] = [
|
||||
{
|
||||
nodeId: "child-root",
|
||||
role: { type: "role", value: "RootWebArea" },
|
||||
backendDOMNodeId: 21,
|
||||
childIds: ["child-button"],
|
||||
},
|
||||
{
|
||||
nodeId: "child-button",
|
||||
parentId: "child-root",
|
||||
role: { type: "role", value: "button" },
|
||||
name: { type: "computedString", value: "Frame action" },
|
||||
backendDOMNodeId: 22,
|
||||
},
|
||||
];
|
||||
const send = vi.fn(async (_tabId: number, method: string) => {
|
||||
if (method === "Page.getLayoutMetrics") return VP_METRICS;
|
||||
if (method === "DOMSnapshot.enable" || method === "Accessibility.enable") return {};
|
||||
if (method === "DOMSnapshot.captureSnapshot") return snapshot;
|
||||
if (method === "Accessibility.getFullAXTree") return { nodes: mainAx };
|
||||
throw new Error(`unexpected root CDP method ${method}`);
|
||||
});
|
||||
const sendToTarget = vi.fn(async (_target, method: string) => {
|
||||
if (method === "Accessibility.enable") return {};
|
||||
if (method === "Accessibility.getFullAXTree") return { nodes: childAx };
|
||||
throw new Error(`unexpected child CDP method ${method}`);
|
||||
});
|
||||
const cdp = {
|
||||
send: send as unknown as CdpRunner["send"],
|
||||
sendToTarget: sendToTarget as unknown as NonNullable<CdpRunner["sendToTarget"]>,
|
||||
getFrameGraph: vi.fn(async () => ({
|
||||
rootFrameId: "main",
|
||||
frames: [
|
||||
{ frameId: "main", target: { tabId: 4 } },
|
||||
{
|
||||
frameId: "child",
|
||||
parentFrameId: "main",
|
||||
ownerBackendNodeId: 12,
|
||||
target: { tabId: 4, sessionId: "child-session" },
|
||||
},
|
||||
],
|
||||
})),
|
||||
trackSessionTab: vi.fn(),
|
||||
} satisfies CdpRunner;
|
||||
return {
|
||||
cdp,
|
||||
tabsApi: {
|
||||
get: vi.fn(
|
||||
async (tabId: number) => ({ id: tabId, windowId: 100, active: true }) as chrome.tabs.Tab,
|
||||
),
|
||||
query: vi.fn(async () => [{ id: 4, windowId: 100, active: true } as chrome.tabs.Tab]),
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
it("exposes the frame-aware observation result for recording", async () => {
|
||||
const ax: CdpAxNode[] = [
|
||||
{
|
||||
nodeId: "root",
|
||||
role: { type: "role", value: "RootWebArea" },
|
||||
backendDOMNodeId: 11,
|
||||
childIds: ["password"],
|
||||
},
|
||||
{
|
||||
nodeId: "password",
|
||||
parentId: "root",
|
||||
role: { type: "role", value: "textbox" },
|
||||
name: { type: "computedString", value: "Password" },
|
||||
value: { value: "hunter2" },
|
||||
backendDOMNodeId: 13,
|
||||
},
|
||||
];
|
||||
const deps = makeOverlayDeps(ax, loginSnapshotReply(), VP_METRICS);
|
||||
|
||||
const result = await captureVomObservation(deps.cdp, 4, "https://example.com", {
|
||||
redactValues: true,
|
||||
});
|
||||
|
||||
expect(result.text).not.toContain("hunter2");
|
||||
expect(result.text).toContain("•••");
|
||||
expect(result.refs[0]).toMatchObject({
|
||||
backendNodeId: 13,
|
||||
role: "textbox",
|
||||
name: "Password",
|
||||
line: expect.any(Number),
|
||||
});
|
||||
expect(result.rootFrameId).toBe("root");
|
||||
expect(result).not.toHaveProperty("frameDocuments");
|
||||
expect(result.frames).toEqual([{ frameId: "root", target: { tabId: 4 } }]);
|
||||
expect(result.matchNodes.find((node) => node.backendNodeId === 13)).toMatchObject({
|
||||
frameId: "root",
|
||||
backendNodeId: 13,
|
||||
tag: "input",
|
||||
rect: { x: 400, y: 300, w: 200, h: 40 },
|
||||
localRect: { x: 400, y: 300, w: 200, h: 40 },
|
||||
});
|
||||
});
|
||||
|
||||
it("does not leak form secrets through the record-safe observation payload", async () => {
|
||||
const secrets = {
|
||||
email: "user@example.com",
|
||||
password: "hunter2",
|
||||
otp: "847291",
|
||||
card: "4111111111111111",
|
||||
};
|
||||
const ax: CdpAxNode[] = [
|
||||
{
|
||||
nodeId: "root",
|
||||
role: { type: "role", value: "RootWebArea" },
|
||||
backendDOMNodeId: 11,
|
||||
childIds: ["email", "password", "otp", "card"],
|
||||
},
|
||||
{
|
||||
nodeId: "email",
|
||||
parentId: "root",
|
||||
role: { type: "role", value: "textbox" },
|
||||
name: { type: "computedString", value: "Email" },
|
||||
value: { value: secrets.email },
|
||||
backendDOMNodeId: 12,
|
||||
},
|
||||
{
|
||||
nodeId: "password",
|
||||
parentId: "root",
|
||||
role: { type: "role", value: "textbox" },
|
||||
name: { type: "computedString", value: "Password" },
|
||||
value: { value: secrets.password },
|
||||
backendDOMNodeId: 13,
|
||||
},
|
||||
{
|
||||
nodeId: "otp",
|
||||
parentId: "root",
|
||||
role: { type: "role", value: "textbox" },
|
||||
name: { type: "computedString", value: "Code" },
|
||||
value: { value: secrets.otp },
|
||||
backendDOMNodeId: 14,
|
||||
},
|
||||
{
|
||||
nodeId: "card",
|
||||
parentId: "root",
|
||||
role: { type: "role", value: "textbox" },
|
||||
name: { type: "computedString", value: "Card" },
|
||||
value: { value: secrets.card },
|
||||
backendDOMNodeId: 15,
|
||||
},
|
||||
];
|
||||
const deps = makeOverlayDeps(ax, secretsSnapshotReply(secrets), VP_METRICS);
|
||||
|
||||
const result = await captureVomObservation(deps.cdp, 4, "https://example.com", {
|
||||
redactValues: true,
|
||||
});
|
||||
const dumped = JSON.stringify(result);
|
||||
|
||||
expect(result).not.toHaveProperty("frameDocuments");
|
||||
expect(dumped).not.toContain(secrets.email);
|
||||
expect(dumped).not.toContain(secrets.password);
|
||||
expect(dumped).not.toContain(secrets.otp);
|
||||
expect(dumped).not.toContain(secrets.card);
|
||||
expect(dumped).not.toContain("formValue");
|
||||
expect(dumped).not.toContain("formDefaultValue");
|
||||
expect(dumped).not.toContain("axNodes");
|
||||
expect(dumped).not.toContain("domNodes");
|
||||
expect(dumped).not.toContain("attrs");
|
||||
expect(result.text).toContain("•••");
|
||||
});
|
||||
|
||||
it("keeps frames and rendered refs on the same frame identity", async () => {
|
||||
const deps = makeFrameAwareDeps();
|
||||
|
||||
const result = await captureVomObservation(deps.cdp, 4, "https://example.com");
|
||||
|
||||
expect(result.rootFrameId).toBe("main");
|
||||
expect(result.frames).toHaveLength(2);
|
||||
expect(result.frames.find((frame) => frame.frameId === "child")).toEqual({
|
||||
frameId: "child",
|
||||
parentFrameId: "main",
|
||||
ownerBackendNodeId: 12,
|
||||
target: { tabId: 4, sessionId: "child-session" },
|
||||
});
|
||||
expect(result.matchNodes.find((node) => node.backendNodeId === 22)).toMatchObject({
|
||||
frameId: "child",
|
||||
tag: "button",
|
||||
localRect: { x: 20, y: 30, w: 120, h: 40 },
|
||||
});
|
||||
expect(result.refs.find((ref) => ref.backendNodeId === 22)).toMatchObject({
|
||||
frameId: "child",
|
||||
name: "Frame action",
|
||||
});
|
||||
});
|
||||
|
||||
it("stores iframe refs with their owning CDP session", async () => {
|
||||
const sm = new SessionManager({ agentWindow: fakeAgentWindow([100]) });
|
||||
const ctx = await sm.start("aa11");
|
||||
const deps = makeFrameAwareDeps();
|
||||
|
||||
const result = await handleSnapshot(sm, { session_id: "aa11" }, deps);
|
||||
|
||||
if ("code" in result) throw new Error(`unexpected error: ${JSON.stringify(result)}`);
|
||||
expect(result.text).toContain('@e1 button "Frame action"');
|
||||
const frameRef = [...ctx.refStore.entries()].find(([, entry]) => entry.backendNodeId === 22);
|
||||
expect(frameRef?.[1]).toMatchObject({
|
||||
tabId: 4,
|
||||
frameId: "child",
|
||||
cdpSessionId: "child-session",
|
||||
});
|
||||
});
|
||||
|
||||
it("renders a blocking login overlay as the focused top layer", async () => {
|
||||
const sm = new SessionManager({ agentWindow: fakeAgentWindow([100]) });
|
||||
await sm.start("aa11");
|
||||
|
||||
@@ -66,52 +66,6 @@ function makeApis(
|
||||
}
|
||||
|
||||
describe("handleSessionStop with auto-return", () => {
|
||||
it("leaves the session untouched when cancellation arrived before teardown", async () => {
|
||||
const aw = fakeAgentWindow([100]);
|
||||
const sm = new SessionManager({ agentWindow: aw });
|
||||
await sm.start("aa11");
|
||||
const cdp = { detachSession: vi.fn(async () => {}) };
|
||||
const controller = new AbortController();
|
||||
controller.abort();
|
||||
|
||||
const res = await handleSessionStop(
|
||||
sm,
|
||||
{ session_id: "aa11" },
|
||||
{ cdp, signal: controller.signal },
|
||||
);
|
||||
|
||||
expect(res).toMatchObject({ code: "cancelled" });
|
||||
expect(sm.has("aa11")).toBe(true);
|
||||
expect(cdp.detachSession).not.toHaveBeenCalled();
|
||||
expect(aw.remove).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("does not close the Agent Window when cancellation lands during CDP detach", async () => {
|
||||
const aw = fakeAgentWindow([100]);
|
||||
const sm = new SessionManager({ agentWindow: aw });
|
||||
await sm.start("aa11");
|
||||
const controller = new AbortController();
|
||||
let finishDetach: () => void = () => {};
|
||||
const detach = new Promise<void>((resolve) => {
|
||||
finishDetach = resolve;
|
||||
});
|
||||
const cdp = { detachSession: vi.fn(() => detach) };
|
||||
|
||||
const stopping = handleSessionStop(
|
||||
sm,
|
||||
{ session_id: "aa11" },
|
||||
{ cdp, signal: controller.signal },
|
||||
);
|
||||
await vi.waitFor(() => expect(cdp.detachSession).toHaveBeenCalledWith("aa11"));
|
||||
controller.abort();
|
||||
finishDetach();
|
||||
|
||||
const res = await stopping;
|
||||
expect(res).toMatchObject({ code: "cancelled" });
|
||||
expect(sm.has("aa11")).toBe(true);
|
||||
expect(aw.remove).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("returns every borrowed tab and closes the Agent Window in the right order", async () => {
|
||||
const aw = fakeAgentWindow([100]);
|
||||
const sm = new SessionManager({ agentWindow: aw });
|
||||
|
||||
@@ -1,9 +0,0 @@
|
||||
export {
|
||||
type CaptureVomObservationOptions,
|
||||
captureVomObservation,
|
||||
} from "./observation";
|
||||
export {
|
||||
type CaptureVomFrame,
|
||||
type CaptureVomMatchNode,
|
||||
type CaptureVomObservationResult,
|
||||
} from "./vom/record-safe-observation";
|
||||
@@ -4,6 +4,7 @@ import type { Transport } from "@/transport/transport";
|
||||
import type {
|
||||
ClickParams,
|
||||
ConsoleParams,
|
||||
DownloadParams,
|
||||
EmulateParams,
|
||||
EvaluateParams,
|
||||
FillParams,
|
||||
@@ -28,10 +29,12 @@ import type {
|
||||
ScreenshotParams,
|
||||
SelectParams,
|
||||
SnapshotParams,
|
||||
UploadParams,
|
||||
WaitForNavigationParams,
|
||||
} from "@/transport/types";
|
||||
import { isRequestFrame } from "@/transport/types";
|
||||
import { handleConsole } from "./console";
|
||||
import { handleDownload } from "./download";
|
||||
import { type EmulateCdpRunner, handleEmulate } from "./emulate";
|
||||
import { handleEvaluate } from "./evaluate";
|
||||
import { handleRequestHelp } from "./human-loop";
|
||||
@@ -74,6 +77,7 @@ import {
|
||||
type TabReturnParams,
|
||||
type TabSelectParams,
|
||||
} from "./tabs";
|
||||
import { handleUpload } from "./upload";
|
||||
import { handleWaitForNavigation } from "./waits";
|
||||
import { handleWindowResize, type WindowResizeParams } from "./window";
|
||||
|
||||
@@ -290,7 +294,6 @@ export class ToolDispatcher {
|
||||
await this.releaseHoverLatch((req.params as SessionStopParams).session_id);
|
||||
return handleSessionStop(this.sessions, req.params as SessionStopParams, {
|
||||
cdp: this.cdp,
|
||||
signal,
|
||||
});
|
||||
}
|
||||
case "tool.tab_list":
|
||||
@@ -505,6 +508,40 @@ export class ToolDispatcher {
|
||||
),
|
||||
signal,
|
||||
);
|
||||
case "tool.upload":
|
||||
return this.withHoverReleaseForRequest(
|
||||
req.params as UploadParams,
|
||||
() =>
|
||||
this.cdp
|
||||
? handleUpload(this.sessions, req.params as UploadParams, {
|
||||
cdp: this.cdp,
|
||||
tabsApi: chromeTabsApi,
|
||||
signal,
|
||||
bypassOverlay,
|
||||
})
|
||||
: Promise.resolve({
|
||||
code: "unsupported",
|
||||
message: "upload requires CDP",
|
||||
} satisfies RpcError),
|
||||
signal,
|
||||
);
|
||||
case "tool.download":
|
||||
return this.withHoverReleaseForRequest(
|
||||
req.params as DownloadParams,
|
||||
() =>
|
||||
this.cdp
|
||||
? handleDownload(this.sessions, req.params as DownloadParams, {
|
||||
cdp: this.cdp,
|
||||
tabsApi: chromeTabsApi,
|
||||
signal,
|
||||
bypassOverlay,
|
||||
})
|
||||
: Promise.resolve({
|
||||
code: "unsupported",
|
||||
message: "download requires CDP",
|
||||
} satisfies RpcError),
|
||||
signal,
|
||||
);
|
||||
case "tool.evaluate":
|
||||
return handleEvaluate(
|
||||
this.sessions,
|
||||
@@ -722,6 +759,8 @@ function sessionIdForBrowserControlMethod(req: RequestFrame): string | null {
|
||||
case "tool.fill":
|
||||
case "tool.press":
|
||||
case "tool.select":
|
||||
case "tool.upload":
|
||||
case "tool.download":
|
||||
case "tool.evaluate":
|
||||
case "tool.observe":
|
||||
case "tool.request_help":
|
||||
|
||||
@@ -0,0 +1,349 @@
|
||||
// Order-independent coordinator for one browser download. CDP supplies the
|
||||
// exact target/frame intent while chrome.downloads supplies the download id
|
||||
// and filename routing hook; neither event is assumed to arrive first.
|
||||
|
||||
import type { CdpTarget } from "@/browser-driver/frame-graph";
|
||||
import type { ClickResult, RpcError, TransferEffectState } from "@/transport/types";
|
||||
import { transferError } from "./errors";
|
||||
import { type CdpRunner, isRpcError } from "./shared";
|
||||
|
||||
const CORRELATION_GRACE_MS = 750;
|
||||
const UNIQUE_SETTLE_MS = 50;
|
||||
const SIZE_POLL_MS = 250;
|
||||
|
||||
type DeterminingFilenameListener = (
|
||||
item: chrome.downloads.DownloadItem,
|
||||
suggest: (suggestion?: chrome.downloads.DownloadFilenameSuggestion) => void,
|
||||
) => void | true;
|
||||
|
||||
interface ListenerEvent<T> {
|
||||
addListener(listener: T): void;
|
||||
removeListener(listener: T): void;
|
||||
}
|
||||
|
||||
export interface DownloadsApi {
|
||||
onCreated: ListenerEvent<(item: chrome.downloads.DownloadItem) => void>;
|
||||
onChanged: ListenerEvent<(delta: chrome.downloads.DownloadDelta) => void>;
|
||||
onDeterminingFilename: ListenerEvent<DeterminingFilenameListener>;
|
||||
search(query: chrome.downloads.DownloadQuery): Promise<chrome.downloads.DownloadItem[]>;
|
||||
cancel(downloadId: number): Promise<void>;
|
||||
removeFile(downloadId: number): Promise<void>;
|
||||
}
|
||||
|
||||
export const chromeDownloadsApi: DownloadsApi = {
|
||||
get onCreated() {
|
||||
return chrome.downloads.onCreated;
|
||||
},
|
||||
get onChanged() {
|
||||
return chrome.downloads.onChanged;
|
||||
},
|
||||
get onDeterminingFilename() {
|
||||
return chrome.downloads.onDeterminingFilename;
|
||||
},
|
||||
search: (query) => chrome.downloads.search(query),
|
||||
cancel: (id) => chrome.downloads.cancel(id),
|
||||
removeFile: (id) => chrome.downloads.removeFile(id),
|
||||
};
|
||||
|
||||
export interface DownloadCaptureOptions {
|
||||
cdp: CdpRunner;
|
||||
target: CdpTarget;
|
||||
expectedFrameId?: string;
|
||||
downloads: DownloadsApi;
|
||||
browserRelativeDir: string;
|
||||
maxByteSize?: number;
|
||||
timeoutMs: number;
|
||||
signal?: AbortSignal;
|
||||
trigger(): Promise<ClickResult | RpcError>;
|
||||
}
|
||||
|
||||
export interface DownloadCaptureResult {
|
||||
click: ClickResult;
|
||||
item: chrome.downloads.DownloadItem;
|
||||
}
|
||||
|
||||
interface DownloadIntent {
|
||||
url: string;
|
||||
suggestedFilename: string;
|
||||
frameId?: string;
|
||||
}
|
||||
|
||||
interface DownloadCandidate {
|
||||
item: chrome.downloads.DownloadItem;
|
||||
suggest: (suggestion?: chrome.downloads.DownloadFilenameSuggestion) => void;
|
||||
suggested: boolean;
|
||||
graceTimer: ReturnType<typeof setTimeout>;
|
||||
}
|
||||
|
||||
function safeBasename(filename: string): string {
|
||||
const basename = filename.split(/[\\/]/).pop()?.trim();
|
||||
return basename && basename !== "." && basename !== ".." ? basename : "download";
|
||||
}
|
||||
|
||||
function sameTarget(source: { tabId?: number; sessionId?: string }, target: CdpTarget): boolean {
|
||||
return source.tabId === target.tabId && source.sessionId === target.sessionId;
|
||||
}
|
||||
|
||||
function matchesIntent(item: chrome.downloads.DownloadItem, intent: DownloadIntent): boolean {
|
||||
const urlMatches = item.url === intent.url || item.finalUrl === intent.url;
|
||||
return urlMatches && safeBasename(item.filename) === safeBasename(intent.suggestedFilename);
|
||||
}
|
||||
|
||||
function knownSize(item: chrome.downloads.DownloadItem): number | undefined {
|
||||
if (item.fileSize >= 0) return item.fileSize;
|
||||
if (item.totalBytes >= 0) return item.totalBytes;
|
||||
return undefined;
|
||||
}
|
||||
|
||||
function captureError(
|
||||
message: string,
|
||||
effectState: TransferEffectState,
|
||||
phase: string,
|
||||
cleanupFailed = false,
|
||||
): RpcError {
|
||||
return transferError("cdp_failed", "download_capture_failed", message, {
|
||||
effectState,
|
||||
phase,
|
||||
...(cleanupFailed ? { cleanupState: "failed" } : {}),
|
||||
});
|
||||
}
|
||||
|
||||
export async function captureBrowserDownload(
|
||||
options: DownloadCaptureOptions,
|
||||
): Promise<DownloadCaptureResult | RpcError> {
|
||||
let click: ClickResult | undefined;
|
||||
let intent: DownloadIntent | undefined;
|
||||
let capturedId: number | undefined;
|
||||
let capturedItem: chrome.downloads.DownloadItem | undefined;
|
||||
let settled = false;
|
||||
let succeeded = false;
|
||||
let failureResult: RpcError | undefined;
|
||||
let uniquenessTimer: ReturnType<typeof setTimeout> | undefined;
|
||||
let operationTimer: ReturnType<typeof setTimeout> | undefined;
|
||||
let sizePoll: ReturnType<typeof setInterval> | undefined;
|
||||
const candidates = new Map<number, DownloadCandidate>();
|
||||
const createdItems = new Map<number, chrome.downloads.DownloadItem>();
|
||||
|
||||
let resolveCompletion!: (item: chrome.downloads.DownloadItem) => void;
|
||||
let rejectCompletion!: (error: Error) => void;
|
||||
const completion = new Promise<chrome.downloads.DownloadItem>((resolve, reject) => {
|
||||
resolveCompletion = resolve;
|
||||
rejectCompletion = reject;
|
||||
});
|
||||
const fail = (error: Error) => {
|
||||
if (settled) return;
|
||||
settled = true;
|
||||
rejectCompletion(error);
|
||||
};
|
||||
const complete = (item: chrome.downloads.DownloadItem) => {
|
||||
if (settled) return;
|
||||
const size = knownSize(item);
|
||||
if (size !== undefined && options.maxByteSize !== undefined && size > options.maxByteSize) {
|
||||
fail(new Error(`download exceeds transfer limit ${options.maxByteSize}`));
|
||||
return;
|
||||
}
|
||||
settled = true;
|
||||
capturedItem = item;
|
||||
resolveCompletion(item);
|
||||
};
|
||||
const suggestDefault = (candidate: DownloadCandidate) => {
|
||||
if (candidate.suggested) return;
|
||||
candidate.suggested = true;
|
||||
candidate.suggest();
|
||||
};
|
||||
const matchingCandidates = (): DownloadCandidate[] => {
|
||||
const currentIntent = intent;
|
||||
return currentIntent
|
||||
? [...candidates.values()].filter(
|
||||
(candidate) => !candidate.suggested && matchesIntent(candidate.item, currentIntent),
|
||||
)
|
||||
: [];
|
||||
};
|
||||
|
||||
const claimUnique = () => {
|
||||
uniquenessTimer = undefined;
|
||||
if (settled || capturedId !== undefined || !intent) return;
|
||||
const matches = matchingCandidates();
|
||||
if (matches.length !== 1) {
|
||||
if (matches.length > 1) {
|
||||
for (const candidate of matches) suggestDefault(candidate);
|
||||
fail(new Error("download attribution is ambiguous"));
|
||||
}
|
||||
return;
|
||||
}
|
||||
const candidate = matches[0];
|
||||
candidate.suggested = true;
|
||||
clearTimeout(candidate.graceTimer);
|
||||
capturedId = candidate.item.id;
|
||||
capturedItem = candidate.item;
|
||||
candidate.suggest({
|
||||
filename: `${options.browserRelativeDir}/${safeBasename(intent.suggestedFilename)}`,
|
||||
conflictAction: "overwrite",
|
||||
});
|
||||
const size = knownSize(candidate.item);
|
||||
if (size !== undefined && options.maxByteSize !== undefined && size > options.maxByteSize) {
|
||||
fail(new Error(`download exceeds transfer limit ${options.maxByteSize}`));
|
||||
return;
|
||||
}
|
||||
const created = createdItems.get(candidate.item.id);
|
||||
if (created?.state === "interrupted") {
|
||||
fail(new Error(created.error ?? "download interrupted"));
|
||||
} else if (created?.state === "complete") {
|
||||
complete(created);
|
||||
}
|
||||
};
|
||||
const reconcile = () => {
|
||||
if (settled || capturedId !== undefined || !intent) return;
|
||||
const matches = matchingCandidates();
|
||||
if (matches.length > 1) {
|
||||
for (const candidate of matches) suggestDefault(candidate);
|
||||
fail(new Error("download attribution is ambiguous"));
|
||||
return;
|
||||
}
|
||||
if (matches.length === 1 && !uniquenessTimer) {
|
||||
uniquenessTimer = setTimeout(claimUnique, UNIQUE_SETTLE_MS);
|
||||
}
|
||||
};
|
||||
|
||||
const determiningListener: DeterminingFilenameListener = (item, suggest) => {
|
||||
const candidate: DownloadCandidate = {
|
||||
item,
|
||||
suggest,
|
||||
suggested: false,
|
||||
graceTimer: setTimeout(() => {
|
||||
suggestDefault(candidate);
|
||||
candidates.delete(item.id);
|
||||
if (intent && matchesIntent(item, intent) && capturedId === undefined) {
|
||||
fail(new Error("download correlation grace elapsed before unique attribution"));
|
||||
}
|
||||
}, CORRELATION_GRACE_MS),
|
||||
};
|
||||
candidates.set(item.id, candidate);
|
||||
reconcile();
|
||||
return true;
|
||||
};
|
||||
const createdListener = (item: chrome.downloads.DownloadItem) => {
|
||||
createdItems.set(item.id, item);
|
||||
if (capturedId !== item.id) return;
|
||||
capturedItem = item;
|
||||
if (item.state === "interrupted") {
|
||||
fail(new Error(item.error ?? "download interrupted"));
|
||||
} else if (item.state === "complete") {
|
||||
complete(item);
|
||||
}
|
||||
};
|
||||
const changedListener = async (delta: chrome.downloads.DownloadDelta) => {
|
||||
if (capturedId === undefined || delta.id !== capturedId || settled) return;
|
||||
if (delta.state?.current === "interrupted" || delta.error?.current) {
|
||||
fail(new Error(delta.error?.current ?? "download interrupted"));
|
||||
return;
|
||||
}
|
||||
if (delta.state?.current === "complete") {
|
||||
try {
|
||||
const [item] = await options.downloads.search({ id: delta.id });
|
||||
if (item) complete(item);
|
||||
else fail(new Error("completed download disappeared"));
|
||||
} catch (err) {
|
||||
fail(err instanceof Error ? err : new Error(String(err)));
|
||||
}
|
||||
}
|
||||
};
|
||||
const onAbort = () => fail(new DOMException("aborted", "AbortError"));
|
||||
const cdpSubscription = options.cdp.onEvent?.((source, method, raw) => {
|
||||
if (method !== "Page.downloadWillBegin" || !sameTarget(source, options.target)) return;
|
||||
const event = raw as { url?: unknown; suggestedFilename?: unknown; frameId?: unknown };
|
||||
if (typeof event.url !== "string" || typeof event.suggestedFilename !== "string") return;
|
||||
if (options.expectedFrameId && event.frameId !== options.expectedFrameId) {
|
||||
fail(new Error("download originated from a different frame"));
|
||||
return;
|
||||
}
|
||||
if (intent) {
|
||||
fail(new Error("download trigger produced more than one browser download intent"));
|
||||
return;
|
||||
}
|
||||
intent = {
|
||||
url: event.url,
|
||||
suggestedFilename: event.suggestedFilename,
|
||||
...(typeof event.frameId === "string" ? { frameId: event.frameId } : {}),
|
||||
};
|
||||
reconcile();
|
||||
});
|
||||
if (!cdpSubscription) {
|
||||
return captureError("CDP download intent subscription unavailable", "none", "arm");
|
||||
}
|
||||
|
||||
options.downloads.onDeterminingFilename.addListener(determiningListener);
|
||||
options.downloads.onCreated.addListener(createdListener);
|
||||
options.downloads.onChanged.addListener(changedListener);
|
||||
options.signal?.addEventListener("abort", onAbort, { once: true });
|
||||
operationTimer = setTimeout(
|
||||
() => fail(new Error("download did not complete before timeout")),
|
||||
options.timeoutMs,
|
||||
);
|
||||
sizePoll = setInterval(() => {
|
||||
if (capturedId === undefined || settled || options.maxByteSize === undefined) return;
|
||||
void options.downloads
|
||||
.search({ id: capturedId })
|
||||
.then(([item]) => {
|
||||
if (!item || settled) return;
|
||||
capturedItem = item;
|
||||
if (item.bytesReceived > (options.maxByteSize as number)) {
|
||||
fail(new Error(`download exceeds transfer limit ${options.maxByteSize}`));
|
||||
}
|
||||
})
|
||||
.catch((err) => fail(err instanceof Error ? err : new Error(String(err))));
|
||||
}, SIZE_POLL_MS);
|
||||
|
||||
try {
|
||||
const triggered = await options.trigger();
|
||||
if (isRpcError(triggered)) {
|
||||
void completion.catch(() => undefined);
|
||||
const effect: TransferEffectState =
|
||||
capturedId !== undefined ? "committed" : intent ? "unknown" : "none";
|
||||
failureResult = {
|
||||
...triggered,
|
||||
data: { ...triggered.data, effect_state: effect, phase: "trigger" },
|
||||
};
|
||||
return failureResult;
|
||||
}
|
||||
click = triggered;
|
||||
const item = await completion;
|
||||
succeeded = true;
|
||||
return { click, item };
|
||||
} catch (err) {
|
||||
const effect: TransferEffectState =
|
||||
capturedId !== undefined ? "committed" : click ? "unknown" : "none";
|
||||
failureResult = captureError(
|
||||
err instanceof Error ? err.message : String(err),
|
||||
effect,
|
||||
capturedId !== undefined ? "download" : "attribution",
|
||||
);
|
||||
return failureResult;
|
||||
} finally {
|
||||
settled = true;
|
||||
if (operationTimer) clearTimeout(operationTimer);
|
||||
if (uniquenessTimer) clearTimeout(uniquenessTimer);
|
||||
if (sizePoll) clearInterval(sizePoll);
|
||||
options.signal?.removeEventListener("abort", onAbort);
|
||||
options.downloads.onDeterminingFilename.removeListener(determiningListener);
|
||||
options.downloads.onCreated.removeListener(createdListener);
|
||||
options.downloads.onChanged.removeListener(changedListener);
|
||||
cdpSubscription.dispose();
|
||||
for (const candidate of candidates.values()) {
|
||||
clearTimeout(candidate.graceTimer);
|
||||
if (candidate.item.id !== capturedId) suggestDefault(candidate);
|
||||
}
|
||||
if (!succeeded && capturedId !== undefined) {
|
||||
const item = capturedItem ?? createdItems.get(capturedId);
|
||||
try {
|
||||
if (item?.state === "complete") {
|
||||
await options.downloads.removeFile(capturedId);
|
||||
} else {
|
||||
await options.downloads.cancel(capturedId);
|
||||
}
|
||||
} catch {
|
||||
if (failureResult?.data) failureResult.data.cleanup_state = "failed";
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,64 @@
|
||||
// Download orchestration: validate session/tab ownership, then delegate one
|
||||
// browser-global chrome.downloads transaction to download-capture.ts.
|
||||
|
||||
import type { SessionManager } from "@/session-manager/manager";
|
||||
import type { DownloadParams, DownloadResult, RpcError } from "@/transport/types";
|
||||
import { captureBrowserDownload, chromeDownloadsApi, type DownloadsApi } from "./download-capture";
|
||||
import { clickResolvedTarget, type InteractionDeps, resolveActionTarget } from "./interaction";
|
||||
import { enforceAgentWindow, isRpcError, lookupSession, resolveTargetTab } from "./shared";
|
||||
|
||||
let downloadActive = false;
|
||||
|
||||
export type { DownloadsApi } from "./download-capture";
|
||||
|
||||
export interface DownloadDeps extends InteractionDeps {
|
||||
downloads?: DownloadsApi;
|
||||
}
|
||||
|
||||
export async function handleDownload(
|
||||
manager: SessionManager,
|
||||
params: DownloadParams,
|
||||
deps: DownloadDeps,
|
||||
): Promise<DownloadResult | RpcError> {
|
||||
if (downloadActive) return { code: "invalid_params", message: "another bsk download is active" };
|
||||
downloadActive = true;
|
||||
try {
|
||||
const ctx = lookupSession(manager, params, "download");
|
||||
if (isRpcError(ctx)) return ctx;
|
||||
const target = await resolveTargetTab(manager, ctx, params.tab_id, deps.tabsApi);
|
||||
if (isRpcError(target)) return target;
|
||||
const denied = enforceAgentWindow(ctx, target, "download");
|
||||
if (denied) return denied;
|
||||
if (!params.browser_relative_dir) {
|
||||
return { code: "invalid_params", message: "download requires a daemon capability directory" };
|
||||
}
|
||||
const address = await resolveActionTarget(deps.cdp, ctx, target, params, "download");
|
||||
if (isRpcError(address)) return address;
|
||||
|
||||
const capture = await captureBrowserDownload({
|
||||
cdp: deps.cdp,
|
||||
target: address.cdpTarget,
|
||||
downloads: deps.downloads ?? chromeDownloadsApi,
|
||||
browserRelativeDir: params.browser_relative_dir,
|
||||
maxByteSize: params.max_byte_size,
|
||||
timeoutMs: params.timeout_ms ?? 120_000,
|
||||
signal: deps.signal,
|
||||
expectedFrameId: address.frameId,
|
||||
trigger: () => clickResolvedTarget(ctx, address, {}, deps),
|
||||
});
|
||||
if (isRpcError(capture)) return capture;
|
||||
const { click, item } = capture;
|
||||
return {
|
||||
tab_id: target.tabId,
|
||||
used_ref: click.used_ref,
|
||||
used_selector: click.used_selector,
|
||||
suggested_filename: item.filename.split(/[\\/]/).pop() ?? "download",
|
||||
byte_size: item.fileSize >= 0 ? item.fileSize : item.totalBytes,
|
||||
mime: item.mime || undefined,
|
||||
danger: item.danger,
|
||||
browser_path: item.filename,
|
||||
};
|
||||
} finally {
|
||||
downloadActive = false;
|
||||
}
|
||||
}
|
||||
@@ -2,10 +2,23 @@
|
||||
// rendering. Extension handlers attach reasons here; human-facing copy
|
||||
// lives in bsk-cli `render_error.rs`.
|
||||
|
||||
import type { ErrorCode, RpcError, RpcErrorData, RpcErrorReason } from "@/transport/types";
|
||||
import type {
|
||||
ErrorCode,
|
||||
RpcError,
|
||||
RpcErrorData,
|
||||
RpcErrorReason,
|
||||
TransferCleanupState,
|
||||
TransferEffectState,
|
||||
} from "@/transport/types";
|
||||
|
||||
export type { RpcErrorData, RpcErrorReason };
|
||||
|
||||
export interface TransferErrorOptions {
|
||||
effectState: TransferEffectState;
|
||||
phase: string;
|
||||
cleanupState?: TransferCleanupState;
|
||||
}
|
||||
|
||||
export function rpcError(
|
||||
code: ErrorCode,
|
||||
reason: RpcErrorReason,
|
||||
@@ -15,3 +28,16 @@ export function rpcError(
|
||||
const data: RpcErrorData = { reason, ...extra };
|
||||
return { code, message, data };
|
||||
}
|
||||
|
||||
export function transferError(
|
||||
code: ErrorCode,
|
||||
reason: RpcErrorReason,
|
||||
message: string,
|
||||
options: TransferErrorOptions,
|
||||
): RpcError {
|
||||
return rpcError(code, reason, message, {
|
||||
effect_state: options.effectState,
|
||||
phase: options.phase,
|
||||
...(options.cleanupState ? { cleanup_state: options.cleanupState } : {}),
|
||||
});
|
||||
}
|
||||
|
||||
@@ -0,0 +1,484 @@
|
||||
// One upload transaction with an explicit browser-side commit boundary.
|
||||
// Chrome interception prevents a native chooser from escaping automation;
|
||||
// chooser events and a frame-scoped DOM probe are independent input-location
|
||||
// signals, so event delivery is not required for standard file inputs.
|
||||
|
||||
import type { ClickResult, RpcError, TransferEffectState } from "@/transport/types";
|
||||
import { transferError } from "./errors";
|
||||
import type { ResolvedActionTarget } from "./interaction";
|
||||
import { type CdpRunner, isRpcError, sendToCdpTarget } from "./shared";
|
||||
|
||||
const CLEANUP_TIMEOUT_MS = 1_000;
|
||||
const ACTIVATION_GRACE_MS = 1_000;
|
||||
const PROBE_INTERVAL_MS = 20;
|
||||
|
||||
type UploadPhase =
|
||||
| "arm_interception"
|
||||
| "arm_input_probe"
|
||||
| "trigger"
|
||||
| "resolve_input"
|
||||
| "set_files"
|
||||
| "cleanup";
|
||||
|
||||
interface RuntimeReply {
|
||||
result?: { value?: unknown; objectId?: string };
|
||||
exceptionDetails?: { text?: string; exception?: { description?: string } };
|
||||
}
|
||||
|
||||
interface ChooserEvent {
|
||||
frameId?: string;
|
||||
backendNodeId?: number;
|
||||
mode?: "selectSingle" | "selectMultiple";
|
||||
}
|
||||
|
||||
export interface FileInputTransactionOptions {
|
||||
cdp: CdpRunner;
|
||||
actionTarget: ResolvedActionTarget;
|
||||
files: string[];
|
||||
timeoutMs: number;
|
||||
signal?: AbortSignal;
|
||||
trigger(): Promise<ClickResult | RpcError>;
|
||||
}
|
||||
|
||||
export interface FileInputTransactionResult {
|
||||
click: ClickResult;
|
||||
multiple: boolean;
|
||||
}
|
||||
|
||||
class BoundedWaitError extends Error {
|
||||
constructor(
|
||||
readonly kind: "timeout" | "aborted",
|
||||
message: string,
|
||||
) {
|
||||
super(message);
|
||||
}
|
||||
}
|
||||
|
||||
function remainingMs(deadline: number): number {
|
||||
return Math.max(0, deadline - Date.now());
|
||||
}
|
||||
|
||||
async function waitBounded<T>(
|
||||
promise: Promise<T>,
|
||||
deadline: number,
|
||||
signal: AbortSignal | undefined,
|
||||
timeoutMessage: string,
|
||||
): Promise<T> {
|
||||
const remaining = remainingMs(deadline);
|
||||
if (signal?.aborted) throw new BoundedWaitError("aborted", "upload transaction aborted");
|
||||
if (remaining === 0) throw new BoundedWaitError("timeout", timeoutMessage);
|
||||
|
||||
let timer: ReturnType<typeof setTimeout> | undefined;
|
||||
let onAbort: (() => void) | undefined;
|
||||
const boundary = new Promise<never>((_resolve, reject) => {
|
||||
timer = setTimeout(() => reject(new BoundedWaitError("timeout", timeoutMessage)), remaining);
|
||||
if (signal) {
|
||||
onAbort = () => reject(new BoundedWaitError("aborted", "upload transaction aborted"));
|
||||
signal.addEventListener("abort", onAbort, { once: true });
|
||||
}
|
||||
});
|
||||
try {
|
||||
return await Promise.race([promise, boundary]);
|
||||
} finally {
|
||||
if (timer) clearTimeout(timer);
|
||||
if (signal && onAbort) signal.removeEventListener("abort", onAbort);
|
||||
}
|
||||
}
|
||||
|
||||
function runtimeError(reply: RuntimeReply, fallback: string): Error | null {
|
||||
if (!reply.exceptionDetails) return null;
|
||||
return new Error(
|
||||
reply.exceptionDetails.exception?.description ?? reply.exceptionDetails.text ?? fallback,
|
||||
);
|
||||
}
|
||||
|
||||
function transferFailure(
|
||||
code: RpcError["code"],
|
||||
reason: "file_input_probe_failed" | "file_input_not_activated" | "set_file_input_failed",
|
||||
message: string,
|
||||
effectState: TransferEffectState,
|
||||
phase: UploadPhase,
|
||||
): RpcError {
|
||||
return transferError(code, reason, message, { effectState, phase });
|
||||
}
|
||||
|
||||
function enrichFailure(
|
||||
error: RpcError,
|
||||
effectState: TransferEffectState,
|
||||
phase: UploadPhase,
|
||||
): RpcError {
|
||||
return {
|
||||
...error,
|
||||
data: { ...error.data, effect_state: effectState, phase },
|
||||
};
|
||||
}
|
||||
|
||||
function sameTarget(
|
||||
source: { tabId?: number; sessionId?: string },
|
||||
target: { tabId: number; sessionId?: string },
|
||||
): boolean {
|
||||
return source.tabId === target.tabId && source.sessionId === target.sessionId;
|
||||
}
|
||||
|
||||
async function delay(ms: number, signal?: AbortSignal): Promise<void> {
|
||||
await waitBounded(
|
||||
new Promise((resolve) => setTimeout(resolve, ms)),
|
||||
Date.now() + ms + 1,
|
||||
signal,
|
||||
"upload activation probe timed out",
|
||||
);
|
||||
}
|
||||
|
||||
async function callOnTrigger<T = RuntimeReply>(
|
||||
options: FileInputTransactionOptions,
|
||||
objectId: string,
|
||||
functionDeclaration: string,
|
||||
args: unknown[],
|
||||
returnByValue: boolean,
|
||||
): Promise<T> {
|
||||
return sendToCdpTarget<T>(options.cdp, options.actionTarget.cdpTarget, "Runtime.callFunctionOn", {
|
||||
objectId,
|
||||
functionDeclaration,
|
||||
arguments: args.map((value) => ({ value })),
|
||||
returnByValue,
|
||||
awaitPromise: false,
|
||||
});
|
||||
}
|
||||
|
||||
export async function uploadThroughActivatedFileInput(
|
||||
options: FileInputTransactionOptions,
|
||||
): Promise<FileInputTransactionResult | RpcError> {
|
||||
const deadline = Date.now() + options.timeoutMs;
|
||||
const target = options.actionTarget.cdpTarget;
|
||||
const objectGroup = `bsk-upload-${crypto.randomUUID()}`;
|
||||
const stateKey = `__bskUpload_${crypto.randomUUID().replaceAll("-", "")}`;
|
||||
const chooserEvents: ChooserEvent[] = [];
|
||||
let interceptionArmed = false;
|
||||
let probeArmed = false;
|
||||
let triggerObjectId: string | undefined;
|
||||
let outcome: FileInputTransactionResult | RpcError = transferFailure(
|
||||
"protocol_error",
|
||||
"file_input_probe_failed",
|
||||
"upload transaction ended without an outcome",
|
||||
"none",
|
||||
"cleanup",
|
||||
);
|
||||
|
||||
const chooserSubscription = options.cdp.onEvent?.((source, method, raw) => {
|
||||
if (method !== "Page.fileChooserOpened" || !sameTarget(source, target)) return;
|
||||
const event = raw as ChooserEvent;
|
||||
chooserEvents.push(event);
|
||||
});
|
||||
|
||||
try {
|
||||
try {
|
||||
await waitBounded(
|
||||
sendToCdpTarget(options.cdp, target, "Page.setInterceptFileChooserDialog", {
|
||||
enabled: true,
|
||||
}),
|
||||
deadline,
|
||||
options.signal,
|
||||
"arming native file chooser interception timed out",
|
||||
);
|
||||
interceptionArmed = true;
|
||||
} catch (err) {
|
||||
outcome = transferFailure(
|
||||
err instanceof BoundedWaitError ? "timeout" : "cdp_failed",
|
||||
"file_input_probe_failed",
|
||||
err instanceof Error ? err.message : String(err),
|
||||
"none",
|
||||
"arm_interception",
|
||||
);
|
||||
return outcome;
|
||||
}
|
||||
|
||||
try {
|
||||
const resolved = await waitBounded(
|
||||
sendToCdpTarget<{ object?: { objectId?: string } }>(
|
||||
options.cdp,
|
||||
target,
|
||||
"DOM.resolveNode",
|
||||
{
|
||||
backendNodeId: options.actionTarget.backendNodeId,
|
||||
objectGroup,
|
||||
},
|
||||
),
|
||||
deadline,
|
||||
options.signal,
|
||||
"resolving upload trigger timed out",
|
||||
);
|
||||
triggerObjectId = resolved.object?.objectId;
|
||||
if (!triggerObjectId) throw new Error("DOM.resolveNode returned no trigger objectId");
|
||||
|
||||
const armed = await waitBounded(
|
||||
callOnTrigger<RuntimeReply>(
|
||||
options,
|
||||
triggerObjectId,
|
||||
`function(key) {
|
||||
const doc = this.ownerDocument;
|
||||
const owner = doc.defaultView;
|
||||
if (!owner) return false;
|
||||
const state = { inputs: [], listener: null };
|
||||
Object.defineProperty(owner, key, { value: state, configurable: true });
|
||||
state.listener = event => {
|
||||
const path = typeof event.composedPath === "function" ? event.composedPath() : [];
|
||||
const candidate = path[0] || event.target;
|
||||
if (candidate && candidate.nodeType === 1 &&
|
||||
candidate.localName === "input" && candidate.type === "file") {
|
||||
if (!state.inputs.includes(candidate)) state.inputs.push(candidate);
|
||||
event.preventDefault();
|
||||
}
|
||||
};
|
||||
doc.addEventListener("click", state.listener, true);
|
||||
return true;
|
||||
}`,
|
||||
[stateKey],
|
||||
true,
|
||||
),
|
||||
deadline,
|
||||
options.signal,
|
||||
"arming frame-scoped file input probe timed out",
|
||||
);
|
||||
const armError = runtimeError(armed, "failed to arm file input probe");
|
||||
if (armError || armed.result?.value !== true) {
|
||||
throw armError ?? new Error("file input probe did not arm");
|
||||
}
|
||||
probeArmed = true;
|
||||
} catch (err) {
|
||||
outcome = transferFailure(
|
||||
err instanceof BoundedWaitError ? "timeout" : "cdp_failed",
|
||||
"file_input_probe_failed",
|
||||
err instanceof Error ? err.message : String(err),
|
||||
"none",
|
||||
"arm_input_probe",
|
||||
);
|
||||
return outcome;
|
||||
}
|
||||
|
||||
let click: ClickResult | RpcError;
|
||||
try {
|
||||
click = await waitBounded(
|
||||
options.trigger(),
|
||||
deadline,
|
||||
options.signal,
|
||||
"upload trigger timed out",
|
||||
);
|
||||
} catch (err) {
|
||||
outcome = transferFailure(
|
||||
err instanceof BoundedWaitError ? "timeout" : "cdp_failed",
|
||||
"file_input_probe_failed",
|
||||
err instanceof Error ? err.message : String(err),
|
||||
"none",
|
||||
"trigger",
|
||||
);
|
||||
return outcome;
|
||||
}
|
||||
if (isRpcError(click)) {
|
||||
outcome = enrichFailure(click, "none", "trigger");
|
||||
return outcome;
|
||||
}
|
||||
|
||||
try {
|
||||
const activationDeadline = Math.min(deadline, Date.now() + ACTIVATION_GRACE_MS);
|
||||
let summary: { count: number; multiple: boolean } = { count: 0, multiple: false };
|
||||
while (Date.now() < activationDeadline) {
|
||||
if (chooserEvents.length > 0) break;
|
||||
const reply = await callOnTrigger<RuntimeReply>(
|
||||
options,
|
||||
triggerObjectId,
|
||||
`function(key) {
|
||||
const state = this.ownerDocument.defaultView?.[key];
|
||||
return state
|
||||
? { count: state.inputs.length, multiple: state.inputs[0]?.multiple === true }
|
||||
: { count: 0, multiple: false };
|
||||
}`,
|
||||
[stateKey],
|
||||
true,
|
||||
);
|
||||
const summaryError = runtimeError(reply, "failed to inspect activated file input");
|
||||
if (summaryError) throw summaryError;
|
||||
const value = reply.result?.value as { count?: unknown; multiple?: unknown } | undefined;
|
||||
summary = {
|
||||
count: typeof value?.count === "number" ? value.count : 0,
|
||||
multiple: value?.multiple === true,
|
||||
};
|
||||
if (summary.count > 0) break;
|
||||
await delay(Math.min(PROBE_INTERVAL_MS, remainingMs(activationDeadline)), options.signal);
|
||||
}
|
||||
|
||||
if (chooserEvents.length > 1) {
|
||||
outcome = transferFailure(
|
||||
"unsupported",
|
||||
"file_input_not_activated",
|
||||
"upload trigger activated more than one file chooser",
|
||||
"none",
|
||||
"resolve_input",
|
||||
);
|
||||
return outcome;
|
||||
}
|
||||
|
||||
const chooser = chooserEvents[0];
|
||||
let backendNodeId: number | undefined;
|
||||
let multiple = summary.multiple;
|
||||
if (chooser) {
|
||||
if (options.actionTarget.frameId && chooser.frameId !== options.actionTarget.frameId) {
|
||||
outcome = transferFailure(
|
||||
"unsupported",
|
||||
"file_input_not_activated",
|
||||
"upload trigger activated a file chooser in a different frame",
|
||||
"none",
|
||||
"resolve_input",
|
||||
);
|
||||
return outcome;
|
||||
}
|
||||
if (typeof chooser.backendNodeId !== "number") {
|
||||
outcome = transferFailure(
|
||||
"unsupported",
|
||||
"file_input_not_activated",
|
||||
"upload trigger invoked a non-input file picker",
|
||||
"none",
|
||||
"resolve_input",
|
||||
);
|
||||
return outcome;
|
||||
}
|
||||
backendNodeId = chooser.backendNodeId;
|
||||
multiple = chooser.mode === "selectMultiple";
|
||||
} else {
|
||||
if (summary.count !== 1) {
|
||||
outcome = transferFailure(
|
||||
"unsupported",
|
||||
"file_input_not_activated",
|
||||
summary.count === 0
|
||||
? "upload trigger did not activate an input[type=file]"
|
||||
: "upload trigger activated more than one input[type=file]",
|
||||
"none",
|
||||
"resolve_input",
|
||||
);
|
||||
return outcome;
|
||||
}
|
||||
const input = await callOnTrigger<RuntimeReply>(
|
||||
options,
|
||||
triggerObjectId,
|
||||
`function(key) { return this.ownerDocument.defaultView?.[key]?.inputs[0]; }`,
|
||||
[stateKey],
|
||||
false,
|
||||
);
|
||||
const inputError = runtimeError(input, "failed to resolve activated file input object");
|
||||
if (inputError) throw inputError;
|
||||
if (!input.result?.objectId) throw new Error("activated file input returned no objectId");
|
||||
const described = await sendToCdpTarget<{ node?: { backendNodeId?: number } }>(
|
||||
options.cdp,
|
||||
target,
|
||||
"DOM.describeNode",
|
||||
{ objectId: input.result.objectId },
|
||||
);
|
||||
backendNodeId = described.node?.backendNodeId;
|
||||
if (typeof backendNodeId !== "number") {
|
||||
throw new Error("DOM.describeNode returned no file input backendNodeId");
|
||||
}
|
||||
}
|
||||
|
||||
if (!multiple && options.files.length !== 1) {
|
||||
outcome = enrichFailure(
|
||||
{ code: "invalid_params", message: "file input accepts exactly one file" },
|
||||
"none",
|
||||
"resolve_input",
|
||||
);
|
||||
return outcome;
|
||||
}
|
||||
|
||||
try {
|
||||
await waitBounded(
|
||||
sendToCdpTarget(options.cdp, target, "DOM.setFileInputFiles", {
|
||||
files: options.files,
|
||||
backendNodeId,
|
||||
}),
|
||||
deadline,
|
||||
options.signal,
|
||||
"setting file input files timed out",
|
||||
);
|
||||
} catch (err) {
|
||||
outcome = transferFailure(
|
||||
err instanceof BoundedWaitError ? "timeout" : "cdp_failed",
|
||||
"set_file_input_failed",
|
||||
err instanceof Error ? err.message : String(err),
|
||||
"unknown",
|
||||
"set_files",
|
||||
);
|
||||
return outcome;
|
||||
}
|
||||
outcome = { click, multiple };
|
||||
return outcome;
|
||||
} catch (err) {
|
||||
outcome = transferFailure(
|
||||
err instanceof BoundedWaitError ? "timeout" : "cdp_failed",
|
||||
"file_input_probe_failed",
|
||||
err instanceof Error ? err.message : String(err),
|
||||
"none",
|
||||
"resolve_input",
|
||||
);
|
||||
return outcome;
|
||||
}
|
||||
} finally {
|
||||
chooserSubscription?.dispose();
|
||||
let cleanupFailed = false;
|
||||
if (probeArmed && triggerObjectId) {
|
||||
try {
|
||||
await waitBounded(
|
||||
callOnTrigger(
|
||||
options,
|
||||
triggerObjectId,
|
||||
`function(key) {
|
||||
const owner = this.ownerDocument.defaultView;
|
||||
const state = owner?.[key];
|
||||
if (state?.listener) this.ownerDocument.removeEventListener("click", state.listener, true);
|
||||
if (owner) delete owner[key];
|
||||
}`,
|
||||
[stateKey],
|
||||
true,
|
||||
),
|
||||
Date.now() + CLEANUP_TIMEOUT_MS,
|
||||
undefined,
|
||||
"cleaning file input probe timed out",
|
||||
);
|
||||
} catch {
|
||||
cleanupFailed = true;
|
||||
}
|
||||
}
|
||||
if (interceptionArmed) {
|
||||
try {
|
||||
await waitBounded(
|
||||
sendToCdpTarget(options.cdp, target, "Page.setInterceptFileChooserDialog", {
|
||||
enabled: false,
|
||||
cancel: true,
|
||||
}),
|
||||
Date.now() + CLEANUP_TIMEOUT_MS,
|
||||
undefined,
|
||||
"disabling file chooser interception timed out",
|
||||
);
|
||||
} catch {
|
||||
cleanupFailed = true;
|
||||
}
|
||||
}
|
||||
try {
|
||||
await waitBounded(
|
||||
sendToCdpTarget(options.cdp, target, "Runtime.releaseObjectGroup", { objectGroup }),
|
||||
Date.now() + CLEANUP_TIMEOUT_MS,
|
||||
undefined,
|
||||
"releasing upload object group timed out",
|
||||
);
|
||||
} catch {
|
||||
cleanupFailed = true;
|
||||
}
|
||||
|
||||
const effect = isRpcError(outcome)
|
||||
? (outcome.data?.effect_state as TransferEffectState | undefined)
|
||||
: "committed";
|
||||
if (effect === "unknown" || cleanupFailed) {
|
||||
await options.cdp.detach?.(target.tabId);
|
||||
}
|
||||
if (cleanupFailed && isRpcError(outcome)) {
|
||||
outcome.data = { ...outcome.data, cleanup_state: "failed" };
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -40,6 +40,7 @@ import {
|
||||
enforceAgentWindow,
|
||||
isRpcError,
|
||||
lookupSession,
|
||||
type ResolvedTargetTab,
|
||||
resolveTargetTab,
|
||||
} from "./shared";
|
||||
import { resolveSnapshotRef } from "./snapshot-ref";
|
||||
@@ -56,6 +57,15 @@ export interface InteractionDeps {
|
||||
keepOverlayBypassAfterHover?: boolean;
|
||||
}
|
||||
|
||||
export interface ResolvedActionTarget {
|
||||
tab: ResolvedTargetTab;
|
||||
backendNodeId: number;
|
||||
cdpTarget: CdpTarget;
|
||||
frameId?: string;
|
||||
usedRef?: string;
|
||||
usedSelector?: string;
|
||||
}
|
||||
|
||||
const DEFAULT_TIMEOUT_MS = 30_000;
|
||||
const DEFAULT_HOVER_SETTLE_MS = 200;
|
||||
|
||||
@@ -128,7 +138,7 @@ async function wait(ms: number, signal?: AbortSignal): Promise<void> {
|
||||
* `RpcError` if the caller supplied neither (or both), or if neither
|
||||
* lookup matched.
|
||||
*/
|
||||
async function resolveBackendNode(
|
||||
export async function resolveBackendNode(
|
||||
cdp: CdpRunner,
|
||||
ctx: SessionContext,
|
||||
target: { tabId: number },
|
||||
@@ -215,6 +225,17 @@ async function resolveBackendNode(
|
||||
}
|
||||
}
|
||||
|
||||
export async function resolveActionTarget(
|
||||
cdp: CdpRunner,
|
||||
ctx: SessionContext,
|
||||
target: ResolvedTargetTab,
|
||||
params: { ref?: string; selector?: string },
|
||||
toolName: string,
|
||||
): Promise<ResolvedActionTarget | RpcError> {
|
||||
const node = await resolveBackendNode(cdp, ctx, target, params, toolName);
|
||||
return isRpcError(node) ? node : { tab: target, ...node };
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// tool.click
|
||||
// ---------------------------------------------------------------------------
|
||||
@@ -233,10 +254,19 @@ export async function handleClick(
|
||||
if (isRpcError(target)) return target;
|
||||
const denied = enforceAgentWindow(ctx, target, "click");
|
||||
if (denied) return denied;
|
||||
const dialogCursor = markDialogCursor(deps.cdp, target.tabId);
|
||||
const resolved = await resolveActionTarget(deps.cdp, ctx, target, params, "click");
|
||||
if (isRpcError(resolved)) return resolved;
|
||||
return clickResolvedTarget(ctx, resolved, params, deps);
|
||||
}
|
||||
|
||||
const node = await resolveBackendNode(deps.cdp, ctx, target, params, "click");
|
||||
if (isRpcError(node)) return node;
|
||||
export async function clickResolvedTarget(
|
||||
ctx: SessionContext,
|
||||
resolved: ResolvedActionTarget,
|
||||
params: Pick<ClickParams, "button" | "click_count" | "modifiers">,
|
||||
deps: InteractionDeps,
|
||||
): Promise<ClickResult | RpcError> {
|
||||
const { tab: target } = resolved;
|
||||
const dialogCursor = markDialogCursor(deps.cdp, target.tabId);
|
||||
|
||||
if (throwIfAborted(deps.signal)) {
|
||||
return { code: "cancelled", message: "click aborted" };
|
||||
@@ -247,9 +277,9 @@ export async function handleClick(
|
||||
deps.cdp,
|
||||
target.tabId,
|
||||
{
|
||||
target: node.cdpTarget,
|
||||
backendNodeId: node.backendNodeId,
|
||||
...(node.frameId ? { frameId: node.frameId } : {}),
|
||||
target: resolved.cdpTarget,
|
||||
backendNodeId: resolved.backendNodeId,
|
||||
...(resolved.frameId ? { frameId: resolved.frameId } : {}),
|
||||
},
|
||||
{ scrollIntoView: true },
|
||||
);
|
||||
@@ -337,8 +367,8 @@ export async function handleClick(
|
||||
|
||||
return attachDialogs(deps.cdp, target.tabId, dialogCursor, {
|
||||
tab_id: target.tabId,
|
||||
used_ref: node.usedRef,
|
||||
used_selector: node.usedSelector,
|
||||
used_ref: resolved.usedRef,
|
||||
used_selector: resolved.usedSelector,
|
||||
x: centre.x,
|
||||
y: centre.y,
|
||||
});
|
||||
|
||||
@@ -10,7 +10,6 @@ import {
|
||||
isVomReferenceNode,
|
||||
renderVom,
|
||||
type VomNode,
|
||||
type VomOptions,
|
||||
type VomScene,
|
||||
} from "@browser-skill/vom";
|
||||
import { ChromiumCdp } from "@/browser-driver/chromium-cdp";
|
||||
@@ -54,10 +53,6 @@ import {
|
||||
collectOverlayExcludedBackendIds,
|
||||
} from "./vom/capture";
|
||||
import { type CapturedFrameDocument, captureFrameData } from "./vom/frame-capture";
|
||||
import {
|
||||
type CaptureVomObservationResult,
|
||||
projectRecordSafeObservation,
|
||||
} from "./vom/record-safe-observation";
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Shared helpers (legacy aliases — observation.ts kept exporting these
|
||||
@@ -416,15 +411,7 @@ function isModalSignal(
|
||||
|
||||
function isSensitive(capturedNode: CapturedNode | undefined): boolean {
|
||||
const attrs = capturedNode?.attrs ?? {};
|
||||
const type = (attrs.type ?? "").toLowerCase();
|
||||
if (type === "password") return true;
|
||||
const autocomplete = (attrs.autocomplete ?? "").toLowerCase();
|
||||
return (
|
||||
autocomplete.startsWith("cc-") ||
|
||||
autocomplete === "one-time-code" ||
|
||||
autocomplete === "current-password" ||
|
||||
autocomplete === "new-password"
|
||||
);
|
||||
return (attrs.type ?? "").toLowerCase() === "password";
|
||||
}
|
||||
|
||||
interface AxNodeSignals {
|
||||
@@ -474,13 +461,7 @@ const AX_TEXT_STOP_ROLES = new Set([
|
||||
"columnheader",
|
||||
"rowheader",
|
||||
]);
|
||||
const SENSITIVE_AX_INPUT_TYPES = new Set([
|
||||
"password",
|
||||
"credit-card",
|
||||
"one-time-code",
|
||||
"current-password",
|
||||
"new-password",
|
||||
]);
|
||||
const SENSITIVE_AX_INPUT_TYPES = new Set(["password", "credit-card"]);
|
||||
|
||||
function axPropertyString(axNode: CdpAxNode, name: string): string | undefined {
|
||||
const prop = axNode.properties?.find((item) => item.name === name);
|
||||
@@ -1556,13 +1537,14 @@ async function fallbackCapturedViewModel(
|
||||
try {
|
||||
const metrics = await cdp.send<LayoutMetricsViewportReply>(tabId, "Page.getLayoutMetrics", {});
|
||||
throwIfAborted(signal, "observation");
|
||||
const source = metrics.cssLayoutViewport ?? metrics.layoutViewport ?? {};
|
||||
const vpSrc = metrics.cssLayoutViewport ?? metrics.layoutViewport ?? {};
|
||||
viewport = {
|
||||
width: source.clientWidth ?? 0,
|
||||
height: source.clientHeight ?? 0,
|
||||
width: vpSrc.clientWidth ?? 0,
|
||||
height: vpSrc.clientHeight ?? 0,
|
||||
};
|
||||
} catch (error) {
|
||||
if (isAbortError(error)) throw error;
|
||||
} catch (err) {
|
||||
if (isAbortError(err)) throw err;
|
||||
// viewport stays zero-sized
|
||||
}
|
||||
const excludedBackendNodeIds = await collectOverlayExcludedBackendIds(cdp, tabId, signal);
|
||||
throwIfAborted(signal, "observation");
|
||||
@@ -1572,58 +1554,22 @@ async function fallbackCapturedViewModel(
|
||||
async function captureForVom(
|
||||
cdp: CdpRunner,
|
||||
tabId: number,
|
||||
options: CaptureVomObservationOptions,
|
||||
conditionalSurfaceProbe: boolean,
|
||||
hoverProbeBypassOverlay?: (tabId: number, enabled: boolean) => Promise<void>,
|
||||
signal?: AbortSignal,
|
||||
): Promise<CapturedViewModel> {
|
||||
try {
|
||||
return await captureViewModel(cdp, tabId, {
|
||||
conditionalSurfaceProbe: options.conditionalSurfaceProbe ?? false,
|
||||
hoverProbeBypassOverlay: options.hoverProbeBypassOverlay,
|
||||
signal: options.signal,
|
||||
conditionalSurfaceProbe,
|
||||
hoverProbeBypassOverlay,
|
||||
signal,
|
||||
});
|
||||
} catch (error) {
|
||||
if (isAbortError(error)) throw error;
|
||||
return fallbackCapturedViewModel(cdp, tabId, options.signal);
|
||||
} catch (err) {
|
||||
if (isAbortError(err)) throw err;
|
||||
return fallbackCapturedViewModel(cdp, tabId, signal);
|
||||
}
|
||||
}
|
||||
|
||||
export interface CaptureVomObservationOptions extends VomOptions {
|
||||
conditionalSurfaceProbe?: boolean;
|
||||
hoverProbeBypassOverlay?: (tabId: number, enabled: boolean) => Promise<void>;
|
||||
signal?: AbortSignal;
|
||||
}
|
||||
|
||||
export async function captureVomObservation(
|
||||
cdp: CdpRunner,
|
||||
tabId: number,
|
||||
url: string | undefined,
|
||||
options: CaptureVomObservationOptions = {},
|
||||
): Promise<CaptureVomObservationResult> {
|
||||
throwIfAborted(options.signal, "observation");
|
||||
await cdp.ensureAttachedToUrl?.(tabId, url);
|
||||
throwIfAborted(options.signal, "observation");
|
||||
const captured = await captureForVom(cdp, tabId, options);
|
||||
throwIfAborted(options.signal, "observation");
|
||||
const documents = await captureFrameData<CdpAxNode>(cdp, tabId, captured, options.signal);
|
||||
throwIfAborted(options.signal, "observation");
|
||||
const scene =
|
||||
captured.rootFrameId && captured.frameNodes?.size
|
||||
? buildFrameVomScene(documents, captured, { pageUrl: url })
|
||||
: buildVomScene(documents[0]?.axNodes ?? [], captured, { pageUrl: url });
|
||||
const rendered = renderVom(scene, {
|
||||
maxDepth: options.maxDepth,
|
||||
maxTokens: options.maxTokens,
|
||||
redactValues: options.redactValues,
|
||||
activeRegionPolicy: options.activeRegionPolicy,
|
||||
});
|
||||
throwIfAborted(options.signal, "observation");
|
||||
return projectRecordSafeObservation({
|
||||
rootFrameId: captured.rootFrameId ?? documents[0]?.frameId ?? "root",
|
||||
frameDocuments: documents,
|
||||
rendered,
|
||||
surfaceProbes: captured.surfaceProbes,
|
||||
});
|
||||
}
|
||||
|
||||
async function handleVomObservation(
|
||||
manager: SessionManager,
|
||||
params: SnapshotParams | ObserveParams,
|
||||
@@ -1653,22 +1599,40 @@ async function handleVomObservation(
|
||||
try {
|
||||
throwIfAborted(signal, toolName);
|
||||
deps.cdp.trackSessionTab?.(ctx.sessionId, target.tabId);
|
||||
await deps.cdp.ensureAttachedToUrl?.(target.tabId, target.url);
|
||||
throwIfAborted(signal, toolName);
|
||||
const effectiveConditionalSurfaceProbe =
|
||||
deps.conditionalSurfaceProbe ?? conditionalSurfaceProbe;
|
||||
const observation = await captureVomObservation(deps.cdp, target.tabId, target.url, {
|
||||
const captured = await captureForVom(
|
||||
deps.cdp,
|
||||
target.tabId,
|
||||
effectiveConditionalSurfaceProbe,
|
||||
deps.hoverProbeBypassOverlay,
|
||||
signal,
|
||||
);
|
||||
throwIfAborted(signal, toolName);
|
||||
const frameDocuments = await captureFrameData<CdpAxNode>(
|
||||
deps.cdp,
|
||||
target.tabId,
|
||||
captured,
|
||||
signal,
|
||||
);
|
||||
throwIfAborted(signal, toolName);
|
||||
const scene =
|
||||
captured.rootFrameId && captured.frameNodes?.size
|
||||
? buildFrameVomScene(frameDocuments, captured, { pageUrl: target.url })
|
||||
: buildVomScene(frameDocuments[0]?.axNodes ?? [], captured, { pageUrl: target.url });
|
||||
const rendered = renderVom(scene, {
|
||||
maxDepth: params.max_depth,
|
||||
maxTokens: params.max_tokens,
|
||||
activeRegionPolicy: true,
|
||||
conditionalSurfaceProbe: effectiveConditionalSurfaceProbe,
|
||||
hoverProbeBypassOverlay: deps.hoverProbeBypassOverlay,
|
||||
signal,
|
||||
});
|
||||
throwIfAborted(signal, toolName);
|
||||
const targetByFrameId = new Map(
|
||||
observation.frames.map((frame) => [frame.frameId, frame.target]),
|
||||
frameDocuments.map((document) => [document.frameId, document.target]),
|
||||
);
|
||||
ctx.refStore.replace(
|
||||
observation.refs.map((ref) => {
|
||||
rendered.refs.map((ref) => {
|
||||
const refTarget = ref.frameId ? targetByFrameId.get(ref.frameId) : undefined;
|
||||
return [
|
||||
ref.ref,
|
||||
@@ -1682,14 +1646,14 @@ async function handleVomObservation(
|
||||
}),
|
||||
);
|
||||
return attachDialogs(deps.cdp, target.tabId, dialogCursor, {
|
||||
text: observation.text,
|
||||
ref_count: observation.refs.length,
|
||||
text: rendered.text,
|
||||
ref_count: rendered.refs.length,
|
||||
tab_id: target.tabId,
|
||||
truncated: observation.truncated,
|
||||
truncated: rendered.truncated,
|
||||
...(toolName === "observe" && (params as ObserveParams).debug_surfaces
|
||||
? {
|
||||
debug: {
|
||||
surface_probes: (observation.surfaceProbes ?? []).map((probe) => ({
|
||||
surface_probes: (captured.surfaceProbes ?? []).map((probe) => ({
|
||||
trigger_backend_node_id: probe.triggerBackendNodeId,
|
||||
...(probe.triggerPoint ? { trigger_point: probe.triggerPoint } : {}),
|
||||
trigger_action: probe.triggerAction,
|
||||
|
||||
@@ -29,7 +29,7 @@ import type {
|
||||
RecordStopParams,
|
||||
RecordStopResult,
|
||||
RpcError,
|
||||
TraceV2,
|
||||
Trace,
|
||||
} from "@/transport/types";
|
||||
import { handleNavigate } from "./navigation";
|
||||
import {
|
||||
@@ -50,8 +50,8 @@ interface ActiveRecording {
|
||||
steps: DraftTraceStep[];
|
||||
startedAt: string;
|
||||
startedAtMs: number;
|
||||
finishPromise: Promise<TraceV2>;
|
||||
resolveFinish: (trace: TraceV2) => void;
|
||||
finishPromise: Promise<Trace>;
|
||||
resolveFinish: (trace: Trace) => void;
|
||||
rejectFinish: (err: Error) => void;
|
||||
settled: boolean;
|
||||
finishing: boolean;
|
||||
@@ -154,7 +154,7 @@ async function sendRecordStartWithAck(
|
||||
throw lastError ?? new Error("failed to start recording in content script");
|
||||
}
|
||||
|
||||
function buildTrace(recording: ActiveRecording): TraceV2 {
|
||||
function buildTrace(recording: ActiveRecording): Trace {
|
||||
const { pages, steps } = reduceTraceSteps(recording.steps, recording.startUrl);
|
||||
const startUrl = resolveTraceStartUrl(recording.steps, recording.startUrl, pages);
|
||||
return {
|
||||
@@ -532,7 +532,7 @@ async function finishRecordingByRequest(
|
||||
}
|
||||
}
|
||||
|
||||
async function finishRecording(sessionId: string, deps: RecordDeps): Promise<TraceV2 | null> {
|
||||
async function finishRecording(sessionId: string, deps: RecordDeps): Promise<Trace | null> {
|
||||
const recording = recordings.get(sessionId);
|
||||
if (!recording || recording.settled || recording.finishing) return null;
|
||||
recording.finishing = true;
|
||||
@@ -574,9 +574,9 @@ export async function handleRecordStart(
|
||||
// on the destination page can RECORD_QUERY → rearm → show RecordOverlay
|
||||
// instead of flashing ControlOverlay ("Agent 正在控制").
|
||||
const requestId = makeRequestId(target.tabId);
|
||||
let resolveFinish!: (trace: TraceV2) => void;
|
||||
let resolveFinish!: (trace: Trace) => void;
|
||||
let rejectFinish!: (err: Error) => void;
|
||||
const finishPromise = new Promise<TraceV2>((resolve, reject) => {
|
||||
const finishPromise = new Promise<Trace>((resolve, reject) => {
|
||||
resolveFinish = resolve;
|
||||
rejectFinish = reject;
|
||||
});
|
||||
@@ -785,9 +785,9 @@ export async function handleRecordAwait(
|
||||
return { code: "cancelled", message: "record_await aborted" };
|
||||
}
|
||||
|
||||
const outcome = await new Promise<{ trace: TraceV2 } | { error: RpcError }>((resolve) => {
|
||||
const outcome = await new Promise<{ trace: Trace } | { error: RpcError }>((resolve) => {
|
||||
let settled = false;
|
||||
const finish = (result: { trace: TraceV2 } | { error: RpcError }) => {
|
||||
const finish = (result: { trace: Trace } | { error: RpcError }) => {
|
||||
if (settled) return;
|
||||
settled = true;
|
||||
if (timer) clearTimeout(timer);
|
||||
|
||||
@@ -76,7 +76,6 @@ export interface SessionStopResult {
|
||||
}
|
||||
|
||||
export interface SessionStopDeps {
|
||||
signal?: AbortSignal;
|
||||
cdp?: {
|
||||
detachSession(sessionId: string): Promise<void>;
|
||||
};
|
||||
@@ -178,9 +177,6 @@ export async function handleSessionStop(
|
||||
message: `session ${params.session_id} unknown`,
|
||||
};
|
||||
}
|
||||
if (deps.signal?.aborted) {
|
||||
return { code: "cancelled", message: "session_stop aborted before teardown" };
|
||||
}
|
||||
|
||||
// Step 1: auto-return borrowed tabs. Iterate over a snapshot of the
|
||||
// ids so deletions during iteration do not break the Map iterator.
|
||||
@@ -191,7 +187,6 @@ export async function handleSessionStop(
|
||||
try {
|
||||
const tabManagement = {
|
||||
...(deps.tabManagement ?? {}),
|
||||
signal: deps.signal,
|
||||
isAgentWindowId:
|
||||
deps.tabManagement?.isAgentWindowId ??
|
||||
((windowId: number) => manager.findByWindowId(windowId) !== null),
|
||||
@@ -219,17 +214,6 @@ export async function handleSessionStop(
|
||||
}
|
||||
}
|
||||
|
||||
if (deps.signal?.aborted) {
|
||||
const nonCancelFailures = returnFailures?.filter((failure) => failure.code !== "cancelled");
|
||||
if (nonCancelFailures && nonCancelFailures.length > 0) {
|
||||
return {
|
||||
...(returnedTabIds.length > 0 ? { returned_tab_ids: returnedTabIds } : {}),
|
||||
return_failures: nonCancelFailures,
|
||||
};
|
||||
}
|
||||
return { code: "cancelled", message: "session_stop aborted during tab return" };
|
||||
}
|
||||
|
||||
const result: SessionStopResult = {};
|
||||
if (returnedTabIds.length > 0) result.returned_tab_ids = returnedTabIds;
|
||||
if (returnFailures && returnFailures.length > 0) {
|
||||
@@ -249,10 +233,6 @@ export async function handleSessionStop(
|
||||
// Step 3: detach CDP sessions this session opened (no-op if none).
|
||||
await deps.cdp?.detachSession(params.session_id);
|
||||
|
||||
if (deps.signal?.aborted) {
|
||||
return { code: "cancelled", message: "session_stop aborted before window close" };
|
||||
}
|
||||
|
||||
// Step 4: close the Agent Window and drop the context.
|
||||
await manager.stop(params.session_id);
|
||||
|
||||
|
||||
@@ -53,6 +53,7 @@ export type { DialogCursor };
|
||||
export interface CdpRunner {
|
||||
send<T = unknown>(tabId: number, method: string, params?: object): Promise<T>;
|
||||
sendToTarget?<T = unknown>(target: CdpTarget, method: string, params?: object): Promise<T>;
|
||||
detach?(tabId: number): Promise<void>;
|
||||
getFrameGraph?(tabId: number): Promise<CdpFrameGraph>;
|
||||
ensureAttachedToUrl?(tabId: number, expectedUrl: string | undefined): Promise<void>;
|
||||
trackSessionTab?(sessionId: string, tabId: number): void;
|
||||
|
||||
@@ -0,0 +1,59 @@
|
||||
// Upload orchestration: validate the session-scoped request, resolve its click
|
||||
// target, then delegate the browser protocol transaction to the file-input
|
||||
// transaction module.
|
||||
|
||||
import type { SessionManager } from "@/session-manager/manager";
|
||||
import type { RpcError, UploadParams, UploadResult } from "@/transport/types";
|
||||
import { uploadThroughActivatedFileInput } from "./file-input-transaction";
|
||||
import { clickResolvedTarget, type InteractionDeps, resolveActionTarget } from "./interaction";
|
||||
import {
|
||||
type CdpRunner,
|
||||
enforceAgentWindow,
|
||||
isRpcError,
|
||||
lookupSession,
|
||||
resolveTargetTab,
|
||||
} from "./shared";
|
||||
|
||||
const DEFAULT_TIMEOUT_MS = 120_000;
|
||||
|
||||
export interface UploadDeps extends InteractionDeps {
|
||||
cdp: CdpRunner;
|
||||
}
|
||||
|
||||
export async function handleUpload(
|
||||
manager: SessionManager,
|
||||
params: UploadParams,
|
||||
deps: UploadDeps,
|
||||
): Promise<UploadResult | RpcError> {
|
||||
const ctx = lookupSession(manager, params, "upload");
|
||||
if (isRpcError(ctx)) return ctx;
|
||||
const target = await resolveTargetTab(manager, ctx, params.tab_id, deps.tabsApi);
|
||||
if (isRpcError(target)) return target;
|
||||
const denied = enforceAgentWindow(ctx, target, "upload");
|
||||
if (denied) return denied;
|
||||
if (
|
||||
params.files.length === 0 ||
|
||||
params.files.length > 20 ||
|
||||
params.files.some((file) => !file.staged_path)
|
||||
) {
|
||||
return { code: "invalid_params", message: "upload requires daemon-staged files" };
|
||||
}
|
||||
const address = await resolveActionTarget(deps.cdp, ctx, target, params, "upload");
|
||||
if (isRpcError(address)) return address;
|
||||
const timeoutMs = params.timeout_ms ?? DEFAULT_TIMEOUT_MS;
|
||||
const transaction = await uploadThroughActivatedFileInput({
|
||||
cdp: deps.cdp,
|
||||
actionTarget: address,
|
||||
files: params.files.map((file) => file.staged_path as string),
|
||||
timeoutMs,
|
||||
signal: deps.signal,
|
||||
trigger: () => clickResolvedTarget(ctx, address, {}, deps),
|
||||
});
|
||||
if (isRpcError(transaction)) return transaction;
|
||||
return {
|
||||
tab_id: target.tabId,
|
||||
used_ref: transaction.click.used_ref,
|
||||
used_selector: transaction.click.used_selector,
|
||||
file_names: params.files.map((file) => file.name),
|
||||
};
|
||||
}
|
||||
@@ -77,12 +77,9 @@ export function buildFrameDocuments<T extends FrameOwnedAxNode>(
|
||||
): FrameDocument<T>[] {
|
||||
const frames = frameList(graph, batches, captured);
|
||||
const frameById = new Map(frames.map((frame) => [frame.frameId, frame]));
|
||||
const rootFrameId = graph?.rootFrameId ?? captured.rootFrameId ?? frames[0]?.frameId;
|
||||
const domNodesForFrame = (frameId: string): CapturedNode[] =>
|
||||
captured.frameNodes?.get(frameId) ?? (frameId === rootFrameId ? captured.nodes : []);
|
||||
const backendOwner = new Map<string, string>();
|
||||
for (const frame of frames) {
|
||||
for (const node of domNodesForFrame(frame.frameId)) {
|
||||
for (const node of captured.frameNodes?.get(frame.frameId) ?? []) {
|
||||
backendOwner.set(targetBackendKey(frame.target, node.backendNodeId), frame.frameId);
|
||||
}
|
||||
}
|
||||
@@ -168,6 +165,6 @@ export function buildFrameDocuments<T extends FrameOwnedAxNode>(
|
||||
...frame,
|
||||
contextScopeId: frame.frameId,
|
||||
axNodes: axNodesByFrame.get(frame.frameId) ?? [],
|
||||
domNodes: domNodesForFrame(frame.frameId),
|
||||
domNodes: captured.frameNodes?.get(frame.frameId) ?? [],
|
||||
}));
|
||||
}
|
||||
|
||||
@@ -1,77 +0,0 @@
|
||||
import type { Rect, RenderedRef, VomResult } from "@browser-skill/vom";
|
||||
import type { CdpTarget } from "@/browser-driver/frame-graph";
|
||||
import type { CapturedSurfaceProbe } from "./capture";
|
||||
import type { CapturedFrameDocument, FrameAxNode } from "./frame-capture";
|
||||
|
||||
/** Frame identity needed to resolve an `@eN` that lives in an iframe. */
|
||||
export interface CaptureVomFrame {
|
||||
frameId: string;
|
||||
target: CdpTarget;
|
||||
parentFrameId?: string;
|
||||
ownerBackendNodeId?: number;
|
||||
}
|
||||
|
||||
/** Allowlisted geometry used to match a recorded action to a rendered ref. */
|
||||
export interface CaptureVomMatchNode {
|
||||
frameId: string;
|
||||
backendNodeId: number;
|
||||
tag: string;
|
||||
/** Top-level viewport-relative geometry. */
|
||||
rect: Rect | null;
|
||||
/** Frame-local viewport-relative geometry, when available. */
|
||||
localRect?: Rect | null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Raw AX/DOM trees are never returned. When `redactValues` is enabled by the
|
||||
* caller, form values in `text` are masked as well.
|
||||
*/
|
||||
export interface CaptureVomObservationResult {
|
||||
text: string;
|
||||
refs: RenderedRef[];
|
||||
truncated: boolean;
|
||||
rootFrameId: string;
|
||||
frames: CaptureVomFrame[];
|
||||
matchNodes: CaptureVomMatchNode[];
|
||||
surfaceProbes?: CapturedSurfaceProbe[];
|
||||
}
|
||||
|
||||
function projectFrames(documents: CapturedFrameDocument<FrameAxNode>[]): CaptureVomFrame[] {
|
||||
return documents.map((document) => ({
|
||||
frameId: document.frameId,
|
||||
target: document.target,
|
||||
...(document.parentFrameId ? { parentFrameId: document.parentFrameId } : {}),
|
||||
...(document.ownerBackendNodeId !== undefined
|
||||
? { ownerBackendNodeId: document.ownerBackendNodeId }
|
||||
: {}),
|
||||
}));
|
||||
}
|
||||
|
||||
function projectMatchNodes(documents: CapturedFrameDocument<FrameAxNode>[]): CaptureVomMatchNode[] {
|
||||
return documents.flatMap((document) =>
|
||||
document.domNodes.map((node) => ({
|
||||
frameId: document.frameId,
|
||||
backendNodeId: node.backendNodeId,
|
||||
tag: node.tag,
|
||||
rect: node.rect,
|
||||
...(node.localRect !== undefined ? { localRect: node.localRect } : {}),
|
||||
})),
|
||||
);
|
||||
}
|
||||
|
||||
export function projectRecordSafeObservation(input: {
|
||||
rootFrameId: string;
|
||||
frameDocuments: CapturedFrameDocument<FrameAxNode>[];
|
||||
rendered: VomResult;
|
||||
surfaceProbes?: CapturedSurfaceProbe[];
|
||||
}): CaptureVomObservationResult {
|
||||
return {
|
||||
text: input.rendered.text,
|
||||
refs: input.rendered.refs,
|
||||
truncated: input.rendered.truncated,
|
||||
rootFrameId: input.rootFrameId,
|
||||
frames: projectFrames(input.frameDocuments),
|
||||
matchNodes: projectMatchNodes(input.frameDocuments),
|
||||
surfaceProbes: input.surfaceProbes,
|
||||
};
|
||||
}
|
||||
@@ -33,10 +33,22 @@ export type RpcErrorReason =
|
||||
| "restricted_tab_url"
|
||||
| "borrow_conflict"
|
||||
| "screenshot_capture_failed"
|
||||
| "file_input_probe_failed"
|
||||
| "file_input_not_activated"
|
||||
| "set_file_input_failed"
|
||||
| "download_capture_failed"
|
||||
| "transfer_outcome_unknown"
|
||||
| "transfer_timeout"
|
||||
| "cleanup_failed";
|
||||
|
||||
export type TransferEffectState = "none" | "committed" | "unknown";
|
||||
export type TransferCleanupState = "complete" | "failed";
|
||||
|
||||
export interface RpcErrorData {
|
||||
reason?: RpcErrorReason;
|
||||
effect_state?: TransferEffectState;
|
||||
phase?: string;
|
||||
cleanup_state?: TransferCleanupState;
|
||||
[key: string]: unknown;
|
||||
}
|
||||
|
||||
@@ -500,6 +512,50 @@ export interface SelectResult {
|
||||
dialogs?: JavaScriptDialogInfo[];
|
||||
}
|
||||
|
||||
export interface UploadFile {
|
||||
transfer_id: string;
|
||||
name: string;
|
||||
staged_path?: string;
|
||||
}
|
||||
|
||||
export interface UploadParams {
|
||||
session_id: string;
|
||||
ref?: string;
|
||||
selector?: string;
|
||||
tab_id?: number;
|
||||
files: UploadFile[];
|
||||
timeout_ms?: number;
|
||||
}
|
||||
|
||||
export interface UploadResult {
|
||||
tab_id: number;
|
||||
used_ref?: string;
|
||||
used_selector?: string;
|
||||
file_names: string[];
|
||||
}
|
||||
|
||||
export interface DownloadParams {
|
||||
session_id: string;
|
||||
ref?: string;
|
||||
selector?: string;
|
||||
tab_id?: number;
|
||||
timeout_ms?: number;
|
||||
browser_relative_dir?: string;
|
||||
max_byte_size?: number;
|
||||
}
|
||||
|
||||
export interface DownloadResult {
|
||||
tab_id: number;
|
||||
used_ref?: string;
|
||||
used_selector?: string;
|
||||
suggested_filename: string;
|
||||
byte_size: number;
|
||||
mime?: string;
|
||||
danger?: string;
|
||||
browser_path?: string;
|
||||
transfer_id?: string;
|
||||
}
|
||||
|
||||
// --------------------------------------------------------------------------
|
||||
// M9 tool payloads — evaluate / wait_for_navigation / wait_ms
|
||||
// --------------------------------------------------------------------------
|
||||
@@ -657,60 +713,10 @@ export interface EmulateResult {
|
||||
}
|
||||
|
||||
// --------------------------------------------------------------------------
|
||||
// Semantic record payloads mirror the versioned Rust protocol models.
|
||||
// Semantic record payloads — mirror bsk-protocol record.rs (Trace v2)
|
||||
// --------------------------------------------------------------------------
|
||||
|
||||
export const TRACE_VERSION_V3 = 3;
|
||||
/** Logical v2 identifier. Not a wire field — v2 envelopes omit `version`. */
|
||||
export const TRACE_VERSION_V2 = 2;
|
||||
export const VOM_FORMAT_VERSION = 1;
|
||||
|
||||
export interface TargetDescriptorV3 {
|
||||
ref?: string;
|
||||
role?: string;
|
||||
name?: string;
|
||||
ctx?: string;
|
||||
unmatched?: boolean;
|
||||
}
|
||||
|
||||
export interface RecorderInfo {
|
||||
bsk: string;
|
||||
vom: number;
|
||||
}
|
||||
|
||||
export type StopReason = "user_finish" | "cli_stop";
|
||||
|
||||
export interface TraceStateV3 {
|
||||
id: string;
|
||||
url: string;
|
||||
title?: string;
|
||||
body: string;
|
||||
truncated?: boolean;
|
||||
}
|
||||
|
||||
export interface StepResultV3 {
|
||||
state: string;
|
||||
}
|
||||
|
||||
export interface StepCommonV3 {
|
||||
id: number;
|
||||
state: string;
|
||||
result: StepResultV3;
|
||||
}
|
||||
|
||||
export type NavigationCause =
|
||||
| "user_typed"
|
||||
| "link"
|
||||
| "form_submit"
|
||||
| "reload"
|
||||
| "history"
|
||||
| "script"
|
||||
| "browser";
|
||||
|
||||
export type FillCommit = "enter" | "suggestion" | "blur";
|
||||
|
||||
/** Legacy v2 target shape retained for existing record producers. */
|
||||
export interface TargetDescriptorV2 {
|
||||
export interface TargetDescriptor {
|
||||
role?: string;
|
||||
name?: string;
|
||||
tag: string;
|
||||
@@ -723,43 +729,43 @@ export interface TraceEntry {
|
||||
start_url: string;
|
||||
}
|
||||
|
||||
export interface PageRefV2 {
|
||||
export interface PageRef {
|
||||
id: string;
|
||||
url: string;
|
||||
title?: string;
|
||||
}
|
||||
|
||||
export interface SelectedOptionV2 {
|
||||
export interface SelectedOption {
|
||||
value: string;
|
||||
label?: string;
|
||||
}
|
||||
|
||||
export interface StepEffectV2 {
|
||||
export interface StepEffect {
|
||||
navigated_to: string;
|
||||
}
|
||||
|
||||
export interface StepCommonV2 {
|
||||
export interface StepCommon {
|
||||
id: number;
|
||||
page: string;
|
||||
effect?: StepEffectV2;
|
||||
effect?: StepEffect;
|
||||
}
|
||||
|
||||
/** Capture/buffer draft before v2 reduction. */
|
||||
export type DraftTraceStep =
|
||||
| {
|
||||
op: "click";
|
||||
target: TargetDescriptorV2;
|
||||
target: TargetDescriptor;
|
||||
navigated_to?: string;
|
||||
page_url?: string;
|
||||
}
|
||||
| {
|
||||
op: "hover";
|
||||
target: TargetDescriptorV2;
|
||||
target: TargetDescriptor;
|
||||
page_url?: string;
|
||||
}
|
||||
| {
|
||||
op: "fill";
|
||||
target: TargetDescriptorV2;
|
||||
target: TargetDescriptor;
|
||||
value: string;
|
||||
redacted?: boolean;
|
||||
page_url?: string;
|
||||
@@ -767,14 +773,14 @@ export type DraftTraceStep =
|
||||
| {
|
||||
op: "press";
|
||||
key: string;
|
||||
target?: TargetDescriptorV2;
|
||||
target?: TargetDescriptor;
|
||||
modifiers?: KeyModifier[];
|
||||
navigated_to?: string;
|
||||
page_url?: string;
|
||||
}
|
||||
| {
|
||||
op: "select";
|
||||
target: TargetDescriptorV2;
|
||||
target: TargetDescriptor;
|
||||
values: string[];
|
||||
labels?: string[];
|
||||
navigated_to?: string;
|
||||
@@ -787,75 +793,34 @@ export type DraftTraceStep =
|
||||
};
|
||||
|
||||
/** Exported record-only step (trace v2). */
|
||||
export type StepV2 =
|
||||
| ({ op: "navigate" } & StepCommonV2 & { to: string })
|
||||
| ({ op: "click" } & StepCommonV2 & { target: TargetDescriptorV2 })
|
||||
| ({ op: "hover" } & StepCommonV2 & { target: TargetDescriptorV2 })
|
||||
| ({ op: "fill" } & StepCommonV2 & {
|
||||
target: TargetDescriptorV2;
|
||||
export type Step =
|
||||
| ({ op: "navigate" } & StepCommon & { to: string })
|
||||
| ({ op: "click" } & StepCommon & { target: TargetDescriptor })
|
||||
| ({ op: "hover" } & StepCommon & { target: TargetDescriptor })
|
||||
| ({ op: "fill" } & StepCommon & {
|
||||
target: TargetDescriptor;
|
||||
value: string;
|
||||
redacted?: boolean;
|
||||
})
|
||||
| ({ op: "select" } & StepCommonV2 & {
|
||||
target: TargetDescriptorV2;
|
||||
selection: SelectedOptionV2[];
|
||||
| ({ op: "select" } & StepCommon & {
|
||||
target: TargetDescriptor;
|
||||
selection: SelectedOption[];
|
||||
})
|
||||
| ({ op: "press" } & StepCommonV2 & {
|
||||
| ({ op: "press" } & StepCommon & {
|
||||
key: string;
|
||||
modifiers?: KeyModifier[];
|
||||
target?: TargetDescriptorV2;
|
||||
target?: TargetDescriptor;
|
||||
});
|
||||
|
||||
export interface TraceV2 {
|
||||
export interface Trace {
|
||||
recorded_at: string;
|
||||
started_at?: string;
|
||||
purpose?: string;
|
||||
entry: TraceEntry;
|
||||
pages: PageRefV2[];
|
||||
steps: StepV2[];
|
||||
pages: PageRef[];
|
||||
steps: Step[];
|
||||
}
|
||||
|
||||
export interface SelectedOptionV3 {
|
||||
value: string;
|
||||
label?: string;
|
||||
}
|
||||
|
||||
export type StepV3 =
|
||||
| ({ op: "navigate" } & StepCommonV3 & { to: string; cause: NavigationCause })
|
||||
| ({ op: "click" } & StepCommonV3 & { target: TargetDescriptorV3 })
|
||||
| ({ op: "hover" } & StepCommonV3 & { target: TargetDescriptorV3 })
|
||||
| ({ op: "fill" } & StepCommonV3 & {
|
||||
target: TargetDescriptorV3;
|
||||
value: string;
|
||||
commit: FillCommit;
|
||||
redacted?: boolean;
|
||||
})
|
||||
| ({ op: "select" } & StepCommonV3 & {
|
||||
target: TargetDescriptorV3;
|
||||
selection?: SelectedOptionV3[];
|
||||
})
|
||||
| ({ op: "press" } & StepCommonV3 & {
|
||||
key: string;
|
||||
modifiers?: KeyModifier[];
|
||||
target?: TargetDescriptorV3;
|
||||
})
|
||||
| ({ op: "scroll" } & StepCommonV3);
|
||||
|
||||
export interface TraceV3 {
|
||||
version: typeof TRACE_VERSION_V3;
|
||||
recorded_at: string;
|
||||
started_at?: string;
|
||||
purpose?: string;
|
||||
stopped_by: StopReason;
|
||||
entry: TraceEntry;
|
||||
recorder: RecorderInfo;
|
||||
states: TraceStateV3[];
|
||||
steps: StepV3[];
|
||||
}
|
||||
|
||||
export type RecordedTrace = TraceV2 | TraceV3;
|
||||
export type RecordedStep = StepV2 | StepV3;
|
||||
|
||||
export interface RecordStartParams {
|
||||
session_id: string;
|
||||
tab_id?: number;
|
||||
@@ -873,7 +838,7 @@ export interface RecordStopParams {
|
||||
}
|
||||
|
||||
export interface RecordStopResult {
|
||||
trace: RecordedTrace;
|
||||
trace: Trace;
|
||||
}
|
||||
|
||||
export interface RecordAwaitParams {
|
||||
@@ -882,5 +847,5 @@ export interface RecordAwaitParams {
|
||||
}
|
||||
|
||||
export interface RecordAwaitResult {
|
||||
trace: RecordedTrace;
|
||||
trace: Trace;
|
||||
}
|
||||
|
||||
@@ -30,6 +30,7 @@ export default defineConfig({
|
||||
permissions: [
|
||||
"alarms",
|
||||
"debugger",
|
||||
"downloads",
|
||||
"idle",
|
||||
"notifications",
|
||||
"tabs",
|
||||
|
||||
@@ -53,6 +53,7 @@ flate2 = { workspace = true }
|
||||
tar = { workspace = true }
|
||||
zip = { workspace = true }
|
||||
sha2 = { workspace = true }
|
||||
base64 = { workspace = true }
|
||||
|
||||
[target.'cfg(unix)'.dependencies]
|
||||
nix = { workspace = true }
|
||||
@@ -63,6 +64,7 @@ windows-sys = { version = "0.59", features = [
|
||||
"Win32_Foundation",
|
||||
"Win32_System_Threading",
|
||||
"Win32_Security",
|
||||
"Win32_Storage_FileSystem",
|
||||
] }
|
||||
|
||||
[dev-dependencies]
|
||||
|
||||
@@ -206,6 +206,29 @@ Both capture from the moment the tab is attached and read a bounded per-tab buff
|
||||
| `bsk select <ref-or-selector> --value <v>` | Set `<select>` option(s) by `value` (repeat `--value` for multi-select) |
|
||||
| `bsk press <key>` | Key/combo (`Enter`, `Ctrl+A`, …; optional `--ref` to focus first) |
|
||||
|
||||
### File transfer (require `--session`)
|
||||
|
||||
| Command | Summary |
|
||||
|---------|---------|
|
||||
| `bsk upload <ref-or-selector> --file <path>` | Click one upload trigger and attach an agent-readable local file (`--file` is repeatable) |
|
||||
| `bsk download <ref-or-selector> --out <path>` | Click one download trigger and copy the single completed file to an exact local path (`--overwrite` is opt-in) |
|
||||
|
||||
The agent/harness decides whether a file transfer is appropriate and which local path belongs to the task. Treat upload as disclosure of that file to the current website, and download as accepting website-controlled bytes onto the local filesystem. Use only paths that are necessary for the user's bounded goal.
|
||||
|
||||
BrowserSkill enforces the mechanical boundary: files are staged under a session-scoped opaque transfer, only daemon-minted capabilities reach the extension, upload/download still obey Agent Window tab checks, and transfers are chunk/size bounded. Upload intercepts the native chooser for one transaction, locates the input activated in the resolved target's document, and assigns only the staged file paths. Download uniquely correlates one exact-target browser intent with one Chrome download in either event order, routes it through a daemon-minted relative directory, and lets the daemon validate and import it. Upload staging remains available for a later form submission and is removed when the session ends. Downloads cannot overwrite an existing destination unless `--overwrite` is explicit. BrowserSkill does not inspect file content or decide whether its meaning is sensitive.
|
||||
|
||||
Do not use `request-help` merely because a native file chooser or browser download is involved; try these commands first. For transfer failures, use the structured error instead of retrying blindly:
|
||||
|
||||
- `reason=file_input_not_activated` means the requested click did not activate exactly one `<input type="file">`; the page may use a non-input picker such as `window.showOpenFilePicker()`. Do not retry blindly. If human help is available, call `request-help` and tell the user the exact original local path to choose. The staged daemon path is internal and must not be shown to the user.
|
||||
- `reason=file_input_probe_failed` means BrowserSkill could not safely establish the browser-side upload transaction. Do not repeat the same action; use `request-help` when available.
|
||||
- `reason=set_file_input_failed` means BrowserSkill found the activated file input but Chrome rejected the staged path or assignment. Check the extension's file-URL access permission; otherwise use `request-help`.
|
||||
- `reason=download_capture_failed` means BrowserSkill could not attribute exactly one completed download to the requested target. Do not retry blindly or accept an unrelated browser download; use `request-help` when available.
|
||||
- `effect_state=none` means BrowserSkill confirmed that no file-transfer effect was committed. Follow the accompanying reason; a corrected target or explicit human fallback may be attempted.
|
||||
- `effect_state=unknown` means the browser may already have attached or created the file. Do not repeat the transfer. Observe the page if that can establish the result; otherwise stop and report the uncertainty.
|
||||
- `effect_state=committed` means the browser-side effect occurred even if later completion or cleanup failed. Do not repeat it; continue only after verifying the resulting page/download state.
|
||||
|
||||
If `request-help` returns `outcome="disabled"`, do not retry it. Stop gracefully and report the transfer mechanism that requires human intervention.
|
||||
|
||||
### Scripting & timing
|
||||
|
||||
| Command | Summary |
|
||||
|
||||
@@ -0,0 +1,75 @@
|
||||
//! Atomic visibility boundary for files received into a same-directory temp.
|
||||
|
||||
use std::path::Path;
|
||||
|
||||
pub fn commit(temp: &Path, out: &Path, overwrite: bool) -> std::io::Result<()> {
|
||||
if !overwrite {
|
||||
std::fs::hard_link(temp, out)?;
|
||||
std::fs::remove_file(temp)?;
|
||||
return Ok(());
|
||||
}
|
||||
replace(temp, out)
|
||||
}
|
||||
|
||||
#[cfg(unix)]
|
||||
fn replace(temp: &Path, out: &Path) -> std::io::Result<()> {
|
||||
// POSIX rename replaces an existing non-directory destination atomically.
|
||||
std::fs::rename(temp, out)
|
||||
}
|
||||
|
||||
#[cfg(windows)]
|
||||
fn replace(temp: &Path, out: &Path) -> std::io::Result<()> {
|
||||
use std::os::windows::ffi::OsStrExt;
|
||||
use windows_sys::Win32::Storage::FileSystem::{
|
||||
MOVEFILE_REPLACE_EXISTING, MOVEFILE_WRITE_THROUGH, MoveFileExW,
|
||||
};
|
||||
|
||||
let from: Vec<u16> = temp.as_os_str().encode_wide().chain(Some(0)).collect();
|
||||
let to: Vec<u16> = out.as_os_str().encode_wide().chain(Some(0)).collect();
|
||||
// SAFETY: both buffers are NUL-terminated and remain alive for the call.
|
||||
let ok = unsafe {
|
||||
MoveFileExW(
|
||||
from.as_ptr(),
|
||||
to.as_ptr(),
|
||||
MOVEFILE_REPLACE_EXISTING | MOVEFILE_WRITE_THROUGH,
|
||||
)
|
||||
};
|
||||
if ok == 0 {
|
||||
Err(std::io::Error::last_os_error())
|
||||
} else {
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(not(any(unix, windows)))]
|
||||
fn replace(temp: &Path, out: &Path) -> std::io::Result<()> {
|
||||
std::fs::rename(temp, out)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn no_replace_never_overwrites_existing_output() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let temp = dir.path().join("new.part");
|
||||
let out = dir.path().join("out.bin");
|
||||
std::fs::write(&temp, b"new").unwrap();
|
||||
std::fs::write(&out, b"old").unwrap();
|
||||
assert!(commit(&temp, &out, false).is_err());
|
||||
assert_eq!(std::fs::read(&out).unwrap(), b"old");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn overwrite_replaces_existing_output_without_predelete() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let temp = dir.path().join("new.part");
|
||||
let out = dir.path().join("out.bin");
|
||||
std::fs::write(&temp, b"new").unwrap();
|
||||
std::fs::write(&out, b"old").unwrap();
|
||||
commit(&temp, &out, true).unwrap();
|
||||
assert_eq!(std::fs::read(&out).unwrap(), b"new");
|
||||
assert!(!temp.exists());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,147 @@
|
||||
//! `bsk download` — capture one browser download and atomically commit it.
|
||||
|
||||
use std::fs::OpenOptions;
|
||||
use std::io::Write;
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::time::Duration;
|
||||
|
||||
use anyhow::Context;
|
||||
use base64::Engine;
|
||||
use bsk_protocol::Method;
|
||||
use bsk_protocol::tools::{
|
||||
DownloadParams, DownloadResult, TransferChunkParams, TransferChunkResult, TransferIdParams,
|
||||
TransferReleaseResult,
|
||||
};
|
||||
use clap::Args;
|
||||
use uuid::Uuid;
|
||||
|
||||
use crate::cli::atomic_output;
|
||||
use crate::cli::ensure_daemon::ensure_daemon;
|
||||
use crate::cli::error::{CliError, Format};
|
||||
use crate::cli::interaction::split_target;
|
||||
use crate::cli::navigate::parse_timeout_ms;
|
||||
|
||||
#[derive(Debug, Clone, Args)]
|
||||
pub struct DownloadArgs {
|
||||
/// Snapshot ref (`@e3`) or CSS selector for the download trigger.
|
||||
pub target: Option<String>,
|
||||
#[arg(long = "ref")]
|
||||
pub ref_: Option<String>,
|
||||
#[arg(long = "selector")]
|
||||
pub selector: Option<String>,
|
||||
#[arg(long)]
|
||||
pub out: PathBuf,
|
||||
#[arg(long)]
|
||||
pub session: String,
|
||||
#[arg(long = "tab-id")]
|
||||
pub tab_id: Option<i64>,
|
||||
#[arg(long, default_value = "2m", value_parser = parse_timeout_ms)]
|
||||
pub timeout: u32,
|
||||
#[arg(long)]
|
||||
pub overwrite: bool,
|
||||
}
|
||||
|
||||
pub fn dispatch(args: DownloadArgs, format: Format) -> Result<(), CliError> {
|
||||
if args.out.exists() && !args.overwrite {
|
||||
return Err(CliError::Local(anyhow::anyhow!(
|
||||
"output already exists (pass --overwrite to replace it): {}",
|
||||
args.out.display()
|
||||
)));
|
||||
}
|
||||
let info = ensure_daemon().context("ensure daemon is running")?;
|
||||
let (ref_, selector) = split_target(args.target, args.ref_, args.selector)?;
|
||||
let params = DownloadParams {
|
||||
session_id: args.session,
|
||||
ref_,
|
||||
selector,
|
||||
tab_id: args.tab_id,
|
||||
timeout_ms: Some(args.timeout),
|
||||
browser_relative_dir: None,
|
||||
max_byte_size: None,
|
||||
};
|
||||
let reply: DownloadResult = crate::cli::business_rpc::call(
|
||||
info.sock_path.clone(),
|
||||
"download",
|
||||
Method::ToolDownload,
|
||||
Some(params),
|
||||
ipc_timeout(args.timeout),
|
||||
)?;
|
||||
let transfer_id = reply.transfer_id.clone().ok_or_else(|| {
|
||||
CliError::Local(anyhow::anyhow!("daemon returned no download transfer id"))
|
||||
})?;
|
||||
let write_result = write_transfer(&info.sock_path, &transfer_id, &args.out, args.overwrite);
|
||||
let _: Result<TransferReleaseResult, CliError> = crate::cli::business_rpc::call(
|
||||
info.sock_path,
|
||||
"transfer-release",
|
||||
Method::TransferRelease,
|
||||
Some(TransferIdParams { transfer_id }),
|
||||
Duration::from_secs(5),
|
||||
);
|
||||
write_result?;
|
||||
match format {
|
||||
Format::Json => {
|
||||
let mut value = serde_json::to_value(&reply).unwrap();
|
||||
value["path"] = serde_json::json!(args.out.to_string_lossy());
|
||||
if let Some(obj) = value.as_object_mut() {
|
||||
obj.remove("transfer_id");
|
||||
obj.remove("browser_path");
|
||||
}
|
||||
println!("{}", serde_json::to_string_pretty(&value).unwrap());
|
||||
}
|
||||
Format::Human => println!("{}", args.out.display()),
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn write_transfer(sock: &Path, id: &str, out: &Path, overwrite: bool) -> Result<(), CliError> {
|
||||
let parent = out
|
||||
.parent()
|
||||
.filter(|p| !p.as_os_str().is_empty())
|
||||
.unwrap_or(Path::new("."));
|
||||
let temp = parent.join(format!(".bsk-download-{}.part", Uuid::new_v4().simple()));
|
||||
let result = (|| {
|
||||
let mut file = OpenOptions::new()
|
||||
.write(true)
|
||||
.create_new(true)
|
||||
.open(&temp)
|
||||
.with_context(|| format!("create temporary download {}", temp.display()))
|
||||
.map_err(CliError::Local)?;
|
||||
let mut offset = 0u64;
|
||||
loop {
|
||||
let chunk: TransferChunkResult = crate::cli::business_rpc::call(
|
||||
sock.to_path_buf(),
|
||||
"transfer-read",
|
||||
Method::TransferRead,
|
||||
Some(TransferChunkParams {
|
||||
transfer_id: id.to_string(),
|
||||
offset,
|
||||
data_base64: String::new(),
|
||||
}),
|
||||
Duration::from_secs(30),
|
||||
)?;
|
||||
let encoded = chunk.data_base64.as_deref().unwrap_or("");
|
||||
let bytes = base64::engine::general_purpose::STANDARD
|
||||
.decode(encoded)
|
||||
.map_err(|e| CliError::Local(anyhow::anyhow!("decode download chunk: {e}")))?;
|
||||
file.write_all(&bytes)
|
||||
.map_err(|e| CliError::Local(e.into()))?;
|
||||
offset = chunk.next_offset;
|
||||
if chunk.eof {
|
||||
break;
|
||||
}
|
||||
}
|
||||
file.sync_all().map_err(|e| CliError::Local(e.into()))?;
|
||||
drop(file);
|
||||
atomic_output::commit(&temp, out, overwrite)
|
||||
.with_context(|| format!("atomically commit download to {}", out.display()))
|
||||
.map_err(CliError::Local)
|
||||
})();
|
||||
if result.is_err() {
|
||||
let _ = std::fs::remove_file(&temp);
|
||||
}
|
||||
result
|
||||
}
|
||||
|
||||
fn ipc_timeout(timeout_ms: u32) -> Duration {
|
||||
Duration::from_millis(u64::from(timeout_ms) + 5_000)
|
||||
}
|
||||
@@ -71,7 +71,7 @@ pub(crate) fn parse_modifiers(input: &str) -> Result<Vec<KeyModifier>, String> {
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
fn split_target(
|
||||
pub(crate) fn split_target(
|
||||
positional: Option<String>,
|
||||
explicit_ref: Option<String>,
|
||||
explicit_selector: Option<String>,
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
|
||||
use std::time::Duration;
|
||||
|
||||
mod atomic_output;
|
||||
pub mod browser_wait;
|
||||
pub mod browsers;
|
||||
pub mod business_rpc;
|
||||
@@ -9,6 +10,7 @@ pub mod console;
|
||||
pub mod daemon;
|
||||
pub mod dialogs;
|
||||
pub mod doctor;
|
||||
pub mod download;
|
||||
pub mod emulate;
|
||||
pub mod ensure_daemon;
|
||||
pub mod error;
|
||||
@@ -30,6 +32,7 @@ pub mod snapshot;
|
||||
pub mod status;
|
||||
pub mod tab;
|
||||
pub mod update;
|
||||
pub mod upload;
|
||||
pub mod waits;
|
||||
pub mod window;
|
||||
|
||||
@@ -37,6 +40,7 @@ use clap::{Args, Parser, Subcommand};
|
||||
|
||||
use crate::cli::console::ConsoleArgs;
|
||||
use crate::cli::daemon::DaemonCmd;
|
||||
use crate::cli::download::DownloadArgs;
|
||||
use crate::cli::emulate::EmulateArgs;
|
||||
use crate::cli::evaluate::EvaluateArgs;
|
||||
use crate::cli::get_html::GetHtmlArgs;
|
||||
@@ -52,6 +56,7 @@ use crate::cli::session::SessionCmd;
|
||||
use crate::cli::snapshot::SnapshotArgs;
|
||||
use crate::cli::tab::TabCmd;
|
||||
use crate::cli::update::UpdateArgs;
|
||||
use crate::cli::upload::UploadArgs;
|
||||
use crate::cli::waits::{WaitForNavigationArgs, WaitMsArgs};
|
||||
use crate::cli::window::WindowCmd;
|
||||
|
||||
@@ -176,6 +181,12 @@ pub enum Command {
|
||||
/// Set `<select>` option values by `value` attribute.
|
||||
Select(SelectArgs),
|
||||
|
||||
/// Upload one or more local files through a page file chooser.
|
||||
Upload(UploadArgs),
|
||||
|
||||
/// Capture one browser download and write it to a local path.
|
||||
Download(DownloadArgs),
|
||||
|
||||
/// Evaluate a JavaScript expression inside the Agent Window.
|
||||
Evaluate(EvaluateArgs),
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@ use anyhow::Context;
|
||||
use bsk_protocol::Method;
|
||||
use bsk_protocol::tools::{
|
||||
RecordAwaitParams, RecordAwaitResult, RecordStartParams, RecordStartResult, RecordStopParams,
|
||||
RecordStopResult, RecordedTrace,
|
||||
RecordStopResult, Trace,
|
||||
};
|
||||
use clap::{Args, Subcommand};
|
||||
|
||||
@@ -190,7 +190,7 @@ fn record_await_ipc_timeout(timeout_ms: u32) -> Duration {
|
||||
.unwrap_or(Duration::from_secs(u64::from(timeout_ms / 1_000) + 15))
|
||||
}
|
||||
|
||||
fn write_trace_file(output: &PathBuf, trace: &RecordedTrace) -> Result<(), CliError> {
|
||||
fn write_trace_file(output: &PathBuf, trace: &Trace) -> Result<(), CliError> {
|
||||
let json = serde_json::to_string_pretty(trace)
|
||||
.context("serialize trace JSON")
|
||||
.map_err(CliError::Local)?;
|
||||
@@ -207,7 +207,7 @@ fn write_trace_file(output: &PathBuf, trace: &RecordedTrace) -> Result<(), CliEr
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn render_finish(trace: &RecordedTrace, output: &PathBuf, format: Format) -> Result<(), CliError> {
|
||||
fn render_finish(trace: &Trace, output: &PathBuf, format: Format) -> Result<(), CliError> {
|
||||
match format {
|
||||
Format::Json => {
|
||||
println!(
|
||||
@@ -221,11 +221,7 @@ fn render_finish(trace: &RecordedTrace, output: &PathBuf, format: Format) -> Res
|
||||
);
|
||||
}
|
||||
Format::Human => {
|
||||
let step_count = match trace {
|
||||
RecordedTrace::V2(trace) => trace.steps.len(),
|
||||
RecordedTrace::V3(trace) => trace.steps.len(),
|
||||
};
|
||||
println!("saved {step_count} steps to {}", output.display());
|
||||
println!("saved {} steps to {}", trace.steps.len(), output.display());
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
|
||||
@@ -49,6 +49,12 @@ pub mod reason {
|
||||
pub const TAB_NOT_ACTIVE: &str = "tab_not_active";
|
||||
pub const BORROW_CONFLICT: &str = "borrow_conflict";
|
||||
pub const SCREENSHOT_CAPTURE_FAILED: &str = "screenshot_capture_failed";
|
||||
pub const FILE_INPUT_PROBE_FAILED: &str = "file_input_probe_failed";
|
||||
pub const FILE_INPUT_NOT_ACTIVATED: &str = "file_input_not_activated";
|
||||
pub const SET_FILE_INPUT_FAILED: &str = "set_file_input_failed";
|
||||
pub const DOWNLOAD_CAPTURE_FAILED: &str = "download_capture_failed";
|
||||
pub const TRANSFER_OUTCOME_UNKNOWN: &str = "transfer_outcome_unknown";
|
||||
pub const TRANSFER_TIMEOUT: &str = "transfer_timeout";
|
||||
pub const SESSION_BUSY: &str = crate::rpc_reason::SESSION_BUSY;
|
||||
}
|
||||
|
||||
@@ -196,6 +202,20 @@ pub fn info_for_error(code: ErrorCode, data: Option<&serde_json::Value>) -> Rend
|
||||
return base;
|
||||
};
|
||||
match (code, reason) {
|
||||
(_, reason::TRANSFER_OUTCOME_UNKNOWN) => RenderInfo {
|
||||
summary: "the file transfer outcome could not be confirmed",
|
||||
hint: Some(
|
||||
"do not retry the transfer: the browser may already have applied it; inspect the page or stop safely",
|
||||
),
|
||||
exit_code: base.exit_code,
|
||||
},
|
||||
(ErrorCode::Timeout, reason::TRANSFER_TIMEOUT) => RenderInfo {
|
||||
summary: "the file transfer timed out after browser dispatch",
|
||||
hint: Some(
|
||||
"do not retry when effect_state is unknown or committed; inspect the page before taking another action",
|
||||
),
|
||||
exit_code: base.exit_code,
|
||||
},
|
||||
(ErrorCode::PermissionDenied, reason::ELEMENT_NOT_VISIBLE) => RenderInfo {
|
||||
summary: "target element has no visible geometry",
|
||||
hint: Some(
|
||||
@@ -269,6 +289,36 @@ pub fn info_for_error(code: ErrorCode, data: Option<&serde_json::Value>) -> Rend
|
||||
),
|
||||
exit_code: base.exit_code,
|
||||
},
|
||||
(ErrorCode::Unsupported, reason::FILE_INPUT_NOT_ACTIVATED) => RenderInfo {
|
||||
summary: "the upload trigger did not activate a file input",
|
||||
hint: Some(
|
||||
"the page may use a non-input picker such as showOpenFilePicker(); do not retry blindly — use `bsk request-help` with the exact original local path, or stop if human help is disabled",
|
||||
),
|
||||
exit_code: base.exit_code,
|
||||
},
|
||||
(ErrorCode::Timeout | ErrorCode::CdpFailed, reason::FILE_INPUT_PROBE_FAILED) => {
|
||||
RenderInfo {
|
||||
summary: "the browser upload transaction could not be established",
|
||||
hint: Some(
|
||||
"do not retry the same upload blindly; use `bsk request-help` when human help is available",
|
||||
),
|
||||
exit_code: base.exit_code,
|
||||
}
|
||||
}
|
||||
(ErrorCode::Timeout | ErrorCode::CdpFailed, reason::SET_FILE_INPUT_FAILED) => RenderInfo {
|
||||
summary: "the browser could not attach the staged file to the input",
|
||||
hint: Some(
|
||||
"check that BrowserSkill has Chrome's 'Allow access to file URLs' permission; otherwise use `bsk request-help`",
|
||||
),
|
||||
exit_code: base.exit_code,
|
||||
},
|
||||
(ErrorCode::CdpFailed, reason::DOWNLOAD_CAPTURE_FAILED) => RenderInfo {
|
||||
summary: "the browser download could not be attributed or completed",
|
||||
hint: Some(
|
||||
"do not retry blindly; the target browser may not expose a reliable download intent signal",
|
||||
),
|
||||
exit_code: base.exit_code,
|
||||
},
|
||||
_ => base,
|
||||
}
|
||||
}
|
||||
@@ -392,6 +442,38 @@ mod tests {
|
||||
assert_eq!(info.exit_code, 3);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn file_transfer_reasons_render_actionable_fallbacks() {
|
||||
let unsupported = serde_json::json!({ "reason": reason::FILE_INPUT_NOT_ACTIVATED });
|
||||
let info = info_for_error(ErrorCode::Unsupported, Some(&unsupported));
|
||||
assert_eq!(
|
||||
info.summary,
|
||||
"the upload trigger did not activate a file input"
|
||||
);
|
||||
assert!(info.hint.unwrap().contains("exact original local path"));
|
||||
|
||||
let control = serde_json::json!({ "reason": reason::FILE_INPUT_PROBE_FAILED });
|
||||
let info = info_for_error(ErrorCode::Timeout, Some(&control));
|
||||
assert!(
|
||||
info.summary
|
||||
.contains("transaction could not be established")
|
||||
);
|
||||
assert!(info.hint.unwrap().contains("do not retry"));
|
||||
|
||||
let download = serde_json::json!({ "reason": reason::DOWNLOAD_CAPTURE_FAILED });
|
||||
let info = info_for_error(ErrorCode::CdpFailed, Some(&download));
|
||||
assert!(info.summary.contains("download could not be attributed"));
|
||||
|
||||
let unknown = serde_json::json!({ "reason": reason::TRANSFER_OUTCOME_UNKNOWN });
|
||||
let info = info_for_error(ErrorCode::ProtocolError, Some(&unknown));
|
||||
assert!(info.summary.contains("outcome could not be confirmed"));
|
||||
assert!(info.hint.unwrap().contains("do not retry"));
|
||||
|
||||
let timeout = serde_json::json!({ "reason": reason::TRANSFER_TIMEOUT });
|
||||
let info = info_for_error(ErrorCode::Timeout, Some(&timeout));
|
||||
assert!(info.summary.contains("timed out after browser dispatch"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ref_not_found_overrides_not_found_copy() {
|
||||
let data = serde_json::json!({ "reason": reason::REF_NOT_FOUND });
|
||||
|
||||
@@ -0,0 +1,185 @@
|
||||
//! `bsk upload` — stage caller-readable files and attach them to a page.
|
||||
|
||||
use std::fs::File;
|
||||
use std::io::Read;
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::time::Duration;
|
||||
|
||||
use anyhow::Context;
|
||||
use base64::Engine;
|
||||
use bsk_protocol::Method;
|
||||
use bsk_protocol::tools::{
|
||||
TransferBeginParams, TransferBeginResult, TransferChunkParams, TransferChunkResult,
|
||||
TransferIdParams, TransferReadyResult, TransferReleaseResult, UploadFile, UploadParams,
|
||||
UploadResult,
|
||||
};
|
||||
use clap::Args;
|
||||
|
||||
use crate::cli::ensure_daemon::ensure_daemon;
|
||||
use crate::cli::error::{CliError, Format};
|
||||
use crate::cli::interaction::split_target;
|
||||
use crate::cli::navigate::parse_timeout_ms;
|
||||
|
||||
#[derive(Debug, Clone, Args)]
|
||||
pub struct UploadArgs {
|
||||
/// Snapshot ref (`@e3`) or CSS selector for the file input / chooser trigger.
|
||||
pub target: Option<String>,
|
||||
#[arg(long = "ref")]
|
||||
pub ref_: Option<String>,
|
||||
#[arg(long = "selector")]
|
||||
pub selector: Option<String>,
|
||||
/// Local file to upload. Repeat for a multiple-file input.
|
||||
#[arg(long = "file", required = true)]
|
||||
pub files: Vec<PathBuf>,
|
||||
#[arg(long)]
|
||||
pub session: String,
|
||||
#[arg(long = "tab-id")]
|
||||
pub tab_id: Option<i64>,
|
||||
#[arg(long, default_value = "2m", value_parser = parse_timeout_ms)]
|
||||
pub timeout: u32,
|
||||
}
|
||||
|
||||
pub fn dispatch(args: UploadArgs, format: Format) -> Result<(), CliError> {
|
||||
if args.files.len() > crate::daemon::file_transfer::MAX_UPLOAD_FILES {
|
||||
return Err(CliError::Local(anyhow::anyhow!(
|
||||
"upload accepts at most {} files",
|
||||
crate::daemon::file_transfer::MAX_UPLOAD_FILES
|
||||
)));
|
||||
}
|
||||
let info = ensure_daemon().context("ensure daemon is running")?;
|
||||
let (ref_, selector) = split_target(args.target, args.ref_, args.selector)?;
|
||||
let mut staged = Vec::new();
|
||||
for path in &args.files {
|
||||
match stage_file(&info.sock_path, &args.session, path) {
|
||||
Ok(file) => staged.push(file),
|
||||
Err(err) => {
|
||||
for (id, _) in &staged {
|
||||
let _ = release(&info.sock_path, id);
|
||||
}
|
||||
return Err(err);
|
||||
}
|
||||
}
|
||||
}
|
||||
let params = UploadParams {
|
||||
session_id: args.session,
|
||||
ref_,
|
||||
selector,
|
||||
tab_id: args.tab_id,
|
||||
files: staged
|
||||
.iter()
|
||||
.map(|(id, name)| UploadFile {
|
||||
transfer_id: id.clone(),
|
||||
name: name.clone(),
|
||||
staged_path: None,
|
||||
})
|
||||
.collect(),
|
||||
timeout_ms: Some(args.timeout),
|
||||
};
|
||||
let result = crate::cli::business_rpc::call::<_, UploadResult>(
|
||||
info.sock_path.clone(),
|
||||
"upload",
|
||||
Method::ToolUpload,
|
||||
Some(params),
|
||||
ipc_timeout(args.timeout),
|
||||
);
|
||||
// Once tool.upload is dispatched, staging ownership belongs to the
|
||||
// session. A transport timeout cannot prove that Chrome did not attach
|
||||
// the file, so releasing here could invalidate a late successful attach.
|
||||
// Session teardown remains the single cleanup boundary after dispatch.
|
||||
let reply = result?;
|
||||
match format {
|
||||
Format::Json => println!("{}", serde_json::to_string_pretty(&reply).unwrap()),
|
||||
Format::Human => println!(
|
||||
"upload ok tab={} files={}",
|
||||
reply.tab_id,
|
||||
reply.file_names.join(", ")
|
||||
),
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn stage_file(sock: &Path, session: &str, path: &PathBuf) -> Result<(String, String), CliError> {
|
||||
let mut file = File::open(path)
|
||||
.with_context(|| format!("open upload file {}", path.display()))
|
||||
.map_err(CliError::Local)?;
|
||||
let meta = file
|
||||
.metadata()
|
||||
.with_context(|| format!("read upload file metadata {}", path.display()))
|
||||
.map_err(CliError::Local)?;
|
||||
if !meta.is_file() {
|
||||
return Err(CliError::Local(anyhow::anyhow!(
|
||||
"upload source is not a regular file: {}",
|
||||
path.display()
|
||||
)));
|
||||
}
|
||||
let name = path
|
||||
.file_name()
|
||||
.and_then(|v| v.to_str())
|
||||
.filter(|v| !v.is_empty())
|
||||
.ok_or_else(|| CliError::Local(anyhow::anyhow!("upload file has no valid basename")))?
|
||||
.to_string();
|
||||
let begin: TransferBeginResult = crate::cli::business_rpc::call(
|
||||
sock.to_path_buf(),
|
||||
"transfer-begin",
|
||||
Method::TransferBegin,
|
||||
Some(TransferBeginParams {
|
||||
session_id: session.to_string(),
|
||||
name: name.clone(),
|
||||
byte_size: meta.len(),
|
||||
}),
|
||||
Duration::from_secs(10),
|
||||
)?;
|
||||
let staged = (|| {
|
||||
let mut offset = 0u64;
|
||||
let mut buf = vec![0u8; begin.chunk_size as usize];
|
||||
loop {
|
||||
let n = file.read(&mut buf).map_err(|e| CliError::Local(e.into()))?;
|
||||
if n == 0 {
|
||||
break;
|
||||
}
|
||||
let reply: TransferChunkResult = crate::cli::business_rpc::call(
|
||||
sock.to_path_buf(),
|
||||
"transfer-chunk",
|
||||
Method::TransferChunk,
|
||||
Some(TransferChunkParams {
|
||||
transfer_id: begin.transfer_id.clone(),
|
||||
offset,
|
||||
data_base64: base64::engine::general_purpose::STANDARD.encode(&buf[..n]),
|
||||
}),
|
||||
Duration::from_secs(30),
|
||||
)?;
|
||||
offset = reply.next_offset;
|
||||
}
|
||||
let _: TransferReadyResult = crate::cli::business_rpc::call(
|
||||
sock.to_path_buf(),
|
||||
"transfer-finish",
|
||||
Method::TransferFinish,
|
||||
Some(TransferIdParams {
|
||||
transfer_id: begin.transfer_id.clone(),
|
||||
}),
|
||||
Duration::from_secs(10),
|
||||
)?;
|
||||
Ok::<_, CliError>(())
|
||||
})();
|
||||
if let Err(err) = staged {
|
||||
let _ = release(sock, &begin.transfer_id);
|
||||
return Err(err);
|
||||
}
|
||||
Ok((begin.transfer_id, name))
|
||||
}
|
||||
|
||||
fn release(sock: &Path, id: &str) -> Result<TransferReleaseResult, CliError> {
|
||||
crate::cli::business_rpc::call(
|
||||
sock.to_path_buf(),
|
||||
"transfer-release",
|
||||
Method::TransferRelease,
|
||||
Some(TransferIdParams {
|
||||
transfer_id: id.to_string(),
|
||||
}),
|
||||
Duration::from_secs(5),
|
||||
)
|
||||
}
|
||||
|
||||
fn ipc_timeout(timeout_ms: u32) -> Duration {
|
||||
Duration::from_millis(u64::from(timeout_ms) + 5_000)
|
||||
}
|
||||
@@ -0,0 +1,728 @@
|
||||
//! Session-scoped, content-transparent staging for upload/download tools.
|
||||
//!
|
||||
//! The registry deliberately exposes no arbitrary-path operations. Upload
|
||||
//! bytes arrive from the invoking CLI in bounded chunks; browser downloads
|
||||
//! land in a daemon-minted directory and are read back by the invoking CLI.
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::fs::{self, File, OpenOptions};
|
||||
use std::io::{Read, Seek, SeekFrom, Write};
|
||||
use std::path::{Component, Path, PathBuf};
|
||||
use std::sync::Mutex;
|
||||
|
||||
use base64::Engine;
|
||||
use bsk_protocol::tools::{
|
||||
TransferBeginParams, TransferBeginResult, TransferChunkParams, TransferChunkResult,
|
||||
TransferIdParams, TransferReadyResult, TransferReleaseResult,
|
||||
};
|
||||
use bsk_protocol::{ErrorCode, RpcError};
|
||||
use uuid::Uuid;
|
||||
|
||||
use super::paths;
|
||||
|
||||
pub const TRANSFER_CHUNK_SIZE: u32 = 512 * 1024;
|
||||
pub const MAX_TRANSFER_BYTES: u64 = 512 * 1024 * 1024;
|
||||
pub const MAX_UPLOAD_FILES: usize = 20;
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
enum Direction {
|
||||
Upload,
|
||||
Download,
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
struct Entry {
|
||||
session_id: String,
|
||||
direction: Direction,
|
||||
path: PathBuf,
|
||||
expected_size: Option<u64>,
|
||||
written: u64,
|
||||
ready: bool,
|
||||
}
|
||||
|
||||
/// Browser download path after the daemon has verified that it belongs to
|
||||
/// the capability minted for this transfer. From this point onward the daemon
|
||||
/// owns cleanup on every error path; unvalidated arbitrary paths are never
|
||||
/// removed.
|
||||
#[derive(Debug)]
|
||||
struct ValidatedBrowserFile {
|
||||
path: PathBuf,
|
||||
parent: PathBuf,
|
||||
cleanup_armed: bool,
|
||||
}
|
||||
|
||||
impl ValidatedBrowserFile {
|
||||
fn validate(reported_path: &Path, transfer_id: &str) -> Result<Self, RpcError> {
|
||||
if !reported_path.is_absolute() {
|
||||
return Err(permission(
|
||||
"download path is outside its browser capability",
|
||||
));
|
||||
}
|
||||
let expected_parent = Path::new("BrowserSkill").join(transfer_id);
|
||||
let parent = reported_path
|
||||
.parent()
|
||||
.ok_or_else(|| permission("download path has no parent directory"))?;
|
||||
if !parent.ends_with(&expected_parent) {
|
||||
return Err(permission(
|
||||
"download escaped its browser capability directory",
|
||||
));
|
||||
}
|
||||
for path in [reported_path, parent] {
|
||||
if fs::symlink_metadata(path)
|
||||
.map_err(io_error)?
|
||||
.file_type()
|
||||
.is_symlink()
|
||||
{
|
||||
return Err(permission("download capability path contains a symlink"));
|
||||
}
|
||||
}
|
||||
Ok(Self {
|
||||
path: reported_path.to_path_buf(),
|
||||
parent: parent.to_path_buf(),
|
||||
cleanup_armed: true,
|
||||
})
|
||||
}
|
||||
|
||||
fn remove_source(mut self) -> Result<(), RpcError> {
|
||||
remove_if_present(&self.path).map_err(io_error)?;
|
||||
remove_dir_if_present(&self.parent).map_err(io_error)?;
|
||||
self.cleanup_armed = false;
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for ValidatedBrowserFile {
|
||||
fn drop(&mut self) {
|
||||
if !self.cleanup_armed {
|
||||
return;
|
||||
}
|
||||
let _ = remove_if_present(&self.path);
|
||||
let _ = remove_dir_if_present(&self.parent);
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
pub struct DownloadStaging {
|
||||
pub transfer_id: String,
|
||||
pub browser_relative_dir: String,
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
pub struct TransferRegistry {
|
||||
root: PathBuf,
|
||||
initialized: Mutex<bool>,
|
||||
entries: Mutex<HashMap<String, Entry>>,
|
||||
}
|
||||
|
||||
impl TransferRegistry {
|
||||
pub fn new() -> anyhow::Result<Self> {
|
||||
Ok(Self {
|
||||
root: paths::bsk_home()?.join("run").join("transfers"),
|
||||
initialized: Mutex::new(false),
|
||||
entries: Mutex::new(HashMap::new()),
|
||||
})
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
fn at_root(root: PathBuf) -> anyhow::Result<Self> {
|
||||
let registry = Self {
|
||||
root,
|
||||
initialized: Mutex::new(false),
|
||||
entries: Mutex::new(HashMap::new()),
|
||||
};
|
||||
registry.ensure_root()?;
|
||||
Ok(registry)
|
||||
}
|
||||
|
||||
pub fn initialize(&self) -> anyhow::Result<()> {
|
||||
self.ensure_root()
|
||||
}
|
||||
|
||||
fn ensure_root(&self) -> anyhow::Result<()> {
|
||||
let mut initialized = self.initialized.lock().unwrap();
|
||||
if *initialized {
|
||||
return Ok(());
|
||||
}
|
||||
if self.root.exists() {
|
||||
// Transfers never survive a daemon; removing stale directories on
|
||||
// startup gives crash cleanup without a second persistence model.
|
||||
let _ = fs::remove_dir_all(&self.root);
|
||||
}
|
||||
fs::create_dir_all(&self.root)?;
|
||||
#[cfg(unix)]
|
||||
{
|
||||
use std::os::unix::fs::PermissionsExt;
|
||||
fs::set_permissions(&self.root, fs::Permissions::from_mode(0o700))?;
|
||||
}
|
||||
*initialized = true;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub fn begin_upload(&self, p: TransferBeginParams) -> Result<TransferBeginResult, RpcError> {
|
||||
self.ensure_root()
|
||||
.map_err(|err| protocol_error(err.to_string()))?;
|
||||
let basename = safe_upload_basename(&p.name)?;
|
||||
if p.byte_size > MAX_TRANSFER_BYTES {
|
||||
return Err(invalid(format!(
|
||||
"file size {} exceeds transfer limit {}",
|
||||
p.byte_size, MAX_TRANSFER_BYTES
|
||||
)));
|
||||
}
|
||||
let mut entries = self.entries.lock().unwrap();
|
||||
let session_bytes: u64 = entries
|
||||
.values()
|
||||
.filter(|entry| entry.session_id == p.session_id)
|
||||
.map(|entry| entry.expected_size.unwrap_or(entry.written))
|
||||
.sum();
|
||||
if session_bytes.saturating_add(p.byte_size) > MAX_TRANSFER_BYTES {
|
||||
return Err(invalid(format!(
|
||||
"session transfer staging exceeds limit {}",
|
||||
MAX_TRANSFER_BYTES
|
||||
)));
|
||||
}
|
||||
let id = new_id();
|
||||
let dir = self.root.join(&id);
|
||||
fs::create_dir(&dir).map_err(io_error)?;
|
||||
set_private_dir(&dir).map_err(io_error)?;
|
||||
let path = dir.join(basename);
|
||||
let file = OpenOptions::new()
|
||||
.write(true)
|
||||
.create_new(true)
|
||||
.open(&path)
|
||||
.map_err(io_error)?;
|
||||
set_private_file(&file).map_err(io_error)?;
|
||||
entries.insert(
|
||||
id.clone(),
|
||||
Entry {
|
||||
session_id: p.session_id,
|
||||
direction: Direction::Upload,
|
||||
path,
|
||||
expected_size: Some(p.byte_size),
|
||||
written: 0,
|
||||
ready: false,
|
||||
},
|
||||
);
|
||||
Ok(TransferBeginResult {
|
||||
transfer_id: id,
|
||||
chunk_size: TRANSFER_CHUNK_SIZE,
|
||||
})
|
||||
}
|
||||
|
||||
pub fn write_chunk(&self, p: TransferChunkParams) -> Result<TransferChunkResult, RpcError> {
|
||||
let bytes = base64::engine::general_purpose::STANDARD
|
||||
.decode(p.data_base64)
|
||||
.map_err(|e| invalid(format!("invalid transfer chunk: {e}")))?;
|
||||
if bytes.len() > TRANSFER_CHUNK_SIZE as usize {
|
||||
return Err(invalid("transfer chunk exceeds negotiated size"));
|
||||
}
|
||||
let mut entries = self.entries.lock().unwrap();
|
||||
let entry = entries
|
||||
.get_mut(&p.transfer_id)
|
||||
.ok_or_else(|| not_found("transfer not found"))?;
|
||||
if entry.direction != Direction::Upload || entry.ready {
|
||||
return Err(invalid("transfer is not writable"));
|
||||
}
|
||||
if p.offset != entry.written {
|
||||
return Err(invalid(format!(
|
||||
"non-sequential transfer offset: expected {}, got {}",
|
||||
entry.written, p.offset
|
||||
)));
|
||||
}
|
||||
let next = entry.written.saturating_add(bytes.len() as u64);
|
||||
if next > entry.expected_size.unwrap_or(MAX_TRANSFER_BYTES) {
|
||||
return Err(invalid("transfer exceeds declared byte size"));
|
||||
}
|
||||
let mut file = OpenOptions::new()
|
||||
.append(true)
|
||||
.open(&entry.path)
|
||||
.map_err(io_error)?;
|
||||
file.write_all(&bytes).map_err(io_error)?;
|
||||
entry.written = next;
|
||||
Ok(TransferChunkResult {
|
||||
next_offset: next,
|
||||
eof: false,
|
||||
data_base64: None,
|
||||
})
|
||||
}
|
||||
|
||||
pub fn finish_upload(&self, p: TransferIdParams) -> Result<TransferReadyResult, RpcError> {
|
||||
let mut entries = self.entries.lock().unwrap();
|
||||
let entry = entries
|
||||
.get_mut(&p.transfer_id)
|
||||
.ok_or_else(|| not_found("transfer not found"))?;
|
||||
if entry.direction != Direction::Upload {
|
||||
return Err(invalid("transfer is not an upload"));
|
||||
}
|
||||
if entry.written != entry.expected_size.unwrap_or(entry.written) {
|
||||
return Err(invalid(format!(
|
||||
"incomplete transfer: expected {} bytes, received {}",
|
||||
entry.expected_size.unwrap_or(0),
|
||||
entry.written
|
||||
)));
|
||||
}
|
||||
entry.ready = true;
|
||||
Ok(TransferReadyResult {
|
||||
transfer_id: p.transfer_id,
|
||||
byte_size: entry.written,
|
||||
})
|
||||
}
|
||||
|
||||
pub fn resolve_uploads(
|
||||
&self,
|
||||
session_id: &str,
|
||||
ids: &[String],
|
||||
) -> Result<Vec<PathBuf>, RpcError> {
|
||||
let entries = self.entries.lock().unwrap();
|
||||
ids.iter()
|
||||
.map(|id| {
|
||||
let entry = entries
|
||||
.get(id)
|
||||
.ok_or_else(|| not_found("upload transfer not found"))?;
|
||||
if entry.session_id != session_id
|
||||
|| entry.direction != Direction::Upload
|
||||
|| !entry.ready
|
||||
{
|
||||
return Err(permission(
|
||||
"upload transfer is outside this session or not ready",
|
||||
));
|
||||
}
|
||||
Ok(entry.path.clone())
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
pub fn begin_download(&self, session_id: &str) -> Result<DownloadStaging, RpcError> {
|
||||
self.ensure_root()
|
||||
.map_err(|err| protocol_error(err.to_string()))?;
|
||||
let id = new_id();
|
||||
let dir = self.root.join(&id);
|
||||
fs::create_dir(&dir).map_err(io_error)?;
|
||||
set_private_dir(&dir).map_err(io_error)?;
|
||||
self.entries.lock().unwrap().insert(
|
||||
id.clone(),
|
||||
Entry {
|
||||
session_id: session_id.to_string(),
|
||||
direction: Direction::Download,
|
||||
path: dir.clone(),
|
||||
expected_size: None,
|
||||
written: 0,
|
||||
ready: false,
|
||||
},
|
||||
);
|
||||
Ok(DownloadStaging {
|
||||
browser_relative_dir: format!("BrowserSkill/{id}"),
|
||||
transfer_id: id,
|
||||
})
|
||||
}
|
||||
|
||||
pub fn import_download(&self, id: &str, reported_path: &Path) -> Result<u64, RpcError> {
|
||||
let mut entries = self.entries.lock().unwrap();
|
||||
let entry = entries
|
||||
.get_mut(id)
|
||||
.ok_or_else(|| not_found("download transfer not found"))?;
|
||||
if entry.direction != Direction::Download {
|
||||
return Err(permission("transfer is not a download capability"));
|
||||
}
|
||||
let browser_file = ValidatedBrowserFile::validate(reported_path, id)?;
|
||||
let canonical = browser_file.path.canonicalize().map_err(io_error)?;
|
||||
let meta = fs::metadata(&canonical).map_err(io_error)?;
|
||||
if !meta.is_file() || meta.len() > MAX_TRANSFER_BYTES {
|
||||
return Err(invalid(
|
||||
"download is not a regular file or exceeds the transfer limit",
|
||||
));
|
||||
}
|
||||
|
||||
let destination = entry.path.join("payload");
|
||||
let source = File::open(&canonical).map_err(io_error)?;
|
||||
let mut destination_file = OpenOptions::new()
|
||||
.write(true)
|
||||
.create_new(true)
|
||||
.open(&destination)
|
||||
.map_err(io_error)?;
|
||||
let copied = std::io::copy(
|
||||
&mut source.take(MAX_TRANSFER_BYTES.saturating_add(1)),
|
||||
&mut destination_file,
|
||||
)
|
||||
.map_err(io_error)?;
|
||||
if copied > MAX_TRANSFER_BYTES {
|
||||
let _ = fs::remove_file(&destination);
|
||||
return Err(invalid("download exceeds the transfer limit"));
|
||||
}
|
||||
entry.path = destination;
|
||||
entry.written = copied;
|
||||
entry.expected_size = Some(copied);
|
||||
entry.ready = true;
|
||||
browser_file.remove_source()?;
|
||||
Ok(copied)
|
||||
}
|
||||
|
||||
pub fn read_chunk(&self, p: TransferChunkParams) -> Result<TransferChunkResult, RpcError> {
|
||||
let entries = self.entries.lock().unwrap();
|
||||
let entry = entries
|
||||
.get(&p.transfer_id)
|
||||
.ok_or_else(|| not_found("transfer not found"))?;
|
||||
if entry.direction != Direction::Download || !entry.ready {
|
||||
return Err(invalid("transfer is not readable"));
|
||||
}
|
||||
if p.offset > entry.written {
|
||||
return Err(invalid("read offset is beyond transfer length"));
|
||||
}
|
||||
let mut file = File::open(&entry.path).map_err(io_error)?;
|
||||
file.seek(SeekFrom::Start(p.offset)).map_err(io_error)?;
|
||||
let mut buf = vec![0u8; TRANSFER_CHUNK_SIZE as usize];
|
||||
let count = file.read(&mut buf).map_err(io_error)?;
|
||||
buf.truncate(count);
|
||||
let next = p.offset + count as u64;
|
||||
Ok(TransferChunkResult {
|
||||
next_offset: next,
|
||||
eof: next >= entry.written,
|
||||
data_base64: Some(base64::engine::general_purpose::STANDARD.encode(buf)),
|
||||
})
|
||||
}
|
||||
|
||||
pub fn release(&self, p: TransferIdParams) -> TransferReleaseResult {
|
||||
let entry = self.entries.lock().unwrap().remove(&p.transfer_id);
|
||||
if let Some(entry) = entry {
|
||||
let dir = if entry.path.is_dir() {
|
||||
entry.path
|
||||
} else {
|
||||
entry.path.parent().unwrap_or(&self.root).to_path_buf()
|
||||
};
|
||||
let _ = fs::remove_dir_all(dir);
|
||||
TransferReleaseResult { released: true }
|
||||
} else {
|
||||
TransferReleaseResult { released: false }
|
||||
}
|
||||
}
|
||||
|
||||
pub fn release_session(&self, session_id: &str) {
|
||||
let ids: Vec<String> = self
|
||||
.entries
|
||||
.lock()
|
||||
.unwrap()
|
||||
.iter()
|
||||
.filter(|(_, entry)| entry.session_id == session_id)
|
||||
.map(|(id, _)| id.clone())
|
||||
.collect();
|
||||
for id in ids {
|
||||
self.release(TransferIdParams { transfer_id: id });
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn new_id() -> String {
|
||||
format!("tr_{}", Uuid::new_v4().simple())
|
||||
}
|
||||
|
||||
fn set_private_dir(path: &Path) -> std::io::Result<()> {
|
||||
#[cfg(unix)]
|
||||
{
|
||||
use std::os::unix::fs::PermissionsExt;
|
||||
fs::set_permissions(path, fs::Permissions::from_mode(0o700))?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn set_private_file(file: &File) -> std::io::Result<()> {
|
||||
#[cfg(unix)]
|
||||
{
|
||||
use std::os::unix::fs::PermissionsExt;
|
||||
file.set_permissions(fs::Permissions::from_mode(0o600))?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn remove_if_present(path: &Path) -> std::io::Result<()> {
|
||||
match fs::remove_file(path) {
|
||||
Ok(()) => Ok(()),
|
||||
Err(err) if err.kind() == std::io::ErrorKind::NotFound => Ok(()),
|
||||
Err(err) => Err(err),
|
||||
}
|
||||
}
|
||||
|
||||
fn remove_dir_if_present(path: &Path) -> std::io::Result<()> {
|
||||
match fs::remove_dir(path) {
|
||||
Ok(()) => Ok(()),
|
||||
Err(err) if err.kind() == std::io::ErrorKind::NotFound => Ok(()),
|
||||
Err(err) => Err(err),
|
||||
}
|
||||
}
|
||||
|
||||
fn safe_upload_basename(name: &str) -> Result<&str, RpcError> {
|
||||
if name.is_empty() || name.contains(['/', '\\', '\0']) {
|
||||
return Err(invalid("upload name must be a safe basename"));
|
||||
}
|
||||
let mut components = Path::new(name).components();
|
||||
match (components.next(), components.next()) {
|
||||
(Some(Component::Normal(_)), None) => Ok(name),
|
||||
_ => Err(invalid("upload name must be a safe basename")),
|
||||
}
|
||||
}
|
||||
|
||||
fn invalid(message: impl Into<String>) -> RpcError {
|
||||
RpcError {
|
||||
code: ErrorCode::InvalidParams,
|
||||
message: message.into(),
|
||||
data: None,
|
||||
}
|
||||
}
|
||||
fn not_found(message: impl Into<String>) -> RpcError {
|
||||
RpcError {
|
||||
code: ErrorCode::NotFound,
|
||||
message: message.into(),
|
||||
data: None,
|
||||
}
|
||||
}
|
||||
fn permission(message: impl Into<String>) -> RpcError {
|
||||
RpcError {
|
||||
code: ErrorCode::PermissionDenied,
|
||||
message: message.into(),
|
||||
data: None,
|
||||
}
|
||||
}
|
||||
fn io_error(err: std::io::Error) -> RpcError {
|
||||
RpcError {
|
||||
code: ErrorCode::ProtocolError,
|
||||
message: err.to_string(),
|
||||
data: None,
|
||||
}
|
||||
}
|
||||
|
||||
fn protocol_error(message: impl Into<String>) -> RpcError {
|
||||
RpcError {
|
||||
code: ErrorCode::ProtocolError,
|
||||
message: message.into(),
|
||||
data: None,
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn registry() -> (tempfile::TempDir, TransferRegistry) {
|
||||
let temp = tempfile::tempdir().unwrap();
|
||||
let registry = TransferRegistry::at_root(temp.path().join("transfers")).unwrap();
|
||||
(temp, registry)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn upload_is_not_resolvable_until_complete_or_from_another_session() {
|
||||
let (_temp, registry) = registry();
|
||||
let begin = registry
|
||||
.begin_upload(TransferBeginParams {
|
||||
session_id: "s1".into(),
|
||||
name: "image.png".into(),
|
||||
byte_size: 3,
|
||||
})
|
||||
.unwrap();
|
||||
assert!(
|
||||
registry
|
||||
.resolve_uploads("s1", std::slice::from_ref(&begin.transfer_id))
|
||||
.is_err()
|
||||
);
|
||||
registry
|
||||
.write_chunk(TransferChunkParams {
|
||||
transfer_id: begin.transfer_id.clone(),
|
||||
offset: 0,
|
||||
data_base64: base64::engine::general_purpose::STANDARD.encode(b"abc"),
|
||||
})
|
||||
.unwrap();
|
||||
registry
|
||||
.finish_upload(TransferIdParams {
|
||||
transfer_id: begin.transfer_id.clone(),
|
||||
})
|
||||
.unwrap();
|
||||
assert!(
|
||||
registry
|
||||
.resolve_uploads("s2", std::slice::from_ref(&begin.transfer_id))
|
||||
.is_err()
|
||||
);
|
||||
let [path] = registry
|
||||
.resolve_uploads("s1", &[begin.transfer_id])
|
||||
.unwrap()
|
||||
.try_into()
|
||||
.unwrap();
|
||||
assert_eq!(path.file_name().unwrap(), "image.png");
|
||||
assert_eq!(fs::read(path).unwrap(), b"abc");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn upload_name_must_be_one_safe_path_component() {
|
||||
let (_temp, registry) = registry();
|
||||
for name in ["", ".", "..", "../secret", "folder/file", "folder\\file"] {
|
||||
let result = registry.begin_upload(TransferBeginParams {
|
||||
session_id: "s1".into(),
|
||||
name: name.into(),
|
||||
byte_size: 0,
|
||||
});
|
||||
assert!(result.is_err(), "accepted unsafe upload name");
|
||||
}
|
||||
assert!(registry.entries.lock().unwrap().is_empty());
|
||||
assert!(fs::read_dir(®istry.root).unwrap().next().is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn same_upload_names_are_isolated_by_transfer_directory() {
|
||||
let (_temp, registry) = registry();
|
||||
let first = registry
|
||||
.begin_upload(TransferBeginParams {
|
||||
session_id: "s1".into(),
|
||||
name: "image.png".into(),
|
||||
byte_size: 0,
|
||||
})
|
||||
.unwrap();
|
||||
let second = registry
|
||||
.begin_upload(TransferBeginParams {
|
||||
session_id: "s1".into(),
|
||||
name: "image.png".into(),
|
||||
byte_size: 0,
|
||||
})
|
||||
.unwrap();
|
||||
let entries = registry.entries.lock().unwrap();
|
||||
let first_path = &entries[&first.transfer_id].path;
|
||||
let second_path = &entries[&second.transfer_id].path;
|
||||
assert_ne!(first_path.parent(), second_path.parent());
|
||||
assert_eq!(first_path.file_name().unwrap(), "image.png");
|
||||
assert_eq!(second_path.file_name().unwrap(), "image.png");
|
||||
}
|
||||
|
||||
#[cfg(unix)]
|
||||
#[test]
|
||||
fn upload_staging_is_private() {
|
||||
use std::os::unix::fs::PermissionsExt;
|
||||
|
||||
let (_temp, registry) = registry();
|
||||
let begin = registry
|
||||
.begin_upload(TransferBeginParams {
|
||||
session_id: "s1".into(),
|
||||
name: "private.png".into(),
|
||||
byte_size: 0,
|
||||
})
|
||||
.unwrap();
|
||||
let entries = registry.entries.lock().unwrap();
|
||||
let path = &entries[&begin.transfer_id].path;
|
||||
assert_eq!(
|
||||
fs::metadata(path.parent().unwrap())
|
||||
.unwrap()
|
||||
.permissions()
|
||||
.mode()
|
||||
& 0o777,
|
||||
0o700
|
||||
);
|
||||
assert_eq!(
|
||||
fs::metadata(path).unwrap().permissions().mode() & 0o777,
|
||||
0o600
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn download_must_import_from_its_browser_capability_directory() {
|
||||
let (temp, registry) = registry();
|
||||
let staging = registry.begin_download("s1").unwrap();
|
||||
let outside = temp.path().join("outside.bin");
|
||||
fs::write(&outside, b"secret").unwrap();
|
||||
assert!(
|
||||
registry
|
||||
.import_download(&staging.transfer_id, &outside)
|
||||
.is_err()
|
||||
);
|
||||
assert_eq!(fs::read(&outside).unwrap(), b"secret");
|
||||
|
||||
let browser_dir = temp
|
||||
.path()
|
||||
.join("Downloads")
|
||||
.join("BrowserSkill")
|
||||
.join(&staging.transfer_id);
|
||||
fs::create_dir_all(&browser_dir).unwrap();
|
||||
let inside = browser_dir.join("result.bin");
|
||||
fs::write(&inside, b"result").unwrap();
|
||||
assert_eq!(
|
||||
registry
|
||||
.import_download(&staging.transfer_id, &inside)
|
||||
.unwrap(),
|
||||
6
|
||||
);
|
||||
assert!(!inside.exists());
|
||||
let chunk = registry
|
||||
.read_chunk(TransferChunkParams {
|
||||
transfer_id: staging.transfer_id,
|
||||
offset: 0,
|
||||
data_base64: String::new(),
|
||||
})
|
||||
.unwrap();
|
||||
assert_eq!(
|
||||
base64::engine::general_purpose::STANDARD
|
||||
.decode(chunk.data_base64.unwrap())
|
||||
.unwrap(),
|
||||
b"result"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn validated_oversized_browser_download_is_removed_on_rejection() {
|
||||
let (temp, registry) = registry();
|
||||
let staging = registry.begin_download("s1").unwrap();
|
||||
let browser_dir = temp
|
||||
.path()
|
||||
.join("Downloads")
|
||||
.join("BrowserSkill")
|
||||
.join(&staging.transfer_id);
|
||||
fs::create_dir_all(&browser_dir).unwrap();
|
||||
let inside = browser_dir.join("oversized.bin");
|
||||
File::create(&inside)
|
||||
.unwrap()
|
||||
.set_len(MAX_TRANSFER_BYTES + 1)
|
||||
.unwrap();
|
||||
|
||||
assert!(
|
||||
registry
|
||||
.import_download(&staging.transfer_id, &inside)
|
||||
.is_err()
|
||||
);
|
||||
assert!(!inside.exists());
|
||||
assert!(!browser_dir.exists());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn releasing_a_session_removes_all_staging() {
|
||||
let (_temp, registry) = registry();
|
||||
let first = registry.begin_download("s1").unwrap();
|
||||
let second = registry.begin_download("s2").unwrap();
|
||||
let first_directory = registry.root.join(&first.transfer_id);
|
||||
let second_directory = registry.root.join(&second.transfer_id);
|
||||
registry.release_session("s1");
|
||||
assert!(!first_directory.exists());
|
||||
assert!(second_directory.exists());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn upload_staging_is_bounded_across_a_session() {
|
||||
let (_temp, registry) = registry();
|
||||
registry
|
||||
.begin_upload(TransferBeginParams {
|
||||
session_id: "s1".into(),
|
||||
name: "large.bin".into(),
|
||||
byte_size: MAX_TRANSFER_BYTES,
|
||||
})
|
||||
.unwrap();
|
||||
assert!(
|
||||
registry
|
||||
.begin_upload(TransferBeginParams {
|
||||
session_id: "s1".into(),
|
||||
name: "one-more-byte.bin".into(),
|
||||
byte_size: 1,
|
||||
})
|
||||
.is_err()
|
||||
);
|
||||
assert!(
|
||||
registry
|
||||
.begin_upload(TransferBeginParams {
|
||||
session_id: "s2".into(),
|
||||
name: "other-session.bin".into(),
|
||||
byte_size: 1,
|
||||
})
|
||||
.is_ok()
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -27,7 +27,8 @@ use bsk_protocol::system::{
|
||||
VersionSkewEntry,
|
||||
};
|
||||
use bsk_protocol::tools::{
|
||||
HelpOutcome, RequestHelpResult, ReturnFailure, WaitMsParams, WaitMsResult,
|
||||
DownloadParams, DownloadResult, HelpOutcome, RequestHelpResult, ReturnFailure,
|
||||
TransferBeginParams, TransferIdParams, UploadParams, WaitMsParams, WaitMsResult,
|
||||
};
|
||||
use bsk_protocol::{
|
||||
CancelParams, CancelResult, ErrorCode, Method, PingResult, ResponseBody, RpcError, RpcId,
|
||||
@@ -61,6 +62,10 @@ pub type RpcHandler = Arc<
|
||||
>;
|
||||
|
||||
const DEFAULT_RPC_TIMEOUT: Duration = Duration::from_secs(15);
|
||||
// The upload transaction owns its operation deadline and may need a bounded
|
||||
// cleanup before it can return a useful structured error. Keep only that
|
||||
// transport alive slightly longer so it does not replace the result.
|
||||
const EXTENSION_RESPONSE_GRACE: Duration = Duration::from_secs(2);
|
||||
/// Upper bound on `wait_for_browser_ms` accepted over IPC.
|
||||
const MAX_BROWSER_WAIT: Duration = Duration::from_secs(60);
|
||||
// `session.stop` fast-fails while another tool is active; this budget
|
||||
@@ -206,15 +211,15 @@ pub fn full_handler(status: DaemonStatus, state: Arc<DaemonState>) -> RpcHandler
|
||||
Ok(v) => ResponseBody::Ok(v),
|
||||
Err(e) => ResponseBody::Err(e),
|
||||
},
|
||||
Method::SessionStart => match handle_session_start(&state, rpc_id, params).await {
|
||||
Method::SessionStart => match handle_session_start(&state, params).await {
|
||||
Ok(v) => ResponseBody::Ok(v),
|
||||
Err(e) => ResponseBody::Err(e),
|
||||
},
|
||||
Method::SessionStop => match handle_session_stop(&state, rpc_id, params).await {
|
||||
Method::SessionStop => match handle_session_stop(&state, params).await {
|
||||
Ok(v) => ResponseBody::Ok(v),
|
||||
Err(e) => ResponseBody::Err(e),
|
||||
},
|
||||
Method::SessionStopAll => match handle_session_stop_all_rpc(&state, rpc_id).await {
|
||||
Method::SessionStopAll => match handle_session_stop_all(&state).await {
|
||||
Ok(v) => ResponseBody::Ok(v),
|
||||
Err(e) => ResponseBody::Err(e),
|
||||
},
|
||||
@@ -223,6 +228,11 @@ pub fn full_handler(status: DaemonStatus, state: Arc<DaemonState>) -> RpcHandler
|
||||
Ok(v) => ResponseBody::Ok(v),
|
||||
Err(e) => ResponseBody::Err(e),
|
||||
},
|
||||
Method::TransferBegin => handle_transfer_begin(&state, params),
|
||||
Method::TransferChunk => handle_transfer_chunk(&state, params),
|
||||
Method::TransferFinish => handle_transfer_finish(&state, params),
|
||||
Method::TransferRead => handle_transfer_read(&state, params),
|
||||
Method::TransferRelease => handle_transfer_release(&state, params),
|
||||
Method::ToolTabList
|
||||
| Method::ToolTabCreate
|
||||
| Method::ToolTabClose
|
||||
@@ -246,6 +256,8 @@ pub fn full_handler(status: DaemonStatus, state: Arc<DaemonState>) -> RpcHandler
|
||||
| Method::ToolFill
|
||||
| Method::ToolPress
|
||||
| Method::ToolSelect
|
||||
| Method::ToolUpload
|
||||
| Method::ToolDownload
|
||||
| Method::ToolEvaluate
|
||||
| Method::ToolWaitForNavigation
|
||||
| Method::ToolRequestHelp
|
||||
@@ -324,7 +336,7 @@ async fn handle_tool_dispatch(
|
||||
data: None,
|
||||
});
|
||||
}
|
||||
let timeout = match tool_dispatch_timeout(¶ms) {
|
||||
let timeout = match tool_dispatch_transport_timeout(&method, ¶ms) {
|
||||
Ok(timeout) => timeout,
|
||||
Err(err) => return ResponseBody::Err(err),
|
||||
};
|
||||
@@ -338,24 +350,180 @@ async fn handle_tool_dispatch(
|
||||
});
|
||||
}
|
||||
};
|
||||
let mut params = params;
|
||||
let mut download_transfer_id: Option<String> = None;
|
||||
if method == Method::ToolUpload {
|
||||
let mut upload: UploadParams = match serde_json::from_value(params) {
|
||||
Ok(v) => v,
|
||||
Err(err) => return ResponseBody::Err(invalid_params(err.to_string())),
|
||||
};
|
||||
if upload.files.is_empty() || upload.files.len() > super::file_transfer::MAX_UPLOAD_FILES {
|
||||
return ResponseBody::Err(invalid_params(format!(
|
||||
"upload requires 1..={} files",
|
||||
super::file_transfer::MAX_UPLOAD_FILES
|
||||
)));
|
||||
}
|
||||
let ids: Vec<String> = upload.files.iter().map(|f| f.transfer_id.clone()).collect();
|
||||
let paths = match state.transfers.resolve_uploads(&session_id.0, &ids) {
|
||||
Ok(v) => v,
|
||||
Err(err) => return ResponseBody::Err(err),
|
||||
};
|
||||
for (file, path) in upload.files.iter_mut().zip(paths) {
|
||||
file.staged_path = Some(path.to_string_lossy().into_owned());
|
||||
}
|
||||
params = serde_json::to_value(upload).unwrap_or(Value::Null);
|
||||
} else if method == Method::ToolDownload {
|
||||
let mut download: DownloadParams = match serde_json::from_value(params) {
|
||||
Ok(v) => v,
|
||||
Err(err) => return ResponseBody::Err(invalid_params(err.to_string())),
|
||||
};
|
||||
let staging = match state.transfers.begin_download(&session_id.0) {
|
||||
Ok(v) => v,
|
||||
Err(err) => return ResponseBody::Err(err),
|
||||
};
|
||||
download.browser_relative_dir = Some(staging.browser_relative_dir);
|
||||
download.max_byte_size = Some(super::file_transfer::MAX_TRANSFER_BYTES);
|
||||
download_transfer_id = Some(staging.transfer_id);
|
||||
params = serde_json::to_value(download).unwrap_or(Value::Null);
|
||||
}
|
||||
let entry = inflight_guard.entry();
|
||||
// `record_stop` must reach the extension while `record_await` holds the
|
||||
// serial busy lock — finishing the recording unblocks await.
|
||||
let outcome = if method == Method::ToolRecordStop {
|
||||
state
|
||||
.tool_queues
|
||||
.dispatch_unlocked(&session_id, method, params, timeout, Some(entry))
|
||||
.dispatch_unlocked(&session_id, method.clone(), params, timeout, Some(entry))
|
||||
.await
|
||||
} else {
|
||||
state
|
||||
.tool_queues
|
||||
.dispatch(&session_id, method, params, timeout, Some(entry))
|
||||
.dispatch(&session_id, method.clone(), params, timeout, Some(entry))
|
||||
.await
|
||||
};
|
||||
drop(inflight_guard);
|
||||
match outcome {
|
||||
Ok(v) if method == Method::ToolDownload => {
|
||||
let id = download_transfer_id.expect("download transfer allocated");
|
||||
let mut result: DownloadResult = match serde_json::from_value(v) {
|
||||
Ok(v) => v,
|
||||
Err(err) => {
|
||||
state
|
||||
.transfers
|
||||
.release(TransferIdParams { transfer_id: id });
|
||||
return ResponseBody::Err(RpcError {
|
||||
code: ErrorCode::ProtocolError,
|
||||
message: format!("invalid tool.download result: {err}"),
|
||||
data: None,
|
||||
});
|
||||
}
|
||||
};
|
||||
let Some(path) = result.browser_path.take() else {
|
||||
state
|
||||
.transfers
|
||||
.release(TransferIdParams { transfer_id: id });
|
||||
return ResponseBody::Err(RpcError {
|
||||
code: ErrorCode::ProtocolError,
|
||||
message: "tool.download returned no browser_path".into(),
|
||||
data: None,
|
||||
});
|
||||
};
|
||||
match state
|
||||
.transfers
|
||||
.import_download(&id, std::path::Path::new(&path))
|
||||
{
|
||||
Ok(size) => {
|
||||
result.byte_size = size;
|
||||
result.transfer_id = Some(id);
|
||||
ResponseBody::Ok(serde_json::to_value(result).unwrap_or(Value::Null))
|
||||
}
|
||||
Err(err) => {
|
||||
state
|
||||
.transfers
|
||||
.release(TransferIdParams { transfer_id: id });
|
||||
ResponseBody::Err(err)
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(v) => ResponseBody::Ok(v),
|
||||
Err(err) => ResponseBody::Err(err.into_rpc()),
|
||||
Err(err) => {
|
||||
if let Some(id) = download_transfer_id {
|
||||
state
|
||||
.transfers
|
||||
.release(TransferIdParams { transfer_id: id });
|
||||
}
|
||||
ResponseBody::Err(err.into_rpc())
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn handle_transfer_begin(state: &Arc<DaemonState>, params: Value) -> ResponseBody {
|
||||
let p: TransferBeginParams = match serde_json::from_value(params) {
|
||||
Ok(v) => v,
|
||||
Err(err) => return ResponseBody::Err(invalid_params(err.to_string())),
|
||||
};
|
||||
if state
|
||||
.sessions
|
||||
.get(&SessionId(p.session_id.clone()))
|
||||
.is_none()
|
||||
{
|
||||
return ResponseBody::Err(RpcError {
|
||||
code: ErrorCode::NotFound,
|
||||
message: format!("session {} unknown", p.session_id),
|
||||
data: None,
|
||||
});
|
||||
}
|
||||
match state.transfers.begin_upload(p) {
|
||||
Ok(v) => ResponseBody::Ok(serde_json::to_value(v).unwrap_or(Value::Null)),
|
||||
Err(err) => ResponseBody::Err(err),
|
||||
}
|
||||
}
|
||||
|
||||
fn handle_transfer_chunk(state: &Arc<DaemonState>, params: Value) -> ResponseBody {
|
||||
let p = match serde_json::from_value(params) {
|
||||
Ok(v) => v,
|
||||
Err(err) => return ResponseBody::Err(invalid_params(err.to_string())),
|
||||
};
|
||||
match state.transfers.write_chunk(p) {
|
||||
Ok(v) => ResponseBody::Ok(serde_json::to_value(v).unwrap_or(Value::Null)),
|
||||
Err(err) => ResponseBody::Err(err),
|
||||
}
|
||||
}
|
||||
|
||||
fn handle_transfer_finish(state: &Arc<DaemonState>, params: Value) -> ResponseBody {
|
||||
let p = match serde_json::from_value(params) {
|
||||
Ok(v) => v,
|
||||
Err(err) => return ResponseBody::Err(invalid_params(err.to_string())),
|
||||
};
|
||||
match state.transfers.finish_upload(p) {
|
||||
Ok(v) => ResponseBody::Ok(serde_json::to_value(v).unwrap_or(Value::Null)),
|
||||
Err(err) => ResponseBody::Err(err),
|
||||
}
|
||||
}
|
||||
|
||||
fn handle_transfer_read(state: &Arc<DaemonState>, params: Value) -> ResponseBody {
|
||||
let p = match serde_json::from_value(params) {
|
||||
Ok(v) => v,
|
||||
Err(err) => return ResponseBody::Err(invalid_params(err.to_string())),
|
||||
};
|
||||
match state.transfers.read_chunk(p) {
|
||||
Ok(v) => ResponseBody::Ok(serde_json::to_value(v).unwrap_or(Value::Null)),
|
||||
Err(err) => ResponseBody::Err(err),
|
||||
}
|
||||
}
|
||||
|
||||
fn handle_transfer_release(state: &Arc<DaemonState>, params: Value) -> ResponseBody {
|
||||
let p: TransferIdParams = match serde_json::from_value(params) {
|
||||
Ok(v) => v,
|
||||
Err(err) => return ResponseBody::Err(invalid_params(err.to_string())),
|
||||
};
|
||||
ResponseBody::Ok(serde_json::to_value(state.transfers.release(p)).unwrap_or(Value::Null))
|
||||
}
|
||||
|
||||
fn invalid_params(message: impl Into<String>) -> RpcError {
|
||||
RpcError {
|
||||
code: ErrorCode::InvalidParams,
|
||||
message: message.into(),
|
||||
data: None,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -434,9 +602,8 @@ async fn handle_wait_ms(
|
||||
/// cancellation surfaces (M10.2 + review C2):
|
||||
///
|
||||
/// 1. [`AbortRegistry`] — answers daemon-local cancellable runners
|
||||
/// (`tool.wait_ms` and `session.*` lifecycle calls). If a token is
|
||||
/// registered we trip it and stop. Lifecycle handlers forward their
|
||||
/// own WS cancel after preserving request-before-cancel ordering.
|
||||
/// (currently `tool.wait_ms`). If a token is registered we trip it
|
||||
/// and stop.
|
||||
/// 2. [`super::inflight::ToolInflightRegistry`] — every IPC-tracked
|
||||
/// `tool.*` RPC, registered the moment the IPC handler accepts the
|
||||
/// request. Trips the entry's cancel token regardless of whether
|
||||
@@ -533,6 +700,16 @@ fn tool_dispatch_timeout(params: &Value) -> Result<Duration, RpcError> {
|
||||
Ok(Duration::from_millis(u64::from(ms)))
|
||||
}
|
||||
|
||||
fn tool_dispatch_transport_timeout(method: &Method, params: &Value) -> Result<Duration, RpcError> {
|
||||
tool_dispatch_timeout(params).map(|timeout| {
|
||||
if matches!(method, Method::ToolUpload | Method::ToolDownload) {
|
||||
timeout.saturating_add(EXTENSION_RESPONSE_GRACE)
|
||||
} else {
|
||||
timeout
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
// Local CLI-facing shapes. Intentionally distinct from
|
||||
// `bsk_protocol::tools::SessionStart*` (which describes the WS-facing
|
||||
// `tool.session_*` round-trip with the extension) so the two sides can
|
||||
@@ -635,20 +812,7 @@ fn clamp_browser_wait(wait_ms: Option<u64>) -> Option<Duration> {
|
||||
))
|
||||
}
|
||||
|
||||
async fn handle_session_start(
|
||||
state: &Arc<DaemonState>,
|
||||
rpc_id: RpcId,
|
||||
params: Value,
|
||||
) -> Result<Value, RpcError> {
|
||||
let abort_guard = state
|
||||
.abort_registry
|
||||
.register(rpc_id)
|
||||
.map_err(|err| RpcError {
|
||||
code: ErrorCode::ProtocolError,
|
||||
message: format!("session.start cancellation registration failed: {err:?}"),
|
||||
data: None,
|
||||
})?;
|
||||
let cancel = abort_guard.token().clone();
|
||||
async fn handle_session_start(state: &Arc<DaemonState>, params: Value) -> Result<Value, RpcError> {
|
||||
let params: CliSessionStartParams = if params.is_null() {
|
||||
CliSessionStartParams {
|
||||
browser_instance_id: None,
|
||||
@@ -687,7 +851,6 @@ async fn handle_session_start(
|
||||
},
|
||||
state.config.extension_connect_wait,
|
||||
DEFAULT_RPC_TIMEOUT,
|
||||
Some(cancel),
|
||||
)
|
||||
.await
|
||||
{
|
||||
@@ -711,8 +874,6 @@ fn map_start_error(err: StartSessionError) -> RpcError {
|
||||
StartSessionError::AmbiguousBrowserLabel { .. } => ErrorCode::InvalidParams,
|
||||
StartSessionError::IdExhausted => ErrorCode::ProtocolError,
|
||||
StartSessionError::Timeout => ErrorCode::Timeout,
|
||||
StartSessionError::Cancelled => ErrorCode::Cancelled,
|
||||
StartSessionError::CleanupFailed { .. } => ErrorCode::ProtocolError,
|
||||
StartSessionError::TransportClosed => ErrorCode::ProtocolError,
|
||||
StartSessionError::ExtensionError(inner) => inner.code,
|
||||
};
|
||||
@@ -728,16 +889,6 @@ fn map_start_error(err: StartSessionError) -> RpcError {
|
||||
"label": label,
|
||||
"instance_ids": instance_ids,
|
||||
})),
|
||||
StartSessionError::CleanupFailed {
|
||||
session_id,
|
||||
agent_window_id,
|
||||
..
|
||||
} => Some(serde_json::json!({
|
||||
"reason": "cleanup_failed",
|
||||
"session_id": session_id.0,
|
||||
"agent_window_id": agent_window_id,
|
||||
})),
|
||||
StartSessionError::ExtensionError(inner) => inner.data.clone(),
|
||||
_ => None,
|
||||
};
|
||||
RpcError {
|
||||
@@ -747,20 +898,7 @@ fn map_start_error(err: StartSessionError) -> RpcError {
|
||||
}
|
||||
}
|
||||
|
||||
async fn handle_session_stop(
|
||||
state: &Arc<DaemonState>,
|
||||
rpc_id: RpcId,
|
||||
params: Value,
|
||||
) -> Result<Value, RpcError> {
|
||||
let abort_guard = state
|
||||
.abort_registry
|
||||
.register(rpc_id)
|
||||
.map_err(|err| RpcError {
|
||||
code: ErrorCode::ProtocolError,
|
||||
message: format!("session.stop cancellation registration failed: {err:?}"),
|
||||
data: None,
|
||||
})?;
|
||||
let cancel = abort_guard.token().clone();
|
||||
async fn handle_session_stop(state: &Arc<DaemonState>, params: Value) -> Result<Value, RpcError> {
|
||||
let params: CliSessionStopParams = if params.is_null() {
|
||||
CliSessionStopParams {
|
||||
session_id: None,
|
||||
@@ -774,7 +912,7 @@ async fn handle_session_stop(
|
||||
})?
|
||||
};
|
||||
if params.all {
|
||||
return handle_session_stop_all(state, Some(cancel)).await;
|
||||
return handle_session_stop_all(state).await;
|
||||
}
|
||||
let session_id = match params.session_id {
|
||||
Some(s) => SessionId(s),
|
||||
@@ -793,11 +931,11 @@ async fn handle_session_stop(
|
||||
&state.session_interrupts,
|
||||
&session_id,
|
||||
DEFAULT_SESSION_STOP_TIMEOUT,
|
||||
Some(cancel),
|
||||
)
|
||||
.await
|
||||
{
|
||||
Ok(stop) => {
|
||||
state.transfers.release_session(&session_id.0);
|
||||
let result = CliSessionStopResult {
|
||||
stopped: vec![session_id.0],
|
||||
failed: Vec::new(),
|
||||
@@ -845,25 +983,7 @@ fn map_stop_error(err: StopSessionError) -> RpcError {
|
||||
}
|
||||
}
|
||||
|
||||
async fn handle_session_stop_all_rpc(
|
||||
state: &Arc<DaemonState>,
|
||||
rpc_id: RpcId,
|
||||
) -> Result<Value, RpcError> {
|
||||
let abort_guard = state
|
||||
.abort_registry
|
||||
.register(rpc_id)
|
||||
.map_err(|err| RpcError {
|
||||
code: ErrorCode::ProtocolError,
|
||||
message: format!("session.stop_all cancellation registration failed: {err:?}"),
|
||||
data: None,
|
||||
})?;
|
||||
handle_session_stop_all(state, Some(abort_guard.token().clone())).await
|
||||
}
|
||||
|
||||
async fn handle_session_stop_all(
|
||||
state: &Arc<DaemonState>,
|
||||
cancel: Option<super::abort::AbortToken>,
|
||||
) -> Result<Value, RpcError> {
|
||||
async fn handle_session_stop_all(state: &Arc<DaemonState>) -> Result<Value, RpcError> {
|
||||
let ids: Vec<SessionId> = state
|
||||
.sessions
|
||||
.snapshot()
|
||||
@@ -875,16 +995,6 @@ async fn handle_session_stop_all(
|
||||
let mut returned_tab_ids = Vec::new();
|
||||
let mut return_failures = Vec::new();
|
||||
for id in ids {
|
||||
if cancel
|
||||
.as_ref()
|
||||
.is_some_and(super::abort::AbortToken::is_cancelled)
|
||||
{
|
||||
return Err(RpcError {
|
||||
code: ErrorCode::Cancelled,
|
||||
message: "session.stop_all was cancelled".into(),
|
||||
data: None,
|
||||
});
|
||||
}
|
||||
match stop_session(
|
||||
&state.browsers,
|
||||
&state.sessions,
|
||||
@@ -892,11 +1002,11 @@ async fn handle_session_stop_all(
|
||||
&state.session_interrupts,
|
||||
&id,
|
||||
DEFAULT_SESSION_STOP_TIMEOUT,
|
||||
cancel.clone(),
|
||||
)
|
||||
.await
|
||||
{
|
||||
Ok(stop) => {
|
||||
state.transfers.release_session(&id.0);
|
||||
stopped.push(id.0);
|
||||
returned_tab_ids.extend(stop.returned_tab_ids);
|
||||
return_failures.extend(stop.return_failures);
|
||||
@@ -918,15 +1028,6 @@ async fn handle_session_stop_all(
|
||||
returned_tab_ids.extend(stop.returned_tab_ids);
|
||||
return_failures.extend(stop.return_failures);
|
||||
}
|
||||
Err(err)
|
||||
if matches!(
|
||||
&err,
|
||||
StopSessionError::ExtensionError(inner)
|
||||
if inner.code == ErrorCode::Cancelled
|
||||
) =>
|
||||
{
|
||||
return Err(map_stop_error(err));
|
||||
}
|
||||
Err(err) => {
|
||||
debug!(session = %id, ?err, "session.stop_all: failure (continuing)");
|
||||
let rpc = map_stop_error(err);
|
||||
@@ -1329,6 +1430,30 @@ mod tests {
|
||||
assert_eq!(got, std::time::Duration::from_millis(300_000));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn extension_transport_outlives_the_operation_deadline() {
|
||||
let params = serde_json::json!({
|
||||
"session_id": "abcd",
|
||||
"timeout_ms": 60_000,
|
||||
});
|
||||
let dispatch_timeout =
|
||||
tool_dispatch_transport_timeout(&Method::ToolUpload, ¶ms).unwrap();
|
||||
|
||||
assert_eq!(dispatch_timeout, Duration::from_secs(62));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn extension_response_grace_does_not_change_other_tools() {
|
||||
let params = serde_json::json!({
|
||||
"session_id": "abcd",
|
||||
"timeout_ms": 60_000,
|
||||
});
|
||||
let dispatch_timeout =
|
||||
tool_dispatch_transport_timeout(&Method::ToolClick, ¶ms).unwrap();
|
||||
|
||||
assert_eq!(dispatch_timeout, Duration::from_secs(60));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn tool_dispatch_timeout_rejects_zero_timeout_ms() {
|
||||
let params = serde_json::json!({
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
pub mod abort;
|
||||
pub mod browsers;
|
||||
mod cancel_forward;
|
||||
pub mod file_transfer;
|
||||
pub mod inflight;
|
||||
pub mod info;
|
||||
pub mod ipc;
|
||||
|
||||
+210
-143
@@ -43,7 +43,6 @@ use serde_json::Value;
|
||||
use tokio::sync::{mpsc, oneshot};
|
||||
use tracing::{debug, warn};
|
||||
|
||||
use super::abort::AbortToken;
|
||||
use super::browsers::BrowserRegistry;
|
||||
use super::inflight::{PromoteOutcome, ToolInflightEntry};
|
||||
use super::sessions::{SessionId, SessionRegistry};
|
||||
@@ -84,14 +83,6 @@ pub struct ToolJob {
|
||||
/// IPC request id (e.g. `session.stop`'s queued teardown call —
|
||||
/// those already carry their own bespoke retry / abort path).
|
||||
pub inflight: Option<Arc<ToolInflightEntry>>,
|
||||
/// Cancellation token for session-lifecycle work registered in the
|
||||
/// daemon-local abort registry. Unlike `inflight`, lifecycle callers
|
||||
/// forward their own WS cancel after the request has been queued.
|
||||
pub lifecycle_cancel: Option<AbortToken>,
|
||||
/// How long cancellation keeps the worker busy while the extension
|
||||
/// finishes compensation. Session teardown uses its full RPC budget
|
||||
/// so daemon state is reconciled even after the CLI exits.
|
||||
pub cancel_cleanup_timeout: Duration,
|
||||
/// Owning session — used by `ToolInflightRegistry::cancel_session`
|
||||
/// to drain queued jobs that have not yet been promoted to
|
||||
/// "forwarded". Worker callers also have access to it via the
|
||||
@@ -331,8 +322,6 @@ impl ToolQueueRegistry {
|
||||
timeout,
|
||||
false,
|
||||
inflight,
|
||||
None,
|
||||
CANCEL_CLEANUP_TIMEOUT,
|
||||
)
|
||||
.await;
|
||||
// A long-running request may outlive the idle threshold. Touching
|
||||
@@ -371,8 +360,6 @@ impl ToolQueueRegistry {
|
||||
timeout,
|
||||
respond: respond_tx,
|
||||
inflight,
|
||||
lifecycle_cancel: None,
|
||||
cancel_cleanup_timeout: CANCEL_CLEANUP_TIMEOUT,
|
||||
session_id: sid.clone(),
|
||||
};
|
||||
match forward_one(sid, &self.browsers, &self.sessions, &job).await {
|
||||
@@ -391,7 +378,6 @@ impl ToolQueueRegistry {
|
||||
method: Method,
|
||||
params: Value,
|
||||
timeout: Duration,
|
||||
cancel: Option<AbortToken>,
|
||||
) -> Result<Value, DispatchError> {
|
||||
let (sender, state) = {
|
||||
let mut guard = self.queues.lock().expect("tool queue registry poisoned");
|
||||
@@ -418,8 +404,6 @@ impl ToolQueueRegistry {
|
||||
timeout,
|
||||
true,
|
||||
None,
|
||||
cancel,
|
||||
timeout,
|
||||
)
|
||||
.await
|
||||
}
|
||||
@@ -435,9 +419,12 @@ async fn dispatch_with_sender(
|
||||
timeout: Duration,
|
||||
wait_for_capacity: bool,
|
||||
inflight: Option<Arc<ToolInflightEntry>>,
|
||||
lifecycle_cancel: Option<AbortToken>,
|
||||
cancel_cleanup_timeout: Duration,
|
||||
) -> Result<Value, DispatchError> {
|
||||
let response_grace = if is_effect_aware_transfer(&method) {
|
||||
CANCEL_CLEANUP_TIMEOUT
|
||||
} else {
|
||||
Duration::ZERO
|
||||
};
|
||||
let (respond_tx, respond_rx) = oneshot::channel();
|
||||
let job = ToolJob {
|
||||
method,
|
||||
@@ -445,11 +432,8 @@ async fn dispatch_with_sender(
|
||||
timeout,
|
||||
respond: respond_tx,
|
||||
inflight,
|
||||
lifecycle_cancel,
|
||||
cancel_cleanup_timeout,
|
||||
session_id,
|
||||
};
|
||||
let lifecycle_cancellable = job.lifecycle_cancel.is_some();
|
||||
if wait_for_capacity {
|
||||
let waited = tokio::time::timeout(
|
||||
timeout.saturating_add(Duration::from_secs(1)),
|
||||
@@ -474,15 +458,13 @@ async fn dispatch_with_sender(
|
||||
mpsc::error::TrySendError::Closed(_) => DispatchError::QueueClosed,
|
||||
});
|
||||
}
|
||||
let mut response_timeout = timeout.saturating_add(Duration::from_secs(1));
|
||||
if lifecycle_cancellable {
|
||||
// A lifecycle cancel can arrive at the original RPC deadline and
|
||||
// intentionally keeps the worker alive while extension cleanup
|
||||
// settles. Do not let this outer waiter abandon that reconciliation
|
||||
// early, which could reopen a daemon session whose window is gone.
|
||||
response_timeout = response_timeout.saturating_add(cancel_cleanup_timeout);
|
||||
}
|
||||
let waited = tokio::time::timeout(response_timeout, respond_rx).await;
|
||||
let waited = tokio::time::timeout(
|
||||
timeout
|
||||
.saturating_add(response_grace)
|
||||
.saturating_add(Duration::from_secs(1)),
|
||||
respond_rx,
|
||||
)
|
||||
.await;
|
||||
match waited {
|
||||
Ok(Ok(Ok(v))) => Ok(v),
|
||||
Ok(Ok(Err(rpc))) => Err(DispatchError::Rpc(rpc)),
|
||||
@@ -520,17 +502,11 @@ async fn forward_one(
|
||||
// / browser resolution work, and before any WS frame leaves the
|
||||
// daemon (review C2). We keep the `tokio::select!` later so
|
||||
// cancels arriving mid-WS-hop also unblock the worker.
|
||||
if job
|
||||
.inflight
|
||||
.as_ref()
|
||||
.is_some_and(|entry| entry.is_cancelled())
|
||||
|| job
|
||||
.lifecycle_cancel
|
||||
.as_ref()
|
||||
.is_some_and(AbortToken::is_cancelled)
|
||||
if let Some(entry) = job.inflight.as_ref()
|
||||
&& entry.is_cancelled()
|
||||
{
|
||||
return Err(cancelled_error(
|
||||
job.inflight.as_deref(),
|
||||
Some(entry),
|
||||
"tool dispatch cancelled before forwarding",
|
||||
));
|
||||
}
|
||||
@@ -594,20 +570,11 @@ async fn forward_one(
|
||||
}
|
||||
}
|
||||
None => {
|
||||
// Daemon-internal callers do not carry a tool-inflight entry.
|
||||
// Session lifecycle jobs instead use the abort-registry token
|
||||
// below; other internal callers keep the plain send path.
|
||||
if job
|
||||
.lifecycle_cancel
|
||||
.as_ref()
|
||||
.is_some_and(AbortToken::is_cancelled)
|
||||
{
|
||||
client.pending.lock().unwrap().cancel(&rpc_id);
|
||||
return Err(cancelled_error(
|
||||
None,
|
||||
"session lifecycle cancelled before forwarding",
|
||||
));
|
||||
}
|
||||
// Daemon-internal callers (e.g. queued session.stop drain)
|
||||
// don't carry an inflight entry; they cannot be cancelled
|
||||
// through the IPC `cancel` surface, so a plain sink send
|
||||
// is sufficient and keeps the existing transport-error
|
||||
// behaviour.
|
||||
if client.sink.send(request).is_err() {
|
||||
client.pending.lock().unwrap().cancel(&rpc_id);
|
||||
return Err(RpcError {
|
||||
@@ -616,92 +583,120 @@ async fn forward_one(
|
||||
data: None,
|
||||
});
|
||||
}
|
||||
job.lifecycle_cancel.clone()
|
||||
None
|
||||
}
|
||||
};
|
||||
let forward_lifecycle_cancel = || {
|
||||
if job.lifecycle_cancel.is_none() {
|
||||
return;
|
||||
}
|
||||
let cancel = Frame::Request(RequestFrame {
|
||||
id: format!("cancel-{rpc_id}"),
|
||||
method: Method::Cancel,
|
||||
params: Some(serde_json::json!({ "rpc_id": rpc_id })),
|
||||
});
|
||||
if let Err(err) = client.sink.send(cancel) {
|
||||
warn!(%rpc_id, ?err, "failed to forward session lifecycle cancel");
|
||||
}
|
||||
let send_deadline_cancel = || {
|
||||
client
|
||||
.sink
|
||||
.send(Frame::Request(RequestFrame {
|
||||
id: format!("deadline-cancel-{rpc_id}"),
|
||||
method: Method::Cancel,
|
||||
params: Some(serde_json::json!({ "rpc_id": rpc_id })),
|
||||
}))
|
||||
.is_ok()
|
||||
};
|
||||
let on_abort: Option<&(dyn Fn() + Sync)> = job
|
||||
.lifecycle_cancel
|
||||
.as_ref()
|
||||
.map(|_| &forward_lifecycle_cancel as &(dyn Fn() + Sync));
|
||||
let deadline_cancel: Option<&(dyn Fn() -> bool + Sync)> =
|
||||
is_effect_aware_transfer(&job.method).then_some(&send_deadline_cancel);
|
||||
let waited = await_with_optional_cancel(
|
||||
job.timeout,
|
||||
job.cancel_cleanup_timeout,
|
||||
CANCEL_CLEANUP_TIMEOUT,
|
||||
waiter,
|
||||
cancel_token.as_ref(),
|
||||
on_abort,
|
||||
deadline_cancel,
|
||||
)
|
||||
.await;
|
||||
let response = match waited {
|
||||
WaitOutcome::Response(resp) => resp,
|
||||
WaitOutcome::CancelledAfterResponse(resp) => {
|
||||
if job.method == Method::ToolSessionStop
|
||||
&& let ResponseBody::Ok(value) = resp.body
|
||||
{
|
||||
// Teardown crossed its final irreversible boundary before
|
||||
// the cancel landed. Commit the real extension result so the
|
||||
// daemon does not retain a session whose window is gone.
|
||||
return Ok(value);
|
||||
// File transfer commits are irreversible. A late cancel cannot
|
||||
// overwrite a confirmed success, and an unknown transfer effect
|
||||
// must remain explicit so callers do not retry or release upload
|
||||
// staging as though nothing happened.
|
||||
if is_effect_aware_transfer(&job.method) {
|
||||
match &resp.body {
|
||||
ResponseBody::Ok(_) => resp,
|
||||
ResponseBody::Err(err)
|
||||
if transfer_effect(err) == Some("unknown")
|
||||
|| transfer_effect(err) == Some("committed") =>
|
||||
{
|
||||
return Err(err.clone());
|
||||
}
|
||||
ResponseBody::Err(err)
|
||||
if !matches!(err.code, ErrorCode::Cancelled | ErrorCode::UserAborted) =>
|
||||
{
|
||||
return Err(err.clone());
|
||||
}
|
||||
ResponseBody::Err(_) => {
|
||||
return Err(cancelled_error(
|
||||
job.inflight.as_deref(),
|
||||
"tool dispatch cancelled after extension cleanup",
|
||||
));
|
||||
}
|
||||
}
|
||||
} else {
|
||||
// For ordinary tools cancellation keeps the existing verdict,
|
||||
// while non-cancel errors still expose compensation failures.
|
||||
if let ResponseBody::Err(err) = resp.body
|
||||
&& !matches!(err.code, ErrorCode::Cancelled | ErrorCode::UserAborted)
|
||||
{
|
||||
return Err(err);
|
||||
}
|
||||
return Err(cancelled_error(
|
||||
job.inflight.as_deref(),
|
||||
"tool dispatch cancelled after extension cleanup",
|
||||
));
|
||||
}
|
||||
// Cancellation wins the external verdict, but only after the
|
||||
// extension's original RPC has settled. Preserve any non-cancel
|
||||
// error so compensation failures remain explicit instead of
|
||||
// being hidden behind a generic cancelled result.
|
||||
if let ResponseBody::Err(err) = resp.body
|
||||
&& !matches!(err.code, ErrorCode::Cancelled | ErrorCode::UserAborted)
|
||||
{
|
||||
return Err(err);
|
||||
}
|
||||
WaitOutcome::CleanupTimeout => {
|
||||
client.pending.lock().unwrap().cancel(&rpc_id);
|
||||
if is_effect_aware_transfer(&job.method) {
|
||||
return Err(unknown_transfer_error(
|
||||
ErrorCode::Timeout,
|
||||
"cancelled file transfer did not confirm its outcome before cleanup timed out",
|
||||
"cleanup",
|
||||
true,
|
||||
));
|
||||
}
|
||||
return Err(cancelled_error(
|
||||
job.inflight.as_deref(),
|
||||
"tool dispatch cancelled after extension cleanup",
|
||||
));
|
||||
return Err(RpcError {
|
||||
code: ErrorCode::Timeout,
|
||||
message: format!(
|
||||
"cancelled tool did not finish cleanup within {:?}",
|
||||
CANCEL_CLEANUP_TIMEOUT
|
||||
),
|
||||
data: Some(serde_json::json!({ "reason": "cancel_cleanup_timeout" })),
|
||||
});
|
||||
}
|
||||
WaitOutcome::TimedOutAfterResponse(resp) => match resp.body {
|
||||
ResponseBody::Ok(value) if job.method == Method::ToolSessionStop => {
|
||||
// The close crossed its irreversible boundary during the
|
||||
// timeout cleanup grace. Reconcile daemon state instead of
|
||||
// reopening a session whose window is already gone.
|
||||
return Ok(value);
|
||||
}
|
||||
ResponseBody::Ok(value) => return Ok(value),
|
||||
ResponseBody::Err(err)
|
||||
if !matches!(err.code, ErrorCode::Cancelled | ErrorCode::UserAborted) =>
|
||||
{
|
||||
return Err(err);
|
||||
}
|
||||
_ => {
|
||||
return Err(RpcError {
|
||||
code: ErrorCode::Timeout,
|
||||
message: format!("tool RPC timed out after {:?}", job.timeout),
|
||||
data: None,
|
||||
});
|
||||
ResponseBody::Err(err) => {
|
||||
return Err(timed_out_transfer_error(&err));
|
||||
}
|
||||
},
|
||||
WaitOutcome::CleanupTimeout => {
|
||||
WaitOutcome::TimeoutCleanupFailed => {
|
||||
client.pending.lock().unwrap().cancel(&rpc_id);
|
||||
return Err(RpcError {
|
||||
code: ErrorCode::Timeout,
|
||||
message: format!(
|
||||
"cancelled tool did not finish cleanup within {:?}",
|
||||
job.cancel_cleanup_timeout
|
||||
),
|
||||
data: Some(serde_json::json!({ "reason": "cancel_cleanup_timeout" })),
|
||||
});
|
||||
return Err(unknown_transfer_error(
|
||||
ErrorCode::Timeout,
|
||||
"file transfer timed out and cleanup could not be confirmed",
|
||||
"cleanup",
|
||||
true,
|
||||
));
|
||||
}
|
||||
WaitOutcome::WaiterClosed => {
|
||||
client.pending.lock().unwrap().cancel(&rpc_id);
|
||||
if is_effect_aware_transfer(&job.method) {
|
||||
return Err(unknown_transfer_error(
|
||||
ErrorCode::ProtocolError,
|
||||
"file transfer transport closed after dispatch; outcome is unknown",
|
||||
"transport",
|
||||
false,
|
||||
));
|
||||
}
|
||||
return Err(RpcError {
|
||||
code: ErrorCode::ProtocolError,
|
||||
message: "transport closed mid-call".into(),
|
||||
@@ -710,6 +705,14 @@ async fn forward_one(
|
||||
}
|
||||
WaitOutcome::Timeout => {
|
||||
client.pending.lock().unwrap().cancel(&rpc_id);
|
||||
if is_effect_aware_transfer(&job.method) {
|
||||
return Err(unknown_transfer_error(
|
||||
ErrorCode::Timeout,
|
||||
"file transfer timed out after dispatch; outcome is unknown",
|
||||
"transport",
|
||||
false,
|
||||
));
|
||||
}
|
||||
return Err(RpcError {
|
||||
code: ErrorCode::Timeout,
|
||||
message: format!("tool RPC timed out after {:?}", job.timeout),
|
||||
@@ -723,12 +726,65 @@ async fn forward_one(
|
||||
}
|
||||
}
|
||||
|
||||
fn is_effect_aware_transfer(method: &Method) -> bool {
|
||||
matches!(method, Method::ToolUpload | Method::ToolDownload)
|
||||
}
|
||||
|
||||
fn transfer_effect(err: &RpcError) -> Option<&str> {
|
||||
err.data.as_ref()?.get("effect_state")?.as_str()
|
||||
}
|
||||
|
||||
fn unknown_transfer_error(
|
||||
code: ErrorCode,
|
||||
message: impl Into<String>,
|
||||
phase: &str,
|
||||
cleanup_failed: bool,
|
||||
) -> RpcError {
|
||||
let mut data = serde_json::json!({
|
||||
"reason": "transfer_outcome_unknown",
|
||||
"effect_state": "unknown",
|
||||
"phase": phase,
|
||||
});
|
||||
if cleanup_failed {
|
||||
data["cleanup_state"] = serde_json::json!("failed");
|
||||
}
|
||||
RpcError {
|
||||
code,
|
||||
message: message.into(),
|
||||
data: Some(data),
|
||||
}
|
||||
}
|
||||
|
||||
fn timed_out_transfer_error(err: &RpcError) -> RpcError {
|
||||
let data = err.data.as_ref();
|
||||
RpcError {
|
||||
code: ErrorCode::Timeout,
|
||||
message: "file transfer timed out after dispatch".into(),
|
||||
data: Some(serde_json::json!({
|
||||
"reason": "transfer_timeout",
|
||||
"effect_state": data
|
||||
.and_then(|value| value.get("effect_state"))
|
||||
.and_then(Value::as_str)
|
||||
.unwrap_or("unknown"),
|
||||
"phase": data
|
||||
.and_then(|value| value.get("phase"))
|
||||
.and_then(Value::as_str)
|
||||
.unwrap_or("cleanup"),
|
||||
"cleanup_state": data
|
||||
.and_then(|value| value.get("cleanup_state"))
|
||||
.and_then(Value::as_str)
|
||||
.unwrap_or("complete"),
|
||||
})),
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
enum WaitOutcome {
|
||||
Response(bsk_protocol::ResponseFrame),
|
||||
CancelledAfterResponse(bsk_protocol::ResponseFrame),
|
||||
TimedOutAfterResponse(bsk_protocol::ResponseFrame),
|
||||
CleanupTimeout,
|
||||
TimedOutAfterResponse(bsk_protocol::ResponseFrame),
|
||||
TimeoutCleanupFailed,
|
||||
WaiterClosed,
|
||||
Timeout,
|
||||
}
|
||||
@@ -738,7 +794,7 @@ async fn await_with_optional_cancel(
|
||||
cleanup_timeout: Duration,
|
||||
mut waiter: oneshot::Receiver<bsk_protocol::ResponseFrame>,
|
||||
cancel: Option<&super::abort::AbortToken>,
|
||||
on_abort: Option<&(dyn Fn() + Sync)>,
|
||||
on_deadline: Option<&(dyn Fn() -> bool + Sync)>,
|
||||
) -> WaitOutcome {
|
||||
match cancel {
|
||||
Some(token) => {
|
||||
@@ -750,9 +806,6 @@ async fn await_with_optional_cancel(
|
||||
// after compensation or the cleanup deadline expires.
|
||||
biased;
|
||||
_ = token.cancelled() => {
|
||||
if let Some(on_abort) = on_abort {
|
||||
on_abort();
|
||||
}
|
||||
match tokio::time::timeout(cleanup_timeout, &mut waiter).await {
|
||||
Ok(Ok(resp)) => WaitOutcome::CancelledAfterResponse(resp),
|
||||
Ok(Err(_)) => WaitOutcome::WaiterClosed,
|
||||
@@ -763,23 +816,31 @@ async fn await_with_optional_cancel(
|
||||
Ok(resp) => WaitOutcome::Response(resp),
|
||||
Err(_) => WaitOutcome::WaiterClosed,
|
||||
},
|
||||
_ = &mut deadline => {
|
||||
let Some(on_abort) = on_abort else {
|
||||
return WaitOutcome::Timeout;
|
||||
};
|
||||
on_abort();
|
||||
match tokio::time::timeout(cleanup_timeout, &mut waiter).await {
|
||||
Ok(Ok(resp)) => WaitOutcome::TimedOutAfterResponse(resp),
|
||||
Ok(Err(_)) => WaitOutcome::WaiterClosed,
|
||||
Err(_) => WaitOutcome::CleanupTimeout,
|
||||
_ = &mut deadline => match on_deadline {
|
||||
Some(send_cancel) if send_cancel() => {
|
||||
match tokio::time::timeout(cleanup_timeout, &mut waiter).await {
|
||||
Ok(Ok(resp)) => WaitOutcome::TimedOutAfterResponse(resp),
|
||||
Ok(Err(_)) | Err(_) => WaitOutcome::TimeoutCleanupFailed,
|
||||
}
|
||||
}
|
||||
Some(_) => WaitOutcome::TimeoutCleanupFailed,
|
||||
None => WaitOutcome::Timeout,
|
||||
},
|
||||
}
|
||||
}
|
||||
None => match tokio::time::timeout(timeout, &mut waiter).await {
|
||||
Ok(Ok(resp)) => WaitOutcome::Response(resp),
|
||||
Ok(Err(_)) => WaitOutcome::WaiterClosed,
|
||||
Err(_) => WaitOutcome::Timeout,
|
||||
Err(_) => match on_deadline {
|
||||
Some(send_cancel) if send_cancel() => {
|
||||
match tokio::time::timeout(cleanup_timeout, &mut waiter).await {
|
||||
Ok(Ok(resp)) => WaitOutcome::TimedOutAfterResponse(resp),
|
||||
Ok(Err(_)) | Err(_) => WaitOutcome::TimeoutCleanupFailed,
|
||||
}
|
||||
}
|
||||
Some(_) => WaitOutcome::TimeoutCleanupFailed,
|
||||
None => WaitOutcome::Timeout,
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
@@ -954,30 +1015,38 @@ mod await_with_optional_cancel_tests {
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn lifecycle_timeout_forwards_abort_and_waits_for_cleanup_response() {
|
||||
use std::sync::atomic::{AtomicBool, Ordering};
|
||||
|
||||
let token = AbortToken::new();
|
||||
async fn transfer_deadline_sends_cancel_and_waits_for_cleanup_response() {
|
||||
let (tx, rx) = oneshot::channel();
|
||||
let abort_forwarded = AtomicBool::new(false);
|
||||
let forward_abort = || abort_forwarded.store(true, Ordering::SeqCst);
|
||||
tokio::spawn(async move {
|
||||
tokio::time::sleep(Duration::from_millis(20)).await;
|
||||
tokio::time::sleep(Duration::from_millis(5)).await;
|
||||
tx.send(dummy_response()).unwrap();
|
||||
});
|
||||
|
||||
let send_cancel = || true;
|
||||
let outcome = await_with_optional_cancel(
|
||||
Duration::from_millis(5),
|
||||
Duration::from_millis(100),
|
||||
Duration::from_millis(1),
|
||||
Duration::from_secs(1),
|
||||
rx,
|
||||
Some(&token),
|
||||
Some(&forward_abort),
|
||||
None,
|
||||
Some(&send_cancel),
|
||||
)
|
||||
.await;
|
||||
|
||||
assert!(abort_forwarded.load(Ordering::SeqCst));
|
||||
assert!(matches!(outcome, WaitOutcome::TimedOutAfterResponse(_)));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn transfer_deadline_is_unknown_when_cancel_cannot_be_sent() {
|
||||
let (_tx, rx) = oneshot::channel();
|
||||
let send_cancel = || false;
|
||||
let outcome = await_with_optional_cancel(
|
||||
Duration::from_millis(1),
|
||||
Duration::from_secs(1),
|
||||
rx,
|
||||
None,
|
||||
Some(&send_cancel),
|
||||
)
|
||||
.await;
|
||||
assert!(matches!(outcome, WaitOutcome::TimeoutCleanupFailed));
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
@@ -1040,8 +1109,6 @@ mod tool_job_session_id_tests {
|
||||
timeout: Duration::from_secs(1),
|
||||
respond: tx,
|
||||
inflight: None,
|
||||
lifecycle_cancel: None,
|
||||
cancel_cleanup_timeout: CANCEL_CLEANUP_TIMEOUT,
|
||||
session_id: SessionId("sess-A".into()),
|
||||
};
|
||||
assert_eq!(job.session_id.0, "sess-A");
|
||||
|
||||
@@ -12,14 +12,13 @@ use bsk_protocol::system::{BrowserStatusEntry, SessionStatusEntry};
|
||||
use bsk_protocol::tools::{
|
||||
SessionStartParams, SessionStartResult, SessionStopParams, SessionStopResult,
|
||||
};
|
||||
use bsk_protocol::{ErrorCode, Frame, Method, RequestFrame, ResponseBody, RpcError, RpcId};
|
||||
use bsk_protocol::{Frame, RequestFrame, ResponseBody, RpcError, RpcId};
|
||||
use rand::Rng;
|
||||
use serde::{Deserialize, Serialize};
|
||||
use tokio::time::{Duration, timeout};
|
||||
|
||||
use super::abort::AbortToken;
|
||||
use super::browsers::{BrowserClient, BrowserId, BrowserRegistry, SelectError};
|
||||
use super::queue::{CANCEL_CLEANUP_TIMEOUT, DispatchError, ToolQueueRegistry};
|
||||
use super::queue::{DispatchError, ToolQueueRegistry};
|
||||
use super::session_interrupt::SessionInterruptRegistry;
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize)]
|
||||
@@ -340,14 +339,6 @@ pub enum StartSessionError {
|
||||
IdExhausted,
|
||||
#[error("session creation timed out waiting for extension")]
|
||||
Timeout,
|
||||
#[error("session creation was cancelled")]
|
||||
Cancelled,
|
||||
#[error("session creation cancellation cleanup failed for {session_id}: {message}")]
|
||||
CleanupFailed {
|
||||
session_id: SessionId,
|
||||
agent_window_id: Option<i64>,
|
||||
message: String,
|
||||
},
|
||||
#[error("extension rejected tool.session_start: {0:?}")]
|
||||
ExtensionError(RpcError),
|
||||
#[error("transport closed while waiting for extension response")]
|
||||
@@ -363,8 +354,6 @@ impl StartSessionError {
|
||||
StartSessionError::AmbiguousBrowserLabel { .. } => "invalid_params",
|
||||
StartSessionError::IdExhausted => "protocol_error",
|
||||
StartSessionError::Timeout => "timeout",
|
||||
StartSessionError::Cancelled => "cancelled",
|
||||
StartSessionError::CleanupFailed { .. } => "protocol_error",
|
||||
StartSessionError::TransportClosed => "protocol_error",
|
||||
StartSessionError::ExtensionError(err) => match err.code {
|
||||
bsk_protocol::ErrorCode::Timeout => "timeout",
|
||||
@@ -418,7 +407,6 @@ pub struct AgentWindowOptions {
|
||||
/// On success the matching per-session dispatch queue (M6.5,
|
||||
/// design §5) is also spawned so that subsequent `tool.*` RPCs serialise
|
||||
/// against this session's Agent Window / ref-store.
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
pub async fn start_session(
|
||||
registry: &Arc<BrowserRegistry>,
|
||||
sessions: &Arc<SessionRegistry>,
|
||||
@@ -427,32 +415,24 @@ pub async fn start_session(
|
||||
window: AgentWindowOptions,
|
||||
connect_wait: Duration,
|
||||
timeout_dur: Duration,
|
||||
cancel: Option<AbortToken>,
|
||||
) -> Result<Session, StartSessionError> {
|
||||
let selection = registry.select_with_connect_wait(requested, connect_wait);
|
||||
tokio::pin!(selection);
|
||||
let selected = match cancel.as_ref() {
|
||||
Some(token) => tokio::select! {
|
||||
biased;
|
||||
_ = token.cancelled() => return Err(StartSessionError::Cancelled),
|
||||
selected = &mut selection => selected,
|
||||
},
|
||||
None => selection.await,
|
||||
};
|
||||
let client: Arc<BrowserClient> = selected.map_err(|e| match e {
|
||||
SelectError::NoBrowserConnected => StartSessionError::NoBrowserConnected,
|
||||
SelectError::MultipleBrowsersOnline => StartSessionError::MultipleBrowsersOnline {
|
||||
browsers: snapshot_status_entries(registry, sessions),
|
||||
},
|
||||
SelectError::NotFound => StartSessionError::BrowserNotFound,
|
||||
SelectError::AmbiguousLabel {
|
||||
label,
|
||||
instance_ids,
|
||||
} => StartSessionError::AmbiguousBrowserLabel {
|
||||
label,
|
||||
instance_ids,
|
||||
},
|
||||
})?;
|
||||
let client: Arc<BrowserClient> = registry
|
||||
.select_with_connect_wait(requested, connect_wait)
|
||||
.await
|
||||
.map_err(|e| match e {
|
||||
SelectError::NoBrowserConnected => StartSessionError::NoBrowserConnected,
|
||||
SelectError::MultipleBrowsersOnline => StartSessionError::MultipleBrowsersOnline {
|
||||
browsers: snapshot_status_entries(registry, sessions),
|
||||
},
|
||||
SelectError::NotFound => StartSessionError::BrowserNotFound,
|
||||
SelectError::AmbiguousLabel {
|
||||
label,
|
||||
instance_ids,
|
||||
} => StartSessionError::AmbiguousBrowserLabel {
|
||||
label,
|
||||
instance_ids,
|
||||
},
|
||||
})?;
|
||||
let session_id = sessions
|
||||
.reserve_id(client.id.clone(), SESSION_ID_MAX_RESERVE_ATTEMPTS, now_ms)
|
||||
.ok_or(StartSessionError::IdExhausted)?;
|
||||
@@ -473,71 +453,22 @@ pub async fn start_session(
|
||||
let mut pending = client.pending.lock().unwrap();
|
||||
pending.register(rpc_id.clone())
|
||||
};
|
||||
if cancel.as_ref().is_some_and(AbortToken::is_cancelled) {
|
||||
sessions.cancel_reservation(&session_id);
|
||||
client.pending.lock().unwrap().cancel(&rpc_id);
|
||||
return Err(StartSessionError::Cancelled);
|
||||
}
|
||||
if client.sink.send(Frame::Request(request)).is_err() {
|
||||
sessions.cancel_reservation(&session_id);
|
||||
client.pending.lock().unwrap().cancel(&rpc_id);
|
||||
return Err(StartSessionError::TransportClosed);
|
||||
}
|
||||
let mut waiter = waiter;
|
||||
enum StartWaitOutcome {
|
||||
Response(bsk_protocol::ResponseFrame),
|
||||
WaiterClosed,
|
||||
Cancelled,
|
||||
Timeout,
|
||||
}
|
||||
let wait_outcome = match cancel.as_ref() {
|
||||
Some(token) => {
|
||||
let deadline = tokio::time::sleep(timeout_dur);
|
||||
tokio::pin!(deadline);
|
||||
tokio::select! {
|
||||
biased;
|
||||
_ = token.cancelled() => StartWaitOutcome::Cancelled,
|
||||
response = &mut waiter => match response {
|
||||
Ok(response) => StartWaitOutcome::Response(response),
|
||||
Err(_) => StartWaitOutcome::WaiterClosed,
|
||||
},
|
||||
_ = &mut deadline => StartWaitOutcome::Timeout,
|
||||
}
|
||||
}
|
||||
None => match timeout(timeout_dur, &mut waiter).await {
|
||||
Ok(Ok(response)) => StartWaitOutcome::Response(response),
|
||||
Ok(Err(_)) => StartWaitOutcome::WaiterClosed,
|
||||
Err(_) => StartWaitOutcome::Timeout,
|
||||
},
|
||||
};
|
||||
let response = match wait_outcome {
|
||||
StartWaitOutcome::Response(response) => response,
|
||||
StartWaitOutcome::WaiterClosed => {
|
||||
let response = match timeout(timeout_dur, waiter).await {
|
||||
Ok(Ok(resp)) => resp,
|
||||
Ok(Err(_)) => {
|
||||
sessions.cancel_reservation(&session_id);
|
||||
client.pending.lock().unwrap().cancel(&rpc_id);
|
||||
return Err(StartSessionError::TransportClosed);
|
||||
}
|
||||
StartWaitOutcome::Cancelled => {
|
||||
return Err(finish_aborted_start(
|
||||
&client,
|
||||
sessions,
|
||||
&session_id,
|
||||
&rpc_id,
|
||||
waiter,
|
||||
StartAbortReason::Cancelled,
|
||||
)
|
||||
.await);
|
||||
}
|
||||
StartWaitOutcome::Timeout => {
|
||||
return Err(finish_aborted_start(
|
||||
&client,
|
||||
sessions,
|
||||
&session_id,
|
||||
&rpc_id,
|
||||
waiter,
|
||||
StartAbortReason::Timeout,
|
||||
)
|
||||
.await);
|
||||
Err(_) => {
|
||||
sessions.cancel_reservation(&session_id);
|
||||
client.pending.lock().unwrap().cancel(&rpc_id);
|
||||
return Err(StartSessionError::Timeout);
|
||||
}
|
||||
};
|
||||
let agent_window_id = match response.body {
|
||||
@@ -573,124 +504,6 @@ pub async fn start_session(
|
||||
Ok(session)
|
||||
}
|
||||
|
||||
#[derive(Clone, Copy)]
|
||||
enum StartAbortReason {
|
||||
Cancelled,
|
||||
Timeout,
|
||||
}
|
||||
|
||||
impl StartAbortReason {
|
||||
fn error(self) -> StartSessionError {
|
||||
match self {
|
||||
Self::Cancelled => StartSessionError::Cancelled,
|
||||
Self::Timeout => StartSessionError::Timeout,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async fn finish_aborted_start(
|
||||
client: &Arc<BrowserClient>,
|
||||
sessions: &Arc<SessionRegistry>,
|
||||
session_id: &SessionId,
|
||||
rpc_id: &RpcId,
|
||||
mut waiter: tokio::sync::oneshot::Receiver<bsk_protocol::ResponseFrame>,
|
||||
reason: StartAbortReason,
|
||||
) -> StartSessionError {
|
||||
sessions.cancel_reservation(session_id);
|
||||
let cancel = Frame::Request(RequestFrame {
|
||||
id: format!("cancel-{rpc_id}"),
|
||||
method: Method::Cancel,
|
||||
params: Some(serde_json::json!({ "rpc_id": rpc_id })),
|
||||
});
|
||||
if client.sink.send(cancel).is_err() {
|
||||
client.pending.lock().unwrap().cancel(rpc_id);
|
||||
return StartSessionError::TransportClosed;
|
||||
}
|
||||
|
||||
match timeout(CANCEL_CLEANUP_TIMEOUT, &mut waiter).await {
|
||||
Ok(Ok(response)) => match response.body {
|
||||
ResponseBody::Err(err)
|
||||
if matches!(err.code, ErrorCode::Cancelled | ErrorCode::UserAborted) =>
|
||||
{
|
||||
reason.error()
|
||||
}
|
||||
ResponseBody::Err(err) => StartSessionError::ExtensionError(err),
|
||||
ResponseBody::Ok(value) => {
|
||||
let agent_window_id = serde_json::from_value::<SessionStartResult>(value)
|
||||
.ok()
|
||||
.and_then(|result| result.agent_window_id);
|
||||
match rollback_extension_session(client, session_id).await {
|
||||
Ok(()) => reason.error(),
|
||||
Err(message) => StartSessionError::CleanupFailed {
|
||||
session_id: session_id.clone(),
|
||||
agent_window_id,
|
||||
message,
|
||||
},
|
||||
}
|
||||
}
|
||||
},
|
||||
Ok(Err(_)) => {
|
||||
client.pending.lock().unwrap().cancel(rpc_id);
|
||||
StartSessionError::TransportClosed
|
||||
}
|
||||
Err(_) => {
|
||||
// The cancel frame is already queued behind session_start. Even
|
||||
// if chrome.windows.create is still blocked, the extension will
|
||||
// observe the aborted signal and compensate before replying.
|
||||
client.pending.lock().unwrap().cancel(rpc_id);
|
||||
reason.error()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async fn rollback_extension_session(
|
||||
client: &Arc<BrowserClient>,
|
||||
session_id: &SessionId,
|
||||
) -> Result<(), String> {
|
||||
let rollback_id = next_rpc_id("sess-start-rollback");
|
||||
let waiter = {
|
||||
let mut pending = client.pending.lock().unwrap();
|
||||
pending.register(rollback_id.clone())
|
||||
};
|
||||
let request = RequestFrame {
|
||||
id: rollback_id.clone(),
|
||||
method: Method::ToolSessionStop,
|
||||
params: Some(
|
||||
serde_json::to_value(SessionStopParams {
|
||||
session_id: session_id.0.clone(),
|
||||
})
|
||||
.expect("serialize session_start rollback params"),
|
||||
),
|
||||
};
|
||||
if client.sink.send(Frame::Request(request)).is_err() {
|
||||
client.pending.lock().unwrap().cancel(&rollback_id);
|
||||
return Err("browser disconnected before rollback was queued".into());
|
||||
}
|
||||
let response = match timeout(CANCEL_CLEANUP_TIMEOUT, waiter).await {
|
||||
Ok(Ok(response)) => response,
|
||||
Ok(Err(_)) => return Err("browser disconnected during rollback".into()),
|
||||
Err(_) => {
|
||||
client.pending.lock().unwrap().cancel(&rollback_id);
|
||||
return Err(format!(
|
||||
"extension did not finish rollback within {CANCEL_CLEANUP_TIMEOUT:?}"
|
||||
));
|
||||
}
|
||||
};
|
||||
match response.body {
|
||||
ResponseBody::Ok(value) => {
|
||||
let result: SessionStopResult = serde_json::from_value(value)
|
||||
.map_err(|err| format!("invalid rollback response: {err}"))?;
|
||||
if result.return_failures.is_empty() {
|
||||
Ok(())
|
||||
} else {
|
||||
Err("rollback reported borrowed-tab return failures".into())
|
||||
}
|
||||
}
|
||||
ResponseBody::Err(err) if err.code == ErrorCode::NotFound => Ok(()),
|
||||
ResponseBody::Err(err) => Err(format!("rollback failed: {} ({:?})", err.message, err.code)),
|
||||
}
|
||||
}
|
||||
|
||||
/// Tear down a single session id (round-trips `tool.session_stop` to the
|
||||
/// owning extension and unregisters on success). Also tears down the
|
||||
/// per-session dispatch queue so the worker task exits.
|
||||
@@ -701,7 +514,6 @@ pub async fn stop_session(
|
||||
interrupts: &Arc<SessionInterruptRegistry>,
|
||||
session_id: &SessionId,
|
||||
timeout_dur: Duration,
|
||||
cancel: Option<AbortToken>,
|
||||
) -> Result<SessionStopResult, StopSessionError> {
|
||||
let session = sessions.get(session_id).ok_or(StopSessionError::NotFound)?;
|
||||
if registry.get(&session.browser_id).is_none() {
|
||||
@@ -712,17 +524,12 @@ pub async fn stop_session(
|
||||
session_id: session_id.0.clone(),
|
||||
})
|
||||
.unwrap();
|
||||
// Internal callers (idle reaping and shutdown) do not have a CLI-owned
|
||||
// token, but still need timeout-driven WS cancellation so a late window
|
||||
// close cannot leave daemon state pointing at a vanished session.
|
||||
let lifecycle_cancel = Some(cancel.unwrap_or_default());
|
||||
match queues
|
||||
.dispatch_after_closing(
|
||||
session_id,
|
||||
bsk_protocol::Method::ToolSessionStop,
|
||||
params,
|
||||
timeout_dur,
|
||||
lifecycle_cancel,
|
||||
)
|
||||
.await
|
||||
{
|
||||
@@ -760,10 +567,6 @@ pub async fn stop_session(
|
||||
drop_session_local(queues, interrupts, session_id);
|
||||
return Ok(SessionStopResult::default());
|
||||
}
|
||||
if err.code == bsk_protocol::ErrorCode::Timeout {
|
||||
queues.reopen(session_id);
|
||||
return Err(StopSessionError::Timeout);
|
||||
}
|
||||
queues.reopen(session_id);
|
||||
Err(StopSessionError::ExtensionError(err))
|
||||
}
|
||||
|
||||
@@ -15,6 +15,7 @@ use std::net::{IpAddr, Ipv4Addr, SocketAddr};
|
||||
use std::path::Path;
|
||||
use std::sync::Arc;
|
||||
use std::sync::Mutex;
|
||||
use std::sync::atomic::{AtomicUsize, Ordering};
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
use anyhow::{Context, Result};
|
||||
@@ -225,6 +226,10 @@ pub fn run_foreground(cfg: DaemonConfig) -> Result<()> {
|
||||
.with_context(|| format!("bind IPC endpoint {}", sock_path.display()))?;
|
||||
|
||||
let state = Arc::new(DaemonState::new(cfg.clone()));
|
||||
state
|
||||
.transfers
|
||||
.initialize()
|
||||
.context("initialize transfer staging")?;
|
||||
let session_idle_task = spawn_session_idle_reaper(Arc::clone(&state));
|
||||
let browser_liveness_task = spawn_browser_liveness_reaper(Arc::clone(&state));
|
||||
// Fired by the update check task after a successful auto-update:
|
||||
@@ -290,7 +295,8 @@ pub fn run_foreground(cfg: DaemonConfig) -> Result<()> {
|
||||
});
|
||||
|
||||
let started_at = Instant::now();
|
||||
let activity = Arc::new(Mutex::new(IpcActivityState::new(started_at)));
|
||||
let activity: Arc<Mutex<Instant>> = Arc::new(Mutex::new(started_at));
|
||||
let active_ipc_connections = Arc::new(AtomicUsize::new(0));
|
||||
let (ipc_shutdown_tx, ipc_shutdown_rx) = tokio::sync::oneshot::channel::<()>();
|
||||
let status = ipc::DaemonStatus {
|
||||
started_at,
|
||||
@@ -312,14 +318,18 @@ pub fn run_foreground(cfg: DaemonConfig) -> Result<()> {
|
||||
let handler = handler.clone();
|
||||
let ipc_open = {
|
||||
let activity = activity.clone();
|
||||
let active = active_ipc_connections.clone();
|
||||
move || {
|
||||
record_ipc_open(&activity);
|
||||
active.fetch_add(1, Ordering::SeqCst);
|
||||
record_activity(&activity);
|
||||
}
|
||||
};
|
||||
let ipc_close = {
|
||||
let activity = activity.clone();
|
||||
let active = active_ipc_connections.clone();
|
||||
move || {
|
||||
record_ipc_close(&activity);
|
||||
active.fetch_sub(1, Ordering::SeqCst);
|
||||
record_activity(&activity);
|
||||
}
|
||||
};
|
||||
tokio::spawn(ipc::serve(
|
||||
@@ -338,6 +348,7 @@ pub fn run_foreground(cfg: DaemonConfig) -> Result<()> {
|
||||
let idle_task = {
|
||||
let activity = activity.clone();
|
||||
let state = Arc::clone(&state);
|
||||
let active_ipc_connections = active_ipc_connections.clone();
|
||||
let daemon_idle = cfg.daemon_idle;
|
||||
tokio::spawn(async move {
|
||||
let tick = (daemon_idle / 4).max(Duration::from_millis(250));
|
||||
@@ -345,19 +356,22 @@ pub fn run_foreground(cfg: DaemonConfig) -> Result<()> {
|
||||
ticker.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Delay);
|
||||
loop {
|
||||
ticker.tick().await;
|
||||
let ipc_is_idle =
|
||||
lock_activity(&activity).should_exit_for_idle(Instant::now(), daemon_idle);
|
||||
if !ipc_is_idle {
|
||||
let last = match activity.lock() {
|
||||
Ok(a) => *a,
|
||||
Err(p) => *p.into_inner(),
|
||||
};
|
||||
if last.elapsed() < daemon_idle {
|
||||
continue;
|
||||
}
|
||||
// IPC liveness (open-connection count + idle window) is already
|
||||
// enforced above by `should_exit_for_idle`, which reads
|
||||
// `IpcActivityState` under one lock and returns true only when
|
||||
// there are zero connections AND the idle interval has elapsed.
|
||||
// Here we additionally hold the daemon alive while any browser
|
||||
// is paired or any session is live (design §3.2, M4/M5
|
||||
// registries).
|
||||
if !state.browsers.is_empty() || !state.sessions.is_empty() {
|
||||
// Bridge from M2/M3 (which only knew about connection
|
||||
// counts) to M4/M5 (browser + session registries):
|
||||
// hold the daemon alive while any IPC client is
|
||||
// connected, any browser is paired, or any session
|
||||
// is live (design §3.2).
|
||||
if active_ipc_connections.load(Ordering::SeqCst) > 0
|
||||
|| !state.browsers.is_empty()
|
||||
|| !state.sessions.is_empty()
|
||||
{
|
||||
continue;
|
||||
}
|
||||
info!(
|
||||
@@ -448,6 +462,7 @@ pub(crate) fn spawn_browser_liveness_reaper(
|
||||
for s in state.sessions.purge_browser(&client.id) {
|
||||
state.tool_queues.remove(&s.id);
|
||||
state.session_interrupts.drop_session(&s.id);
|
||||
state.transfers.release_session(&s.id.0);
|
||||
debug!(session = %s.id, "purged session on browser liveness timeout");
|
||||
}
|
||||
}
|
||||
@@ -481,11 +496,13 @@ pub(crate) fn spawn_session_idle_reaper(state: Arc<DaemonState>) -> tokio::task:
|
||||
&state.session_interrupts,
|
||||
&session_id,
|
||||
Duration::from_secs(10),
|
||||
None,
|
||||
)
|
||||
.await
|
||||
{
|
||||
Ok(_) => info!(session = %session_id, "idle session stopped"),
|
||||
Ok(_) => {
|
||||
state.transfers.release_session(&session_id.0);
|
||||
info!(session = %session_id, "idle session stopped");
|
||||
}
|
||||
Err(StopSessionError::SessionBusy | StopSessionError::Stopping) => {
|
||||
debug!(session = %session_id, "idle session still active; retrying later");
|
||||
}
|
||||
@@ -496,6 +513,7 @@ pub(crate) fn spawn_session_idle_reaper(state: Arc<DaemonState>) -> tokio::task:
|
||||
&state.session_interrupts,
|
||||
&session_id,
|
||||
);
|
||||
state.transfers.release_session(&session_id.0);
|
||||
}
|
||||
Err(err) => {
|
||||
warn!(session = %session_id, error = %err, "failed to stop idle session");
|
||||
@@ -672,55 +690,10 @@ fn restart_start_args(cfg: &DaemonConfig) -> StartArgs {
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
struct IpcActivityState {
|
||||
last_activity: Instant,
|
||||
active_connections: usize,
|
||||
}
|
||||
|
||||
impl IpcActivityState {
|
||||
fn new(now: Instant) -> Self {
|
||||
Self {
|
||||
last_activity: now,
|
||||
active_connections: 0,
|
||||
}
|
||||
fn record_activity(activity: &Arc<Mutex<Instant>>) {
|
||||
if let Ok(mut a) = activity.lock() {
|
||||
*a = Instant::now();
|
||||
}
|
||||
|
||||
fn touch(&mut self, now: Instant) {
|
||||
self.last_activity = now;
|
||||
}
|
||||
|
||||
fn opened(&mut self, now: Instant) {
|
||||
self.active_connections = self.active_connections.saturating_add(1);
|
||||
self.last_activity = now;
|
||||
}
|
||||
|
||||
fn closed(&mut self, now: Instant) {
|
||||
self.active_connections = self.active_connections.saturating_sub(1);
|
||||
self.last_activity = now;
|
||||
}
|
||||
|
||||
fn should_exit_for_idle(&self, now: Instant, idle: Duration) -> bool {
|
||||
self.active_connections == 0 && now.saturating_duration_since(self.last_activity) >= idle
|
||||
}
|
||||
}
|
||||
|
||||
fn lock_activity(
|
||||
activity: &Arc<Mutex<IpcActivityState>>,
|
||||
) -> std::sync::MutexGuard<'_, IpcActivityState> {
|
||||
activity.lock().unwrap_or_else(|poison| poison.into_inner())
|
||||
}
|
||||
|
||||
fn record_activity(activity: &Arc<Mutex<IpcActivityState>>) {
|
||||
lock_activity(activity).touch(Instant::now());
|
||||
}
|
||||
|
||||
fn record_ipc_open(activity: &Arc<Mutex<IpcActivityState>>) {
|
||||
lock_activity(activity).opened(Instant::now());
|
||||
}
|
||||
|
||||
fn record_ipc_close(activity: &Arc<Mutex<IpcActivityState>>) {
|
||||
lock_activity(activity).closed(Instant::now());
|
||||
}
|
||||
|
||||
/// Initialise tracing for the daemon: write a daily-rotated file in
|
||||
@@ -1103,37 +1076,4 @@ mod tests {
|
||||
assert_eq!(args.session_idle, Some(Duration::from_secs(11)));
|
||||
assert_eq!(args.daemon_idle, Some(Duration::from_secs(22)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ipc_close_restarts_the_idle_window_atomically() {
|
||||
let started = Instant::now();
|
||||
let idle = Duration::from_secs(10);
|
||||
let mut activity = IpcActivityState::new(started);
|
||||
|
||||
activity.opened(started);
|
||||
let closed_at = started + Duration::from_secs(30);
|
||||
assert!(
|
||||
!activity.should_exit_for_idle(closed_at, idle),
|
||||
"an open IPC connection must hold the daemon alive"
|
||||
);
|
||||
|
||||
activity.closed(closed_at);
|
||||
assert_eq!(activity.active_connections, 0);
|
||||
assert_eq!(activity.last_activity, closed_at);
|
||||
assert!(!activity.should_exit_for_idle(closed_at, idle));
|
||||
assert!(!activity.should_exit_for_idle(closed_at + idle - Duration::from_millis(1), idle));
|
||||
assert!(activity.should_exit_for_idle(closed_at + idle, idle));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ipc_activity_keeps_the_daemon_alive_until_a_full_idle_interval_passes() {
|
||||
let started = Instant::now();
|
||||
let idle = Duration::from_secs(10);
|
||||
let mut activity = IpcActivityState::new(started);
|
||||
let request_at = started + Duration::from_secs(20);
|
||||
|
||||
activity.touch(request_at);
|
||||
assert!(!activity.should_exit_for_idle(request_at + Duration::from_secs(9), idle));
|
||||
assert!(activity.should_exit_for_idle(request_at + idle, idle));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -7,6 +7,7 @@ use tokio::task::JoinHandle;
|
||||
|
||||
use super::abort::AbortRegistry;
|
||||
use super::browsers::BrowserRegistry;
|
||||
use super::file_transfer::TransferRegistry;
|
||||
use super::inflight::ToolInflightRegistry;
|
||||
use super::ipc::IpcHandle;
|
||||
use super::queue::ToolQueueRegistry;
|
||||
@@ -34,8 +35,8 @@ pub struct DaemonState {
|
||||
/// `stop_session` / browser disconnect.
|
||||
pub tool_queues: Arc<ToolQueueRegistry>,
|
||||
/// Per-rpc-id cancellation tokens for daemon-side long-runners
|
||||
/// (`tool.wait_ms` plus `session.*` lifecycle calls). The CLI's
|
||||
/// `cancel { rpc_id }` consults this registry first.
|
||||
/// (M9.3 — currently only `tool.wait_ms`). The CLI's `cancel
|
||||
/// { rpc_id }` consults this registry first.
|
||||
pub abort_registry: Arc<AbortRegistry>,
|
||||
/// Tracks `tool.*` RPCs that have been forwarded to an extension
|
||||
/// over WS but have not yet received a response. Indexed by the
|
||||
@@ -51,6 +52,9 @@ pub struct DaemonState {
|
||||
/// `SessionRegistry` because the signal is a transient runtime
|
||||
/// control state.
|
||||
pub session_interrupts: Arc<SessionInterruptRegistry>,
|
||||
/// Operation-scoped local file staging. The extension only sees paths
|
||||
/// minted here; agent-facing RPCs use opaque transfer ids.
|
||||
pub transfers: Arc<TransferRegistry>,
|
||||
}
|
||||
|
||||
impl DaemonState {
|
||||
@@ -64,6 +68,7 @@ impl DaemonState {
|
||||
));
|
||||
let abort_registry = Arc::new(AbortRegistry::new());
|
||||
let session_interrupts = Arc::new(SessionInterruptRegistry::new());
|
||||
let transfers = Arc::new(TransferRegistry::new().expect("initialise transfer staging"));
|
||||
Self {
|
||||
config,
|
||||
browsers,
|
||||
@@ -72,6 +77,7 @@ impl DaemonState {
|
||||
abort_registry,
|
||||
tool_inflight,
|
||||
session_interrupts,
|
||||
transfers,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -295,6 +295,7 @@ async fn drive_connection(
|
||||
for session in state.sessions.purge_browser(&browser_id) {
|
||||
state.tool_queues.remove(&session.id);
|
||||
state.session_interrupts.drop_session(&session.id);
|
||||
state.transfers.release_session(&session.id.0);
|
||||
debug!(session = %session.id, "purged stale session before browser reconnect");
|
||||
}
|
||||
let (tx, mut rx) = mpsc::unbounded_channel::<Frame>();
|
||||
@@ -407,6 +408,7 @@ async fn drive_connection(
|
||||
for s in state.sessions.purge_browser(&browser_id) {
|
||||
state.tool_queues.remove(&s.id);
|
||||
state.session_interrupts.drop_session(&s.id);
|
||||
state.transfers.release_session(&s.id.0);
|
||||
debug!(session = %s.id, "purged session on browser disconnect");
|
||||
}
|
||||
} else {
|
||||
@@ -503,6 +505,7 @@ fn handle_session_window_closed(
|
||||
&state.session_interrupts,
|
||||
&session_id,
|
||||
) {
|
||||
state.transfers.release_session(&session_id.0);
|
||||
info!(session = %session_id, "session removed: user closed Agent Window");
|
||||
} else {
|
||||
debug!(session = %session_id, "session.window_closed for unknown session id");
|
||||
|
||||
@@ -97,6 +97,8 @@ fn dispatch(cli: Cli, format: Format) -> Result<(), CliError> {
|
||||
Command::Fill(args) => cli::interaction::dispatch_fill(args, format),
|
||||
Command::Press(args) => cli::interaction::dispatch_press(args, format),
|
||||
Command::Select(args) => cli::interaction::dispatch_select(args, format),
|
||||
Command::Upload(args) => cli::upload::dispatch(args, format),
|
||||
Command::Download(args) => cli::download::dispatch(args, format),
|
||||
Command::Evaluate(args) => cli::evaluate::dispatch(args, format),
|
||||
Command::WaitForNavigation(args) => cli::waits::dispatch_wait_for_navigation(args, format),
|
||||
Command::WaitMs(args) => cli::waits::dispatch_wait_ms(args, format),
|
||||
|
||||
@@ -213,6 +213,45 @@ fn rejects_zero_click_count() {
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parses_upload_with_repeated_files() {
|
||||
let cli = parse(&[
|
||||
"bsk",
|
||||
"upload",
|
||||
"@e3",
|
||||
"--file",
|
||||
"one.png",
|
||||
"--file",
|
||||
"two.png",
|
||||
"--session",
|
||||
"s1",
|
||||
]);
|
||||
let Command::Upload(args) = cli.command else {
|
||||
panic!("expected upload command");
|
||||
};
|
||||
assert_eq!(args.target.as_deref(), Some("@e3"));
|
||||
assert_eq!(args.files.len(), 2);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parses_download_with_exact_output_policy() {
|
||||
let cli = parse(&[
|
||||
"bsk",
|
||||
"download",
|
||||
"#export",
|
||||
"--out",
|
||||
"result.zip",
|
||||
"--session",
|
||||
"s1",
|
||||
"--overwrite",
|
||||
]);
|
||||
let Command::Download(args) = cli.command else {
|
||||
panic!("expected download command");
|
||||
};
|
||||
assert_eq!(args.target.as_deref(), Some("#export"));
|
||||
assert!(args.overwrite);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parses_record_start_with_browser_and_url() {
|
||||
let cli = parse(&[
|
||||
|
||||
@@ -325,7 +325,6 @@ async fn session_stop_fast_fails_while_tool_is_in_flight() {
|
||||
&state.session_interrupts,
|
||||
&sid,
|
||||
Duration::from_secs(5),
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
assert!(
|
||||
@@ -367,7 +366,6 @@ async fn session_stop_fast_fails_while_tool_is_in_flight() {
|
||||
&state.session_interrupts,
|
||||
&sid,
|
||||
Duration::from_secs(5),
|
||||
None,
|
||||
)
|
||||
.await
|
||||
.expect("session.stop ok after tool completes");
|
||||
|
||||
@@ -10,14 +10,9 @@ use std::time::Duration;
|
||||
|
||||
use bsk::daemon::{self, DaemonConfig};
|
||||
use bsk::ipc_client::IpcClient;
|
||||
use bsk_protocol::cancel::{CancelParams, CancelResult};
|
||||
use bsk_protocol::system::{HandshakeParams, HandshakeResult, StatusResult};
|
||||
use bsk_protocol::tools::{
|
||||
SessionStartParams, SessionStartResult, SessionStopParams, SessionStopResult,
|
||||
};
|
||||
use bsk_protocol::{
|
||||
BrowserPeerInfo, ErrorCode, Frame, Method, RequestFrame, ResponseBody, ResponseFrame, RpcError,
|
||||
};
|
||||
use bsk_protocol::tools::{SessionStartParams, SessionStartResult, SessionStopParams};
|
||||
use bsk_protocol::{BrowserPeerInfo, Frame, Method, RequestFrame, ResponseBody, ResponseFrame};
|
||||
use futures_util::{SinkExt, StreamExt};
|
||||
use rand::Rng;
|
||||
use tokio_tungstenite::tungstenite::handshake::client::generate_key;
|
||||
@@ -28,9 +23,6 @@ use support::{wait_for_browser_count, wait_for_no_sessions};
|
||||
|
||||
const TEST_EXT_ID: &str = "abcdefghijklmnopabcdefghijklmnop";
|
||||
|
||||
type TestWs =
|
||||
tokio_tungstenite::WebSocketStream<tokio_tungstenite::MaybeTlsStream<tokio::net::TcpStream>>;
|
||||
|
||||
fn tempfile_path(prefix: &str) -> PathBuf {
|
||||
let mut p = std::env::temp_dir();
|
||||
let mut rng = rand::thread_rng();
|
||||
@@ -64,7 +56,9 @@ async fn spawn_daemon_with_session_idle(session_idle: Duration) -> (daemon::Daem
|
||||
(handle, sock)
|
||||
}
|
||||
|
||||
async fn connect_ext(addr: std::net::SocketAddr) -> TestWs {
|
||||
async fn connect_ext(
|
||||
addr: std::net::SocketAddr,
|
||||
) -> tokio_tungstenite::WebSocketStream<tokio_tungstenite::MaybeTlsStream<tokio::net::TcpStream>> {
|
||||
let origin = format!("chrome-extension://{TEST_EXT_ID}");
|
||||
let url = format!("ws://{addr}/");
|
||||
let req = Request::builder()
|
||||
@@ -82,7 +76,11 @@ async fn connect_ext(addr: std::net::SocketAddr) -> TestWs {
|
||||
ws
|
||||
}
|
||||
|
||||
async fn handshake_as_ext(ws: &mut TestWs) -> HandshakeResult {
|
||||
async fn handshake_as_ext(
|
||||
ws: &mut tokio_tungstenite::WebSocketStream<
|
||||
tokio_tungstenite::MaybeTlsStream<tokio::net::TcpStream>,
|
||||
>,
|
||||
) -> HandshakeResult {
|
||||
let params = HandshakeParams {
|
||||
client: "browser-skill-extension".into(),
|
||||
version: "0.1.0-dev.0".parse().unwrap(),
|
||||
@@ -116,128 +114,6 @@ async fn handshake_as_ext(ws: &mut TestWs) -> HandshakeResult {
|
||||
}
|
||||
}
|
||||
|
||||
async fn next_extension_request(ws: &mut TestWs) -> RequestFrame {
|
||||
loop {
|
||||
let message = ws.next().await.expect("daemon closed websocket").unwrap();
|
||||
match message {
|
||||
Message::Text(text) => match serde_json::from_str::<Frame>(&text).unwrap() {
|
||||
Frame::Request(request) => return request,
|
||||
Frame::Response(_) | Frame::Event(_) => continue,
|
||||
},
|
||||
Message::Ping(payload) => ws.send(Message::Pong(payload)).await.unwrap(),
|
||||
Message::Close(_) => panic!("daemon closed websocket before sending request"),
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async fn send_extension_response(ws: &mut TestWs, response: ResponseFrame) {
|
||||
ws.send(Message::Text(serde_json::to_string(&response).unwrap()))
|
||||
.await
|
||||
.unwrap();
|
||||
}
|
||||
|
||||
async fn acknowledge_extension_cancel(ws: &mut TestWs, cancel: RequestFrame, target_rpc_id: &str) {
|
||||
assert_eq!(cancel.method, Method::Cancel);
|
||||
let params: CancelParams = serde_json::from_value(cancel.params.unwrap()).unwrap();
|
||||
assert_eq!(params.rpc_id, target_rpc_id);
|
||||
send_extension_response(
|
||||
ws,
|
||||
ResponseFrame {
|
||||
id: cancel.id,
|
||||
body: ResponseBody::Ok(serde_json::to_value(CancelResult { cancelled: true }).unwrap()),
|
||||
},
|
||||
)
|
||||
.await;
|
||||
}
|
||||
|
||||
async fn respond_to_aborted_start(
|
||||
ws: &mut TestWs,
|
||||
start_seen: Option<tokio::sync::oneshot::Sender<()>>,
|
||||
agent_window_id: i64,
|
||||
) {
|
||||
let start = next_extension_request(ws).await;
|
||||
assert_eq!(start.method, Method::ToolSessionStart);
|
||||
let params: SessionStartParams = serde_json::from_value(start.params.clone().unwrap()).unwrap();
|
||||
if let Some(start_seen) = start_seen {
|
||||
start_seen.send(()).unwrap();
|
||||
}
|
||||
|
||||
let cancel = next_extension_request(ws).await;
|
||||
acknowledge_extension_cancel(ws, cancel, &start.id).await;
|
||||
|
||||
// Model the hardest race: chrome.windows.create completed just as
|
||||
// cancellation arrived, so the extension's original call succeeds.
|
||||
send_extension_response(
|
||||
ws,
|
||||
ResponseFrame {
|
||||
id: start.id,
|
||||
body: ResponseBody::Ok(
|
||||
serde_json::to_value(SessionStartResult {
|
||||
agent_window_id: Some(agent_window_id),
|
||||
})
|
||||
.unwrap(),
|
||||
),
|
||||
},
|
||||
)
|
||||
.await;
|
||||
|
||||
let rollback = next_extension_request(ws).await;
|
||||
assert_eq!(rollback.method, Method::ToolSessionStop);
|
||||
let rollback_params: SessionStopParams =
|
||||
serde_json::from_value(rollback.params.unwrap()).unwrap();
|
||||
assert_eq!(rollback_params.session_id, params.session_id);
|
||||
send_extension_response(
|
||||
ws,
|
||||
ResponseFrame {
|
||||
id: rollback.id,
|
||||
body: ResponseBody::Ok(serde_json::to_value(SessionStopResult::default()).unwrap()),
|
||||
},
|
||||
)
|
||||
.await;
|
||||
}
|
||||
|
||||
async fn respond_to_aborted_stop(
|
||||
ws: &mut TestWs,
|
||||
stop_seen: Option<tokio::sync::oneshot::Sender<()>>,
|
||||
) {
|
||||
let start = next_extension_request(ws).await;
|
||||
assert_eq!(start.method, Method::ToolSessionStart);
|
||||
send_extension_response(
|
||||
ws,
|
||||
ResponseFrame {
|
||||
id: start.id,
|
||||
body: ResponseBody::Ok(
|
||||
serde_json::to_value(SessionStartResult {
|
||||
agent_window_id: Some(4242),
|
||||
})
|
||||
.unwrap(),
|
||||
),
|
||||
},
|
||||
)
|
||||
.await;
|
||||
|
||||
let stop = next_extension_request(ws).await;
|
||||
assert_eq!(stop.method, Method::ToolSessionStop);
|
||||
if let Some(stop_seen) = stop_seen {
|
||||
stop_seen.send(()).unwrap();
|
||||
}
|
||||
let cancel = next_extension_request(ws).await;
|
||||
acknowledge_extension_cancel(ws, cancel, &stop.id).await;
|
||||
send_extension_response(
|
||||
ws,
|
||||
ResponseFrame {
|
||||
id: stop.id,
|
||||
body: ResponseBody::Err(RpcError {
|
||||
code: ErrorCode::Cancelled,
|
||||
message: "session_stop aborted before window close".into(),
|
||||
data: None,
|
||||
}),
|
||||
},
|
||||
)
|
||||
.await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn session_start_stop_round_trip_via_ipc() {
|
||||
let (handle, sock) = spawn_daemon().await;
|
||||
@@ -374,240 +250,6 @@ async fn session_start_stop_round_trip_via_ipc() {
|
||||
handle.shutdown().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn cancelling_session_start_rolls_back_a_late_extension_success() {
|
||||
const START_RPC_ID: &str = "session-start-cancel";
|
||||
|
||||
let (handle, sock) = spawn_daemon().await;
|
||||
let mut ws = connect_ext(handle.ws_addr()).await;
|
||||
let _ = handshake_as_ext(&mut ws).await;
|
||||
let (start_seen_tx, start_seen_rx) = tokio::sync::oneshot::channel();
|
||||
let (release_tx, release_rx) = tokio::sync::oneshot::channel();
|
||||
|
||||
let responder = tokio::spawn(async move {
|
||||
respond_to_aborted_start(&mut ws, Some(start_seen_tx), 4242).await;
|
||||
let _ = release_rx.await;
|
||||
});
|
||||
|
||||
let mut start_ipc = IpcClient::connect(&sock).await.unwrap();
|
||||
let start_call = tokio::spawn(async move {
|
||||
let outcome: Result<serde_json::Value, RpcError> = start_ipc
|
||||
.call_with_id(
|
||||
START_RPC_ID.to_string(),
|
||||
Method::SessionStart,
|
||||
Some(serde_json::json!({ "focused": false })),
|
||||
Duration::from_secs(5),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
outcome
|
||||
});
|
||||
|
||||
start_seen_rx.await.unwrap();
|
||||
let mut cancel_ipc = IpcClient::connect(&sock).await.unwrap();
|
||||
let cancel: CancelResult = cancel_ipc
|
||||
.call(
|
||||
"cancel-start",
|
||||
Method::Cancel,
|
||||
Some(CancelParams {
|
||||
rpc_id: START_RPC_ID.to_string(),
|
||||
}),
|
||||
Duration::from_secs(2),
|
||||
)
|
||||
.await
|
||||
.unwrap()
|
||||
.unwrap();
|
||||
assert!(cancel.cancelled);
|
||||
|
||||
let start_error = start_call.await.unwrap().unwrap_err();
|
||||
assert_eq!(start_error.code, ErrorCode::Cancelled);
|
||||
|
||||
let mut status_ipc = IpcClient::connect(&sock).await.unwrap();
|
||||
let status: StatusResult = status_ipc
|
||||
.call::<(), _>(
|
||||
"status-after-start-cancel",
|
||||
Method::SystemStatus,
|
||||
None,
|
||||
Duration::from_secs(2),
|
||||
)
|
||||
.await
|
||||
.unwrap()
|
||||
.unwrap();
|
||||
assert!(status.sessions.is_empty());
|
||||
assert_eq!(status.browsers[0].session_count, 0);
|
||||
|
||||
let _ = release_tx.send(());
|
||||
responder.await.unwrap();
|
||||
handle.shutdown().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn timing_out_session_start_rolls_back_a_late_extension_success() {
|
||||
let (handle, _sock) = spawn_daemon().await;
|
||||
let state = handle.state();
|
||||
let mut ws = connect_ext(handle.ws_addr()).await;
|
||||
let _ = handshake_as_ext(&mut ws).await;
|
||||
let (release_tx, release_rx) = tokio::sync::oneshot::channel();
|
||||
|
||||
let responder = tokio::spawn(async move {
|
||||
respond_to_aborted_start(&mut ws, None, 4343).await;
|
||||
let _ = release_rx.await;
|
||||
});
|
||||
|
||||
let start = bsk::daemon::sessions::start_session(
|
||||
&state.browsers,
|
||||
&state.sessions,
|
||||
&state.tool_queues,
|
||||
None,
|
||||
bsk::daemon::sessions::AgentWindowOptions::default(),
|
||||
Duration::ZERO,
|
||||
Duration::from_millis(20),
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
assert!(matches!(
|
||||
start,
|
||||
Err(bsk::daemon::sessions::StartSessionError::Timeout)
|
||||
));
|
||||
assert!(state.sessions.is_empty());
|
||||
|
||||
let _ = release_tx.send(());
|
||||
responder.await.unwrap();
|
||||
handle.shutdown().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn cancelling_session_stop_keeps_the_session_retryable() {
|
||||
const STOP_RPC_ID: &str = "session-stop-cancel";
|
||||
|
||||
let (handle, sock) = spawn_daemon().await;
|
||||
let mut ws = connect_ext(handle.ws_addr()).await;
|
||||
let _ = handshake_as_ext(&mut ws).await;
|
||||
let (stop_seen_tx, stop_seen_rx) = tokio::sync::oneshot::channel();
|
||||
let (release_tx, release_rx) = tokio::sync::oneshot::channel();
|
||||
|
||||
let responder = tokio::spawn(async move {
|
||||
respond_to_aborted_stop(&mut ws, Some(stop_seen_tx)).await;
|
||||
let _ = release_rx.await;
|
||||
});
|
||||
|
||||
let mut start_ipc = IpcClient::connect(&sock).await.unwrap();
|
||||
let start: serde_json::Value = start_ipc
|
||||
.call(
|
||||
"start-before-stop-cancel",
|
||||
Method::SessionStart,
|
||||
Some(serde_json::json!({ "focused": false })),
|
||||
Duration::from_secs(5),
|
||||
)
|
||||
.await
|
||||
.unwrap()
|
||||
.unwrap();
|
||||
let session_id = start["session_id"].as_str().unwrap().to_string();
|
||||
|
||||
let mut stop_ipc = IpcClient::connect(&sock).await.unwrap();
|
||||
let stop_session_id = session_id.clone();
|
||||
let stop_call = tokio::spawn(async move {
|
||||
let outcome: Result<serde_json::Value, RpcError> = stop_ipc
|
||||
.call_with_id(
|
||||
STOP_RPC_ID.to_string(),
|
||||
Method::SessionStop,
|
||||
Some(serde_json::json!({
|
||||
"session_id": stop_session_id,
|
||||
"all": false,
|
||||
})),
|
||||
Duration::from_secs(5),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
outcome
|
||||
});
|
||||
|
||||
stop_seen_rx.await.unwrap();
|
||||
let mut cancel_ipc = IpcClient::connect(&sock).await.unwrap();
|
||||
let cancel: CancelResult = cancel_ipc
|
||||
.call(
|
||||
"cancel-stop",
|
||||
Method::Cancel,
|
||||
Some(CancelParams {
|
||||
rpc_id: STOP_RPC_ID.to_string(),
|
||||
}),
|
||||
Duration::from_secs(2),
|
||||
)
|
||||
.await
|
||||
.unwrap()
|
||||
.unwrap();
|
||||
assert!(cancel.cancelled);
|
||||
|
||||
let stop_error = stop_call.await.unwrap().unwrap_err();
|
||||
assert_eq!(stop_error.code, ErrorCode::Cancelled);
|
||||
|
||||
let mut status_ipc = IpcClient::connect(&sock).await.unwrap();
|
||||
let status: StatusResult = status_ipc
|
||||
.call::<(), _>(
|
||||
"status-after-stop-cancel",
|
||||
Method::SystemStatus,
|
||||
None,
|
||||
Duration::from_secs(2),
|
||||
)
|
||||
.await
|
||||
.unwrap()
|
||||
.unwrap();
|
||||
assert_eq!(status.sessions.len(), 1);
|
||||
assert_eq!(status.sessions[0].session_id, session_id);
|
||||
|
||||
let _ = release_tx.send(());
|
||||
responder.await.unwrap();
|
||||
handle.shutdown().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn timing_out_session_stop_cancels_extension_teardown_and_keeps_session() {
|
||||
let (handle, sock) = spawn_daemon().await;
|
||||
let state = handle.state();
|
||||
let mut ws = connect_ext(handle.ws_addr()).await;
|
||||
let _ = handshake_as_ext(&mut ws).await;
|
||||
let (release_tx, release_rx) = tokio::sync::oneshot::channel();
|
||||
|
||||
let responder = tokio::spawn(async move {
|
||||
respond_to_aborted_stop(&mut ws, None).await;
|
||||
let _ = release_rx.await;
|
||||
});
|
||||
|
||||
let mut start_ipc = IpcClient::connect(&sock).await.unwrap();
|
||||
let start: serde_json::Value = start_ipc
|
||||
.call(
|
||||
"start-before-stop-timeout",
|
||||
Method::SessionStart,
|
||||
Some(serde_json::json!({ "focused": false })),
|
||||
Duration::from_secs(5),
|
||||
)
|
||||
.await
|
||||
.unwrap()
|
||||
.unwrap();
|
||||
let session_id =
|
||||
bsk::daemon::sessions::SessionId(start["session_id"].as_str().unwrap().to_string());
|
||||
|
||||
let stop = bsk::daemon::sessions::stop_session(
|
||||
&state.browsers,
|
||||
&state.sessions,
|
||||
&state.tool_queues,
|
||||
&state.session_interrupts,
|
||||
&session_id,
|
||||
Duration::from_millis(20),
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
assert!(matches!(
|
||||
stop,
|
||||
Err(bsk::daemon::sessions::StopSessionError::Timeout)
|
||||
));
|
||||
assert!(state.sessions.get(&session_id).is_some());
|
||||
|
||||
let _ = release_tx.send(());
|
||||
responder.await.unwrap();
|
||||
handle.shutdown().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn session_idle_timeout_stops_and_unregisters_session() {
|
||||
let (handle, _sock) = spawn_daemon_with_session_idle(Duration::from_millis(50)).await;
|
||||
|
||||
@@ -0,0 +1,56 @@
|
||||
{
|
||||
"$schema": "http://json-schema.org/draft-07/schema#",
|
||||
"title": "DownloadParams",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"session_id"
|
||||
],
|
||||
"properties": {
|
||||
"browser_relative_dir": {
|
||||
"description": "Daemon-injected relative directory beneath Chrome's Downloads root.",
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"max_byte_size": {
|
||||
"description": "Daemon-injected authoritative transfer size limit. The extension uses it only for early cancellation; daemon import remains the final check.",
|
||||
"type": [
|
||||
"integer",
|
||||
"null"
|
||||
],
|
||||
"format": "uint64",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"ref": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"selector": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"session_id": {
|
||||
"type": "string"
|
||||
},
|
||||
"tab_id": {
|
||||
"type": [
|
||||
"integer",
|
||||
"null"
|
||||
],
|
||||
"format": "int64"
|
||||
},
|
||||
"timeout_ms": {
|
||||
"type": [
|
||||
"integer",
|
||||
"null"
|
||||
],
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,62 @@
|
||||
{
|
||||
"$schema": "http://json-schema.org/draft-07/schema#",
|
||||
"title": "DownloadResult",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"byte_size",
|
||||
"suggested_filename",
|
||||
"tab_id"
|
||||
],
|
||||
"properties": {
|
||||
"browser_path": {
|
||||
"description": "Extension-internal completed Chrome download path, stripped by the daemon.",
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"byte_size": {
|
||||
"type": "integer",
|
||||
"format": "uint64",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"danger": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"mime": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"suggested_filename": {
|
||||
"type": "string"
|
||||
},
|
||||
"tab_id": {
|
||||
"type": "integer",
|
||||
"format": "int64"
|
||||
},
|
||||
"transfer_id": {
|
||||
"description": "Opaque id returned by the daemon to the CLI.",
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"used_ref": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"used_selector": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -7,18 +7,10 @@
|
||||
],
|
||||
"properties": {
|
||||
"trace": {
|
||||
"$ref": "#/definitions/RecordedTrace"
|
||||
"$ref": "#/definitions/Trace"
|
||||
}
|
||||
},
|
||||
"definitions": {
|
||||
"FillCommit": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"enter",
|
||||
"suggestion",
|
||||
"blur"
|
||||
]
|
||||
},
|
||||
"KeyModifier": {
|
||||
"description": "Keyboard modifier flags. Multiple flags may be combined; the extension folds them into CDP's bitfield (`alt=1, ctrl=2, meta=4, shift=8`).",
|
||||
"type": "string",
|
||||
@@ -29,19 +21,7 @@
|
||||
"shift"
|
||||
]
|
||||
},
|
||||
"NavigationCause": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"user_typed",
|
||||
"link",
|
||||
"form_submit",
|
||||
"reload",
|
||||
"history",
|
||||
"script",
|
||||
"browser"
|
||||
]
|
||||
},
|
||||
"PageRefV2": {
|
||||
"PageRef": {
|
||||
"description": "Page context dictionary entry — referenced by steps via `page` id.",
|
||||
"type": "object",
|
||||
"required": [
|
||||
@@ -63,42 +43,7 @@
|
||||
}
|
||||
}
|
||||
},
|
||||
"RecordedTrace": {
|
||||
"if": {
|
||||
"required": [
|
||||
"version"
|
||||
],
|
||||
"properties": {
|
||||
"version": {
|
||||
"type": "integer"
|
||||
}
|
||||
}
|
||||
},
|
||||
"then": {
|
||||
"$ref": "#/definitions/TraceV3"
|
||||
},
|
||||
"else": {
|
||||
"$ref": "#/definitions/TraceV2"
|
||||
}
|
||||
},
|
||||
"RecorderInfo": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"bsk",
|
||||
"vom"
|
||||
],
|
||||
"properties": {
|
||||
"bsk": {
|
||||
"type": "string"
|
||||
},
|
||||
"vom": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
}
|
||||
}
|
||||
},
|
||||
"SelectedOptionV2": {
|
||||
"SelectedOption": {
|
||||
"description": "One selected option (`select` op).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
@@ -116,53 +61,11 @@
|
||||
}
|
||||
}
|
||||
},
|
||||
"SelectedOptionV3": {
|
||||
"description": "One selected option (`select` op).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"value"
|
||||
],
|
||||
"properties": {
|
||||
"label": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"value": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"StepEffectV2": {
|
||||
"description": "Observed navigation after a step (objective fact only).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"navigated_to"
|
||||
],
|
||||
"properties": {
|
||||
"navigated_to": {
|
||||
"description": "Reference into `pages[]` for the destination page.",
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"StepResultV3": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"state"
|
||||
],
|
||||
"properties": {
|
||||
"state": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"StepV2": {
|
||||
"description": "One recorded user action — discriminated union by `op` (v2).",
|
||||
"Step": {
|
||||
"description": "One recorded user action — discriminated union by `op`.",
|
||||
"oneOf": [
|
||||
{
|
||||
"description": "Fields shared by every v2 step variant (flattened in JSON).",
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
@@ -174,7 +77,7 @@
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffectV2"
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
@@ -202,7 +105,7 @@
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every v2 step variant (flattened in JSON).",
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
@@ -214,7 +117,7 @@
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffectV2"
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
@@ -237,12 +140,12 @@
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV2"
|
||||
"$ref": "#/definitions/TargetDescriptor"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every v2 step variant (flattened in JSON).",
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
@@ -254,7 +157,7 @@
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffectV2"
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
@@ -277,12 +180,12 @@
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV2"
|
||||
"$ref": "#/definitions/TargetDescriptor"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every v2 step variant (flattened in JSON).",
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
@@ -295,7 +198,7 @@
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffectV2"
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
@@ -324,7 +227,7 @@
|
||||
]
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV2"
|
||||
"$ref": "#/definitions/TargetDescriptor"
|
||||
},
|
||||
"value": {
|
||||
"type": "string"
|
||||
@@ -332,7 +235,7 @@
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every v2 step variant (flattened in JSON).",
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
@@ -345,7 +248,7 @@
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffectV2"
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
@@ -370,16 +273,16 @@
|
||||
"selection": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/SelectedOptionV2"
|
||||
"$ref": "#/definitions/SelectedOption"
|
||||
}
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV2"
|
||||
"$ref": "#/definitions/TargetDescriptor"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every v2 step variant (flattened in JSON).",
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
@@ -391,7 +294,7 @@
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffectV2"
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
@@ -428,7 +331,7 @@
|
||||
"target": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/TargetDescriptorV2"
|
||||
"$ref": "#/definitions/TargetDescriptor"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
@@ -439,294 +342,21 @@
|
||||
}
|
||||
]
|
||||
},
|
||||
"StepV3": {
|
||||
"description": "One recorded user action — discriminated union by `op`.",
|
||||
"oneOf": [
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"cause",
|
||||
"id",
|
||||
"op",
|
||||
"result",
|
||||
"state",
|
||||
"to"
|
||||
],
|
||||
"properties": {
|
||||
"cause": {
|
||||
"$ref": "#/definitions/NavigationCause"
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"navigate"
|
||||
]
|
||||
},
|
||||
"result": {
|
||||
"$ref": "#/definitions/StepResultV3"
|
||||
},
|
||||
"state": {
|
||||
"description": "Observation id immediately before this action.",
|
||||
"type": "string"
|
||||
},
|
||||
"to": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"result",
|
||||
"state",
|
||||
"target"
|
||||
],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"click"
|
||||
]
|
||||
},
|
||||
"result": {
|
||||
"$ref": "#/definitions/StepResultV3"
|
||||
},
|
||||
"state": {
|
||||
"description": "Observation id immediately before this action.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV3"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"result",
|
||||
"state",
|
||||
"target"
|
||||
],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"hover"
|
||||
]
|
||||
},
|
||||
"result": {
|
||||
"$ref": "#/definitions/StepResultV3"
|
||||
},
|
||||
"state": {
|
||||
"description": "Observation id immediately before this action.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV3"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"commit",
|
||||
"id",
|
||||
"op",
|
||||
"result",
|
||||
"state",
|
||||
"target",
|
||||
"value"
|
||||
],
|
||||
"properties": {
|
||||
"commit": {
|
||||
"$ref": "#/definitions/FillCommit"
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"fill"
|
||||
]
|
||||
},
|
||||
"redacted": {
|
||||
"type": "boolean"
|
||||
},
|
||||
"result": {
|
||||
"$ref": "#/definitions/StepResultV3"
|
||||
},
|
||||
"state": {
|
||||
"description": "Observation id immediately before this action.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV3"
|
||||
},
|
||||
"value": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"result",
|
||||
"state",
|
||||
"target"
|
||||
],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"select"
|
||||
]
|
||||
},
|
||||
"result": {
|
||||
"$ref": "#/definitions/StepResultV3"
|
||||
},
|
||||
"selection": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/SelectedOptionV3"
|
||||
}
|
||||
},
|
||||
"state": {
|
||||
"description": "Observation id immediately before this action.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV3"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"key",
|
||||
"op",
|
||||
"result",
|
||||
"state"
|
||||
],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"key": {
|
||||
"type": "string"
|
||||
},
|
||||
"modifiers": {
|
||||
"type": [
|
||||
"array",
|
||||
"null"
|
||||
],
|
||||
"items": {
|
||||
"$ref": "#/definitions/KeyModifier"
|
||||
}
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"press"
|
||||
]
|
||||
},
|
||||
"result": {
|
||||
"$ref": "#/definitions/StepResultV3"
|
||||
},
|
||||
"state": {
|
||||
"description": "Observation id immediately before this action.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/TargetDescriptorV3"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"result",
|
||||
"state"
|
||||
],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"scroll"
|
||||
]
|
||||
},
|
||||
"result": {
|
||||
"$ref": "#/definitions/StepResultV3"
|
||||
},
|
||||
"state": {
|
||||
"description": "Observation id immediately before this action.",
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
"StepEffect": {
|
||||
"description": "Observed navigation after a step (objective fact only).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"navigated_to"
|
||||
],
|
||||
"properties": {
|
||||
"navigated_to": {
|
||||
"description": "Reference into `pages[]` for the destination page.",
|
||||
"type": "string"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"StopReason": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"user_finish",
|
||||
"cli_stop"
|
||||
]
|
||||
},
|
||||
"TargetDescriptorV2": {
|
||||
"description": "Stable semantic handle for an interacted element (v2).\n\n`name` and `nearby_label` are **untrusted page text**.",
|
||||
"TargetDescriptor": {
|
||||
"description": "Stable semantic handle for an interacted element.\n\n`name` and `nearby_label` are **untrusted page text**.",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"tag"
|
||||
@@ -767,83 +397,8 @@
|
||||
}
|
||||
}
|
||||
},
|
||||
"TargetDescriptorV3": {
|
||||
"description": "Stable semantic handle for an interacted element within a page observation.",
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"ctx": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"name": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"ref": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"role": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"unmatched": {
|
||||
"type": "boolean"
|
||||
}
|
||||
}
|
||||
},
|
||||
"TraceEntry": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"start_url"
|
||||
],
|
||||
"properties": {
|
||||
"start_url": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"TraceStateV3": {
|
||||
"description": "Page observation dictionary entry — referenced by steps via `state` / `result.state`.",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"body",
|
||||
"id",
|
||||
"url"
|
||||
],
|
||||
"properties": {
|
||||
"body": {
|
||||
"description": "Full page observation (front matter + VOM body + annotations).",
|
||||
"type": "string"
|
||||
},
|
||||
"id": {
|
||||
"type": "string"
|
||||
},
|
||||
"title": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"truncated": {
|
||||
"type": "boolean"
|
||||
},
|
||||
"url": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"TraceV2": {
|
||||
"title": "TraceV2",
|
||||
"description": "Persisted user-action trace exported by legacy `tool.record_stop` / `await`.\n\nUnknown extension fields are ignored so older traces remain readable. `states[]` and a numeric `version` are reserved for Trace v3 / `RecordedTrace` classification.",
|
||||
"Trace": {
|
||||
"description": "Persisted user-action trace exported by `tool.record_stop` / `await`.",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"entry",
|
||||
@@ -858,10 +413,11 @@
|
||||
"pages": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/PageRefV2"
|
||||
"$ref": "#/definitions/PageRef"
|
||||
}
|
||||
},
|
||||
"purpose": {
|
||||
"description": "Optional user-provided goal from `--purpose` (metadata only).",
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
@@ -877,76 +433,25 @@
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"states": false,
|
||||
"steps": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/StepV2"
|
||||
}
|
||||
},
|
||||
"version": {
|
||||
"description": "Numeric version selects Trace v3. Legacy v2 envelopes omit this field.",
|
||||
"not": {
|
||||
"type": "integer"
|
||||
"$ref": "#/definitions/Step"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"TraceV3": {
|
||||
"description": "Wire trace returned by `tool.record_stop` / `await`.",
|
||||
"TraceEntry": {
|
||||
"description": "Recording entry point — first URL the flow starts from.",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"entry",
|
||||
"recorded_at",
|
||||
"recorder",
|
||||
"states",
|
||||
"steps",
|
||||
"stopped_by",
|
||||
"version"
|
||||
"start_url"
|
||||
],
|
||||
"properties": {
|
||||
"entry": {
|
||||
"$ref": "#/definitions/TraceEntry"
|
||||
},
|
||||
"purpose": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"recorded_at": {
|
||||
"start_url": {
|
||||
"type": "string"
|
||||
},
|
||||
"recorder": {
|
||||
"$ref": "#/definitions/RecorderInfo"
|
||||
},
|
||||
"started_at": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"states": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/TraceStateV3"
|
||||
}
|
||||
},
|
||||
"steps": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/StepV3"
|
||||
}
|
||||
},
|
||||
"stopped_by": {
|
||||
"$ref": "#/definitions/StopReason"
|
||||
},
|
||||
"version": {
|
||||
"type": "integer",
|
||||
"const": 3
|
||||
}
|
||||
},
|
||||
"additionalProperties": false
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -7,18 +7,10 @@
|
||||
],
|
||||
"properties": {
|
||||
"trace": {
|
||||
"$ref": "#/definitions/RecordedTrace"
|
||||
"$ref": "#/definitions/Trace"
|
||||
}
|
||||
},
|
||||
"definitions": {
|
||||
"FillCommit": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"enter",
|
||||
"suggestion",
|
||||
"blur"
|
||||
]
|
||||
},
|
||||
"KeyModifier": {
|
||||
"description": "Keyboard modifier flags. Multiple flags may be combined; the extension folds them into CDP's bitfield (`alt=1, ctrl=2, meta=4, shift=8`).",
|
||||
"type": "string",
|
||||
@@ -29,19 +21,7 @@
|
||||
"shift"
|
||||
]
|
||||
},
|
||||
"NavigationCause": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"user_typed",
|
||||
"link",
|
||||
"form_submit",
|
||||
"reload",
|
||||
"history",
|
||||
"script",
|
||||
"browser"
|
||||
]
|
||||
},
|
||||
"PageRefV2": {
|
||||
"PageRef": {
|
||||
"description": "Page context dictionary entry — referenced by steps via `page` id.",
|
||||
"type": "object",
|
||||
"required": [
|
||||
@@ -63,42 +43,7 @@
|
||||
}
|
||||
}
|
||||
},
|
||||
"RecordedTrace": {
|
||||
"if": {
|
||||
"required": [
|
||||
"version"
|
||||
],
|
||||
"properties": {
|
||||
"version": {
|
||||
"type": "integer"
|
||||
}
|
||||
}
|
||||
},
|
||||
"then": {
|
||||
"$ref": "#/definitions/TraceV3"
|
||||
},
|
||||
"else": {
|
||||
"$ref": "#/definitions/TraceV2"
|
||||
}
|
||||
},
|
||||
"RecorderInfo": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"bsk",
|
||||
"vom"
|
||||
],
|
||||
"properties": {
|
||||
"bsk": {
|
||||
"type": "string"
|
||||
},
|
||||
"vom": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
}
|
||||
}
|
||||
},
|
||||
"SelectedOptionV2": {
|
||||
"SelectedOption": {
|
||||
"description": "One selected option (`select` op).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
@@ -116,53 +61,11 @@
|
||||
}
|
||||
}
|
||||
},
|
||||
"SelectedOptionV3": {
|
||||
"description": "One selected option (`select` op).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"value"
|
||||
],
|
||||
"properties": {
|
||||
"label": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"value": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"StepEffectV2": {
|
||||
"description": "Observed navigation after a step (objective fact only).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"navigated_to"
|
||||
],
|
||||
"properties": {
|
||||
"navigated_to": {
|
||||
"description": "Reference into `pages[]` for the destination page.",
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"StepResultV3": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"state"
|
||||
],
|
||||
"properties": {
|
||||
"state": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"StepV2": {
|
||||
"description": "One recorded user action — discriminated union by `op` (v2).",
|
||||
"Step": {
|
||||
"description": "One recorded user action — discriminated union by `op`.",
|
||||
"oneOf": [
|
||||
{
|
||||
"description": "Fields shared by every v2 step variant (flattened in JSON).",
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
@@ -174,7 +77,7 @@
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffectV2"
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
@@ -202,7 +105,7 @@
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every v2 step variant (flattened in JSON).",
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
@@ -214,7 +117,7 @@
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffectV2"
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
@@ -237,12 +140,12 @@
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV2"
|
||||
"$ref": "#/definitions/TargetDescriptor"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every v2 step variant (flattened in JSON).",
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
@@ -254,7 +157,7 @@
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffectV2"
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
@@ -277,12 +180,12 @@
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV2"
|
||||
"$ref": "#/definitions/TargetDescriptor"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every v2 step variant (flattened in JSON).",
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
@@ -295,7 +198,7 @@
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffectV2"
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
@@ -324,7 +227,7 @@
|
||||
]
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV2"
|
||||
"$ref": "#/definitions/TargetDescriptor"
|
||||
},
|
||||
"value": {
|
||||
"type": "string"
|
||||
@@ -332,7 +235,7 @@
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every v2 step variant (flattened in JSON).",
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
@@ -345,7 +248,7 @@
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffectV2"
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
@@ -370,16 +273,16 @@
|
||||
"selection": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/SelectedOptionV2"
|
||||
"$ref": "#/definitions/SelectedOption"
|
||||
}
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV2"
|
||||
"$ref": "#/definitions/TargetDescriptor"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every v2 step variant (flattened in JSON).",
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
@@ -391,7 +294,7 @@
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffectV2"
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
@@ -428,7 +331,7 @@
|
||||
"target": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/TargetDescriptorV2"
|
||||
"$ref": "#/definitions/TargetDescriptor"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
@@ -439,294 +342,21 @@
|
||||
}
|
||||
]
|
||||
},
|
||||
"StepV3": {
|
||||
"description": "One recorded user action — discriminated union by `op`.",
|
||||
"oneOf": [
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"cause",
|
||||
"id",
|
||||
"op",
|
||||
"result",
|
||||
"state",
|
||||
"to"
|
||||
],
|
||||
"properties": {
|
||||
"cause": {
|
||||
"$ref": "#/definitions/NavigationCause"
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"navigate"
|
||||
]
|
||||
},
|
||||
"result": {
|
||||
"$ref": "#/definitions/StepResultV3"
|
||||
},
|
||||
"state": {
|
||||
"description": "Observation id immediately before this action.",
|
||||
"type": "string"
|
||||
},
|
||||
"to": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"result",
|
||||
"state",
|
||||
"target"
|
||||
],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"click"
|
||||
]
|
||||
},
|
||||
"result": {
|
||||
"$ref": "#/definitions/StepResultV3"
|
||||
},
|
||||
"state": {
|
||||
"description": "Observation id immediately before this action.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV3"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"result",
|
||||
"state",
|
||||
"target"
|
||||
],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"hover"
|
||||
]
|
||||
},
|
||||
"result": {
|
||||
"$ref": "#/definitions/StepResultV3"
|
||||
},
|
||||
"state": {
|
||||
"description": "Observation id immediately before this action.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV3"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"commit",
|
||||
"id",
|
||||
"op",
|
||||
"result",
|
||||
"state",
|
||||
"target",
|
||||
"value"
|
||||
],
|
||||
"properties": {
|
||||
"commit": {
|
||||
"$ref": "#/definitions/FillCommit"
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"fill"
|
||||
]
|
||||
},
|
||||
"redacted": {
|
||||
"type": "boolean"
|
||||
},
|
||||
"result": {
|
||||
"$ref": "#/definitions/StepResultV3"
|
||||
},
|
||||
"state": {
|
||||
"description": "Observation id immediately before this action.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV3"
|
||||
},
|
||||
"value": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"result",
|
||||
"state",
|
||||
"target"
|
||||
],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"select"
|
||||
]
|
||||
},
|
||||
"result": {
|
||||
"$ref": "#/definitions/StepResultV3"
|
||||
},
|
||||
"selection": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/SelectedOptionV3"
|
||||
}
|
||||
},
|
||||
"state": {
|
||||
"description": "Observation id immediately before this action.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV3"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"key",
|
||||
"op",
|
||||
"result",
|
||||
"state"
|
||||
],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"key": {
|
||||
"type": "string"
|
||||
},
|
||||
"modifiers": {
|
||||
"type": [
|
||||
"array",
|
||||
"null"
|
||||
],
|
||||
"items": {
|
||||
"$ref": "#/definitions/KeyModifier"
|
||||
}
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"press"
|
||||
]
|
||||
},
|
||||
"result": {
|
||||
"$ref": "#/definitions/StepResultV3"
|
||||
},
|
||||
"state": {
|
||||
"description": "Observation id immediately before this action.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/TargetDescriptorV3"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"result",
|
||||
"state"
|
||||
],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"scroll"
|
||||
]
|
||||
},
|
||||
"result": {
|
||||
"$ref": "#/definitions/StepResultV3"
|
||||
},
|
||||
"state": {
|
||||
"description": "Observation id immediately before this action.",
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
"StepEffect": {
|
||||
"description": "Observed navigation after a step (objective fact only).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"navigated_to"
|
||||
],
|
||||
"properties": {
|
||||
"navigated_to": {
|
||||
"description": "Reference into `pages[]` for the destination page.",
|
||||
"type": "string"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"StopReason": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"user_finish",
|
||||
"cli_stop"
|
||||
]
|
||||
},
|
||||
"TargetDescriptorV2": {
|
||||
"description": "Stable semantic handle for an interacted element (v2).\n\n`name` and `nearby_label` are **untrusted page text**.",
|
||||
"TargetDescriptor": {
|
||||
"description": "Stable semantic handle for an interacted element.\n\n`name` and `nearby_label` are **untrusted page text**.",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"tag"
|
||||
@@ -767,83 +397,8 @@
|
||||
}
|
||||
}
|
||||
},
|
||||
"TargetDescriptorV3": {
|
||||
"description": "Stable semantic handle for an interacted element within a page observation.",
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"ctx": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"name": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"ref": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"role": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"unmatched": {
|
||||
"type": "boolean"
|
||||
}
|
||||
}
|
||||
},
|
||||
"TraceEntry": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"start_url"
|
||||
],
|
||||
"properties": {
|
||||
"start_url": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"TraceStateV3": {
|
||||
"description": "Page observation dictionary entry — referenced by steps via `state` / `result.state`.",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"body",
|
||||
"id",
|
||||
"url"
|
||||
],
|
||||
"properties": {
|
||||
"body": {
|
||||
"description": "Full page observation (front matter + VOM body + annotations).",
|
||||
"type": "string"
|
||||
},
|
||||
"id": {
|
||||
"type": "string"
|
||||
},
|
||||
"title": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"truncated": {
|
||||
"type": "boolean"
|
||||
},
|
||||
"url": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"TraceV2": {
|
||||
"title": "TraceV2",
|
||||
"description": "Persisted user-action trace exported by legacy `tool.record_stop` / `await`.\n\nUnknown extension fields are ignored so older traces remain readable. `states[]` and a numeric `version` are reserved for Trace v3 / `RecordedTrace` classification.",
|
||||
"Trace": {
|
||||
"description": "Persisted user-action trace exported by `tool.record_stop` / `await`.",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"entry",
|
||||
@@ -858,10 +413,11 @@
|
||||
"pages": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/PageRefV2"
|
||||
"$ref": "#/definitions/PageRef"
|
||||
}
|
||||
},
|
||||
"purpose": {
|
||||
"description": "Optional user-provided goal from `--purpose` (metadata only).",
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
@@ -877,76 +433,25 @@
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"states": false,
|
||||
"steps": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/StepV2"
|
||||
}
|
||||
},
|
||||
"version": {
|
||||
"description": "Numeric version selects Trace v3. Legacy v2 envelopes omit this field.",
|
||||
"not": {
|
||||
"type": "integer"
|
||||
"$ref": "#/definitions/Step"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"TraceV3": {
|
||||
"description": "Wire trace returned by `tool.record_stop` / `await`.",
|
||||
"TraceEntry": {
|
||||
"description": "Recording entry point — first URL the flow starts from.",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"entry",
|
||||
"recorded_at",
|
||||
"recorder",
|
||||
"states",
|
||||
"steps",
|
||||
"stopped_by",
|
||||
"version"
|
||||
"start_url"
|
||||
],
|
||||
"properties": {
|
||||
"entry": {
|
||||
"$ref": "#/definitions/TraceEntry"
|
||||
},
|
||||
"purpose": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"recorded_at": {
|
||||
"start_url": {
|
||||
"type": "string"
|
||||
},
|
||||
"recorder": {
|
||||
"$ref": "#/definitions/RecorderInfo"
|
||||
},
|
||||
"started_at": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"states": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/TraceStateV3"
|
||||
}
|
||||
},
|
||||
"steps": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/StepV3"
|
||||
}
|
||||
},
|
||||
"stopped_by": {
|
||||
"$ref": "#/definitions/StopReason"
|
||||
},
|
||||
"version": {
|
||||
"type": "integer",
|
||||
"const": 3
|
||||
}
|
||||
},
|
||||
"additionalProperties": false
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,70 @@
|
||||
{
|
||||
"$schema": "http://json-schema.org/draft-07/schema#",
|
||||
"title": "UploadParams",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"files",
|
||||
"session_id"
|
||||
],
|
||||
"properties": {
|
||||
"files": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/UploadFile"
|
||||
}
|
||||
},
|
||||
"ref": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"selector": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"session_id": {
|
||||
"type": "string"
|
||||
},
|
||||
"tab_id": {
|
||||
"type": [
|
||||
"integer",
|
||||
"null"
|
||||
],
|
||||
"format": "int64"
|
||||
},
|
||||
"timeout_ms": {
|
||||
"type": [
|
||||
"integer",
|
||||
"null"
|
||||
],
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
}
|
||||
},
|
||||
"definitions": {
|
||||
"UploadFile": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"name",
|
||||
"transfer_id"
|
||||
],
|
||||
"properties": {
|
||||
"name": {
|
||||
"type": "string"
|
||||
},
|
||||
"staged_path": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"transfer_id": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,33 @@
|
||||
{
|
||||
"$schema": "http://json-schema.org/draft-07/schema#",
|
||||
"title": "UploadResult",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"file_names",
|
||||
"tab_id"
|
||||
],
|
||||
"properties": {
|
||||
"file_names": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"tab_id": {
|
||||
"type": "integer",
|
||||
"format": "int64"
|
||||
},
|
||||
"used_ref": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"used_selector": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,31 +1,49 @@
|
||||
{
|
||||
"$schema": "http://json-schema.org/draft-07/schema#",
|
||||
"title": "RecordedTrace",
|
||||
"if": {
|
||||
"required": [
|
||||
"version"
|
||||
],
|
||||
"properties": {
|
||||
"version": {
|
||||
"type": "integer"
|
||||
"title": "Trace",
|
||||
"description": "Persisted user-action trace exported by `tool.record_stop` / `await`.",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"entry",
|
||||
"pages",
|
||||
"recorded_at",
|
||||
"steps"
|
||||
],
|
||||
"properties": {
|
||||
"entry": {
|
||||
"$ref": "#/definitions/TraceEntry"
|
||||
},
|
||||
"pages": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/PageRef"
|
||||
}
|
||||
},
|
||||
"purpose": {
|
||||
"description": "Optional user-provided goal from `--purpose` (metadata only).",
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"recorded_at": {
|
||||
"description": "RFC 3339 timestamp when recording stopped.",
|
||||
"type": "string"
|
||||
},
|
||||
"started_at": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"steps": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/Step"
|
||||
}
|
||||
}
|
||||
},
|
||||
"then": {
|
||||
"$ref": "#/definitions/TraceV3"
|
||||
},
|
||||
"else": {
|
||||
"$ref": "#/definitions/TraceV2"
|
||||
},
|
||||
"definitions": {
|
||||
"FillCommit": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"enter",
|
||||
"suggestion",
|
||||
"blur"
|
||||
]
|
||||
},
|
||||
"KeyModifier": {
|
||||
"description": "Keyboard modifier flags. Multiple flags may be combined; the extension folds them into CDP's bitfield (`alt=1, ctrl=2, meta=4, shift=8`).",
|
||||
"type": "string",
|
||||
@@ -36,19 +54,7 @@
|
||||
"shift"
|
||||
]
|
||||
},
|
||||
"NavigationCause": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"user_typed",
|
||||
"link",
|
||||
"form_submit",
|
||||
"reload",
|
||||
"history",
|
||||
"script",
|
||||
"browser"
|
||||
]
|
||||
},
|
||||
"PageRefV2": {
|
||||
"PageRef": {
|
||||
"description": "Page context dictionary entry — referenced by steps via `page` id.",
|
||||
"type": "object",
|
||||
"required": [
|
||||
@@ -70,24 +76,7 @@
|
||||
}
|
||||
}
|
||||
},
|
||||
"RecorderInfo": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"bsk",
|
||||
"vom"
|
||||
],
|
||||
"properties": {
|
||||
"bsk": {
|
||||
"type": "string"
|
||||
},
|
||||
"vom": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
}
|
||||
}
|
||||
},
|
||||
"SelectedOptionV2": {
|
||||
"SelectedOption": {
|
||||
"description": "One selected option (`select` op).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
@@ -105,53 +94,11 @@
|
||||
}
|
||||
}
|
||||
},
|
||||
"SelectedOptionV3": {
|
||||
"description": "One selected option (`select` op).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"value"
|
||||
],
|
||||
"properties": {
|
||||
"label": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"value": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"StepEffectV2": {
|
||||
"description": "Observed navigation after a step (objective fact only).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"navigated_to"
|
||||
],
|
||||
"properties": {
|
||||
"navigated_to": {
|
||||
"description": "Reference into `pages[]` for the destination page.",
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"StepResultV3": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"state"
|
||||
],
|
||||
"properties": {
|
||||
"state": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"StepV2": {
|
||||
"description": "One recorded user action — discriminated union by `op` (v2).",
|
||||
"Step": {
|
||||
"description": "One recorded user action — discriminated union by `op`.",
|
||||
"oneOf": [
|
||||
{
|
||||
"description": "Fields shared by every v2 step variant (flattened in JSON).",
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
@@ -163,7 +110,7 @@
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffectV2"
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
@@ -191,7 +138,7 @@
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every v2 step variant (flattened in JSON).",
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
@@ -203,7 +150,7 @@
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffectV2"
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
@@ -226,12 +173,12 @@
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV2"
|
||||
"$ref": "#/definitions/TargetDescriptor"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every v2 step variant (flattened in JSON).",
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
@@ -243,7 +190,7 @@
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffectV2"
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
@@ -266,12 +213,12 @@
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV2"
|
||||
"$ref": "#/definitions/TargetDescriptor"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every v2 step variant (flattened in JSON).",
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
@@ -284,7 +231,7 @@
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffectV2"
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
@@ -313,7 +260,7 @@
|
||||
]
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV2"
|
||||
"$ref": "#/definitions/TargetDescriptor"
|
||||
},
|
||||
"value": {
|
||||
"type": "string"
|
||||
@@ -321,7 +268,7 @@
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every v2 step variant (flattened in JSON).",
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
@@ -334,7 +281,7 @@
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffectV2"
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
@@ -359,16 +306,16 @@
|
||||
"selection": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/SelectedOptionV2"
|
||||
"$ref": "#/definitions/SelectedOption"
|
||||
}
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV2"
|
||||
"$ref": "#/definitions/TargetDescriptor"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every v2 step variant (flattened in JSON).",
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
@@ -380,7 +327,7 @@
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffectV2"
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
@@ -417,7 +364,7 @@
|
||||
"target": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/TargetDescriptorV2"
|
||||
"$ref": "#/definitions/TargetDescriptor"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
@@ -428,294 +375,21 @@
|
||||
}
|
||||
]
|
||||
},
|
||||
"StepV3": {
|
||||
"description": "One recorded user action — discriminated union by `op`.",
|
||||
"oneOf": [
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"cause",
|
||||
"id",
|
||||
"op",
|
||||
"result",
|
||||
"state",
|
||||
"to"
|
||||
],
|
||||
"properties": {
|
||||
"cause": {
|
||||
"$ref": "#/definitions/NavigationCause"
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"navigate"
|
||||
]
|
||||
},
|
||||
"result": {
|
||||
"$ref": "#/definitions/StepResultV3"
|
||||
},
|
||||
"state": {
|
||||
"description": "Observation id immediately before this action.",
|
||||
"type": "string"
|
||||
},
|
||||
"to": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"result",
|
||||
"state",
|
||||
"target"
|
||||
],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"click"
|
||||
]
|
||||
},
|
||||
"result": {
|
||||
"$ref": "#/definitions/StepResultV3"
|
||||
},
|
||||
"state": {
|
||||
"description": "Observation id immediately before this action.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV3"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"result",
|
||||
"state",
|
||||
"target"
|
||||
],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"hover"
|
||||
]
|
||||
},
|
||||
"result": {
|
||||
"$ref": "#/definitions/StepResultV3"
|
||||
},
|
||||
"state": {
|
||||
"description": "Observation id immediately before this action.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV3"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"commit",
|
||||
"id",
|
||||
"op",
|
||||
"result",
|
||||
"state",
|
||||
"target",
|
||||
"value"
|
||||
],
|
||||
"properties": {
|
||||
"commit": {
|
||||
"$ref": "#/definitions/FillCommit"
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"fill"
|
||||
]
|
||||
},
|
||||
"redacted": {
|
||||
"type": "boolean"
|
||||
},
|
||||
"result": {
|
||||
"$ref": "#/definitions/StepResultV3"
|
||||
},
|
||||
"state": {
|
||||
"description": "Observation id immediately before this action.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV3"
|
||||
},
|
||||
"value": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"result",
|
||||
"state",
|
||||
"target"
|
||||
],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"select"
|
||||
]
|
||||
},
|
||||
"result": {
|
||||
"$ref": "#/definitions/StepResultV3"
|
||||
},
|
||||
"selection": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/SelectedOptionV3"
|
||||
}
|
||||
},
|
||||
"state": {
|
||||
"description": "Observation id immediately before this action.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV3"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"key",
|
||||
"op",
|
||||
"result",
|
||||
"state"
|
||||
],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"key": {
|
||||
"type": "string"
|
||||
},
|
||||
"modifiers": {
|
||||
"type": [
|
||||
"array",
|
||||
"null"
|
||||
],
|
||||
"items": {
|
||||
"$ref": "#/definitions/KeyModifier"
|
||||
}
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"press"
|
||||
]
|
||||
},
|
||||
"result": {
|
||||
"$ref": "#/definitions/StepResultV3"
|
||||
},
|
||||
"state": {
|
||||
"description": "Observation id immediately before this action.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/TargetDescriptorV3"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"result",
|
||||
"state"
|
||||
],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"scroll"
|
||||
]
|
||||
},
|
||||
"result": {
|
||||
"$ref": "#/definitions/StepResultV3"
|
||||
},
|
||||
"state": {
|
||||
"description": "Observation id immediately before this action.",
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
"StepEffect": {
|
||||
"description": "Observed navigation after a step (objective fact only).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"navigated_to"
|
||||
],
|
||||
"properties": {
|
||||
"navigated_to": {
|
||||
"description": "Reference into `pages[]` for the destination page.",
|
||||
"type": "string"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"StopReason": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"user_finish",
|
||||
"cli_stop"
|
||||
]
|
||||
},
|
||||
"TargetDescriptorV2": {
|
||||
"description": "Stable semantic handle for an interacted element (v2).\n\n`name` and `nearby_label` are **untrusted page text**.",
|
||||
"TargetDescriptor": {
|
||||
"description": "Stable semantic handle for an interacted element.\n\n`name` and `nearby_label` are **untrusted page text**.",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"tag"
|
||||
@@ -756,40 +430,8 @@
|
||||
}
|
||||
}
|
||||
},
|
||||
"TargetDescriptorV3": {
|
||||
"description": "Stable semantic handle for an interacted element within a page observation.",
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"ctx": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"name": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"ref": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"role": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"unmatched": {
|
||||
"type": "boolean"
|
||||
}
|
||||
}
|
||||
},
|
||||
"TraceEntry": {
|
||||
"description": "Recording entry point — first URL the flow starts from.",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"start_url"
|
||||
@@ -799,143 +441,6 @@
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"TraceStateV3": {
|
||||
"description": "Page observation dictionary entry — referenced by steps via `state` / `result.state`.",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"body",
|
||||
"id",
|
||||
"url"
|
||||
],
|
||||
"properties": {
|
||||
"body": {
|
||||
"description": "Full page observation (front matter + VOM body + annotations).",
|
||||
"type": "string"
|
||||
},
|
||||
"id": {
|
||||
"type": "string"
|
||||
},
|
||||
"title": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"truncated": {
|
||||
"type": "boolean"
|
||||
},
|
||||
"url": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"TraceV2": {
|
||||
"title": "TraceV2",
|
||||
"description": "Persisted user-action trace exported by legacy `tool.record_stop` / `await`.\n\nUnknown extension fields are ignored so older traces remain readable. `states[]` and a numeric `version` are reserved for Trace v3 / `RecordedTrace` classification.",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"entry",
|
||||
"pages",
|
||||
"recorded_at",
|
||||
"steps"
|
||||
],
|
||||
"properties": {
|
||||
"entry": {
|
||||
"$ref": "#/definitions/TraceEntry"
|
||||
},
|
||||
"pages": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/PageRefV2"
|
||||
}
|
||||
},
|
||||
"purpose": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"recorded_at": {
|
||||
"description": "RFC 3339 timestamp when recording stopped.",
|
||||
"type": "string"
|
||||
},
|
||||
"started_at": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"states": false,
|
||||
"steps": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/StepV2"
|
||||
}
|
||||
},
|
||||
"version": {
|
||||
"description": "Numeric version selects Trace v3. Legacy v2 envelopes omit this field.",
|
||||
"not": {
|
||||
"type": "integer"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"TraceV3": {
|
||||
"description": "Wire trace returned by `tool.record_stop` / `await`.",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"entry",
|
||||
"recorded_at",
|
||||
"recorder",
|
||||
"states",
|
||||
"steps",
|
||||
"stopped_by",
|
||||
"version"
|
||||
],
|
||||
"properties": {
|
||||
"entry": {
|
||||
"$ref": "#/definitions/TraceEntry"
|
||||
},
|
||||
"purpose": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"recorded_at": {
|
||||
"type": "string"
|
||||
},
|
||||
"recorder": {
|
||||
"$ref": "#/definitions/RecorderInfo"
|
||||
},
|
||||
"started_at": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"states": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/TraceStateV3"
|
||||
}
|
||||
},
|
||||
"steps": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/StepV3"
|
||||
}
|
||||
},
|
||||
"stopped_by": {
|
||||
"$ref": "#/definitions/StopReason"
|
||||
},
|
||||
"version": {
|
||||
"type": "integer",
|
||||
"const": 3
|
||||
}
|
||||
},
|
||||
"additionalProperties": false
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,32 +1,286 @@
|
||||
{
|
||||
"$schema": "http://json-schema.org/draft-07/schema#",
|
||||
"title": "RecordedStep",
|
||||
"not": {
|
||||
"required": [
|
||||
"page",
|
||||
"state"
|
||||
]
|
||||
},
|
||||
"if": {
|
||||
"required": [
|
||||
"state"
|
||||
]
|
||||
},
|
||||
"then": {
|
||||
"$ref": "#/definitions/StepV3"
|
||||
},
|
||||
"else": {
|
||||
"$ref": "#/definitions/StepV2"
|
||||
},
|
||||
"definitions": {
|
||||
"FillCommit": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"enter",
|
||||
"suggestion",
|
||||
"blur"
|
||||
]
|
||||
"title": "Step",
|
||||
"description": "One recorded user action — discriminated union by `op`.",
|
||||
"oneOf": [
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"page",
|
||||
"to"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"navigate"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"to": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"page",
|
||||
"target"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"click"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptor"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"page",
|
||||
"target"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"hover"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptor"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"page",
|
||||
"target",
|
||||
"value"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"fill"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"redacted": {
|
||||
"type": [
|
||||
"boolean",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptor"
|
||||
},
|
||||
"value": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"page",
|
||||
"selection",
|
||||
"target"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"select"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"selection": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/SelectedOption"
|
||||
}
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptor"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"key",
|
||||
"op",
|
||||
"page"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"key": {
|
||||
"type": "string"
|
||||
},
|
||||
"modifiers": {
|
||||
"type": [
|
||||
"array",
|
||||
"null"
|
||||
],
|
||||
"items": {
|
||||
"$ref": "#/definitions/KeyModifier"
|
||||
}
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"press"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/TargetDescriptor"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"definitions": {
|
||||
"KeyModifier": {
|
||||
"description": "Keyboard modifier flags. Multiple flags may be combined; the extension folds them into CDP's bitfield (`alt=1, ctrl=2, meta=4, shift=8`).",
|
||||
"type": "string",
|
||||
@@ -37,19 +291,7 @@
|
||||
"shift"
|
||||
]
|
||||
},
|
||||
"NavigationCause": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"user_typed",
|
||||
"link",
|
||||
"form_submit",
|
||||
"reload",
|
||||
"history",
|
||||
"script",
|
||||
"browser"
|
||||
]
|
||||
},
|
||||
"SelectedOptionV2": {
|
||||
"SelectedOption": {
|
||||
"description": "One selected option (`select` op).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
@@ -67,25 +309,7 @@
|
||||
}
|
||||
}
|
||||
},
|
||||
"SelectedOptionV3": {
|
||||
"description": "One selected option (`select` op).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"value"
|
||||
],
|
||||
"properties": {
|
||||
"label": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"value": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"StepEffectV2": {
|
||||
"StepEffect": {
|
||||
"description": "Observed navigation after a step (objective fact only).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
@@ -98,579 +322,8 @@
|
||||
}
|
||||
}
|
||||
},
|
||||
"StepResultV3": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"state"
|
||||
],
|
||||
"properties": {
|
||||
"state": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"StepV2": {
|
||||
"description": "One recorded user action — discriminated union by `op` (v2).",
|
||||
"oneOf": [
|
||||
{
|
||||
"description": "Fields shared by every v2 step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"page",
|
||||
"to"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffectV2"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"navigate"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"to": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every v2 step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"page",
|
||||
"target"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffectV2"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"click"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV2"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every v2 step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"page",
|
||||
"target"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffectV2"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"hover"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV2"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every v2 step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"page",
|
||||
"target",
|
||||
"value"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffectV2"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"fill"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"redacted": {
|
||||
"type": [
|
||||
"boolean",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV2"
|
||||
},
|
||||
"value": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every v2 step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"page",
|
||||
"selection",
|
||||
"target"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffectV2"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"select"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"selection": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/SelectedOptionV2"
|
||||
}
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV2"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every v2 step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"key",
|
||||
"op",
|
||||
"page"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffectV2"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"key": {
|
||||
"type": "string"
|
||||
},
|
||||
"modifiers": {
|
||||
"type": [
|
||||
"array",
|
||||
"null"
|
||||
],
|
||||
"items": {
|
||||
"$ref": "#/definitions/KeyModifier"
|
||||
}
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"press"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/TargetDescriptorV2"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
},
|
||||
"StepV3": {
|
||||
"description": "One recorded user action — discriminated union by `op`.",
|
||||
"oneOf": [
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"cause",
|
||||
"id",
|
||||
"op",
|
||||
"result",
|
||||
"state",
|
||||
"to"
|
||||
],
|
||||
"properties": {
|
||||
"cause": {
|
||||
"$ref": "#/definitions/NavigationCause"
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"navigate"
|
||||
]
|
||||
},
|
||||
"result": {
|
||||
"$ref": "#/definitions/StepResultV3"
|
||||
},
|
||||
"state": {
|
||||
"description": "Observation id immediately before this action.",
|
||||
"type": "string"
|
||||
},
|
||||
"to": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"result",
|
||||
"state",
|
||||
"target"
|
||||
],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"click"
|
||||
]
|
||||
},
|
||||
"result": {
|
||||
"$ref": "#/definitions/StepResultV3"
|
||||
},
|
||||
"state": {
|
||||
"description": "Observation id immediately before this action.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV3"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"result",
|
||||
"state",
|
||||
"target"
|
||||
],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"hover"
|
||||
]
|
||||
},
|
||||
"result": {
|
||||
"$ref": "#/definitions/StepResultV3"
|
||||
},
|
||||
"state": {
|
||||
"description": "Observation id immediately before this action.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV3"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"commit",
|
||||
"id",
|
||||
"op",
|
||||
"result",
|
||||
"state",
|
||||
"target",
|
||||
"value"
|
||||
],
|
||||
"properties": {
|
||||
"commit": {
|
||||
"$ref": "#/definitions/FillCommit"
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"fill"
|
||||
]
|
||||
},
|
||||
"redacted": {
|
||||
"type": "boolean"
|
||||
},
|
||||
"result": {
|
||||
"$ref": "#/definitions/StepResultV3"
|
||||
},
|
||||
"state": {
|
||||
"description": "Observation id immediately before this action.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV3"
|
||||
},
|
||||
"value": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"result",
|
||||
"state",
|
||||
"target"
|
||||
],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"select"
|
||||
]
|
||||
},
|
||||
"result": {
|
||||
"$ref": "#/definitions/StepResultV3"
|
||||
},
|
||||
"selection": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/SelectedOptionV3"
|
||||
}
|
||||
},
|
||||
"state": {
|
||||
"description": "Observation id immediately before this action.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV3"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"key",
|
||||
"op",
|
||||
"result",
|
||||
"state"
|
||||
],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"key": {
|
||||
"type": "string"
|
||||
},
|
||||
"modifiers": {
|
||||
"type": [
|
||||
"array",
|
||||
"null"
|
||||
],
|
||||
"items": {
|
||||
"$ref": "#/definitions/KeyModifier"
|
||||
}
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"press"
|
||||
]
|
||||
},
|
||||
"result": {
|
||||
"$ref": "#/definitions/StepResultV3"
|
||||
},
|
||||
"state": {
|
||||
"description": "Observation id immediately before this action.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/TargetDescriptorV3"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"result",
|
||||
"state"
|
||||
],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"scroll"
|
||||
]
|
||||
},
|
||||
"result": {
|
||||
"$ref": "#/definitions/StepResultV3"
|
||||
},
|
||||
"state": {
|
||||
"description": "Observation id immediately before this action.",
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
},
|
||||
"TargetDescriptorV2": {
|
||||
"description": "Stable semantic handle for an interacted element (v2).\n\n`name` and `nearby_label` are **untrusted page text**.",
|
||||
"TargetDescriptor": {
|
||||
"description": "Stable semantic handle for an interacted element.\n\n`name` and `nearby_label` are **untrusted page text**.",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"tag"
|
||||
@@ -710,39 +363,6 @@
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"TargetDescriptorV3": {
|
||||
"description": "Stable semantic handle for an interacted element within a page observation.",
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"ctx": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"name": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"ref": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"role": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"unmatched": {
|
||||
"type": "boolean"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,368 +0,0 @@
|
||||
{
|
||||
"$schema": "http://json-schema.org/draft-07/schema#",
|
||||
"title": "StepV2",
|
||||
"description": "One recorded user action — discriminated union by `op` (v2).",
|
||||
"oneOf": [
|
||||
{
|
||||
"description": "Fields shared by every v2 step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"page",
|
||||
"to"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffectV2"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"navigate"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"to": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every v2 step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"page",
|
||||
"target"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffectV2"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"click"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV2"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every v2 step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"page",
|
||||
"target"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffectV2"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"hover"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV2"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every v2 step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"page",
|
||||
"target",
|
||||
"value"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffectV2"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"fill"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"redacted": {
|
||||
"type": [
|
||||
"boolean",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV2"
|
||||
},
|
||||
"value": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every v2 step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"page",
|
||||
"selection",
|
||||
"target"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffectV2"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"select"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"selection": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/SelectedOptionV2"
|
||||
}
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV2"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every v2 step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"key",
|
||||
"op",
|
||||
"page"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffectV2"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"key": {
|
||||
"type": "string"
|
||||
},
|
||||
"modifiers": {
|
||||
"type": [
|
||||
"array",
|
||||
"null"
|
||||
],
|
||||
"items": {
|
||||
"$ref": "#/definitions/KeyModifier"
|
||||
}
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"press"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/TargetDescriptorV2"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"definitions": {
|
||||
"KeyModifier": {
|
||||
"description": "Keyboard modifier flags. Multiple flags may be combined; the extension folds them into CDP's bitfield (`alt=1, ctrl=2, meta=4, shift=8`).",
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"alt",
|
||||
"ctrl",
|
||||
"meta",
|
||||
"shift"
|
||||
]
|
||||
},
|
||||
"SelectedOptionV2": {
|
||||
"description": "One selected option (`select` op).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"value"
|
||||
],
|
||||
"properties": {
|
||||
"label": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"value": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"StepEffectV2": {
|
||||
"description": "Observed navigation after a step (objective fact only).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"navigated_to"
|
||||
],
|
||||
"properties": {
|
||||
"navigated_to": {
|
||||
"description": "Reference into `pages[]` for the destination page.",
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"TargetDescriptorV2": {
|
||||
"description": "Stable semantic handle for an interacted element (v2).\n\n`name` and `nearby_label` are **untrusted page text**.",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"tag"
|
||||
],
|
||||
"properties": {
|
||||
"name": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"name_attr": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"nearby_label": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"placeholder": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"role": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"tag": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,375 +0,0 @@
|
||||
{
|
||||
"$schema": "http://json-schema.org/draft-07/schema#",
|
||||
"title": "StepV3",
|
||||
"description": "One recorded user action — discriminated union by `op`.",
|
||||
"oneOf": [
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"cause",
|
||||
"id",
|
||||
"op",
|
||||
"result",
|
||||
"state",
|
||||
"to"
|
||||
],
|
||||
"properties": {
|
||||
"cause": {
|
||||
"$ref": "#/definitions/NavigationCause"
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"navigate"
|
||||
]
|
||||
},
|
||||
"result": {
|
||||
"$ref": "#/definitions/StepResultV3"
|
||||
},
|
||||
"state": {
|
||||
"description": "Observation id immediately before this action.",
|
||||
"type": "string"
|
||||
},
|
||||
"to": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"result",
|
||||
"state",
|
||||
"target"
|
||||
],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"click"
|
||||
]
|
||||
},
|
||||
"result": {
|
||||
"$ref": "#/definitions/StepResultV3"
|
||||
},
|
||||
"state": {
|
||||
"description": "Observation id immediately before this action.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV3"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"result",
|
||||
"state",
|
||||
"target"
|
||||
],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"hover"
|
||||
]
|
||||
},
|
||||
"result": {
|
||||
"$ref": "#/definitions/StepResultV3"
|
||||
},
|
||||
"state": {
|
||||
"description": "Observation id immediately before this action.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV3"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"commit",
|
||||
"id",
|
||||
"op",
|
||||
"result",
|
||||
"state",
|
||||
"target",
|
||||
"value"
|
||||
],
|
||||
"properties": {
|
||||
"commit": {
|
||||
"$ref": "#/definitions/FillCommit"
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"fill"
|
||||
]
|
||||
},
|
||||
"redacted": {
|
||||
"type": "boolean"
|
||||
},
|
||||
"result": {
|
||||
"$ref": "#/definitions/StepResultV3"
|
||||
},
|
||||
"state": {
|
||||
"description": "Observation id immediately before this action.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV3"
|
||||
},
|
||||
"value": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"result",
|
||||
"state",
|
||||
"target"
|
||||
],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"select"
|
||||
]
|
||||
},
|
||||
"result": {
|
||||
"$ref": "#/definitions/StepResultV3"
|
||||
},
|
||||
"selection": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/SelectedOptionV3"
|
||||
}
|
||||
},
|
||||
"state": {
|
||||
"description": "Observation id immediately before this action.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV3"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"key",
|
||||
"op",
|
||||
"result",
|
||||
"state"
|
||||
],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"key": {
|
||||
"type": "string"
|
||||
},
|
||||
"modifiers": {
|
||||
"type": [
|
||||
"array",
|
||||
"null"
|
||||
],
|
||||
"items": {
|
||||
"$ref": "#/definitions/KeyModifier"
|
||||
}
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"press"
|
||||
]
|
||||
},
|
||||
"result": {
|
||||
"$ref": "#/definitions/StepResultV3"
|
||||
},
|
||||
"state": {
|
||||
"description": "Observation id immediately before this action.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/TargetDescriptorV3"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"result",
|
||||
"state"
|
||||
],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"scroll"
|
||||
]
|
||||
},
|
||||
"result": {
|
||||
"$ref": "#/definitions/StepResultV3"
|
||||
},
|
||||
"state": {
|
||||
"description": "Observation id immediately before this action.",
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"definitions": {
|
||||
"FillCommit": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"enter",
|
||||
"suggestion",
|
||||
"blur"
|
||||
]
|
||||
},
|
||||
"KeyModifier": {
|
||||
"description": "Keyboard modifier flags. Multiple flags may be combined; the extension folds them into CDP's bitfield (`alt=1, ctrl=2, meta=4, shift=8`).",
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"alt",
|
||||
"ctrl",
|
||||
"meta",
|
||||
"shift"
|
||||
]
|
||||
},
|
||||
"NavigationCause": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"user_typed",
|
||||
"link",
|
||||
"form_submit",
|
||||
"reload",
|
||||
"history",
|
||||
"script",
|
||||
"browser"
|
||||
]
|
||||
},
|
||||
"SelectedOptionV3": {
|
||||
"description": "One selected option (`select` op).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"value"
|
||||
],
|
||||
"properties": {
|
||||
"label": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"value": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"StepResultV3": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"state"
|
||||
],
|
||||
"properties": {
|
||||
"state": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"TargetDescriptorV3": {
|
||||
"description": "Stable semantic handle for an interacted element within a page observation.",
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"ctx": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"name": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"ref": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"role": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"unmatched": {
|
||||
"type": "boolean"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,451 +0,0 @@
|
||||
{
|
||||
"$schema": "http://json-schema.org/draft-07/schema#",
|
||||
"title": "TraceV2",
|
||||
"description": "Persisted user-action trace exported by legacy `tool.record_stop` / `await`.\n\nUnknown extension fields are ignored so older traces remain readable. `states[]` and a numeric `version` are reserved for Trace v3 / `RecordedTrace` classification.",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"entry",
|
||||
"pages",
|
||||
"recorded_at",
|
||||
"steps"
|
||||
],
|
||||
"properties": {
|
||||
"entry": {
|
||||
"$ref": "#/definitions/TraceEntry"
|
||||
},
|
||||
"pages": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/PageRefV2"
|
||||
}
|
||||
},
|
||||
"purpose": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"recorded_at": {
|
||||
"description": "RFC 3339 timestamp when recording stopped.",
|
||||
"type": "string"
|
||||
},
|
||||
"started_at": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"states": false,
|
||||
"steps": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/StepV2"
|
||||
}
|
||||
},
|
||||
"version": {
|
||||
"description": "Numeric version selects Trace v3. Legacy v2 envelopes omit this field.",
|
||||
"not": {
|
||||
"type": "integer"
|
||||
}
|
||||
}
|
||||
},
|
||||
"definitions": {
|
||||
"KeyModifier": {
|
||||
"description": "Keyboard modifier flags. Multiple flags may be combined; the extension folds them into CDP's bitfield (`alt=1, ctrl=2, meta=4, shift=8`).",
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"alt",
|
||||
"ctrl",
|
||||
"meta",
|
||||
"shift"
|
||||
]
|
||||
},
|
||||
"PageRefV2": {
|
||||
"description": "Page context dictionary entry — referenced by steps via `page` id.",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"url"
|
||||
],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "string"
|
||||
},
|
||||
"title": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"url": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"SelectedOptionV2": {
|
||||
"description": "One selected option (`select` op).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"value"
|
||||
],
|
||||
"properties": {
|
||||
"label": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"value": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"StepEffectV2": {
|
||||
"description": "Observed navigation after a step (objective fact only).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"navigated_to"
|
||||
],
|
||||
"properties": {
|
||||
"navigated_to": {
|
||||
"description": "Reference into `pages[]` for the destination page.",
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"StepV2": {
|
||||
"description": "One recorded user action — discriminated union by `op` (v2).",
|
||||
"oneOf": [
|
||||
{
|
||||
"description": "Fields shared by every v2 step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"page",
|
||||
"to"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffectV2"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"navigate"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"to": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every v2 step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"page",
|
||||
"target"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffectV2"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"click"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV2"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every v2 step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"page",
|
||||
"target"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffectV2"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"hover"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV2"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every v2 step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"page",
|
||||
"target",
|
||||
"value"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffectV2"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"fill"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"redacted": {
|
||||
"type": [
|
||||
"boolean",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV2"
|
||||
},
|
||||
"value": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every v2 step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"page",
|
||||
"selection",
|
||||
"target"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffectV2"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"select"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"selection": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/SelectedOptionV2"
|
||||
}
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV2"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every v2 step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"key",
|
||||
"op",
|
||||
"page"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffectV2"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"key": {
|
||||
"type": "string"
|
||||
},
|
||||
"modifiers": {
|
||||
"type": [
|
||||
"array",
|
||||
"null"
|
||||
],
|
||||
"items": {
|
||||
"$ref": "#/definitions/KeyModifier"
|
||||
}
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"press"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/TargetDescriptorV2"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
},
|
||||
"TargetDescriptorV2": {
|
||||
"description": "Stable semantic handle for an interacted element (v2).\n\n`name` and `nearby_label` are **untrusted page text**.",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"tag"
|
||||
],
|
||||
"properties": {
|
||||
"name": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"name_attr": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"nearby_label": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"placeholder": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"role": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"tag": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"TraceEntry": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"start_url"
|
||||
],
|
||||
"properties": {
|
||||
"start_url": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,496 +0,0 @@
|
||||
{
|
||||
"$schema": "http://json-schema.org/draft-07/schema#",
|
||||
"title": "TraceV3",
|
||||
"description": "Wire trace returned by `tool.record_stop` / `await`.",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"entry",
|
||||
"recorded_at",
|
||||
"recorder",
|
||||
"states",
|
||||
"steps",
|
||||
"stopped_by",
|
||||
"version"
|
||||
],
|
||||
"properties": {
|
||||
"entry": {
|
||||
"$ref": "#/definitions/TraceEntry"
|
||||
},
|
||||
"purpose": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"recorded_at": {
|
||||
"type": "string"
|
||||
},
|
||||
"recorder": {
|
||||
"$ref": "#/definitions/RecorderInfo"
|
||||
},
|
||||
"started_at": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"states": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/TraceStateV3"
|
||||
}
|
||||
},
|
||||
"steps": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/StepV3"
|
||||
}
|
||||
},
|
||||
"stopped_by": {
|
||||
"$ref": "#/definitions/StopReason"
|
||||
},
|
||||
"version": {
|
||||
"type": "integer",
|
||||
"const": 3
|
||||
}
|
||||
},
|
||||
"additionalProperties": false,
|
||||
"definitions": {
|
||||
"FillCommit": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"enter",
|
||||
"suggestion",
|
||||
"blur"
|
||||
]
|
||||
},
|
||||
"KeyModifier": {
|
||||
"description": "Keyboard modifier flags. Multiple flags may be combined; the extension folds them into CDP's bitfield (`alt=1, ctrl=2, meta=4, shift=8`).",
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"alt",
|
||||
"ctrl",
|
||||
"meta",
|
||||
"shift"
|
||||
]
|
||||
},
|
||||
"NavigationCause": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"user_typed",
|
||||
"link",
|
||||
"form_submit",
|
||||
"reload",
|
||||
"history",
|
||||
"script",
|
||||
"browser"
|
||||
]
|
||||
},
|
||||
"RecorderInfo": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"bsk",
|
||||
"vom"
|
||||
],
|
||||
"properties": {
|
||||
"bsk": {
|
||||
"type": "string"
|
||||
},
|
||||
"vom": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
}
|
||||
}
|
||||
},
|
||||
"SelectedOptionV3": {
|
||||
"description": "One selected option (`select` op).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"value"
|
||||
],
|
||||
"properties": {
|
||||
"label": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"value": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"StepResultV3": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"state"
|
||||
],
|
||||
"properties": {
|
||||
"state": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"StepV3": {
|
||||
"description": "One recorded user action — discriminated union by `op`.",
|
||||
"oneOf": [
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"cause",
|
||||
"id",
|
||||
"op",
|
||||
"result",
|
||||
"state",
|
||||
"to"
|
||||
],
|
||||
"properties": {
|
||||
"cause": {
|
||||
"$ref": "#/definitions/NavigationCause"
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"navigate"
|
||||
]
|
||||
},
|
||||
"result": {
|
||||
"$ref": "#/definitions/StepResultV3"
|
||||
},
|
||||
"state": {
|
||||
"description": "Observation id immediately before this action.",
|
||||
"type": "string"
|
||||
},
|
||||
"to": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"result",
|
||||
"state",
|
||||
"target"
|
||||
],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"click"
|
||||
]
|
||||
},
|
||||
"result": {
|
||||
"$ref": "#/definitions/StepResultV3"
|
||||
},
|
||||
"state": {
|
||||
"description": "Observation id immediately before this action.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV3"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"result",
|
||||
"state",
|
||||
"target"
|
||||
],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"hover"
|
||||
]
|
||||
},
|
||||
"result": {
|
||||
"$ref": "#/definitions/StepResultV3"
|
||||
},
|
||||
"state": {
|
||||
"description": "Observation id immediately before this action.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV3"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"commit",
|
||||
"id",
|
||||
"op",
|
||||
"result",
|
||||
"state",
|
||||
"target",
|
||||
"value"
|
||||
],
|
||||
"properties": {
|
||||
"commit": {
|
||||
"$ref": "#/definitions/FillCommit"
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"fill"
|
||||
]
|
||||
},
|
||||
"redacted": {
|
||||
"type": "boolean"
|
||||
},
|
||||
"result": {
|
||||
"$ref": "#/definitions/StepResultV3"
|
||||
},
|
||||
"state": {
|
||||
"description": "Observation id immediately before this action.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV3"
|
||||
},
|
||||
"value": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"result",
|
||||
"state",
|
||||
"target"
|
||||
],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"select"
|
||||
]
|
||||
},
|
||||
"result": {
|
||||
"$ref": "#/definitions/StepResultV3"
|
||||
},
|
||||
"selection": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/SelectedOptionV3"
|
||||
}
|
||||
},
|
||||
"state": {
|
||||
"description": "Observation id immediately before this action.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptorV3"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"key",
|
||||
"op",
|
||||
"result",
|
||||
"state"
|
||||
],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"key": {
|
||||
"type": "string"
|
||||
},
|
||||
"modifiers": {
|
||||
"type": [
|
||||
"array",
|
||||
"null"
|
||||
],
|
||||
"items": {
|
||||
"$ref": "#/definitions/KeyModifier"
|
||||
}
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"press"
|
||||
]
|
||||
},
|
||||
"result": {
|
||||
"$ref": "#/definitions/StepResultV3"
|
||||
},
|
||||
"state": {
|
||||
"description": "Observation id immediately before this action.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/TargetDescriptorV3"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"result",
|
||||
"state"
|
||||
],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"scroll"
|
||||
]
|
||||
},
|
||||
"result": {
|
||||
"$ref": "#/definitions/StepResultV3"
|
||||
},
|
||||
"state": {
|
||||
"description": "Observation id immediately before this action.",
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
},
|
||||
"StopReason": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"user_finish",
|
||||
"cli_stop"
|
||||
]
|
||||
},
|
||||
"TargetDescriptorV3": {
|
||||
"description": "Stable semantic handle for an interacted element within a page observation.",
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"ctx": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"name": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"ref": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"role": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"unmatched": {
|
||||
"type": "boolean"
|
||||
}
|
||||
}
|
||||
},
|
||||
"TraceEntry": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"start_url"
|
||||
],
|
||||
"properties": {
|
||||
"start_url": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"TraceStateV3": {
|
||||
"description": "Page observation dictionary entry — referenced by steps via `state` / `result.state`.",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"body",
|
||||
"id",
|
||||
"url"
|
||||
],
|
||||
"properties": {
|
||||
"body": {
|
||||
"description": "Full page observation (front matter + VOM body + annotations).",
|
||||
"type": "string"
|
||||
},
|
||||
"id": {
|
||||
"type": "string"
|
||||
},
|
||||
"title": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"truncated": {
|
||||
"type": "boolean"
|
||||
},
|
||||
"url": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -85,6 +85,10 @@ fn main() {
|
||||
dump!(PressResult, "tool_press_result");
|
||||
dump!(SelectParams, "tool_select_params");
|
||||
dump!(SelectResult, "tool_select_result");
|
||||
dump!(UploadParams, "tool_upload_params");
|
||||
dump!(UploadResult, "tool_upload_result");
|
||||
dump!(DownloadParams, "tool_download_params");
|
||||
dump!(DownloadResult, "tool_download_result");
|
||||
|
||||
dump!(SnapshotParams, "tool_snapshot_params");
|
||||
dump!(SnapshotResult, "tool_snapshot_result");
|
||||
@@ -113,12 +117,8 @@ fn main() {
|
||||
dump!(RequestHelpParams, "tool_request_help_params");
|
||||
dump!(RequestHelpResult, "tool_request_help_result");
|
||||
|
||||
dump!(TraceV2, "trace_v2");
|
||||
dump!(TraceV3, "trace_v3");
|
||||
dump!(RecordedTrace, "trace");
|
||||
dump!(StepV2, "trace_step_v2");
|
||||
dump!(StepV3, "trace_step_v3");
|
||||
dump!(RecordedStep, "trace_step");
|
||||
dump!(Trace, "trace");
|
||||
dump!(Step, "trace_step");
|
||||
dump!(RecordStartParams, "tool_record_start_params");
|
||||
dump!(RecordStartResult, "tool_record_start_result");
|
||||
dump!(RecordStopParams, "tool_record_stop_params");
|
||||
|
||||
@@ -78,6 +78,10 @@ pub enum Method {
|
||||
ToolPress,
|
||||
#[serde(rename = "tool.select")]
|
||||
ToolSelect,
|
||||
#[serde(rename = "tool.upload")]
|
||||
ToolUpload,
|
||||
#[serde(rename = "tool.download")]
|
||||
ToolDownload,
|
||||
#[serde(rename = "tool.snapshot")]
|
||||
ToolSnapshot,
|
||||
#[serde(rename = "tool.observe")]
|
||||
@@ -105,6 +109,17 @@ pub enum Method {
|
||||
#[serde(rename = "tool.record_await")]
|
||||
ToolRecordAwait,
|
||||
|
||||
#[serde(rename = "transfer.begin")]
|
||||
TransferBegin,
|
||||
#[serde(rename = "transfer.chunk")]
|
||||
TransferChunk,
|
||||
#[serde(rename = "transfer.finish")]
|
||||
TransferFinish,
|
||||
#[serde(rename = "transfer.read")]
|
||||
TransferRead,
|
||||
#[serde(rename = "transfer.release")]
|
||||
TransferRelease,
|
||||
|
||||
#[serde(rename = "cancel")]
|
||||
Cancel,
|
||||
}
|
||||
@@ -156,6 +171,8 @@ impl Method {
|
||||
| Method::ToolFill
|
||||
| Method::ToolPress
|
||||
| Method::ToolSelect
|
||||
| Method::ToolUpload
|
||||
| Method::ToolDownload
|
||||
| Method::ToolEvaluate
|
||||
// May navigate via optional `url` and changes Agent Window
|
||||
// chrome; gate behind pending-interrupt like other writes.
|
||||
@@ -194,6 +211,11 @@ impl Method {
|
||||
| Method::SystemPing
|
||||
| Method::SystemStatus
|
||||
| Method::BrowserList
|
||||
| Method::TransferBegin
|
||||
| Method::TransferChunk
|
||||
| Method::TransferFinish
|
||||
| Method::TransferRead
|
||||
| Method::TransferRelease
|
||||
| Method::Cancel => MethodEffect::ControlPlane,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,176 @@
|
||||
//! Content-transparent file transfer contracts.
|
||||
//!
|
||||
//! Public tool calls carry opaque transfer ids. Filesystem paths are injected
|
||||
//! by the daemon only on the daemon -> extension hop and are never accepted as
|
||||
//! an agent-facing source or destination.
|
||||
|
||||
use schemars::JsonSchema;
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct UploadFile {
|
||||
pub transfer_id: String,
|
||||
pub name: String,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub staged_path: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct UploadParams {
|
||||
pub session_id: String,
|
||||
#[serde(
|
||||
rename = "ref",
|
||||
alias = "ref_",
|
||||
default,
|
||||
skip_serializing_if = "Option::is_none"
|
||||
)]
|
||||
pub ref_: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub selector: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub tab_id: Option<i64>,
|
||||
pub files: Vec<UploadFile>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub timeout_ms: Option<u32>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct UploadResult {
|
||||
pub tab_id: i64,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub used_ref: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub used_selector: Option<String>,
|
||||
pub file_names: Vec<String>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct DownloadParams {
|
||||
pub session_id: String,
|
||||
#[serde(
|
||||
rename = "ref",
|
||||
alias = "ref_",
|
||||
default,
|
||||
skip_serializing_if = "Option::is_none"
|
||||
)]
|
||||
pub ref_: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub selector: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub tab_id: Option<i64>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub timeout_ms: Option<u32>,
|
||||
/// Daemon-injected relative directory beneath Chrome's Downloads root.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub browser_relative_dir: Option<String>,
|
||||
/// Daemon-injected authoritative transfer size limit. The extension uses
|
||||
/// it only for early cancellation; daemon import remains the final check.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub max_byte_size: Option<u64>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct DownloadResult {
|
||||
pub tab_id: i64,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub used_ref: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub used_selector: Option<String>,
|
||||
pub suggested_filename: String,
|
||||
pub byte_size: u64,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub mime: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub danger: Option<String>,
|
||||
/// Extension-internal completed Chrome download path, stripped by the daemon.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub browser_path: Option<String>,
|
||||
/// Opaque id returned by the daemon to the CLI.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub transfer_id: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct TransferBeginParams {
|
||||
pub session_id: String,
|
||||
pub name: String,
|
||||
pub byte_size: u64,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct TransferBeginResult {
|
||||
pub transfer_id: String,
|
||||
pub chunk_size: u32,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct TransferChunkParams {
|
||||
pub transfer_id: String,
|
||||
pub offset: u64,
|
||||
pub data_base64: String,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct TransferChunkResult {
|
||||
pub next_offset: u64,
|
||||
#[serde(default)]
|
||||
pub eof: bool,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub data_base64: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct TransferIdParams {
|
||||
pub transfer_id: String,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct TransferReadyResult {
|
||||
pub transfer_id: String,
|
||||
pub byte_size: u64,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct TransferReleaseResult {
|
||||
pub released: bool,
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn upload_agent_shape_omits_daemon_path_and_uses_ref_wire_name() {
|
||||
let value = serde_json::to_value(UploadParams {
|
||||
session_id: "s1".into(),
|
||||
ref_: Some("@e3".into()),
|
||||
selector: None,
|
||||
tab_id: None,
|
||||
files: vec![UploadFile {
|
||||
transfer_id: "tr_1".into(),
|
||||
name: "image.png".into(),
|
||||
staged_path: None,
|
||||
}],
|
||||
timeout_ms: None,
|
||||
})
|
||||
.unwrap();
|
||||
assert_eq!(value["ref"], "@e3");
|
||||
assert!(value["files"][0].get("staged_path").is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn download_agent_shape_omits_daemon_path() {
|
||||
let value = serde_json::to_value(DownloadParams {
|
||||
session_id: "s1".into(),
|
||||
ref_: None,
|
||||
selector: Some("#export".into()),
|
||||
tab_id: None,
|
||||
timeout_ms: None,
|
||||
browser_relative_dir: None,
|
||||
max_byte_size: None,
|
||||
})
|
||||
.unwrap();
|
||||
assert!(value.get("browser_relative_dir").is_none());
|
||||
assert!(value.get("max_byte_size").is_none());
|
||||
}
|
||||
}
|
||||
@@ -3,15 +3,13 @@
|
||||
pub mod console;
|
||||
pub mod dialog;
|
||||
pub mod emulate;
|
||||
pub mod file_transfer;
|
||||
pub mod human_loop;
|
||||
pub mod interaction;
|
||||
pub mod navigation;
|
||||
pub mod network;
|
||||
pub mod observation;
|
||||
pub mod record;
|
||||
mod record_common;
|
||||
mod record_v2;
|
||||
mod record_v3;
|
||||
pub mod script;
|
||||
pub mod session;
|
||||
pub mod tabs;
|
||||
@@ -21,6 +19,7 @@ pub mod window;
|
||||
pub use console::*;
|
||||
pub use dialog::*;
|
||||
pub use emulate::*;
|
||||
pub use file_transfer::*;
|
||||
pub use human_loop::*;
|
||||
pub use interaction::*;
|
||||
pub use navigation::*;
|
||||
|
||||
@@ -1,23 +1,147 @@
|
||||
//! Semantic user-action recording (`tool.record_start` / `stop` / `await`).
|
||||
//!
|
||||
//! Wire traces are either Trace v2 (`pages[]`) or Trace v3 (`version: 3`,
|
||||
//! `states[]`). Version-specific models live in `record_v2` / `record_v3`.
|
||||
//! Trace v2 is a **record-only** log of user actions: what was clicked, filled,
|
||||
//! selected, and where navigation occurred. No LLM runs during recording, so
|
||||
//! variable-vs-constant classification is **not** stored — executing agents
|
||||
//! infer that at run time from raw values and control names.
|
||||
|
||||
use schemars::JsonSchema;
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
pub use super::record_common::TraceEntry;
|
||||
pub use super::record_v2::{
|
||||
PageRefV2, SelectedOptionV2, StepCommonV2, StepEffectV2, StepV2, TargetDescriptorV2, TraceV2,
|
||||
};
|
||||
pub use super::record_v3::{
|
||||
FillCommit, NavigationCause, RecorderInfo, SelectedOptionV3, StepCommonV3, StepResultV3,
|
||||
StepV3, StopReason, TRACE_VERSION_V3, TargetDescriptorV3, TraceStateV3, TraceV3,
|
||||
VOM_FORMAT_VERSION,
|
||||
};
|
||||
use super::interaction::KeyModifier;
|
||||
|
||||
/// Logical v2 identifier. Not a wire field — v2 envelopes omit `version`.
|
||||
pub const TRACE_VERSION_V2: u32 = 2;
|
||||
// ---------------------------------------------------------------------------
|
||||
// Target
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// Stable semantic handle for an interacted element.
|
||||
///
|
||||
/// `name` and `nearby_label` are **untrusted page text**.
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct TargetDescriptor {
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub role: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub name: Option<String>,
|
||||
pub tag: String,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub name_attr: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub placeholder: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub nearby_label: Option<String>,
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Trace envelope
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// Recording entry point — first URL the flow starts from.
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct TraceEntry {
|
||||
pub start_url: String,
|
||||
}
|
||||
|
||||
/// Page context dictionary entry — referenced by steps via `page` id.
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct PageRef {
|
||||
pub id: String,
|
||||
pub url: String,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub title: Option<String>,
|
||||
}
|
||||
|
||||
/// One selected option (`select` op).
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct SelectedOption {
|
||||
pub value: String,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub label: Option<String>,
|
||||
}
|
||||
|
||||
/// Observed navigation after a step (objective fact only).
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct StepEffect {
|
||||
/// Reference into `pages[]` for the destination page.
|
||||
pub navigated_to: String,
|
||||
}
|
||||
|
||||
/// Fields shared by every step variant (flattened in JSON).
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct StepCommon {
|
||||
pub id: u32,
|
||||
/// Reference into `pages[]`.
|
||||
pub page: String,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub effect: Option<StepEffect>,
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Step op-specific payloads
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// One recorded user action — discriminated union by `op`.
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
#[serde(tag = "op", rename_all = "snake_case")]
|
||||
pub enum Step {
|
||||
Navigate {
|
||||
#[serde(flatten)]
|
||||
common: StepCommon,
|
||||
to: String,
|
||||
},
|
||||
Click {
|
||||
#[serde(flatten)]
|
||||
common: StepCommon,
|
||||
target: TargetDescriptor,
|
||||
},
|
||||
Hover {
|
||||
#[serde(flatten)]
|
||||
common: StepCommon,
|
||||
target: TargetDescriptor,
|
||||
},
|
||||
Fill {
|
||||
#[serde(flatten)]
|
||||
common: StepCommon,
|
||||
target: TargetDescriptor,
|
||||
value: String,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
redacted: Option<bool>,
|
||||
},
|
||||
Select {
|
||||
#[serde(flatten)]
|
||||
common: StepCommon,
|
||||
target: TargetDescriptor,
|
||||
selection: Vec<SelectedOption>,
|
||||
},
|
||||
Press {
|
||||
#[serde(flatten)]
|
||||
common: StepCommon,
|
||||
key: String,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
modifiers: Option<Vec<KeyModifier>>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
target: Option<TargetDescriptor>,
|
||||
},
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Trace root
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// Persisted user-action trace exported by `tool.record_stop` / `await`.
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct Trace {
|
||||
/// RFC 3339 timestamp when recording stopped.
|
||||
pub recorded_at: String,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub started_at: Option<String>,
|
||||
/// Optional user-provided goal from `--purpose` (metadata only).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub purpose: Option<String>,
|
||||
pub entry: TraceEntry,
|
||||
pub pages: Vec<PageRef>,
|
||||
pub steps: Vec<Step>,
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// RPC params / results
|
||||
@@ -45,209 +169,9 @@ pub struct RecordStopParams {
|
||||
pub session_id: String,
|
||||
}
|
||||
|
||||
/// Wire trace payload — v2 (legacy `pages[]`) or v3 (`version: 3`, `states[]`).
|
||||
///
|
||||
/// Classification matches [`RecordedTrace::classify_value`]: a numeric `version`
|
||||
/// selects v3 (and forbids `pages[]`); otherwise the envelope is v2 and must
|
||||
/// not include `states[]`.
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub enum RecordedTrace {
|
||||
V2(TraceV2),
|
||||
V3(TraceV3),
|
||||
}
|
||||
|
||||
impl RecordedTrace {
|
||||
pub fn classify_value(v: &serde_json::Value) -> Result<Self, String> {
|
||||
if let Some(ver) = v.get("version").and_then(|x| x.as_u64()) {
|
||||
if ver != u64::from(TRACE_VERSION_V3) {
|
||||
return Err(format!("unsupported trace version {ver}"));
|
||||
}
|
||||
if v.get("pages").is_some() {
|
||||
return Err("trace v3 must not include legacy pages[]".into());
|
||||
}
|
||||
return serde_json::from_value(v.clone())
|
||||
.map(RecordedTrace::V3)
|
||||
.map_err(|e| e.to_string());
|
||||
}
|
||||
if v.get("states").is_some() {
|
||||
return Err("trace v2 must not include states[]; set version: 3 for Trace v3".into());
|
||||
}
|
||||
if v.get("pages").is_some() {
|
||||
return serde_json::from_value(v.clone())
|
||||
.map(RecordedTrace::V2)
|
||||
.map_err(|e| e.to_string());
|
||||
}
|
||||
Err("ambiguous or unparseable trace".into())
|
||||
}
|
||||
}
|
||||
|
||||
impl Serialize for RecordedTrace {
|
||||
fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
|
||||
where
|
||||
S: serde::Serializer,
|
||||
{
|
||||
match self {
|
||||
RecordedTrace::V2(t) => t.serialize(serializer),
|
||||
RecordedTrace::V3(t) => t.serialize(serializer),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl<'de> Deserialize<'de> for RecordedTrace {
|
||||
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
|
||||
where
|
||||
D: serde::Deserializer<'de>,
|
||||
{
|
||||
let value = serde_json::Value::deserialize(deserializer)?;
|
||||
Self::classify_value(&value).map_err(serde::de::Error::custom)
|
||||
}
|
||||
}
|
||||
|
||||
impl JsonSchema for RecordedTrace {
|
||||
fn schema_name() -> String {
|
||||
"RecordedTrace".into()
|
||||
}
|
||||
|
||||
fn json_schema(generator: &mut schemars::SchemaGenerator) -> schemars::schema::Schema {
|
||||
// Mirror classify_value: a numeric `version` selects v3; otherwise v2.
|
||||
// TraceV2 stays open for unknown extension fields but forbids `states[]`
|
||||
// and a numeric `version`. TraceV3 already denies `pages[]`.
|
||||
let mut version_props = schemars::Map::new();
|
||||
version_props.insert(
|
||||
"version".into(),
|
||||
schemars::schema::SchemaObject {
|
||||
instance_type: Some(schemars::schema::InstanceType::Integer.into()),
|
||||
..Default::default()
|
||||
}
|
||||
.into(),
|
||||
);
|
||||
let mut required = schemars::Set::new();
|
||||
required.insert("version".into());
|
||||
|
||||
schemars::schema::SchemaObject {
|
||||
subschemas: Some(Box::new(schemars::schema::SubschemaValidation {
|
||||
if_schema: Some(Box::new(
|
||||
schemars::schema::SchemaObject {
|
||||
object: Some(Box::new(schemars::schema::ObjectValidation {
|
||||
properties: version_props,
|
||||
required,
|
||||
..Default::default()
|
||||
})),
|
||||
..Default::default()
|
||||
}
|
||||
.into(),
|
||||
)),
|
||||
then_schema: Some(Box::new(generator.subschema_for::<TraceV3>())),
|
||||
else_schema: Some(Box::new(generator.subschema_for::<TraceV2>())),
|
||||
..Default::default()
|
||||
})),
|
||||
..Default::default()
|
||||
}
|
||||
.into()
|
||||
}
|
||||
}
|
||||
|
||||
/// One recorded step — v2 (`page`) or v3 (`state` / `result.state`).
|
||||
///
|
||||
/// Classification matches [`RecordedStep::classify_value`]: `state` selects v3
|
||||
/// (and forbids `page`); otherwise the step is v2 and must include `page`.
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub enum RecordedStep {
|
||||
V2(StepV2),
|
||||
V3(StepV3),
|
||||
}
|
||||
|
||||
impl RecordedStep {
|
||||
pub fn classify_value(v: &serde_json::Value) -> Result<Self, String> {
|
||||
if v.get("state").is_some() {
|
||||
if v.get("page").is_some() {
|
||||
return Err("step v3 must not include legacy page".into());
|
||||
}
|
||||
return serde_json::from_value(v.clone())
|
||||
.map(RecordedStep::V3)
|
||||
.map_err(|e| e.to_string());
|
||||
}
|
||||
if v.get("page").is_some() {
|
||||
return serde_json::from_value(v.clone())
|
||||
.map(RecordedStep::V2)
|
||||
.map_err(|e| e.to_string());
|
||||
}
|
||||
Err("ambiguous or unparseable step".into())
|
||||
}
|
||||
}
|
||||
|
||||
impl Serialize for RecordedStep {
|
||||
fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
|
||||
where
|
||||
S: serde::Serializer,
|
||||
{
|
||||
match self {
|
||||
RecordedStep::V2(s) => s.serialize(serializer),
|
||||
RecordedStep::V3(s) => s.serialize(serializer),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl<'de> Deserialize<'de> for RecordedStep {
|
||||
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
|
||||
where
|
||||
D: serde::Deserializer<'de>,
|
||||
{
|
||||
let value = serde_json::Value::deserialize(deserializer)?;
|
||||
Self::classify_value(&value).map_err(serde::de::Error::custom)
|
||||
}
|
||||
}
|
||||
|
||||
impl JsonSchema for RecordedStep {
|
||||
fn schema_name() -> String {
|
||||
"RecordedStep".into()
|
||||
}
|
||||
|
||||
fn json_schema(generator: &mut schemars::SchemaGenerator) -> schemars::schema::Schema {
|
||||
// Mirror classify_value: `state` selects v3; otherwise v2.
|
||||
// Mixed page + state is rejected at this layer because StepV2/StepV3
|
||||
// still allow unknown extension fields.
|
||||
let mut required = schemars::Set::new();
|
||||
required.insert("state".into());
|
||||
let mut mixed_keys = schemars::Set::new();
|
||||
mixed_keys.insert("page".into());
|
||||
mixed_keys.insert("state".into());
|
||||
|
||||
schemars::schema::SchemaObject {
|
||||
subschemas: Some(Box::new(schemars::schema::SubschemaValidation {
|
||||
if_schema: Some(Box::new(
|
||||
schemars::schema::SchemaObject {
|
||||
object: Some(Box::new(schemars::schema::ObjectValidation {
|
||||
required,
|
||||
..Default::default()
|
||||
})),
|
||||
..Default::default()
|
||||
}
|
||||
.into(),
|
||||
)),
|
||||
then_schema: Some(Box::new(generator.subschema_for::<StepV3>())),
|
||||
else_schema: Some(Box::new(generator.subschema_for::<StepV2>())),
|
||||
not: Some(Box::new(
|
||||
schemars::schema::SchemaObject {
|
||||
object: Some(Box::new(schemars::schema::ObjectValidation {
|
||||
required: mixed_keys,
|
||||
..Default::default()
|
||||
})),
|
||||
..Default::default()
|
||||
}
|
||||
.into(),
|
||||
)),
|
||||
..Default::default()
|
||||
})),
|
||||
..Default::default()
|
||||
}
|
||||
.into()
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct RecordStopResult {
|
||||
pub trace: RecordedTrace,
|
||||
pub trace: Trace,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
@@ -259,197 +183,281 @@ pub struct RecordAwaitParams {
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct RecordAwaitResult {
|
||||
pub trace: RecordedTrace,
|
||||
pub trace: Trace,
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Tests
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use serde_json::json;
|
||||
|
||||
#[test]
|
||||
fn recorded_trace_classifies_v2_and_v3() {
|
||||
let v2 = json!({
|
||||
"recorded_at": "2026-07-21T08:00:00Z",
|
||||
"entry": { "start_url": "https://example.com/" },
|
||||
"pages": [{ "id": "p1", "url": "https://example.com/" }],
|
||||
"steps": []
|
||||
});
|
||||
match RecordedTrace::classify_value(&v2).unwrap() {
|
||||
RecordedTrace::V2(_) => {}
|
||||
other => panic!("expected v2, got {other:?}"),
|
||||
fn sample_common(id: u32) -> StepCommon {
|
||||
StepCommon {
|
||||
id,
|
||||
page: "p1".into(),
|
||||
effect: None,
|
||||
}
|
||||
}
|
||||
|
||||
let v3 = json!({
|
||||
"version": 3,
|
||||
"recorded_at": "2026-07-21T08:00:00Z",
|
||||
"stopped_by": "user_finish",
|
||||
"entry": { "start_url": "https://example.com/" },
|
||||
"recorder": { "bsk": "0.1.10", "vom": 1 },
|
||||
"states": [],
|
||||
"steps": []
|
||||
});
|
||||
match RecordedTrace::classify_value(&v3).unwrap() {
|
||||
RecordedTrace::V3(t) => assert_eq!(t.version, 3),
|
||||
other => panic!("expected v3, got {other:?}"),
|
||||
fn sample_target() -> TargetDescriptor {
|
||||
TargetDescriptor {
|
||||
role: Some("button".into()),
|
||||
name: Some("发布".into()),
|
||||
tag: "button".into(),
|
||||
name_attr: None,
|
||||
placeholder: None,
|
||||
nearby_label: None,
|
||||
}
|
||||
}
|
||||
|
||||
fn sample_trace() -> Trace {
|
||||
Trace {
|
||||
recorded_at: "2026-07-17T09:01:10Z".into(),
|
||||
started_at: Some("2026-07-17T09:00:00Z".into()),
|
||||
purpose: Some("发布一篇文章".into()),
|
||||
entry: TraceEntry {
|
||||
start_url: "https://x.com/editor".into(),
|
||||
},
|
||||
pages: vec![
|
||||
PageRef {
|
||||
id: "p1".into(),
|
||||
url: "https://x.com/editor".into(),
|
||||
title: Some("写文章".into()),
|
||||
},
|
||||
PageRef {
|
||||
id: "p2".into(),
|
||||
url: "https://x.com/p/99".into(),
|
||||
title: None,
|
||||
},
|
||||
],
|
||||
steps: vec![
|
||||
Step::Fill {
|
||||
common: sample_common(1),
|
||||
target: TargetDescriptor {
|
||||
role: Some("textbox".into()),
|
||||
name: Some("标题".into()),
|
||||
tag: "input".into(),
|
||||
name_attr: None,
|
||||
placeholder: None,
|
||||
nearby_label: None,
|
||||
},
|
||||
value: "我的第一篇文章".into(),
|
||||
redacted: None,
|
||||
},
|
||||
Step::Select {
|
||||
common: sample_common(3),
|
||||
target: TargetDescriptor {
|
||||
role: Some("combobox".into()),
|
||||
name: Some("分类".into()),
|
||||
tag: "select".into(),
|
||||
name_attr: None,
|
||||
placeholder: None,
|
||||
nearby_label: None,
|
||||
},
|
||||
selection: vec![SelectedOption {
|
||||
value: "tech".into(),
|
||||
label: Some("技术分享".into()),
|
||||
}],
|
||||
},
|
||||
Step::Click {
|
||||
common: StepCommon {
|
||||
effect: Some(StepEffect {
|
||||
navigated_to: "p2".into(),
|
||||
}),
|
||||
..sample_common(4)
|
||||
},
|
||||
target: sample_target(),
|
||||
},
|
||||
],
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn recorded_trace_rejects_mixed_and_unsupported_versions() {
|
||||
let v2_with_states = json!({
|
||||
"recorded_at": "2026-07-21T08:00:00Z",
|
||||
"entry": { "start_url": "https://example.com/" },
|
||||
"pages": [{ "id": "p1", "url": "https://example.com/" }],
|
||||
"states": [],
|
||||
"steps": []
|
||||
});
|
||||
assert!(RecordedTrace::classify_value(&v2_with_states).is_err());
|
||||
|
||||
let v3_with_pages = json!({
|
||||
"version": 3,
|
||||
"recorded_at": "2026-07-21T08:00:00Z",
|
||||
"stopped_by": "user_finish",
|
||||
"entry": { "start_url": "https://example.com/" },
|
||||
"recorder": { "bsk": "0.1.10", "vom": 1 },
|
||||
"pages": [{ "id": "p1", "url": "https://example.com/" }],
|
||||
"states": [],
|
||||
"steps": []
|
||||
});
|
||||
assert!(RecordedTrace::classify_value(&v3_with_pages).is_err());
|
||||
|
||||
let unsupported = json!({
|
||||
"version": 2,
|
||||
"recorded_at": "2026-07-21T08:00:00Z",
|
||||
"stopped_by": "user_finish",
|
||||
"entry": { "start_url": "https://example.com/" },
|
||||
"recorder": { "bsk": "0.1.10", "vom": 1 },
|
||||
"states": [],
|
||||
"steps": []
|
||||
});
|
||||
assert!(RecordedTrace::classify_value(&unsupported).is_err());
|
||||
assert!(serde_json::from_value::<TraceV3>(unsupported).is_err());
|
||||
fn step_click_round_trips() {
|
||||
let step = Step::Click {
|
||||
common: sample_common(1),
|
||||
target: sample_target(),
|
||||
};
|
||||
let v = serde_json::to_value(&step).unwrap();
|
||||
assert_eq!(v.get("op").and_then(|v| v.as_str()), Some("click"));
|
||||
assert_eq!(v.get("page").and_then(|v| v.as_str()), Some("p1"));
|
||||
assert!(v.get("key").is_none());
|
||||
let round: Step = serde_json::from_value(v).unwrap();
|
||||
assert_eq!(round, step);
|
||||
}
|
||||
|
||||
fn assert_classifies_by_integer_version(schema: &serde_json::Value) {
|
||||
#[test]
|
||||
fn step_fill_with_raw_value_round_trips() {
|
||||
let step = Step::Fill {
|
||||
common: sample_common(2),
|
||||
target: TargetDescriptor {
|
||||
role: Some("textbox".into()),
|
||||
name: Some("服务名称".into()),
|
||||
tag: "input".into(),
|
||||
name_attr: Some("serviceName".into()),
|
||||
placeholder: None,
|
||||
nearby_label: None,
|
||||
},
|
||||
value: "my-svc".into(),
|
||||
redacted: None,
|
||||
};
|
||||
let v = serde_json::to_value(&step).unwrap();
|
||||
assert_eq!(v["op"], "fill");
|
||||
assert_eq!(v["value"], "my-svc");
|
||||
let round: Step = serde_json::from_value(v).unwrap();
|
||||
assert_eq!(round, step);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn step_fill_password_is_redacted() {
|
||||
let step = Step::Fill {
|
||||
common: sample_common(1),
|
||||
target: TargetDescriptor {
|
||||
role: Some("textbox".into()),
|
||||
name: Some("密码".into()),
|
||||
tag: "input".into(),
|
||||
name_attr: None,
|
||||
placeholder: None,
|
||||
nearby_label: None,
|
||||
},
|
||||
value: "***".into(),
|
||||
redacted: Some(true),
|
||||
};
|
||||
let v = serde_json::to_value(&step).unwrap();
|
||||
assert_eq!(v["value"], "***");
|
||||
assert_eq!(v["redacted"], true);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn step_select_uses_object_array() {
|
||||
let step = Step::Select {
|
||||
common: sample_common(3),
|
||||
target: TargetDescriptor {
|
||||
role: Some("combobox".into()),
|
||||
name: Some("分类".into()),
|
||||
tag: "select".into(),
|
||||
name_attr: None,
|
||||
placeholder: None,
|
||||
nearby_label: None,
|
||||
},
|
||||
selection: vec![SelectedOption {
|
||||
value: "tech".into(),
|
||||
label: Some("技术分享".into()),
|
||||
}],
|
||||
};
|
||||
let v = serde_json::to_value(&step).unwrap();
|
||||
assert_eq!(v["selection"][0]["value"], "tech");
|
||||
assert_eq!(v["selection"][0]["label"], "技术分享");
|
||||
let round: Step = serde_json::from_value(v).unwrap();
|
||||
assert_eq!(round, step);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn step_navigate_round_trips() {
|
||||
let step = Step::Navigate {
|
||||
common: sample_common(1),
|
||||
to: "https://example.com".into(),
|
||||
};
|
||||
let v = serde_json::to_value(&step).unwrap();
|
||||
assert_eq!(v["op"], "navigate");
|
||||
assert_eq!(v["to"], "https://example.com");
|
||||
let round: Step = serde_json::from_value(v).unwrap();
|
||||
assert_eq!(round, step);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn step_press_with_modifiers_round_trips() {
|
||||
let step = Step::Press {
|
||||
common: StepCommon {
|
||||
effect: Some(StepEffect {
|
||||
navigated_to: "p2".into(),
|
||||
}),
|
||||
..sample_common(2)
|
||||
},
|
||||
key: "Enter".into(),
|
||||
modifiers: Some(vec![KeyModifier::Ctrl, KeyModifier::Shift]),
|
||||
target: Some(sample_target()),
|
||||
};
|
||||
let v = serde_json::to_value(&step).unwrap();
|
||||
assert_eq!(v["key"], "Enter");
|
||||
assert_eq!(v["modifiers"], json!(["ctrl", "shift"]));
|
||||
assert_eq!(v["effect"]["navigated_to"], "p2");
|
||||
let round: Step = serde_json::from_value(v).unwrap();
|
||||
assert_eq!(round, step);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn trace_record_only_round_trips() {
|
||||
let trace = sample_trace();
|
||||
let v = serde_json::to_value(&trace).unwrap();
|
||||
assert!(v.get("version").is_none());
|
||||
assert_eq!(
|
||||
schema["if"]["required"],
|
||||
json!(["version"]),
|
||||
"numeric version is the RecordedTrace classification key"
|
||||
v.get("purpose").and_then(|v| v.as_str()),
|
||||
Some("发布一篇文章")
|
||||
);
|
||||
assert_eq!(schema["if"]["properties"]["version"]["type"], "integer");
|
||||
assert_eq!(schema["then"]["$ref"], "#/definitions/TraceV3");
|
||||
assert_eq!(schema["else"]["$ref"], "#/definitions/TraceV2");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn recorded_trace_schema_includes_v2_and_v3() {
|
||||
let schema = serde_json::to_value(schemars::schema_for!(RecordStopResult)).unwrap();
|
||||
assert_classifies_by_integer_version(&schema["definitions"]["RecordedTrace"]);
|
||||
assert!(v.get("parameters").is_none());
|
||||
assert!(v.get("site").is_none());
|
||||
assert!(v.get("goal").is_none());
|
||||
assert_eq!(
|
||||
schema["definitions"]["TraceV3"]["properties"]["version"]["const"],
|
||||
3
|
||||
v.get("entry")
|
||||
.and_then(|e| e.get("start_url"))
|
||||
.and_then(|u| u.as_str()),
|
||||
Some("https://x.com/editor")
|
||||
);
|
||||
let round: Trace = serde_json::from_value(v).unwrap();
|
||||
assert_eq!(round, trace);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn recorded_trace_schema_follows_classify_value() {
|
||||
let standalone = serde_json::to_value(schemars::schema_for!(RecordedTrace)).unwrap();
|
||||
assert_classifies_by_integer_version(&standalone);
|
||||
assert!(standalone["definitions"].get("TraceV2").is_some());
|
||||
assert!(standalone["definitions"].get("TraceV3").is_some());
|
||||
|
||||
let stop_result = serde_json::to_value(schemars::schema_for!(RecordStopResult)).unwrap();
|
||||
assert_classifies_by_integer_version(&stop_result["definitions"]["RecordedTrace"]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn standalone_trace_schema_is_recorded_trace_union() {
|
||||
let schema = serde_json::to_value(schemars::schema_for!(RecordedTrace)).unwrap();
|
||||
assert_eq!(schema["title"], "RecordedTrace");
|
||||
assert_classifies_by_integer_version(&schema);
|
||||
assert!(schema["definitions"].get("TraceV2").is_some());
|
||||
assert!(schema["definitions"].get("TraceV3").is_some());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn recorded_trace_accepts_v2_with_unknown_extension_fields() {
|
||||
let v2 = json!({
|
||||
"recorded_at": "2026-07-21T08:00:00Z",
|
||||
"entry": { "start_url": "https://example.com/" },
|
||||
"pages": [{ "id": "p1", "url": "https://example.com/" }],
|
||||
"steps": [],
|
||||
"meta": { "tool": "legacy-exporter" }
|
||||
});
|
||||
match RecordedTrace::classify_value(&v2).unwrap() {
|
||||
RecordedTrace::V2(trace) => assert_eq!(trace.pages.len(), 1),
|
||||
other => panic!("expected v2, got {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
fn v2_click() -> serde_json::Value {
|
||||
json!({
|
||||
fn discriminated_union_click_ignores_extra_key_field() {
|
||||
let bad = json!({
|
||||
"op": "click",
|
||||
"id": 1,
|
||||
"page": "p1",
|
||||
"target": { "tag": "button", "role": "button", "name": "发布" }
|
||||
})
|
||||
}
|
||||
|
||||
fn v3_click() -> serde_json::Value {
|
||||
json!({
|
||||
"op": "click",
|
||||
"id": 1,
|
||||
"state": "s1",
|
||||
"result": { "state": "s2" },
|
||||
"target": { "ref": "e1", "role": "button", "name": "发布" }
|
||||
})
|
||||
}
|
||||
|
||||
fn assert_step_classifies_by_state(schema: &serde_json::Value) {
|
||||
assert_eq!(
|
||||
schema["if"]["required"],
|
||||
json!(["state"]),
|
||||
"state is the RecordedStep classification key"
|
||||
);
|
||||
assert_eq!(schema["then"]["$ref"], "#/definitions/StepV3");
|
||||
assert_eq!(schema["else"]["$ref"], "#/definitions/StepV2");
|
||||
let mut forbidden = schema["not"]["required"]
|
||||
.as_array()
|
||||
.expect("RecordedStep schema must reject mixed page + state steps")
|
||||
.iter()
|
||||
.filter_map(|v| v.as_str())
|
||||
.collect::<Vec<_>>();
|
||||
forbidden.sort_unstable();
|
||||
assert_eq!(forbidden, ["page", "state"]);
|
||||
"target": { "tag": "button", "name": "OK" },
|
||||
"key": "Enter"
|
||||
});
|
||||
let step: Step = serde_json::from_value(bad).unwrap();
|
||||
assert!(matches!(step, Step::Click { .. }));
|
||||
let v = serde_json::to_value(&step).unwrap();
|
||||
assert!(v.get("key").is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn recorded_step_classifies_v2_and_v3() {
|
||||
match RecordedStep::classify_value(&v2_click()).unwrap() {
|
||||
RecordedStep::V2(StepV2::Click { .. }) => {}
|
||||
other => panic!("expected v2 click, got {other:?}"),
|
||||
}
|
||||
match RecordedStep::classify_value(&v3_click()).unwrap() {
|
||||
RecordedStep::V3(StepV3::Click { .. }) => {}
|
||||
other => panic!("expected v3 click, got {other:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn recorded_step_rejects_mixed_page_and_state() {
|
||||
let mut mixed = v2_click();
|
||||
mixed["state"] = json!("s1");
|
||||
mixed["result"] = json!({ "state": "s2" });
|
||||
assert!(RecordedStep::classify_value(&mixed).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn recorded_step_schema_follows_classify_value() {
|
||||
let schema = serde_json::to_value(schemars::schema_for!(RecordedStep)).unwrap();
|
||||
assert_eq!(schema["title"], "RecordedStep");
|
||||
assert_step_classifies_by_state(&schema);
|
||||
assert!(schema["definitions"].get("StepV2").is_some());
|
||||
assert!(schema["definitions"].get("StepV3").is_some());
|
||||
fn extension_trace_deserializes() {
|
||||
let v = json!({
|
||||
"recorded_at": "2026-07-21T08:00:00Z",
|
||||
"started_at": "2026-07-21T07:59:00Z",
|
||||
"purpose": "demo",
|
||||
"entry": { "start_url": "https://example.com/editor" },
|
||||
"pages": [
|
||||
{ "id": "p1", "url": "https://example.com/editor" },
|
||||
{ "id": "p2", "url": "https://example.com/p/99" }
|
||||
],
|
||||
"steps": [
|
||||
{
|
||||
"op": "fill",
|
||||
"id": 1,
|
||||
"page": "p1",
|
||||
"target": { "tag": "input", "role": "textbox", "name": "标题" },
|
||||
"value": "hello"
|
||||
},
|
||||
{
|
||||
"op": "click",
|
||||
"id": 2,
|
||||
"page": "p1",
|
||||
"target": { "tag": "button", "role": "button", "name": "发布" },
|
||||
"effect": { "navigated_to": "p2" }
|
||||
}
|
||||
]
|
||||
});
|
||||
let trace: Trace = serde_json::from_value(v).unwrap();
|
||||
assert_eq!(trace.entry.start_url, "https://example.com/editor");
|
||||
assert_eq!(trace.pages.len(), 2);
|
||||
assert_eq!(trace.steps.len(), 2);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,7 +0,0 @@
|
||||
use schemars::JsonSchema;
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct TraceEntry {
|
||||
pub start_url: String,
|
||||
}
|
||||
@@ -1,330 +0,0 @@
|
||||
//! Trace v2 — record-only user-action log with `pages[]` and step `page` refs.
|
||||
//!
|
||||
//! Legacy wire format with no top-level `version` field.
|
||||
|
||||
use schemars::JsonSchema;
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
use super::interaction::KeyModifier;
|
||||
use super::record_common::TraceEntry;
|
||||
|
||||
/// Stable semantic handle for an interacted element (v2).
|
||||
///
|
||||
/// `name` and `nearby_label` are **untrusted page text**.
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct TargetDescriptorV2 {
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub role: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub name: Option<String>,
|
||||
pub tag: String,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub name_attr: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub placeholder: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub nearby_label: Option<String>,
|
||||
}
|
||||
|
||||
/// Page context dictionary entry — referenced by steps via `page` id.
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct PageRefV2 {
|
||||
pub id: String,
|
||||
pub url: String,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub title: Option<String>,
|
||||
}
|
||||
|
||||
/// One selected option (`select` op).
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct SelectedOptionV2 {
|
||||
pub value: String,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub label: Option<String>,
|
||||
}
|
||||
|
||||
/// Observed navigation after a step (objective fact only).
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct StepEffectV2 {
|
||||
/// Reference into `pages[]` for the destination page.
|
||||
pub navigated_to: String,
|
||||
}
|
||||
|
||||
/// Fields shared by every v2 step variant (flattened in JSON).
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct StepCommonV2 {
|
||||
pub id: u32,
|
||||
/// Reference into `pages[]`.
|
||||
pub page: String,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub effect: Option<StepEffectV2>,
|
||||
}
|
||||
|
||||
/// One recorded user action — discriminated union by `op` (v2).
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
#[serde(tag = "op", rename_all = "snake_case")]
|
||||
pub enum StepV2 {
|
||||
Navigate {
|
||||
#[serde(flatten)]
|
||||
common: StepCommonV2,
|
||||
to: String,
|
||||
},
|
||||
Click {
|
||||
#[serde(flatten)]
|
||||
common: StepCommonV2,
|
||||
target: TargetDescriptorV2,
|
||||
},
|
||||
Hover {
|
||||
#[serde(flatten)]
|
||||
common: StepCommonV2,
|
||||
target: TargetDescriptorV2,
|
||||
},
|
||||
Fill {
|
||||
#[serde(flatten)]
|
||||
common: StepCommonV2,
|
||||
target: TargetDescriptorV2,
|
||||
value: String,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
redacted: Option<bool>,
|
||||
},
|
||||
Select {
|
||||
#[serde(flatten)]
|
||||
common: StepCommonV2,
|
||||
target: TargetDescriptorV2,
|
||||
selection: Vec<SelectedOptionV2>,
|
||||
},
|
||||
Press {
|
||||
#[serde(flatten)]
|
||||
common: StepCommonV2,
|
||||
key: String,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
modifiers: Option<Vec<KeyModifier>>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
target: Option<TargetDescriptorV2>,
|
||||
},
|
||||
}
|
||||
|
||||
/// Persisted user-action trace exported by legacy `tool.record_stop` / `await`.
|
||||
///
|
||||
/// Unknown extension fields are ignored so older traces remain readable.
|
||||
/// `states[]` and a numeric `version` are reserved for Trace v3 / `RecordedTrace`
|
||||
/// classification — serde still ignores them, but the JSON Schema rejects them.
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
|
||||
pub struct TraceV2 {
|
||||
/// RFC 3339 timestamp when recording stopped.
|
||||
pub recorded_at: String,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub started_at: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub purpose: Option<String>,
|
||||
pub entry: TraceEntry,
|
||||
pub pages: Vec<PageRefV2>,
|
||||
pub steps: Vec<StepV2>,
|
||||
}
|
||||
|
||||
fn constrain_trace_v2_schema(mut schema: schemars::schema::Schema) -> schemars::schema::Schema {
|
||||
let schemars::schema::Schema::Object(obj) = &mut schema else {
|
||||
return schema;
|
||||
};
|
||||
obj.metadata().title = Some("TraceV2".into());
|
||||
obj.metadata().description = Some(
|
||||
"Persisted user-action trace exported by legacy `tool.record_stop` / `await`.\n\n\
|
||||
Unknown extension fields are ignored so older traces remain readable. \
|
||||
`states[]` and a numeric `version` are reserved for Trace v3 / `RecordedTrace` classification."
|
||||
.into(),
|
||||
);
|
||||
|
||||
let object = obj.object.get_or_insert_with(Default::default);
|
||||
object
|
||||
.properties
|
||||
.insert("states".into(), schemars::schema::Schema::Bool(false));
|
||||
object.properties.insert(
|
||||
"version".into(),
|
||||
schemars::schema::SchemaObject {
|
||||
metadata: Some(Box::new(schemars::schema::Metadata {
|
||||
description: Some(
|
||||
"Numeric version selects Trace v3. Legacy v2 envelopes omit this field.".into(),
|
||||
),
|
||||
..Default::default()
|
||||
})),
|
||||
subschemas: Some(Box::new(schemars::schema::SubschemaValidation {
|
||||
not: Some(Box::new(
|
||||
schemars::schema::SchemaObject {
|
||||
instance_type: Some(schemars::schema::InstanceType::Integer.into()),
|
||||
..Default::default()
|
||||
}
|
||||
.into(),
|
||||
)),
|
||||
..Default::default()
|
||||
})),
|
||||
..Default::default()
|
||||
}
|
||||
.into(),
|
||||
);
|
||||
schema
|
||||
}
|
||||
|
||||
impl JsonSchema for TraceV2 {
|
||||
fn schema_name() -> String {
|
||||
"TraceV2".into()
|
||||
}
|
||||
|
||||
fn json_schema(generator: &mut schemars::SchemaGenerator) -> schemars::schema::Schema {
|
||||
#[derive(JsonSchema)]
|
||||
#[allow(dead_code)]
|
||||
struct TraceV2Shape {
|
||||
/// RFC 3339 timestamp when recording stopped.
|
||||
recorded_at: String,
|
||||
started_at: Option<String>,
|
||||
purpose: Option<String>,
|
||||
entry: TraceEntry,
|
||||
pages: Vec<PageRefV2>,
|
||||
steps: Vec<StepV2>,
|
||||
}
|
||||
|
||||
constrain_trace_v2_schema(TraceV2Shape::json_schema(generator))
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use serde_json::json;
|
||||
|
||||
fn sample_common(id: u32) -> StepCommonV2 {
|
||||
StepCommonV2 {
|
||||
id,
|
||||
page: "p1".into(),
|
||||
effect: None,
|
||||
}
|
||||
}
|
||||
|
||||
fn sample_target() -> TargetDescriptorV2 {
|
||||
TargetDescriptorV2 {
|
||||
role: Some("button".into()),
|
||||
name: Some("发布".into()),
|
||||
tag: "button".into(),
|
||||
name_attr: None,
|
||||
placeholder: None,
|
||||
nearby_label: None,
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn trace_v2_has_no_version_field() {
|
||||
let trace = TraceV2 {
|
||||
recorded_at: "2026-07-17T09:01:10Z".into(),
|
||||
started_at: None,
|
||||
purpose: None,
|
||||
entry: TraceEntry {
|
||||
start_url: "https://example.com/".into(),
|
||||
},
|
||||
pages: vec![PageRefV2 {
|
||||
id: "p1".into(),
|
||||
url: "https://example.com/".into(),
|
||||
title: None,
|
||||
}],
|
||||
steps: vec![StepV2::Click {
|
||||
common: sample_common(1),
|
||||
target: sample_target(),
|
||||
}],
|
||||
};
|
||||
let v = serde_json::to_value(&trace).unwrap();
|
||||
assert!(v.get("version").is_none());
|
||||
assert!(v.get("pages").is_some());
|
||||
assert!(v.get("states").is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn extension_v2_trace_deserializes() {
|
||||
let v = json!({
|
||||
"recorded_at": "2026-07-21T08:00:00Z",
|
||||
"started_at": "2026-07-21T07:59:00Z",
|
||||
"purpose": "demo",
|
||||
"entry": { "start_url": "https://example.com/editor" },
|
||||
"pages": [
|
||||
{ "id": "p1", "url": "https://example.com/editor" },
|
||||
{ "id": "p2", "url": "https://example.com/p/99" }
|
||||
],
|
||||
"steps": [
|
||||
{
|
||||
"op": "fill",
|
||||
"id": 1,
|
||||
"page": "p1",
|
||||
"target": { "tag": "input", "role": "textbox", "name": "标题" },
|
||||
"value": "hello"
|
||||
},
|
||||
{
|
||||
"op": "click",
|
||||
"id": 2,
|
||||
"page": "p1",
|
||||
"target": { "tag": "button", "role": "button", "name": "发布" },
|
||||
"effect": { "navigated_to": "p2" }
|
||||
}
|
||||
]
|
||||
});
|
||||
let trace: TraceV2 = serde_json::from_value(v).unwrap();
|
||||
assert_eq!(trace.pages.len(), 2);
|
||||
assert_eq!(trace.steps.len(), 2);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn v2_hover_steps_round_trip() {
|
||||
let value = json!({
|
||||
"recorded_at": "2026-07-21T08:00:00Z",
|
||||
"entry": { "start_url": "https://example.com/" },
|
||||
"pages": [{ "id": "p1", "url": "https://example.com/" }],
|
||||
"steps": [{
|
||||
"op": "hover",
|
||||
"id": 1,
|
||||
"page": "p1",
|
||||
"target": { "tag": "span", "role": "button", "name": "Account" }
|
||||
}]
|
||||
});
|
||||
|
||||
let trace: TraceV2 = serde_json::from_value(value).unwrap();
|
||||
assert!(matches!(trace.steps.as_slice(), [StepV2::Hover { .. }]));
|
||||
assert_eq!(
|
||||
serde_json::to_value(&trace).unwrap()["steps"][0]["op"],
|
||||
"hover"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn trace_v2_ignores_unknown_extension_fields() {
|
||||
let value = json!({
|
||||
"recorded_at": "2026-07-21T08:00:00Z",
|
||||
"entry": { "start_url": "https://example.com/" },
|
||||
"pages": [{ "id": "p1", "url": "https://example.com/" }],
|
||||
"steps": [],
|
||||
"meta": { "tool": "legacy-exporter" }
|
||||
});
|
||||
let trace: TraceV2 = serde_json::from_value(value).unwrap();
|
||||
assert_eq!(trace.pages.len(), 1);
|
||||
assert!(trace.steps.is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn trace_v2_schema_allows_additional_properties() {
|
||||
let schema = serde_json::to_value(schemars::schema_for!(TraceV2)).unwrap();
|
||||
assert_ne!(
|
||||
schema.get("additionalProperties"),
|
||||
Some(&serde_json::Value::Bool(false)),
|
||||
"Trace v2 must keep accepting traces with unknown extension fields"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn trace_v2_schema_forbids_classification_keys() {
|
||||
let schema = serde_json::to_value(schemars::schema_for!(TraceV2)).unwrap();
|
||||
assert_eq!(
|
||||
schema["properties"]["states"],
|
||||
json!(false),
|
||||
"Trace v2 schema must reject states[] (reserved for v3)"
|
||||
);
|
||||
assert_eq!(
|
||||
schema["properties"]["version"]["not"]["type"], "integer",
|
||||
"Trace v2 schema must reject a numeric version (that selects the v3 path)"
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -1,428 +0,0 @@
|
||||
//! Trace v3 — state-action-state log with `states[]` and step `state` refs.
|
||||
//!
|
||||
//! Wire format with top-level `version: 3`.
|
||||
|
||||
use schemars::JsonSchema;
|
||||
use serde::{Deserialize, Deserializer, Serialize};
|
||||
|
||||
use super::interaction::KeyModifier;
|
||||
use super::record_common::TraceEntry;
|
||||
|
||||
pub const TRACE_VERSION_V3: u32 = 3;
|
||||
pub const VOM_FORMAT_VERSION: u32 = 1;
|
||||
|
||||
fn deserialize_trace_v3_version<'de, D>(deserializer: D) -> Result<u32, D::Error>
|
||||
where
|
||||
D: Deserializer<'de>,
|
||||
{
|
||||
let version = u32::deserialize(deserializer)?;
|
||||
if version != TRACE_VERSION_V3 {
|
||||
return Err(serde::de::Error::custom(format!(
|
||||
"unsupported trace version {version} (expected {TRACE_VERSION_V3})"
|
||||
)));
|
||||
}
|
||||
Ok(version)
|
||||
}
|
||||
|
||||
fn trace_v3_version_schema(_: &mut schemars::r#gen::SchemaGenerator) -> schemars::schema::Schema {
|
||||
schemars::schema::SchemaObject {
|
||||
instance_type: Some(schemars::schema::InstanceType::Integer.into()),
|
||||
const_value: Some(serde_json::json!(TRACE_VERSION_V3)),
|
||||
..Default::default()
|
||||
}
|
||||
.into()
|
||||
}
|
||||
|
||||
/// Stable semantic handle for an interacted element within a page observation.
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct TargetDescriptorV3 {
|
||||
#[serde(default, skip_serializing_if = "Option::is_none", rename = "ref")]
|
||||
pub element_ref: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub role: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub name: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub ctx: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "std::ops::Not::not")]
|
||||
pub unmatched: bool,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct RecorderInfo {
|
||||
pub bsk: String,
|
||||
pub vom: u32,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
|
||||
#[serde(rename_all = "snake_case")]
|
||||
pub enum StopReason {
|
||||
UserFinish,
|
||||
CliStop,
|
||||
}
|
||||
|
||||
/// Page observation dictionary entry — referenced by steps via `state` / `result.state`.
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct TraceStateV3 {
|
||||
pub id: String,
|
||||
pub url: String,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub title: Option<String>,
|
||||
/// Full page observation (front matter + VOM body + annotations).
|
||||
pub body: String,
|
||||
#[serde(default, skip_serializing_if = "std::ops::Not::not")]
|
||||
pub truncated: bool,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct StepResultV3 {
|
||||
pub state: String,
|
||||
}
|
||||
|
||||
/// Fields shared by every step variant (flattened in JSON).
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct StepCommonV3 {
|
||||
pub id: u32,
|
||||
/// Observation id immediately before this action.
|
||||
pub state: String,
|
||||
pub result: StepResultV3,
|
||||
}
|
||||
|
||||
/// One selected option (`select` op).
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct SelectedOptionV3 {
|
||||
pub value: String,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub label: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
|
||||
#[serde(rename_all = "snake_case")]
|
||||
pub enum NavigationCause {
|
||||
UserTyped,
|
||||
Link,
|
||||
FormSubmit,
|
||||
Reload,
|
||||
History,
|
||||
Script,
|
||||
Browser,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
|
||||
#[serde(rename_all = "snake_case")]
|
||||
pub enum FillCommit {
|
||||
Enter,
|
||||
Suggestion,
|
||||
Blur,
|
||||
}
|
||||
|
||||
/// One recorded user action — discriminated union by `op`.
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
#[serde(tag = "op", rename_all = "snake_case")]
|
||||
pub enum StepV3 {
|
||||
Navigate {
|
||||
#[serde(flatten)]
|
||||
common: StepCommonV3,
|
||||
to: String,
|
||||
cause: NavigationCause,
|
||||
},
|
||||
Click {
|
||||
#[serde(flatten)]
|
||||
common: StepCommonV3,
|
||||
target: TargetDescriptorV3,
|
||||
},
|
||||
Hover {
|
||||
#[serde(flatten)]
|
||||
common: StepCommonV3,
|
||||
target: TargetDescriptorV3,
|
||||
},
|
||||
Fill {
|
||||
#[serde(flatten)]
|
||||
common: StepCommonV3,
|
||||
target: TargetDescriptorV3,
|
||||
value: String,
|
||||
commit: FillCommit,
|
||||
#[serde(default, skip_serializing_if = "std::ops::Not::not")]
|
||||
redacted: bool,
|
||||
},
|
||||
Select {
|
||||
#[serde(flatten)]
|
||||
common: StepCommonV3,
|
||||
target: TargetDescriptorV3,
|
||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
||||
selection: Vec<SelectedOptionV3>,
|
||||
},
|
||||
Press {
|
||||
#[serde(flatten)]
|
||||
common: StepCommonV3,
|
||||
key: String,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
modifiers: Option<Vec<KeyModifier>>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
target: Option<TargetDescriptorV3>,
|
||||
},
|
||||
Scroll {
|
||||
#[serde(flatten)]
|
||||
common: StepCommonV3,
|
||||
},
|
||||
}
|
||||
|
||||
/// Wire trace returned by `tool.record_stop` / `await`.
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
#[serde(deny_unknown_fields)]
|
||||
pub struct TraceV3 {
|
||||
#[serde(deserialize_with = "deserialize_trace_v3_version")]
|
||||
#[schemars(schema_with = "trace_v3_version_schema")]
|
||||
pub version: u32,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub purpose: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub started_at: Option<String>,
|
||||
pub recorded_at: String,
|
||||
pub stopped_by: StopReason,
|
||||
pub entry: TraceEntry,
|
||||
pub recorder: RecorderInfo,
|
||||
pub states: Vec<TraceStateV3>,
|
||||
pub steps: Vec<StepV3>,
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use serde_json::json;
|
||||
|
||||
fn sample_common(id: u32, state: &str, result_state: &str) -> StepCommonV3 {
|
||||
StepCommonV3 {
|
||||
id,
|
||||
state: state.into(),
|
||||
result: StepResultV3 {
|
||||
state: result_state.into(),
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
fn sample_target() -> TargetDescriptorV3 {
|
||||
TargetDescriptorV3 {
|
||||
element_ref: Some("e21".into()),
|
||||
role: Some("button".into()),
|
||||
name: Some("发布".into()),
|
||||
ctx: Some("金桔柠檬 6 号".into()),
|
||||
unmatched: false,
|
||||
}
|
||||
}
|
||||
|
||||
fn sample_trace() -> TraceV3 {
|
||||
TraceV3 {
|
||||
version: TRACE_VERSION_V3,
|
||||
purpose: Some("把草稿商品发布上架".into()),
|
||||
started_at: Some("2026-08-10T02:10:41.080Z".into()),
|
||||
recorded_at: "2026-08-10T02:12:55.360Z".into(),
|
||||
stopped_by: StopReason::UserFinish,
|
||||
entry: TraceEntry {
|
||||
start_url: "https://example.com/".into(),
|
||||
},
|
||||
recorder: RecorderInfo {
|
||||
bsk: "0.1.10".into(),
|
||||
vom: VOM_FORMAT_VERSION,
|
||||
},
|
||||
states: vec![
|
||||
TraceStateV3 {
|
||||
id: "s1".into(),
|
||||
url: "https://example.com/".into(),
|
||||
title: Some("Example Domain".into()),
|
||||
body: "@vom 1\nRootWebArea \"Example Domain\"".into(),
|
||||
truncated: false,
|
||||
},
|
||||
TraceStateV3 {
|
||||
id: "s2".into(),
|
||||
url: "https://shop.example.com/products?status=draft".into(),
|
||||
title: Some("商品管理".into()),
|
||||
body: "@vom 1\nRootWebArea \"商品管理\"".into(),
|
||||
truncated: false,
|
||||
},
|
||||
],
|
||||
steps: vec![
|
||||
StepV3::Navigate {
|
||||
common: sample_common(1, "s1", "s2"),
|
||||
to: "https://shop.example.com/products?status=draft".into(),
|
||||
cause: NavigationCause::UserTyped,
|
||||
},
|
||||
StepV3::Fill {
|
||||
common: sample_common(2, "s2", "s2"),
|
||||
target: TargetDescriptorV3 {
|
||||
element_ref: Some("e12".into()),
|
||||
role: Some("textbox".into()),
|
||||
name: Some("搜索商品".into()),
|
||||
ctx: None,
|
||||
unmatched: false,
|
||||
},
|
||||
value: "金桔柠檬".into(),
|
||||
commit: FillCommit::Enter,
|
||||
redacted: false,
|
||||
},
|
||||
StepV3::Click {
|
||||
common: sample_common(3, "s2", "s2"),
|
||||
target: sample_target(),
|
||||
},
|
||||
],
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn step_click_round_trips() {
|
||||
let step = StepV3::Click {
|
||||
common: sample_common(1, "s1", "s2"),
|
||||
target: sample_target(),
|
||||
};
|
||||
let v = serde_json::to_value(&step).unwrap();
|
||||
assert_eq!(v.get("op").and_then(|v| v.as_str()), Some("click"));
|
||||
assert_eq!(v.get("state").and_then(|v| v.as_str()), Some("s1"));
|
||||
assert_eq!(v["result"]["state"], "s2");
|
||||
assert_eq!(v["target"]["ref"], "e21");
|
||||
let round: StepV3 = serde_json::from_value(v).unwrap();
|
||||
assert_eq!(round, step);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn step_fill_with_commit_round_trips() {
|
||||
let step = StepV3::Fill {
|
||||
common: sample_common(2, "s2", "s3"),
|
||||
target: TargetDescriptorV3 {
|
||||
element_ref: Some("e12".into()),
|
||||
role: Some("textbox".into()),
|
||||
name: Some("搜索商品".into()),
|
||||
ctx: None,
|
||||
unmatched: false,
|
||||
},
|
||||
value: "browser skill".into(),
|
||||
commit: FillCommit::Enter,
|
||||
redacted: false,
|
||||
};
|
||||
let v = serde_json::to_value(&step).unwrap();
|
||||
assert_eq!(v["op"], "fill");
|
||||
assert_eq!(v["commit"], "enter");
|
||||
assert!(v.get("redacted").is_none());
|
||||
let round: StepV3 = serde_json::from_value(v).unwrap();
|
||||
assert_eq!(round, step);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn step_fill_password_is_redacted() {
|
||||
let step = StepV3::Fill {
|
||||
common: sample_common(1, "s1", "s1"),
|
||||
target: TargetDescriptorV3 {
|
||||
element_ref: Some("e3".into()),
|
||||
role: Some("textbox".into()),
|
||||
name: Some("密码".into()),
|
||||
ctx: None,
|
||||
unmatched: false,
|
||||
},
|
||||
value: "***".into(),
|
||||
commit: FillCommit::Blur,
|
||||
redacted: true,
|
||||
};
|
||||
let v = serde_json::to_value(&step).unwrap();
|
||||
assert_eq!(v["value"], "***");
|
||||
assert_eq!(v["redacted"], true);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn step_navigate_with_cause_round_trips() {
|
||||
let step = StepV3::Navigate {
|
||||
common: sample_common(1, "s1", "s2"),
|
||||
to: "https://example.com".into(),
|
||||
cause: NavigationCause::UserTyped,
|
||||
};
|
||||
let v = serde_json::to_value(&step).unwrap();
|
||||
assert_eq!(v["op"], "navigate");
|
||||
assert_eq!(v["cause"], "user_typed");
|
||||
let round: StepV3 = serde_json::from_value(v).unwrap();
|
||||
assert_eq!(round, step);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn trace_v3_round_trips() {
|
||||
let trace = sample_trace();
|
||||
let v = serde_json::to_value(&trace).unwrap();
|
||||
assert_eq!(v.get("version").and_then(|v| v.as_u64()), Some(3));
|
||||
assert!(v.get("pages").is_none());
|
||||
assert_eq!(v["recorder"]["vom"], 1);
|
||||
assert_eq!(v["states"].as_array().unwrap().len(), 2);
|
||||
let round: TraceV3 = serde_json::from_value(v).unwrap();
|
||||
assert_eq!(round, trace);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn default_fields_are_omitted() {
|
||||
let step = StepV3::Click {
|
||||
common: sample_common(1, "s1", "s1"),
|
||||
target: TargetDescriptorV3 {
|
||||
element_ref: Some("e1".into()),
|
||||
role: Some("button".into()),
|
||||
name: Some("OK".into()),
|
||||
ctx: None,
|
||||
unmatched: false,
|
||||
},
|
||||
};
|
||||
let v = serde_json::to_value(&step).unwrap();
|
||||
assert!(v.get("unmatched").is_none());
|
||||
assert!(v["target"].get("ctx").is_none());
|
||||
assert!(v["target"].get("unmatched").is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn unmatched_target_serializes_flag() {
|
||||
let step = StepV3::Click {
|
||||
common: sample_common(1, "s1", "s2"),
|
||||
target: TargetDescriptorV3 {
|
||||
element_ref: None,
|
||||
role: Some("button".into()),
|
||||
name: Some("发布".into()),
|
||||
ctx: None,
|
||||
unmatched: true,
|
||||
},
|
||||
};
|
||||
let v = serde_json::to_value(&step).unwrap();
|
||||
assert_eq!(v["target"]["unmatched"], true);
|
||||
assert!(v["target"].get("ref").is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn extension_trace_deserializes() {
|
||||
let v = json!({
|
||||
"version": 3,
|
||||
"recorded_at": "2026-07-21T08:00:00Z",
|
||||
"started_at": "2026-07-21T07:59:00Z",
|
||||
"purpose": "demo",
|
||||
"stopped_by": "user_finish",
|
||||
"entry": { "start_url": "https://example.com/editor" },
|
||||
"recorder": { "bsk": "0.1.10", "vom": 1 },
|
||||
"states": [
|
||||
{ "id": "s1", "url": "https://example.com/editor", "body": "@vom 1" },
|
||||
{ "id": "s2", "url": "https://example.com/p/99", "body": "@vom 1" }
|
||||
],
|
||||
"steps": [
|
||||
{
|
||||
"op": "fill",
|
||||
"id": 1,
|
||||
"state": "s1",
|
||||
"result": { "state": "s1" },
|
||||
"target": { "ref": "e1", "role": "textbox", "name": "标题" },
|
||||
"value": "hello",
|
||||
"commit": "blur"
|
||||
},
|
||||
{
|
||||
"op": "click",
|
||||
"id": 2,
|
||||
"state": "s1",
|
||||
"result": { "state": "s2" },
|
||||
"target": { "ref": "e2", "role": "button", "name": "发布" }
|
||||
}
|
||||
]
|
||||
});
|
||||
let trace: TraceV3 = serde_json::from_value(v).unwrap();
|
||||
assert_eq!(trace.version, 3);
|
||||
assert_eq!(trace.states.len(), 2);
|
||||
assert_eq!(trace.steps.len(), 2);
|
||||
}
|
||||
}
|
||||
@@ -143,6 +143,15 @@ flowchart LR
|
||||
- No credential storage in bsk — cookies stay in the user's browser profile.
|
||||
- `evaluate` restricted to Agent Window tabs in sandbox mode.
|
||||
|
||||
### File-transfer boundary
|
||||
|
||||
- The invoking agent/harness decides whether a transfer is authorized and supplies the task-local source or destination path.
|
||||
- The CLI is the only component that reads an upload source or writes the final download destination. Before browser dispatch it owns rollback of partially staged uploads; after dispatch, ownership moves to the session because a transport timeout cannot prove that Chrome did not attach the file. Download output becomes visible through one atomic commit, and replacement is opt-in without a pre-delete window. The extension never receives either agent-facing path.
|
||||
- The daemon is the authority for storage capabilities and limits. It issues opaque session-scoped transfer IDs, stages bounded chunks in a private runtime directory, and injects only private staged upload paths. For download it mints one relative Chrome directory capability. Only after validating the reported path, file type, symlink boundary, and authoritative byte limit does it take ownership of browser-file cleanup and import the bytes.
|
||||
- The extension owns only the browser transaction. Both tools resolve one `ResolvedActionTarget` and use that same target for protocol setup and the triggering click. Upload arms Chrome's chooser interception before clicking, then accepts either an exact `Page.fileChooserOpened` input node or an independent probe anchored in the trigger node's own document; one verified input is committed with `DOM.setFileInputFiles`. A non-input picker is rejected immediately as unsupported. Download correlates exact-target CDP intent and `chrome.downloads` filename candidates in either arrival order, claims only one unique match, and never cancels an unclaimed candidate.
|
||||
- Browser-side operations report `effect_state` (`none`, `committed`, or `unknown`), `phase`, and `cleanup_state`. Confirmed success wins over a late cancel; an unknown effect is preserved across timeout or transport loss and must not be retried blindly. A transfer deadline sends cancellation to the extension and keeps the session queue occupied for bounded compensation rather than abandoning an in-flight browser effect.
|
||||
- Download staging is released after CLI commit. Upload staging remains until session teardown because the page may read an attached file only on a later form submission. Remaining staging is released on session stop/browser disconnect and on daemon startup after a crash. BrowserSkill does not inspect content or decide whether a transfer is appropriate.
|
||||
|
||||
## Repository layout
|
||||
|
||||
```
|
||||
|
||||
@@ -12,7 +12,7 @@
|
||||
* events, and never move the registry's current pointer.
|
||||
*/
|
||||
|
||||
import { createHash, randomUUID } from "node:crypto";
|
||||
import { createHash } from "node:crypto";
|
||||
import { readFile, unlink } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
@@ -78,7 +78,6 @@ const FAILURE_BACKOFF_THRESHOLD = 3;
|
||||
const FRAME_RING_SIZE = 2;
|
||||
|
||||
export class ObservationService {
|
||||
private readonly scratchNamespace = randomUUID();
|
||||
private readonly observations = new Map<string, SessionObservation>();
|
||||
private readonly listeners = new Set<(event: ObservationEvent) => void>();
|
||||
private readonly captureTimers = new Map<string, unknown>();
|
||||
@@ -305,9 +304,9 @@ export class ObservationService {
|
||||
if (current === undefined || current.dead === true) return;
|
||||
if (this.captureInFlight.has(sessionId)) return;
|
||||
this.captureInFlight.add(sessionId);
|
||||
// Stable within this service, isolated from captures owned by other instances.
|
||||
const sessionKey = createHash("sha256").update(sessionId).digest("hex").slice(0, 16);
|
||||
const outPath = join(tmpdir(), `bsk-obs-${this.scratchNamespace}-${sessionKey}.png`);
|
||||
// One fixed scratch path per session (overwritten each frame) and always
|
||||
// unlinked after the bytes are read — /tmp must not grow without bound.
|
||||
const outPath = join(tmpdir(), `bsk-obs-${sessionId}.png`);
|
||||
let writtenPath = outPath;
|
||||
try {
|
||||
const result = await this.deps.queue.run(sessionId, () =>
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
import { existsSync, mkdtempSync, writeFileSync } from "node:fs";
|
||||
import type { IncomingMessage, ServerResponse } from "node:http";
|
||||
import { tmpdir } from "node:os";
|
||||
import { basename, dirname } from "node:path";
|
||||
import { join } from "node:path";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import {
|
||||
actionForLabel,
|
||||
@@ -548,12 +548,8 @@ describe("scratch files and thumbnail references", () => {
|
||||
);
|
||||
const firstOut = runner.calls.filter((c) => c.args[0] === "screenshot").at(-1)?.args;
|
||||
const pathOf = (args: string[]) => args[args.indexOf("--out") + 1];
|
||||
const firstPath = pathOf(firstOut ?? []);
|
||||
expect(dirname(firstPath)).toBe(tmpdir());
|
||||
expect(basename(firstPath)).toMatch(
|
||||
/^bsk-obs-[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}-[0-9a-f]{16}\.png$/,
|
||||
);
|
||||
expect(existsSync(firstPath)).toBe(false);
|
||||
expect(pathOf(firstOut ?? [])).toBe(join(tmpdir(), "bsk-obs-s1.png"));
|
||||
expect(existsSync(pathOf(firstOut ?? []))).toBe(false);
|
||||
// second frame: same path, deleted again
|
||||
service.endAction("s1");
|
||||
scheduler.runNext();
|
||||
@@ -563,33 +559,11 @@ describe("scratch files and thumbnail references", () => {
|
||||
scheduler.pending().length === 1,
|
||||
);
|
||||
const secondOut = runner.calls.filter((c) => c.args[0] === "screenshot").at(-1)?.args;
|
||||
const secondPath = pathOf(secondOut ?? []);
|
||||
expect(secondPath).toBe(firstPath);
|
||||
await waitFor(() => !existsSync(secondPath));
|
||||
expect(pathOf(secondOut ?? [])).toBe(join(tmpdir(), "bsk-obs-s1.png"));
|
||||
await waitFor(() => !existsSync(pathOf(secondOut ?? [])));
|
||||
service.dispose();
|
||||
});
|
||||
|
||||
it("isolates scratch paths between service instances", async () => {
|
||||
const first = setup({ scheduler: fakeScheduler(), runner: fakeRunner() });
|
||||
const second = setup({ scheduler: fakeScheduler(), runner: fakeRunner() });
|
||||
first.service.addSession("s1");
|
||||
second.service.addSession("s1");
|
||||
first.scheduler.runNext();
|
||||
second.scheduler.runNext();
|
||||
await waitFor(
|
||||
() =>
|
||||
first.service.getState()[0]?.thumbnailAttachmentId !== undefined &&
|
||||
second.service.getState()[0]?.thumbnailAttachmentId !== undefined,
|
||||
);
|
||||
const pathOf = (runner: FakeRunner) => {
|
||||
const args = runner.calls.find((call) => call.args[0] === "screenshot")?.args ?? [];
|
||||
return args[args.indexOf("--out") + 1];
|
||||
};
|
||||
expect(pathOf(first.runner)).not.toBe(pathOf(second.runner));
|
||||
first.service.dispose();
|
||||
second.service.dispose();
|
||||
});
|
||||
|
||||
it("keeps the last two distinct frames and evicts the third", async () => {
|
||||
let n = 0;
|
||||
const runner = fakeRunner({
|
||||
|
||||
@@ -24,10 +24,6 @@ function scene(nodes: VomNode[]): VomScene {
|
||||
};
|
||||
}
|
||||
|
||||
function coreRefs(refs: ReturnType<typeof renderVom>["refs"]) {
|
||||
return refs.map(({ ref, backendNodeId }) => ({ ref, backendNodeId }));
|
||||
}
|
||||
|
||||
describe("renderVom single-layer page", () => {
|
||||
it("does not derive handle context across frame scopes", () => {
|
||||
const out = renderVom(
|
||||
@@ -134,7 +130,7 @@ describe("renderVom single-layer page", () => {
|
||||
' @e2 button "加入购物车"',
|
||||
].join("\n"),
|
||||
);
|
||||
expect(coreRefs(out.refs)).toEqual([
|
||||
expect(out.refs).toEqual([
|
||||
{ ref: "e1", backendNodeId: 3 },
|
||||
{ ref: "e2", backendNodeId: 6 },
|
||||
]);
|
||||
@@ -155,7 +151,7 @@ describe("renderVom single-layer page", () => {
|
||||
expect(out.text).toContain(' heading "Title"');
|
||||
expect(out.text).toContain(' img "Logo"');
|
||||
expect(out.text).toContain(' @e1 button "Continue"');
|
||||
expect(coreRefs(out.refs)).toEqual([{ ref: "e1", backendNodeId: 4 }]);
|
||||
expect(out.refs).toEqual([{ ref: "e1", backendNodeId: 4 }]);
|
||||
});
|
||||
|
||||
it("assigns refs to listbox controls", () => {
|
||||
@@ -167,7 +163,7 @@ describe("renderVom single-layer page", () => {
|
||||
);
|
||||
|
||||
expect(out.text).toContain(' @e1 listbox "Choices"');
|
||||
expect(coreRefs(out.refs)).toEqual([{ ref: "e1", backendNodeId: 2 }]);
|
||||
expect(out.refs).toEqual([{ ref: "e1", backendNodeId: 2 }]);
|
||||
});
|
||||
|
||||
it("annotates external links regardless of role casing", () => {
|
||||
@@ -204,7 +200,7 @@ describe("renderVom single-layer page", () => {
|
||||
expect(out.text).toContain(" alertdialog");
|
||||
expect(out.text).toContain(" section");
|
||||
expect(out.text).toContain(' @e1 button "Close"');
|
||||
expect(coreRefs(out.refs)).toEqual([{ ref: "e1", backendNodeId: 5 }]);
|
||||
expect(out.refs).toEqual([{ ref: "e1", backendNodeId: 5 }]);
|
||||
});
|
||||
|
||||
it("skips generic nodes without adding an extra indentation level", () => {
|
||||
@@ -295,7 +291,7 @@ describe("renderVom single-layer page", () => {
|
||||
expect(out.text).toContain(" section");
|
||||
expect(out.text).not.toContain("Too deep");
|
||||
expect(out.text).toContain(' @e1 button "Later"');
|
||||
expect(coreRefs(out.refs)).toEqual([{ ref: "e1", backendNodeId: 4 }]);
|
||||
expect(out.refs).toEqual([{ ref: "e1", backendNodeId: 4 }]);
|
||||
});
|
||||
|
||||
it("truncates when maxTokens is exceeded while keeping refs in sync with rendered text", () => {
|
||||
@@ -330,7 +326,7 @@ describe("renderVom single-layer page", () => {
|
||||
);
|
||||
|
||||
expect(out.text).toContain('@e1 button "Open settings"');
|
||||
expect(coreRefs(out.refs)).toEqual([{ ref: "e1", backendNodeId: 2 }]);
|
||||
expect(out.refs).toEqual([{ ref: "e1", backendNodeId: 2 }]);
|
||||
});
|
||||
|
||||
it("keeps the deepest custom control when wrapper and child both look clickable", () => {
|
||||
@@ -360,7 +356,7 @@ describe("renderVom single-layer page", () => {
|
||||
|
||||
expect(out.text).not.toContain('@e1 button "Card"');
|
||||
expect(out.text).toContain('@e1 button "Details"');
|
||||
expect(coreRefs(out.refs)).toEqual([{ ref: "e1", backendNodeId: 3 }]);
|
||||
expect(out.refs).toEqual([{ ref: "e1", backendNodeId: 3 }]);
|
||||
});
|
||||
|
||||
it("does not recover clickable wrappers around native controls", () => {
|
||||
@@ -383,7 +379,7 @@ describe("renderVom single-layer page", () => {
|
||||
|
||||
expect(out.text).not.toContain('@e1 button "Checkout"');
|
||||
expect(out.text).toContain('@e1 button "Pay"');
|
||||
expect(coreRefs(out.refs)).toEqual([{ ref: "e1", backendNodeId: 3 }]);
|
||||
expect(out.refs).toEqual([{ ref: "e1", backendNodeId: 3 }]);
|
||||
});
|
||||
|
||||
it("adds context to duplicate weak action labels", () => {
|
||||
@@ -446,7 +442,7 @@ describe("renderVom single-layer page", () => {
|
||||
|
||||
expect(out.text).toContain("@layers 1 focus=L1");
|
||||
expect(out.text).toContain('@e1 button "Background"');
|
||||
expect(coreRefs(out.refs)).toEqual([{ ref: "e1", backendNodeId: 2 }]);
|
||||
expect(out.refs).toEqual([{ ref: "e1", backendNodeId: 2 }]);
|
||||
});
|
||||
|
||||
it("can filter refs blocked by active positioned regions without changing layer output", () => {
|
||||
@@ -489,7 +485,7 @@ describe("renderVom single-layer page", () => {
|
||||
expect(out.text).not.toContain("active-region");
|
||||
expect(out.text).not.toContain("Background");
|
||||
expect(out.text).toContain('@e1 button "Foreground"');
|
||||
expect(coreRefs(out.refs)).toEqual([{ ref: "e1", backendNodeId: 4 }]);
|
||||
expect(out.refs).toEqual([{ ref: "e1", backendNodeId: 4 }]);
|
||||
});
|
||||
|
||||
it("keeps paint-order comparisons inside their frame document", () => {
|
||||
@@ -604,7 +600,7 @@ describe("renderVom single-layer page", () => {
|
||||
});
|
||||
|
||||
expect(out.text).toContain('@e1 button "Products" [hover first: Shoes | Bags | Accessories]');
|
||||
expect(coreRefs(out.refs)).toEqual([{ ref: "e1", backendNodeId: 2 }]);
|
||||
expect(out.refs).toEqual([{ ref: "e1", backendNodeId: 2 }]);
|
||||
});
|
||||
|
||||
it("injects active scope blocks immediately after their trigger without refs", () => {
|
||||
@@ -630,7 +626,7 @@ describe("renderVom single-layer page", () => {
|
||||
" Bob - great sound",
|
||||
].join("\n"),
|
||||
);
|
||||
expect(coreRefs(out.refs)).toEqual([{ ref: "e1", backendNodeId: 2 }]);
|
||||
expect(out.refs).toEqual([{ ref: "e1", backendNodeId: 2 }]);
|
||||
});
|
||||
|
||||
it("does not render redundant children inside named ref controls", () => {
|
||||
@@ -644,7 +640,7 @@ describe("renderVom single-layer page", () => {
|
||||
|
||||
expect(out.text).toContain('@e1 button "Save"');
|
||||
expect(out.text).not.toContain("disk icon");
|
||||
expect(coreRefs(out.refs)).toEqual([{ ref: "e1", backendNodeId: 2 }]);
|
||||
expect(out.refs).toEqual([{ ref: "e1", backendNodeId: 2 }]);
|
||||
});
|
||||
|
||||
it("does not render redundant children inside native text inputs", () => {
|
||||
@@ -811,7 +807,7 @@ describe("renderVom double-layer page", () => {
|
||||
].join("\n"),
|
||||
);
|
||||
expect(out.text).not.toContain("底层按钮");
|
||||
expect(coreRefs(out.refs)).toEqual([
|
||||
expect(out.refs).toEqual([
|
||||
{ ref: "e1", backendNodeId: 4 },
|
||||
{ ref: "e2", backendNodeId: 5 },
|
||||
]);
|
||||
@@ -863,36 +859,4 @@ describe("renderVom double-layer page", () => {
|
||||
expect(out.text).toContain(' @e2 textbox "密码" [filled] ="•••"');
|
||||
expect(out.refs.map((r) => r.backendNodeId)).toEqual([101, 102]);
|
||||
});
|
||||
|
||||
it("redacts form values and returns semantic metadata for rendered refs", () => {
|
||||
const out = renderVom(
|
||||
scene([
|
||||
node({ id: 1, role: "RootWebArea", name: "Form" }),
|
||||
node({
|
||||
id: 2,
|
||||
backendNodeId: 42,
|
||||
frameId: "child-frame",
|
||||
parentId: 1,
|
||||
role: "textbox",
|
||||
name: "Email",
|
||||
value: "user@example.com",
|
||||
inputState: "filled",
|
||||
tag: "input",
|
||||
}),
|
||||
]),
|
||||
{ redactValues: true },
|
||||
);
|
||||
|
||||
expect(out.text).toContain('[filled] ="•••"');
|
||||
expect(out.text).not.toContain("user@example.com");
|
||||
expect(out.refs[0]).toMatchObject({
|
||||
ref: "e1",
|
||||
backendNodeId: 42,
|
||||
frameId: "child-frame",
|
||||
role: "textbox",
|
||||
name: "Email",
|
||||
line: expect.any(Number),
|
||||
});
|
||||
expect(out.text.split("\n")[out.refs[0]?.line ?? -1]).toContain('@e1 textbox "Email"');
|
||||
});
|
||||
});
|
||||
|
||||
@@ -5,7 +5,6 @@ export type {
|
||||
CondSurface,
|
||||
LayerKind,
|
||||
Rect,
|
||||
RenderedRef,
|
||||
Viewport,
|
||||
VomNode,
|
||||
VomOptions,
|
||||
|
||||
@@ -4,9 +4,9 @@ import type {
|
||||
BlockingLayer,
|
||||
CondSurface,
|
||||
Rect,
|
||||
RenderedRef,
|
||||
VomNode,
|
||||
VomOptions,
|
||||
VomRef,
|
||||
VomResult,
|
||||
VomScene,
|
||||
} from "./types";
|
||||
@@ -615,7 +615,6 @@ function renderNodeLine(
|
||||
ref: string | undefined,
|
||||
context: string[] = [],
|
||||
surface: CondSurface | undefined = undefined,
|
||||
redactValues = false,
|
||||
): string {
|
||||
let line = `${" ".repeat(depth)}${ref ? `@${ref} ` : ""}${node.role}`;
|
||||
|
||||
@@ -637,14 +636,13 @@ function renderNodeLine(
|
||||
}
|
||||
const placeholder = cleaned(node.placeholder);
|
||||
if (placeholder) line += ` placeholder=${JSON.stringify(placeholder)}`;
|
||||
const hasValue =
|
||||
const sensitiveHasValue =
|
||||
rawValue !== undefined || node.inputState === "filled" || node.inputState === "default";
|
||||
const value =
|
||||
node.sensitive || redactValues
|
||||
? hasValue
|
||||
? SENSITIVE_MASK
|
||||
: undefined
|
||||
: rawValue?.slice(0, MAX_VALUE_LENGTH);
|
||||
const value = node.sensitive
|
||||
? sensitiveHasValue
|
||||
? SENSITIVE_MASK
|
||||
: undefined
|
||||
: rawValue?.slice(0, MAX_VALUE_LENGTH);
|
||||
if (value !== undefined) line += ` =${JSON.stringify(value)}`;
|
||||
|
||||
if (surface && surface.subItems.length > 0) {
|
||||
@@ -697,12 +695,11 @@ function shouldSkipRedundantChildren(node: VomNode, state: RenderState): boolean
|
||||
|
||||
interface RenderState {
|
||||
lines: string[];
|
||||
refs: RenderedRef[];
|
||||
refs: VomRef[];
|
||||
nextRef: number;
|
||||
tokens: number;
|
||||
maxDepth: number;
|
||||
maxTokens: number;
|
||||
redactValues: boolean;
|
||||
truncated: boolean;
|
||||
stopped: boolean;
|
||||
children: Map<number | null, VomNode[]>;
|
||||
@@ -767,7 +764,7 @@ function renderTree(
|
||||
const ref = isVomReferenceNode(node) ? `e${state.nextRef}` : undefined;
|
||||
const context = ref ? handleContext(node, state) : [];
|
||||
const surface = state.surfaceMap.get(node.id);
|
||||
const line = renderNodeLine(node, depth, ref, context, surface, state.redactValues);
|
||||
const line = renderNodeLine(node, depth, ref, context, surface);
|
||||
const nextTokens = state.tokens + estimateTokens(line);
|
||||
if (nextTokens > state.maxTokens) {
|
||||
state.truncated = true;
|
||||
@@ -778,15 +775,10 @@ function renderTree(
|
||||
state.lines.push(line);
|
||||
state.tokens = nextTokens;
|
||||
if (ref) {
|
||||
const name = cleaned(node.name);
|
||||
state.refs.push({
|
||||
ref,
|
||||
backendNodeId: node.backendNodeId ?? node.id,
|
||||
...(node.frameId ? { frameId: node.frameId } : {}),
|
||||
...(node.role ? { role: node.role } : {}),
|
||||
...(name ? { name } : {}),
|
||||
...(context.length > 0 ? { ctx: context.join(" > ") } : {}),
|
||||
line: state.lines.length - 1,
|
||||
});
|
||||
state.nextRef += 1;
|
||||
}
|
||||
@@ -816,7 +808,6 @@ function renderNodes(
|
||||
tokens: estimateTokens(initialLines.join("\n")),
|
||||
maxDepth: options.maxDepth ?? DEFAULT_MAX_DEPTH,
|
||||
maxTokens: options.maxTokens ?? DEFAULT_MAX_TOKENS,
|
||||
redactValues: options.redactValues ?? false,
|
||||
truncated: false,
|
||||
stopped: false,
|
||||
children,
|
||||
|
||||
@@ -64,10 +64,6 @@ export interface VomScene {
|
||||
export interface VomOptions {
|
||||
maxDepth?: number;
|
||||
maxTokens?: number;
|
||||
/**
|
||||
* When true, form values are rendered as masks instead of literal values.
|
||||
*/
|
||||
redactValues?: boolean;
|
||||
/**
|
||||
* Experimental: filter refs that are geometrically blocked by foreground
|
||||
* fixed/absolute/sticky regions. Disabled by default and does not alter the
|
||||
@@ -94,17 +90,9 @@ export interface VomRef {
|
||||
frameId?: string;
|
||||
}
|
||||
|
||||
export interface RenderedRef extends VomRef {
|
||||
role?: string;
|
||||
name?: string;
|
||||
ctx?: string;
|
||||
/** Zero-based line index in `VomResult.text`. */
|
||||
line: number;
|
||||
}
|
||||
|
||||
export interface VomResult {
|
||||
text: string;
|
||||
refs: RenderedRef[];
|
||||
refs: VomRef[];
|
||||
truncated: boolean;
|
||||
}
|
||||
|
||||
|
||||
@@ -206,6 +206,29 @@ Both capture from the moment the tab is attached and read a bounded per-tab buff
|
||||
| `bsk select <ref-or-selector> --value <v>` | Set `<select>` option(s) by `value` (repeat `--value` for multi-select) |
|
||||
| `bsk press <key>` | Key/combo (`Enter`, `Ctrl+A`, …; optional `--ref` to focus first) |
|
||||
|
||||
### File transfer (require `--session`)
|
||||
|
||||
| Command | Summary |
|
||||
|---------|---------|
|
||||
| `bsk upload <ref-or-selector> --file <path>` | Click one upload trigger and attach an agent-readable local file (`--file` is repeatable) |
|
||||
| `bsk download <ref-or-selector> --out <path>` | Click one download trigger and copy the single completed file to an exact local path (`--overwrite` is opt-in) |
|
||||
|
||||
The agent/harness decides whether a file transfer is appropriate and which local path belongs to the task. Treat upload as disclosure of that file to the current website, and download as accepting website-controlled bytes onto the local filesystem. Use only paths that are necessary for the user's bounded goal.
|
||||
|
||||
BrowserSkill enforces the mechanical boundary: files are staged under a session-scoped opaque transfer, only daemon-minted capabilities reach the extension, upload/download still obey Agent Window tab checks, and transfers are chunk/size bounded. Upload intercepts the native chooser for one transaction, locates the input activated in the resolved target's document, and assigns only the staged file paths. Download uniquely correlates one exact-target browser intent with one Chrome download in either event order, routes it through a daemon-minted relative directory, and lets the daemon validate and import it. Upload staging remains available for a later form submission and is removed when the session ends. Downloads cannot overwrite an existing destination unless `--overwrite` is explicit. BrowserSkill does not inspect file content or decide whether its meaning is sensitive.
|
||||
|
||||
Do not use `request-help` merely because a native file chooser or browser download is involved; try these commands first. For transfer failures, use the structured error instead of retrying blindly:
|
||||
|
||||
- `reason=file_input_not_activated` means the requested click did not activate exactly one `<input type="file">`; the page may use a non-input picker such as `window.showOpenFilePicker()`. Do not retry blindly. If human help is available, call `request-help` and tell the user the exact original local path to choose. The staged daemon path is internal and must not be shown to the user.
|
||||
- `reason=file_input_probe_failed` means BrowserSkill could not safely establish the browser-side upload transaction. Do not repeat the same action; use `request-help` when available.
|
||||
- `reason=set_file_input_failed` means BrowserSkill found the activated file input but Chrome rejected the staged path or assignment. Check the extension's file-URL access permission; otherwise use `request-help`.
|
||||
- `reason=download_capture_failed` means BrowserSkill could not attribute exactly one completed download to the requested target. Do not retry blindly or accept an unrelated browser download; use `request-help` when available.
|
||||
- `effect_state=none` means BrowserSkill confirmed that no file-transfer effect was committed. Follow the accompanying reason; a corrected target or explicit human fallback may be attempted.
|
||||
- `effect_state=unknown` means the browser may already have attached or created the file. Do not repeat the transfer. Observe the page if that can establish the result; otherwise stop and report the uncertainty.
|
||||
- `effect_state=committed` means the browser-side effect occurred even if later completion or cleanup failed. Do not repeat it; continue only after verifying the resulting page/download state.
|
||||
|
||||
If `request-help` returns `outcome="disabled"`, do not retry it. Stop gracefully and report the transfer mechanism that requires human intervention.
|
||||
|
||||
### Scripting & timing
|
||||
|
||||
| Command | Summary |
|
||||
|
||||
Reference in New Issue
Block a user