Compare commits

..
Author SHA1 Message Date
Ljy-0827 c08fed5de3 fix(record): saftey policy and rebase bug fix 2026-08-24 10:42:45 +08:00
“haonan” 887d1ed0b9 fix(record): drop 'hover' in tracev2 for compatible 2026-08-24 10:42:45 +08:00
Ljy-0827 1a3d3b7de8 fix(record): bug fix 2026-08-24 10:42:45 +08:00
“haonan”andCursor 72834980d2 fix(record): record switch_tab and keep Trace v3 states per tab
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-24 10:42:45 +08:00
“haonan” 38b0debba9 fix(test): share one BSK_HOME lock across lib tests 2026-08-24 10:40:37 +08:00
“haonan” 4677149e5e fix(record): recover failed Trace exports and hint --url when the default start page fails
Reject legacy JSON --output before recording starts, persist the completed Trace before writing the bundle so `bsk record stop --output <dir>` can retry after an export failure, and tell users to pass --url when https://example.com/ does not load.
2026-08-23 21:26:40 +08:00
Ljy-0827 8ea5477e11 fix(record): safety problem and bug fix 2026-08-23 21:26:40 +08:00
“haonan” 17af1af65a feat(record): export Trace v3 bundles with settled page states 2026-08-23 21:26:40 +08:00
Ljy-0827 6300eda0aa fix(record): rebase and dto policy 2026-08-23 21:26:40 +08:00
Ljy-0827 87cdcfe5ca fix(recording): using unified geometry 2026-08-23 21:26:40 +08:00
“haonan”andCursor 21476c3884 feat(record): capture Trace v3 states after pages settle
Wait for DOM quiet after actions, match targets to VOM refs, and reduce
drafts into deduped state-linked Trace v3 while keeping the v2 reducer.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-23 21:26:40 +08:00
“haonan” c2a225377b fix(protocol): align trace schemas with v2/v3 classify rules
RecordedTrace validation now follows classify_value — numeric version
selects v3, otherwise v2 — so mixed pages/states and v2 envelopes with
a version field fail Schema checks before Recall import. Keep standalone
trace_step as a v2|v3 union for tools that validate a single step.
2026-08-23 21:26:39 +08:00
“haonan” e64f4cac9f fix(protocol): keep generic trace schema compatible with v2 and v3
Split standalone v2/v3 schemas and drop TraceV2 deny_unknown_fields so legacy traces with extra keys still parse; mix and version checks stay in RecordedTrace.
2026-08-23 21:26:39 +08:00
Ljy-0827 368507b07d fix(protocol): compatible with dto 2026-08-23 21:26:39 +08:00
Ljy-0827 b750eed5d4 fix(protocol): integrate with vom 2026-08-23 21:26:39 +08:00
“haonan” 3d159c0254 test(protocol): align handshake fixtures with 1.0 floor and clarify v2/v3 naming 2026-08-23 21:26:39 +08:00
“haonan”andCursor 524516cb56 fix(protocol): align Trace v2/v3 classification and hover steps
Require Trace v3 version const 3, reject mixed pages/states payloads, and
align TypeScript selection optionality with the generated schema. Trace v2
on main already records hover steps, so the frozen v2 schema must accept
them instead of dropping them during reduction.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-23 21:26:39 +08:00
“haonan”andCursor f98198949a feat(protocol): negotiate Trace v3 with v2 compatibility
Define Trace v3 wire types and generated schemas while keeping protocol
1.1 peers compatible with 1.0. Default record start still produces Trace
v2, and only an explicit trace_version=3 request enables the new shape.
Refs #90

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-23 21:26:39 +08:00
Zhang GH d05279f991 Merge pull request #98 from NianJiuZst/codex/fix-daemon-idle-close-race
fix(daemon): make idle disconnect updates atomic
2026-08-23 21:25:24 +08:00
Zhang GH 2719e76438 Merge pull request #97 from NianJiuZst/codex/cancel-session-lifecycle-rpcs
fix(session): cancel lifecycle RPCs cooperatively
2026-08-22 14:51:38 +08:00
polarday dd9fc9edba Merge pull request #100 from Tencent/feat/record-vom-observation
feat(vom): expose record-safe semantic observations
2026-08-21 16:21:41 +08:00
NianJiuZst 8352c3c67a fix(daemon): refresh stale idle-task comment
After folding the IPC connection count and activity timestamp into a
single mutex-protected IpcActivityState, the idle task no longer
references the connection count directly; should_exit_for_idle already
returns false while a connection is open or within the idle window.
Update the comment to describe the post-refactor control flow so it
matches the code.
2026-08-20 13:53:23 +08:00
NianJiuZst 4eedd98c1d fix(daemon): make idle disconnect updates atomic 2026-08-20 13:53:23 +08:00
NianJiuZst 7035e46024 fix(session): cancel lifecycle RPCs cooperatively 2026-08-17 15:13:53 +08:00
84 changed files with 13824 additions and 1510 deletions
@@ -1,10 +1,12 @@
import { beforeEach, describe, expect, it, vi } from "vitest";
import { RECORD_STOP, type RecordStepPayload } from "@/lib/record-bridge";
import { RECORD_START, RECORD_STOP, type RecordStepPayload } from "@/lib/record-bridge";
import { handleRecordContentMessage, startRecordCapture } from "../record-capture";
vi.stubGlobal("chrome", {
runtime: {
sendMessage: vi.fn(() => Promise.resolve()),
sendMessage: vi.fn((message: { sequence?: number }) =>
Promise.resolve({ ok: true, sequence: message.sequence }),
),
},
});
@@ -96,6 +98,66 @@ describe("handleRecordContentMessage stop/cancel", () => {
expect(dispose).not.toHaveBeenCalled();
expect(onStop).not.toHaveBeenCalled();
});
it("keeps a failed STOP retryable and redelivers its recorded step", async () => {
document.body.innerHTML = `<label for="retry">Draft</label><input id="retry" />`;
const sendMessage = vi.mocked(chrome.runtime.sendMessage);
sendMessage.mockReset();
sendMessage.mockRejectedValueOnce(new Error("service worker unavailable"));
sendMessage.mockRejectedValueOnce(new Error("service worker still unavailable"));
sendMessage.mockResolvedValueOnce({ ok: true, sequence: 1 });
let activeRequestId: string | null = null;
let capture: ReturnType<typeof startRecordCapture> | null = null;
const onStop = vi.fn();
const dispatch = (
message:
| { type: typeof RECORD_START; requestId: string; startedAtMs?: number }
| { type: typeof RECORD_STOP; requestId: string },
sendResponse?: (response: unknown) => void,
) =>
handleRecordContentMessage(
message,
{
activeRequestId,
capture,
setActiveRequestId: (id) => {
activeRequestId = id;
},
setCapture: (next) => {
capture = next;
},
onStart: vi.fn(),
onStop,
},
sendResponse,
);
dispatch({ type: RECORD_START, requestId: "rec-retry" });
const input = document.querySelector("input")!;
input.dispatchEvent(new FocusEvent("focusin", { bubbles: true }));
input.value = "dirty final value";
input.dispatchEvent(new Event("input", { bubbles: true }));
const firstResponse = vi.fn();
expect(dispatch({ type: RECORD_STOP, requestId: "rec-retry" }, firstResponse)).toBe(true);
await vi.waitFor(() =>
expect(firstResponse).toHaveBeenCalledWith({
ok: false,
error: "failed to deliver one or more recorded steps",
}),
);
expect(activeRequestId).toBe("rec-retry");
expect(onStop).not.toHaveBeenCalled();
const retryResponse = vi.fn();
expect(dispatch({ type: RECORD_STOP, requestId: "rec-retry" }, retryResponse)).toBe(true);
await vi.waitFor(() => expect(retryResponse).toHaveBeenCalledWith({ ok: true }));
expect(sendMessage).toHaveBeenCalledTimes(3);
expect(activeRequestId).toBeNull();
expect(onStop).toHaveBeenCalledTimes(1);
});
});
describe("record-capture semantic", () => {
@@ -228,6 +290,7 @@ describe("record-capture semantic", () => {
expect(steps[0]).toMatchObject({
op: "hover",
target: { role: "button", name: "Open user navigation menu" },
geometry: { tag: "button", rect: { x: 900, y: 8, w: 32, h: 32 } },
});
expect(steps[1]).toMatchObject({
op: "click",
@@ -0,0 +1,36 @@
import { describe, expect, it, vi } from "vitest";
import type { RecordStepMessage } from "@/lib/record-bridge";
import { RecordStepDelivery } from "../record-step-delivery";
describe("RecordStepDelivery", () => {
it("retries from the first unacknowledged sequence before sending later steps", async () => {
const sent: RecordStepMessage[] = [];
const send = vi.fn(async (message: RecordStepMessage) => {
sent.push(message);
if (sent.length === 1) throw new Error("service worker unavailable");
return { ok: true, sequence: message.sequence };
});
const delivery = new RecordStepDelivery("rec-ordered", send, "document-1");
delivery.enqueue({ op: "click", target: { tag: "button", name: "First" } });
delivery.enqueue({ op: "click", target: { tag: "button", name: "Second" } });
await expect(delivery.flush()).resolves.toBe(true);
expect(sent.map((message) => message.sequence)).toEqual([1, 1, 2]);
expect(sent.map((message) => message.step.target?.name)).toEqual(["First", "First", "Second"]);
});
it("keeps unacknowledged steps pending for a later flush", async () => {
const send = vi
.fn<(message: RecordStepMessage) => Promise<unknown>>()
.mockRejectedValueOnce(new Error("offline"))
.mockRejectedValueOnce(new Error("still offline"))
.mockImplementation(async (message) => ({ ok: true, sequence: message.sequence }));
const delivery = new RecordStepDelivery("rec-retry", send, "document-1");
delivery.enqueue({ op: "click", target: { tag: "button", name: "Save" } });
await expect(delivery.flush()).resolves.toBe(false);
await expect(delivery.flush()).resolves.toBe(true);
expect(send).toHaveBeenCalledTimes(3);
});
});
+80 -53
View File
@@ -1,9 +1,9 @@
import {
type CaptureTargetDescriptor,
describeEventTarget,
describeTarget,
resolveClickableElement,
resolveHoverElement,
type TargetDescriptor,
} from "@/lib/describe-target";
import {
evaluateHoverTrigger,
@@ -27,7 +27,7 @@ import {
type RecordStopAck,
type RecordStopMessage,
} from "@/lib/record-bridge";
import { shouldRecordPress } from "@/lib/trace-reducer";
import { shouldRecordPress } from "@/lib/recording/draft-policy";
import {
closestHoverSurfaceCandidate,
collectHoverSurfaceStates,
@@ -36,27 +36,17 @@ import {
isHoverSurfaceCandidateElement,
isLikelyHoverSurfaceOwner,
} from "./record-hover-surface";
import { RecordStepDelivery } from "./record-step-delivery";
const pendingStepSends = new Map<string, Set<Promise<boolean>>>();
const failedStepDeliveries = new Set<string>();
const knownRecordRequests = new Set<string>();
const stepDeliveries = new Map<string, RecordStepDelivery>();
const pendingStopFlushes = new Map<string, Promise<RecordStopAck>>();
function sendRecordStep(requestId: string, step: RecordStepPayload): void {
const payload = { type: RECORD_STEP, requestId, step };
const pending = Promise.resolve(chrome.runtime.sendMessage(payload)).then(
() => true,
() => {
failedStepDeliveries.add(requestId);
return false;
},
);
const sends = pendingStepSends.get(requestId) ?? new Set<Promise<boolean>>();
sends.add(pending);
pendingStepSends.set(requestId, sends);
void pending.then(() => {
sends.delete(pending);
if (sends.size === 0) pendingStepSends.delete(requestId);
});
function deliveryFor(requestId: string): RecordStepDelivery {
const existing = stepDeliveries.get(requestId);
if (existing) return existing;
const created = new RecordStepDelivery(requestId);
stepDeliveries.set(requestId, created);
return created;
}
export interface RecordCaptureController {
@@ -65,14 +55,15 @@ export interface RecordCaptureController {
interface FillSession {
element: FillableElement;
target: TargetDescriptor;
target: CaptureTargetDescriptor;
baselineValue: string;
lastValue: string;
pendingCommit?: "enter" | "suggestion" | "blur";
}
interface HoverCandidate {
element: Element;
target: TargetDescriptor;
target: CaptureTargetDescriptor;
recordedAt: number;
score: number;
eligible: boolean;
@@ -157,6 +148,24 @@ function nearbyFillableFromSearchChrome(target: Element): FillableElement | null
return null;
}
function captureGeometry(el: Element): RecordStepPayload["geometry"] {
const rect = el.getBoundingClientRect();
return {
rect: { x: rect.x, y: rect.y, w: rect.width, h: rect.height },
tag: el.tagName.toLowerCase(),
};
}
function geometryForEventTarget(
target: EventTarget | null,
): RecordStepPayload["geometry"] | undefined {
if (!(target instanceof Element)) return undefined;
const clickable = target.closest(
'a,button,input,select,textarea,[role="button"],[role="link"],[role="menuitem"],[contenteditable="true"]',
);
return captureGeometry(clickable ?? target);
}
function fillableValue(el: FillableElement): string {
if (el instanceof HTMLInputElement || el instanceof HTMLTextAreaElement) {
return el.value;
@@ -232,7 +241,10 @@ function collectHoverTriggerLabelText(root: Element): string {
return normalizeLabelText(text);
}
function compactHoverTargetName(el: Element, desc: TargetDescriptor): TargetDescriptor {
function compactHoverTargetName(
el: Element,
desc: CaptureTargetDescriptor,
): CaptureTargetDescriptor {
if (!desc.name) return desc;
const fullText = normalizeLabelText(el.textContent ?? "");
const compactName = collectHoverTriggerLabelText(el);
@@ -250,7 +262,7 @@ function compactHoverTargetName(el: Element, desc: TargetDescriptor): TargetDesc
return desc;
}
function isWeakHoverTarget(target: TargetDescriptor): boolean {
function isWeakHoverTarget(target: CaptureTargetDescriptor): boolean {
return !target.role && !target.name && target.tag === "div";
}
@@ -261,9 +273,9 @@ function looksLikeAvatarElement(el: Element): boolean {
function normalizeHoverTarget(
el: Element,
desc: TargetDescriptor,
desc: CaptureTargetDescriptor,
decision: HoverTriggerDecision,
): TargetDescriptor {
): CaptureTargetDescriptor {
if (desc.role === "img" && !desc.name && looksLikeAvatarElement(el)) {
return { ...desc, name: "image" };
}
@@ -277,7 +289,7 @@ function normalizeHoverTarget(
return desc;
}
function hoverTriggerSignals(el: Element, desc: TargetDescriptor) {
function hoverTriggerSignals(el: Element, desc: CaptureTargetDescriptor) {
if (!(el instanceof HTMLElement)) return null;
const style = hoverTriggerStyle(el);
return {
@@ -411,14 +423,17 @@ export function startRecordCapture(
emitStep({
op: "fill",
target: session.target,
geometry: captureGeometry(session.element),
value,
commit: session.pendingCommit ?? "blur",
...(isPassword ? { redacted: true } : {}),
});
};
const commitFillSession = () => {
const commitFillSession = (commit: "enter" | "suggestion" | "blur" = "blur") => {
if (!fillSession || composing) return;
const session = fillSession;
session.pendingCommit = commit;
fillSession = null;
emitFill(session);
committedValues.set(session.element, session.lastValue);
@@ -624,6 +639,7 @@ export function startRecordCapture(
emitStep({
op: "hover",
target: hover.target,
geometry: captureGeometry(hover.element),
});
emittedHoverElements.add(hover.element);
};
@@ -678,6 +694,7 @@ export function startRecordCapture(
emitStep({
op: "click",
target,
geometry: geometryForEventTarget(eventTarget(event)),
expects_navigation: true,
});
};
@@ -779,7 +796,7 @@ export function startRecordCapture(
scheduleInputCompletionCommit(
sessionElement,
syncFillSessionValue,
commitFillSession,
() => commitFillSession("suggestion"),
(el) => fillSession?.element === el,
);
return;
@@ -839,6 +856,7 @@ export function startRecordCapture(
emitStep({
op: "select",
target: desc,
geometry: captureGeometry(target),
values,
labels,
expects_navigation: true,
@@ -879,7 +897,7 @@ export function startRecordCapture(
};
}
if (fillable) {
commitFillSession();
commitFillSession(event.key === "Enter" ? "enter" : "blur");
}
const desc = describeEventTarget(target);
if (!desc && !event.key) return;
@@ -887,7 +905,7 @@ export function startRecordCapture(
emitStep({
op: "press",
key: event.key,
...(desc ? { target: desc } : {}),
...(desc ? { target: desc, geometry: geometryForEventTarget(target) } : {}),
...(modifiers.length ? { modifiers } : {}),
expects_navigation: event.key === "Enter",
});
@@ -906,9 +924,12 @@ export function startRecordCapture(
const urlObserver = new MutationObserver(() => emitNavigateIfChanged());
urlObserver.observe(document, { subtree: true, childList: true });
const onUrlEvent = () => emitNavigateIfChanged();
// Wrap: passing commitFillSession directly would forward the DOM event as
// the `commit` argument and stamp it onto the recorded fill step.
const onPageHide = () => commitFillSession();
window.addEventListener("hashchange", onUrlEvent);
window.addEventListener("popstate", onUrlEvent);
window.addEventListener("pagehide", commitFillSession);
window.addEventListener("pagehide", onPageHide);
const originalPushState = history.pushState;
const originalReplaceState = history.replaceState;
@@ -936,7 +957,7 @@ export function startRecordCapture(
urlObserver.disconnect();
window.removeEventListener("hashchange", onUrlEvent);
window.removeEventListener("popstate", onUrlEvent);
window.removeEventListener("pagehide", commitFillSession);
window.removeEventListener("pagehide", onPageHide);
history.pushState = originalPushState;
history.replaceState = originalReplaceState;
},
@@ -962,14 +983,11 @@ export function handleRecordContentMessage(
sendResponse?: (response: RecordStartAck | RecordStopAck) => void,
): boolean {
if (isRecordStartMessage(message)) {
if (!knownRecordRequests.has(message.requestId)) {
knownRecordRequests.add(message.requestId);
failedStepDeliveries.delete(message.requestId);
}
const delivery = deliveryFor(message.requestId);
state.capture?.dispose();
state.setCapture(
startRecordCapture(message.requestId, (step) => {
sendRecordStep(message.requestId, step);
delivery.enqueue(step);
}),
);
state.setActiveRequestId(message.requestId);
@@ -992,28 +1010,37 @@ export function handleRecordContentMessage(
state.setActiveRequestId(null);
};
if (isRecordStopMessage(message) && sendResponse) {
const pending = [...(pendingStepSends.get(message.requestId) ?? [])];
void Promise.all(pending).then((delivered) => {
finishStop();
const succeeded = delivered.every(Boolean) && !failedStepDeliveries.has(message.requestId);
if (succeeded) {
failedStepDeliveries.delete(message.requestId);
knownRecordRequests.delete(message.requestId);
}
sendResponse(
succeeded
const existingFlush = pendingStopFlushes.get(message.requestId);
if (existingFlush) {
void existingFlush.then(sendResponse);
return true;
}
const flush = deliveryFor(message.requestId)
.flush()
.then((succeeded): RecordStopAck => {
if (succeeded) {
stepDeliveries.delete(message.requestId);
finishStop();
}
return succeeded
? { ok: true }
: {
ok: false,
error: "failed to deliver one or more recorded steps",
},
);
};
});
pendingStopFlushes.set(message.requestId, flush);
void flush.then(sendResponse).finally(() => {
if (pendingStopFlushes.get(message.requestId) === flush) {
pendingStopFlushes.delete(message.requestId);
}
});
return true;
}
if (isRecordCancelMessage(message)) {
failedStepDeliveries.delete(message.requestId);
knownRecordRequests.delete(message.requestId);
stepDeliveries.delete(message.requestId);
pendingStopFlushes.delete(message.requestId);
}
finishStop();
return false;
@@ -0,0 +1,65 @@
import {
isAcceptedRecordStepAck,
RECORD_STEP,
type RecordStepMessage,
type RecordStepPayload,
} from "@/lib/record-bridge";
export type SendRecordStepMessage = (message: RecordStepMessage) => Promise<unknown>;
export class RecordStepDelivery {
readonly #requestId: string;
readonly #producerId: string;
readonly #send: SendRecordStepMessage;
readonly #pending = new Map<number, RecordStepPayload>();
#nextSequence = 1;
#tail = Promise.resolve();
constructor(
requestId: string,
send: SendRecordStepMessage = (message) => chrome.runtime.sendMessage(message),
producerId: string = crypto.randomUUID(),
) {
this.#requestId = requestId;
this.#producerId = producerId;
this.#send = send;
}
enqueue(step: RecordStepPayload): void {
const sequence = this.#nextSequence;
this.#nextSequence += 1;
this.#pending.set(sequence, step);
void this.#schedule();
}
async flush(): Promise<boolean> {
await this.#tail;
await this.#schedule();
return this.#pending.size === 0;
}
#schedule(): Promise<void> {
const drain = () => this.#drain();
this.#tail = this.#tail.then(drain, drain);
return this.#tail;
}
async #drain(): Promise<void> {
for (const [sequence, step] of this.#pending) {
const message: RecordStepMessage = {
type: RECORD_STEP,
requestId: this.#requestId,
producerId: this.#producerId,
sequence,
step,
};
try {
const response = await this.#send(message);
if (!isAcceptedRecordStepAck(response, sequence)) return;
this.#pending.delete(sequence);
} catch {
return;
}
}
}
}
+2 -1
View File
@@ -172,6 +172,7 @@ export default defineBackground(() => {
dispatcher.start();
const recordDeps = {
tabsApi: chrome.tabs,
cdp,
sendToTab: (tabId: number, msg: Parameters<typeof chrome.tabs.sendMessage>[1]) =>
chrome.tabs.sendMessage(tabId, msg),
bypassOverlay: async (tabId: number, enabled: boolean) => {
@@ -188,7 +189,7 @@ export default defineBackground(() => {
// Message listeners stay up (cheap; fire only for record message types).
// Tab / webNavigation observation attaches lazily while a recording is
// active — see ensureBrowserObservationListeners in tools/record.ts.
attachRecordStepListener();
attachRecordStepListener(recordDeps);
attachRecordFinishListener(recordDeps);
attachRecordQueryListener(recordDeps);
if (typeof chrome.notifications?.onClicked?.addListener === "function") {
@@ -200,6 +200,9 @@ describe("App", () => {
expect(copied).toContain("--browser 03c3e47f");
expect(copied).toContain('--purpose "发布 wiki 文档"');
expect(copied).not.toMatch(/bsk record start[^\n]*--url/);
expect(copied).toContain("./trace");
expect(copied).toContain("trace.json");
expect(copied).toContain("states/");
// Button label stays static; a transient toast confirms the copy.
expect(copyButton.textContent).toContain("复制录制指令");
await waitFor(() => expect(screen.getByRole("status").textContent).toContain("已复制"));
@@ -26,20 +26,20 @@ function handshake(
}
describe("computeConnectedState (protocol-based compat)", () => {
it("returns connected when protocol strings match", () => {
expect(computeConnectedState(handshake("1.0", "1.0"), MIN_COMPATIBLE_PROTOCOL)).toEqual({
it("returns connected when daemon protocol equals extension protocol", () => {
expect(computeConnectedState(handshake("1.1", "1.0"), MIN_COMPATIBLE_PROTOCOL)).toEqual({
kind: "connected",
});
});
it("returns version_skew when daemon protocol minor is newer", () => {
expect(computeConnectedState(handshake("1.1", "1.0"))).toEqual({
expect(computeConnectedState(handshake("1.2", "1.0"))).toEqual({
kind: "version_skew",
});
});
it("returns version_skew when daemon protocol string differs but floor is satisfied", () => {
expect(computeConnectedState(handshake("1", "1.0"))).toEqual({
expect(computeConnectedState(handshake("1.1.0", "1.0"))).toEqual({
kind: "version_skew",
});
});
@@ -53,7 +53,7 @@ describe("computeConnectedState (protocol-based compat)", () => {
});
it("rejects when extension is below daemon min_compatible_protocol", () => {
const result = computeConnectedState(handshake("1.0", "1.5"));
const result = computeConnectedState(handshake("1.1", "1.5"));
expect(result.kind).toBe("rejected");
if (result.kind === "rejected") {
expect(result.reason).toContain("min_compatible_protocol");
@@ -65,7 +65,7 @@ describe("computeConnectedState (protocol-based compat)", () => {
const result = computeConnectedState({
server: "browser-skill-daemon",
version: "0.1.0",
protocol_version: "1.0",
protocol_version: "1.1",
min_compatible_peer: "0.1.0",
});
expect(result).toEqual({ kind: "connected" });
@@ -80,7 +80,7 @@ describe("computeConnectedState (protocol-based compat)", () => {
});
it("rejects malformed daemon min_compatible_protocol with a daemon-floor reason", () => {
const result = computeConnectedState(handshake("1.0", "not-a-protocol"));
const result = computeConnectedState(handshake("1.1", "not-a-protocol"));
expect(result.kind).toBe("rejected");
if (result.kind === "rejected") {
expect(result.reason).toContain("daemon min_compatible_protocol");
@@ -242,11 +242,11 @@ describe("ConnectionController connectionEnabled", () => {
const second = transport.send.mock.calls[1]?.[0] as { id: string };
expect(second.id).not.toBe(first.id);
transport.emitMessage({ id: first.id, result: handshake("1.0", "1.0") });
transport.emitMessage({ id: first.id, result: handshake("1.1", "1.0") });
await Promise.resolve();
expect(controller.snapshot().state).not.toBe("connected");
transport.emitMessage({ id: second.id, result: handshake("1.0", "1.0") });
transport.emitMessage({ id: second.id, result: handshake("1.1", "1.0") });
await vi.waitFor(() => expect(controller.snapshot().state).toBe("connected"));
});
});
@@ -0,0 +1,101 @@
import { afterEach, describe, expect, it, vi } from "vitest";
import type { CdpRunner } from "@/tools/shared";
import { waitForDocumentSettled } from "../recording/document-settle";
function quietCdp(): { cdp: CdpRunner; send: ReturnType<typeof vi.fn> } {
const send = vi.fn(async () => ({
result: { value: { idleMs: 1_000, readyState: "complete" } },
}));
return {
cdp: { send: send as unknown as CdpRunner["send"] },
send,
};
}
describe("waitForDocumentSettled", () => {
afterEach(() => {
vi.useRealTimers();
});
it("waits through the minimum observation floor before accepting a quiet page", async () => {
vi.useFakeTimers();
const { cdp, send } = quietCdp();
const settled = waitForDocumentSettled(cdp, { target: { tabId: 7 } });
await vi.advanceTimersByTimeAsync(180);
await expect(settled).resolves.toBe("quiet");
expect(send).toHaveBeenCalledTimes(3);
});
it("cancels before probing a superseded observation", async () => {
vi.useFakeTimers();
const { cdp, send } = quietCdp();
const controller = new AbortController();
const settled = waitForDocumentSettled(
cdp,
{ target: { tabId: 7 } },
{
signal: controller.signal,
},
);
controller.abort();
await vi.advanceTimersByTimeAsync(60);
await expect(settled).resolves.toBe("cancelled");
expect(send).not.toHaveBeenCalled();
});
it("probes a same-process iframe through its isolated execution context", async () => {
vi.useFakeTimers();
const send = vi.fn(async (_tabId: number, method: string) => {
if (method === "Page.createIsolatedWorld") return { executionContextId: 91 };
return { result: { value: { idleMs: 1_000, readyState: "complete" } } };
});
const cdp: CdpRunner = { send: send as unknown as CdpRunner["send"] };
const settled = waitForDocumentSettled(cdp, {
target: { tabId: 7 },
frameId: "child-frame",
});
await vi.advanceTimersByTimeAsync(180);
await expect(settled).resolves.toBe("quiet");
expect(send).toHaveBeenCalledWith(
7,
"Page.createIsolatedWorld",
expect.objectContaining({ frameId: "child-frame" }),
);
expect(send).toHaveBeenCalledWith(
7,
"Runtime.evaluate",
expect.objectContaining({ contextId: 91 }),
);
});
it("routes an OOPIF probe through its CDP target session", async () => {
vi.useFakeTimers();
const send = vi.fn();
const sendToTarget = vi.fn(async (_target, method: string) => {
if (method === "Page.createIsolatedWorld") return { executionContextId: 27 };
return { result: { value: { idleMs: 1_000, readyState: "complete" } } };
});
const cdp: CdpRunner = {
send: send as unknown as CdpRunner["send"],
sendToTarget: sendToTarget as unknown as NonNullable<CdpRunner["sendToTarget"]>,
};
const target = { tabId: 7, sessionId: "oopif-session" };
const settled = waitForDocumentSettled(cdp, { target, frameId: "oopif-frame" });
await vi.advanceTimersByTimeAsync(180);
await expect(settled).resolves.toBe("quiet");
expect(send).not.toHaveBeenCalled();
expect(sendToTarget).toHaveBeenCalledWith(
target,
"Runtime.evaluate",
expect.objectContaining({ contextId: 27 }),
);
});
});
@@ -0,0 +1,32 @@
import { describe, expect, it } from "vitest";
import {
isAcceptedRecordStepAck,
isRecordStepMessage,
RECORD_STEP,
type RecordStepMessage,
} from "../record-bridge";
const message: RecordStepMessage = {
type: RECORD_STEP,
requestId: "rec-1",
producerId: "document-1",
sequence: 1,
step: { op: "click", target: { tag: "button", name: "Submit" } },
};
describe("record step delivery protocol", () => {
it("requires a producer and a positive integer sequence", () => {
expect(isRecordStepMessage(message)).toBe(true);
expect(isRecordStepMessage({ ...message, producerId: "" })).toBe(false);
expect(isRecordStepMessage({ ...message, sequence: 0 })).toBe(false);
expect(isRecordStepMessage({ ...message, sequence: 1.5 })).toBe(false);
});
it("accepts only the acknowledgement for the sent sequence", () => {
expect(isAcceptedRecordStepAck({ ok: true, sequence: 2 }, 2)).toBe(true);
expect(isAcceptedRecordStepAck({ ok: true, sequence: 1 }, 2)).toBe(false);
expect(isAcceptedRecordStepAck({ ok: false, expectedSequence: 2, error: "gap" }, 2)).toBe(
false,
);
});
});
@@ -0,0 +1,146 @@
import { describe, expect, it } from "vitest";
import { ObservationNodeIndex } from "../recording/observation-capture";
import { RecordingObservationSession } from "../recording/observation-session";
import { RecordingStateRegistry } from "../recording/state-registry";
import { buildTraceV3 } from "../recording/trace-builder-v3";
import type { RecordingDraftStep } from "../recording/types";
const URL = "https://example.com/login";
function sessionWithInput(redactValues = false): RecordingObservationSession {
const session = new RecordingObservationSession({ redactValues });
const state = session.registry.register({
url: URL,
vomText: '@vom 1\ntextbox "Password" value="••••••" [ref=e1]',
});
session.cursor.lastSettled = {
stateId: state.id,
rootFrameId: "root",
index: new ObservationNodeIndex({
rootFrameId: "root",
matchNodes: [
{
frameId: "root",
backendNodeId: 42,
tag: "input",
rect: { x: 20, y: 40, w: 200, h: 30 },
localRect: { x: 20, y: 40, w: 200, h: 30 },
},
],
refs: [{ ref: "e1", backendNodeId: 42, role: "textbox", name: "Password", line: 1 }],
}),
url: URL,
};
return session;
}
function finalizedFillTrace(value: string, redactValues: boolean) {
const session = sessionWithInput(redactValues);
const draft: RecordingDraftStep = {
op: "fill",
captureTarget: { tag: "input", role: "textbox", name: "Password" },
value,
targetHint: {
geometry: { rect: { x: 20, y: 40, w: 200, h: 30 }, tag: "input" },
},
};
session.bindDraft(draft, 1);
draft.postStateId = draft.preStateId;
return buildTraceV3({
registry: session.registry,
drafts: [draft],
annotations: session.annotations,
startedAt: "2026-08-12T00:00:00.000Z",
stoppedBy: "user_finish",
bskVersion: "test",
redactValues,
});
}
function finalizedFillBody(value: string, redactValues: boolean): string {
return finalizedFillTrace(value, redactValues).states[0]!.body;
}
describe("record observation annotations", () => {
it("omits a fill literal when values are redacted", () => {
const secret = "hunter2-private";
const dumped = JSON.stringify(finalizedFillTrace(secret, true));
expect(dumped).toContain("step 1: fill");
expect(dumped).not.toContain(secret);
});
it("keeps ordinary fill details", () => {
expect(finalizedFillBody("ordinary text", false)).toContain('step 1: fill: "ordinary text"');
});
it("encodes title and URL so line breaks cannot corrupt state metadata", () => {
const registry = new RecordingStateRegistry();
const state = registry.register({
url: "https://example.com/a\nb",
title: "hello\nworld",
vomText: "@vom 1",
});
const trace = buildTraceV3({
registry,
drafts: [{ op: "scroll", preStateId: state.id, postStateId: state.id }],
startedAt: "2026-08-12T00:00:00.000Z",
stoppedBy: "user_finish",
bskVersion: "test",
});
expect(trace.states[0]?.body).toContain('url: "https://example.com/a\\nb"');
expect(trace.states[0]?.body).toContain('title: "hello\\nworld"');
});
});
describe("recording state ownership", () => {
it("deduplicates within one recording and isolates ids between recordings", () => {
const first = new RecordingStateRegistry();
const second = new RecordingStateRegistry();
expect(first.register({ url: URL, vomText: "same" }).id).toBe("s1");
expect(first.register({ url: URL, vomText: "same" }).id).toBe("s1");
expect(second.register({ url: URL, vomText: "other" }).id).toBe("s1");
});
it("enriches metadata when a deduplicated observation becomes more complete", () => {
const registry = new RecordingStateRegistry();
registry.register({ url: URL, vomText: "same" });
const state = registry.register({
url: URL,
title: "Login",
vomText: "same",
truncated: true,
});
expect(state).toMatchObject({ id: "s1", title: "Login", truncated: true });
});
});
describe("draft binding", () => {
it("keeps capture semantics when no observation exists", () => {
const session = new RecordingObservationSession();
const draft: RecordingDraftStep = {
op: "hover",
captureTarget: { tag: "button", role: "button", name: "新建" },
};
session.bindDraft(draft, 1);
expect(draft.matchedTarget).toEqual({ role: "button", name: "新建", unmatched: true });
expect(draft.preStateId).toBeUndefined();
});
it("does not bind an unmatched new action to a stale observation", () => {
const session = sessionWithInput();
const draft: RecordingDraftStep = {
op: "click",
captureTarget: { tag: "button", role: "button", name: "Confirm" },
targetHint: {
geometry: { rect: { x: 400, y: 300, w: 80, h: 30 }, tag: "button" },
},
};
session.bindDraft(draft, 2, true);
expect(draft.preStateId).toBeUndefined();
expect("matchedTarget" in draft ? draft.matchedTarget : undefined).toEqual({
role: "button",
name: "Confirm",
unmatched: true,
});
});
});
@@ -0,0 +1,152 @@
import { afterEach, describe, expect, it, vi } from "vitest";
import type { CdpRunner, ChromeTabsApi } from "@/tools/shared";
const captureRecordingObservation = vi.hoisted(() => vi.fn());
vi.mock("../recording/observation-capture", async (importOriginal) => {
const actual = await importOriginal<typeof import("../recording/observation-capture")>();
return { ...actual, captureRecordingObservation };
});
import { ObservationNodeIndex } from "../recording/observation-capture";
import { RecordingObservationRuntime } from "../recording/recording-runtime";
function observation(url = "https://example.com/") {
return {
rootFrameId: "root",
index: new ObservationNodeIndex({ rootFrameId: "root", matchNodes: [], refs: [] }),
url,
title: "Example",
vomText: '@vom 1\nRootWebArea "Example"',
truncated: false,
};
}
function runtime(): RecordingObservationRuntime {
return new RecordingObservationRuntime({
cdp: { send: vi.fn() as unknown as CdpRunner["send"] },
tabsApi: { get: vi.fn(), query: vi.fn() } as unknown as ChromeTabsApi,
});
}
describe("RecordingObservationRuntime", () => {
afterEach(() => captureRecordingObservation.mockReset());
it("shares one initial capture and includes it in flush", async () => {
let release!: (value: ReturnType<typeof observation>) => void;
captureRecordingObservation.mockReturnValueOnce(
new Promise((resolve) => {
release = resolve;
}),
);
const recording = runtime();
const first = recording.captureInitial(7);
const second = recording.captureInitial(7);
const flushed = recording.flush();
let flushCompleted = false;
void flushed.then(() => {
flushCompleted = true;
});
await Promise.resolve();
expect(captureRecordingObservation).toHaveBeenCalledTimes(1);
expect(flushCompleted).toBe(false);
release(observation());
await Promise.all([first, second, flushed]);
expect(flushCompleted).toBe(true);
});
it("cancels an in-flight initial capture", async () => {
captureRecordingObservation.mockImplementationOnce(
(input: { signal?: AbortSignal }) =>
new Promise((_, reject) => {
input.signal?.addEventListener(
"abort",
() => reject(new DOMException("observation aborted", "AbortError")),
{ once: true },
);
}),
);
const recording = runtime();
const pending = recording.captureInitial(7);
recording.cancel();
await expect(pending).rejects.toMatchObject({ name: "AbortError" });
});
it("retries a failed initial capture when the recording settles at stop", async () => {
captureRecordingObservation
.mockRejectedValueOnce(new Error("document swapped"))
.mockResolvedValueOnce(observation());
const recording = runtime();
await expect(recording.captureInitial(7)).rejects.toThrow("document swapped");
await recording.settleTrailing(7, []);
const trace = recording.buildTrace({
drafts: [],
startedAt: "2026-08-19T00:00:00.000Z",
stoppedBy: "user_finish",
bskVersion: "0.1.6",
});
expect(captureRecordingObservation).toHaveBeenCalledTimes(2);
expect(trace.states).toHaveLength(1);
});
it("keeps runtime matching data out of the serialized trace", async () => {
const captured = observation();
captured.index = new ObservationNodeIndex({
rootFrameId: "root",
matchNodes: [
{
frameId: "root",
backendNodeId: 42,
tag: "button",
rect: { x: 10, y: 20, w: 100, h: 30 },
localRect: { x: 10, y: 20, w: 100, h: 30 },
},
],
refs: [{ ref: "e1", backendNodeId: 42, role: "button", name: "Submit", line: 1 }],
});
captureRecordingObservation.mockResolvedValueOnce(captured);
const recording = runtime();
await recording.captureInitial(7);
const dumped = JSON.stringify(
recording.buildTrace({
drafts: [],
startedAt: "2026-08-19T00:00:00.000Z",
stoppedBy: "user_finish",
bskVersion: "0.1.6",
}),
);
expect(dumped).toContain("RootWebArea");
expect(dumped).not.toContain("matchNodes");
expect(dumped).not.toContain("localRect");
expect(dumped).not.toContain("backendNodeId");
expect(dumped).not.toContain("ObservationNodeIndex");
});
it("captures a tab transition without sharing observation cursors", async () => {
captureRecordingObservation
.mockResolvedValueOnce(observation("https://example.com/first"))
.mockResolvedValueOnce(observation("https://example.com/second"));
const recording = runtime();
await recording.captureInitial(4);
const transition = await recording.captureTabTransition(4, 5);
await recording.captureInitial(4);
await recording.captureInitial(5);
expect(transition).toEqual({
preStateId: "s1",
postStateId: "s2",
targetUrl: "https://example.com/second",
});
expect(captureRecordingObservation.mock.calls.map(([input]) => input.tabId)).toEqual([4, 5]);
});
});
@@ -1,80 +0,0 @@
import { describe, expect, it } from "vitest";
import { appendRecordedPayload, observeRecordedNavigation } from "../recording-step-buffer";
describe("recording-step-buffer", () => {
it("stores semantic click without summary", () => {
const buffer = { steps: [], pendingNavigation: false };
appendRecordedPayload(buffer, {
op: "click",
target: { tag: "button", role: "button", name: "发布" },
expects_navigation: true,
});
expect(buffer.steps).toEqual([
{
op: "click",
target: { tag: "button", role: "button", name: "发布" },
},
]);
expect(buffer.pendingNavigation).toBe(true);
});
it("annotates navigated_to on action-caused navigation instead of wait_for_navigation", () => {
const buffer = {
steps: [
{
op: "click" as const,
target: { tag: "button", role: "button", name: "发布" },
},
],
currentUrl: "https://example.com/a",
pendingNavigation: true,
pendingNavigationDeadline: Date.now() + 5_000,
};
observeRecordedNavigation(buffer, "https://example.com/b", true);
expect(buffer.steps).toEqual([
{
op: "click",
target: { tag: "button", role: "button", name: "发布" },
navigated_to: "https://example.com/b",
},
]);
expect(JSON.stringify(buffer.steps)).not.toContain("wait_for_navigation");
});
it("annotates navigated_to onto a select that auto-submits a navigation", () => {
const buffer = {
steps: [
{
op: "select" as const,
target: { tag: "select", role: "combobox", name: "分类" },
values: ["tech"],
},
],
currentUrl: "https://example.com/list",
pendingNavigation: true,
pendingNavigationDeadline: Date.now() + 5_000,
};
observeRecordedNavigation(buffer, "https://example.com/list?cat=tech", true);
expect(buffer.steps).toEqual([
{
op: "select",
target: { tag: "select", role: "combobox", name: "分类" },
values: ["tech"],
navigated_to: "https://example.com/list?cat=tech",
},
]);
});
it("emits navigate for uncaused URL changes", () => {
const buffer = {
steps: [],
currentUrl: "https://example.com/a",
pendingNavigation: false,
};
observeRecordedNavigation(buffer, "https://example.com/b", false);
expect(buffer.steps[0]).toMatchObject({
op: "navigate",
url: "https://example.com/b",
});
});
});
@@ -0,0 +1,136 @@
import { afterEach, describe, expect, it, vi } from "vitest";
import type { CdpRunner, ChromeTabsApi } from "@/tools/shared";
import { ObservationNodeIndex, type RegisteredObservation } from "../recording/observation-capture";
import { RecordingObservationSession } from "../recording/observation-session";
import { SettleController } from "../recording/settle-controller";
import type { RecordingDraftStep } from "../recording/types";
const OBSERVATION: RegisteredObservation = {
stateId: "s-next",
rootFrameId: "root",
index: new ObservationNodeIndex({ rootFrameId: "root", matchNodes: [], refs: [] }),
url: "https://example.com/next",
};
describe("SettleController", () => {
afterEach(() => vi.useRealTimers());
it("aborts superseded capture work and lands the prior action on the next origin", async () => {
vi.useFakeTimers();
const cdp: CdpRunner = {
send: vi.fn(async () => ({
result: { value: { idleMs: 1_000, readyState: "complete" } },
})) as unknown as CdpRunner["send"],
};
const tabsApi: ChromeTabsApi = {
get: vi.fn(async () => ({ id: 7, url: OBSERVATION.url }) as chrome.tabs.Tab),
query: vi.fn(async () => []),
};
const session = new RecordingObservationSession();
let firstSignal: AbortSignal | undefined;
vi.spyOn(session, "capture")
.mockImplementationOnce(async (_cdp, _tabs, _tabId, signal) => {
firstSignal = signal;
return new Promise<RegisteredObservation>((_resolve, reject) => {
signal?.addEventListener(
"abort",
() => reject(new DOMException("observation aborted", "AbortError")),
{ once: true },
);
});
})
.mockResolvedValueOnce(OBSERVATION);
const drafts: RecordingDraftStep[] = [
{ op: "click", captureTarget: { tag: "button", name: "First" }, preStateId: "s1" },
];
const controller = new SettleController({ session, cdp, tabsApi, tabId: 7 });
controller.schedule(drafts, 0);
await vi.advanceTimersByTimeAsync(180);
drafts.push({
op: "click",
captureTarget: { tag: "button", name: "Second" },
preStateId: "s-next",
});
controller.schedule(drafts, 1);
expect(firstSignal?.aborted).toBe(true);
expect(drafts[0]?.postStateId).toBe("s-next");
await vi.advanceTimersByTimeAsync(500);
await controller.flush();
expect(drafts[1]?.postStateId).toBe("s-next");
});
it("does not consume a newer redirect while reading the prior landing URL", async () => {
vi.useFakeTimers();
const cdp: CdpRunner = {
send: vi.fn(async () => ({
result: { value: { idleMs: 1_000, readyState: "complete" } },
})) as unknown as CdpRunner["send"],
};
let resolveFirstTab!: (tab: chrome.tabs.Tab) => void;
const firstTab = new Promise<chrome.tabs.Tab>((resolve) => {
resolveFirstTab = resolve;
});
const tabsApi: ChromeTabsApi = {
get: vi
.fn()
.mockImplementationOnce(() => firstTab)
.mockResolvedValue({ id: 7, url: OBSERVATION.url }),
query: vi.fn(async () => []),
};
const session = new RecordingObservationSession();
session.cursor.lastSettled = OBSERVATION;
const drafts: RecordingDraftStep[] = [];
const controller = new SettleController({ session, cdp, tabsApi, tabId: 7 });
controller.scheduleRedirect(drafts, "https://example.com/intermediate");
await vi.advanceTimersByTimeAsync(180);
expect(tabsApi.get).toHaveBeenCalledTimes(1);
controller.scheduleRedirect(drafts, OBSERVATION.url);
resolveFirstTab({ id: 7, url: "https://example.com/intermediate" } as chrome.tabs.Tab);
await vi.advanceTimersByTimeAsync(500);
await controller.flushRedirects();
expect(drafts).toEqual([]);
expect(controller.hasPending).toBe(false);
});
it("settles an action in its own OOPIF document scope", async () => {
vi.useFakeTimers();
const send = vi.fn();
const sendToTarget = vi.fn(async (_target, method: string) => {
if (method === "Page.createIsolatedWorld") return { executionContextId: 39 };
return { result: { value: { idleMs: 1_000, readyState: "complete" } } };
});
const cdp: CdpRunner = {
send: send as unknown as CdpRunner["send"],
sendToTarget: sendToTarget as unknown as NonNullable<CdpRunner["sendToTarget"]>,
};
const tabsApi: ChromeTabsApi = {
get: vi.fn(async () => ({ id: 7, url: OBSERVATION.url }) as chrome.tabs.Tab),
query: vi.fn(async () => []),
};
const session = new RecordingObservationSession();
vi.spyOn(session, "capture").mockResolvedValue(OBSERVATION);
const drafts: RecordingDraftStep[] = [
{ op: "click", captureTarget: { tag: "button", name: "Inside frame" } },
];
const controller = new SettleController({ session, cdp, tabsApi, tabId: 7 });
const target = { tabId: 7, sessionId: "oopif-session" };
controller.schedule(drafts, 0, { target, frameId: "oopif-frame" });
await vi.advanceTimersByTimeAsync(180);
await controller.flush();
expect(send).not.toHaveBeenCalled();
expect(sendToTarget).toHaveBeenCalledWith(
target,
"Runtime.evaluate",
expect.objectContaining({ contextId: 39 }),
);
expect(drafts[0]?.postStateId).toBe("s-next");
});
});
@@ -0,0 +1,164 @@
import { describe, expect, it } from "vitest";
import { appendRecordedPayload, observeRecordedNavigation } from "../recording/step-buffer";
describe("recording-step-buffer", () => {
it("stores semantic click without summary", () => {
const buffer = { steps: [], navigation: { pendingNavigation: false } };
appendRecordedPayload(buffer, {
op: "click",
target: { tag: "button", role: "button", name: "发布" },
expects_navigation: true,
});
expect(buffer.steps).toEqual([
{
op: "click",
captureTarget: { tag: "button", role: "button", name: "发布" },
},
]);
expect(buffer.navigation.pendingNavigation).toBe(true);
});
it("keeps the hovered element description as a capture fallback", () => {
const buffer = { steps: [], navigation: { pendingNavigation: false } };
appendRecordedPayload(
buffer,
{
op: "hover",
target: { tag: "button", role: "button", name: "新建" },
geometry: {
rect: { x: 900, y: 8, w: 60, h: 32 },
tag: "button",
},
},
{
geometry: {
rect: { x: 900, y: 8, w: 60, h: 32 },
tag: "button",
},
},
);
expect(buffer.steps).toEqual([
{
op: "hover",
captureTarget: { tag: "button", role: "button", name: "新建" },
targetHint: {
geometry: {
rect: { x: 900, y: 8, w: 60, h: 32 },
tag: "button",
},
},
},
]);
});
it("annotates navigated_to on action-caused navigation instead of wait_for_navigation", () => {
const buffer = {
steps: [
{
op: "click" as const,
captureTarget: { tag: "button", role: "button", name: "发布" },
},
],
navigation: {
currentUrl: "https://example.com/a",
pendingNavigation: true,
pendingNavigationDeadline: Date.now() + 5_000,
},
};
observeRecordedNavigation(buffer, "https://example.com/b", true);
expect(buffer.steps).toEqual([
{
op: "click",
captureTarget: { tag: "button", role: "button", name: "发布" },
navigatedTo: "https://example.com/b",
},
]);
expect(JSON.stringify(buffer.steps)).not.toContain("wait_for_navigation");
});
it("annotates navigated_to onto a select that auto-submits a navigation", () => {
const buffer = {
steps: [
{
op: "select" as const,
captureTarget: { tag: "select", role: "combobox", name: "分类" },
values: ["tech"],
},
],
navigation: {
currentUrl: "https://example.com/list",
pendingNavigation: true,
pendingNavigationDeadline: Date.now() + 5_000,
},
};
observeRecordedNavigation(buffer, "https://example.com/list?cat=tech", true);
expect(buffer.steps).toEqual([
{
op: "select",
captureTarget: { tag: "select", role: "combobox", name: "分类" },
values: ["tech"],
navigatedTo: "https://example.com/list?cat=tech",
},
]);
});
it("emits navigate for uncaused URL changes", () => {
const buffer = {
steps: [],
navigation: { currentUrl: "https://example.com/a", pendingNavigation: false },
};
const result = observeRecordedNavigation(buffer, "https://example.com/b", false);
expect(result).toEqual({ kind: "appended", index: 0 });
expect(buffer.steps[0]).toMatchObject({
op: "navigate",
url: "https://example.com/b",
});
});
it("asks the recorder to coalesce redirect hops instead of emitting each one", () => {
const buffer = {
steps: [],
navigation: {
currentUrl: "https://passport.example/login",
pendingNavigation: false,
},
};
const hop1 = observeRecordedNavigation(
buffer,
"https://passport.example/callback",
false,
"link",
["server_redirect"],
);
expect(hop1).toEqual({
kind: "coalesce_redirect",
url: "https://passport.example/callback",
});
expect(buffer.steps).toEqual([]);
const hop2 = observeRecordedNavigation(buffer, "https://app.example/dashboard", false, "link", [
"client_redirect",
]);
expect(hop2).toEqual({
kind: "coalesce_redirect",
url: "https://app.example/dashboard",
});
expect(buffer.steps).toEqual([]);
expect(buffer.navigation.currentUrl).toBe("https://app.example/dashboard");
});
it("does not reuse redirect metadata for a later content-observed URL change", () => {
const buffer = {
steps: [],
navigation: {
currentUrl: "https://example.com/redirect",
pendingNavigation: false,
},
};
const result = observeRecordedNavigation(buffer, "https://example.com/spa");
expect(result).toEqual({ kind: "appended", index: 0 });
expect(buffer.steps[0]).toMatchObject({ op: "navigate", url: "https://example.com/spa" });
});
});
@@ -0,0 +1,39 @@
import { describe, expect, it } from "vitest";
import { RecordingTabCoordinator } from "../recording/tab-coordinator";
describe("RecordingTabCoordinator", () => {
it("keeps navigation state isolated per tab", () => {
const tabs = new RecordingTabCoordinator(4, "https://example.com/first");
const first = tabs.navigation(4);
first.pendingNavigation = true;
const second = tabs.navigation(5, "https://example.com/second");
expect(second).toEqual({
currentUrl: "https://example.com/second",
pendingNavigation: false,
});
expect(tabs.navigation(4)).toBe(first);
expect(tabs.navigation(4).pendingNavigation).toBe(true);
});
it("invalidates an earlier activation when a newer tab becomes active", () => {
const tabs = new RecordingTabCoordinator(4);
const second = tabs.noteActivation(5);
const third = tabs.noteActivation(6);
expect(tabs.isLatest(second)).toBe(false);
expect(tabs.isLatest(third)).toBe(true);
tabs.commit(6);
expect(tabs.currentTabId).toBe(6);
});
it("commits the freshly observed URL when returning to an existing tab", () => {
const tabs = new RecordingTabCoordinator(4);
tabs.navigation(5, "https://example.com/old");
tabs.commit(5, "https://example.com/new");
expect(tabs.currentTabId).toBe(5);
expect(tabs.navigation(5).currentUrl).toBe("https://example.com/new");
});
});
@@ -0,0 +1,127 @@
import type { RenderedRef } from "@browser-skill/vom";
import { describe, expect, it } from "vitest";
import type { CaptureVomMatchNode } from "@/tools/capture-vom-observation";
import { ObservationNodeIndex, type RegisteredObservation } from "../recording/observation-capture";
import { matchObservationTarget } from "../recording/target-matcher";
function node(backendNodeId: number, frameId: string, x = 10): CaptureVomMatchNode {
return {
backendNodeId,
frameId,
tag: "button",
rect: { x, y: 20, w: 100, h: 30 },
localRect: { x, y: 20, w: 100, h: 30 },
};
}
function observation(
matchNodes: CaptureVomMatchNode[],
refs: RenderedRef[],
): RegisteredObservation {
return {
stateId: "s1",
rootFrameId: "root",
index: new ObservationNodeIndex({ rootFrameId: "root", matchNodes, refs }),
url: "https://example.com",
};
}
describe("matchObservationTarget", () => {
it("indexes the safe geometry DTO without discarding frame-local geometry", () => {
const geometry = node(42, "child");
geometry.localRect = { x: 5, y: 6, w: 100, h: 30 };
const ref: RenderedRef = {
ref: "e1",
backendNodeId: 42,
frameId: "child",
line: 1,
};
const candidate = observation([geometry], [ref]).index.candidates("child", "button")[0];
expect(candidate).toEqual({ frameId: "child", geometry, ref });
});
it("matches a unique node using canonical top-level viewport geometry", () => {
const target = matchObservationTarget({
observation: observation(
[node(42, "root")],
[{ ref: "e1", backendNodeId: 42, role: "button", name: "发布", line: 1 }],
),
hint: { geometry: { rect: { x: 10, y: 20, w: 100, h: 30 }, tag: "button" } },
});
expect(target).toEqual({ ref: "e1", role: "button", name: "发布" });
});
it("uses frame id with backend node id so sibling frames cannot collide", () => {
const target = matchObservationTarget({
observation: observation(
[node(42, "left"), node(42, "right")],
[
{ ref: "e1", backendNodeId: 42, frameId: "left", line: 1 },
{ ref: "e2", backendNodeId: 42, frameId: "right", line: 2 },
],
),
hint: {
frameId: "right",
geometry: { rect: { x: 10, y: 20, w: 100, h: 30 }, tag: "button" },
},
});
expect(target.ref).toBe("e2");
});
it("restricts a missing frame hint to the root frame", () => {
const target = matchObservationTarget({
observation: observation(
[node(42, "child")],
[{ ref: "e1", backendNodeId: 42, frameId: "child", line: 1 }],
),
hint: { geometry: { rect: { x: 10, y: 20, w: 100, h: 30 }, tag: "button" } },
fallback: { tag: "button", name: "发布" },
});
expect(target).toEqual({ name: "发布", unmatched: true });
});
it("returns unmatched for ambiguous geometry", () => {
const target = matchObservationTarget({
observation: observation(
[node(42, "root"), node(43, "root")],
[
{ ref: "e1", backendNodeId: 42, line: 1 },
{ ref: "e2", backendNodeId: 43, line: 2 },
],
),
hint: { geometry: { rect: { x: 10, y: 20, w: 100, h: 30 }, tag: "button" } },
});
expect(target.unmatched).toBe(true);
});
it("uses semantics when geometry is unavailable without crossing frame boundaries", () => {
const target = matchObservationTarget({
observation: observation(
[node(41, "root"), node(42, "child")],
[
{ ref: "e1", backendNodeId: 41, role: "button", name: "保存", line: 1 },
{ ref: "e2", backendNodeId: 42, frameId: "child", role: "button", name: "保存", line: 2 },
],
),
hint: { frameId: "child" },
fallback: { tag: "button", role: "button", name: "保存" },
});
expect(target.ref).toBe("e2");
});
it("uses semantics to disambiguate equal geometry in one frame", () => {
const target = matchObservationTarget({
observation: observation(
[node(42, "root"), node(43, "root")],
[
{ ref: "e1", backendNodeId: 42, role: "button", name: "保存", line: 1 },
{ ref: "e2", backendNodeId: 43, role: "button", name: "取消", line: 2 },
],
),
hint: { geometry: { rect: { x: 10, y: 20, w: 100, h: 30 }, tag: "button" } },
fallback: { tag: "button", role: "button", name: "取消" },
});
expect(target.ref).toBe("e2");
});
});
@@ -1,6 +1,15 @@
import { describe, expect, it } from "vitest";
import type { DraftTraceStep } from "@/transport/types";
import { reduceTraceSteps, resolveTraceStartUrl, shouldRecordPress } from "../trace-reducer";
import type { RecordingDraftStep } from "@/lib/recording/types";
import { shouldRecordPress } from "../recording/draft-policy";
import { buildTraceV2 } from "../recording/trace-reducer-v2";
function reduceTraceSteps(steps: RecordingDraftStep[], startUrl?: string) {
return buildTraceV2({
steps,
startedAt: "2026-01-01T00:00:00.000Z",
...(startUrl ? { startUrl } : {}),
});
}
describe("shouldRecordPress", () => {
it("keeps Enter and Escape", () => {
@@ -18,36 +27,47 @@ describe("shouldRecordPress", () => {
});
describe("reduceTraceSteps", () => {
it("does not expose internal tab transitions in trace v2", () => {
const trace = reduceTraceSteps([
{
op: "switch_tab",
preStateId: "s1",
postStateId: "s2",
},
]);
expect(trace.steps).toEqual([]);
});
it("builds steps with pages dictionary and page id references", () => {
const drafts: DraftTraceStep[] = [
const drafts: RecordingDraftStep[] = [
{
op: "navigate",
url: "https://example.com/search?q=hello&utm_source=x",
page_url: "https://example.com/search?q=hello&utm_source=x",
pageUrl: "https://example.com/search?q=hello&utm_source=x",
},
{
op: "fill",
target: { tag: "input", role: "textbox", name: "搜索", name_attr: "q" },
captureTarget: { tag: "input", role: "textbox", name: "搜索", name_attr: "q" },
value: "browser skill",
page_url: "https://example.com/search?q=hello&utm_source=x",
pageUrl: "https://example.com/search?q=hello&utm_source=x",
},
{
op: "press",
key: "Enter",
target: { tag: "input", role: "textbox", name: "搜索", name_attr: "q" },
navigated_to: "https://example.com/results/42",
page_url: "https://example.com/search?q=hello&utm_source=x",
captureTarget: { tag: "input", role: "textbox", name: "搜索", name_attr: "q" },
navigatedTo: "https://example.com/results/42",
pageUrl: "https://example.com/search?q=hello&utm_source=x",
},
{
op: "click",
target: { tag: "button", role: "button", name: "发布" },
navigated_to: "https://example.com/p/99",
page_url: "https://example.com/results/42",
captureTarget: { tag: "button", role: "button", name: "发布" },
navigatedTo: "https://example.com/p/99",
pageUrl: "https://example.com/results/42",
},
{
op: "press",
key: "a",
page_url: "https://example.com/p/99",
pageUrl: "https://example.com/p/99",
},
];
@@ -109,9 +129,9 @@ describe("reduceTraceSteps", () => {
[
{
op: "fill",
target: { tag: "input", role: "textbox", name: "Search query" },
captureTarget: { tag: "input", role: "textbox", name: "Search query" },
value: "",
page_url: "https://example.com/search",
pageUrl: "https://example.com/search",
},
],
"https://example.com/search",
@@ -125,16 +145,16 @@ describe("reduceTraceSteps", () => {
]);
});
it("maps select navigated_to onto effect.navigated_to (page id)", () => {
it("maps select navigatedTo onto effect.navigatedTo (page id)", () => {
const { pages, steps } = reduceTraceSteps(
[
{
op: "select",
target: { tag: "select", role: "combobox", name: "分类" },
captureTarget: { tag: "select", role: "combobox", name: "分类" },
values: ["tech"],
labels: ["技术"],
navigated_to: "https://example.com/list?cat=tech",
page_url: "https://example.com/list",
navigatedTo: "https://example.com/list?cat=tech",
pageUrl: "https://example.com/list",
},
],
"https://example.com/list",
@@ -151,36 +171,32 @@ describe("reduceTraceSteps", () => {
});
});
it("keeps hover steps before menu clicks", () => {
it("drops hover steps, which only exist in trace v3", () => {
const { steps } = reduceTraceSteps(
[
{
op: "hover",
target: { tag: "span", role: "button", name: "Account" },
page_url: "https://example.com/app",
captureTarget: { tag: "span", role: "button", name: "Account" },
pageUrl: "https://example.com/app",
},
{
op: "click",
target: { tag: "a", role: "link", name: "Profile" },
page_url: "https://example.com/app",
captureTarget: { tag: "a", role: "link", name: "Profile" },
pageUrl: "https://example.com/app",
},
],
"https://example.com/app",
);
expect(steps.map((s) => s.op)).toEqual(["hover", "click"]);
expect(steps[0]).toMatchObject({
op: "hover",
page: "p1",
target: { name: "Account" },
});
expect(steps.map((s) => s.op)).toEqual(["click"]);
});
it("resolveTraceStartUrl prefers explicit start URL", () => {
expect(
resolveTraceStartUrl(
[{ op: "navigate", url: "https://example.com/other" }],
"https://example.com/start",
),
buildTraceV2({
steps: [{ op: "navigate", url: "https://example.com/other" }],
startedAt: "2026-01-01T00:00:00.000Z",
startUrl: "https://example.com/start",
}).entry.start_url,
).toBe("https://example.com/start");
});
});
@@ -0,0 +1,132 @@
import { describe, expect, it } from "vitest";
import { TRACE_VERSION_V3, VOM_FORMAT_VERSION } from "@/transport/types";
import { RecordingStateRegistry } from "../recording/state-registry";
import { buildTraceV3 } from "../recording/trace-builder-v3";
import { reduceTraceStepsV3 } from "../recording/trace-reducer-v3";
import type { RecordingDraftStep } from "../recording/types";
describe("trace reducer v3", () => {
it("removes form literals when value redaction is requested", () => {
const reduced = reduceTraceStepsV3(
[
{
op: "fill",
value: "private@example.com",
preStateId: "s1",
postStateId: "s1",
},
{
op: "select",
values: ["private-account-id"],
labels: ["Private account"],
preStateId: "s1",
postStateId: "s1",
},
],
{ redactValues: true },
);
expect(reduced.steps[0]).toMatchObject({ op: "fill", value: "***", redacted: true });
expect(reduced.steps[1]).toMatchObject({ op: "select" });
expect(reduced.steps[1]).not.toHaveProperty("selection");
expect(JSON.stringify(reduced.steps)).not.toContain("private@example.com");
expect(JSON.stringify(reduced.steps)).not.toContain("private-account-id");
});
it("emits tab transitions only for callers that advertised support", () => {
const drafts: RecordingDraftStep[] = [
{
op: "switch_tab",
preStateId: "s1",
postStateId: "s2",
},
];
expect(reduceTraceStepsV3(drafts).steps).toEqual([]);
expect(reduceTraceStepsV3(drafts, { includeTabSwitches: true }).steps).toEqual([
{ op: "switch_tab", id: 1, state: "s1", result: { state: "s2" } },
]);
});
it("keeps hover steps before menu clicks", () => {
const drafts: RecordingDraftStep[] = [
{
op: "hover",
captureTarget: { tag: "span", role: "button", name: "Account" },
preStateId: "s1",
postStateId: "s1",
},
{
op: "click",
captureTarget: { tag: "a", role: "link", name: "Profile" },
preStateId: "s1",
postStateId: "s1",
},
];
expect(reduceTraceStepsV3(drafts).steps.map((s) => s.op)).toEqual(["hover", "click"]);
});
it("does not export a tab transition without both observation endpoints", () => {
const sourceOnly: RecordingDraftStep = { op: "switch_tab", preStateId: "s1" };
const targetOnly: RecordingDraftStep = { op: "switch_tab", postStateId: "s2" };
expect(
reduceTraceStepsV3([sourceOnly, targetOnly], { includeTabSwitches: true }).steps,
).toEqual([]);
});
it("collapses redirect hops while retaining draft-to-step identity", () => {
const drafts: RecordingDraftStep[] = [
{ op: "navigate", url: "https://example.com/start", preStateId: "s1", postStateId: "s2" },
{
op: "navigate",
url: "https://example.com/final",
transitionQualifiers: ["server_redirect"],
preStateId: "s2",
postStateId: "s3",
},
];
const before = structuredClone(drafts);
const reduced = reduceTraceStepsV3(drafts);
expect(drafts).toEqual(before);
expect(reduced.steps).toEqual([
expect.objectContaining({
id: 1,
op: "navigate",
state: "s1",
to: "https://example.com/final",
result: { state: "s3" },
}),
]);
expect(reduced.stepIdByDraftId.get(1)).toBe(1);
expect(reduced.stepIdByDraftId.get(2)).toBe(1);
});
it("builds the wire model from protocol constants", () => {
const registry = new RecordingStateRegistry();
const state = registry.register({ url: "https://example.com", vomText: "@vom 1" });
const trace = buildTraceV3({
registry,
drafts: [
{
op: "click",
captureTarget: { tag: "button", role: "button", name: "Save" },
preStateId: state.id,
postStateId: state.id,
},
],
startedAt: "2026-08-12T00:00:00.000Z",
stoppedBy: "user_finish",
bskVersion: "test",
});
expect(trace.version).toBe(TRACE_VERSION_V3);
expect(trace.recorder.vom).toBe(VOM_FORMAT_VERSION);
expect(trace.steps[0]).toMatchObject({
op: "click",
target: { role: "button", name: "Save", unmatched: true },
});
});
});
+6 -5
View File
@@ -1,12 +1,13 @@
/**
* Build a semantic TargetDescriptor for an interacted element.
* Build a semantic capture descriptor for an interacted element.
*
* Trace steps are an LLM *textbook*: each click must say what to look for
* on screen (usually a short visible name). Tag-only noise like
* `{ "tag": "div" }` / “点击div” is useless and must not be recorded.
*/
export interface TargetDescriptor {
/** Content-script capture descriptor before VOM geometric matching. */
export interface CaptureTargetDescriptor {
role?: string;
name?: string;
tag: string;
@@ -247,7 +248,7 @@ export function resolveHoverElement(target: Element): Element | null {
* (visible name), or at least a form `name_attr` for checkbox/radio.
* Recording “点击div” with no name fails this bar.
*/
export function isMeaningfulClickTarget(target: TargetDescriptor): boolean {
export function isMeaningfulClickTarget(target: CaptureTargetDescriptor): boolean {
const name = target.name?.trim();
if (name && isActionableLabel(name)) return true;
if (
@@ -323,7 +324,7 @@ function nearbyLabelText(el: Element): string | undefined {
return undefined;
}
export function describeTarget(el: Element): TargetDescriptor {
export function describeTarget(el: Element): CaptureTargetDescriptor {
const tag = el.tagName.toLowerCase();
const role = inferRole(el);
const name = accessibleName(el);
@@ -350,7 +351,7 @@ export function describeTarget(el: Element): TargetDescriptor {
};
}
export function describeEventTarget(target: EventTarget | null): TargetDescriptor | null {
export function describeEventTarget(target: EventTarget | null): CaptureTargetDescriptor | null {
if (!(target instanceof Element)) return null;
const clickable = resolveClickableElement(target);
if (!clickable) return null;
+39 -3
View File
@@ -3,7 +3,7 @@
* service worker and a tab's content script.
*/
import type { TargetDescriptor } from "./describe-target";
import type { CaptureTargetDescriptor } from "./describe-target";
export const RECORD_START = "bsk-record-start";
export const RECORD_STEP = "bsk-record-step";
@@ -16,6 +16,10 @@ export interface RecordStartAck {
ok: true;
}
export type RecordStepAck =
| { ok: true; sequence: number }
| { ok: false; expectedSequence: number; error: string };
export type RecordStopAck = { ok: true } | { ok: false; error: string };
export interface RecordQueryMessage {
@@ -42,7 +46,7 @@ export interface RecordStartMessage {
export interface RecordStepPayload {
op: "click" | "hover" | "fill" | "press" | "select" | "navigate";
target?: TargetDescriptor;
target?: CaptureTargetDescriptor;
value?: string;
key?: string;
modifiers?: Array<"alt" | "ctrl" | "meta" | "shift">;
@@ -50,17 +54,30 @@ export interface RecordStepPayload {
labels?: string[];
url?: string;
redacted?: boolean;
commit?: "enter" | "suggestion" | "blur";
/** Page URL when the step was captured. */
page_url?: string;
/** Event-target bounds in the source frame's viewport coordinate space. */
geometry?: {
rect: { x: number; y: number; w: number; h: number };
tag: string;
};
/** Capture-only hint; never persisted unless converted to navigated_to. */
expects_navigation?: boolean;
/** Whether an observed URL change was synchronously caused by the action. */
navigation_caused_by_action?: boolean;
/** Raw webNavigation transition metadata for cause mapping. */
transitionType?: string;
transitionQualifiers?: string[];
}
export interface RecordStepMessage {
type: typeof RECORD_STEP;
requestId: string;
/** Stable for the lifetime of one content-script document. */
producerId: string;
/** Monotonic within one content-script producer and recording request. */
sequence: number;
step: RecordStepPayload;
}
@@ -112,8 +129,27 @@ export function isRecordQueryMessage(msg: unknown): msg is RecordQueryMessage {
export function isRecordStepMessage(msg: unknown): msg is RecordStepMessage {
if (typeof msg !== "object" || msg === null) return false;
const m = msg as Record<string, unknown>;
if (m.type !== RECORD_STEP || typeof m.requestId !== "string") return false;
if (
m.type !== RECORD_STEP ||
typeof m.requestId !== "string" ||
typeof m.producerId !== "string" ||
m.producerId.length === 0 ||
typeof m.sequence !== "number" ||
!Number.isSafeInteger(m.sequence) ||
m.sequence < 1
) {
return false;
}
const step = m.step;
if (typeof step !== "object" || step === null) return false;
return typeof (step as RecordStepPayload).op === "string";
}
export function isAcceptedRecordStepAck(
value: unknown,
sequence: number,
): value is Extract<RecordStepAck, { ok: true }> {
if (typeof value !== "object" || value === null) return false;
const ack = value as Partial<RecordStepAck>;
return ack.ok === true && ack.sequence === sequence;
}
@@ -1,136 +0,0 @@
import type { DraftTraceStep } from "@/transport/types";
import type { RecordStepPayload } from "./record-bridge";
export interface RecordingStepBuffer {
steps: DraftTraceStep[];
currentUrl?: string;
pendingNavigation: boolean;
pendingNavigationDeadline?: number;
}
const NAVIGATION_TRIGGER_WINDOW_MS = 3_000;
function toDraftStep(payload: RecordStepPayload): DraftTraceStep | null {
const pageUrl = payload.page_url;
switch (payload.op) {
case "click":
return payload.target
? {
op: "click",
target: payload.target,
...(pageUrl ? { page_url: pageUrl } : {}),
}
: null;
case "hover":
return payload.target
? {
op: "hover",
target: payload.target,
...(pageUrl ? { page_url: pageUrl } : {}),
}
: null;
case "fill":
return payload.target
? {
op: "fill",
target: payload.target,
value: payload.value ?? "",
...(payload.redacted ? { redacted: true } : {}),
...(pageUrl ? { page_url: pageUrl } : {}),
}
: null;
case "press":
return payload.key
? {
op: "press",
key: payload.key,
...(payload.target ? { target: payload.target } : {}),
...(payload.modifiers?.length ? { modifiers: payload.modifiers } : {}),
...(pageUrl ? { page_url: pageUrl } : {}),
}
: null;
case "select":
return payload.target && payload.values
? {
op: "select",
target: payload.target,
values: payload.values,
...(payload.labels?.length ? { labels: payload.labels } : {}),
...(pageUrl ? { page_url: pageUrl } : {}),
}
: null;
case "navigate":
return null;
}
}
function annotateLastStepNavigation(buffer: RecordingStepBuffer, url: string): boolean {
for (let i = buffer.steps.length - 1; i >= 0; i -= 1) {
const step = buffer.steps[i];
if (!step) continue;
if (step.op === "click" || step.op === "press" || step.op === "select") {
buffer.steps[i] = { ...step, navigated_to: url };
return true;
}
break;
}
return false;
}
export function observeRecordedNavigation(
buffer: RecordingStepBuffer,
url: string,
causedByAction?: boolean,
): void {
if (!url || url === buffer.currentUrl) return;
buffer.currentUrl = url;
const pendingIsCurrent =
buffer.pendingNavigation &&
(buffer.pendingNavigationDeadline === undefined ||
buffer.pendingNavigationDeadline >= Date.now());
if (causedByAction === true || (causedByAction === undefined && pendingIsCurrent)) {
buffer.pendingNavigation = false;
buffer.pendingNavigationDeadline = undefined;
if (!annotateLastStepNavigation(buffer, url)) {
buffer.steps.push({
op: "navigate",
url,
page_url: url,
});
}
return;
}
buffer.pendingNavigation = false;
buffer.pendingNavigationDeadline = undefined;
buffer.steps.push({
op: "navigate",
url,
page_url: url,
});
}
export function appendRecordedPayload(
buffer: RecordingStepBuffer,
payload: RecordStepPayload,
): void {
if (payload.op === "navigate") {
if (payload.url) {
observeRecordedNavigation(buffer, payload.url, payload.navigation_caused_by_action);
}
return;
}
const step = toDraftStep({
...payload,
page_url: payload.page_url ?? buffer.currentUrl,
});
if (!step) return;
buffer.steps.push(step);
if (step.op === "click" || step.op === "press" || step.op === "select" || step.op === "fill") {
buffer.pendingNavigation = payload.expects_navigation === true;
buffer.pendingNavigationDeadline = buffer.pendingNavigation
? Date.now() + NAVIGATION_TRIGGER_WINDOW_MS
: undefined;
}
}
@@ -0,0 +1,134 @@
import type { CdpTarget } from "@/browser-driver/frame-graph";
import { type CdpRunner, sendToCdpTarget } from "@/tools/shared";
const SETTLE_MIN_MS = 150;
const SETTLE_QUIET_MS = 250;
const SETTLE_MAX_MS = 2_000;
const SETTLE_POLL_MS = 60;
const SETTLE_WORLD_NAME = "__bsk_record_settle__";
const QUIET_PROBE = `(() => {
const scope = window;
let probe = scope.__bskRecordQuiet;
if (!probe) {
probe = { changedAt: Date.now() };
const observer = new MutationObserver(() => {
probe.changedAt = Date.now();
});
observer.observe(document.documentElement, {
subtree: true,
childList: true,
attributes: true,
characterData: true,
});
scope.__bskRecordQuiet = probe;
}
return { idleMs: Date.now() - probe.changedAt, readyState: document.readyState };
})()`;
export interface DocumentSettleScope {
target: CdpTarget;
/** Omit only for the target's root document. */
frameId?: string;
}
type SettleOutcome = "quiet" | "timeout" | "cancelled";
interface QuietProbe {
idleMs: number;
readyState: string;
}
interface ProbeContext {
executionContextId?: number;
}
function sleep(ms: number, signal?: AbortSignal): Promise<void> {
return new Promise((resolve) => {
if (signal?.aborted) return resolve();
const timer = setTimeout(resolve, ms);
signal?.addEventListener(
"abort",
() => {
clearTimeout(timer);
resolve();
},
{ once: true },
);
});
}
async function executionContextForScope(
cdp: CdpRunner,
scope: DocumentSettleScope,
): Promise<number | undefined> {
if (!scope.frameId) return undefined;
const result = await sendToCdpTarget<{ executionContextId?: number }>(
cdp,
scope.target,
"Page.createIsolatedWorld",
{
frameId: scope.frameId,
worldName: SETTLE_WORLD_NAME,
grantUniveralAccess: false,
},
);
return result.executionContextId;
}
async function readQuietProbe(
cdp: CdpRunner,
scope: DocumentSettleScope,
context: ProbeContext,
): Promise<QuietProbe | null> {
try {
if (scope.frameId && context.executionContextId === undefined) {
context.executionContextId = await executionContextForScope(cdp, scope);
if (context.executionContextId === undefined) return null;
}
const reply = await sendToCdpTarget<{ result?: { value?: unknown } }>(
cdp,
scope.target,
"Runtime.evaluate",
{
expression: QUIET_PROBE,
returnByValue: true,
...(context.executionContextId !== undefined
? { contextId: context.executionContextId }
: {}),
},
);
const value = reply.result?.value;
if (!value || typeof value !== "object") return null;
const { idleMs, readyState } = value as { idleMs?: unknown; readyState?: unknown };
if (typeof idleMs !== "number" || typeof readyState !== "string") return null;
return { idleMs, readyState };
} catch {
context.executionContextId = undefined;
return null;
}
}
export async function waitForDocumentSettled(
cdp: CdpRunner,
scope: DocumentSettleScope,
options: { signal?: AbortSignal } = {},
): Promise<SettleOutcome> {
const startedAt = Date.now();
const floor = startedAt + SETTLE_MIN_MS;
const deadline = startedAt + SETTLE_MAX_MS;
const context: ProbeContext = {};
for (;;) {
if (options.signal?.aborted) return "cancelled";
await sleep(SETTLE_POLL_MS, options.signal);
if (options.signal?.aborted) return "cancelled";
const probe = await readQuietProbe(cdp, scope, context);
const now = Date.now();
if (now < floor) continue;
if (now >= deadline) return "timeout";
if (!probe || probe.readyState === "loading") continue;
if (probe.idleMs >= SETTLE_QUIET_MS) return "quiet";
}
}
@@ -0,0 +1,34 @@
import type { RecordingDraftStep } from "./types";
const CLIPBOARD_KEYS = new Set(["a", "c", "v", "x", "A", "C", "V", "X"]);
const MODIFIER_ONLY_KEYS = new Set(["Meta", "Control", "Alt", "Shift", "OS", "Hyper", "Super"]);
export function shouldRecordPress(
key: string,
modifiers?: Array<"alt" | "ctrl" | "meta" | "shift">,
): boolean {
if (MODIFIER_ONLY_KEYS.has(key)) return false;
const mods = modifiers ?? [];
const hasCtrlOrMeta = mods.includes("ctrl") || mods.includes("meta");
if (hasCtrlOrMeta && CLIPBOARD_KEYS.has(key)) return false;
if (key === "Enter" || key === "Escape") return true;
if (key.length === 1 && !hasCtrlOrMeta && !mods.includes("alt")) return false;
return false;
}
export function shouldIncludeDraft(step: RecordingDraftStep): boolean {
return step.op !== "press" || shouldRecordPress(step.key, step.modifiers);
}
export function resolveDraftStartUrl(
drafts: RecordingDraftStep[],
explicitStartUrl?: string,
fallbackUrl?: string,
): string {
if (explicitStartUrl) return explicitStartUrl;
const navigation = drafts.find(
(step): step is Extract<RecordingDraftStep, { op: "navigate" }> => step.op === "navigate",
);
if (navigation) return navigation.url;
return drafts.find((step) => step.pageUrl)?.pageUrl ?? fallbackUrl ?? "about:blank";
}
@@ -0,0 +1,5 @@
const REDIRECT_QUALIFIERS = new Set(["client_redirect", "server_redirect"]);
export function hasRedirectQualifier(qualifiers?: readonly string[]): boolean {
return (qualifiers ?? []).some((qualifier) => REDIRECT_QUALIFIERS.has(qualifier));
}
@@ -0,0 +1,116 @@
import type { RenderedRef } from "@browser-skill/vom";
import {
type CaptureVomMatchNode,
type CaptureVomObservationResult,
captureVomObservation,
} from "@/tools/capture-vom-observation";
import type { CdpRunner, ChromeTabsApi } from "@/tools/shared";
export interface IndexedObservationNode {
frameId: string;
geometry: CaptureVomMatchNode;
ref?: RenderedRef;
}
export interface CapturedRecordingObservation {
rootFrameId: string;
index: ObservationNodeIndex;
url: string;
title?: string;
vomText: string;
truncated: boolean;
}
export interface RegisteredObservation {
stateId: string;
rootFrameId: string;
index: ObservationNodeIndex;
url: string;
}
function nodeKey(frameId: string, backendNodeId: number): string {
return `${frameId}:${backendNodeId}`;
}
function frameTagKey(frameId: string, tag: string): string {
return `${frameId}:${tag.toLowerCase()}`;
}
export class ObservationNodeIndex {
readonly #nodesByFrameTag = new Map<string, IndexedObservationNode[]>();
readonly #refById = new Map<string, RenderedRef>();
readonly #refsByFrame = new Map<string, RenderedRef[]>();
constructor(input: Pick<CaptureVomObservationResult, "rootFrameId" | "matchNodes" | "refs">) {
const refByNode = new Map<string, RenderedRef>();
for (const ref of input.refs) {
const frameId = ref.frameId ?? input.rootFrameId;
refByNode.set(nodeKey(frameId, ref.backendNodeId), ref);
this.#refById.set(ref.ref, ref);
const frameRefs = this.#refsByFrame.get(frameId) ?? [];
frameRefs.push(ref);
this.#refsByFrame.set(frameId, frameRefs);
}
for (const geometry of input.matchNodes) {
const { frameId } = geometry;
const entry = {
frameId,
geometry,
ref: refByNode.get(nodeKey(frameId, geometry.backendNodeId)),
};
const key = frameTagKey(frameId, geometry.tag);
const bucket = this.#nodesByFrameTag.get(key) ?? [];
bucket.push(entry);
this.#nodesByFrameTag.set(key, bucket);
}
}
candidates(frameId: string, tag: string): readonly IndexedObservationNode[] {
return this.#nodesByFrameTag.get(frameTagKey(frameId, tag)) ?? [];
}
ref(refId: string): RenderedRef | undefined {
return this.#refById.get(refId);
}
refs(frameId: string): readonly RenderedRef[] {
return this.#refsByFrame.get(frameId) ?? [];
}
}
async function readTabMeta(
tabsApi: ChromeTabsApi,
tabId: number,
): Promise<{ url: string; title?: string }> {
try {
const tab = await tabsApi.get(tabId);
return { url: tab.url ?? "about:blank", title: tab.title };
} catch {
return { url: "about:blank" };
}
}
export async function captureRecordingObservation(input: {
cdp: CdpRunner;
tabsApi: ChromeTabsApi;
tabId: number;
maxTokens: number;
redactValues: boolean;
signal?: AbortSignal;
}): Promise<CapturedRecordingObservation> {
const { url, title } = await readTabMeta(input.tabsApi, input.tabId);
const captured = await captureVomObservation(input.cdp, input.tabId, url, {
maxTokens: input.maxTokens,
redactValues: input.redactValues,
conditionalSurfaceProbe: false,
signal: input.signal,
});
return {
rootFrameId: captured.rootFrameId,
index: new ObservationNodeIndex(captured),
url,
title,
vomText: captured.text,
truncated: captured.truncated,
};
}
@@ -0,0 +1,128 @@
import type { CdpRunner, ChromeTabsApi } from "@/tools/shared";
import { captureRecordingObservation, type RegisteredObservation } from "./observation-capture";
import { RecordingStateRegistry } from "./state-registry";
import { matchObservationTarget, unmatchedTarget } from "./target-matcher";
import type { RecordingDraftStep, StepAnnotation, TargetedRecordingDraft } from "./types";
const DEFAULT_MAX_PAGE_TOKENS = 3_000;
const MIN_CAPTURE_INTERVAL_MS = 200;
export interface TabObservationCursor {
lastSettled: RegisteredObservation | null;
lastCaptureAt: number;
}
function abortableDelay(ms: number, signal?: AbortSignal): Promise<void> {
return new Promise((resolve) => {
if (signal?.aborted) return resolve();
const timer = setTimeout(resolve, ms);
signal?.addEventListener(
"abort",
() => {
clearTimeout(timer);
resolve();
},
{ once: true },
);
});
}
function isTargeted(draft: RecordingDraftStep): draft is TargetedRecordingDraft {
return (
draft.op === "click" ||
draft.op === "hover" ||
draft.op === "fill" ||
draft.op === "press" ||
draft.op === "select"
);
}
export class RecordingObservationSession {
readonly registry: RecordingStateRegistry;
readonly cursor: TabObservationCursor;
readonly annotations: StepAnnotation[];
readonly #maxTokens: number;
readonly #redactValues: boolean;
constructor(
options: {
registry?: RecordingStateRegistry;
cursor?: TabObservationCursor;
annotations?: StepAnnotation[];
maxTokens?: number;
redactValues?: boolean;
} = {},
) {
this.registry = options.registry ?? new RecordingStateRegistry();
this.cursor = options.cursor ?? { lastSettled: null, lastCaptureAt: 0 };
this.annotations = options.annotations ?? [];
this.#maxTokens = options.maxTokens ?? DEFAULT_MAX_PAGE_TOKENS;
this.#redactValues = options.redactValues ?? false;
}
async capture(
cdp: CdpRunner,
tabsApi: ChromeTabsApi,
tabId: number,
signal?: AbortSignal,
): Promise<RegisteredObservation> {
const waitMs = Math.max(0, MIN_CAPTURE_INTERVAL_MS - (Date.now() - this.cursor.lastCaptureAt));
if (waitMs > 0) await abortableDelay(waitMs, signal);
if (signal?.aborted) throw new DOMException("observation aborted", "AbortError");
const captured = await captureRecordingObservation({
cdp,
tabsApi,
tabId,
maxTokens: this.#maxTokens,
redactValues: this.#redactValues,
signal,
});
const state = this.registry.register({
url: captured.url,
title: captured.title,
vomText: captured.vomText,
truncated: captured.truncated,
});
const observation: RegisteredObservation = {
stateId: state.id,
rootFrameId: captured.rootFrameId,
index: captured.index,
url: captured.url,
};
this.cursor.lastSettled = observation;
this.cursor.lastCaptureAt = Date.now();
return observation;
}
bindDraft(draft: RecordingDraftStep, draftId: number, previousActionPending = false): void {
const observation = this.cursor.lastSettled;
if (isTargeted(draft)) {
draft.matchedTarget = observation
? matchObservationTarget({
observation,
hint: draft.targetHint,
fallback: draft.captureTarget,
})
: unmatchedTarget(draft.captureTarget);
}
if (!observation) return;
const unmatched = isTargeted(draft) && draft.matchedTarget?.unmatched === true;
if (previousActionPending && unmatched) return;
draft.preStateId = observation.stateId;
this.registry.markStep(observation.stateId, draftId);
if (!isTargeted(draft) || !draft.matchedTarget?.ref) return;
const ref = observation.index.ref(draft.matchedTarget.ref);
if (!ref) return;
this.annotations.push({
draftId,
op: draft.op,
line: ref.line,
stateId: observation.stateId,
...(draft.op === "fill" && !this.#redactValues
? { detail: JSON.stringify(draft.value) }
: {}),
});
}
}
@@ -0,0 +1,219 @@
import type { CdpRunner, ChromeTabsApi } from "@/tools/shared";
import type { StopReason, TraceV3 } from "@/transport/types";
import type { DocumentSettleScope } from "./document-settle";
import type { RegisteredObservation } from "./observation-capture";
import { RecordingObservationSession } from "./observation-session";
import { inferMissingPostStates, SettleController } from "./settle-controller";
import { RecordingStateRegistry } from "./state-registry";
import { buildTraceV3 } from "./trace-builder-v3";
import type { RecordingDraftStep, StepAnnotation } from "./types";
interface TabRecordingContext {
session: RecordingObservationSession;
settle: SettleController;
pendingCapture: PendingCapture | null;
}
interface PendingCapture {
promise: Promise<RegisteredObservation>;
abort: AbortController;
}
export class RecordingObservationRuntime {
readonly #cdp: CdpRunner;
readonly #tabsApi: ChromeTabsApi;
readonly #registry = new RecordingStateRegistry();
readonly #annotations: StepAnnotation[] = [];
readonly #contexts = new Map<number, TabRecordingContext>();
readonly #maxTokens?: number;
readonly #redactValues: boolean;
constructor(input: {
cdp: CdpRunner;
tabsApi: ChromeTabsApi;
maxTokens?: number;
redactValues?: boolean;
}) {
this.#cdp = input.cdp;
this.#tabsApi = input.tabsApi;
this.#maxTokens = input.maxTokens;
this.#redactValues = input.redactValues ?? false;
}
#context(tabId: number): TabRecordingContext {
const existing = this.#contexts.get(tabId);
if (existing) return existing;
const session = new RecordingObservationSession({
registry: this.#registry,
annotations: this.#annotations,
maxTokens: this.#maxTokens,
redactValues: this.#redactValues,
});
const context: TabRecordingContext = {
session,
settle: new SettleController({
session,
cdp: this.#cdp,
tabsApi: this.#tabsApi,
tabId,
}),
pendingCapture: null,
};
this.#contexts.set(tabId, context);
return context;
}
async #capture(tabId: number): Promise<RegisteredObservation> {
const context = this.#context(tabId);
if (context.pendingCapture) return context.pendingCapture.promise;
const abort = new AbortController();
let pending!: PendingCapture;
const promise = context.session.capture(this.#cdp, this.#tabsApi, tabId, abort.signal);
pending = {
abort,
promise: promise.finally(() => {
if (context.pendingCapture === pending) context.pendingCapture = null;
}),
};
context.pendingCapture = pending;
return pending.promise;
}
async captureInitial(tabId: number): Promise<void> {
const context = this.#context(tabId);
if (context.session.cursor.lastSettled) return;
await this.#capture(tabId);
}
async captureTabTransition(
fromTabId: number,
toTabId: number,
): Promise<{ preStateId?: string; postStateId?: string; targetUrl?: string }> {
const from = this.#context(fromTabId);
await this.#flushContext(from);
if (!from.session.cursor.lastSettled) {
try {
await this.captureInitial(fromTabId);
} catch {
// A target-tab observation can still preserve the forward transition.
}
}
const to = this.#context(toTabId);
let target = to.session.cursor.lastSettled;
try {
target = await this.#capture(toTabId);
} catch {
// Reuse the last settled target state when a refresh races navigation.
}
const source = from.session.cursor.lastSettled;
return {
...(source ? { preStateId: source.stateId } : {}),
...(target ? { postStateId: target.stateId, targetUrl: target.url } : {}),
};
}
bindTabTransition(
draft: Extract<RecordingDraftStep, { op: "switch_tab" }>,
draftId: number,
): void {
if (draft.preStateId) this.#registry.markStep(draft.preStateId, draftId);
}
async processDraft(
tabId: number,
drafts: RecordingDraftStep[],
draftIndex: number,
scope?: DocumentSettleScope,
): Promise<void> {
const draft = drafts[draftIndex];
if (!draft) return;
const context = this.#context(tabId);
if (!context.session.cursor.lastSettled) {
try {
await this.captureInitial(tabId);
} catch {
// Post-action settle can still provide a usable state.
}
}
context.session.bindDraft(draft, draftIndex + 1, context.settle.hasPending);
context.settle.schedule(drafts, draftIndex, scope);
}
scheduleSettle(
tabId: number,
drafts: RecordingDraftStep[],
draftIndex: number,
scope?: DocumentSettleScope,
): void {
this.#context(tabId).settle.schedule(drafts, draftIndex, scope);
}
clearRedirect(tabId: number): void {
this.#contexts.get(tabId)?.settle.clearRedirect();
}
scheduleRedirect(tabId: number, drafts: RecordingDraftStep[], url: string): void {
this.#context(tabId).settle.scheduleRedirect(drafts, url);
}
async #flushContext(context: TabRecordingContext): Promise<void> {
if (context.pendingCapture) await Promise.allSettled([context.pendingCapture.promise]);
await context.settle.flushRedirects();
await context.settle.flush();
}
async flushRedirects(tabId: number): Promise<void> {
await this.#contexts.get(tabId)?.settle.flushRedirects();
}
async flush(): Promise<void> {
for (const context of this.#contexts.values()) await this.#flushContext(context);
}
async settleTrailing(tabId: number, drafts: RecordingDraftStep[]): Promise<void> {
const context = this.#context(tabId);
if (context.pendingCapture) await Promise.allSettled([context.pendingCapture.promise]);
if (!context.session.cursor.lastSettled) {
try {
await this.captureInitial(tabId);
} catch {
// The trace may remain observation-free when CDP is unavailable.
}
}
await context.settle.settleTrailing(drafts);
inferMissingPostStates(drafts);
}
cancel(): void {
for (const context of this.#contexts.values()) {
context.pendingCapture?.abort.abort();
context.settle.cancel();
}
}
buildTrace(input: {
drafts: RecordingDraftStep[];
startedAt: string;
purpose?: string;
startUrl?: string;
stoppedBy: StopReason;
bskVersion: string;
includeTabSwitches?: boolean;
}): TraceV3 {
return buildTraceV3({
registry: this.#registry,
drafts: input.drafts,
annotations: this.#annotations,
startedAt: input.startedAt,
purpose: input.purpose,
startUrl: input.startUrl,
stoppedBy: input.stoppedBy,
bskVersion: input.bskVersion,
redactValues: this.#redactValues,
includeTabSwitches: input.includeTabSwitches,
});
}
}
@@ -0,0 +1,233 @@
import type { CdpRunner, ChromeTabsApi } from "@/tools/shared";
import { type DocumentSettleScope, waitForDocumentSettled } from "./document-settle";
import { RecordingObservationSession } from "./observation-session";
import type { RecordingDraftStep } from "./types";
interface PendingSettle {
abort: AbortController;
scope: DocumentSettleScope;
}
interface PendingRedirect {
url: string;
abort: AbortController;
}
const CAPTURE_RETRY_DELAY_MS = 250;
function delay(ms: number, signal?: AbortSignal): Promise<void> {
return new Promise((resolve) => {
if (signal?.aborted) return resolve();
const timer = setTimeout(resolve, ms);
signal?.addEventListener(
"abort",
() => {
clearTimeout(timer);
resolve();
},
{ once: true },
);
});
}
function isAbortError(error: unknown): boolean {
return (
typeof error === "object" &&
error !== null &&
(error as { name?: string }).name === "AbortError"
);
}
export function inferMissingPostStates(drafts: RecordingDraftStep[]): void {
for (let index = 0; index < drafts.length - 1; index += 1) {
const draft = drafts[index];
const next = drafts[index + 1];
if (draft && next && !draft.postStateId && next.preStateId) {
draft.postStateId = next.preStateId;
}
}
}
export class SettleController {
readonly #session: RecordingObservationSession;
readonly #cdp: CdpRunner;
readonly #tabsApi: ChromeTabsApi;
readonly #tabId: number;
readonly #rootScope: DocumentSettleScope;
readonly #pending = new Map<number, PendingSettle>();
#queue = Promise.resolve();
#pendingRedirect: PendingRedirect | null = null;
#redirectQueue = Promise.resolve();
constructor(input: {
session: RecordingObservationSession;
cdp: CdpRunner;
tabsApi: ChromeTabsApi;
tabId: number;
}) {
this.#session = input.session;
this.#cdp = input.cdp;
this.#tabsApi = input.tabsApi;
this.#tabId = input.tabId;
this.#rootScope = { target: { tabId: input.tabId } };
}
get hasPending(): boolean {
return this.#pending.size > 0 || this.#pendingRedirect !== null;
}
async #captureWithRetry(signal?: AbortSignal) {
try {
return await this.#session.capture(this.#cdp, this.#tabsApi, this.#tabId, signal);
} catch (error) {
if (signal?.aborted) throw error;
await delay(CAPTURE_RETRY_DELAY_MS, signal);
return this.#session.capture(this.#cdp, this.#tabsApi, this.#tabId, signal);
}
}
schedule(
drafts: RecordingDraftStep[],
draftIndex: number,
scope: DocumentSettleScope = this.#rootScope,
): void {
if (scope.target.tabId !== this.#tabId) {
throw new Error(
`settle scope tab ${scope.target.tabId} does not belong to tab ${this.#tabId}`,
);
}
const landing = drafts[draftIndex]?.preStateId;
for (const [index, pending] of this.#pending) {
if (index >= draftIndex) continue;
pending.abort.abort();
this.#pending.delete(index);
if (landing && drafts[index] && !drafts[index].postStateId) {
drafts[index].postStateId = landing;
}
}
this.#pending.get(draftIndex)?.abort.abort();
const pending = { abort: new AbortController(), scope };
this.#pending.set(draftIndex, pending);
const task = async () => {
if (pending.abort.signal.aborted) return;
try {
const outcome = await waitForDocumentSettled(this.#cdp, pending.scope, {
signal: pending.abort.signal,
});
if (outcome === "cancelled") return;
const observation = await this.#captureWithRetry(pending.abort.signal);
if (!pending.abort.signal.aborted && drafts[draftIndex]) {
drafts[draftIndex].postStateId = observation.stateId;
}
} catch (error) {
if (!isAbortError(error)) {
console.warn(
`[bsk record] post-action observation failed for step ${draftIndex + 1}`,
error,
);
}
} finally {
if (this.#pending.get(draftIndex) === pending) this.#pending.delete(draftIndex);
}
};
this.#queue = this.#queue.then(task, task).catch(() => {});
}
cancel(): void {
for (const pending of this.#pending.values()) pending.abort.abort();
this.#pending.clear();
this.clearRedirect();
}
async flush(): Promise<void> {
for (let round = 0; round < 10; round += 1) {
const current = this.#queue;
await current;
if (current === this.#queue) return;
}
console.warn("[bsk record] settle queue kept growing; using completed observations");
}
async settleTrailing(drafts: RecordingDraftStep[]): Promise<void> {
const trailing: RecordingDraftStep[] = [];
for (let index = drafts.length - 1; index >= 0; index -= 1) {
const draft = drafts[index];
if (!draft || draft.postStateId) break;
trailing.push(draft);
}
if (trailing.length === 0) return;
try {
const observation = await this.#captureWithRetry();
for (const draft of trailing) draft.postStateId = observation.stateId;
} catch (error) {
console.warn("[bsk record] final observation at stop failed", error);
}
}
clearRedirect(): void {
this.#pendingRedirect?.abort.abort();
this.#pendingRedirect = null;
}
scheduleRedirect(drafts: RecordingDraftStep[], url: string): void {
this.#pendingRedirect?.abort.abort();
const pending: PendingRedirect = { url, abort: new AbortController() };
this.#pendingRedirect = pending;
const task = () => this.#settleRedirect(drafts, pending);
this.#redirectQueue = this.#redirectQueue.then(task, task).catch(() => {});
}
async #settleRedirect(drafts: RecordingDraftStep[], pending: PendingRedirect): Promise<void> {
const outcome = await waitForDocumentSettled(this.#cdp, this.#rootScope, {
signal: pending.abort.signal,
});
if (outcome === "cancelled" || this.#pendingRedirect !== pending) return;
let finalUrl = pending.url;
try {
finalUrl = (await this.#tabsApi.get(this.#tabId)).url || finalUrl;
} catch {
// The navigation event URL remains the best available destination.
}
if (this.#pendingRedirect !== pending) return;
this.#pendingRedirect = null;
if (
!finalUrl ||
finalUrl === "about:blank" ||
this.#session.cursor.lastSettled?.url === finalUrl
) {
return;
}
const last = drafts[drafts.length - 1];
if (last?.op === "navigate" && last.url === finalUrl) {
if (!last.postStateId) this.schedule(drafts, drafts.length - 1);
await this.flush();
return;
}
const draft: RecordingDraftStep = {
op: "navigate",
url: finalUrl,
pageUrl: finalUrl,
cause: "browser",
transitionQualifiers: ["server_redirect"],
preStateId: this.#session.cursor.lastSettled?.stateId,
};
drafts.push(draft);
const draftIndex = drafts.length - 1;
if (draft.preStateId) this.#session.registry.markStep(draft.preStateId, draftIndex + 1);
this.schedule(drafts, draftIndex);
await this.flush();
}
async flushRedirects(): Promise<void> {
for (let round = 0; round < 10; round += 1) {
const current = this.#redirectQueue;
await current;
if (current === this.#redirectQueue) return;
}
console.warn("[bsk record] redirect queue kept growing; using the final completed landing");
}
}
@@ -0,0 +1,57 @@
export interface RecordedStateEntry {
id: string;
url: string;
title?: string;
/** Rendered VOM text; capture indexes and source DOM/AX data never enter the registry. */
vomText: string;
truncated: boolean;
stepsHere: number[];
}
function stateIdentity(url: string, body: string): string {
return `${url}\0${body}`;
}
export class RecordingStateRegistry {
readonly #entriesById = new Map<string, RecordedStateEntry>();
readonly #idByIdentity = new Map<string, string>();
#nextId = 1;
register(input: {
url: string;
title?: string;
vomText: string;
truncated?: boolean;
}): RecordedStateEntry {
const identity = stateIdentity(input.url, input.vomText);
const existingId = this.#idByIdentity.get(identity);
if (existingId) {
const existing = this.#entriesById.get(existingId)!;
if (!existing.title && input.title) existing.title = input.title;
if (input.truncated) existing.truncated = true;
return existing;
}
const entry: RecordedStateEntry = {
id: `s${this.#nextId}`,
url: input.url,
...(input.title ? { title: input.title } : {}),
vomText: input.vomText,
truncated: input.truncated ?? false,
stepsHere: [],
};
this.#nextId += 1;
this.#entriesById.set(entry.id, entry);
this.#idByIdentity.set(identity, entry.id);
return entry;
}
values(): RecordedStateEntry[] {
return [...this.#entriesById.values()];
}
markStep(stateId: string, draftId: number): void {
const entry = this.#entriesById.get(stateId);
if (entry && !entry.stepsHere.includes(draftId)) entry.stepsHere.push(draftId);
}
}
@@ -0,0 +1,155 @@
import type { RecordStepPayload } from "../record-bridge";
import { hasRedirectQualifier } from "./navigation-policy";
import type { RecordingDraftStep, TargetMatchHint } from "./types";
export interface RecordingStepBuffer {
steps: RecordingDraftStep[];
navigation: RecordingNavigationCursor;
}
export interface RecordingNavigationCursor {
currentUrl?: string;
pendingNavigation: boolean;
pendingNavigationDeadline?: number;
}
const NAVIGATION_TRIGGER_WINDOW_MS = 3_000;
function toDraftStep(
payload: RecordStepPayload,
targetHint?: TargetMatchHint,
): RecordingDraftStep | null {
const pageUrl = payload.page_url;
const common = {
...(pageUrl ? { pageUrl } : {}),
...(targetHint ? { targetHint } : {}),
};
switch (payload.op) {
case "click":
return payload.target ? { op: "click", captureTarget: payload.target, ...common } : null;
case "hover":
return payload.target ? { op: "hover", captureTarget: payload.target, ...common } : null;
case "fill":
return payload.target
? {
op: "fill",
captureTarget: payload.target,
value: payload.value ?? "",
...(payload.commit ? { commit: payload.commit } : {}),
...(payload.redacted ? { redacted: true } : {}),
...common,
}
: null;
case "press":
return payload.key
? {
op: "press",
key: payload.key,
...(payload.target ? { captureTarget: payload.target } : {}),
...(payload.modifiers?.length ? { modifiers: payload.modifiers } : {}),
...common,
}
: null;
case "select":
return payload.target && payload.values
? {
op: "select",
captureTarget: payload.target,
values: payload.values,
...(payload.labels?.length ? { labels: payload.labels } : {}),
...common,
}
: null;
case "navigate":
return null;
}
}
function annotateLastStepNavigation(buffer: RecordingStepBuffer, url: string): number {
for (let index = buffer.steps.length - 1; index >= 0; index -= 1) {
const step = buffer.steps[index];
if (!step) continue;
if (step.op === "click" || step.op === "press" || step.op === "select" || step.op === "fill") {
step.navigatedTo = url;
return index;
}
break;
}
return -1;
}
export type NavigationObserveResult =
| { kind: "noop" }
| { kind: "annotated"; index: number }
| { kind: "appended"; index: number }
| { kind: "coalesce_redirect"; url: string };
export function observeRecordedNavigation(
buffer: RecordingStepBuffer,
url: string,
causedByAction?: boolean,
transitionType?: string,
transitionQualifiers?: string[],
): NavigationObserveResult {
const navigation = buffer.navigation;
if (!url || url === navigation.currentUrl) return { kind: "noop" };
navigation.currentUrl = url;
const pendingIsCurrent =
navigation.pendingNavigation &&
(navigation.pendingNavigationDeadline === undefined ||
navigation.pendingNavigationDeadline >= Date.now());
if (causedByAction === true || (causedByAction === undefined && pendingIsCurrent)) {
navigation.pendingNavigation = false;
navigation.pendingNavigationDeadline = undefined;
const annotatedIndex = annotateLastStepNavigation(buffer, url);
if (annotatedIndex >= 0) return { kind: "annotated", index: annotatedIndex };
} else {
navigation.pendingNavigation = false;
navigation.pendingNavigationDeadline = undefined;
}
if (hasRedirectQualifier(transitionQualifiers)) {
return { kind: "coalesce_redirect", url };
}
buffer.steps.push({
op: "navigate",
url,
pageUrl: url,
transitionType,
transitionQualifiers,
});
return { kind: "appended", index: buffer.steps.length - 1 };
}
export function appendRecordedPayload(
buffer: RecordingStepBuffer,
payload: RecordStepPayload,
targetHint?: TargetMatchHint,
): number | null {
if (payload.op === "navigate") {
if (!payload.url) return null;
const result = observeRecordedNavigation(
buffer,
payload.url,
payload.navigation_caused_by_action,
payload.transitionType,
payload.transitionQualifiers,
);
return result.kind === "appended" ? result.index : null;
}
const step = toDraftStep(
{ ...payload, page_url: payload.page_url ?? buffer.navigation.currentUrl },
targetHint,
);
if (!step) return null;
buffer.steps.push(step);
if (step.op === "click" || step.op === "press" || step.op === "select" || step.op === "fill") {
buffer.navigation.pendingNavigation = payload.expects_navigation === true;
buffer.navigation.pendingNavigationDeadline = buffer.navigation.pendingNavigation
? Date.now() + NAVIGATION_TRIGGER_WINDOW_MS
: undefined;
}
return buffer.steps.length - 1;
}
@@ -0,0 +1,61 @@
import type { RecordingNavigationCursor } from "./step-buffer";
export interface TabActivation {
tabId: number;
revision: number;
}
export class RecordingTabCoordinator {
readonly #navigationByTab = new Map<number, RecordingNavigationCursor>();
#activeTabId: number;
#currentTabId: number;
#activationRevision = 0;
constructor(initialTabId: number, initialUrl?: string) {
this.#activeTabId = initialTabId;
this.#currentTabId = initialTabId;
this.#navigationByTab.set(initialTabId, {
currentUrl: initialUrl,
pendingNavigation: false,
});
}
get activeTabId(): number {
return this.#activeTabId;
}
get currentTabId(): number {
return this.#currentTabId;
}
noteActivation(tabId: number): TabActivation {
this.#activeTabId = tabId;
this.#activationRevision += 1;
return { tabId, revision: this.#activationRevision };
}
isLatest(activation: TabActivation): boolean {
return (
activation.tabId === this.#activeTabId && activation.revision === this.#activationRevision
);
}
commit(tabId: number, currentUrl?: string): void {
if (currentUrl !== undefined) this.navigation(tabId).currentUrl = currentUrl;
this.#currentTabId = tabId;
}
navigation(tabId: number, fallbackUrl?: string): RecordingNavigationCursor {
const existing = this.#navigationByTab.get(tabId);
if (existing) {
if (!existing.currentUrl && fallbackUrl) existing.currentUrl = fallbackUrl;
return existing;
}
const created: RecordingNavigationCursor = {
currentUrl: fallbackUrl,
pendingNavigation: false,
};
this.#navigationByTab.set(tabId, created);
return created;
}
}
@@ -0,0 +1,76 @@
import type { RenderedRef } from "@browser-skill/vom";
import type { TargetDescriptorV3 } from "@/transport/types";
import type { CaptureTargetDescriptor } from "../describe-target";
import type { IndexedObservationNode, RegisteredObservation } from "./observation-capture";
import type { TargetGeometry, TargetMatchHint } from "./types";
const MATCH_TOLERANCE_PX = 2;
function close(a: number, b: number): boolean {
return Math.abs(a - b) <= MATCH_TOLERANCE_PX;
}
function rectMatches(a: TargetGeometry["rect"], b: TargetGeometry["rect"]): boolean {
return close(a.x, b.x) && close(a.y, b.y) && close(a.w, b.w) && close(a.h, b.h);
}
function candidateMatches(candidate: IndexedObservationNode, geometry: TargetGeometry): boolean {
return candidate.geometry.rect !== null && rectMatches(geometry.rect, candidate.geometry.rect);
}
export function unmatchedTarget(fallback?: CaptureTargetDescriptor): TargetDescriptorV3 {
return {
...(fallback?.role ? { role: fallback.role } : {}),
...(fallback?.name ? { name: fallback.name } : {}),
unmatched: true,
};
}
function normalized(value: string | undefined): string {
return value?.replace(/\s+/g, " ").trim().toLowerCase() ?? "";
}
function matchesSemantics(ref: RenderedRef, fallback?: CaptureTargetDescriptor): boolean {
const role = normalized(fallback?.role);
const name = normalized(fallback?.name);
if (!role && !name) return false;
if (role && normalized(ref.role) !== role) return false;
if (name && normalized(ref.name) !== name) return false;
return true;
}
function descriptor(ref: RenderedRef): TargetDescriptorV3 {
return {
ref: ref.ref,
...(ref.role ? { role: ref.role } : {}),
...(ref.name ? { name: ref.name } : {}),
...(ref.ctx ? { ctx: ref.ctx } : {}),
};
}
export function matchObservationTarget(input: {
observation: RegisteredObservation;
hint?: TargetMatchHint;
fallback?: CaptureTargetDescriptor;
}): TargetDescriptorV3 {
const frameId = input.hint?.frameId ?? input.observation.rootFrameId;
const geometry = input.hint?.geometry;
if (geometry) {
const matches = input.observation.index
.candidates(frameId, geometry.tag)
.filter((candidate) => candidate.ref && candidateMatches(candidate, geometry));
if (matches.length === 1) return descriptor(matches[0]!.ref!);
const semanticMatches = matches.filter(
(candidate) => candidate.ref && matchesSemantics(candidate.ref, input.fallback),
);
if (semanticMatches.length === 1) return descriptor(semanticMatches[0]!.ref!);
return unmatchedTarget(input.fallback);
}
const semanticMatches = input.observation.index
.refs(frameId)
.filter((ref) => matchesSemantics(ref, input.fallback));
return semanticMatches.length === 1
? descriptor(semanticMatches[0]!)
: unmatchedTarget(input.fallback);
}
@@ -0,0 +1,92 @@
import {
type StepV3,
type StopReason,
TRACE_VERSION_V3,
type TraceStateV3,
type TraceV3,
VOM_FORMAT_VERSION,
} from "@/transport/types";
import { resolveDraftStartUrl } from "./draft-policy";
import type { RecordedStateEntry, RecordingStateRegistry } from "./state-registry";
import { reduceTraceStepsV3 } from "./trace-reducer-v3";
import { formatTraceStateBody } from "./trace-state-body";
import type { RecordingDraftStep, StepAnnotation } from "./types";
function publishedEntries(registry: RecordingStateRegistry, steps: StepV3[]): RecordedStateEntry[] {
const entries = registry.values();
if (steps.length === 0) return entries.slice(0, 1);
const referenced = new Set(steps.flatMap((step) => [step.state, step.result.state]));
return entries.filter((entry) => referenced.has(entry.id));
}
function remapDraftIds(draftIds: number[], stepIdByDraftId: Map<number, number>): number[] {
return [
...new Set(
draftIds.flatMap((id) => {
const stepId = stepIdByDraftId.get(id);
return stepId === undefined ? [] : [stepId];
}),
),
].sort((a, b) => a - b);
}
export function buildTraceV3(input: {
registry: RecordingStateRegistry;
drafts: RecordingDraftStep[];
annotations?: StepAnnotation[];
startedAt: string;
purpose?: string;
startUrl?: string;
stoppedBy: StopReason;
bskVersion: string;
redactValues?: boolean;
includeTabSwitches?: boolean;
}): TraceV3 {
const reduced = reduceTraceStepsV3(input.drafts, {
includeTabSwitches: input.includeTabSwitches,
redactValues: input.redactValues,
});
const entries = publishedEntries(input.registry, reduced.steps);
const publishedId = new Map(entries.map((entry, index) => [entry.id, `s${index + 1}`]));
const annotationsByState = new Map<string, StepAnnotation[]>();
for (const annotation of input.annotations ?? []) {
const bucket = annotationsByState.get(annotation.stateId) ?? [];
bucket.push(annotation);
annotationsByState.set(annotation.stateId, bucket);
}
const steps = reduced.steps.map((step) => ({
...step,
state: publishedId.get(step.state) ?? step.state,
result: { state: publishedId.get(step.result.state) ?? step.result.state },
}));
const states: TraceStateV3[] = entries.map((entry) => {
const id = publishedId.get(entry.id) ?? entry.id;
return {
id,
url: entry.url,
...(entry.title ? { title: entry.title } : {}),
body: formatTraceStateBody({
stateId: id,
url: entry.url,
title: entry.title,
stepIds: remapDraftIds(entry.stepsHere, reduced.stepIdByDraftId),
vomText: entry.vomText,
annotations: annotationsByState.get(entry.id) ?? [],
stepIdByDraftId: reduced.stepIdByDraftId,
}),
...(entry.truncated ? { truncated: true } : {}),
};
});
return {
version: TRACE_VERSION_V3,
...(input.purpose ? { purpose: input.purpose } : {}),
recorded_at: new Date().toISOString(),
started_at: input.startedAt,
stopped_by: input.stoppedBy,
entry: { start_url: resolveDraftStartUrl(input.drafts, input.startUrl, states[0]?.url) },
recorder: { bsk: input.bskVersion, vom: VOM_FORMAT_VERSION },
states,
steps,
};
}
@@ -1,30 +1,10 @@
import type { DraftTraceStep, PageRef, SelectedOption, Step } from "@/transport/types";
const CLIPBOARD_KEYS = new Set(["a", "c", "v", "x", "A", "C", "V", "X"]);
const MODIFIER_ONLY_KEYS = new Set(["Meta", "Control", "Alt", "Shift", "OS", "Hyper", "Super"]);
export function shouldRecordPress(
key: string,
modifiers?: Array<"alt" | "ctrl" | "meta" | "shift">,
): boolean {
if (MODIFIER_ONLY_KEYS.has(key)) return false;
const mods = modifiers ?? [];
const hasCtrlOrMeta = mods.includes("ctrl") || mods.includes("meta");
if (hasCtrlOrMeta && CLIPBOARD_KEYS.has(key)) return false;
if (key === "Enter" || key === "Escape") return true;
// Drop bare character typing — FillSession already records the value.
if (key.length === 1 && !hasCtrlOrMeta && !mods.includes("alt")) return false;
return false;
}
function shouldIncludeDraft(step: DraftTraceStep): boolean {
if (step.op === "press" && !shouldRecordPress(step.key, step.modifiers)) return false;
return true;
}
import type { PageRefV2, SelectedOptionV2, StepV2, TraceV2 } from "@/transport/types";
import { resolveDraftStartUrl, shouldIncludeDraft } from "./draft-policy";
import type { RecordingDraftStep } from "./types";
/** Collapse consecutive navigations to the last hop. */
function collapseNavigations(steps: DraftTraceStep[]): DraftTraceStep[] {
const out: DraftTraceStep[] = [];
function collapseNavigations(steps: RecordingDraftStep[]): RecordingDraftStep[] {
const out: RecordingDraftStep[] = [];
for (const step of steps) {
const prev = out[out.length - 1];
if (step.op === "navigate" && prev?.op === "navigate") {
@@ -36,7 +16,7 @@ function collapseNavigations(steps: DraftTraceStep[]): DraftTraceStep[] {
return out;
}
function collectUrls(steps: DraftTraceStep[], startUrl?: string): string[] {
function collectUrls(steps: RecordingDraftStep[], startUrl?: string): string[] {
const urls: string[] = [];
if (startUrl) urls.push(startUrl);
for (const step of steps) {
@@ -44,8 +24,8 @@ function collectUrls(steps: DraftTraceStep[], startUrl?: string): string[] {
urls.push(step.url);
continue;
}
if ("page_url" in step && step.page_url) urls.push(step.page_url);
if ("navigated_to" in step && step.navigated_to) urls.push(step.navigated_to);
if (step.pageUrl) urls.push(step.pageUrl);
if ("navigatedTo" in step && step.navigatedTo) urls.push(step.navigatedTo);
}
const seen = new Set<string>();
const unique: string[] = [];
@@ -58,9 +38,9 @@ function collectUrls(steps: DraftTraceStep[], startUrl?: string): string[] {
}
function buildPageRegistry(
steps: DraftTraceStep[],
steps: RecordingDraftStep[],
startUrl?: string,
): { pages: PageRef[]; urlToId: Map<string, string> } {
): { pages: PageRefV2[]; urlToId: Map<string, string> } {
const urls = collectUrls(steps, startUrl);
const urlToId = new Map<string, string>();
const pages = urls.map((url, index) => {
@@ -81,28 +61,28 @@ function pageIdFor(
return urlToId.values().next().value ?? "p1";
}
function pageUrlForDraft(step: DraftTraceStep, fallbackUrl?: string): string | undefined {
if (step.op === "navigate") return step.page_url ?? step.url;
if ("page_url" in step && step.page_url) return step.page_url;
function pageUrlForDraft(step: RecordingDraftStep, fallbackUrl?: string): string | undefined {
if (step.op === "navigate") return step.pageUrl ?? step.url;
if (step.pageUrl) return step.pageUrl;
return fallbackUrl;
}
function effectForNavigation(
navigatedTo: string | undefined,
urlToId: Map<string, string>,
): Step["effect"] {
): StepV2["effect"] {
if (!navigatedTo) return undefined;
const pageId = urlToId.get(navigatedTo);
if (!pageId) return undefined;
return { navigated_to: pageId };
}
function withEffect(step: Step, effect: Step["effect"]): Step {
function withEffect(step: StepV2, effect: StepV2["effect"]): StepV2 {
if (!effect) return step;
return { ...step, effect };
}
function toSelection(values: string[], labels?: string[]): SelectedOption[] {
function toSelection(values: string[], labels?: string[]): SelectedOptionV2[] {
return values.map((value, index) => ({
value,
...(labels?.[index] ? { label: labels[index] } : {}),
@@ -110,11 +90,11 @@ function toSelection(values: string[], labels?: string[]): SelectedOption[] {
}
function toV2Step(
step: DraftTraceStep,
step: RecordingDraftStep,
id: number,
urlToId: Map<string, string>,
fallbackUrl?: string,
): Step | null {
): StepV2 | null {
if (!shouldIncludeDraft(step)) return null;
const pageUrl = pageUrlForDraft(step, fallbackUrl);
@@ -129,28 +109,23 @@ function toV2Step(
to: step.url,
};
case "click":
if (!step.captureTarget) return null;
return withEffect(
{
op: "click",
id,
page,
target: step.target,
target: step.captureTarget,
},
effectForNavigation(step.navigated_to, urlToId),
effectForNavigation(step.navigatedTo, urlToId),
);
case "hover":
return {
op: "hover",
id,
page,
target: step.target,
};
case "fill":
if (!step.captureTarget) return null;
return {
op: "fill",
id,
page,
target: step.target,
target: step.captureTarget,
value: step.value,
...(step.redacted ? { redacted: true } : {}),
};
@@ -161,44 +136,51 @@ function toV2Step(
id,
page,
key: step.key,
...(step.target ? { target: step.target } : {}),
...(step.captureTarget ? { target: step.captureTarget } : {}),
...(step.modifiers?.length ? { modifiers: step.modifiers } : {}),
},
effectForNavigation(step.navigated_to, urlToId),
effectForNavigation(step.navigatedTo, urlToId),
);
case "select":
if (!step.captureTarget) return null;
return withEffect(
{
op: "select",
id,
page,
target: step.target,
target: step.captureTarget,
selection: toSelection(step.values, step.labels),
},
effectForNavigation(step.navigated_to, urlToId),
effectForNavigation(step.navigatedTo, urlToId),
);
// `hover` only exists in Trace v3. Peers that negotiate v2 predate the
// step variant and fail to decode the whole result if we emit it.
case "hover":
case "scroll":
case "switch_tab":
return null;
}
}
export interface ReducedTrace {
pages: PageRef[];
steps: Step[];
interface ReducedTrace {
pages: PageRefV2[];
steps: StepV2[];
}
/**
* Compile capture drafts into record-only trace v2 steps.
* Variable inputs are NOT classified here — executing agents infer that at run time.
*/
export function reduceTraceSteps(steps: DraftTraceStep[], startUrl?: string): ReducedTrace {
function reduceTraceSteps(steps: RecordingDraftStep[], startUrl?: string): ReducedTrace {
const collapsed = collapseNavigations(steps);
const { pages, urlToId } = buildPageRegistry(collapsed, startUrl);
const out: Step[] = [];
const out: StepV2[] = [];
let id = 1;
let lastUrl = startUrl;
for (const draft of collapsed) {
if (draft.op === "navigate") lastUrl = draft.url;
else if ("navigated_to" in draft && draft.navigated_to) lastUrl = draft.navigated_to;
else if ("page_url" in draft && draft.page_url) lastUrl = draft.page_url;
else if ("navigatedTo" in draft && draft.navigatedTo) lastUrl = draft.navigatedTo;
else if (draft.pageUrl) lastUrl = draft.pageUrl;
const step = toV2Step(draft, id, urlToId, lastUrl);
if (!step) continue;
out.push(step);
@@ -207,18 +189,27 @@ export function reduceTraceSteps(steps: DraftTraceStep[], startUrl?: string): Re
return { pages, steps: out };
}
export function resolveTraceStartUrl(
drafts: DraftTraceStep[],
function resolveTraceStartUrl(
drafts: RecordingDraftStep[],
startUrl?: string,
pages?: PageRef[],
pages?: PageRefV2[],
): string {
if (startUrl) return startUrl;
const navigate = drafts.find((step): step is Extract<DraftTraceStep, { op: "navigate" }> => {
return step.op === "navigate";
});
if (navigate) return navigate.url;
for (const draft of drafts) {
if ("page_url" in draft && draft.page_url) return draft.page_url;
}
return pages?.[0]?.url ?? "about:blank";
return resolveDraftStartUrl(drafts, startUrl, pages?.[0]?.url);
}
export function buildTraceV2(input: {
steps: RecordingDraftStep[];
startedAt: string;
startUrl?: string;
purpose?: string;
}): TraceV2 {
const { pages, steps } = reduceTraceSteps(input.steps, input.startUrl);
return {
recorded_at: new Date().toISOString(),
started_at: input.startedAt,
...(input.purpose ? { purpose: input.purpose } : {}),
entry: { start_url: resolveTraceStartUrl(input.steps, input.startUrl, pages) },
pages,
steps,
};
}
@@ -0,0 +1,154 @@
import type { NavigationCause, StepV3 } from "@/transport/types";
import { shouldIncludeDraft } from "./draft-policy";
import { hasRedirectQualifier } from "./navigation-policy";
import { unmatchedTarget } from "./target-matcher";
import type { RecordingDraftStep } from "./types";
interface CollapsedDraft {
draft: RecordingDraftStep;
draftIds: number[];
}
const TRANSITION_CAUSES: Record<string, NavigationCause> = {
typed: "user_typed",
generated: "user_typed",
keyword: "user_typed",
keyword_generated: "user_typed",
link: "link",
form_submit: "form_submit",
reload: "reload",
auto_bookmark: "browser",
start_page: "browser",
};
function isRedirect(step: Extract<RecordingDraftStep, { op: "navigate" }>): boolean {
return hasRedirectQualifier(step.transitionQualifiers);
}
function collapseRedirects(steps: RecordingDraftStep[]): CollapsedDraft[] {
const output: CollapsedDraft[] = [];
steps.forEach((step, index) => {
const previous = output[output.length - 1];
if (step.op === "navigate" && previous?.draft.op === "navigate" && isRedirect(step)) {
previous.draft = {
...previous.draft,
url: step.url,
postStateId: step.postStateId ?? previous.draft.postStateId,
};
previous.draftIds.push(index + 1);
return;
}
output.push({ draft: { ...step }, draftIds: [index + 1] });
});
return output;
}
function navigationCause(step: Extract<RecordingDraftStep, { op: "navigate" }>): NavigationCause {
if (step.cause) return step.cause;
const qualifiers = step.transitionQualifiers ?? [];
if (qualifiers.includes("forward_back")) return "history";
if (qualifiers.includes("from_address_bar")) return "user_typed";
return TRANSITION_CAUSES[step.transitionType ?? ""] ?? "browser";
}
function selection(values: string[], labels?: string[]): Array<{ value: string; label?: string }> {
return values.map((value, index) => ({
value,
...(labels?.[index] ? { label: labels[index] } : {}),
}));
}
interface ReduceDraftOptions {
includeTabSwitches: boolean;
redactValues: boolean;
}
function reduceDraft(
draft: RecordingDraftStep,
id: number,
options: ReduceDraftOptions,
): StepV3 | null {
if (!shouldIncludeDraft(draft)) return null;
if (draft.op === "switch_tab") {
if (!options.includeTabSwitches || !draft.preStateId || !draft.postStateId) return null;
return {
op: "switch_tab",
id,
state: draft.preStateId,
result: { state: draft.postStateId },
};
}
const state = draft.preStateId ?? draft.postStateId;
const resultState = draft.postStateId ?? draft.preStateId;
if (!state || !resultState) return null;
const common = { id, state, result: { state: resultState } };
switch (draft.op) {
case "navigate":
return { op: "navigate", ...common, to: draft.url, cause: navigationCause(draft) };
case "click":
return {
op: "click",
...common,
target: draft.matchedTarget ?? unmatchedTarget(draft.captureTarget),
};
case "hover":
return {
op: "hover",
...common,
target: draft.matchedTarget ?? unmatchedTarget(draft.captureTarget),
};
case "fill":
const fillIsRedacted = options.redactValues || draft.redacted === true;
return {
op: "fill",
...common,
target: draft.matchedTarget ?? unmatchedTarget(draft.captureTarget),
value: fillIsRedacted ? "***" : draft.value,
commit: draft.commit ?? "blur",
...(fillIsRedacted ? { redacted: true } : {}),
};
case "press":
return {
op: "press",
...common,
key: draft.key,
...(draft.captureTarget || draft.matchedTarget
? { target: draft.matchedTarget ?? unmatchedTarget(draft.captureTarget) }
: {}),
...(draft.modifiers?.length ? { modifiers: draft.modifiers } : {}),
};
case "select":
return {
op: "select",
...common,
target: draft.matchedTarget ?? unmatchedTarget(draft.captureTarget),
...(!options.redactValues ? { selection: selection(draft.values, draft.labels) } : {}),
};
case "scroll":
return { op: "scroll", ...common };
}
}
export interface ReducedTraceV3 {
steps: StepV3[];
stepIdByDraftId: Map<number, number>;
}
export function reduceTraceStepsV3(
steps: RecordingDraftStep[],
options: { includeTabSwitches?: boolean; redactValues?: boolean } = {},
): ReducedTraceV3 {
const output: StepV3[] = [];
const stepIdByDraftId = new Map<number, number>();
for (const { draft, draftIds } of collapseRedirects(steps)) {
const step = reduceDraft(draft, output.length + 1, {
includeTabSwitches: options.includeTabSwitches === true,
redactValues: options.redactValues === true,
});
if (!step) continue;
output.push(step);
for (const draftId of draftIds) stepIdByDraftId.set(draftId, step.id);
}
return { steps: output, stepIdByDraftId };
}
@@ -0,0 +1,39 @@
import type { StepAnnotation } from "./types";
function annotationText(annotation: StepAnnotation, stepId: number): string {
return ` ⟵ step ${stepId}: ${annotation.op}${annotation.detail ? `: ${annotation.detail}` : ""}`;
}
export function formatTraceStateBody(input: {
stateId: string;
url: string;
title?: string;
stepIds: number[];
vomText: string;
annotations: StepAnnotation[];
stepIdByDraftId: Map<number, number>;
}): string {
const lines = ["# bsk-observation 1", `state: ${JSON.stringify(input.stateId)}`];
lines.push(`url: ${JSON.stringify(input.url)}`);
if (input.title) lines.push(`title: ${JSON.stringify(input.title)}`);
if (input.stepIds.length > 0) lines.push(`steps_here: [${input.stepIds.join(", ")}]`);
lines.push("---");
const byLine = new Map<number, Array<{ annotation: StepAnnotation; stepId: number }>>();
for (const annotation of input.annotations) {
const stepId = input.stepIdByDraftId.get(annotation.draftId);
if (stepId === undefined) continue;
const bucket = byLine.get(annotation.line) ?? [];
bucket.push({ annotation, stepId });
byLine.set(annotation.line, bucket);
}
input.vomText.split("\n").forEach((bodyLine, lineIndex) => {
let line = bodyLine;
for (const item of byLine.get(lineIndex) ?? []) {
line += annotationText(item.annotation, item.stepId);
}
lines.push(line);
});
return `${lines.join("\n")}\n`;
}
+84
View File
@@ -0,0 +1,84 @@
import type { CaptureTargetDescriptor } from "@/lib/describe-target";
import type {
FillCommit,
KeyModifier,
NavigationCause,
StepV3,
TargetDescriptorV3,
} from "@/transport/types";
export interface TargetGeometry {
/** Top-level viewport-relative CSS pixels, as defined by the geometry module. */
rect: { x: number; y: number; w: number; h: number };
tag: string;
}
export interface TargetMatchHint {
geometry?: TargetGeometry;
/** Missing means the current top frame, never an unrestricted frame search. */
frameId?: string;
}
export interface StepAnnotation {
draftId: number;
op: StepV3["op"];
line: number;
stateId: string;
detail?: string;
}
interface DraftStateLink {
pageUrl?: string;
preStateId?: string;
postStateId?: string;
}
interface DraftTarget {
captureTarget?: CaptureTargetDescriptor;
targetHint?: TargetMatchHint;
matchedTarget?: TargetDescriptorV3;
}
interface DraftNavigationEffect {
navigatedTo?: string;
}
export type RecordingDraftStep =
| ({ op: "click" } & DraftStateLink & DraftTarget & DraftNavigationEffect)
| ({ op: "hover" } & DraftStateLink & DraftTarget)
| ({
op: "fill";
value: string;
commit?: FillCommit;
redacted?: boolean;
} & DraftStateLink &
DraftTarget &
DraftNavigationEffect)
| ({
op: "press";
key: string;
modifiers?: KeyModifier[];
} & DraftStateLink &
DraftTarget &
DraftNavigationEffect)
| ({
op: "select";
values: string[];
labels?: string[];
} & DraftStateLink &
DraftTarget &
DraftNavigationEffect)
| ({ op: "scroll" } & DraftStateLink)
| ({ op: "switch_tab" } & DraftStateLink)
| ({
op: "navigate";
url: string;
cause?: NavigationCause;
transitionType?: string;
transitionQualifiers?: string[];
} & DraftStateLink);
export type TargetedRecordingDraft = Extract<
RecordingDraftStep,
{ op: "click" | "hover" | "fill" | "press" | "select" }
>;
@@ -203,6 +203,55 @@ describe("ToolDispatcher", () => {
expect(sent[0]).toEqual({ id: "r-1", result: {} });
});
it("forwards daemon cancellation into an in-flight session stop", async () => {
const { transport, sent, deliver } = fakeTransport();
const remove = vi.fn(async () => {});
const sessions = new SessionManager({
agentWindow: {
create: vi.fn(async () => 4242),
remove,
ensureActiveTab: vi.fn(async () => {}),
},
});
await sessions.start("aa11");
let finishDetach: () => void = () => {};
const detach = new Promise<void>((resolve) => {
finishDetach = resolve;
});
const cdp = {
send: vi.fn(),
detachSession: vi.fn(() => detach),
ensureNetworkCapture: vi.fn(async () => {}),
networkEntriesSince: vi.fn(() => ({
tab_id: 7,
entries: [],
next_since: 0,
truncated: false,
})),
setDeviceMetricsOverride: vi.fn(async () => {}),
clearDeviceMetricsOverride: vi.fn(async () => {}),
setUserAgentOverride: vi.fn(async () => {}),
setTouchEmulationEnabled: vi.fn(async () => {}),
};
const dispatcher = new ToolDispatcher({ transport, sessions, cdp: cdp as TestDispatcherCdp });
dispatcher.start();
deliver(makeRequest("tool.session_stop", { session_id: "aa11" }));
await vi.waitFor(() => expect(cdp.detachSession).toHaveBeenCalledWith("aa11"));
deliver({ id: "cancel-stop", method: "cancel", params: { rpc_id: "r-1" } });
await flushMicrotasks();
finishDetach();
await flushMicrotasks();
expect(sent).toContainEqual({ id: "cancel-stop", result: { cancelled: true } });
expect(sent).toContainEqual({
id: "r-1",
error: expect.objectContaining({ code: "cancelled" }),
});
expect(sessions.has("aa11")).toBe(true);
expect(remove).not.toHaveBeenCalled();
});
it("returns not_found when stopping an unknown session", async () => {
const { transport, sent, deliver } = fakeTransport();
const sessions = new SessionManager({
File diff suppressed because it is too large Load Diff
@@ -66,6 +66,52 @@ function makeApis(
}
describe("handleSessionStop with auto-return", () => {
it("leaves the session untouched when cancellation arrived before teardown", async () => {
const aw = fakeAgentWindow([100]);
const sm = new SessionManager({ agentWindow: aw });
await sm.start("aa11");
const cdp = { detachSession: vi.fn(async () => {}) };
const controller = new AbortController();
controller.abort();
const res = await handleSessionStop(
sm,
{ session_id: "aa11" },
{ cdp, signal: controller.signal },
);
expect(res).toMatchObject({ code: "cancelled" });
expect(sm.has("aa11")).toBe(true);
expect(cdp.detachSession).not.toHaveBeenCalled();
expect(aw.remove).not.toHaveBeenCalled();
});
it("does not close the Agent Window when cancellation lands during CDP detach", async () => {
const aw = fakeAgentWindow([100]);
const sm = new SessionManager({ agentWindow: aw });
await sm.start("aa11");
const controller = new AbortController();
let finishDetach: () => void = () => {};
const detach = new Promise<void>((resolve) => {
finishDetach = resolve;
});
const cdp = { detachSession: vi.fn(() => detach) };
const stopping = handleSessionStop(
sm,
{ session_id: "aa11" },
{ cdp, signal: controller.signal },
);
await vi.waitFor(() => expect(cdp.detachSession).toHaveBeenCalledWith("aa11"));
controller.abort();
finishDetach();
const res = await stopping;
expect(res).toMatchObject({ code: "cancelled" });
expect(sm.has("aa11")).toBe(true);
expect(aw.remove).not.toHaveBeenCalled();
});
it("returns every borrowed tab and closes the Agent Window in the right order", async () => {
const aw = fakeAgentWindow([100]);
const sm = new SessionManager({ agentWindow: aw });
+1
View File
@@ -290,6 +290,7 @@ export class ToolDispatcher {
await this.releaseHoverLatch((req.params as SessionStopParams).session_id);
return handleSessionStop(this.sessions, req.params as SessionStopParams, {
cdp: this.cdp,
signal,
});
}
case "tool.tab_list":
+347 -74
View File
@@ -15,22 +15,32 @@ import {
type RecordQueryResponse,
type RecordStartAck,
type RecordStartMessage,
type RecordStepAck,
type RecordStopMessage,
} from "@/lib/record-bridge";
import { appendRecordedPayload, observeRecordedNavigation } from "@/lib/recording-step-buffer";
import { reduceTraceSteps, resolveTraceStartUrl } from "@/lib/trace-reducer";
import { RecordingObservationRuntime } from "@/lib/recording/recording-runtime";
import {
appendRecordedPayload,
observeRecordedNavigation,
type RecordingStepBuffer,
} from "@/lib/recording/step-buffer";
import { RecordingTabCoordinator, type TabActivation } from "@/lib/recording/tab-coordinator";
import { buildTraceV2 } from "@/lib/recording/trace-reducer-v2";
import type { RecordingDraftStep } from "@/lib/recording/types";
import type { SessionManager } from "@/session-manager/manager";
import { EXTENSION_VERSION } from "@/transport/handshake";
import type {
DraftTraceStep,
RecordAwaitParams,
RecordAwaitResult,
RecordedTrace,
RecordStartParams,
RecordStartResult,
RecordStopParams,
RecordStopResult,
RpcError,
Trace,
StopReason,
} from "@/transport/types";
import { TRACE_VERSION_V3 } from "@/transport/types";
import { handleNavigate } from "./navigation";
import {
type CdpRunner,
@@ -43,21 +53,46 @@ import {
interface ActiveRecording {
requestId: string;
tabId: number;
tabs: RecordingTabCoordinator;
agentWindowId: number;
startUrl?: string;
purpose?: string;
steps: DraftTraceStep[];
steps: RecordingDraftStep[];
startedAt: string;
startedAtMs: number;
finishPromise: Promise<Trace>;
resolveFinish: (trace: Trace) => void;
traceVersion: 2 | 3;
supportsTabSwitchSteps: boolean;
finishPromise: Promise<RecordedTrace>;
resolveFinish: (trace: RecordedTrace) => void;
rejectFinish: (err: Error) => void;
settled: boolean;
finishing: boolean;
currentUrl?: string;
pendingNavigation: boolean;
pendingNavigationDeadline?: number;
finishAttempt: Promise<RecordedTrace | null> | null;
observation: RecordingObservationRuntime | null;
stoppedBy: StopReason;
/** Navigation callbacks tracked from event receipt through action enqueue. */
navigationCallbacks: Set<Promise<void>>;
/** Synchronous intake gate closed only after finish drains to stability. */
acceptingNavigation: boolean;
/**
* Serializes step appends with navigation observation so a click is always
* in `steps` before a same-turn `webNavigation` tries to annotate it.
*/
actionQueue: Promise<void>;
/** Last accepted sequence for each content-script document producer. */
lastStepSequenceByProducer: Map<string, number>;
}
function enqueueRecordingAction(
recording: ActiveRecording,
task: () => Promise<void>,
): Promise<void> {
const queued = recording.actionQueue.then(task, task);
recording.actionQueue = queued.catch(() => {});
return queued;
}
function isRecordingFinishing(recording: ActiveRecording): boolean {
return recording.settled || recording.finishAttempt !== null;
}
const recordings = new Map<string, ActiveRecording>();
@@ -78,6 +113,9 @@ function sleep(ms: number): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, ms));
}
/** Recording producer version mirrored into trace.recorder.bsk. */
export const BSK_TRACE_VERSION = EXTENSION_VERSION;
/** Injectable http(s) landing page when `tool.record_start` omits `url`. */
export const RECORD_DEFAULT_START_URL = "https://example.com/";
@@ -137,9 +175,11 @@ async function sendRecordStartWithAck(
tabId: number,
msg: RecordStartMessage,
sendToTab: RecordDeps["sendToTab"],
cancelled?: () => boolean,
): Promise<void> {
let lastError: unknown;
for (let attempt = 0; attempt < RECORD_START_RETRIES; attempt += 1) {
if (cancelled?.()) throw new Error("record start cancelled");
try {
const response = await sendToTab(tabId, msg);
if (isRecordStartAck(response)) return;
@@ -147,6 +187,7 @@ async function sendRecordStartWithAck(
} catch (err) {
lastError = err;
}
if (cancelled?.()) throw new Error("record start cancelled");
if (attempt + 1 < RECORD_START_RETRIES) {
await sleep(RECORD_START_RETRY_DELAY_MS);
}
@@ -154,19 +195,82 @@ async function sendRecordStartWithAck(
throw lastError ?? new Error("failed to start recording in content script");
}
function buildTrace(recording: ActiveRecording): Trace {
const { pages, steps } = reduceTraceSteps(recording.steps, recording.startUrl);
const startUrl = resolveTraceStartUrl(recording.steps, recording.startUrl, pages);
function negotiatedTraceVersion(params: RecordStartParams): 2 | 3 | RpcError {
if (params.trace_version === undefined) return 2;
if (params.trace_version === TRACE_VERSION_V3) return 3;
return {
...(recording.purpose ? { purpose: recording.purpose } : {}),
recorded_at: new Date().toISOString(),
started_at: recording.startedAt,
entry: { start_url: startUrl },
pages,
steps,
code: "invalid_params",
message: `unsupported trace_version ${params.trace_version}; supported values are omitted (v2) or ${TRACE_VERSION_V3} (v3)`,
};
}
function buildTrace(recording: ActiveRecording): RecordedTrace {
if (recording.traceVersion === 3 && recording.observation) {
return recording.observation.buildTrace({
drafts: recording.steps,
startedAt: recording.startedAt,
purpose: recording.purpose,
startUrl: recording.startUrl,
stoppedBy: recording.stoppedBy,
bskVersion: BSK_TRACE_VERSION,
includeTabSwitches: recording.supportsTabSwitchSteps,
});
}
if (recording.traceVersion === 3) {
throw new Error("trace v3 observation runtime is unavailable");
}
return buildTraceV2({
steps: recording.steps,
startedAt: recording.startedAt,
...(recording.startUrl ? { startUrl: recording.startUrl } : {}),
...(recording.purpose ? { purpose: recording.purpose } : {}),
});
}
function stepBufferFor(
recording: ActiveRecording,
tabId: number,
fallbackUrl?: string,
): RecordingStepBuffer {
return {
steps: recording.steps,
navigation: recording.tabs.navigation(tabId, fallbackUrl),
};
}
async function activateRecordingTab(
recording: ActiveRecording,
targetTabId: number,
): Promise<void> {
const previousTabId = recording.tabs.currentTabId;
if (targetTabId === previousTabId) return;
const transition = recording.observation
? await recording.observation.captureTabTransition(previousTabId, targetTabId)
: {};
const { targetUrl, ...stateLinks } = transition;
const draft: Extract<RecordingDraftStep, { op: "switch_tab" }> = {
op: "switch_tab",
...stateLinks,
};
recording.steps.push(draft);
recording.observation?.bindTabTransition(draft, recording.steps.length);
recording.tabs.commit(targetTabId, targetUrl);
}
async function processRecordedStep(
recording: ActiveRecording,
draftIndex: number,
tabId: number,
): Promise<void> {
if (!recording.observation) return;
try {
await recording.observation.processDraft(tabId, recording.steps, draftIndex);
} catch (err) {
console.warn(`[bsk record] observation failed for step ${draftIndex + 1}`, err);
}
}
export interface RecordDeps {
tabsApi: ChromeTabsApi;
sendToTab(
@@ -241,16 +345,52 @@ export function releaseBrowserObservationListenersIfIdle(): void {
detachBrowserObservation = null;
}
export function attachRecordStepListener(): () => void {
export function attachRecordStepListener(deps: RecordDeps = getDefaultDeps()): () => void {
const listener = (
message: unknown,
_sender: chrome.runtime.MessageSender,
_sendResponse: () => void,
sender: chrome.runtime.MessageSender,
sendResponse: (response: RecordStepAck) => void,
) => {
if (!isRecordStepMessage(message)) return false;
for (const recording of recordings.values()) {
if (recording.requestId !== message.requestId) continue;
appendRecordedPayload(recording, message.step);
const sourceTabId = sender.tab?.id ?? recording.tabs.currentTabId;
const sourceWasActive = sender.tab?.active ?? sourceTabId === recording.tabs.activeTabId;
const producerKey = `${sourceTabId}:${message.producerId}`;
const expectedSequence = (recording.lastStepSequenceByProducer.get(producerKey) ?? 0) + 1;
if (message.sequence < expectedSequence) {
sendResponse({ ok: true, sequence: message.sequence });
return false;
}
if (message.sequence > expectedSequence) {
sendResponse({
ok: false,
expectedSequence,
error: `out-of-order recorded step ${message.sequence}`,
});
return false;
}
recording.lastStepSequenceByProducer.set(producerKey, message.sequence);
enqueueRecordingAction(recording, async () => {
if (sourceTabId !== recording.tabs.currentTabId) {
if (!sourceWasActive) return;
if (sourceTabId !== recording.tabs.activeTabId)
recording.tabs.noteActivation(sourceTabId);
await activateRecordingTab(recording, sourceTabId);
}
await recording.observation?.flushRedirects(sourceTabId);
const targetHint = message.step.geometry ? { geometry: message.step.geometry } : undefined;
const draftIndex = appendRecordedPayload(
stepBufferFor(recording, sourceTabId, message.step.page_url),
message.step,
targetHint,
);
if (draftIndex !== null) {
await processRecordedStep(recording, draftIndex, sourceTabId);
}
});
sendResponse({ ok: true, sequence: message.sequence });
return false;
}
return false;
@@ -277,7 +417,7 @@ export function attachRecordFinishListener(deps: RecordDeps = getDefaultDeps()):
function findRecordingByTabId(tabId: number): ActiveRecording | null {
for (const recording of recordings.values()) {
if (recording.tabId === tabId && !recording.settled) return recording;
if (recording.tabs.currentTabId === tabId && !recording.settled) return recording;
}
return null;
}
@@ -314,7 +454,7 @@ async function clearRearmTimersForRecording(
recording: ActiveRecording,
deps: RecordDeps,
): Promise<void> {
clearRearmTimer(recording.tabId);
clearRearmTimer(recording.tabs.currentTabId);
try {
const tabs = await deps.tabsApi.query({ windowId: recording.agentWindowId });
for (const tab of tabs) {
@@ -330,12 +470,12 @@ async function stopRecordingOnAllAgentTabs(
deps: RecordDeps,
): Promise<void> {
const stopMsg: RecordStopMessage = { type: RECORD_STOP, requestId: recording.requestId };
let tabIds = [recording.tabId];
let tabIds = [recording.tabs.currentTabId];
try {
const tabs = await deps.tabsApi.query({ windowId: recording.agentWindowId });
tabIds = [
...new Set([
recording.tabId,
recording.tabs.currentTabId,
...tabs.flatMap((tab) => (typeof tab.id === "number" ? [tab.id] : [])),
]),
];
@@ -346,11 +486,11 @@ async function stopRecordingOnAllAgentTabs(
for (const tabId of tabIds) {
try {
const response = await deps.sendToTab(tabId, stopMsg);
if (tabId === recording.tabId && !isRecordStartAck(response)) {
if (tabId === recording.tabs.currentTabId && !isRecordStartAck(response)) {
throw new Error("content script did not confirm recorded steps");
}
} catch {
if (tabId === recording.tabId) {
if (tabId === recording.tabs.currentTabId) {
throw new Error("failed to flush recorded steps");
}
}
@@ -368,22 +508,35 @@ async function rearmRecording(
recording: ActiveRecording,
targetTabId: number,
deps: RecordDeps,
activation?: TabActivation,
): Promise<boolean> {
// Do NOT toggle automation-bypass here: each retry used to increment the
// content-script counter, and a single stop decrement left the ControlOverlay
// stuck with pointer-events:none (page usable, Interrupt dead). RecordOverlay
// already hides the control chrome while activeRecord is set.
const isFinishing = () => isRecordingFinishing(recording);
for (let attempt = 0; attempt < RECORD_REARM_MAX_ATTEMPTS; attempt += 1) {
if (isFinishing()) return false;
const startMsg: RecordStartMessage = {
type: RECORD_START,
requestId: recording.requestId,
startedAtMs: recording.startedAtMs,
};
try {
await sendRecordStartWithAck(targetTabId, startMsg, deps.sendToTab);
recording.tabId = targetTabId;
await sendRecordStartWithAck(targetTabId, startMsg, deps.sendToTab, isFinishing);
if (isFinishing()) return false;
if (activation) {
if (!recording.tabs.isLatest(activation)) return true;
await enqueueRecordingAction(recording, async () => {
if (isFinishing() || !recording.tabs.isLatest(activation)) {
return;
}
await activateRecordingTab(recording, targetTabId);
});
}
return true;
} catch {
if (isFinishing()) return false;
if (attempt + 1 < RECORD_REARM_MAX_ATTEMPTS) {
await sleep(RECORD_REARM_RETRY_DELAY_MS);
}
@@ -392,7 +545,7 @@ async function rearmRecording(
return false;
}
function scheduleRearmForTab(tabId: number, deps: RecordDeps): void {
function scheduleRearmForTab(tabId: number, deps: RecordDeps, activation?: TabActivation): void {
const existing = rearmTimers.get(tabId);
if (existing) clearTimeout(existing);
rearmTimers.set(
@@ -401,7 +554,7 @@ function scheduleRearmForTab(tabId: number, deps: RecordDeps): void {
rearmTimers.delete(tabId);
void (async () => {
const current = await findRecordingForTab(tabId, deps);
if (current) await rearmRecording(current, tabId, deps);
if (current) await rearmRecording(current, tabId, deps, activation);
})();
}, RECORD_REARM_DEBOUNCE_MS),
);
@@ -413,14 +566,20 @@ export function attachRecordTabListener(deps: RecordDeps = getDefaultDeps()): ()
const windowId = tab.windowId;
if (tabId === undefined || windowId === undefined) return;
for (const recording of recordings.values()) {
if (recording.settled || recording.agentWindowId !== windowId) continue;
if (isRecordingFinishing(recording) || recording.agentWindowId !== windowId) continue;
scheduleRearmForTab(tabId, deps);
return;
}
};
const onActivated = (activeInfo: chrome.tabs.TabActiveInfo) => {
scheduleRearmForTab(activeInfo.tabId, deps);
for (const recording of recordings.values()) {
if (isRecordingFinishing(recording) || recording.agentWindowId !== activeInfo.windowId)
continue;
const activation = recording.tabs.noteActivation(activeInfo.tabId);
scheduleRearmForTab(activeInfo.tabId, deps, activation);
return;
}
};
chrome.tabs.onCreated.addListener(onCreated);
@@ -432,13 +591,47 @@ export function attachRecordTabListener(deps: RecordDeps = getDefaultDeps()): ()
}
export function attachRecordNavigationListener(deps: RecordDeps = getDefaultDeps()): () => void {
const observeMainFrame = (tabId: number, url?: string, causedByAction?: boolean) => {
void (async () => {
const recording = await findRecordingForTab(tabId, deps);
if (recording && url) {
observeRecordedNavigation(recording, url, causedByAction);
}
})();
const observeMainFrame = (
tabId: number,
url?: string,
causedByAction?: boolean,
transitionType?: string,
transitionQualifiers?: string[],
) => {
if (!url) return;
const candidates = [...recordings.values()].filter(
(recording) =>
!recording.settled && recording.acceptingNavigation && recording.tabs.activeTabId === tabId,
);
for (const recording of candidates) {
const queued = enqueueRecordingAction(recording, async () => {
if (tabId !== recording.tabs.currentTabId) {
await activateRecordingTab(recording, tabId);
}
const result = observeRecordedNavigation(
stepBufferFor(recording, tabId, url),
url,
causedByAction,
transitionType,
transitionQualifiers,
);
if (result.kind === "coalesce_redirect") {
recording.observation?.scheduleRedirect(tabId, recording.steps, result.url);
return;
}
if (result.kind === "noop") return;
recording.observation?.clearRedirect(tabId);
if (result.kind === "appended") {
await processRecordedStep(recording, result.index, tabId);
} else {
recording.observation?.scheduleSettle(tabId, recording.steps, result.index);
}
});
const tracked = queued.catch(() => {});
recording.navigationCallbacks.add(tracked);
void tracked.finally(() => recording.navigationCallbacks.delete(tracked));
}
};
const onMainFrameComplete = (tabId: number, url?: string) => {
void (async () => {
@@ -458,18 +651,13 @@ export function attachRecordNavigationListener(deps: RecordDeps = getDefaultDeps
details: chrome.webNavigation.WebNavigationTransitionCallbackDetails,
) => {
if (details.frameId !== 0) return;
const causedByAction =
details.transitionType === "link" || details.transitionType === "form_submit"
? true
: details.transitionType === "typed" ||
details.transitionType === "auto_bookmark" ||
details.transitionType === "generated" ||
details.transitionType === "keyword" ||
details.transitionType === "keyword_generated" ||
details.transitionType === "reload"
? false
: undefined;
observeMainFrame(details.tabId, details.url, causedByAction);
observeMainFrame(
details.tabId,
details.url,
undefined,
details.transitionType,
details.transitionQualifiers,
);
};
chrome.webNavigation.onCompleted.addListener(completedListener);
chrome.webNavigation.onCommitted?.addListener(committedListener);
@@ -487,6 +675,26 @@ export function attachRecordNavigationListener(deps: RecordDeps = getDefaultDeps
return () => chrome.tabs.onUpdated.removeListener(listener);
}
const MAX_FINISH_DRAIN_ROUNDS = 10;
async function drainRecordingToStability(recording: ActiveRecording): Promise<boolean> {
for (let round = 0; round < MAX_FINISH_DRAIN_ROUNDS; round += 1) {
await Promise.all([...recording.navigationCallbacks]);
const actionTail = recording.actionQueue;
await actionTail;
await recording.observation?.flush();
if (recording.navigationCallbacks.size === 0 && recording.actionQueue === actionTail) {
// No event or promise continuation can interleave with this synchronous
// check-and-close, so work accepted before the cutoff is fully drained.
recording.acceptingNavigation = false;
return true;
}
}
console.warn("[bsk record] navigation/action queues did not stabilize at stop");
return false;
}
export function attachRecordQueryListener(deps: RecordDeps = getDefaultDeps()): () => void {
const listener = (
message: unknown,
@@ -505,7 +713,8 @@ export function attachRecordQueryListener(deps: RecordDeps = getDefaultDeps()):
sendResponse({ active: false });
return;
}
await rearmRecording(recording, tabId, deps);
const activation = sender.tab?.active ? recording.tabs.noteActivation(tabId) : undefined;
await rearmRecording(recording, tabId, deps, activation);
sendResponse({
active: true,
requestId: recording.requestId,
@@ -527,23 +736,50 @@ async function finishRecordingByRequest(
if (recording.requestId !== requestId || recording.settled) continue;
const match = await findRecordingForTab(tabId, deps);
if (match !== recording) continue;
await finishRecording(sessionId, deps);
await finishRecording(sessionId, deps, "user_finish");
return;
}
}
async function finishRecording(sessionId: string, deps: RecordDeps): Promise<Trace | null> {
async function finishRecording(
sessionId: string,
deps: RecordDeps,
stoppedBy: StopReason,
): Promise<RecordedTrace | null> {
const recording = recordings.get(sessionId);
if (!recording || recording.settled || recording.finishing) return null;
recording.finishing = true;
if (!recording || recording.settled) return null;
if (recording.finishAttempt) return recording.finishAttempt;
recording.stoppedBy = stoppedBy;
const attempt = finishRecordingAttempt(sessionId, recording, deps);
recording.finishAttempt = attempt;
try {
return await attempt;
} finally {
if (!recording.settled) {
recording.finishAttempt = null;
}
}
}
async function finishRecordingAttempt(
sessionId: string,
recording: ActiveRecording,
deps: RecordDeps,
): Promise<RecordedTrace | null> {
await clearRearmTimersForRecording(recording, deps);
try {
// Disposing content capture may commit a final dirty fill. Flush content
// first so all resulting RECORD_STEP messages enter actionQueue before it
// and the observation queues are drained.
await stopRecordingOnAllAgentTabs(recording, deps);
} catch {
recording.finishing = false;
return null;
}
if (!(await drainRecordingToStability(recording))) {
return null;
}
await recording.observation?.settleTrailing(recording.tabs.currentTabId, recording.steps);
recording.settled = true;
recordings.delete(sessionId);
@@ -558,6 +794,15 @@ export async function handleRecordStart(
params: RecordStartParams,
deps: RecordDeps = getDefaultDeps(),
): Promise<RecordStartResult | RpcError> {
const traceVersionOrErr = negotiatedTraceVersion(params);
if (typeof traceVersionOrErr !== "number") return traceVersionOrErr;
if (traceVersionOrErr === 3 && !deps.cdp) {
return {
code: "protocol_error",
message: "trace v3 recording requires an active CDP connection",
};
}
const ctxOrErr = lookupSession(manager, params, "record_start");
if (isRpcError(ctxOrErr)) return ctxOrErr;
const ctx = ctxOrErr;
@@ -574,37 +819,53 @@ export async function handleRecordStart(
// on the destination page can RECORD_QUERY → rearm → show RecordOverlay
// instead of flashing ControlOverlay ("Agent 正在控制").
const requestId = makeRequestId(target.tabId);
let resolveFinish!: (trace: Trace) => void;
let resolveFinish!: (trace: RecordedTrace) => void;
let rejectFinish!: (err: Error) => void;
const finishPromise = new Promise<Trace>((resolve, reject) => {
const finishPromise = new Promise<RecordedTrace>((resolve, reject) => {
resolveFinish = resolve;
rejectFinish = reject;
});
const navigateUrl = params.url ?? RECORD_DEFAULT_START_URL;
const startedAtMs = Date.now();
const maxPageTokens = params.max_page_tokens;
const redactValues = params.redact_values ?? false;
recordings.set(params.session_id, {
requestId,
tabId: target.tabId,
tabs: new RecordingTabCoordinator(target.tabId, navigateUrl),
agentWindowId: ctx.agentWindowId,
startUrl: navigateUrl,
...(params.purpose ? { purpose: params.purpose } : {}),
steps: [],
startedAt: new Date(startedAtMs).toISOString(),
startedAtMs,
traceVersion: traceVersionOrErr,
supportsTabSwitchSteps: params.supports_tab_switch_steps === true,
finishPromise,
resolveFinish,
rejectFinish,
settled: false,
finishing: false,
currentUrl: navigateUrl,
pendingNavigation: false,
pendingNavigationDeadline: undefined,
finishAttempt: null,
observation:
traceVersionOrErr === 3 && deps.cdp
? new RecordingObservationRuntime({
cdp: deps.cdp,
tabsApi: deps.tabsApi,
maxTokens: maxPageTokens,
redactValues,
})
: null,
stoppedBy: "user_finish",
navigationCallbacks: new Set(),
acceptingNavigation: true,
actionQueue: Promise.resolve(),
lastStepSequenceByProducer: new Map(),
});
// Observe navigations for the whole recording lifetime; attach before
// optional navigate so the destination load can rearm capture.
ensureBrowserObservationListeners(deps);
const abortPending = async (notifyContent: boolean) => {
recordings.get(params.session_id)?.observation?.cancel();
recordings.delete(params.session_id);
releaseBrowserObservationListenersIfIdle();
if (notifyContent) {
@@ -686,7 +947,7 @@ export async function handleRecordStart(
return cancelledError();
}
active.startUrl = startUrl;
active.currentUrl = startUrl;
active.tabs.navigation(target.tabId, startUrl).currentUrl = startUrl;
if (isContentScriptRestrictedUrl(startUrl)) {
await abortPending(false);
@@ -694,7 +955,7 @@ export async function handleRecordStart(
code: "invalid_params",
message: params.url
? `cannot record on restricted URL (${startUrl}); use an http(s) page`
: `cannot record on restricted URL (${startUrl}); default start page must be injectable http(s)`,
: `cannot record on restricted URL (${startUrl}); default start page https://example.com/ did not load — pass --url with a page you can open`,
};
}
@@ -731,6 +992,17 @@ export async function handleRecordStart(
};
}
if (active.observation) {
const activeRecording = recordings.get(params.session_id);
if (activeRecording) {
try {
await activeRecording.observation?.captureInitial(target.tabId);
} catch {
// Proceed without initial observation; steps may be dropped by reducer.
}
}
}
{
const cancelled = await abortIfCancelled(true);
if (cancelled) return cancelled;
@@ -755,7 +1027,7 @@ export async function handleRecordStop(
};
}
const trace = await finishRecording(params.session_id, deps);
const trace = await finishRecording(params.session_id, deps, "cli_stop");
if (!trace) {
return {
code: "protocol_error",
@@ -785,9 +1057,9 @@ export async function handleRecordAwait(
return { code: "cancelled", message: "record_await aborted" };
}
const outcome = await new Promise<{ trace: Trace } | { error: RpcError }>((resolve) => {
const outcome = await new Promise<{ trace: RecordedTrace } | { error: RpcError }>((resolve) => {
let settled = false;
const finish = (result: { trace: Trace } | { error: RpcError }) => {
const finish = (result: { trace: RecordedTrace } | { error: RpcError }) => {
if (settled) return;
settled = true;
if (timer) clearTimeout(timer);
@@ -829,6 +1101,7 @@ export function clearRecordingForSession(sessionId: string): void {
}
void clearRearmTimersForRecording(recording, getDefaultDeps());
if (!recording.settled) {
recording.observation?.cancel();
recording.settled = true;
recording.rejectFinish(new Error("recording cleared"));
}
+20
View File
@@ -76,6 +76,7 @@ export interface SessionStopResult {
}
export interface SessionStopDeps {
signal?: AbortSignal;
cdp?: {
detachSession(sessionId: string): Promise<void>;
};
@@ -177,6 +178,9 @@ export async function handleSessionStop(
message: `session ${params.session_id} unknown`,
};
}
if (deps.signal?.aborted) {
return { code: "cancelled", message: "session_stop aborted before teardown" };
}
// Step 1: auto-return borrowed tabs. Iterate over a snapshot of the
// ids so deletions during iteration do not break the Map iterator.
@@ -187,6 +191,7 @@ export async function handleSessionStop(
try {
const tabManagement = {
...(deps.tabManagement ?? {}),
signal: deps.signal,
isAgentWindowId:
deps.tabManagement?.isAgentWindowId ??
((windowId: number) => manager.findByWindowId(windowId) !== null),
@@ -214,6 +219,17 @@ export async function handleSessionStop(
}
}
if (deps.signal?.aborted) {
const nonCancelFailures = returnFailures?.filter((failure) => failure.code !== "cancelled");
if (nonCancelFailures && nonCancelFailures.length > 0) {
return {
...(returnedTabIds.length > 0 ? { returned_tab_ids: returnedTabIds } : {}),
return_failures: nonCancelFailures,
};
}
return { code: "cancelled", message: "session_stop aborted during tab return" };
}
const result: SessionStopResult = {};
if (returnedTabIds.length > 0) result.returned_tab_ids = returnedTabIds;
if (returnFailures && returnFailures.length > 0) {
@@ -233,6 +249,10 @@ export async function handleSessionStop(
// Step 3: detach CDP sessions this session opened (no-op if none).
await deps.cdp?.detachSession(params.session_id);
if (deps.signal?.aborted) {
return { code: "cancelled", message: "session_stop aborted before window close" };
}
// Step 4: close the Agent Window and drop the context.
await manager.stop(params.session_id);
@@ -65,6 +65,11 @@ function deferredFakeTransport(): { transport: Transport; emit: (frame: Protocol
}
describe("performHandshake", () => {
it("advertises the protocol compatibility boundary", () => {
expect(PROTOCOL_VERSION).toBe("1.1");
expect(MIN_COMPATIBLE_PROTOCOL).toBe("1.0");
});
it("sends system.handshake with identity and both compat fields", async () => {
let sentFrame: ProtocolFrame | null = null;
const transport = fakeTransport((req) => {
@@ -74,7 +79,7 @@ describe("performHandshake", () => {
result: {
server: "browser-skill-daemon",
version: "0.1.0",
protocol_version: "1.0",
protocol_version: "1.1",
min_compatible_peer: "0.0.0",
min_compatible_protocol: "1.0",
},
@@ -129,8 +134,8 @@ describe("performHandshake", () => {
const response = {
server: "browser-skill-daemon",
version: "0.1.0",
protocol_version: "1.0",
min_compatible_protocol: "1.0",
protocol_version: "1.1",
min_compatible_protocol: "1.1",
} satisfies HandshakeResult;
const transport = fakeTransport((req) => ({
id: (req as { id: string }).id,
@@ -145,7 +150,7 @@ describe("performHandshake", () => {
});
expect(outcome.result.min_compatible_peer).toBeUndefined();
expect(outcome.result.min_compatible_protocol).toBe("1.0");
expect(outcome.result.min_compatible_protocol).toBe("1.1");
});
it("rejects when the daemon responds with an error", async () => {
@@ -178,9 +183,9 @@ describe("performHandshake", () => {
result: {
server: "browser-skill-daemon",
version: "0.1.0",
protocol_version: "1.0",
protocol_version: "1.1",
min_compatible_peer: "0.0.0",
min_compatible_protocol: "1.0",
min_compatible_protocol: "1.1",
},
});
emit({
@@ -188,14 +193,14 @@ describe("performHandshake", () => {
result: {
server: "browser-skill-daemon",
version: "0.1.0",
protocol_version: "1.0",
protocol_version: "1.1",
min_compatible_peer: "0.0.0",
min_compatible_protocol: "1.0",
min_compatible_protocol: "1.1",
},
});
await expect(pending).resolves.toMatchObject({
result: { server: "browser-skill-daemon", protocol_version: "1.0" },
result: { server: "browser-skill-daemon", protocol_version: "1.1" },
});
});
+1 -1
View File
@@ -7,7 +7,7 @@ import type {
ResponseFrame,
} from "./types";
export const PROTOCOL_VERSION = "1.0";
export const PROTOCOL_VERSION = "1.1";
/**
* Extension semver, injected at build time from `package.json` via
* Vite's `define` (see `wxt.config.ts` and `vitest.config.ts`).
+121 -65
View File
@@ -657,10 +657,60 @@ export interface EmulateResult {
}
// --------------------------------------------------------------------------
// Semantic record payloads — mirror bsk-protocol record.rs (Trace v2)
// Semantic record payloads mirror the versioned Rust protocol models.
// --------------------------------------------------------------------------
export interface TargetDescriptor {
export const TRACE_VERSION_V3 = 3;
/** Logical v2 identifier. Not a wire field — v2 envelopes omit `version`. */
export const TRACE_VERSION_V2 = 2;
export const VOM_FORMAT_VERSION = 1;
export interface TargetDescriptorV3 {
ref?: string;
role?: string;
name?: string;
ctx?: string;
unmatched?: boolean;
}
export interface RecorderInfo {
bsk: string;
vom: number;
}
export type StopReason = "user_finish" | "cli_stop";
export interface TraceStateV3 {
id: string;
url: string;
title?: string;
body: string;
truncated?: boolean;
}
export interface StepResultV3 {
state: string;
}
export interface StepCommonV3 {
id: number;
state: string;
result: StepResultV3;
}
export type NavigationCause =
| "user_typed"
| "link"
| "form_submit"
| "reload"
| "history"
| "script"
| "browser";
export type FillCommit = "enter" | "suggestion" | "blur";
/** Legacy v2 target shape retained for existing record producers. */
export interface TargetDescriptorV2 {
role?: string;
name?: string;
tag: string;
@@ -673,103 +723,109 @@ export interface TraceEntry {
start_url: string;
}
export interface PageRef {
export interface PageRefV2 {
id: string;
url: string;
title?: string;
}
export interface SelectedOption {
export interface SelectedOptionV2 {
value: string;
label?: string;
}
export interface StepEffect {
export interface StepEffectV2 {
navigated_to: string;
}
export interface StepCommon {
export interface StepCommonV2 {
id: number;
page: string;
effect?: StepEffect;
effect?: StepEffectV2;
}
/** Capture/buffer draft before v2 reduction. */
export type DraftTraceStep =
| {
op: "click";
target: TargetDescriptor;
navigated_to?: string;
page_url?: string;
}
| {
op: "hover";
target: TargetDescriptor;
page_url?: string;
}
| {
op: "fill";
target: TargetDescriptor;
value: string;
redacted?: boolean;
page_url?: string;
}
| {
op: "press";
key: string;
target?: TargetDescriptor;
modifiers?: KeyModifier[];
navigated_to?: string;
page_url?: string;
}
| {
op: "select";
target: TargetDescriptor;
values: string[];
labels?: string[];
navigated_to?: string;
page_url?: string;
}
| {
op: "navigate";
url: string;
page_url?: string;
};
/** Exported record-only step (trace v2). */
export type Step =
| ({ op: "navigate" } & StepCommon & { to: string })
| ({ op: "click" } & StepCommon & { target: TargetDescriptor })
| ({ op: "hover" } & StepCommon & { target: TargetDescriptor })
| ({ op: "fill" } & StepCommon & {
target: TargetDescriptor;
export type StepV2 =
| ({ op: "navigate" } & StepCommonV2 & { to: string })
| ({ op: "click" } & StepCommonV2 & { target: TargetDescriptorV2 })
| ({ op: "hover" } & StepCommonV2 & { target: TargetDescriptorV2 })
| ({ op: "fill" } & StepCommonV2 & {
target: TargetDescriptorV2;
value: string;
redacted?: boolean;
})
| ({ op: "select" } & StepCommon & {
target: TargetDescriptor;
selection: SelectedOption[];
| ({ op: "select" } & StepCommonV2 & {
target: TargetDescriptorV2;
selection: SelectedOptionV2[];
})
| ({ op: "press" } & StepCommon & {
| ({ op: "press" } & StepCommonV2 & {
key: string;
modifiers?: KeyModifier[];
target?: TargetDescriptor;
target?: TargetDescriptorV2;
});
export interface Trace {
export interface TraceV2 {
recorded_at: string;
started_at?: string;
purpose?: string;
entry: TraceEntry;
pages: PageRef[];
steps: Step[];
pages: PageRefV2[];
steps: StepV2[];
}
export interface SelectedOptionV3 {
value: string;
label?: string;
}
export type StepV3 =
| ({ op: "navigate" } & StepCommonV3 & { to: string; cause: NavigationCause })
| ({ op: "switch_tab" } & StepCommonV3)
| ({ op: "click" } & StepCommonV3 & { target: TargetDescriptorV3 })
| ({ op: "hover" } & StepCommonV3 & { target: TargetDescriptorV3 })
| ({ op: "fill" } & StepCommonV3 & {
target: TargetDescriptorV3;
value: string;
commit: FillCommit;
redacted?: boolean;
})
| ({ op: "select" } & StepCommonV3 & {
target: TargetDescriptorV3;
selection?: SelectedOptionV3[];
})
| ({ op: "press" } & StepCommonV3 & {
key: string;
modifiers?: KeyModifier[];
target?: TargetDescriptorV3;
})
| ({ op: "scroll" } & StepCommonV3);
export interface TraceV3 {
version: typeof TRACE_VERSION_V3;
recorded_at: string;
started_at?: string;
purpose?: string;
stopped_by: StopReason;
entry: TraceEntry;
recorder: RecorderInfo;
states: TraceStateV3[];
steps: StepV3[];
}
export type RecordedTrace = TraceV2 | TraceV3;
export type RecordedStep = StepV2 | StepV3;
export interface RecordStartParams {
session_id: string;
tab_id?: number;
url?: string;
purpose?: string;
max_page_tokens?: number;
redact_values?: boolean;
/** Omitted means v2; `3` requests a state-linked v3 trace. */
trace_version?: number;
/** Client can decode the v3 `switch_tab` step variant. */
supports_tab_switch_steps?: boolean;
}
export interface RecordStartResult {
@@ -782,7 +838,7 @@ export interface RecordStopParams {
}
export interface RecordStopResult {
trace: Trace;
trace: RecordedTrace;
}
export interface RecordAwaitParams {
@@ -791,5 +847,5 @@ export interface RecordAwaitParams {
}
export interface RecordAwaitResult {
trace: Trace;
trace: RecordedTrace;
}
+14 -6
View File
@@ -265,19 +265,27 @@ retry — complete the task autonomously or stop gracefully.
### Recording — `bsk record`
Capture the user's own actions in the Agent Window to a `trace.json`, for later LLM-driven automation:
Capture the user's own actions in the Agent Window to a **trace bundle**, for later LLM-driven automation. New CLI builds request **trace v3** (page observations + action chain); older extensions may still return **trace v2** (actions only), which the CLI exports as a single `trace.json` without `states/`.
```bash
bsk record start --browser <instance-id-or-label> [--url https://…] [--purpose "publish a wiki doc"] [--output trace.json]
bsk record start --browser <instance-id-or-label> \
[--url https://…] [--purpose "publish a wiki doc"] \
[--max-page-tokens 3000] [--redact-values] \
[--output trace]
# `--url` is optional; default https://example.com/ when omitted (must be http(s)).
# Blocks until the user clicks Finish in the recording panel, then writes ./trace.json and closes the window.
# Blocks until the user clicks Finish in the recording panel, then writes:
# trace/trace.json — action chain (+ state index when v3)
# trace/states/ — v3 only: one `sN.txt` observe snapshot per settled page state
bsk record stop [--output trace.json] # terminal fallback if the browser panel is unavailable
bsk record stop [--output trace] # terminal fallback if the browser panel is unavailable
```
- The trace is a **record-only action log** (a `pages[]` dictionary + `navigate`/`click`/`fill`/`select`/`press` steps with `target` descriptors). It records *what the user did*; deciding which inputs are variable is left to the executing agent.
- **v3 bundle (preferred):** `--output` is a directory (default `./trace`) containing `trace.json` and `states/`. `trace.json` lists `states[]` (page observation ids) and `steps[]` (each step binds `state` = observe snapshot *before* the action and `result.state` = snapshot *after* settle). Page bodies live in `states/sN.txt` using the same VOM format as `bsk observe`.
- **v2 fallback:** when the connected extension is older, export may contain only legacy `trace.json` with `pages[]` action context and no `states/` observation files. Update the extension for full v3 bundles.
- Each `states[]` entry is one **settled page observation** (captured after recording start, navigation landing, or action settle — not on a timer).
- `target.ref` values like `@e12` exist **only inside** the bundle for disambiguation; do **not** copy `@eN` refs into SKILL.md or agent runbooks — use visible names from the observation text instead.
- `--purpose` is optional context metadata; it does **not** change what gets captured.
- There is **no** `bsk replay` — to redo a flow, read the trace and reuse the existing `session` / `snapshot` / `@eN` / `click` / `fill` tools. Follow **Stop when the goal is met**.
- `--redact-values` masks all form values in page files as `[filled]` / `[empty]`.
- Do **not** record on banking/SSO/password-manager pages; passwords are redacted but traces may still contain sensitive text.
## Error handling
+1
View File
@@ -22,6 +22,7 @@ pub mod navigate;
pub mod network;
pub mod observe;
pub mod record;
pub mod record_recovery;
pub mod record_state;
pub mod render_error;
pub mod screenshot;
+269 -67
View File
@@ -1,23 +1,30 @@
//! `bsk record start|stop` — capture user actions in the Agent Window.
use std::fs;
use std::path::PathBuf;
use std::path::{Path, PathBuf};
use std::time::Duration;
use anyhow::Context;
use bsk_protocol::Method;
use bsk_protocol::tools::{
RecordAwaitParams, RecordAwaitResult, RecordStartParams, RecordStartResult, RecordStopParams,
RecordStopResult, Trace,
RecordStopResult, RecordedTrace, TRACE_VERSION_V3,
};
use bsk_protocol::{ErrorCode, Method};
use clap::{Args, Subcommand};
use crate::cli::TOOL_IPC_TIMEOUT;
mod export;
use crate::cli::business_rpc;
use crate::cli::ensure_daemon::ensure_daemon;
use crate::cli::error::{CliError, Format};
use crate::cli::record_recovery;
use crate::cli::record_state;
use crate::cli::session::{SessionStartOptions, start_session, stop_session};
use export::{
ExportMeta, export_with_recovery, states_dir_for_output, trace_json_path,
validate_record_output,
};
/// Max wait for the user to click 结束 in the browser (24 hours).
const RECORD_AWAIT_TIMEOUT_MS: u32 = 86_400_000;
@@ -50,7 +57,8 @@ pub struct RecordStartArgs {
pub tab_id: Option<i64>,
/// Navigate to this http(s) URL before recording. When omitted, defaults
/// to `https://example.com/`.
/// to `https://example.com/`. If that page does not load, pass `--url`
/// with a site you can open in this browser.
#[arg(long)]
pub url: Option<String>,
@@ -58,15 +66,25 @@ pub struct RecordStartArgs {
#[arg(long)]
pub purpose: Option<String>,
/// Output path for the trace JSON (default `./trace.json`).
#[arg(long, default_value = "trace.json")]
/// Max VOM tokens per page observation file (default 3000).
#[arg(long = "max-page-tokens")]
pub max_page_tokens: Option<u32>,
/// Redact all form values in page observations (`[filled]` only).
#[arg(long = "redact-values")]
pub redact_values: bool,
/// Output directory for the trace bundle (default `./trace`).
/// Writes `<dir>/trace.json` and `<dir>/states/*.txt`.
#[arg(long, default_value = "trace")]
pub output: PathBuf,
}
#[derive(Debug, Clone, Args)]
pub struct RecordStopArgs {
/// Output path for the trace JSON (default `./trace.json`).
#[arg(long, default_value = "trace.json")]
/// Output directory for the trace bundle (default `./trace`).
/// Writes `<dir>/trace.json` and `<dir>/states/*.txt`.
#[arg(long, default_value = "trace")]
pub output: PathBuf,
}
@@ -78,11 +96,7 @@ pub fn dispatch(cmd: RecordCmd, format: Format) -> Result<(), CliError> {
}
fn dispatch_start(args: RecordStartArgs, format: Format) -> Result<(), CliError> {
if record_state::read().is_ok() {
return Err(CliError::Local(anyhow::anyhow!(
"a recording is already in progress; run `bsk record stop` first"
)));
}
prepare_record_start(&args.output)?;
let info = ensure_daemon().context("ensure daemon is running")?;
let session = start_session(
@@ -96,8 +110,12 @@ fn dispatch_start(args: RecordStartArgs, format: Format) -> Result<(), CliError>
let start_params = RecordStartParams {
session_id: session.session_id.clone(),
tab_id: args.tab_id,
url: args.url,
url: args.url.clone(),
purpose: args.purpose.clone(),
max_page_tokens: args.max_page_tokens,
redact_values: Some(args.redact_values),
trace_version: Some(TRACE_VERSION_V3),
supports_tab_switch_steps: Some(true),
};
let start_result = business_rpc::call::<RecordStartParams, RecordStartResult>(
info.sock_path.clone(),
@@ -111,7 +129,7 @@ fn dispatch_start(args: RecordStartArgs, format: Format) -> Result<(), CliError>
Ok(result) => result,
Err(err) => {
let _ = stop_session(info.sock_path, &session.session_id);
return Err(err);
return Err(annotate_default_start_page_error(err, args.url.as_deref()));
}
};
@@ -142,8 +160,8 @@ fn dispatch_start(args: RecordStartArgs, format: Format) -> Result<(), CliError>
// Keep write/render inside a Result so `?` cannot skip session teardown.
let run_result: Result<(), CliError> = match await_result {
Ok(result) => (|| {
write_trace_file(&args.output, &result.trace)?;
render_finish(&result.trace, &args.output, format)
let exported = export_with_recovery(&args.output, &result.trace)?;
render_finish(&result.trace, &args.output, &exported, format)
})(),
Err(err) => Err(err),
};
@@ -157,11 +175,12 @@ fn dispatch_start(args: RecordStartArgs, format: Format) -> Result<(), CliError>
}
fn dispatch_stop(args: RecordStopArgs, format: Format) -> Result<(), CliError> {
let state = record_state::read().map_err(CliError::Local)?;
let info = ensure_daemon().context("ensure daemon is running")?;
let session_id = state.session_id.clone();
validate_record_output(&args.output)?;
if let Ok(state) = record_state::read() {
let info = ensure_daemon().context("ensure daemon is running")?;
let session_id = state.session_id.clone();
let run_result: Result<(), CliError> = (|| {
let params = RecordStopParams {
session_id: session_id.clone(),
};
@@ -172,16 +191,85 @@ fn dispatch_stop(args: RecordStopArgs, format: Format) -> Result<(), CliError> {
Some(params),
TOOL_IPC_TIMEOUT,
)?;
write_trace_file(&args.output, &result.trace)?;
render_stop(&result, &args.output, format)
})();
let session_stop_result = stop_session(info.sock_path, &session_id);
record_state::clear();
let run_result: Result<(), CliError> = (|| {
let exported = export_with_recovery(&args.output, &result.trace)?;
render_stop(&result, &args.output, &exported, format)
})();
run_result?;
session_stop_result?;
Ok(())
let session_stop_result = stop_session(info.sock_path, &session_id);
record_state::clear();
run_result?;
session_stop_result?;
return Ok(());
}
let Some(trace) = record_recovery::load().map_err(CliError::Local)? else {
return Err(CliError::Local(anyhow::anyhow!(
"no recording in progress; run `bsk record start` first"
)));
};
let exported = export_with_recovery(&args.output, &trace)?;
render_finish(&trace, &args.output, &exported, format)
}
/// When `record start` omitted `--url` and the default example.com page
/// never loaded, rewrite the RPC error so the CLI hint points at `--url`.
fn annotate_default_start_page_error(err: CliError, user_url: Option<&str>) -> CliError {
if user_url.is_some() {
return err;
}
let CliError::Rpc {
code,
message,
data,
source,
} = err
else {
return err;
};
if !is_default_start_page_failure(code, &message) {
return CliError::Rpc {
code,
message,
data,
source,
};
}
CliError::Rpc {
code,
message,
data: Some(serde_json::json!({
"reason": crate::cli::render_error::reason::RECORD_START_PAGE_UNREACHABLE
})),
source,
}
}
fn is_default_start_page_failure(code: ErrorCode, message: &str) -> bool {
match code {
ErrorCode::CdpFailed | ErrorCode::Timeout => true,
ErrorCode::InvalidParams => {
message.contains("restricted URL") || message.contains("about:blank")
}
_ => false,
}
}
fn prepare_record_start(output: &Path) -> Result<(), CliError> {
if record_state::read().is_ok() {
return Err(CliError::Local(anyhow::anyhow!(
"a recording is already in progress; run `bsk record stop` first"
)));
}
if record_recovery::exists() {
return Err(CliError::Local(anyhow::anyhow!(
"a previous recording was not exported; run `bsk record stop --output <dir>` to recover it first"
)));
}
validate_record_output(output)
}
fn record_await_ipc_timeout(timeout_ms: u32) -> Duration {
@@ -190,39 +278,66 @@ fn record_await_ipc_timeout(timeout_ms: u32) -> Duration {
.unwrap_or(Duration::from_secs(u64::from(timeout_ms / 1_000) + 15))
}
fn write_trace_file(output: &PathBuf, trace: &Trace) -> Result<(), CliError> {
let json = serde_json::to_string_pretty(trace)
.context("serialize trace JSON")
.map_err(CliError::Local)?;
if let Some(parent) = output.parent() {
if !parent.as_os_str().is_empty() {
fs::create_dir_all(parent)
.with_context(|| format!("create output directory {}", parent.display()))
.map_err(CliError::Local)?;
}
}
fs::write(output, format!("{json}\n"))
.with_context(|| format!("write trace to {}", output.display()))
.map_err(CliError::Local)?;
Ok(())
}
fn render_finish(trace: &Trace, output: &PathBuf, format: Format) -> Result<(), CliError> {
fn render_finish(
trace: &RecordedTrace,
output_dir: &PathBuf,
exported: &ExportMeta,
format: Format,
) -> Result<(), CliError> {
let trace_path = trace_json_path(output_dir);
match format {
Format::Json => {
let payload = match trace {
RecordedTrace::V3(t) => serde_json::json!({
"output": output_dir,
"trace_json": trace_path,
"trace_version": exported.trace_version,
"states_dir": exported.states_dir,
"trace": t,
"window_closed": true,
}),
RecordedTrace::V2(t) => serde_json::json!({
"output": output_dir,
"trace_json": trace_path,
"trace_version": exported.trace_version,
"states_dir": null,
"trace": t,
"window_closed": true,
}),
};
println!(
"{}",
serde_json::to_string(&serde_json::json!({
"output": output,
"trace": trace,
"window_closed": true,
}))
.map_err(|e| CliError::Local(anyhow::anyhow!(e)))?
serde_json::to_string(&payload).map_err(|e| CliError::Local(anyhow::anyhow!(e)))?
);
}
Format::Human => {
println!("saved {} steps to {}", trace.steps.len(), output.display());
}
Format::Human => match trace {
RecordedTrace::V3(t) => {
let states_dir = exported
.states_dir
.as_ref()
.map(|p| p.display().to_string())
.unwrap_or_else(|| states_dir_for_output(output_dir).display().to_string());
println!(
"saved {} steps to {} and {} states to {}",
t.steps.len(),
trace_path.display(),
t.states.len(),
states_dir
);
}
RecordedTrace::V2(t) => {
if exported.v2_fallback {
eprintln!(
"note: extension returned trace v2 (no page observations); update the BrowserSkill extension for v3 bundles"
);
}
println!(
"saved {} steps to {} (trace v2)",
t.steps.len(),
trace_path.display()
);
}
},
}
Ok(())
}
@@ -230,33 +345,89 @@ fn render_finish(trace: &Trace, output: &PathBuf, format: Format) -> Result<(),
fn render_stop(
result: &RecordStopResult,
output: &PathBuf,
exported: &ExportMeta,
format: Format,
) -> Result<(), CliError> {
render_finish(&result.trace, output, format)
render_finish(&result.trace, output, exported, format)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn default_output_is_trace_json() {
let args = RecordStopArgs {
output: PathBuf::from("trace.json"),
};
assert_eq!(args.output, PathBuf::from("trace.json"));
fn with_temp_home<F: FnOnce()>(f: F) {
let _lock = record_recovery::test_env_lock();
let tmp = tempfile::tempdir().unwrap();
unsafe {
std::env::set_var(crate::daemon::paths::BSK_HOME_ENV, tmp.path());
}
f();
record_recovery::clear();
record_state::clear();
unsafe {
std::env::remove_var(crate::daemon::paths::BSK_HOME_ENV);
}
}
#[test]
fn start_args_default_output_is_trace_json() {
fn prepare_record_start_rejects_existing_trace_json_file() {
with_temp_home(|| {
let dir = tempfile::tempdir().unwrap();
let output = dir.path().join("trace.json");
std::fs::write(&output, "{}\n").unwrap();
let err = prepare_record_start(&output).unwrap_err();
let msg = err.to_string();
assert!(
msg.contains("JSON file") || msg.contains("json file"),
"{msg}"
);
assert!(
msg.contains("states/") || msg.contains("--output trace"),
"{msg}"
);
});
}
#[test]
fn prepare_record_start_rejects_unexported_recovery() {
with_temp_home(|| {
crate::daemon::paths::ensure_bsk_home().unwrap();
std::fs::write(
crate::daemon::paths::record_recovery_path().unwrap(),
"{}\n",
)
.unwrap();
let err = prepare_record_start(Path::new("trace")).unwrap_err();
let msg = err.to_string();
assert!(
msg.contains("not exported") || msg.contains("recover"),
"{msg}"
);
});
}
#[test]
fn default_output_is_trace_dir() {
let args = RecordStopArgs {
output: PathBuf::from("trace"),
};
assert_eq!(args.output, PathBuf::from("trace"));
}
#[test]
fn start_args_default_output_is_trace_dir() {
let args = RecordStartArgs {
browser: None,
tab_id: None,
url: None,
purpose: None,
output: PathBuf::from("trace.json"),
max_page_tokens: None,
redact_values: false,
output: PathBuf::from("trace"),
};
assert_eq!(args.output, PathBuf::from("trace.json"));
assert_eq!(args.output, PathBuf::from("trace"));
}
#[test]
@@ -264,4 +435,35 @@ mod tests {
let got = record_await_ipc_timeout(RECORD_AWAIT_TIMEOUT_MS);
assert!(got >= Duration::from_secs(86_400));
}
#[test]
fn annotate_default_start_page_error_adds_url_reason() {
let err = CliError::from_rpc(bsk_protocol::RpcError {
code: ErrorCode::CdpFailed,
message: "Page.navigate rejected: net::ERR_ABORTED".into(),
data: None,
});
let annotated = annotate_default_start_page_error(err, None);
assert_eq!(
crate::cli::render_error::reason_for_data(annotated.data()),
Some(crate::cli::render_error::reason::RECORD_START_PAGE_UNREACHABLE)
);
let info =
crate::cli::render_error::info_for_error(annotated.code().unwrap(), annotated.data());
assert!(info.hint.unwrap().contains("--url"));
}
#[test]
fn annotate_default_start_page_error_skips_when_user_passed_url() {
let err = CliError::from_rpc(bsk_protocol::RpcError {
code: ErrorCode::CdpFailed,
message: "Page.navigate rejected: net::ERR_ABORTED".into(),
data: None,
});
let annotated = annotate_default_start_page_error(err, Some("https://www.example.org/"));
assert_eq!(
crate::cli::render_error::reason_for_data(annotated.data()),
None
);
}
}
+976
View File
@@ -0,0 +1,976 @@
use std::collections::HashSet;
use std::fs::{self, OpenOptions};
use std::io;
use std::path::{Path, PathBuf};
use anyhow::Context;
use bsk_protocol::tools::{RecordedTrace, TRACE_VERSION_V2, TRACE_VERSION_V3, TraceV2, TraceV3};
use serde::Serialize;
use crate::cli::error::CliError;
#[derive(Debug)]
pub(super) struct ExportMeta {
pub(super) states_dir: Option<PathBuf>,
pub(super) trace_version: u32,
/// True when the CLI requested v3 but the extension returned legacy v2.
pub(super) v2_fallback: bool,
}
pub(super) fn export_recorded_trace(
output_dir: &Path,
trace: &RecordedTrace,
) -> Result<ExportMeta, CliError> {
match trace {
RecordedTrace::V3(trace) => {
let states_dir = write_trace_bundle(output_dir, trace)?;
Ok(ExportMeta {
states_dir: Some(states_dir),
trace_version: TRACE_VERSION_V3,
v2_fallback: false,
})
}
RecordedTrace::V2(trace) => {
write_trace_v2(output_dir, trace)?;
Ok(ExportMeta {
states_dir: None,
trace_version: TRACE_VERSION_V2,
v2_fallback: true,
})
}
}
}
/// Save the completed Trace first, then write the bundle. Recovery is
/// removed only after the export commits so a bad `--output` cannot drop
/// a recording the extension already returned.
pub(super) fn export_with_recovery(
output_dir: &Path,
trace: &RecordedTrace,
) -> Result<ExportMeta, CliError> {
let save_result = crate::cli::record_recovery::save(trace);
match export_recorded_trace(output_dir, trace) {
Ok(meta) => {
crate::cli::record_recovery::clear();
Ok(meta)
}
Err(export_err) => {
let export_err = annotate_recovery_hint(export_err, save_result.is_ok());
if let Err(save_err) = save_result {
return Err(match export_err {
CliError::Local(inner) => CliError::Local(
inner.context(format!("also failed to save recovery data: {save_err}")),
),
other => other,
});
}
Err(export_err)
}
}
}
fn annotate_recovery_hint(err: CliError, recovery_saved: bool) -> CliError {
if !recovery_saved {
return err;
}
match err {
CliError::Local(inner) => CliError::Local(inner.context(
"export failed; the recorded trace was saved and can be recovered with `bsk record stop --output <dir>`",
)),
other => other,
}
}
fn looks_like_json_output(path: &Path) -> bool {
path.extension()
.and_then(|ext| ext.to_str())
.is_some_and(|ext| ext.eq_ignore_ascii_case("json"))
}
fn legacy_json_output_error(path: &Path) -> CliError {
CliError::Local(anyhow::anyhow!(
"--output {} is a JSON file path; Trace v3 writes a bundle directory \
(`<dir>/trace.json` and `<dir>/states/`), not a single file. \
Use `--output trace` or another directory.",
path.display()
))
}
/// Reject legacy `--output trace.json` (and any existing non-directory)
/// before a recording starts, so the user is not blocked only after Finish.
pub(super) fn validate_record_output(path: &Path) -> Result<(), CliError> {
match fs::symlink_metadata(path) {
Ok(metadata) if metadata.file_type().is_symlink() => Err(CliError::Local(anyhow::anyhow!(
"--output {} must not be a symlink",
path.display()
))),
Ok(metadata) if metadata.is_dir() => Ok(()),
Ok(_) if looks_like_json_output(path) => Err(legacy_json_output_error(path)),
Ok(_) => Err(CliError::Local(anyhow::anyhow!(
"--output {} is not a directory; Trace v3 writes a bundle directory \
(`<dir>/trace.json` and `<dir>/states/`). Use `--output trace`.",
path.display()
))),
Err(err) if err.kind() == io::ErrorKind::NotFound => {
if looks_like_json_output(path) {
Err(legacy_json_output_error(path))
} else {
Ok(())
}
}
Err(err) => Err(CliError::Local(
anyhow::Error::new(err).context(format!("inspect --output {}", path.display())),
)),
}
}
fn acquire_export_lock(output_dir: &Path) -> Result<std::fs::File, CliError> {
validate_bundle_directory(output_dir, "output bundle directory")?;
fs::create_dir_all(output_dir)
.with_context(|| format!("create output dir {}", output_dir.display()))
.map_err(CliError::Local)?;
let lock_path = output_dir.join(".bsk-record-export.lock");
let lock_file = OpenOptions::new()
.read(true)
.write(true)
.create(true)
.truncate(false)
.open(&lock_path)
.with_context(|| format!("open export lock {}", lock_path.display()))
.map_err(CliError::Local)?;
fs2::FileExt::lock_exclusive(&lock_file)
.with_context(|| format!("lock export bundle {}", output_dir.display()))
.map_err(CliError::Local)?;
Ok(lock_file)
}
pub(super) fn write_trace_v2(output_dir: &Path, trace: &TraceV2) -> Result<(), CliError> {
let trace_path = trace_json_path(output_dir);
let states_dir = states_dir_for_output(output_dir);
let json = serde_json::to_string_pretty(trace)
.context("serialize trace JSON")
.map_err(CliError::Local)?;
let _lock = acquire_export_lock(output_dir)?;
validate_replaceable_file(&trace_path, "trace JSON")?;
let transaction_id = uuid::Uuid::new_v4();
let staging_dir = output_dir.join(format!(".bsk-record-stage-{transaction_id}"));
fs::create_dir_all(&staging_dir)
.with_context(|| format!("create staging dir {}", staging_dir.display()))
.map_err(CliError::Local)?;
let staged_trace = staging_dir.join("trace.json");
let stage_result: Result<(), CliError> = (|| {
fs::write(&staged_trace, format!("{json}\n"))
.with_context(|| format!("write staged trace to {}", staged_trace.display()))
.map_err(CliError::Local)?;
Ok(())
})();
if let Err(err) = stage_result {
let _ = fs::remove_dir_all(&staging_dir);
return Err(err);
}
commit_export_transaction(
&staging_dir,
&states_dir,
&HashSet::new(),
transaction_id,
vec![(
staged_trace,
trace_path,
output_dir.join(format!(".trace.json.{transaction_id}.bak")),
)],
)?;
Ok(())
}
pub(super) fn states_dir_for_output(output_dir: &Path) -> PathBuf {
output_dir.join("states")
}
pub(super) fn trace_json_path(output_dir: &Path) -> PathBuf {
output_dir.join("trace.json")
}
fn is_canonical_state_id(id: &str) -> bool {
let Some(number) = id.strip_prefix('s') else {
return false;
};
let mut digits = number.bytes();
matches!(digits.next(), Some(b'1'..=b'9')) && digits.all(|byte| byte.is_ascii_digit())
}
fn validate_bundle_directory(path: &Path, description: &str) -> Result<(), CliError> {
match fs::symlink_metadata(path) {
Ok(metadata) if metadata.file_type().is_symlink() => Err(CliError::Local(anyhow::anyhow!(
"{description} {} must not be a symlink",
path.display()
))),
Ok(metadata) if metadata.is_dir() => Ok(()),
Ok(_) if looks_like_json_output(path) => Err(legacy_json_output_error(path)),
Ok(_) => Err(CliError::Local(anyhow::anyhow!(
"{description} {} is not a directory; Trace v3 writes `<dir>/trace.json` and `<dir>/states/`. Use `--output trace`.",
path.display()
))),
Err(err) if err.kind() == io::ErrorKind::NotFound => Ok(()),
Err(err) => Err(CliError::Local(
anyhow::Error::new(err).context(format!("inspect {description} {}", path.display())),
)),
}
}
fn validate_replaceable_file(path: &Path, description: &str) -> Result<(), CliError> {
match fs::symlink_metadata(path) {
Ok(metadata) if metadata.file_type().is_file() => Ok(()),
Ok(_) => Err(CliError::Local(anyhow::anyhow!(
"{description} {} is not a regular file",
path.display()
))),
Err(err) if err.kind() == io::ErrorKind::NotFound => Ok(()),
Err(err) => Err(CliError::Local(
anyhow::Error::new(err).context(format!("inspect {description} {}", path.display())),
)),
}
}
fn commit_staged_file(staged: &Path, target: &Path, backup: &Path) -> io::Result<Option<PathBuf>> {
let old_file = match fs::symlink_metadata(target) {
Ok(_) => {
fs::rename(target, backup)?;
Some(backup.to_path_buf())
}
Err(err) if err.kind() == io::ErrorKind::NotFound => None,
Err(err) => return Err(err),
};
if let Err(err) = fs::rename(staged, target) {
if let Some(old_file) = old_file.as_deref() {
if let Err(restore_err) = fs::rename(old_file, target) {
return Err(io::Error::new(
restore_err.kind(),
format!(
"{err}; additionally failed to restore {}: {restore_err}",
target.display()
),
));
}
}
return Err(err);
}
Ok(old_file)
}
fn rollback_installed_files(installed: &[(PathBuf, Option<PathBuf>)]) -> io::Result<()> {
let mut rollback_error = None;
for (target, backup) in installed.iter().rev() {
match fs::remove_file(target) {
Ok(()) => {}
Err(err) if err.kind() == io::ErrorKind::NotFound => {}
Err(err) => {
rollback_error.get_or_insert(err);
continue;
}
}
if let Some(backup) = backup {
if let Err(err) = fs::rename(backup, target) {
rollback_error.get_or_insert(err);
}
}
}
match rollback_error {
Some(err) => Err(err),
None => Ok(()),
}
}
fn is_generated_state_name(name: &str) -> bool {
name.strip_suffix(".txt").is_some_and(is_canonical_state_id)
}
fn restore_staged_stale_states(staged: &[(PathBuf, PathBuf)]) -> io::Result<()> {
let mut restore_error = None;
for (original, backup) in staged.iter().rev() {
if let Err(err) = fs::rename(backup, original) {
restore_error.get_or_insert(err);
}
}
match restore_error {
Some(err) => Err(err),
None => Ok(()),
}
}
/// Move stale generated states aside so replacement and cleanup form one
/// rollback boundary. User-owned files, including arbitrary non-`sN.txt`, stay.
fn stage_stale_states(
states_dir: &Path,
keep: &HashSet<String>,
transaction_id: uuid::Uuid,
) -> Result<Vec<(PathBuf, PathBuf)>, CliError> {
let entries = match fs::read_dir(states_dir) {
Ok(entries) => entries,
Err(err) if err.kind() == io::ErrorKind::NotFound => return Ok(Vec::new()),
Err(err) => {
return Err(CliError::Local(
anyhow::Error::new(err)
.context(format!("read states dir {}", states_dir.display())),
));
}
};
let mut staged = Vec::new();
for entry in entries {
let entry = match entry {
Ok(entry) => entry,
Err(err) => {
let restore_result = restore_staged_stale_states(&staged);
let message = match restore_result {
Ok(()) => format!("read stale state entry: {err}"),
Err(restore_err) => {
format!("read stale state entry: {err}; restore also failed: {restore_err}")
}
};
return Err(CliError::Local(anyhow::anyhow!(message)));
}
};
let name = entry.file_name();
let Some(name) = name.to_str() else { continue };
if !is_generated_state_name(name) || keep.contains(name) {
continue;
}
let original = entry.path();
let file_type = match entry.file_type() {
Ok(file_type) => file_type,
Err(err) => {
let restore_result = restore_staged_stale_states(&staged);
let message = match restore_result {
Ok(()) => format!("inspect stale state {}: {err}", original.display()),
Err(restore_err) => format!(
"inspect stale state {}: {err}; restore also failed: {restore_err}",
original.display()
),
};
return Err(CliError::Local(anyhow::anyhow!(message)));
}
};
if file_type.is_dir() {
let restore_result = restore_staged_stale_states(&staged);
let message = match restore_result {
Ok(()) => format!("stale state {} is a directory", original.display()),
Err(restore_err) => format!(
"stale state {} is a directory; restore also failed: {restore_err}",
original.display()
),
};
return Err(CliError::Local(anyhow::anyhow!(message)));
}
let backup = states_dir.join(format!(".{name}.{transaction_id}.stale"));
if let Err(err) = fs::rename(&original, &backup) {
let restore_result = restore_staged_stale_states(&staged);
let message = match restore_result {
Ok(()) => format!("stage stale state {}: {err}", original.display()),
Err(restore_err) => format!(
"stage stale state {}: {err}; restore also failed: {restore_err}",
original.display()
),
};
return Err(CliError::Local(anyhow::anyhow!(message)));
}
staged.push((original, backup));
}
Ok(staged)
}
fn commit_export_transaction(
staging_dir: &Path,
states_dir: &Path,
keep_states: &HashSet<String>,
transaction_id: uuid::Uuid,
replacements: Vec<(PathBuf, PathBuf, PathBuf)>,
) -> Result<(), CliError> {
let mut installed = Vec::with_capacity(replacements.len());
for (staged, target, backup) in replacements {
match commit_staged_file(&staged, &target, &backup) {
Ok(old_file) => installed.push((target, old_file)),
Err(err) => {
let rollback_result = rollback_installed_files(&installed);
let _ = fs::remove_dir_all(staging_dir);
let message = match rollback_result {
Ok(()) => format!("commit replacement {}: {err}", target.display()),
Err(rollback_err) => format!(
"commit replacement {}: {err}; rollback also failed: {rollback_err}",
target.display()
),
};
return Err(CliError::Local(anyhow::anyhow!(message)));
}
}
}
let stale_states = match stage_stale_states(states_dir, keep_states, transaction_id) {
Ok(stale) => stale,
Err(err) => {
let rollback_result = rollback_installed_files(&installed);
let _ = fs::remove_dir_all(staging_dir);
let message = match rollback_result {
Ok(()) => err.to_string(),
Err(rollback_err) => {
format!("{err}; replacement rollback also failed: {rollback_err}")
}
};
return Err(CliError::Local(anyhow::anyhow!(message)));
}
};
for (_, backup) in &installed {
if let Some(backup) = backup {
let _ = fs::remove_file(backup);
}
}
for (_, backup) in stale_states {
let _ = fs::remove_file(backup);
}
let _ = fs::remove_dir_all(staging_dir);
Ok(())
}
#[derive(Serialize)]
struct BundleState<'a> {
id: &'a str,
url: &'a str,
#[serde(skip_serializing_if = "Option::is_none")]
title: Option<&'a str>,
page: String,
#[serde(skip_serializing_if = "std::ops::Not::not")]
truncated: bool,
}
#[derive(Serialize)]
struct BundleTrace<'a> {
version: u32,
#[serde(skip_serializing_if = "Option::is_none")]
purpose: Option<&'a str>,
#[serde(skip_serializing_if = "Option::is_none")]
started_at: Option<&'a str>,
recorded_at: &'a str,
stopped_by: &'a bsk_protocol::tools::StopReason,
entry: &'a bsk_protocol::tools::TraceEntry,
recorder: &'a bsk_protocol::tools::RecorderInfo,
states: Vec<BundleState<'a>>,
steps: &'a [bsk_protocol::tools::StepV3],
}
pub(super) fn write_trace_bundle(output_dir: &Path, trace: &TraceV3) -> Result<PathBuf, CliError> {
let states_dir = states_dir_for_output(output_dir);
let trace_path = trace_json_path(output_dir);
if trace.version != TRACE_VERSION_V3 {
return Err(CliError::Local(anyhow::anyhow!(
"trace version {} is not supported (expected {})",
trace.version,
TRACE_VERSION_V3
)));
}
let mut state_ids = HashSet::with_capacity(trace.states.len());
for state in &trace.states {
if !is_canonical_state_id(&state.id) {
return Err(CliError::Local(anyhow::anyhow!(
"state id {:?} is not canonical (expected s1, s2, ...)",
state.id
)));
}
if !state_ids.insert(state.id.as_str()) {
return Err(CliError::Local(anyhow::anyhow!(
"duplicate state id {:?}",
state.id
)));
}
}
let mut disk_states = Vec::with_capacity(trace.states.len());
let mut state_writes = Vec::new();
let mut keep = HashSet::new();
for state in &trace.states {
let filename = format!("{}.txt", state.id);
let state_path = states_dir.join(&filename);
if state_path.parent() != Some(states_dir.as_path()) {
return Err(CliError::Local(anyhow::anyhow!(
"state path {} escapes states directory {}",
state_path.display(),
states_dir.display()
)));
}
keep.insert(filename.clone());
state_writes.push((filename.clone(), state.body.as_str()));
disk_states.push(BundleState {
id: &state.id,
url: &state.url,
title: state.title.as_deref(),
page: filename,
truncated: state.truncated,
});
}
let disk_trace = BundleTrace {
version: trace.version,
purpose: trace.purpose.as_deref(),
started_at: trace.started_at.as_deref(),
recorded_at: &trace.recorded_at,
stopped_by: &trace.stopped_by,
entry: &trace.entry,
recorder: &trace.recorder,
states: disk_states,
steps: &trace.steps,
};
let json = serde_json::to_string_pretty(&disk_trace)
.context("serialize trace JSON")
.map_err(CliError::Local)?;
validate_bundle_directory(output_dir, "output bundle directory")?;
fs::create_dir_all(output_dir)
.with_context(|| format!("create output dir {}", output_dir.display()))
.map_err(CliError::Local)?;
validate_bundle_directory(&states_dir, "states directory")?;
let _lock = acquire_export_lock(output_dir)?;
fs::create_dir_all(&states_dir)
.with_context(|| format!("create states dir {}", states_dir.display()))
.map_err(CliError::Local)?;
for (filename, _) in &state_writes {
validate_replaceable_file(&states_dir.join(filename), "state observation")?;
}
validate_replaceable_file(&trace_path, "trace JSON")?;
let transaction_id = uuid::Uuid::new_v4();
let staging_dir = output_dir.join(format!(".bsk-record-stage-{transaction_id}"));
let staging_states = staging_dir.join("states");
fs::create_dir_all(&staging_states)
.with_context(|| format!("create staging dir {}", staging_states.display()))
.map_err(CliError::Local)?;
let stage_result: Result<(), CliError> = (|| {
for (filename, body) in &state_writes {
let staged_path = staging_states.join(filename);
fs::write(&staged_path, body)
.with_context(|| {
format!("write staged state observation {}", staged_path.display())
})
.map_err(CliError::Local)?;
}
let staged_trace = staging_dir.join("trace.json");
fs::write(&staged_trace, format!("{json}\n"))
.with_context(|| format!("write staged trace to {}", staged_trace.display()))
.map_err(CliError::Local)?;
Ok(())
})();
if let Err(err) = stage_result {
let _ = fs::remove_dir_all(&staging_dir);
return Err(err);
}
let mut replacements: Vec<(PathBuf, PathBuf, PathBuf)> =
Vec::with_capacity(state_writes.len() + 1);
for (filename, _) in &state_writes {
replacements.push((
staging_states.join(filename),
states_dir.join(filename),
states_dir.join(format!(".{filename}.{transaction_id}.bak")),
));
}
replacements.push((
staging_dir.join("trace.json"),
trace_path.clone(),
output_dir.join(format!(".trace.json.{transaction_id}.bak")),
));
commit_export_transaction(
&staging_dir,
&states_dir,
&keep,
transaction_id,
replacements,
)?;
Ok(states_dir)
}
#[cfg(test)]
mod tests {
use std::fs;
use super::*;
use bsk_protocol::tools::{
NavigationCause, PageRefV2 as PageRef, RecorderInfo, StepCommonV3 as StepCommon,
StepResultV3 as StepResult, StepV3 as Step, StopReason, TRACE_VERSION_V2, TRACE_VERSION_V3,
TraceEntry, TraceStateV3, TraceV2, TraceV3, VOM_FORMAT_VERSION,
};
fn sample_trace(state_id: &str, body: &str) -> TraceV3 {
TraceV3 {
version: TRACE_VERSION_V3,
recorded_at: "2026-08-10T02:12:55.360Z".into(),
stopped_by: StopReason::UserFinish,
entry: TraceEntry {
start_url: "https://example.com/".into(),
},
recorder: RecorderInfo {
bsk: "0.1.10".into(),
vom: VOM_FORMAT_VERSION,
},
states: vec![TraceStateV3 {
id: state_id.into(),
url: "https://example.com/".into(),
title: Some("Example".into()),
body: body.into(),
truncated: false,
}],
steps: vec![Step::Navigate {
common: StepCommon {
id: 1,
state: state_id.into(),
result: StepResult {
state: state_id.into(),
},
},
to: "https://example.com/".into(),
cause: NavigationCause::UserTyped,
}],
purpose: None,
started_at: None,
}
}
#[test]
fn write_trace_bundle_writes_dir_layout() {
let dir = tempfile::tempdir().unwrap();
let output = dir.path().join("flow");
let trace = sample_trace("s1", "@vom 1\nL1 page");
let states_dir = write_trace_bundle(&output, &trace).unwrap();
let trace_path = output.join("trace.json");
assert!(trace_path.exists());
assert_eq!(states_dir, output.join("states"));
assert!(states_dir.join("s1.txt").exists());
let disk: serde_json::Value =
serde_json::from_str(&fs::read_to_string(&trace_path).unwrap()).unwrap();
assert_eq!(disk["states"][0]["page"], "s1.txt");
assert!(disk["states"][0].get("body").is_none());
assert_eq!(
fs::read_to_string(states_dir.join("s1.txt")).unwrap(),
"@vom 1\nL1 page"
);
// Re-exporting drops states from the previous run but keeps the
// directory in place for the concurrent writer.
fs::write(states_dir.join("s9.txt"), "stale").unwrap();
fs::write(states_dir.join("notes.md"), "keep me").unwrap();
fs::write(states_dir.join("reference.txt"), "user file").unwrap();
write_trace_bundle(&output, &trace).unwrap();
assert!(!states_dir.join("s9.txt").exists());
assert!(states_dir.join("s1.txt").exists());
assert!(states_dir.join("notes.md").exists());
assert_eq!(
fs::read_to_string(states_dir.join("reference.txt")).unwrap(),
"user file"
);
}
#[test]
fn write_trace_bundle_rejects_traversal_state_id() {
let dir = tempfile::tempdir().unwrap();
let output = dir.path().join("flow");
let trace = sample_trace("../escaped", "@vom 1\nL1 page");
let err = write_trace_bundle(&output, &trace).unwrap_err();
assert!(err.to_string().contains("state id"));
assert!(!output.join("escaped.txt").exists());
assert!(!output.join("trace.json").exists());
}
#[test]
fn write_trace_bundle_rejects_duplicate_state_id() {
let dir = tempfile::tempdir().unwrap();
let output = dir.path().join("flow");
let mut trace = sample_trace("s1", "@vom 1\nL1 first");
trace.states.push(TraceStateV3 {
body: "@vom 1\nL1 duplicate".into(),
..trace.states[0].clone()
});
let err = write_trace_bundle(&output, &trace).unwrap_err();
assert!(err.to_string().contains("duplicate state id"));
assert!(!output.exists());
}
#[test]
fn unsupported_version_preserves_existing_bundle() {
let dir = tempfile::tempdir().unwrap();
let output = dir.path().join("flow");
write_trace_bundle(&output, &sample_trace("s1", "original")).unwrap();
let original_json = fs::read(output.join("trace.json")).unwrap();
let mut replacement = sample_trace("s2", "replacement");
replacement.version += 1;
let err = write_trace_bundle(&output, &replacement).unwrap_err();
assert!(err.to_string().contains("not supported"));
assert_eq!(fs::read(output.join("trace.json")).unwrap(), original_json);
assert_eq!(
fs::read_to_string(output.join("states/s1.txt")).unwrap(),
"original"
);
assert!(!output.join("states/s2.txt").exists());
}
#[test]
fn state_replacement_failure_preserves_existing_bundle() {
let dir = tempfile::tempdir().unwrap();
let output = dir.path().join("flow");
write_trace_bundle(&output, &sample_trace("s1", "original")).unwrap();
let original_json = fs::read(output.join("trace.json")).unwrap();
fs::write(output.join("states/notes.md"), "keep me").unwrap();
fs::create_dir(output.join("states/s2.txt")).unwrap();
let err = write_trace_bundle(&output, &sample_trace("s2", "replacement")).unwrap_err();
assert!(err.to_string().contains("state"));
assert_eq!(fs::read(output.join("trace.json")).unwrap(), original_json);
assert_eq!(
fs::read_to_string(output.join("states/s1.txt")).unwrap(),
"original"
);
assert_eq!(
fs::read_to_string(output.join("states/notes.md")).unwrap(),
"keep me"
);
}
#[test]
fn stale_state_cleanup_failure_rolls_back_replacement() {
let dir = tempfile::tempdir().unwrap();
let output = dir.path().join("flow");
write_trace_bundle(&output, &sample_trace("s1", "original")).unwrap();
let original_json = fs::read(output.join("trace.json")).unwrap();
fs::create_dir(output.join("states/s9.txt")).unwrap();
let err = write_trace_bundle(&output, &sample_trace("s2", "replacement")).unwrap_err();
assert!(err.to_string().contains("stale state"));
assert_eq!(fs::read(output.join("trace.json")).unwrap(), original_json);
assert_eq!(
fs::read_to_string(output.join("states/s1.txt")).unwrap(),
"original"
);
assert!(!output.join("states/s2.txt").exists());
assert!(output.join("states/s9.txt").is_dir());
}
#[cfg(unix)]
#[test]
fn write_trace_bundle_rejects_symlinked_states_directory() {
use std::os::unix::fs::symlink;
let dir = tempfile::tempdir().unwrap();
let output = dir.path().join("flow");
let external_states = dir.path().join("external-states");
fs::create_dir_all(&output).unwrap();
fs::create_dir_all(&external_states).unwrap();
fs::write(external_states.join("s9.txt"), "external").unwrap();
symlink(&external_states, output.join("states")).unwrap();
let err = write_trace_bundle(&output, &sample_trace("s1", "replacement")).unwrap_err();
assert!(err.to_string().contains("symlink"));
assert_eq!(
fs::read_to_string(external_states.join("s9.txt")).unwrap(),
"external"
);
assert!(!external_states.join("s1.txt").exists());
assert!(!output.join("trace.json").exists());
}
#[cfg(unix)]
#[test]
fn write_trace_bundle_rejects_symlinked_output_directory() {
use std::os::unix::fs::symlink;
let dir = tempfile::tempdir().unwrap();
let external_output = dir.path().join("external-output");
let output = dir.path().join("flow");
fs::create_dir_all(&external_output).unwrap();
fs::write(external_output.join("notes.md"), "external").unwrap();
symlink(&external_output, &output).unwrap();
let err = write_trace_bundle(&output, &sample_trace("s1", "replacement")).unwrap_err();
assert!(err.to_string().contains("symlink"));
assert_eq!(
fs::read_to_string(external_output.join("notes.md")).unwrap(),
"external"
);
assert!(!external_output.join("trace.json").exists());
assert!(!external_output.join("states").exists());
}
#[test]
fn write_trace_v2_writes_single_json_and_prunes_stale_states() {
let dir = tempfile::tempdir().unwrap();
let output = dir.path().join("flow");
fs::create_dir_all(output.join("states")).unwrap();
fs::write(output.join("states/s1.txt"), "stale").unwrap();
let trace = TraceV2 {
recorded_at: "2026-08-10T02:12:55.360Z".into(),
started_at: None,
purpose: None,
entry: bsk_protocol::tools::TraceEntry {
start_url: "https://example.com/".into(),
},
pages: vec![PageRef {
id: "p1".into(),
url: "https://example.com/".into(),
title: None,
}],
steps: vec![],
};
write_trace_v2(&output, &trace).unwrap();
assert!(output.join("trace.json").exists());
assert!(!output.join("states/s1.txt").exists());
}
#[test]
fn write_trace_v2_stale_cleanup_failure_rolls_back_trace() {
let dir = tempfile::tempdir().unwrap();
let output = dir.path().join("flow");
write_trace_bundle(&output, &sample_trace("s1", "original")).unwrap();
let original_json = fs::read(output.join("trace.json")).unwrap();
fs::create_dir(output.join("states/s9.txt")).unwrap();
let trace = TraceV2 {
recorded_at: "2026-08-10T02:12:55.360Z".into(),
started_at: None,
purpose: None,
entry: bsk_protocol::tools::TraceEntry {
start_url: "https://example.com/".into(),
},
pages: vec![],
steps: vec![],
};
let err = write_trace_v2(&output, &trace).unwrap_err();
assert!(err.to_string().contains("stale state"));
assert_eq!(fs::read(output.join("trace.json")).unwrap(), original_json);
assert_eq!(
fs::read_to_string(output.join("states/s1.txt")).unwrap(),
"original"
);
assert!(output.join("states/s9.txt").is_dir());
}
#[test]
fn export_recorded_trace_accepts_v2() {
let dir = tempfile::tempdir().unwrap();
let output = dir.path().join("flow");
let trace = RecordedTrace::V2(TraceV2 {
recorded_at: "2026-08-10T02:12:55.360Z".into(),
started_at: None,
purpose: None,
entry: bsk_protocol::tools::TraceEntry {
start_url: "https://example.com/".into(),
},
pages: vec![],
steps: vec![],
});
let exported = export_recorded_trace(&output, &trace).unwrap();
assert_eq!(exported.trace_version, TRACE_VERSION_V2);
assert!(exported.states_dir.is_none());
assert!(output.join("trace.json").exists());
}
#[test]
fn existing_trace_json_file_is_rejected_with_bundle_usage() {
let dir = tempfile::tempdir().unwrap();
let output = dir.path().join("trace.json");
fs::write(&output, "{}\n").unwrap();
let err = validate_record_output(&output).unwrap_err();
let msg = err.to_string();
assert!(
msg.contains("JSON file") || msg.contains("json file"),
"should name the legacy JSON file usage: {msg}"
);
assert!(
msg.contains("states/") || msg.contains("--output trace"),
"should hint at the new bundle directory usage: {msg}"
);
assert_eq!(fs::read_to_string(&output).unwrap(), "{}\n");
}
#[test]
fn missing_trace_json_path_is_rejected_as_legacy_output() {
let err = validate_record_output(Path::new("trace.json")).unwrap_err();
let msg = err.to_string();
assert!(
msg.contains("JSON file") || msg.contains("json file"),
"{msg}"
);
assert!(
msg.contains("states/") || msg.contains("--output trace"),
"{msg}"
);
}
#[test]
fn export_failure_keeps_recovery_for_retry() {
let _lock = crate::cli::record_recovery::test_env_lock();
let tmp = tempfile::tempdir().unwrap();
unsafe {
std::env::set_var(crate::daemon::paths::BSK_HOME_ENV, tmp.path());
}
let dir = tempfile::tempdir().unwrap();
let bad_output = dir.path().join("trace.json");
fs::write(&bad_output, "{}\n").unwrap();
let trace = RecordedTrace::V3(sample_trace("s1", "@vom 1\nL1 page"));
let err = export_with_recovery(&bad_output, &trace).unwrap_err();
let msg = err.to_string();
assert!(
msg.contains("recover") || msg.contains("record stop"),
"export failure should tell the user how to recover: {msg}"
);
assert!(
crate::cli::record_recovery::exists(),
"recovery data must survive a failed bundle export"
);
let recovered = crate::cli::record_recovery::load()
.unwrap()
.expect("recovery file");
assert_eq!(recovered, trace);
assert_eq!(fs::read_to_string(&bad_output).unwrap(), "{}\n");
let good_output = dir.path().join("bundle");
export_with_recovery(&good_output, &recovered).unwrap();
assert!(good_output.join("trace.json").exists());
assert!(good_output.join("states/s1.txt").exists());
assert!(
!crate::cli::record_recovery::exists(),
"successful export must delete recovery data"
);
unsafe {
std::env::remove_var(crate::daemon::paths::BSK_HOME_ENV);
}
}
}
+112
View File
@@ -0,0 +1,112 @@
//! Persist a completed Trace so a failed bundle export can be retried.
use std::fs;
use anyhow::{Context, Result};
use bsk_protocol::tools::RecordedTrace;
use crate::daemon::paths;
pub fn save(trace: &RecordedTrace) -> Result<()> {
let path = paths::record_recovery_path()?;
paths::ensure_bsk_home()?;
let json = serde_json::to_string_pretty(trace).context("serialize record recovery")?;
let tmp = path.with_extension("tmp");
fs::write(&tmp, format!("{json}\n")).with_context(|| format!("write {}", tmp.display()))?;
fs::rename(&tmp, &path).with_context(|| format!("commit {}", path.display()))?;
Ok(())
}
pub fn load() -> Result<Option<RecordedTrace>> {
let path = paths::record_recovery_path()?;
if !path.exists() {
return Ok(None);
}
let raw = fs::read_to_string(&path).with_context(|| format!("read {}", path.display()))?;
let trace = serde_json::from_str(&raw).with_context(|| format!("parse {}", path.display()))?;
Ok(Some(trace))
}
pub fn exists() -> bool {
paths::record_recovery_path()
.map(|path| path.exists())
.unwrap_or(false)
}
pub fn clear() {
if let Ok(path) = paths::record_recovery_path() {
let _ = fs::remove_file(path);
}
}
#[cfg(test)]
pub(crate) fn test_env_lock() -> std::sync::MutexGuard<'static, ()> {
crate::daemon::paths::test_env_lock()
}
#[cfg(test)]
mod tests {
use super::*;
use bsk_protocol::tools::{
RecorderInfo, StopReason, TRACE_VERSION_V3, TraceEntry, TraceStateV3, TraceV3,
VOM_FORMAT_VERSION,
};
fn sample_trace() -> RecordedTrace {
RecordedTrace::V3(TraceV3 {
version: TRACE_VERSION_V3,
recorded_at: "2026-08-10T02:12:55.360Z".into(),
stopped_by: StopReason::UserFinish,
entry: TraceEntry {
start_url: "https://example.com/".into(),
},
recorder: RecorderInfo {
bsk: "0.1.10".into(),
vom: VOM_FORMAT_VERSION,
},
states: vec![TraceStateV3 {
id: "s1".into(),
url: "https://example.com/".into(),
title: Some("Example".into()),
body: "@vom 1\nL1 page".into(),
truncated: false,
}],
steps: vec![],
purpose: None,
started_at: None,
})
}
fn with_temp_home<F: FnOnce()>(f: F) {
let _lock = test_env_lock();
let tmp = tempfile::tempdir().unwrap();
unsafe {
std::env::set_var(crate::daemon::paths::BSK_HOME_ENV, tmp.path());
}
f();
clear();
unsafe {
std::env::remove_var(crate::daemon::paths::BSK_HOME_ENV);
}
}
#[test]
fn save_load_round_trips() {
with_temp_home(|| {
let trace = sample_trace();
save(&trace).unwrap();
assert!(exists());
assert_eq!(load().unwrap(), Some(trace));
});
}
#[test]
fn clear_removes_recovery_file() {
with_temp_home(|| {
save(&sample_trace()).unwrap();
clear();
assert!(!exists());
assert_eq!(load().unwrap(), None);
});
}
}
+1 -7
View File
@@ -59,15 +59,9 @@ fn started_at_unix_ms() -> String {
#[cfg(test)]
mod tests {
use super::*;
use std::sync::Mutex;
fn env_guard() -> &'static Mutex<()> {
static GUARD: Mutex<()> = Mutex::new(());
&GUARD
}
fn with_temp_home<F: FnOnce()>(f: F) {
let _lock = env_guard().lock().unwrap_or_else(|e| e.into_inner());
let _lock = crate::daemon::paths::test_env_lock();
let tmp = tempfile::tempdir().unwrap();
unsafe {
std::env::set_var(crate::daemon::paths::BSK_HOME_ENV, tmp.path());
+27
View File
@@ -50,6 +50,7 @@ pub mod reason {
pub const BORROW_CONFLICT: &str = "borrow_conflict";
pub const SCREENSHOT_CAPTURE_FAILED: &str = "screenshot_capture_failed";
pub const SESSION_BUSY: &str = crate::rpc_reason::SESSION_BUSY;
pub const RECORD_START_PAGE_UNREACHABLE: &str = "record_start_page_unreachable";
}
/// Per-code rendering metadata. The `summary` is what the CLI prints
@@ -269,6 +270,16 @@ pub fn info_for_error(code: ErrorCode, data: Option<&serde_json::Value>) -> Rend
),
exit_code: base.exit_code,
},
(
ErrorCode::CdpFailed | ErrorCode::Timeout | ErrorCode::InvalidParams,
reason::RECORD_START_PAGE_UNREACHABLE,
) => RenderInfo {
summary: "default start page https://example.com/ could not be opened",
hint: Some(
"that site may be unreachable from your network. Pass --url with a page you can open, e.g. `bsk record start --url https://<site>/`",
),
exit_code: base.exit_code,
},
_ => base,
}
}
@@ -378,6 +389,22 @@ mod tests {
assert_eq!(info.exit_code, 4);
}
#[test]
fn record_start_page_unreachable_overrides_cdp_failed_copy() {
let data = serde_json::json!({ "reason": reason::RECORD_START_PAGE_UNREACHABLE });
let info = info_for_error(ErrorCode::CdpFailed, Some(&data));
assert_eq!(
info.summary,
"default start page https://example.com/ could not be opened"
);
assert!(
info.hint.unwrap().contains("--url"),
"expected --url hint, got {:?}",
info.hint
);
assert_eq!(info.exit_code, 3);
}
#[test]
fn screenshot_capture_failed_overrides_cdp_failed_copy() {
let data = serde_json::json!({ "reason": reason::SCREENSHOT_CAPTURE_FAILED });
+88 -9
View File
@@ -206,15 +206,15 @@ pub fn full_handler(status: DaemonStatus, state: Arc<DaemonState>) -> RpcHandler
Ok(v) => ResponseBody::Ok(v),
Err(e) => ResponseBody::Err(e),
},
Method::SessionStart => match handle_session_start(&state, params).await {
Method::SessionStart => match handle_session_start(&state, rpc_id, params).await {
Ok(v) => ResponseBody::Ok(v),
Err(e) => ResponseBody::Err(e),
},
Method::SessionStop => match handle_session_stop(&state, params).await {
Method::SessionStop => match handle_session_stop(&state, rpc_id, params).await {
Ok(v) => ResponseBody::Ok(v),
Err(e) => ResponseBody::Err(e),
},
Method::SessionStopAll => match handle_session_stop_all(&state).await {
Method::SessionStopAll => match handle_session_stop_all_rpc(&state, rpc_id).await {
Ok(v) => ResponseBody::Ok(v),
Err(e) => ResponseBody::Err(e),
},
@@ -434,8 +434,9 @@ async fn handle_wait_ms(
/// cancellation surfaces (M10.2 + review C2):
///
/// 1. [`AbortRegistry`] — answers daemon-local cancellable runners
/// (currently `tool.wait_ms`). If a token is registered we trip it
/// and stop.
/// (`tool.wait_ms` and `session.*` lifecycle calls). If a token is
/// registered we trip it and stop. Lifecycle handlers forward their
/// own WS cancel after preserving request-before-cancel ordering.
/// 2. [`super::inflight::ToolInflightRegistry`] — every IPC-tracked
/// `tool.*` RPC, registered the moment the IPC handler accepts the
/// request. Trips the entry's cancel token regardless of whether
@@ -634,7 +635,20 @@ fn clamp_browser_wait(wait_ms: Option<u64>) -> Option<Duration> {
))
}
async fn handle_session_start(state: &Arc<DaemonState>, params: Value) -> Result<Value, RpcError> {
async fn handle_session_start(
state: &Arc<DaemonState>,
rpc_id: RpcId,
params: Value,
) -> Result<Value, RpcError> {
let abort_guard = state
.abort_registry
.register(rpc_id)
.map_err(|err| RpcError {
code: ErrorCode::ProtocolError,
message: format!("session.start cancellation registration failed: {err:?}"),
data: None,
})?;
let cancel = abort_guard.token().clone();
let params: CliSessionStartParams = if params.is_null() {
CliSessionStartParams {
browser_instance_id: None,
@@ -673,6 +687,7 @@ async fn handle_session_start(state: &Arc<DaemonState>, params: Value) -> Result
},
state.config.extension_connect_wait,
DEFAULT_RPC_TIMEOUT,
Some(cancel),
)
.await
{
@@ -696,6 +711,8 @@ fn map_start_error(err: StartSessionError) -> RpcError {
StartSessionError::AmbiguousBrowserLabel { .. } => ErrorCode::InvalidParams,
StartSessionError::IdExhausted => ErrorCode::ProtocolError,
StartSessionError::Timeout => ErrorCode::Timeout,
StartSessionError::Cancelled => ErrorCode::Cancelled,
StartSessionError::CleanupFailed { .. } => ErrorCode::ProtocolError,
StartSessionError::TransportClosed => ErrorCode::ProtocolError,
StartSessionError::ExtensionError(inner) => inner.code,
};
@@ -711,6 +728,16 @@ fn map_start_error(err: StartSessionError) -> RpcError {
"label": label,
"instance_ids": instance_ids,
})),
StartSessionError::CleanupFailed {
session_id,
agent_window_id,
..
} => Some(serde_json::json!({
"reason": "cleanup_failed",
"session_id": session_id.0,
"agent_window_id": agent_window_id,
})),
StartSessionError::ExtensionError(inner) => inner.data.clone(),
_ => None,
};
RpcError {
@@ -720,7 +747,20 @@ fn map_start_error(err: StartSessionError) -> RpcError {
}
}
async fn handle_session_stop(state: &Arc<DaemonState>, params: Value) -> Result<Value, RpcError> {
async fn handle_session_stop(
state: &Arc<DaemonState>,
rpc_id: RpcId,
params: Value,
) -> Result<Value, RpcError> {
let abort_guard = state
.abort_registry
.register(rpc_id)
.map_err(|err| RpcError {
code: ErrorCode::ProtocolError,
message: format!("session.stop cancellation registration failed: {err:?}"),
data: None,
})?;
let cancel = abort_guard.token().clone();
let params: CliSessionStopParams = if params.is_null() {
CliSessionStopParams {
session_id: None,
@@ -734,7 +774,7 @@ async fn handle_session_stop(state: &Arc<DaemonState>, params: Value) -> Result<
})?
};
if params.all {
return handle_session_stop_all(state).await;
return handle_session_stop_all(state, Some(cancel)).await;
}
let session_id = match params.session_id {
Some(s) => SessionId(s),
@@ -753,6 +793,7 @@ async fn handle_session_stop(state: &Arc<DaemonState>, params: Value) -> Result<
&state.session_interrupts,
&session_id,
DEFAULT_SESSION_STOP_TIMEOUT,
Some(cancel),
)
.await
{
@@ -804,7 +845,25 @@ fn map_stop_error(err: StopSessionError) -> RpcError {
}
}
async fn handle_session_stop_all(state: &Arc<DaemonState>) -> Result<Value, RpcError> {
async fn handle_session_stop_all_rpc(
state: &Arc<DaemonState>,
rpc_id: RpcId,
) -> Result<Value, RpcError> {
let abort_guard = state
.abort_registry
.register(rpc_id)
.map_err(|err| RpcError {
code: ErrorCode::ProtocolError,
message: format!("session.stop_all cancellation registration failed: {err:?}"),
data: None,
})?;
handle_session_stop_all(state, Some(abort_guard.token().clone())).await
}
async fn handle_session_stop_all(
state: &Arc<DaemonState>,
cancel: Option<super::abort::AbortToken>,
) -> Result<Value, RpcError> {
let ids: Vec<SessionId> = state
.sessions
.snapshot()
@@ -816,6 +875,16 @@ async fn handle_session_stop_all(state: &Arc<DaemonState>) -> Result<Value, RpcE
let mut returned_tab_ids = Vec::new();
let mut return_failures = Vec::new();
for id in ids {
if cancel
.as_ref()
.is_some_and(super::abort::AbortToken::is_cancelled)
{
return Err(RpcError {
code: ErrorCode::Cancelled,
message: "session.stop_all was cancelled".into(),
data: None,
});
}
match stop_session(
&state.browsers,
&state.sessions,
@@ -823,6 +892,7 @@ async fn handle_session_stop_all(state: &Arc<DaemonState>) -> Result<Value, RpcE
&state.session_interrupts,
&id,
DEFAULT_SESSION_STOP_TIMEOUT,
cancel.clone(),
)
.await
{
@@ -848,6 +918,15 @@ async fn handle_session_stop_all(state: &Arc<DaemonState>) -> Result<Value, RpcE
returned_tab_ids.extend(stop.returned_tab_ids);
return_failures.extend(stop.return_failures);
}
Err(err)
if matches!(
&err,
StopSessionError::ExtensionError(inner)
if inner.code == ErrorCode::Cancelled
) =>
{
return Err(map_stop_error(err));
}
Err(err) => {
debug!(session = %id, ?err, "session.stop_all: failure (continuing)");
let rpc = map_stop_error(err);
+21 -9
View File
@@ -100,6 +100,14 @@ pub fn record_session_path() -> Result<PathBuf> {
Ok(bsk_home()?.join("record-session.json"))
}
/// Path to a completed Trace saved before bundle export (`record-recovery.json`).
///
/// Kept until export succeeds so a failed `--output` write cannot drop the
/// recording the extension already returned.
pub fn record_recovery_path() -> Result<PathBuf> {
Ok(bsk_home()?.join("record-recovery.json"))
}
/// Windows named-pipe name. Include the resolved `BSK_HOME` path in the
/// token so test homes and custom installs do not share a predictable
/// per-username pipe.
@@ -119,6 +127,13 @@ pub fn pipe_name() -> String {
render_pipe_name(&user, &home)
}
/// Process-wide lock for tests that mutate `BSK_HOME`.
#[cfg(test)]
pub(crate) fn test_env_lock() -> std::sync::MutexGuard<'static, ()> {
static GUARD: std::sync::Mutex<()> = std::sync::Mutex::new(());
GUARD.lock().unwrap_or_else(|e| e.into_inner())
}
/// Render the Windows named-pipe name for a user/home pair. Kept
/// platform-agnostic so unit tests on any host can pin the output
/// character set.
@@ -136,19 +151,12 @@ fn render_pipe_name(user: &str, home: &str) -> String {
#[cfg(test)]
mod tests {
use super::*;
use std::sync::Mutex;
use tempfile::TempDir;
/// Serialise tests that mutate the global `BSK_HOME` env var.
fn env_guard() -> &'static Mutex<()> {
static GUARD: Mutex<()> = Mutex::new(());
&GUARD
}
fn with_temp_home<F: FnOnce(&Path)>(f: F) {
let _lock = env_guard().lock().unwrap_or_else(|e| e.into_inner());
let _lock = test_env_lock();
let tmp = TempDir::new().unwrap();
// SAFETY: serialised by env_guard above.
// SAFETY: serialised by test_env_lock above.
unsafe {
std::env::set_var(BSK_HOME_ENV, tmp.path().join("bsk"));
}
@@ -187,6 +195,10 @@ mod tests {
record_session_path().unwrap(),
home.join("record-session.json")
);
assert_eq!(
record_recovery_path().unwrap(),
home.join("record-recovery.json")
);
});
}
+158 -17
View File
@@ -43,6 +43,7 @@ use serde_json::Value;
use tokio::sync::{mpsc, oneshot};
use tracing::{debug, warn};
use super::abort::AbortToken;
use super::browsers::BrowserRegistry;
use super::inflight::{PromoteOutcome, ToolInflightEntry};
use super::sessions::{SessionId, SessionRegistry};
@@ -83,6 +84,14 @@ pub struct ToolJob {
/// IPC request id (e.g. `session.stop`'s queued teardown call —
/// those already carry their own bespoke retry / abort path).
pub inflight: Option<Arc<ToolInflightEntry>>,
/// Cancellation token for session-lifecycle work registered in the
/// daemon-local abort registry. Unlike `inflight`, lifecycle callers
/// forward their own WS cancel after the request has been queued.
pub lifecycle_cancel: Option<AbortToken>,
/// How long cancellation keeps the worker busy while the extension
/// finishes compensation. Session teardown uses its full RPC budget
/// so daemon state is reconciled even after the CLI exits.
pub cancel_cleanup_timeout: Duration,
/// Owning session — used by `ToolInflightRegistry::cancel_session`
/// to drain queued jobs that have not yet been promoted to
/// "forwarded". Worker callers also have access to it via the
@@ -322,6 +331,8 @@ impl ToolQueueRegistry {
timeout,
false,
inflight,
None,
CANCEL_CLEANUP_TIMEOUT,
)
.await;
// A long-running request may outlive the idle threshold. Touching
@@ -360,6 +371,8 @@ impl ToolQueueRegistry {
timeout,
respond: respond_tx,
inflight,
lifecycle_cancel: None,
cancel_cleanup_timeout: CANCEL_CLEANUP_TIMEOUT,
session_id: sid.clone(),
};
match forward_one(sid, &self.browsers, &self.sessions, &job).await {
@@ -378,6 +391,7 @@ impl ToolQueueRegistry {
method: Method,
params: Value,
timeout: Duration,
cancel: Option<AbortToken>,
) -> Result<Value, DispatchError> {
let (sender, state) = {
let mut guard = self.queues.lock().expect("tool queue registry poisoned");
@@ -404,6 +418,8 @@ impl ToolQueueRegistry {
timeout,
true,
None,
cancel,
timeout,
)
.await
}
@@ -419,6 +435,8 @@ async fn dispatch_with_sender(
timeout: Duration,
wait_for_capacity: bool,
inflight: Option<Arc<ToolInflightEntry>>,
lifecycle_cancel: Option<AbortToken>,
cancel_cleanup_timeout: Duration,
) -> Result<Value, DispatchError> {
let (respond_tx, respond_rx) = oneshot::channel();
let job = ToolJob {
@@ -427,8 +445,11 @@ async fn dispatch_with_sender(
timeout,
respond: respond_tx,
inflight,
lifecycle_cancel,
cancel_cleanup_timeout,
session_id,
};
let lifecycle_cancellable = job.lifecycle_cancel.is_some();
if wait_for_capacity {
let waited = tokio::time::timeout(
timeout.saturating_add(Duration::from_secs(1)),
@@ -453,8 +474,15 @@ async fn dispatch_with_sender(
mpsc::error::TrySendError::Closed(_) => DispatchError::QueueClosed,
});
}
let waited =
tokio::time::timeout(timeout.saturating_add(Duration::from_secs(1)), respond_rx).await;
let mut response_timeout = timeout.saturating_add(Duration::from_secs(1));
if lifecycle_cancellable {
// A lifecycle cancel can arrive at the original RPC deadline and
// intentionally keeps the worker alive while extension cleanup
// settles. Do not let this outer waiter abandon that reconciliation
// early, which could reopen a daemon session whose window is gone.
response_timeout = response_timeout.saturating_add(cancel_cleanup_timeout);
}
let waited = tokio::time::timeout(response_timeout, respond_rx).await;
match waited {
Ok(Ok(Ok(v))) => Ok(v),
Ok(Ok(Err(rpc))) => Err(DispatchError::Rpc(rpc)),
@@ -492,11 +520,17 @@ async fn forward_one(
// / browser resolution work, and before any WS frame leaves the
// daemon (review C2). We keep the `tokio::select!` later so
// cancels arriving mid-WS-hop also unblock the worker.
if let Some(entry) = job.inflight.as_ref()
&& entry.is_cancelled()
if job
.inflight
.as_ref()
.is_some_and(|entry| entry.is_cancelled())
|| job
.lifecycle_cancel
.as_ref()
.is_some_and(AbortToken::is_cancelled)
{
return Err(cancelled_error(
Some(entry),
job.inflight.as_deref(),
"tool dispatch cancelled before forwarding",
));
}
@@ -560,11 +594,20 @@ async fn forward_one(
}
}
None => {
// Daemon-internal callers (e.g. queued session.stop drain)
// don't carry an inflight entry; they cannot be cancelled
// through the IPC `cancel` surface, so a plain sink send
// is sufficient and keeps the existing transport-error
// behaviour.
// Daemon-internal callers do not carry a tool-inflight entry.
// Session lifecycle jobs instead use the abort-registry token
// below; other internal callers keep the plain send path.
if job
.lifecycle_cancel
.as_ref()
.is_some_and(AbortToken::is_cancelled)
{
client.pending.lock().unwrap().cancel(&rpc_id);
return Err(cancelled_error(
None,
"session lifecycle cancelled before forwarding",
));
}
if client.sink.send(request).is_err() {
client.pending.lock().unwrap().cancel(&rpc_id);
return Err(RpcError {
@@ -573,19 +616,45 @@ async fn forward_one(
data: None,
});
}
None
job.lifecycle_cancel.clone()
}
};
let forward_lifecycle_cancel = || {
if job.lifecycle_cancel.is_none() {
return;
}
let cancel = Frame::Request(RequestFrame {
id: format!("cancel-{rpc_id}"),
method: Method::Cancel,
params: Some(serde_json::json!({ "rpc_id": rpc_id })),
});
if let Err(err) = client.sink.send(cancel) {
warn!(%rpc_id, ?err, "failed to forward session lifecycle cancel");
}
};
let on_abort: Option<&(dyn Fn() + Sync)> = job
.lifecycle_cancel
.as_ref()
.map(|_| &forward_lifecycle_cancel as &(dyn Fn() + Sync));
let waited = await_with_optional_cancel(
job.timeout,
CANCEL_CLEANUP_TIMEOUT,
job.cancel_cleanup_timeout,
waiter,
cancel_token.as_ref(),
on_abort,
)
.await;
let response = match waited {
WaitOutcome::Response(resp) => resp,
WaitOutcome::CancelledAfterResponse(resp) => {
if job.method == Method::ToolSessionStop
&& let ResponseBody::Ok(value) = resp.body
{
// Teardown crossed its final irreversible boundary before
// the cancel landed. Commit the real extension result so the
// daemon does not retain a session whose window is gone.
return Ok(value);
}
// Cancellation wins the external verdict, but only after the
// extension's original RPC has settled. Preserve any non-cancel
// error so compensation failures remain explicit instead of
@@ -600,13 +669,33 @@ async fn forward_one(
"tool dispatch cancelled after extension cleanup",
));
}
WaitOutcome::TimedOutAfterResponse(resp) => match resp.body {
ResponseBody::Ok(value) if job.method == Method::ToolSessionStop => {
// The close crossed its irreversible boundary during the
// timeout cleanup grace. Reconcile daemon state instead of
// reopening a session whose window is already gone.
return Ok(value);
}
ResponseBody::Err(err)
if !matches!(err.code, ErrorCode::Cancelled | ErrorCode::UserAborted) =>
{
return Err(err);
}
_ => {
return Err(RpcError {
code: ErrorCode::Timeout,
message: format!("tool RPC timed out after {:?}", job.timeout),
data: None,
});
}
},
WaitOutcome::CleanupTimeout => {
client.pending.lock().unwrap().cancel(&rpc_id);
return Err(RpcError {
code: ErrorCode::Timeout,
message: format!(
"cancelled tool did not finish cleanup within {:?}",
CANCEL_CLEANUP_TIMEOUT
job.cancel_cleanup_timeout
),
data: Some(serde_json::json!({ "reason": "cancel_cleanup_timeout" })),
});
@@ -638,6 +727,7 @@ async fn forward_one(
enum WaitOutcome {
Response(bsk_protocol::ResponseFrame),
CancelledAfterResponse(bsk_protocol::ResponseFrame),
TimedOutAfterResponse(bsk_protocol::ResponseFrame),
CleanupTimeout,
WaiterClosed,
Timeout,
@@ -648,6 +738,7 @@ async fn await_with_optional_cancel(
cleanup_timeout: Duration,
mut waiter: oneshot::Receiver<bsk_protocol::ResponseFrame>,
cancel: Option<&super::abort::AbortToken>,
on_abort: Option<&(dyn Fn() + Sync)>,
) -> WaitOutcome {
match cancel {
Some(token) => {
@@ -659,6 +750,9 @@ async fn await_with_optional_cancel(
// after compensation or the cleanup deadline expires.
biased;
_ = token.cancelled() => {
if let Some(on_abort) = on_abort {
on_abort();
}
match tokio::time::timeout(cleanup_timeout, &mut waiter).await {
Ok(Ok(resp)) => WaitOutcome::CancelledAfterResponse(resp),
Ok(Err(_)) => WaitOutcome::WaiterClosed,
@@ -669,7 +763,17 @@ async fn await_with_optional_cancel(
Ok(resp) => WaitOutcome::Response(resp),
Err(_) => WaitOutcome::WaiterClosed,
},
_ = &mut deadline => WaitOutcome::Timeout,
_ = &mut deadline => {
let Some(on_abort) = on_abort else {
return WaitOutcome::Timeout;
};
on_abort();
match tokio::time::timeout(cleanup_timeout, &mut waiter).await {
Ok(Ok(resp)) => WaitOutcome::TimedOutAfterResponse(resp),
Ok(Err(_)) => WaitOutcome::WaiterClosed,
Err(_) => WaitOutcome::CleanupTimeout,
}
},
}
}
None => match tokio::time::timeout(timeout, &mut waiter).await {
@@ -759,6 +863,7 @@ mod await_with_optional_cancel_tests {
Duration::from_secs(1),
rx,
Some(&token),
None,
)
.await;
assert!(
@@ -781,6 +886,7 @@ mod await_with_optional_cancel_tests {
Duration::from_secs(1),
rx,
Some(&token),
None,
)
.await;
match outcome {
@@ -795,9 +901,14 @@ mod await_with_optional_cancel_tests {
async fn no_cancel_token_still_returns_response() {
let (tx, rx) = oneshot::channel();
tx.send(dummy_response()).unwrap();
let outcome =
await_with_optional_cancel(Duration::from_secs(10), Duration::from_secs(1), rx, None)
.await;
let outcome = await_with_optional_cancel(
Duration::from_secs(10),
Duration::from_secs(1),
rx,
None,
None,
)
.await;
assert!(matches!(outcome, WaitOutcome::Response(_)));
}
@@ -812,6 +923,7 @@ mod await_with_optional_cancel_tests {
Duration::from_secs(1),
rx,
Some(&token),
None,
)
.await
});
@@ -835,10 +947,37 @@ mod await_with_optional_cancel_tests {
Duration::from_millis(10),
rx,
Some(&token),
None,
)
.await;
assert!(matches!(outcome, WaitOutcome::CleanupTimeout));
}
#[tokio::test]
async fn lifecycle_timeout_forwards_abort_and_waits_for_cleanup_response() {
use std::sync::atomic::{AtomicBool, Ordering};
let token = AbortToken::new();
let (tx, rx) = oneshot::channel();
let abort_forwarded = AtomicBool::new(false);
let forward_abort = || abort_forwarded.store(true, Ordering::SeqCst);
tokio::spawn(async move {
tokio::time::sleep(Duration::from_millis(20)).await;
tx.send(dummy_response()).unwrap();
});
let outcome = await_with_optional_cancel(
Duration::from_millis(5),
Duration::from_millis(100),
rx,
Some(&token),
Some(&forward_abort),
)
.await;
assert!(abort_forwarded.load(Ordering::SeqCst));
assert!(matches!(outcome, WaitOutcome::TimedOutAfterResponse(_)));
}
}
#[cfg(test)]
@@ -901,6 +1040,8 @@ mod tool_job_session_id_tests {
timeout: Duration::from_secs(1),
respond: tx,
inflight: None,
lifecycle_cancel: None,
cancel_cleanup_timeout: CANCEL_CLEANUP_TIMEOUT,
session_id: SessionId("sess-A".into()),
};
assert_eq!(job.session_id.0, "sess-A");
+223 -26
View File
@@ -12,13 +12,14 @@ use bsk_protocol::system::{BrowserStatusEntry, SessionStatusEntry};
use bsk_protocol::tools::{
SessionStartParams, SessionStartResult, SessionStopParams, SessionStopResult,
};
use bsk_protocol::{Frame, RequestFrame, ResponseBody, RpcError, RpcId};
use bsk_protocol::{ErrorCode, Frame, Method, RequestFrame, ResponseBody, RpcError, RpcId};
use rand::Rng;
use serde::{Deserialize, Serialize};
use tokio::time::{Duration, timeout};
use super::abort::AbortToken;
use super::browsers::{BrowserClient, BrowserId, BrowserRegistry, SelectError};
use super::queue::{DispatchError, ToolQueueRegistry};
use super::queue::{CANCEL_CLEANUP_TIMEOUT, DispatchError, ToolQueueRegistry};
use super::session_interrupt::SessionInterruptRegistry;
#[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize)]
@@ -339,6 +340,14 @@ pub enum StartSessionError {
IdExhausted,
#[error("session creation timed out waiting for extension")]
Timeout,
#[error("session creation was cancelled")]
Cancelled,
#[error("session creation cancellation cleanup failed for {session_id}: {message}")]
CleanupFailed {
session_id: SessionId,
agent_window_id: Option<i64>,
message: String,
},
#[error("extension rejected tool.session_start: {0:?}")]
ExtensionError(RpcError),
#[error("transport closed while waiting for extension response")]
@@ -354,6 +363,8 @@ impl StartSessionError {
StartSessionError::AmbiguousBrowserLabel { .. } => "invalid_params",
StartSessionError::IdExhausted => "protocol_error",
StartSessionError::Timeout => "timeout",
StartSessionError::Cancelled => "cancelled",
StartSessionError::CleanupFailed { .. } => "protocol_error",
StartSessionError::TransportClosed => "protocol_error",
StartSessionError::ExtensionError(err) => match err.code {
bsk_protocol::ErrorCode::Timeout => "timeout",
@@ -407,6 +418,7 @@ pub struct AgentWindowOptions {
/// On success the matching per-session dispatch queue (M6.5,
/// design §5) is also spawned so that subsequent `tool.*` RPCs serialise
/// against this session's Agent Window / ref-store.
#[allow(clippy::too_many_arguments)]
pub async fn start_session(
registry: &Arc<BrowserRegistry>,
sessions: &Arc<SessionRegistry>,
@@ -415,24 +427,32 @@ pub async fn start_session(
window: AgentWindowOptions,
connect_wait: Duration,
timeout_dur: Duration,
cancel: Option<AbortToken>,
) -> Result<Session, StartSessionError> {
let client: Arc<BrowserClient> = registry
.select_with_connect_wait(requested, connect_wait)
.await
.map_err(|e| match e {
SelectError::NoBrowserConnected => StartSessionError::NoBrowserConnected,
SelectError::MultipleBrowsersOnline => StartSessionError::MultipleBrowsersOnline {
browsers: snapshot_status_entries(registry, sessions),
},
SelectError::NotFound => StartSessionError::BrowserNotFound,
SelectError::AmbiguousLabel {
label,
instance_ids,
} => StartSessionError::AmbiguousBrowserLabel {
label,
instance_ids,
},
})?;
let selection = registry.select_with_connect_wait(requested, connect_wait);
tokio::pin!(selection);
let selected = match cancel.as_ref() {
Some(token) => tokio::select! {
biased;
_ = token.cancelled() => return Err(StartSessionError::Cancelled),
selected = &mut selection => selected,
},
None => selection.await,
};
let client: Arc<BrowserClient> = selected.map_err(|e| match e {
SelectError::NoBrowserConnected => StartSessionError::NoBrowserConnected,
SelectError::MultipleBrowsersOnline => StartSessionError::MultipleBrowsersOnline {
browsers: snapshot_status_entries(registry, sessions),
},
SelectError::NotFound => StartSessionError::BrowserNotFound,
SelectError::AmbiguousLabel {
label,
instance_ids,
} => StartSessionError::AmbiguousBrowserLabel {
label,
instance_ids,
},
})?;
let session_id = sessions
.reserve_id(client.id.clone(), SESSION_ID_MAX_RESERVE_ATTEMPTS, now_ms)
.ok_or(StartSessionError::IdExhausted)?;
@@ -453,22 +473,71 @@ pub async fn start_session(
let mut pending = client.pending.lock().unwrap();
pending.register(rpc_id.clone())
};
if cancel.as_ref().is_some_and(AbortToken::is_cancelled) {
sessions.cancel_reservation(&session_id);
client.pending.lock().unwrap().cancel(&rpc_id);
return Err(StartSessionError::Cancelled);
}
if client.sink.send(Frame::Request(request)).is_err() {
sessions.cancel_reservation(&session_id);
client.pending.lock().unwrap().cancel(&rpc_id);
return Err(StartSessionError::TransportClosed);
}
let response = match timeout(timeout_dur, waiter).await {
Ok(Ok(resp)) => resp,
Ok(Err(_)) => {
let mut waiter = waiter;
enum StartWaitOutcome {
Response(bsk_protocol::ResponseFrame),
WaiterClosed,
Cancelled,
Timeout,
}
let wait_outcome = match cancel.as_ref() {
Some(token) => {
let deadline = tokio::time::sleep(timeout_dur);
tokio::pin!(deadline);
tokio::select! {
biased;
_ = token.cancelled() => StartWaitOutcome::Cancelled,
response = &mut waiter => match response {
Ok(response) => StartWaitOutcome::Response(response),
Err(_) => StartWaitOutcome::WaiterClosed,
},
_ = &mut deadline => StartWaitOutcome::Timeout,
}
}
None => match timeout(timeout_dur, &mut waiter).await {
Ok(Ok(response)) => StartWaitOutcome::Response(response),
Ok(Err(_)) => StartWaitOutcome::WaiterClosed,
Err(_) => StartWaitOutcome::Timeout,
},
};
let response = match wait_outcome {
StartWaitOutcome::Response(response) => response,
StartWaitOutcome::WaiterClosed => {
sessions.cancel_reservation(&session_id);
client.pending.lock().unwrap().cancel(&rpc_id);
return Err(StartSessionError::TransportClosed);
}
Err(_) => {
sessions.cancel_reservation(&session_id);
client.pending.lock().unwrap().cancel(&rpc_id);
return Err(StartSessionError::Timeout);
StartWaitOutcome::Cancelled => {
return Err(finish_aborted_start(
&client,
sessions,
&session_id,
&rpc_id,
waiter,
StartAbortReason::Cancelled,
)
.await);
}
StartWaitOutcome::Timeout => {
return Err(finish_aborted_start(
&client,
sessions,
&session_id,
&rpc_id,
waiter,
StartAbortReason::Timeout,
)
.await);
}
};
let agent_window_id = match response.body {
@@ -504,6 +573,124 @@ pub async fn start_session(
Ok(session)
}
#[derive(Clone, Copy)]
enum StartAbortReason {
Cancelled,
Timeout,
}
impl StartAbortReason {
fn error(self) -> StartSessionError {
match self {
Self::Cancelled => StartSessionError::Cancelled,
Self::Timeout => StartSessionError::Timeout,
}
}
}
async fn finish_aborted_start(
client: &Arc<BrowserClient>,
sessions: &Arc<SessionRegistry>,
session_id: &SessionId,
rpc_id: &RpcId,
mut waiter: tokio::sync::oneshot::Receiver<bsk_protocol::ResponseFrame>,
reason: StartAbortReason,
) -> StartSessionError {
sessions.cancel_reservation(session_id);
let cancel = Frame::Request(RequestFrame {
id: format!("cancel-{rpc_id}"),
method: Method::Cancel,
params: Some(serde_json::json!({ "rpc_id": rpc_id })),
});
if client.sink.send(cancel).is_err() {
client.pending.lock().unwrap().cancel(rpc_id);
return StartSessionError::TransportClosed;
}
match timeout(CANCEL_CLEANUP_TIMEOUT, &mut waiter).await {
Ok(Ok(response)) => match response.body {
ResponseBody::Err(err)
if matches!(err.code, ErrorCode::Cancelled | ErrorCode::UserAborted) =>
{
reason.error()
}
ResponseBody::Err(err) => StartSessionError::ExtensionError(err),
ResponseBody::Ok(value) => {
let agent_window_id = serde_json::from_value::<SessionStartResult>(value)
.ok()
.and_then(|result| result.agent_window_id);
match rollback_extension_session(client, session_id).await {
Ok(()) => reason.error(),
Err(message) => StartSessionError::CleanupFailed {
session_id: session_id.clone(),
agent_window_id,
message,
},
}
}
},
Ok(Err(_)) => {
client.pending.lock().unwrap().cancel(rpc_id);
StartSessionError::TransportClosed
}
Err(_) => {
// The cancel frame is already queued behind session_start. Even
// if chrome.windows.create is still blocked, the extension will
// observe the aborted signal and compensate before replying.
client.pending.lock().unwrap().cancel(rpc_id);
reason.error()
}
}
}
async fn rollback_extension_session(
client: &Arc<BrowserClient>,
session_id: &SessionId,
) -> Result<(), String> {
let rollback_id = next_rpc_id("sess-start-rollback");
let waiter = {
let mut pending = client.pending.lock().unwrap();
pending.register(rollback_id.clone())
};
let request = RequestFrame {
id: rollback_id.clone(),
method: Method::ToolSessionStop,
params: Some(
serde_json::to_value(SessionStopParams {
session_id: session_id.0.clone(),
})
.expect("serialize session_start rollback params"),
),
};
if client.sink.send(Frame::Request(request)).is_err() {
client.pending.lock().unwrap().cancel(&rollback_id);
return Err("browser disconnected before rollback was queued".into());
}
let response = match timeout(CANCEL_CLEANUP_TIMEOUT, waiter).await {
Ok(Ok(response)) => response,
Ok(Err(_)) => return Err("browser disconnected during rollback".into()),
Err(_) => {
client.pending.lock().unwrap().cancel(&rollback_id);
return Err(format!(
"extension did not finish rollback within {CANCEL_CLEANUP_TIMEOUT:?}"
));
}
};
match response.body {
ResponseBody::Ok(value) => {
let result: SessionStopResult = serde_json::from_value(value)
.map_err(|err| format!("invalid rollback response: {err}"))?;
if result.return_failures.is_empty() {
Ok(())
} else {
Err("rollback reported borrowed-tab return failures".into())
}
}
ResponseBody::Err(err) if err.code == ErrorCode::NotFound => Ok(()),
ResponseBody::Err(err) => Err(format!("rollback failed: {} ({:?})", err.message, err.code)),
}
}
/// Tear down a single session id (round-trips `tool.session_stop` to the
/// owning extension and unregisters on success). Also tears down the
/// per-session dispatch queue so the worker task exits.
@@ -514,6 +701,7 @@ pub async fn stop_session(
interrupts: &Arc<SessionInterruptRegistry>,
session_id: &SessionId,
timeout_dur: Duration,
cancel: Option<AbortToken>,
) -> Result<SessionStopResult, StopSessionError> {
let session = sessions.get(session_id).ok_or(StopSessionError::NotFound)?;
if registry.get(&session.browser_id).is_none() {
@@ -524,12 +712,17 @@ pub async fn stop_session(
session_id: session_id.0.clone(),
})
.unwrap();
// Internal callers (idle reaping and shutdown) do not have a CLI-owned
// token, but still need timeout-driven WS cancellation so a late window
// close cannot leave daemon state pointing at a vanished session.
let lifecycle_cancel = Some(cancel.unwrap_or_default());
match queues
.dispatch_after_closing(
session_id,
bsk_protocol::Method::ToolSessionStop,
params,
timeout_dur,
lifecycle_cancel,
)
.await
{
@@ -567,6 +760,10 @@ pub async fn stop_session(
drop_session_local(queues, interrupts, session_id);
return Ok(SessionStopResult::default());
}
if err.code == bsk_protocol::ErrorCode::Timeout {
queues.reopen(session_id);
return Err(StopSessionError::Timeout);
}
queues.reopen(session_id);
Err(StopSessionError::ExtensionError(err))
}
+98 -29
View File
@@ -15,7 +15,6 @@ use std::net::{IpAddr, Ipv4Addr, SocketAddr};
use std::path::Path;
use std::sync::Arc;
use std::sync::Mutex;
use std::sync::atomic::{AtomicUsize, Ordering};
use std::time::{Duration, Instant};
use anyhow::{Context, Result};
@@ -27,7 +26,7 @@ use crate::daemon::{
browsers::{BROWSER_LIVENESS_TICK, BROWSER_LIVENESS_TIMEOUT, EXTENSION_CONNECT_WAIT},
info as daemon_info, ipc, lockfile, paths,
sessions::{StopSessionError, forget_session, stop_session},
state::DaemonState,
state::{DaemonState, PROTOCOL_VERSION},
ws,
};
@@ -291,15 +290,14 @@ pub fn run_foreground(cfg: DaemonConfig) -> Result<()> {
});
let started_at = Instant::now();
let activity: Arc<Mutex<Instant>> = Arc::new(Mutex::new(started_at));
let active_ipc_connections = Arc::new(AtomicUsize::new(0));
let activity = Arc::new(Mutex::new(IpcActivityState::new(started_at)));
let (ipc_shutdown_tx, ipc_shutdown_rx) = tokio::sync::oneshot::channel::<()>();
let status = ipc::DaemonStatus {
started_at,
ws_port,
sock_path: sock_path.clone(),
daemon_version: env!("CARGO_PKG_VERSION"),
protocol_version: "1.0",
protocol_version: PROTOCOL_VERSION,
};
let handler = ipc::full_handler(status, Arc::clone(&state));
@@ -314,18 +312,14 @@ pub fn run_foreground(cfg: DaemonConfig) -> Result<()> {
let handler = handler.clone();
let ipc_open = {
let activity = activity.clone();
let active = active_ipc_connections.clone();
move || {
active.fetch_add(1, Ordering::SeqCst);
record_activity(&activity);
record_ipc_open(&activity);
}
};
let ipc_close = {
let activity = activity.clone();
let active = active_ipc_connections.clone();
move || {
active.fetch_sub(1, Ordering::SeqCst);
record_activity(&activity);
record_ipc_close(&activity);
}
};
tokio::spawn(ipc::serve(
@@ -344,7 +338,6 @@ pub fn run_foreground(cfg: DaemonConfig) -> Result<()> {
let idle_task = {
let activity = activity.clone();
let state = Arc::clone(&state);
let active_ipc_connections = active_ipc_connections.clone();
let daemon_idle = cfg.daemon_idle;
tokio::spawn(async move {
let tick = (daemon_idle / 4).max(Duration::from_millis(250));
@@ -352,22 +345,19 @@ pub fn run_foreground(cfg: DaemonConfig) -> Result<()> {
ticker.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Delay);
loop {
ticker.tick().await;
let last = match activity.lock() {
Ok(a) => *a,
Err(p) => *p.into_inner(),
};
if last.elapsed() < daemon_idle {
let ipc_is_idle =
lock_activity(&activity).should_exit_for_idle(Instant::now(), daemon_idle);
if !ipc_is_idle {
continue;
}
// Bridge from M2/M3 (which only knew about connection
// counts) to M4/M5 (browser + session registries):
// hold the daemon alive while any IPC client is
// connected, any browser is paired, or any session
// is live (design §3.2).
if active_ipc_connections.load(Ordering::SeqCst) > 0
|| !state.browsers.is_empty()
|| !state.sessions.is_empty()
{
// IPC liveness (open-connection count + idle window) is already
// enforced above by `should_exit_for_idle`, which reads
// `IpcActivityState` under one lock and returns true only when
// there are zero connections AND the idle interval has elapsed.
// Here we additionally hold the daemon alive while any browser
// is paired or any session is live (design §3.2, M4/M5
// registries).
if !state.browsers.is_empty() || !state.sessions.is_empty() {
continue;
}
info!(
@@ -491,6 +481,7 @@ pub(crate) fn spawn_session_idle_reaper(state: Arc<DaemonState>) -> tokio::task:
&state.session_interrupts,
&session_id,
Duration::from_secs(10),
None,
)
.await
{
@@ -681,10 +672,55 @@ fn restart_start_args(cfg: &DaemonConfig) -> StartArgs {
}
}
fn record_activity(activity: &Arc<Mutex<Instant>>) {
if let Ok(mut a) = activity.lock() {
*a = Instant::now();
#[derive(Debug)]
struct IpcActivityState {
last_activity: Instant,
active_connections: usize,
}
impl IpcActivityState {
fn new(now: Instant) -> Self {
Self {
last_activity: now,
active_connections: 0,
}
}
fn touch(&mut self, now: Instant) {
self.last_activity = now;
}
fn opened(&mut self, now: Instant) {
self.active_connections = self.active_connections.saturating_add(1);
self.last_activity = now;
}
fn closed(&mut self, now: Instant) {
self.active_connections = self.active_connections.saturating_sub(1);
self.last_activity = now;
}
fn should_exit_for_idle(&self, now: Instant, idle: Duration) -> bool {
self.active_connections == 0 && now.saturating_duration_since(self.last_activity) >= idle
}
}
fn lock_activity(
activity: &Arc<Mutex<IpcActivityState>>,
) -> std::sync::MutexGuard<'_, IpcActivityState> {
activity.lock().unwrap_or_else(|poison| poison.into_inner())
}
fn record_activity(activity: &Arc<Mutex<IpcActivityState>>) {
lock_activity(activity).touch(Instant::now());
}
fn record_ipc_open(activity: &Arc<Mutex<IpcActivityState>>) {
lock_activity(activity).opened(Instant::now());
}
fn record_ipc_close(activity: &Arc<Mutex<IpcActivityState>>) {
lock_activity(activity).closed(Instant::now());
}
/// Initialise tracing for the daemon: write a daily-rotated file in
@@ -1067,4 +1103,37 @@ mod tests {
assert_eq!(args.session_idle, Some(Duration::from_secs(11)));
assert_eq!(args.daemon_idle, Some(Duration::from_secs(22)));
}
#[test]
fn ipc_close_restarts_the_idle_window_atomically() {
let started = Instant::now();
let idle = Duration::from_secs(10);
let mut activity = IpcActivityState::new(started);
activity.opened(started);
let closed_at = started + Duration::from_secs(30);
assert!(
!activity.should_exit_for_idle(closed_at, idle),
"an open IPC connection must hold the daemon alive"
);
activity.closed(closed_at);
assert_eq!(activity.active_connections, 0);
assert_eq!(activity.last_activity, closed_at);
assert!(!activity.should_exit_for_idle(closed_at, idle));
assert!(!activity.should_exit_for_idle(closed_at + idle - Duration::from_millis(1), idle));
assert!(activity.should_exit_for_idle(closed_at + idle, idle));
}
#[test]
fn ipc_activity_keeps_the_daemon_alive_until_a_full_idle_interval_passes() {
let started = Instant::now();
let idle = Duration::from_secs(10);
let mut activity = IpcActivityState::new(started);
let request_at = started + Duration::from_secs(20);
activity.touch(request_at);
assert!(!activity.should_exit_for_idle(request_at + Duration::from_secs(9), idle));
assert!(activity.should_exit_for_idle(request_at + idle, idle));
}
}
+3 -3
View File
@@ -16,7 +16,7 @@ use super::start::DaemonConfig;
use super::ws::WsHandle;
pub const DAEMON_VERSION: &str = env!("CARGO_PKG_VERSION");
pub const PROTOCOL_VERSION: &str = "1.0";
pub const PROTOCOL_VERSION: &str = "1.1";
/// Lowest **protocol** version peers must speak (e.g. `"1.0"`).
pub const MIN_COMPATIBLE_PROTOCOL: &str = "1.0";
/// Legacy app-semver floor used only when `HandshakeResult.min_compatible_peer`
@@ -34,8 +34,8 @@ pub struct DaemonState {
/// `stop_session` / browser disconnect.
pub tool_queues: Arc<ToolQueueRegistry>,
/// Per-rpc-id cancellation tokens for daemon-side long-runners
/// (M9.3 — currently only `tool.wait_ms`). The CLI's `cancel
/// { rpc_id }` consults this registry first.
/// (`tool.wait_ms` plus `session.*` lifecycle calls). The CLI's
/// `cancel { rpc_id }` consults this registry first.
pub abort_registry: Arc<AbortRegistry>,
/// Tracks `tool.*` RPCs that have been forwarded to an extension
/// over WS but have not yet received a response. Indexed by the
+10 -10
View File
@@ -108,12 +108,12 @@ async fn send_handshake_with_floors(
async fn handshake_ok_when_protocol_matches() {
let (handle, _sock) = spawn_daemon().await;
let mut ws = open_ws(handle.ws_addr()).await;
let resp = send_handshake(&mut ws, "1.0", env!("CARGO_PKG_VERSION")).await;
let resp = send_handshake(&mut ws, "1.1", env!("CARGO_PKG_VERSION")).await;
let result: HandshakeResult = match resp.body {
ResponseBody::Ok(v) => serde_json::from_value(v).unwrap(),
ResponseBody::Err(e) => panic!("expected ok handshake, got {e:?}"),
};
assert_eq!(result.protocol_version, "1.0");
assert_eq!(result.protocol_version, "1.1");
assert_eq!(
result
.min_compatible_peer
@@ -135,7 +135,7 @@ async fn handshake_ok_when_app_versions_differ_but_protocol_matches() {
let (handle, _sock) = spawn_daemon().await;
let mut ws = open_ws(handle.ws_addr()).await;
let resp =
send_handshake_with_floors(&mut ws, "1.0", "9.9.9", Some("0.0.0"), Some("1.0")).await;
send_handshake_with_floors(&mut ws, "1.1", "9.9.9", Some("0.0.0"), Some("1.1")).await;
match resp.body {
ResponseBody::Ok(_) => {}
other => panic!("expected ok when protocol matches, got {other:?}"),
@@ -149,10 +149,10 @@ async fn handshake_skew_when_protocol_minor_differs() {
let mut ws = open_ws(handle.ws_addr()).await;
let resp = send_handshake_with_floors(
&mut ws,
"1.1",
"1.2",
env!("CARGO_PKG_VERSION"),
Some("0.0.0"),
Some("1.0"),
Some("1.1"),
)
.await;
match resp.body {
@@ -210,7 +210,7 @@ async fn handshake_legacy_ext_without_protocol_floor_still_ok() {
let mut ws = open_ws(handle.ws_addr()).await;
let resp = send_handshake_with_floors(
&mut ws,
"1.0",
"1.1",
env!("CARGO_PKG_VERSION"),
Some("0.1.0"),
None,
@@ -235,7 +235,7 @@ async fn status_surfaces_version_skew_for_skewed_browser() {
browser_name: "chrome".into(),
browser_version: "131.0".into(),
extension_version: "9.9.9".into(),
extension_protocol_version: "1.1".into(),
extension_protocol_version: "1.2".into(),
label: "Older".into(),
sink: bsk::daemon::browsers::BrowserSink { tx },
pending: Mutex::new(bsk::daemon::browsers::Pending::default()),
@@ -263,8 +263,8 @@ async fn status_surfaces_version_skew_for_skewed_browser() {
.iter()
.find(|s| s.instance_id == "skew-only-test")
.expect("status must list our skew client");
assert_eq!(skew.client_protocol_version, "1.1");
assert_eq!(skew.server_protocol_version, "1.0");
assert_eq!(skew.client_protocol_version, "1.2");
assert_eq!(skew.server_protocol_version, "1.1");
assert_eq!(skew.client_version, "9.9.9");
let entry = status
.browsers
@@ -281,7 +281,7 @@ async fn handshake_rejects_when_local_below_peer_min_compatible_protocol() {
let mut ws = open_ws(handle.ws_addr()).await;
let resp = send_handshake_with_floors(
&mut ws,
"1.0",
"1.1",
env!("CARGO_PKG_VERSION"),
Some("0.0.0"),
Some("99.0.0"),
@@ -325,6 +325,7 @@ async fn session_stop_fast_fails_while_tool_is_in_flight() {
&state.session_interrupts,
&sid,
Duration::from_secs(5),
None,
)
.await;
assert!(
@@ -366,6 +367,7 @@ async fn session_stop_fast_fails_while_tool_is_in_flight() {
&state.session_interrupts,
&sid,
Duration::from_secs(5),
None,
)
.await
.expect("session.stop ok after tool completes");
@@ -0,0 +1,327 @@
use std::path::{Path, PathBuf};
use std::sync::Arc;
use std::sync::atomic::{AtomicUsize, Ordering};
use std::time::Duration;
use bsk::daemon::info::DaemonInfo;
use bsk::daemon::paths::BSK_HOME_ENV;
use bsk::daemon::{self, DaemonConfig};
use bsk::ipc_client::IpcClient;
use bsk_protocol::system::{HandshakeParams, HandshakeResult};
use bsk_protocol::tools::{
RecordStopResult, RecordedTrace, RecorderInfo, SessionStartParams, SessionStartResult,
StopReason, TraceEntry, TraceStateV3, TraceV3, VOM_FORMAT_VERSION,
};
use bsk_protocol::{BrowserPeerInfo, Frame, Method, RequestFrame, ResponseBody, ResponseFrame};
use futures_util::{SinkExt, StreamExt};
use serde_json::json;
use tokio::process::Command;
use tokio_tungstenite::tungstenite::handshake::client::generate_key;
use tokio_tungstenite::tungstenite::http::Request;
use tokio_tungstenite::tungstenite::protocol::Message;
const TEST_EXT_ID: &str = "abcdefghijklmnopabcdefghijklmnop";
type Ws =
tokio_tungstenite::WebSocketStream<tokio_tungstenite::MaybeTlsStream<tokio::net::TcpStream>>;
fn bsk_bin() -> PathBuf {
PathBuf::from(env!("CARGO_BIN_EXE_bsk"))
}
async fn connect_extension(addr: std::net::SocketAddr) -> Ws {
let origin = format!("chrome-extension://{TEST_EXT_ID}");
let request = Request::builder()
.method("GET")
.uri(format!("ws://{addr}/"))
.header("Host", addr.to_string())
.header("Upgrade", "websocket")
.header("Connection", "Upgrade")
.header("Sec-WebSocket-Version", "13")
.header("Sec-WebSocket-Key", generate_key())
.header("Origin", origin)
.body(())
.unwrap();
tokio_tungstenite::connect_async(request).await.unwrap().0
}
async fn handshake(ws: &mut Ws) {
let request = RequestFrame {
id: "handshake".into(),
method: Method::SystemHandshake,
params: Some(
serde_json::to_value(HandshakeParams {
client: "browser-skill-extension".into(),
version: "0.1.10".parse().unwrap(),
protocol_version: "1.1".into(),
instance_id: TEST_EXT_ID.into(),
browser: BrowserPeerInfo {
name: "chrome".into(),
version: "131.0".into(),
},
min_compatible_peer: Some("0.1.10".parse().unwrap()),
min_compatible_protocol: Some("1.1".into()),
label: "Record export recovery test".into(),
})
.unwrap(),
),
};
ws.send(Message::Text(serde_json::to_string(&request).unwrap()))
.await
.unwrap();
let Message::Text(response) = ws.next().await.unwrap().unwrap() else {
panic!("expected text handshake response");
};
let response: ResponseFrame = serde_json::from_str(&response).unwrap();
let ResponseBody::Ok(value) = response.body else {
panic!("handshake failed");
};
let _: HandshakeResult = serde_json::from_value(value).unwrap();
}
fn sample_trace() -> TraceV3 {
TraceV3 {
version: 3,
purpose: Some("export recovery".into()),
started_at: Some("2026-08-12T10:00:00Z".into()),
recorded_at: "2026-08-12T10:01:00Z".into(),
stopped_by: StopReason::CliStop,
entry: TraceEntry {
start_url: "https://example.com/".into(),
},
recorder: RecorderInfo {
bsk: "0.1.10".into(),
vom: VOM_FORMAT_VERSION,
},
states: vec![TraceStateV3 {
id: "s1".into(),
url: "https://example.com/".into(),
title: Some("Example".into()),
body: "@vom 1\nL1 page".into(),
truncated: false,
}],
steps: vec![],
}
}
fn run_extension(
mut ws: Ws,
record_stop_calls: Arc<AtomicUsize>,
session_stop_calls: Arc<AtomicUsize>,
) {
tokio::spawn(async move {
while let Some(Ok(Message::Text(text))) = ws.next().await {
let Ok(Frame::Request(request)) = serde_json::from_str::<Frame>(&text) else {
continue;
};
let body = match request.method {
Method::ToolSessionStart => {
let _: SessionStartParams =
serde_json::from_value(request.params.clone().unwrap()).unwrap();
ResponseBody::Ok(
serde_json::to_value(SessionStartResult {
agent_window_id: Some(100),
})
.unwrap(),
)
}
Method::ToolRecordStop => {
record_stop_calls.fetch_add(1, Ordering::SeqCst);
ResponseBody::Ok(
serde_json::to_value(RecordStopResult {
trace: RecordedTrace::V3(sample_trace()),
})
.unwrap(),
)
}
Method::ToolSessionStop => {
session_stop_calls.fetch_add(1, Ordering::SeqCst);
ResponseBody::Ok(json!({}))
}
_ => ResponseBody::Err(bsk_protocol::RpcError {
code: bsk_protocol::ErrorCode::ProtocolError,
message: format!("unexpected method {:?}", request.method),
data: None,
}),
};
let response = ResponseFrame {
id: request.id,
body,
};
ws.send(Message::Text(serde_json::to_string(&response).unwrap()))
.await
.unwrap();
}
});
}
async fn start_session(sock: &Path) -> String {
#[derive(serde::Deserialize)]
struct StartResult {
session_id: String,
}
let mut client = IpcClient::connect(sock).await.unwrap();
client
.call::<(), StartResult>(
"session-start",
Method::SessionStart,
None,
Duration::from_secs(5),
)
.await
.unwrap()
.unwrap()
.session_id
}
fn write_record_session(home: &Path, session_id: &str) {
std::fs::create_dir_all(home).unwrap();
std::fs::write(
home.join("record-session.json"),
format!("{{\"session_id\":\"{session_id}\",\"started_at\":\"1\"}}\n"),
)
.unwrap();
}
async fn run_record_stop(home: &Path, output: &Path) -> std::process::Output {
Command::new(bsk_bin())
.arg("record")
.arg("stop")
.arg("--output")
.arg(output)
.env(BSK_HOME_ENV, home)
.output()
.await
.unwrap()
}
async fn setup_daemon(temp: &Path) -> (bsk::daemon::DaemonHandle, PathBuf, PathBuf) {
let home = temp.join("bsk-home");
let sock = temp.join("daemon.sock");
let daemon = daemon::run(DaemonConfig::new(0), Some(sock.clone()))
.await
.unwrap();
bsk::daemon::info::write_to_path(
&DaemonInfo::now(
std::process::id(),
sock.clone(),
daemon.ws_addr().port(),
env!("CARGO_PKG_VERSION"),
),
&home.join("daemon.json"),
)
.unwrap();
(daemon, home, sock)
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn existing_trace_json_file_is_rejected_before_record_stop() {
let temp = tempfile::Builder::new()
.prefix("bsk-record-json-")
.tempdir()
.unwrap();
let (daemon, home, sock) = setup_daemon(temp.path()).await;
let output = temp.path().join("trace.json");
std::fs::write(&output, "{}\n").unwrap();
let mut extension = connect_extension(daemon.ws_addr()).await;
handshake(&mut extension).await;
let record_stop_calls = Arc::new(AtomicUsize::new(0));
let session_stop_calls = Arc::new(AtomicUsize::new(0));
run_extension(
extension,
Arc::clone(&record_stop_calls),
Arc::clone(&session_stop_calls),
);
let session_id = start_session(&sock).await;
write_record_session(&home, &session_id);
let first = run_record_stop(&home, &output).await;
let stderr = String::from_utf8_lossy(&first.stderr);
assert!(
!first.status.success(),
"legacy JSON --output should fail: {stderr}"
);
assert!(
stderr.contains("JSON file") || stderr.contains("bundle directory"),
"{stderr}"
);
let state: serde_json::Value =
serde_json::from_str(&std::fs::read_to_string(home.join("record-session.json")).unwrap())
.unwrap();
assert_eq!(state["session_id"], session_id);
assert_eq!(
record_stop_calls.load(Ordering::SeqCst),
0,
"existing trace.json must be rejected before the extension returns the Trace"
);
assert_eq!(session_stop_calls.load(Ordering::SeqCst), 0);
assert_eq!(std::fs::read_to_string(&output).unwrap(), "{}\n");
daemon.shutdown().await;
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn failed_bundle_export_can_be_recovered() {
let temp = tempfile::Builder::new()
.prefix("bsk-record-recov-")
.tempdir()
.unwrap();
let (daemon, home, sock) = setup_daemon(temp.path()).await;
let output = temp.path().join("trace");
std::fs::create_dir_all(&output).unwrap();
std::fs::write(output.join("states"), "not a directory\n").unwrap();
let mut extension = connect_extension(daemon.ws_addr()).await;
handshake(&mut extension).await;
let record_stop_calls = Arc::new(AtomicUsize::new(0));
let session_stop_calls = Arc::new(AtomicUsize::new(0));
run_extension(
extension,
Arc::clone(&record_stop_calls),
Arc::clone(&session_stop_calls),
);
let session_id = start_session(&sock).await;
write_record_session(&home, &session_id);
let first = run_record_stop(&home, &output).await;
let stderr = String::from_utf8_lossy(&first.stderr);
assert!(
!first.status.success(),
"blocked states path should fail export: {stderr}"
);
assert!(
home.join("record-recovery.json").exists(),
"completed Trace must be saved before the failed export"
);
assert!(
!home.join("record-session.json").exists(),
"session state is cleared after the extension returns the Trace"
);
assert_eq!(record_stop_calls.load(Ordering::SeqCst), 1);
assert_eq!(session_stop_calls.load(Ordering::SeqCst), 1);
std::fs::remove_file(output.join("states")).unwrap();
let second = run_record_stop(&home, &output).await;
assert!(
second.status.success(),
"recovery export failed: {}",
String::from_utf8_lossy(&second.stderr)
);
assert_eq!(
record_stop_calls.load(Ordering::SeqCst),
1,
"recovery must not call the extension again"
);
assert!(output.join("trace.json").exists());
assert!(output.join("states/s1.txt").exists());
assert!(
!home.join("record-recovery.json").exists(),
"successful export must delete recovery data"
);
daemon.shutdown().await;
}
+271
View File
@@ -0,0 +1,271 @@
use std::path::{Path, PathBuf};
use std::sync::Arc;
use std::sync::atomic::{AtomicUsize, Ordering};
use std::time::Duration;
use bsk::cli::record_state;
use bsk::daemon::info::DaemonInfo;
use bsk::daemon::paths::BSK_HOME_ENV;
use bsk::daemon::{self, DaemonConfig};
use bsk::ipc_client::IpcClient;
use bsk_protocol::system::{HandshakeParams, HandshakeResult};
use bsk_protocol::tools::{
RecordStopResult, RecordedTrace, RecorderInfo, SessionStartParams, SessionStartResult,
StopReason, TraceEntry, TraceStateV3, TraceV3, VOM_FORMAT_VERSION,
};
use bsk_protocol::{
BrowserPeerInfo, ErrorCode, Frame, Method, RequestFrame, ResponseBody, ResponseFrame, RpcError,
};
use futures_util::{SinkExt, StreamExt};
use serde_json::json;
use tokio::process::Command;
use tokio_tungstenite::tungstenite::handshake::client::generate_key;
use tokio_tungstenite::tungstenite::http::Request;
use tokio_tungstenite::tungstenite::protocol::Message;
const TEST_EXT_ID: &str = "abcdefghijklmnopabcdefghijklmnop";
type Ws =
tokio_tungstenite::WebSocketStream<tokio_tungstenite::MaybeTlsStream<tokio::net::TcpStream>>;
fn bsk_bin() -> PathBuf {
PathBuf::from(env!("CARGO_BIN_EXE_bsk"))
}
async fn connect_extension(addr: std::net::SocketAddr) -> Ws {
let origin = format!("chrome-extension://{TEST_EXT_ID}");
let request = Request::builder()
.method("GET")
.uri(format!("ws://{addr}/"))
.header("Host", addr.to_string())
.header("Upgrade", "websocket")
.header("Connection", "Upgrade")
.header("Sec-WebSocket-Version", "13")
.header("Sec-WebSocket-Key", generate_key())
.header("Origin", origin)
.body(())
.unwrap();
tokio_tungstenite::connect_async(request).await.unwrap().0
}
async fn handshake(ws: &mut Ws) {
let request = RequestFrame {
id: "handshake".into(),
method: Method::SystemHandshake,
params: Some(
serde_json::to_value(HandshakeParams {
client: "browser-skill-extension".into(),
version: "0.1.10".parse().unwrap(),
protocol_version: "1.1".into(),
instance_id: TEST_EXT_ID.into(),
browser: BrowserPeerInfo {
name: "chrome".into(),
version: "131.0".into(),
},
min_compatible_peer: Some("0.1.10".parse().unwrap()),
min_compatible_protocol: Some("1.1".into()),
label: "Record stop retry test".into(),
})
.unwrap(),
),
};
ws.send(Message::Text(serde_json::to_string(&request).unwrap()))
.await
.unwrap();
let Message::Text(response) = ws.next().await.unwrap().unwrap() else {
panic!("expected text handshake response");
};
let response: ResponseFrame = serde_json::from_str(&response).unwrap();
let ResponseBody::Ok(value) = response.body else {
panic!("handshake failed");
};
let _: HandshakeResult = serde_json::from_value(value).unwrap();
}
fn sample_trace() -> TraceV3 {
TraceV3 {
version: 3,
purpose: Some("retry record stop".into()),
started_at: Some("2026-08-12T10:00:00Z".into()),
recorded_at: "2026-08-12T10:01:00Z".into(),
stopped_by: StopReason::CliStop,
entry: TraceEntry {
start_url: "https://example.com/".into(),
},
recorder: RecorderInfo {
bsk: "0.1.10".into(),
vom: VOM_FORMAT_VERSION,
},
states: vec![TraceStateV3 {
id: "s1".into(),
url: "https://example.com/".into(),
title: Some("Example".into()),
body: "@vom 1\nL1 page".into(),
truncated: false,
}],
steps: vec![],
}
}
fn run_extension(
mut ws: Ws,
record_stop_calls: Arc<AtomicUsize>,
session_stop_calls: Arc<AtomicUsize>,
) {
tokio::spawn(async move {
while let Some(Ok(Message::Text(text))) = ws.next().await {
let Ok(Frame::Request(request)) = serde_json::from_str::<Frame>(&text) else {
continue;
};
let body = match request.method {
Method::ToolSessionStart => {
let _: SessionStartParams =
serde_json::from_value(request.params.clone().unwrap()).unwrap();
ResponseBody::Ok(
serde_json::to_value(SessionStartResult {
agent_window_id: Some(100),
})
.unwrap(),
)
}
Method::ToolRecordStop => {
if record_stop_calls.fetch_add(1, Ordering::SeqCst) == 0 {
ResponseBody::Err(RpcError {
code: ErrorCode::ProtocolError,
message: "recording remains active; retry record stop".into(),
data: Some(json!({ "recording": true, "retryable": true })),
})
} else {
ResponseBody::Ok(
serde_json::to_value(RecordStopResult {
trace: RecordedTrace::V3(sample_trace()),
})
.unwrap(),
)
}
}
Method::ToolSessionStop => {
session_stop_calls.fetch_add(1, Ordering::SeqCst);
ResponseBody::Ok(json!({}))
}
_ => ResponseBody::Err(RpcError {
code: ErrorCode::ProtocolError,
message: format!("unexpected method {:?}", request.method),
data: None,
}),
};
let response = ResponseFrame {
id: request.id,
body,
};
ws.send(Message::Text(serde_json::to_string(&response).unwrap()))
.await
.unwrap();
}
});
}
async fn start_session(sock: &Path) -> String {
#[derive(serde::Deserialize)]
struct StartResult {
session_id: String,
}
let mut client = IpcClient::connect(sock).await.unwrap();
client
.call::<(), StartResult>(
"session-start",
Method::SessionStart,
None,
Duration::from_secs(5),
)
.await
.unwrap()
.unwrap()
.session_id
}
async fn run_record_stop(home: &Path, output: &Path) -> std::process::Output {
Command::new(bsk_bin())
.arg("record")
.arg("stop")
.arg("--output")
.arg(output)
.env(BSK_HOME_ENV, home)
.output()
.await
.unwrap()
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn failed_record_stop_keeps_state_and_session_for_retry() {
// Keep the Unix-domain socket below macOS's SUN_LEN even from a deep worktree.
let temp = tempfile::Builder::new()
.prefix("bsk-record-stop-")
.tempdir()
.unwrap();
let home = temp.path().join("bsk-home");
let sock = temp.path().join("daemon.sock");
let output = temp.path().join("trace");
let daemon = daemon::run(DaemonConfig::new(0), Some(sock.clone()))
.await
.unwrap();
bsk::daemon::info::write_to_path(
&DaemonInfo::now(
std::process::id(),
sock.clone(),
daemon.ws_addr().port(),
env!("CARGO_PKG_VERSION"),
),
&home.join("daemon.json"),
)
.unwrap();
let mut extension = connect_extension(daemon.ws_addr()).await;
handshake(&mut extension).await;
let record_stop_calls = Arc::new(AtomicUsize::new(0));
let session_stop_calls = Arc::new(AtomicUsize::new(0));
run_extension(
extension,
Arc::clone(&record_stop_calls),
Arc::clone(&session_stop_calls),
);
let session_id = start_session(&sock).await;
unsafe {
std::env::set_var(BSK_HOME_ENV, &home);
}
record_state::write(&session_id).unwrap();
let first = run_record_stop(&home, &output).await;
assert!(
!first.status.success(),
"first stop unexpectedly succeeded: {}",
String::from_utf8_lossy(&first.stdout)
);
assert_eq!(record_state::read().unwrap().session_id, session_id);
assert_eq!(record_stop_calls.load(Ordering::SeqCst), 1);
assert_eq!(
session_stop_calls.load(Ordering::SeqCst),
0,
"failed record stop must leave the session active"
);
let second = run_record_stop(&home, &output).await;
assert!(
second.status.success(),
"retry failed: {}",
String::from_utf8_lossy(&second.stderr)
);
assert!(record_state::read().is_err());
assert_eq!(record_stop_calls.load(Ordering::SeqCst), 2);
assert_eq!(session_stop_calls.load(Ordering::SeqCst), 1);
assert!(output.join("trace.json").exists());
assert!(output.join("states/s1.txt").exists());
unsafe {
std::env::remove_var(BSK_HOME_ENV);
}
daemon.shutdown().await;
}
+368 -10
View File
@@ -10,9 +10,14 @@ use std::time::Duration;
use bsk::daemon::{self, DaemonConfig};
use bsk::ipc_client::IpcClient;
use bsk_protocol::cancel::{CancelParams, CancelResult};
use bsk_protocol::system::{HandshakeParams, HandshakeResult, StatusResult};
use bsk_protocol::tools::{SessionStartParams, SessionStartResult, SessionStopParams};
use bsk_protocol::{BrowserPeerInfo, Frame, Method, RequestFrame, ResponseBody, ResponseFrame};
use bsk_protocol::tools::{
SessionStartParams, SessionStartResult, SessionStopParams, SessionStopResult,
};
use bsk_protocol::{
BrowserPeerInfo, ErrorCode, Frame, Method, RequestFrame, ResponseBody, ResponseFrame, RpcError,
};
use futures_util::{SinkExt, StreamExt};
use rand::Rng;
use tokio_tungstenite::tungstenite::handshake::client::generate_key;
@@ -23,6 +28,9 @@ use support::{wait_for_browser_count, wait_for_no_sessions};
const TEST_EXT_ID: &str = "abcdefghijklmnopabcdefghijklmnop";
type TestWs =
tokio_tungstenite::WebSocketStream<tokio_tungstenite::MaybeTlsStream<tokio::net::TcpStream>>;
fn tempfile_path(prefix: &str) -> PathBuf {
let mut p = std::env::temp_dir();
let mut rng = rand::thread_rng();
@@ -56,9 +64,7 @@ async fn spawn_daemon_with_session_idle(session_idle: Duration) -> (daemon::Daem
(handle, sock)
}
async fn connect_ext(
addr: std::net::SocketAddr,
) -> tokio_tungstenite::WebSocketStream<tokio_tungstenite::MaybeTlsStream<tokio::net::TcpStream>> {
async fn connect_ext(addr: std::net::SocketAddr) -> TestWs {
let origin = format!("chrome-extension://{TEST_EXT_ID}");
let url = format!("ws://{addr}/");
let req = Request::builder()
@@ -76,11 +82,7 @@ async fn connect_ext(
ws
}
async fn handshake_as_ext(
ws: &mut tokio_tungstenite::WebSocketStream<
tokio_tungstenite::MaybeTlsStream<tokio::net::TcpStream>,
>,
) -> HandshakeResult {
async fn handshake_as_ext(ws: &mut TestWs) -> HandshakeResult {
let params = HandshakeParams {
client: "browser-skill-extension".into(),
version: "0.1.0-dev.0".parse().unwrap(),
@@ -114,6 +116,128 @@ async fn handshake_as_ext(
}
}
async fn next_extension_request(ws: &mut TestWs) -> RequestFrame {
loop {
let message = ws.next().await.expect("daemon closed websocket").unwrap();
match message {
Message::Text(text) => match serde_json::from_str::<Frame>(&text).unwrap() {
Frame::Request(request) => return request,
Frame::Response(_) | Frame::Event(_) => continue,
},
Message::Ping(payload) => ws.send(Message::Pong(payload)).await.unwrap(),
Message::Close(_) => panic!("daemon closed websocket before sending request"),
_ => {}
}
}
}
async fn send_extension_response(ws: &mut TestWs, response: ResponseFrame) {
ws.send(Message::Text(serde_json::to_string(&response).unwrap()))
.await
.unwrap();
}
async fn acknowledge_extension_cancel(ws: &mut TestWs, cancel: RequestFrame, target_rpc_id: &str) {
assert_eq!(cancel.method, Method::Cancel);
let params: CancelParams = serde_json::from_value(cancel.params.unwrap()).unwrap();
assert_eq!(params.rpc_id, target_rpc_id);
send_extension_response(
ws,
ResponseFrame {
id: cancel.id,
body: ResponseBody::Ok(serde_json::to_value(CancelResult { cancelled: true }).unwrap()),
},
)
.await;
}
async fn respond_to_aborted_start(
ws: &mut TestWs,
start_seen: Option<tokio::sync::oneshot::Sender<()>>,
agent_window_id: i64,
) {
let start = next_extension_request(ws).await;
assert_eq!(start.method, Method::ToolSessionStart);
let params: SessionStartParams = serde_json::from_value(start.params.clone().unwrap()).unwrap();
if let Some(start_seen) = start_seen {
start_seen.send(()).unwrap();
}
let cancel = next_extension_request(ws).await;
acknowledge_extension_cancel(ws, cancel, &start.id).await;
// Model the hardest race: chrome.windows.create completed just as
// cancellation arrived, so the extension's original call succeeds.
send_extension_response(
ws,
ResponseFrame {
id: start.id,
body: ResponseBody::Ok(
serde_json::to_value(SessionStartResult {
agent_window_id: Some(agent_window_id),
})
.unwrap(),
),
},
)
.await;
let rollback = next_extension_request(ws).await;
assert_eq!(rollback.method, Method::ToolSessionStop);
let rollback_params: SessionStopParams =
serde_json::from_value(rollback.params.unwrap()).unwrap();
assert_eq!(rollback_params.session_id, params.session_id);
send_extension_response(
ws,
ResponseFrame {
id: rollback.id,
body: ResponseBody::Ok(serde_json::to_value(SessionStopResult::default()).unwrap()),
},
)
.await;
}
async fn respond_to_aborted_stop(
ws: &mut TestWs,
stop_seen: Option<tokio::sync::oneshot::Sender<()>>,
) {
let start = next_extension_request(ws).await;
assert_eq!(start.method, Method::ToolSessionStart);
send_extension_response(
ws,
ResponseFrame {
id: start.id,
body: ResponseBody::Ok(
serde_json::to_value(SessionStartResult {
agent_window_id: Some(4242),
})
.unwrap(),
),
},
)
.await;
let stop = next_extension_request(ws).await;
assert_eq!(stop.method, Method::ToolSessionStop);
if let Some(stop_seen) = stop_seen {
stop_seen.send(()).unwrap();
}
let cancel = next_extension_request(ws).await;
acknowledge_extension_cancel(ws, cancel, &stop.id).await;
send_extension_response(
ws,
ResponseFrame {
id: stop.id,
body: ResponseBody::Err(RpcError {
code: ErrorCode::Cancelled,
message: "session_stop aborted before window close".into(),
data: None,
}),
},
)
.await;
}
#[tokio::test]
async fn session_start_stop_round_trip_via_ipc() {
let (handle, sock) = spawn_daemon().await;
@@ -250,6 +374,240 @@ async fn session_start_stop_round_trip_via_ipc() {
handle.shutdown().await;
}
#[tokio::test]
async fn cancelling_session_start_rolls_back_a_late_extension_success() {
const START_RPC_ID: &str = "session-start-cancel";
let (handle, sock) = spawn_daemon().await;
let mut ws = connect_ext(handle.ws_addr()).await;
let _ = handshake_as_ext(&mut ws).await;
let (start_seen_tx, start_seen_rx) = tokio::sync::oneshot::channel();
let (release_tx, release_rx) = tokio::sync::oneshot::channel();
let responder = tokio::spawn(async move {
respond_to_aborted_start(&mut ws, Some(start_seen_tx), 4242).await;
let _ = release_rx.await;
});
let mut start_ipc = IpcClient::connect(&sock).await.unwrap();
let start_call = tokio::spawn(async move {
let outcome: Result<serde_json::Value, RpcError> = start_ipc
.call_with_id(
START_RPC_ID.to_string(),
Method::SessionStart,
Some(serde_json::json!({ "focused": false })),
Duration::from_secs(5),
)
.await
.unwrap();
outcome
});
start_seen_rx.await.unwrap();
let mut cancel_ipc = IpcClient::connect(&sock).await.unwrap();
let cancel: CancelResult = cancel_ipc
.call(
"cancel-start",
Method::Cancel,
Some(CancelParams {
rpc_id: START_RPC_ID.to_string(),
}),
Duration::from_secs(2),
)
.await
.unwrap()
.unwrap();
assert!(cancel.cancelled);
let start_error = start_call.await.unwrap().unwrap_err();
assert_eq!(start_error.code, ErrorCode::Cancelled);
let mut status_ipc = IpcClient::connect(&sock).await.unwrap();
let status: StatusResult = status_ipc
.call::<(), _>(
"status-after-start-cancel",
Method::SystemStatus,
None,
Duration::from_secs(2),
)
.await
.unwrap()
.unwrap();
assert!(status.sessions.is_empty());
assert_eq!(status.browsers[0].session_count, 0);
let _ = release_tx.send(());
responder.await.unwrap();
handle.shutdown().await;
}
#[tokio::test]
async fn timing_out_session_start_rolls_back_a_late_extension_success() {
let (handle, _sock) = spawn_daemon().await;
let state = handle.state();
let mut ws = connect_ext(handle.ws_addr()).await;
let _ = handshake_as_ext(&mut ws).await;
let (release_tx, release_rx) = tokio::sync::oneshot::channel();
let responder = tokio::spawn(async move {
respond_to_aborted_start(&mut ws, None, 4343).await;
let _ = release_rx.await;
});
let start = bsk::daemon::sessions::start_session(
&state.browsers,
&state.sessions,
&state.tool_queues,
None,
bsk::daemon::sessions::AgentWindowOptions::default(),
Duration::ZERO,
Duration::from_millis(20),
None,
)
.await;
assert!(matches!(
start,
Err(bsk::daemon::sessions::StartSessionError::Timeout)
));
assert!(state.sessions.is_empty());
let _ = release_tx.send(());
responder.await.unwrap();
handle.shutdown().await;
}
#[tokio::test]
async fn cancelling_session_stop_keeps_the_session_retryable() {
const STOP_RPC_ID: &str = "session-stop-cancel";
let (handle, sock) = spawn_daemon().await;
let mut ws = connect_ext(handle.ws_addr()).await;
let _ = handshake_as_ext(&mut ws).await;
let (stop_seen_tx, stop_seen_rx) = tokio::sync::oneshot::channel();
let (release_tx, release_rx) = tokio::sync::oneshot::channel();
let responder = tokio::spawn(async move {
respond_to_aborted_stop(&mut ws, Some(stop_seen_tx)).await;
let _ = release_rx.await;
});
let mut start_ipc = IpcClient::connect(&sock).await.unwrap();
let start: serde_json::Value = start_ipc
.call(
"start-before-stop-cancel",
Method::SessionStart,
Some(serde_json::json!({ "focused": false })),
Duration::from_secs(5),
)
.await
.unwrap()
.unwrap();
let session_id = start["session_id"].as_str().unwrap().to_string();
let mut stop_ipc = IpcClient::connect(&sock).await.unwrap();
let stop_session_id = session_id.clone();
let stop_call = tokio::spawn(async move {
let outcome: Result<serde_json::Value, RpcError> = stop_ipc
.call_with_id(
STOP_RPC_ID.to_string(),
Method::SessionStop,
Some(serde_json::json!({
"session_id": stop_session_id,
"all": false,
})),
Duration::from_secs(5),
)
.await
.unwrap();
outcome
});
stop_seen_rx.await.unwrap();
let mut cancel_ipc = IpcClient::connect(&sock).await.unwrap();
let cancel: CancelResult = cancel_ipc
.call(
"cancel-stop",
Method::Cancel,
Some(CancelParams {
rpc_id: STOP_RPC_ID.to_string(),
}),
Duration::from_secs(2),
)
.await
.unwrap()
.unwrap();
assert!(cancel.cancelled);
let stop_error = stop_call.await.unwrap().unwrap_err();
assert_eq!(stop_error.code, ErrorCode::Cancelled);
let mut status_ipc = IpcClient::connect(&sock).await.unwrap();
let status: StatusResult = status_ipc
.call::<(), _>(
"status-after-stop-cancel",
Method::SystemStatus,
None,
Duration::from_secs(2),
)
.await
.unwrap()
.unwrap();
assert_eq!(status.sessions.len(), 1);
assert_eq!(status.sessions[0].session_id, session_id);
let _ = release_tx.send(());
responder.await.unwrap();
handle.shutdown().await;
}
#[tokio::test]
async fn timing_out_session_stop_cancels_extension_teardown_and_keeps_session() {
let (handle, sock) = spawn_daemon().await;
let state = handle.state();
let mut ws = connect_ext(handle.ws_addr()).await;
let _ = handshake_as_ext(&mut ws).await;
let (release_tx, release_rx) = tokio::sync::oneshot::channel();
let responder = tokio::spawn(async move {
respond_to_aborted_stop(&mut ws, None).await;
let _ = release_rx.await;
});
let mut start_ipc = IpcClient::connect(&sock).await.unwrap();
let start: serde_json::Value = start_ipc
.call(
"start-before-stop-timeout",
Method::SessionStart,
Some(serde_json::json!({ "focused": false })),
Duration::from_secs(5),
)
.await
.unwrap()
.unwrap();
let session_id =
bsk::daemon::sessions::SessionId(start["session_id"].as_str().unwrap().to_string());
let stop = bsk::daemon::sessions::stop_session(
&state.browsers,
&state.sessions,
&state.tool_queues,
&state.session_interrupts,
&session_id,
Duration::from_millis(20),
None,
)
.await;
assert!(matches!(
stop,
Err(bsk::daemon::sessions::StopSessionError::Timeout)
));
assert!(state.sessions.get(&session_id).is_some());
let _ = release_tx.send(());
responder.await.unwrap();
handle.shutdown().await;
}
#[tokio::test]
async fn session_idle_timeout_stops_and_unregisters_session() {
let (handle, _sock) = spawn_daemon_with_session_idle(Duration::from_millis(50)).await;
+1 -1
View File
@@ -61,7 +61,7 @@ fn bsk_status_json_returns_structured_payload() {
assert!(parsed["pid"].as_u64().unwrap() > 0);
assert!(!parsed["daemon_version"].as_str().unwrap().is_empty());
assert_eq!(parsed["protocol_version"], "1.0");
assert_eq!(parsed["protocol_version"], "1.1");
assert!(parsed["sock_path"].as_str().is_some());
assert_eq!(parsed["browsers"], serde_json::json!([]));
assert_eq!(parsed["sessions"], serde_json::json!([]));
+2 -2
View File
@@ -53,7 +53,7 @@ pub async fn send_handshake(
let params = HandshakeParams {
client: "browser-skill-extension".into(),
version: "0.1.0-dev.0".parse().unwrap(),
protocol_version: "1.0".into(),
protocol_version: "1.1".into(),
instance_id: instance_id.into(),
browser: BrowserPeerInfo {
name: "chrome".into(),
@@ -91,7 +91,7 @@ async fn ws_handshake_registers_browser_in_state() {
let mut ws = connect_ext(handle.ws_addr(), &origin).await;
let result = send_handshake(&mut ws, TEST_EXT_ID).await;
assert_eq!(result.server, "browser-skill-daemon");
assert_eq!(result.protocol_version, "1.0");
assert_eq!(result.protocol_version, "1.1");
let state = handle.state();
let browsers = state.browsers.snapshot();
@@ -7,10 +7,18 @@
],
"properties": {
"trace": {
"$ref": "#/definitions/Trace"
"$ref": "#/definitions/RecordedTrace"
}
},
"definitions": {
"FillCommit": {
"type": "string",
"enum": [
"enter",
"suggestion",
"blur"
]
},
"KeyModifier": {
"description": "Keyboard modifier flags. Multiple flags may be combined; the extension folds them into CDP's bitfield (`alt=1, ctrl=2, meta=4, shift=8`).",
"type": "string",
@@ -21,7 +29,19 @@
"shift"
]
},
"PageRef": {
"NavigationCause": {
"type": "string",
"enum": [
"user_typed",
"link",
"form_submit",
"reload",
"history",
"script",
"browser"
]
},
"PageRefV2": {
"description": "Page context dictionary entry — referenced by steps via `page` id.",
"type": "object",
"required": [
@@ -43,7 +63,42 @@
}
}
},
"SelectedOption": {
"RecordedTrace": {
"if": {
"required": [
"version"
],
"properties": {
"version": {
"type": "integer"
}
}
},
"then": {
"$ref": "#/definitions/TraceV3"
},
"else": {
"$ref": "#/definitions/TraceV2"
}
},
"RecorderInfo": {
"type": "object",
"required": [
"bsk",
"vom"
],
"properties": {
"bsk": {
"type": "string"
},
"vom": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
}
}
},
"SelectedOptionV2": {
"description": "One selected option (`select` op).",
"type": "object",
"required": [
@@ -61,11 +116,53 @@
}
}
},
"Step": {
"description": "One recorded user action — discriminated union by `op`.",
"SelectedOptionV3": {
"description": "One selected option (`select` op).",
"type": "object",
"required": [
"value"
],
"properties": {
"label": {
"type": [
"string",
"null"
]
},
"value": {
"type": "string"
}
}
},
"StepEffectV2": {
"description": "Observed navigation after a step (objective fact only).",
"type": "object",
"required": [
"navigated_to"
],
"properties": {
"navigated_to": {
"description": "Reference into `pages[]` for the destination page.",
"type": "string"
}
}
},
"StepResultV3": {
"type": "object",
"required": [
"state"
],
"properties": {
"state": {
"type": "string"
}
}
},
"StepV2": {
"description": "One recorded user action — discriminated union by `op` (v2).",
"oneOf": [
{
"description": "Fields shared by every step variant (flattened in JSON).",
"description": "Fields shared by every v2 step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
@@ -77,7 +174,7 @@
"effect": {
"anyOf": [
{
"$ref": "#/definitions/StepEffect"
"$ref": "#/definitions/StepEffectV2"
},
{
"type": "null"
@@ -105,7 +202,7 @@
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"description": "Fields shared by every v2 step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
@@ -117,7 +214,7 @@
"effect": {
"anyOf": [
{
"$ref": "#/definitions/StepEffect"
"$ref": "#/definitions/StepEffectV2"
},
{
"type": "null"
@@ -140,12 +237,12 @@
"type": "string"
},
"target": {
"$ref": "#/definitions/TargetDescriptor"
"$ref": "#/definitions/TargetDescriptorV2"
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"description": "Fields shared by every v2 step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
@@ -157,7 +254,7 @@
"effect": {
"anyOf": [
{
"$ref": "#/definitions/StepEffect"
"$ref": "#/definitions/StepEffectV2"
},
{
"type": "null"
@@ -180,12 +277,12 @@
"type": "string"
},
"target": {
"$ref": "#/definitions/TargetDescriptor"
"$ref": "#/definitions/TargetDescriptorV2"
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"description": "Fields shared by every v2 step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
@@ -198,7 +295,7 @@
"effect": {
"anyOf": [
{
"$ref": "#/definitions/StepEffect"
"$ref": "#/definitions/StepEffectV2"
},
{
"type": "null"
@@ -227,7 +324,7 @@
]
},
"target": {
"$ref": "#/definitions/TargetDescriptor"
"$ref": "#/definitions/TargetDescriptorV2"
},
"value": {
"type": "string"
@@ -235,7 +332,7 @@
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"description": "Fields shared by every v2 step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
@@ -248,7 +345,7 @@
"effect": {
"anyOf": [
{
"$ref": "#/definitions/StepEffect"
"$ref": "#/definitions/StepEffectV2"
},
{
"type": "null"
@@ -273,16 +370,16 @@
"selection": {
"type": "array",
"items": {
"$ref": "#/definitions/SelectedOption"
"$ref": "#/definitions/SelectedOptionV2"
}
},
"target": {
"$ref": "#/definitions/TargetDescriptor"
"$ref": "#/definitions/TargetDescriptorV2"
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"description": "Fields shared by every v2 step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
@@ -294,7 +391,7 @@
"effect": {
"anyOf": [
{
"$ref": "#/definitions/StepEffect"
"$ref": "#/definitions/StepEffectV2"
},
{
"type": "null"
@@ -331,7 +428,7 @@
"target": {
"anyOf": [
{
"$ref": "#/definitions/TargetDescriptor"
"$ref": "#/definitions/TargetDescriptorV2"
},
{
"type": "null"
@@ -342,21 +439,324 @@
}
]
},
"StepEffect": {
"description": "Observed navigation after a step (objective fact only).",
"type": "object",
"required": [
"navigated_to"
],
"properties": {
"navigated_to": {
"description": "Reference into `pages[]` for the destination page.",
"type": "string"
"StepV3": {
"description": "One recorded user action — discriminated union by `op`.",
"oneOf": [
{
"description": "Fields shared by every step variant (flattened in JSON).",
"type": "object",
"required": [
"cause",
"id",
"op",
"result",
"state",
"to"
],
"properties": {
"cause": {
"$ref": "#/definitions/NavigationCause"
},
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"navigate"
]
},
"result": {
"$ref": "#/definitions/StepResultV3"
},
"state": {
"description": "Observation id immediately before this action.",
"type": "string"
},
"to": {
"type": "string"
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
"op",
"result",
"state"
],
"properties": {
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"switch_tab"
]
},
"result": {
"$ref": "#/definitions/StepResultV3"
},
"state": {
"description": "Observation id immediately before this action.",
"type": "string"
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
"op",
"result",
"state",
"target"
],
"properties": {
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"click"
]
},
"result": {
"$ref": "#/definitions/StepResultV3"
},
"state": {
"description": "Observation id immediately before this action.",
"type": "string"
},
"target": {
"$ref": "#/definitions/TargetDescriptorV3"
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
"op",
"result",
"state",
"target"
],
"properties": {
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"hover"
]
},
"result": {
"$ref": "#/definitions/StepResultV3"
},
"state": {
"description": "Observation id immediately before this action.",
"type": "string"
},
"target": {
"$ref": "#/definitions/TargetDescriptorV3"
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"type": "object",
"required": [
"commit",
"id",
"op",
"result",
"state",
"target",
"value"
],
"properties": {
"commit": {
"$ref": "#/definitions/FillCommit"
},
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"fill"
]
},
"redacted": {
"type": "boolean"
},
"result": {
"$ref": "#/definitions/StepResultV3"
},
"state": {
"description": "Observation id immediately before this action.",
"type": "string"
},
"target": {
"$ref": "#/definitions/TargetDescriptorV3"
},
"value": {
"type": "string"
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
"op",
"result",
"state",
"target"
],
"properties": {
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"select"
]
},
"result": {
"$ref": "#/definitions/StepResultV3"
},
"selection": {
"type": "array",
"items": {
"$ref": "#/definitions/SelectedOptionV3"
}
},
"state": {
"description": "Observation id immediately before this action.",
"type": "string"
},
"target": {
"$ref": "#/definitions/TargetDescriptorV3"
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
"key",
"op",
"result",
"state"
],
"properties": {
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"key": {
"type": "string"
},
"modifiers": {
"type": [
"array",
"null"
],
"items": {
"$ref": "#/definitions/KeyModifier"
}
},
"op": {
"type": "string",
"enum": [
"press"
]
},
"result": {
"$ref": "#/definitions/StepResultV3"
},
"state": {
"description": "Observation id immediately before this action.",
"type": "string"
},
"target": {
"anyOf": [
{
"$ref": "#/definitions/TargetDescriptorV3"
},
{
"type": "null"
}
]
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
"op",
"result",
"state"
],
"properties": {
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"scroll"
]
},
"result": {
"$ref": "#/definitions/StepResultV3"
},
"state": {
"description": "Observation id immediately before this action.",
"type": "string"
}
}
}
}
]
},
"TargetDescriptor": {
"description": "Stable semantic handle for an interacted element.\n\n`name` and `nearby_label` are **untrusted page text**.",
"StopReason": {
"type": "string",
"enum": [
"user_finish",
"cli_stop"
]
},
"TargetDescriptorV2": {
"description": "Stable semantic handle for an interacted element (v2).\n\n`name` and `nearby_label` are **untrusted page text**.",
"type": "object",
"required": [
"tag"
@@ -397,8 +797,83 @@
}
}
},
"Trace": {
"description": "Persisted user-action trace exported by `tool.record_stop` / `await`.",
"TargetDescriptorV3": {
"description": "Stable semantic handle for an interacted element within a page observation.",
"type": "object",
"properties": {
"ctx": {
"type": [
"string",
"null"
]
},
"name": {
"type": [
"string",
"null"
]
},
"ref": {
"type": [
"string",
"null"
]
},
"role": {
"type": [
"string",
"null"
]
},
"unmatched": {
"type": "boolean"
}
}
},
"TraceEntry": {
"type": "object",
"required": [
"start_url"
],
"properties": {
"start_url": {
"type": "string"
}
}
},
"TraceStateV3": {
"description": "Page observation dictionary entry — referenced by steps via `state` / `result.state`.",
"type": "object",
"required": [
"body",
"id",
"url"
],
"properties": {
"body": {
"description": "Full page observation (front matter + VOM body + annotations).",
"type": "string"
},
"id": {
"type": "string"
},
"title": {
"type": [
"string",
"null"
]
},
"truncated": {
"type": "boolean"
},
"url": {
"type": "string"
}
}
},
"TraceV2": {
"title": "TraceV2",
"description": "Persisted user-action trace exported by legacy `tool.record_stop` / `await`.\n\nUnknown extension fields are ignored so older traces remain readable. `states[]` and a numeric `version` are reserved for Trace v3 / `RecordedTrace` classification.",
"type": "object",
"required": [
"entry",
@@ -413,11 +888,10 @@
"pages": {
"type": "array",
"items": {
"$ref": "#/definitions/PageRef"
"$ref": "#/definitions/PageRefV2"
}
},
"purpose": {
"description": "Optional user-provided goal from `--purpose` (metadata only).",
"type": [
"string",
"null"
@@ -433,25 +907,76 @@
"null"
]
},
"states": false,
"steps": {
"type": "array",
"items": {
"$ref": "#/definitions/Step"
"$ref": "#/definitions/StepV2"
}
},
"version": {
"description": "Numeric version selects Trace v3. Legacy v2 envelopes omit this field.",
"not": {
"type": "integer"
}
}
}
},
"TraceEntry": {
"description": "Recording entry point — first URL the flow starts from.",
"TraceV3": {
"description": "Wire trace returned by `tool.record_stop` / `await`.",
"type": "object",
"required": [
"start_url"
"entry",
"recorded_at",
"recorder",
"states",
"steps",
"stopped_by",
"version"
],
"properties": {
"start_url": {
"entry": {
"$ref": "#/definitions/TraceEntry"
},
"purpose": {
"type": [
"string",
"null"
]
},
"recorded_at": {
"type": "string"
},
"recorder": {
"$ref": "#/definitions/RecorderInfo"
},
"started_at": {
"type": [
"string",
"null"
]
},
"states": {
"type": "array",
"items": {
"$ref": "#/definitions/TraceStateV3"
}
},
"steps": {
"type": "array",
"items": {
"$ref": "#/definitions/StepV3"
}
},
"stopped_by": {
"$ref": "#/definitions/StopReason"
},
"version": {
"type": "integer",
"const": 3
}
}
},
"additionalProperties": false
}
}
}
@@ -6,15 +6,36 @@
"session_id"
],
"properties": {
"max_page_tokens": {
"type": [
"integer",
"null"
],
"format": "uint32",
"minimum": 0.0
},
"purpose": {
"type": [
"string",
"null"
]
},
"redact_values": {
"type": [
"boolean",
"null"
]
},
"session_id": {
"type": "string"
},
"supports_tab_switch_steps": {
"description": "Client can decode the v3 `switch_tab` step variant.",
"type": [
"boolean",
"null"
]
},
"tab_id": {
"type": [
"integer",
@@ -22,6 +43,15 @@
],
"format": "int64"
},
"trace_version": {
"description": "Desired trace export format. Omitted means v2; `3` requests v3.",
"type": [
"integer",
"null"
],
"format": "uint32",
"minimum": 0.0
},
"url": {
"type": [
"string",
@@ -7,10 +7,18 @@
],
"properties": {
"trace": {
"$ref": "#/definitions/Trace"
"$ref": "#/definitions/RecordedTrace"
}
},
"definitions": {
"FillCommit": {
"type": "string",
"enum": [
"enter",
"suggestion",
"blur"
]
},
"KeyModifier": {
"description": "Keyboard modifier flags. Multiple flags may be combined; the extension folds them into CDP's bitfield (`alt=1, ctrl=2, meta=4, shift=8`).",
"type": "string",
@@ -21,7 +29,19 @@
"shift"
]
},
"PageRef": {
"NavigationCause": {
"type": "string",
"enum": [
"user_typed",
"link",
"form_submit",
"reload",
"history",
"script",
"browser"
]
},
"PageRefV2": {
"description": "Page context dictionary entry — referenced by steps via `page` id.",
"type": "object",
"required": [
@@ -43,7 +63,42 @@
}
}
},
"SelectedOption": {
"RecordedTrace": {
"if": {
"required": [
"version"
],
"properties": {
"version": {
"type": "integer"
}
}
},
"then": {
"$ref": "#/definitions/TraceV3"
},
"else": {
"$ref": "#/definitions/TraceV2"
}
},
"RecorderInfo": {
"type": "object",
"required": [
"bsk",
"vom"
],
"properties": {
"bsk": {
"type": "string"
},
"vom": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
}
}
},
"SelectedOptionV2": {
"description": "One selected option (`select` op).",
"type": "object",
"required": [
@@ -61,11 +116,53 @@
}
}
},
"Step": {
"description": "One recorded user action — discriminated union by `op`.",
"SelectedOptionV3": {
"description": "One selected option (`select` op).",
"type": "object",
"required": [
"value"
],
"properties": {
"label": {
"type": [
"string",
"null"
]
},
"value": {
"type": "string"
}
}
},
"StepEffectV2": {
"description": "Observed navigation after a step (objective fact only).",
"type": "object",
"required": [
"navigated_to"
],
"properties": {
"navigated_to": {
"description": "Reference into `pages[]` for the destination page.",
"type": "string"
}
}
},
"StepResultV3": {
"type": "object",
"required": [
"state"
],
"properties": {
"state": {
"type": "string"
}
}
},
"StepV2": {
"description": "One recorded user action — discriminated union by `op` (v2).",
"oneOf": [
{
"description": "Fields shared by every step variant (flattened in JSON).",
"description": "Fields shared by every v2 step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
@@ -77,7 +174,7 @@
"effect": {
"anyOf": [
{
"$ref": "#/definitions/StepEffect"
"$ref": "#/definitions/StepEffectV2"
},
{
"type": "null"
@@ -105,7 +202,7 @@
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"description": "Fields shared by every v2 step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
@@ -117,7 +214,7 @@
"effect": {
"anyOf": [
{
"$ref": "#/definitions/StepEffect"
"$ref": "#/definitions/StepEffectV2"
},
{
"type": "null"
@@ -140,12 +237,12 @@
"type": "string"
},
"target": {
"$ref": "#/definitions/TargetDescriptor"
"$ref": "#/definitions/TargetDescriptorV2"
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"description": "Fields shared by every v2 step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
@@ -157,7 +254,7 @@
"effect": {
"anyOf": [
{
"$ref": "#/definitions/StepEffect"
"$ref": "#/definitions/StepEffectV2"
},
{
"type": "null"
@@ -180,12 +277,12 @@
"type": "string"
},
"target": {
"$ref": "#/definitions/TargetDescriptor"
"$ref": "#/definitions/TargetDescriptorV2"
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"description": "Fields shared by every v2 step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
@@ -198,7 +295,7 @@
"effect": {
"anyOf": [
{
"$ref": "#/definitions/StepEffect"
"$ref": "#/definitions/StepEffectV2"
},
{
"type": "null"
@@ -227,7 +324,7 @@
]
},
"target": {
"$ref": "#/definitions/TargetDescriptor"
"$ref": "#/definitions/TargetDescriptorV2"
},
"value": {
"type": "string"
@@ -235,7 +332,7 @@
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"description": "Fields shared by every v2 step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
@@ -248,7 +345,7 @@
"effect": {
"anyOf": [
{
"$ref": "#/definitions/StepEffect"
"$ref": "#/definitions/StepEffectV2"
},
{
"type": "null"
@@ -273,16 +370,16 @@
"selection": {
"type": "array",
"items": {
"$ref": "#/definitions/SelectedOption"
"$ref": "#/definitions/SelectedOptionV2"
}
},
"target": {
"$ref": "#/definitions/TargetDescriptor"
"$ref": "#/definitions/TargetDescriptorV2"
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"description": "Fields shared by every v2 step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
@@ -294,7 +391,7 @@
"effect": {
"anyOf": [
{
"$ref": "#/definitions/StepEffect"
"$ref": "#/definitions/StepEffectV2"
},
{
"type": "null"
@@ -331,7 +428,7 @@
"target": {
"anyOf": [
{
"$ref": "#/definitions/TargetDescriptor"
"$ref": "#/definitions/TargetDescriptorV2"
},
{
"type": "null"
@@ -342,21 +439,324 @@
}
]
},
"StepEffect": {
"description": "Observed navigation after a step (objective fact only).",
"type": "object",
"required": [
"navigated_to"
],
"properties": {
"navigated_to": {
"description": "Reference into `pages[]` for the destination page.",
"type": "string"
"StepV3": {
"description": "One recorded user action — discriminated union by `op`.",
"oneOf": [
{
"description": "Fields shared by every step variant (flattened in JSON).",
"type": "object",
"required": [
"cause",
"id",
"op",
"result",
"state",
"to"
],
"properties": {
"cause": {
"$ref": "#/definitions/NavigationCause"
},
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"navigate"
]
},
"result": {
"$ref": "#/definitions/StepResultV3"
},
"state": {
"description": "Observation id immediately before this action.",
"type": "string"
},
"to": {
"type": "string"
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
"op",
"result",
"state"
],
"properties": {
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"switch_tab"
]
},
"result": {
"$ref": "#/definitions/StepResultV3"
},
"state": {
"description": "Observation id immediately before this action.",
"type": "string"
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
"op",
"result",
"state",
"target"
],
"properties": {
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"click"
]
},
"result": {
"$ref": "#/definitions/StepResultV3"
},
"state": {
"description": "Observation id immediately before this action.",
"type": "string"
},
"target": {
"$ref": "#/definitions/TargetDescriptorV3"
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
"op",
"result",
"state",
"target"
],
"properties": {
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"hover"
]
},
"result": {
"$ref": "#/definitions/StepResultV3"
},
"state": {
"description": "Observation id immediately before this action.",
"type": "string"
},
"target": {
"$ref": "#/definitions/TargetDescriptorV3"
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"type": "object",
"required": [
"commit",
"id",
"op",
"result",
"state",
"target",
"value"
],
"properties": {
"commit": {
"$ref": "#/definitions/FillCommit"
},
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"fill"
]
},
"redacted": {
"type": "boolean"
},
"result": {
"$ref": "#/definitions/StepResultV3"
},
"state": {
"description": "Observation id immediately before this action.",
"type": "string"
},
"target": {
"$ref": "#/definitions/TargetDescriptorV3"
},
"value": {
"type": "string"
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
"op",
"result",
"state",
"target"
],
"properties": {
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"select"
]
},
"result": {
"$ref": "#/definitions/StepResultV3"
},
"selection": {
"type": "array",
"items": {
"$ref": "#/definitions/SelectedOptionV3"
}
},
"state": {
"description": "Observation id immediately before this action.",
"type": "string"
},
"target": {
"$ref": "#/definitions/TargetDescriptorV3"
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
"key",
"op",
"result",
"state"
],
"properties": {
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"key": {
"type": "string"
},
"modifiers": {
"type": [
"array",
"null"
],
"items": {
"$ref": "#/definitions/KeyModifier"
}
},
"op": {
"type": "string",
"enum": [
"press"
]
},
"result": {
"$ref": "#/definitions/StepResultV3"
},
"state": {
"description": "Observation id immediately before this action.",
"type": "string"
},
"target": {
"anyOf": [
{
"$ref": "#/definitions/TargetDescriptorV3"
},
{
"type": "null"
}
]
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
"op",
"result",
"state"
],
"properties": {
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"scroll"
]
},
"result": {
"$ref": "#/definitions/StepResultV3"
},
"state": {
"description": "Observation id immediately before this action.",
"type": "string"
}
}
}
}
]
},
"TargetDescriptor": {
"description": "Stable semantic handle for an interacted element.\n\n`name` and `nearby_label` are **untrusted page text**.",
"StopReason": {
"type": "string",
"enum": [
"user_finish",
"cli_stop"
]
},
"TargetDescriptorV2": {
"description": "Stable semantic handle for an interacted element (v2).\n\n`name` and `nearby_label` are **untrusted page text**.",
"type": "object",
"required": [
"tag"
@@ -397,8 +797,83 @@
}
}
},
"Trace": {
"description": "Persisted user-action trace exported by `tool.record_stop` / `await`.",
"TargetDescriptorV3": {
"description": "Stable semantic handle for an interacted element within a page observation.",
"type": "object",
"properties": {
"ctx": {
"type": [
"string",
"null"
]
},
"name": {
"type": [
"string",
"null"
]
},
"ref": {
"type": [
"string",
"null"
]
},
"role": {
"type": [
"string",
"null"
]
},
"unmatched": {
"type": "boolean"
}
}
},
"TraceEntry": {
"type": "object",
"required": [
"start_url"
],
"properties": {
"start_url": {
"type": "string"
}
}
},
"TraceStateV3": {
"description": "Page observation dictionary entry — referenced by steps via `state` / `result.state`.",
"type": "object",
"required": [
"body",
"id",
"url"
],
"properties": {
"body": {
"description": "Full page observation (front matter + VOM body + annotations).",
"type": "string"
},
"id": {
"type": "string"
},
"title": {
"type": [
"string",
"null"
]
},
"truncated": {
"type": "boolean"
},
"url": {
"type": "string"
}
}
},
"TraceV2": {
"title": "TraceV2",
"description": "Persisted user-action trace exported by legacy `tool.record_stop` / `await`.\n\nUnknown extension fields are ignored so older traces remain readable. `states[]` and a numeric `version` are reserved for Trace v3 / `RecordedTrace` classification.",
"type": "object",
"required": [
"entry",
@@ -413,11 +888,10 @@
"pages": {
"type": "array",
"items": {
"$ref": "#/definitions/PageRef"
"$ref": "#/definitions/PageRefV2"
}
},
"purpose": {
"description": "Optional user-provided goal from `--purpose` (metadata only).",
"type": [
"string",
"null"
@@ -433,25 +907,76 @@
"null"
]
},
"states": false,
"steps": {
"type": "array",
"items": {
"$ref": "#/definitions/Step"
"$ref": "#/definitions/StepV2"
}
},
"version": {
"description": "Numeric version selects Trace v3. Legacy v2 envelopes omit this field.",
"not": {
"type": "integer"
}
}
}
},
"TraceEntry": {
"description": "Recording entry point — first URL the flow starts from.",
"TraceV3": {
"description": "Wire trace returned by `tool.record_stop` / `await`.",
"type": "object",
"required": [
"start_url"
"entry",
"recorded_at",
"recorder",
"states",
"steps",
"stopped_by",
"version"
],
"properties": {
"start_url": {
"entry": {
"$ref": "#/definitions/TraceEntry"
},
"purpose": {
"type": [
"string",
"null"
]
},
"recorded_at": {
"type": "string"
},
"recorder": {
"$ref": "#/definitions/RecorderInfo"
},
"started_at": {
"type": [
"string",
"null"
]
},
"states": {
"type": "array",
"items": {
"$ref": "#/definitions/TraceStateV3"
}
},
"steps": {
"type": "array",
"items": {
"$ref": "#/definitions/StepV3"
}
},
"stopped_by": {
"$ref": "#/definitions/StopReason"
},
"version": {
"type": "integer",
"const": 3
}
}
},
"additionalProperties": false
}
}
}
+601 -76
View File
@@ -1,49 +1,31 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "Trace",
"description": "Persisted user-action trace exported by `tool.record_stop` / `await`.",
"type": "object",
"required": [
"entry",
"pages",
"recorded_at",
"steps"
],
"properties": {
"entry": {
"$ref": "#/definitions/TraceEntry"
},
"pages": {
"type": "array",
"items": {
"$ref": "#/definitions/PageRef"
}
},
"purpose": {
"description": "Optional user-provided goal from `--purpose` (metadata only).",
"type": [
"string",
"null"
]
},
"recorded_at": {
"description": "RFC 3339 timestamp when recording stopped.",
"type": "string"
},
"started_at": {
"type": [
"string",
"null"
]
},
"steps": {
"type": "array",
"items": {
"$ref": "#/definitions/Step"
"title": "RecordedTrace",
"if": {
"required": [
"version"
],
"properties": {
"version": {
"type": "integer"
}
}
},
"then": {
"$ref": "#/definitions/TraceV3"
},
"else": {
"$ref": "#/definitions/TraceV2"
},
"definitions": {
"FillCommit": {
"type": "string",
"enum": [
"enter",
"suggestion",
"blur"
]
},
"KeyModifier": {
"description": "Keyboard modifier flags. Multiple flags may be combined; the extension folds them into CDP's bitfield (`alt=1, ctrl=2, meta=4, shift=8`).",
"type": "string",
@@ -54,7 +36,19 @@
"shift"
]
},
"PageRef": {
"NavigationCause": {
"type": "string",
"enum": [
"user_typed",
"link",
"form_submit",
"reload",
"history",
"script",
"browser"
]
},
"PageRefV2": {
"description": "Page context dictionary entry — referenced by steps via `page` id.",
"type": "object",
"required": [
@@ -76,7 +70,24 @@
}
}
},
"SelectedOption": {
"RecorderInfo": {
"type": "object",
"required": [
"bsk",
"vom"
],
"properties": {
"bsk": {
"type": "string"
},
"vom": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
}
}
},
"SelectedOptionV2": {
"description": "One selected option (`select` op).",
"type": "object",
"required": [
@@ -94,11 +105,53 @@
}
}
},
"Step": {
"description": "One recorded user action — discriminated union by `op`.",
"SelectedOptionV3": {
"description": "One selected option (`select` op).",
"type": "object",
"required": [
"value"
],
"properties": {
"label": {
"type": [
"string",
"null"
]
},
"value": {
"type": "string"
}
}
},
"StepEffectV2": {
"description": "Observed navigation after a step (objective fact only).",
"type": "object",
"required": [
"navigated_to"
],
"properties": {
"navigated_to": {
"description": "Reference into `pages[]` for the destination page.",
"type": "string"
}
}
},
"StepResultV3": {
"type": "object",
"required": [
"state"
],
"properties": {
"state": {
"type": "string"
}
}
},
"StepV2": {
"description": "One recorded user action — discriminated union by `op` (v2).",
"oneOf": [
{
"description": "Fields shared by every step variant (flattened in JSON).",
"description": "Fields shared by every v2 step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
@@ -110,7 +163,7 @@
"effect": {
"anyOf": [
{
"$ref": "#/definitions/StepEffect"
"$ref": "#/definitions/StepEffectV2"
},
{
"type": "null"
@@ -138,7 +191,7 @@
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"description": "Fields shared by every v2 step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
@@ -150,7 +203,7 @@
"effect": {
"anyOf": [
{
"$ref": "#/definitions/StepEffect"
"$ref": "#/definitions/StepEffectV2"
},
{
"type": "null"
@@ -173,12 +226,12 @@
"type": "string"
},
"target": {
"$ref": "#/definitions/TargetDescriptor"
"$ref": "#/definitions/TargetDescriptorV2"
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"description": "Fields shared by every v2 step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
@@ -190,7 +243,7 @@
"effect": {
"anyOf": [
{
"$ref": "#/definitions/StepEffect"
"$ref": "#/definitions/StepEffectV2"
},
{
"type": "null"
@@ -213,12 +266,12 @@
"type": "string"
},
"target": {
"$ref": "#/definitions/TargetDescriptor"
"$ref": "#/definitions/TargetDescriptorV2"
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"description": "Fields shared by every v2 step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
@@ -231,7 +284,7 @@
"effect": {
"anyOf": [
{
"$ref": "#/definitions/StepEffect"
"$ref": "#/definitions/StepEffectV2"
},
{
"type": "null"
@@ -260,7 +313,7 @@
]
},
"target": {
"$ref": "#/definitions/TargetDescriptor"
"$ref": "#/definitions/TargetDescriptorV2"
},
"value": {
"type": "string"
@@ -268,7 +321,7 @@
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"description": "Fields shared by every v2 step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
@@ -281,7 +334,7 @@
"effect": {
"anyOf": [
{
"$ref": "#/definitions/StepEffect"
"$ref": "#/definitions/StepEffectV2"
},
{
"type": "null"
@@ -306,16 +359,16 @@
"selection": {
"type": "array",
"items": {
"$ref": "#/definitions/SelectedOption"
"$ref": "#/definitions/SelectedOptionV2"
}
},
"target": {
"$ref": "#/definitions/TargetDescriptor"
"$ref": "#/definitions/TargetDescriptorV2"
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"description": "Fields shared by every v2 step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
@@ -327,7 +380,7 @@
"effect": {
"anyOf": [
{
"$ref": "#/definitions/StepEffect"
"$ref": "#/definitions/StepEffectV2"
},
{
"type": "null"
@@ -364,7 +417,7 @@
"target": {
"anyOf": [
{
"$ref": "#/definitions/TargetDescriptor"
"$ref": "#/definitions/TargetDescriptorV2"
},
{
"type": "null"
@@ -375,21 +428,324 @@
}
]
},
"StepEffect": {
"description": "Observed navigation after a step (objective fact only).",
"type": "object",
"required": [
"navigated_to"
],
"properties": {
"navigated_to": {
"description": "Reference into `pages[]` for the destination page.",
"type": "string"
"StepV3": {
"description": "One recorded user action — discriminated union by `op`.",
"oneOf": [
{
"description": "Fields shared by every step variant (flattened in JSON).",
"type": "object",
"required": [
"cause",
"id",
"op",
"result",
"state",
"to"
],
"properties": {
"cause": {
"$ref": "#/definitions/NavigationCause"
},
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"navigate"
]
},
"result": {
"$ref": "#/definitions/StepResultV3"
},
"state": {
"description": "Observation id immediately before this action.",
"type": "string"
},
"to": {
"type": "string"
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
"op",
"result",
"state"
],
"properties": {
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"switch_tab"
]
},
"result": {
"$ref": "#/definitions/StepResultV3"
},
"state": {
"description": "Observation id immediately before this action.",
"type": "string"
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
"op",
"result",
"state",
"target"
],
"properties": {
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"click"
]
},
"result": {
"$ref": "#/definitions/StepResultV3"
},
"state": {
"description": "Observation id immediately before this action.",
"type": "string"
},
"target": {
"$ref": "#/definitions/TargetDescriptorV3"
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
"op",
"result",
"state",
"target"
],
"properties": {
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"hover"
]
},
"result": {
"$ref": "#/definitions/StepResultV3"
},
"state": {
"description": "Observation id immediately before this action.",
"type": "string"
},
"target": {
"$ref": "#/definitions/TargetDescriptorV3"
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"type": "object",
"required": [
"commit",
"id",
"op",
"result",
"state",
"target",
"value"
],
"properties": {
"commit": {
"$ref": "#/definitions/FillCommit"
},
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"fill"
]
},
"redacted": {
"type": "boolean"
},
"result": {
"$ref": "#/definitions/StepResultV3"
},
"state": {
"description": "Observation id immediately before this action.",
"type": "string"
},
"target": {
"$ref": "#/definitions/TargetDescriptorV3"
},
"value": {
"type": "string"
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
"op",
"result",
"state",
"target"
],
"properties": {
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"select"
]
},
"result": {
"$ref": "#/definitions/StepResultV3"
},
"selection": {
"type": "array",
"items": {
"$ref": "#/definitions/SelectedOptionV3"
}
},
"state": {
"description": "Observation id immediately before this action.",
"type": "string"
},
"target": {
"$ref": "#/definitions/TargetDescriptorV3"
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
"key",
"op",
"result",
"state"
],
"properties": {
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"key": {
"type": "string"
},
"modifiers": {
"type": [
"array",
"null"
],
"items": {
"$ref": "#/definitions/KeyModifier"
}
},
"op": {
"type": "string",
"enum": [
"press"
]
},
"result": {
"$ref": "#/definitions/StepResultV3"
},
"state": {
"description": "Observation id immediately before this action.",
"type": "string"
},
"target": {
"anyOf": [
{
"$ref": "#/definitions/TargetDescriptorV3"
},
{
"type": "null"
}
]
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
"op",
"result",
"state"
],
"properties": {
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"scroll"
]
},
"result": {
"$ref": "#/definitions/StepResultV3"
},
"state": {
"description": "Observation id immediately before this action.",
"type": "string"
}
}
}
}
]
},
"TargetDescriptor": {
"description": "Stable semantic handle for an interacted element.\n\n`name` and `nearby_label` are **untrusted page text**.",
"StopReason": {
"type": "string",
"enum": [
"user_finish",
"cli_stop"
]
},
"TargetDescriptorV2": {
"description": "Stable semantic handle for an interacted element (v2).\n\n`name` and `nearby_label` are **untrusted page text**.",
"type": "object",
"required": [
"tag"
@@ -430,8 +786,40 @@
}
}
},
"TargetDescriptorV3": {
"description": "Stable semantic handle for an interacted element within a page observation.",
"type": "object",
"properties": {
"ctx": {
"type": [
"string",
"null"
]
},
"name": {
"type": [
"string",
"null"
]
},
"ref": {
"type": [
"string",
"null"
]
},
"role": {
"type": [
"string",
"null"
]
},
"unmatched": {
"type": "boolean"
}
}
},
"TraceEntry": {
"description": "Recording entry point — first URL the flow starts from.",
"type": "object",
"required": [
"start_url"
@@ -441,6 +829,143 @@
"type": "string"
}
}
},
"TraceStateV3": {
"description": "Page observation dictionary entry — referenced by steps via `state` / `result.state`.",
"type": "object",
"required": [
"body",
"id",
"url"
],
"properties": {
"body": {
"description": "Full page observation (front matter + VOM body + annotations).",
"type": "string"
},
"id": {
"type": "string"
},
"title": {
"type": [
"string",
"null"
]
},
"truncated": {
"type": "boolean"
},
"url": {
"type": "string"
}
}
},
"TraceV2": {
"title": "TraceV2",
"description": "Persisted user-action trace exported by legacy `tool.record_stop` / `await`.\n\nUnknown extension fields are ignored so older traces remain readable. `states[]` and a numeric `version` are reserved for Trace v3 / `RecordedTrace` classification.",
"type": "object",
"required": [
"entry",
"pages",
"recorded_at",
"steps"
],
"properties": {
"entry": {
"$ref": "#/definitions/TraceEntry"
},
"pages": {
"type": "array",
"items": {
"$ref": "#/definitions/PageRefV2"
}
},
"purpose": {
"type": [
"string",
"null"
]
},
"recorded_at": {
"description": "RFC 3339 timestamp when recording stopped.",
"type": "string"
},
"started_at": {
"type": [
"string",
"null"
]
},
"states": false,
"steps": {
"type": "array",
"items": {
"$ref": "#/definitions/StepV2"
}
},
"version": {
"description": "Numeric version selects Trace v3. Legacy v2 envelopes omit this field.",
"not": {
"type": "integer"
}
}
}
},
"TraceV3": {
"description": "Wire trace returned by `tool.record_stop` / `await`.",
"type": "object",
"required": [
"entry",
"recorded_at",
"recorder",
"states",
"steps",
"stopped_by",
"version"
],
"properties": {
"entry": {
"$ref": "#/definitions/TraceEntry"
},
"purpose": {
"type": [
"string",
"null"
]
},
"recorded_at": {
"type": "string"
},
"recorder": {
"$ref": "#/definitions/RecorderInfo"
},
"started_at": {
"type": [
"string",
"null"
]
},
"states": {
"type": "array",
"items": {
"$ref": "#/definitions/TraceStateV3"
}
},
"steps": {
"type": "array",
"items": {
"$ref": "#/definitions/StepV3"
}
},
"stopped_by": {
"$ref": "#/definitions/StopReason"
},
"version": {
"type": "integer",
"const": 3
}
},
"additionalProperties": false
}
}
}
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,368 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "StepV2",
"description": "One recorded user action — discriminated union by `op` (v2).",
"oneOf": [
{
"description": "Fields shared by every v2 step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
"op",
"page",
"to"
],
"properties": {
"effect": {
"anyOf": [
{
"$ref": "#/definitions/StepEffectV2"
},
{
"type": "null"
}
]
},
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"navigate"
]
},
"page": {
"description": "Reference into `pages[]`.",
"type": "string"
},
"to": {
"type": "string"
}
}
},
{
"description": "Fields shared by every v2 step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
"op",
"page",
"target"
],
"properties": {
"effect": {
"anyOf": [
{
"$ref": "#/definitions/StepEffectV2"
},
{
"type": "null"
}
]
},
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"click"
]
},
"page": {
"description": "Reference into `pages[]`.",
"type": "string"
},
"target": {
"$ref": "#/definitions/TargetDescriptorV2"
}
}
},
{
"description": "Fields shared by every v2 step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
"op",
"page",
"target"
],
"properties": {
"effect": {
"anyOf": [
{
"$ref": "#/definitions/StepEffectV2"
},
{
"type": "null"
}
]
},
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"hover"
]
},
"page": {
"description": "Reference into `pages[]`.",
"type": "string"
},
"target": {
"$ref": "#/definitions/TargetDescriptorV2"
}
}
},
{
"description": "Fields shared by every v2 step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
"op",
"page",
"target",
"value"
],
"properties": {
"effect": {
"anyOf": [
{
"$ref": "#/definitions/StepEffectV2"
},
{
"type": "null"
}
]
},
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"fill"
]
},
"page": {
"description": "Reference into `pages[]`.",
"type": "string"
},
"redacted": {
"type": [
"boolean",
"null"
]
},
"target": {
"$ref": "#/definitions/TargetDescriptorV2"
},
"value": {
"type": "string"
}
}
},
{
"description": "Fields shared by every v2 step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
"op",
"page",
"selection",
"target"
],
"properties": {
"effect": {
"anyOf": [
{
"$ref": "#/definitions/StepEffectV2"
},
{
"type": "null"
}
]
},
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"select"
]
},
"page": {
"description": "Reference into `pages[]`.",
"type": "string"
},
"selection": {
"type": "array",
"items": {
"$ref": "#/definitions/SelectedOptionV2"
}
},
"target": {
"$ref": "#/definitions/TargetDescriptorV2"
}
}
},
{
"description": "Fields shared by every v2 step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
"key",
"op",
"page"
],
"properties": {
"effect": {
"anyOf": [
{
"$ref": "#/definitions/StepEffectV2"
},
{
"type": "null"
}
]
},
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"key": {
"type": "string"
},
"modifiers": {
"type": [
"array",
"null"
],
"items": {
"$ref": "#/definitions/KeyModifier"
}
},
"op": {
"type": "string",
"enum": [
"press"
]
},
"page": {
"description": "Reference into `pages[]`.",
"type": "string"
},
"target": {
"anyOf": [
{
"$ref": "#/definitions/TargetDescriptorV2"
},
{
"type": "null"
}
]
}
}
}
],
"definitions": {
"KeyModifier": {
"description": "Keyboard modifier flags. Multiple flags may be combined; the extension folds them into CDP's bitfield (`alt=1, ctrl=2, meta=4, shift=8`).",
"type": "string",
"enum": [
"alt",
"ctrl",
"meta",
"shift"
]
},
"SelectedOptionV2": {
"description": "One selected option (`select` op).",
"type": "object",
"required": [
"value"
],
"properties": {
"label": {
"type": [
"string",
"null"
]
},
"value": {
"type": "string"
}
}
},
"StepEffectV2": {
"description": "Observed navigation after a step (objective fact only).",
"type": "object",
"required": [
"navigated_to"
],
"properties": {
"navigated_to": {
"description": "Reference into `pages[]` for the destination page.",
"type": "string"
}
}
},
"TargetDescriptorV2": {
"description": "Stable semantic handle for an interacted element (v2).\n\n`name` and `nearby_label` are **untrusted page text**.",
"type": "object",
"required": [
"tag"
],
"properties": {
"name": {
"type": [
"string",
"null"
]
},
"name_attr": {
"type": [
"string",
"null"
]
},
"nearby_label": {
"type": [
"string",
"null"
]
},
"placeholder": {
"type": [
"string",
"null"
]
},
"role": {
"type": [
"string",
"null"
]
},
"tag": {
"type": "string"
}
}
}
}
}
@@ -0,0 +1,405 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "StepV3",
"description": "One recorded user action — discriminated union by `op`.",
"oneOf": [
{
"description": "Fields shared by every step variant (flattened in JSON).",
"type": "object",
"required": [
"cause",
"id",
"op",
"result",
"state",
"to"
],
"properties": {
"cause": {
"$ref": "#/definitions/NavigationCause"
},
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"navigate"
]
},
"result": {
"$ref": "#/definitions/StepResultV3"
},
"state": {
"description": "Observation id immediately before this action.",
"type": "string"
},
"to": {
"type": "string"
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
"op",
"result",
"state"
],
"properties": {
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"switch_tab"
]
},
"result": {
"$ref": "#/definitions/StepResultV3"
},
"state": {
"description": "Observation id immediately before this action.",
"type": "string"
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
"op",
"result",
"state",
"target"
],
"properties": {
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"click"
]
},
"result": {
"$ref": "#/definitions/StepResultV3"
},
"state": {
"description": "Observation id immediately before this action.",
"type": "string"
},
"target": {
"$ref": "#/definitions/TargetDescriptorV3"
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
"op",
"result",
"state",
"target"
],
"properties": {
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"hover"
]
},
"result": {
"$ref": "#/definitions/StepResultV3"
},
"state": {
"description": "Observation id immediately before this action.",
"type": "string"
},
"target": {
"$ref": "#/definitions/TargetDescriptorV3"
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"type": "object",
"required": [
"commit",
"id",
"op",
"result",
"state",
"target",
"value"
],
"properties": {
"commit": {
"$ref": "#/definitions/FillCommit"
},
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"fill"
]
},
"redacted": {
"type": "boolean"
},
"result": {
"$ref": "#/definitions/StepResultV3"
},
"state": {
"description": "Observation id immediately before this action.",
"type": "string"
},
"target": {
"$ref": "#/definitions/TargetDescriptorV3"
},
"value": {
"type": "string"
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
"op",
"result",
"state",
"target"
],
"properties": {
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"select"
]
},
"result": {
"$ref": "#/definitions/StepResultV3"
},
"selection": {
"type": "array",
"items": {
"$ref": "#/definitions/SelectedOptionV3"
}
},
"state": {
"description": "Observation id immediately before this action.",
"type": "string"
},
"target": {
"$ref": "#/definitions/TargetDescriptorV3"
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
"key",
"op",
"result",
"state"
],
"properties": {
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"key": {
"type": "string"
},
"modifiers": {
"type": [
"array",
"null"
],
"items": {
"$ref": "#/definitions/KeyModifier"
}
},
"op": {
"type": "string",
"enum": [
"press"
]
},
"result": {
"$ref": "#/definitions/StepResultV3"
},
"state": {
"description": "Observation id immediately before this action.",
"type": "string"
},
"target": {
"anyOf": [
{
"$ref": "#/definitions/TargetDescriptorV3"
},
{
"type": "null"
}
]
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
"op",
"result",
"state"
],
"properties": {
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"scroll"
]
},
"result": {
"$ref": "#/definitions/StepResultV3"
},
"state": {
"description": "Observation id immediately before this action.",
"type": "string"
}
}
}
],
"definitions": {
"FillCommit": {
"type": "string",
"enum": [
"enter",
"suggestion",
"blur"
]
},
"KeyModifier": {
"description": "Keyboard modifier flags. Multiple flags may be combined; the extension folds them into CDP's bitfield (`alt=1, ctrl=2, meta=4, shift=8`).",
"type": "string",
"enum": [
"alt",
"ctrl",
"meta",
"shift"
]
},
"NavigationCause": {
"type": "string",
"enum": [
"user_typed",
"link",
"form_submit",
"reload",
"history",
"script",
"browser"
]
},
"SelectedOptionV3": {
"description": "One selected option (`select` op).",
"type": "object",
"required": [
"value"
],
"properties": {
"label": {
"type": [
"string",
"null"
]
},
"value": {
"type": "string"
}
}
},
"StepResultV3": {
"type": "object",
"required": [
"state"
],
"properties": {
"state": {
"type": "string"
}
}
},
"TargetDescriptorV3": {
"description": "Stable semantic handle for an interacted element within a page observation.",
"type": "object",
"properties": {
"ctx": {
"type": [
"string",
"null"
]
},
"name": {
"type": [
"string",
"null"
]
},
"ref": {
"type": [
"string",
"null"
]
},
"role": {
"type": [
"string",
"null"
]
},
"unmatched": {
"type": "boolean"
}
}
}
}
}
+451
View File
@@ -0,0 +1,451 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "TraceV2",
"description": "Persisted user-action trace exported by legacy `tool.record_stop` / `await`.\n\nUnknown extension fields are ignored so older traces remain readable. `states[]` and a numeric `version` are reserved for Trace v3 / `RecordedTrace` classification.",
"type": "object",
"required": [
"entry",
"pages",
"recorded_at",
"steps"
],
"properties": {
"entry": {
"$ref": "#/definitions/TraceEntry"
},
"pages": {
"type": "array",
"items": {
"$ref": "#/definitions/PageRefV2"
}
},
"purpose": {
"type": [
"string",
"null"
]
},
"recorded_at": {
"description": "RFC 3339 timestamp when recording stopped.",
"type": "string"
},
"started_at": {
"type": [
"string",
"null"
]
},
"states": false,
"steps": {
"type": "array",
"items": {
"$ref": "#/definitions/StepV2"
}
},
"version": {
"description": "Numeric version selects Trace v3. Legacy v2 envelopes omit this field.",
"not": {
"type": "integer"
}
}
},
"definitions": {
"KeyModifier": {
"description": "Keyboard modifier flags. Multiple flags may be combined; the extension folds them into CDP's bitfield (`alt=1, ctrl=2, meta=4, shift=8`).",
"type": "string",
"enum": [
"alt",
"ctrl",
"meta",
"shift"
]
},
"PageRefV2": {
"description": "Page context dictionary entry — referenced by steps via `page` id.",
"type": "object",
"required": [
"id",
"url"
],
"properties": {
"id": {
"type": "string"
},
"title": {
"type": [
"string",
"null"
]
},
"url": {
"type": "string"
}
}
},
"SelectedOptionV2": {
"description": "One selected option (`select` op).",
"type": "object",
"required": [
"value"
],
"properties": {
"label": {
"type": [
"string",
"null"
]
},
"value": {
"type": "string"
}
}
},
"StepEffectV2": {
"description": "Observed navigation after a step (objective fact only).",
"type": "object",
"required": [
"navigated_to"
],
"properties": {
"navigated_to": {
"description": "Reference into `pages[]` for the destination page.",
"type": "string"
}
}
},
"StepV2": {
"description": "One recorded user action — discriminated union by `op` (v2).",
"oneOf": [
{
"description": "Fields shared by every v2 step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
"op",
"page",
"to"
],
"properties": {
"effect": {
"anyOf": [
{
"$ref": "#/definitions/StepEffectV2"
},
{
"type": "null"
}
]
},
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"navigate"
]
},
"page": {
"description": "Reference into `pages[]`.",
"type": "string"
},
"to": {
"type": "string"
}
}
},
{
"description": "Fields shared by every v2 step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
"op",
"page",
"target"
],
"properties": {
"effect": {
"anyOf": [
{
"$ref": "#/definitions/StepEffectV2"
},
{
"type": "null"
}
]
},
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"click"
]
},
"page": {
"description": "Reference into `pages[]`.",
"type": "string"
},
"target": {
"$ref": "#/definitions/TargetDescriptorV2"
}
}
},
{
"description": "Fields shared by every v2 step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
"op",
"page",
"target"
],
"properties": {
"effect": {
"anyOf": [
{
"$ref": "#/definitions/StepEffectV2"
},
{
"type": "null"
}
]
},
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"hover"
]
},
"page": {
"description": "Reference into `pages[]`.",
"type": "string"
},
"target": {
"$ref": "#/definitions/TargetDescriptorV2"
}
}
},
{
"description": "Fields shared by every v2 step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
"op",
"page",
"target",
"value"
],
"properties": {
"effect": {
"anyOf": [
{
"$ref": "#/definitions/StepEffectV2"
},
{
"type": "null"
}
]
},
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"fill"
]
},
"page": {
"description": "Reference into `pages[]`.",
"type": "string"
},
"redacted": {
"type": [
"boolean",
"null"
]
},
"target": {
"$ref": "#/definitions/TargetDescriptorV2"
},
"value": {
"type": "string"
}
}
},
{
"description": "Fields shared by every v2 step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
"op",
"page",
"selection",
"target"
],
"properties": {
"effect": {
"anyOf": [
{
"$ref": "#/definitions/StepEffectV2"
},
{
"type": "null"
}
]
},
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"select"
]
},
"page": {
"description": "Reference into `pages[]`.",
"type": "string"
},
"selection": {
"type": "array",
"items": {
"$ref": "#/definitions/SelectedOptionV2"
}
},
"target": {
"$ref": "#/definitions/TargetDescriptorV2"
}
}
},
{
"description": "Fields shared by every v2 step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
"key",
"op",
"page"
],
"properties": {
"effect": {
"anyOf": [
{
"$ref": "#/definitions/StepEffectV2"
},
{
"type": "null"
}
]
},
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"key": {
"type": "string"
},
"modifiers": {
"type": [
"array",
"null"
],
"items": {
"$ref": "#/definitions/KeyModifier"
}
},
"op": {
"type": "string",
"enum": [
"press"
]
},
"page": {
"description": "Reference into `pages[]`.",
"type": "string"
},
"target": {
"anyOf": [
{
"$ref": "#/definitions/TargetDescriptorV2"
},
{
"type": "null"
}
]
}
}
}
]
},
"TargetDescriptorV2": {
"description": "Stable semantic handle for an interacted element (v2).\n\n`name` and `nearby_label` are **untrusted page text**.",
"type": "object",
"required": [
"tag"
],
"properties": {
"name": {
"type": [
"string",
"null"
]
},
"name_attr": {
"type": [
"string",
"null"
]
},
"nearby_label": {
"type": [
"string",
"null"
]
},
"placeholder": {
"type": [
"string",
"null"
]
},
"role": {
"type": [
"string",
"null"
]
},
"tag": {
"type": "string"
}
}
},
"TraceEntry": {
"type": "object",
"required": [
"start_url"
],
"properties": {
"start_url": {
"type": "string"
}
}
}
}
}
+526
View File
@@ -0,0 +1,526 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "TraceV3",
"description": "Wire trace returned by `tool.record_stop` / `await`.",
"type": "object",
"required": [
"entry",
"recorded_at",
"recorder",
"states",
"steps",
"stopped_by",
"version"
],
"properties": {
"entry": {
"$ref": "#/definitions/TraceEntry"
},
"purpose": {
"type": [
"string",
"null"
]
},
"recorded_at": {
"type": "string"
},
"recorder": {
"$ref": "#/definitions/RecorderInfo"
},
"started_at": {
"type": [
"string",
"null"
]
},
"states": {
"type": "array",
"items": {
"$ref": "#/definitions/TraceStateV3"
}
},
"steps": {
"type": "array",
"items": {
"$ref": "#/definitions/StepV3"
}
},
"stopped_by": {
"$ref": "#/definitions/StopReason"
},
"version": {
"type": "integer",
"const": 3
}
},
"additionalProperties": false,
"definitions": {
"FillCommit": {
"type": "string",
"enum": [
"enter",
"suggestion",
"blur"
]
},
"KeyModifier": {
"description": "Keyboard modifier flags. Multiple flags may be combined; the extension folds them into CDP's bitfield (`alt=1, ctrl=2, meta=4, shift=8`).",
"type": "string",
"enum": [
"alt",
"ctrl",
"meta",
"shift"
]
},
"NavigationCause": {
"type": "string",
"enum": [
"user_typed",
"link",
"form_submit",
"reload",
"history",
"script",
"browser"
]
},
"RecorderInfo": {
"type": "object",
"required": [
"bsk",
"vom"
],
"properties": {
"bsk": {
"type": "string"
},
"vom": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
}
}
},
"SelectedOptionV3": {
"description": "One selected option (`select` op).",
"type": "object",
"required": [
"value"
],
"properties": {
"label": {
"type": [
"string",
"null"
]
},
"value": {
"type": "string"
}
}
},
"StepResultV3": {
"type": "object",
"required": [
"state"
],
"properties": {
"state": {
"type": "string"
}
}
},
"StepV3": {
"description": "One recorded user action — discriminated union by `op`.",
"oneOf": [
{
"description": "Fields shared by every step variant (flattened in JSON).",
"type": "object",
"required": [
"cause",
"id",
"op",
"result",
"state",
"to"
],
"properties": {
"cause": {
"$ref": "#/definitions/NavigationCause"
},
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"navigate"
]
},
"result": {
"$ref": "#/definitions/StepResultV3"
},
"state": {
"description": "Observation id immediately before this action.",
"type": "string"
},
"to": {
"type": "string"
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
"op",
"result",
"state"
],
"properties": {
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"switch_tab"
]
},
"result": {
"$ref": "#/definitions/StepResultV3"
},
"state": {
"description": "Observation id immediately before this action.",
"type": "string"
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
"op",
"result",
"state",
"target"
],
"properties": {
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"click"
]
},
"result": {
"$ref": "#/definitions/StepResultV3"
},
"state": {
"description": "Observation id immediately before this action.",
"type": "string"
},
"target": {
"$ref": "#/definitions/TargetDescriptorV3"
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
"op",
"result",
"state",
"target"
],
"properties": {
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"hover"
]
},
"result": {
"$ref": "#/definitions/StepResultV3"
},
"state": {
"description": "Observation id immediately before this action.",
"type": "string"
},
"target": {
"$ref": "#/definitions/TargetDescriptorV3"
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"type": "object",
"required": [
"commit",
"id",
"op",
"result",
"state",
"target",
"value"
],
"properties": {
"commit": {
"$ref": "#/definitions/FillCommit"
},
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"fill"
]
},
"redacted": {
"type": "boolean"
},
"result": {
"$ref": "#/definitions/StepResultV3"
},
"state": {
"description": "Observation id immediately before this action.",
"type": "string"
},
"target": {
"$ref": "#/definitions/TargetDescriptorV3"
},
"value": {
"type": "string"
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
"op",
"result",
"state",
"target"
],
"properties": {
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"select"
]
},
"result": {
"$ref": "#/definitions/StepResultV3"
},
"selection": {
"type": "array",
"items": {
"$ref": "#/definitions/SelectedOptionV3"
}
},
"state": {
"description": "Observation id immediately before this action.",
"type": "string"
},
"target": {
"$ref": "#/definitions/TargetDescriptorV3"
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
"key",
"op",
"result",
"state"
],
"properties": {
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"key": {
"type": "string"
},
"modifiers": {
"type": [
"array",
"null"
],
"items": {
"$ref": "#/definitions/KeyModifier"
}
},
"op": {
"type": "string",
"enum": [
"press"
]
},
"result": {
"$ref": "#/definitions/StepResultV3"
},
"state": {
"description": "Observation id immediately before this action.",
"type": "string"
},
"target": {
"anyOf": [
{
"$ref": "#/definitions/TargetDescriptorV3"
},
{
"type": "null"
}
]
}
}
},
{
"description": "Fields shared by every step variant (flattened in JSON).",
"type": "object",
"required": [
"id",
"op",
"result",
"state"
],
"properties": {
"id": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"op": {
"type": "string",
"enum": [
"scroll"
]
},
"result": {
"$ref": "#/definitions/StepResultV3"
},
"state": {
"description": "Observation id immediately before this action.",
"type": "string"
}
}
}
]
},
"StopReason": {
"type": "string",
"enum": [
"user_finish",
"cli_stop"
]
},
"TargetDescriptorV3": {
"description": "Stable semantic handle for an interacted element within a page observation.",
"type": "object",
"properties": {
"ctx": {
"type": [
"string",
"null"
]
},
"name": {
"type": [
"string",
"null"
]
},
"ref": {
"type": [
"string",
"null"
]
},
"role": {
"type": [
"string",
"null"
]
},
"unmatched": {
"type": "boolean"
}
}
},
"TraceEntry": {
"type": "object",
"required": [
"start_url"
],
"properties": {
"start_url": {
"type": "string"
}
}
},
"TraceStateV3": {
"description": "Page observation dictionary entry — referenced by steps via `state` / `result.state`.",
"type": "object",
"required": [
"body",
"id",
"url"
],
"properties": {
"body": {
"description": "Full page observation (front matter + VOM body + annotations).",
"type": "string"
},
"id": {
"type": "string"
},
"title": {
"type": [
"string",
"null"
]
},
"truncated": {
"type": "boolean"
},
"url": {
"type": "string"
}
}
}
}
}
+6 -2
View File
@@ -113,8 +113,12 @@ fn main() {
dump!(RequestHelpParams, "tool_request_help_params");
dump!(RequestHelpResult, "tool_request_help_result");
dump!(Trace, "trace");
dump!(Step, "trace_step");
dump!(TraceV2, "trace_v2");
dump!(TraceV3, "trace_v3");
dump!(RecordedTrace, "trace");
dump!(StepV2, "trace_step_v2");
dump!(StepV3, "trace_step_v3");
dump!(RecordedStep, "trace_step");
dump!(RecordStartParams, "tool_record_start_params");
dump!(RecordStartResult, "tool_record_start_result");
dump!(RecordStopParams, "tool_record_stop_params");
+3
View File
@@ -9,6 +9,9 @@ pub mod navigation;
pub mod network;
pub mod observation;
pub mod record;
mod record_common;
mod record_v2;
mod record_v3;
pub mod script;
pub mod session;
pub mod tabs;
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,7 @@
use schemars::JsonSchema;
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
pub struct TraceEntry {
pub start_url: String,
}
+330
View File
@@ -0,0 +1,330 @@
//! Trace v2 — record-only user-action log with `pages[]` and step `page` refs.
//!
//! Legacy wire format with no top-level `version` field.
use schemars::JsonSchema;
use serde::{Deserialize, Serialize};
use super::interaction::KeyModifier;
use super::record_common::TraceEntry;
/// Stable semantic handle for an interacted element (v2).
///
/// `name` and `nearby_label` are **untrusted page text**.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
pub struct TargetDescriptorV2 {
#[serde(default, skip_serializing_if = "Option::is_none")]
pub role: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub name: Option<String>,
pub tag: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub name_attr: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub placeholder: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub nearby_label: Option<String>,
}
/// Page context dictionary entry — referenced by steps via `page` id.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
pub struct PageRefV2 {
pub id: String,
pub url: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub title: Option<String>,
}
/// One selected option (`select` op).
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
pub struct SelectedOptionV2 {
pub value: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub label: Option<String>,
}
/// Observed navigation after a step (objective fact only).
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
pub struct StepEffectV2 {
/// Reference into `pages[]` for the destination page.
pub navigated_to: String,
}
/// Fields shared by every v2 step variant (flattened in JSON).
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
pub struct StepCommonV2 {
pub id: u32,
/// Reference into `pages[]`.
pub page: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub effect: Option<StepEffectV2>,
}
/// One recorded user action — discriminated union by `op` (v2).
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
#[serde(tag = "op", rename_all = "snake_case")]
pub enum StepV2 {
Navigate {
#[serde(flatten)]
common: StepCommonV2,
to: String,
},
Click {
#[serde(flatten)]
common: StepCommonV2,
target: TargetDescriptorV2,
},
Hover {
#[serde(flatten)]
common: StepCommonV2,
target: TargetDescriptorV2,
},
Fill {
#[serde(flatten)]
common: StepCommonV2,
target: TargetDescriptorV2,
value: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
redacted: Option<bool>,
},
Select {
#[serde(flatten)]
common: StepCommonV2,
target: TargetDescriptorV2,
selection: Vec<SelectedOptionV2>,
},
Press {
#[serde(flatten)]
common: StepCommonV2,
key: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
modifiers: Option<Vec<KeyModifier>>,
#[serde(default, skip_serializing_if = "Option::is_none")]
target: Option<TargetDescriptorV2>,
},
}
/// Persisted user-action trace exported by legacy `tool.record_stop` / `await`.
///
/// Unknown extension fields are ignored so older traces remain readable.
/// `states[]` and a numeric `version` are reserved for Trace v3 / `RecordedTrace`
/// classification — serde still ignores them, but the JSON Schema rejects them.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct TraceV2 {
/// RFC 3339 timestamp when recording stopped.
pub recorded_at: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub started_at: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub purpose: Option<String>,
pub entry: TraceEntry,
pub pages: Vec<PageRefV2>,
pub steps: Vec<StepV2>,
}
fn constrain_trace_v2_schema(mut schema: schemars::schema::Schema) -> schemars::schema::Schema {
let schemars::schema::Schema::Object(obj) = &mut schema else {
return schema;
};
obj.metadata().title = Some("TraceV2".into());
obj.metadata().description = Some(
"Persisted user-action trace exported by legacy `tool.record_stop` / `await`.\n\n\
Unknown extension fields are ignored so older traces remain readable. \
`states[]` and a numeric `version` are reserved for Trace v3 / `RecordedTrace` classification."
.into(),
);
let object = obj.object.get_or_insert_with(Default::default);
object
.properties
.insert("states".into(), schemars::schema::Schema::Bool(false));
object.properties.insert(
"version".into(),
schemars::schema::SchemaObject {
metadata: Some(Box::new(schemars::schema::Metadata {
description: Some(
"Numeric version selects Trace v3. Legacy v2 envelopes omit this field.".into(),
),
..Default::default()
})),
subschemas: Some(Box::new(schemars::schema::SubschemaValidation {
not: Some(Box::new(
schemars::schema::SchemaObject {
instance_type: Some(schemars::schema::InstanceType::Integer.into()),
..Default::default()
}
.into(),
)),
..Default::default()
})),
..Default::default()
}
.into(),
);
schema
}
impl JsonSchema for TraceV2 {
fn schema_name() -> String {
"TraceV2".into()
}
fn json_schema(generator: &mut schemars::SchemaGenerator) -> schemars::schema::Schema {
#[derive(JsonSchema)]
#[allow(dead_code)]
struct TraceV2Shape {
/// RFC 3339 timestamp when recording stopped.
recorded_at: String,
started_at: Option<String>,
purpose: Option<String>,
entry: TraceEntry,
pages: Vec<PageRefV2>,
steps: Vec<StepV2>,
}
constrain_trace_v2_schema(TraceV2Shape::json_schema(generator))
}
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
fn sample_common(id: u32) -> StepCommonV2 {
StepCommonV2 {
id,
page: "p1".into(),
effect: None,
}
}
fn sample_target() -> TargetDescriptorV2 {
TargetDescriptorV2 {
role: Some("button".into()),
name: Some("发布".into()),
tag: "button".into(),
name_attr: None,
placeholder: None,
nearby_label: None,
}
}
#[test]
fn trace_v2_has_no_version_field() {
let trace = TraceV2 {
recorded_at: "2026-07-17T09:01:10Z".into(),
started_at: None,
purpose: None,
entry: TraceEntry {
start_url: "https://example.com/".into(),
},
pages: vec![PageRefV2 {
id: "p1".into(),
url: "https://example.com/".into(),
title: None,
}],
steps: vec![StepV2::Click {
common: sample_common(1),
target: sample_target(),
}],
};
let v = serde_json::to_value(&trace).unwrap();
assert!(v.get("version").is_none());
assert!(v.get("pages").is_some());
assert!(v.get("states").is_none());
}
#[test]
fn extension_v2_trace_deserializes() {
let v = json!({
"recorded_at": "2026-07-21T08:00:00Z",
"started_at": "2026-07-21T07:59:00Z",
"purpose": "demo",
"entry": { "start_url": "https://example.com/editor" },
"pages": [
{ "id": "p1", "url": "https://example.com/editor" },
{ "id": "p2", "url": "https://example.com/p/99" }
],
"steps": [
{
"op": "fill",
"id": 1,
"page": "p1",
"target": { "tag": "input", "role": "textbox", "name": "标题" },
"value": "hello"
},
{
"op": "click",
"id": 2,
"page": "p1",
"target": { "tag": "button", "role": "button", "name": "发布" },
"effect": { "navigated_to": "p2" }
}
]
});
let trace: TraceV2 = serde_json::from_value(v).unwrap();
assert_eq!(trace.pages.len(), 2);
assert_eq!(trace.steps.len(), 2);
}
#[test]
fn v2_hover_steps_round_trip() {
let value = json!({
"recorded_at": "2026-07-21T08:00:00Z",
"entry": { "start_url": "https://example.com/" },
"pages": [{ "id": "p1", "url": "https://example.com/" }],
"steps": [{
"op": "hover",
"id": 1,
"page": "p1",
"target": { "tag": "span", "role": "button", "name": "Account" }
}]
});
let trace: TraceV2 = serde_json::from_value(value).unwrap();
assert!(matches!(trace.steps.as_slice(), [StepV2::Hover { .. }]));
assert_eq!(
serde_json::to_value(&trace).unwrap()["steps"][0]["op"],
"hover"
);
}
#[test]
fn trace_v2_ignores_unknown_extension_fields() {
let value = json!({
"recorded_at": "2026-07-21T08:00:00Z",
"entry": { "start_url": "https://example.com/" },
"pages": [{ "id": "p1", "url": "https://example.com/" }],
"steps": [],
"meta": { "tool": "legacy-exporter" }
});
let trace: TraceV2 = serde_json::from_value(value).unwrap();
assert_eq!(trace.pages.len(), 1);
assert!(trace.steps.is_empty());
}
#[test]
fn trace_v2_schema_allows_additional_properties() {
let schema = serde_json::to_value(schemars::schema_for!(TraceV2)).unwrap();
assert_ne!(
schema.get("additionalProperties"),
Some(&serde_json::Value::Bool(false)),
"Trace v2 must keep accepting traces with unknown extension fields"
);
}
#[test]
fn trace_v2_schema_forbids_classification_keys() {
let schema = serde_json::to_value(schemars::schema_for!(TraceV2)).unwrap();
assert_eq!(
schema["properties"]["states"],
json!(false),
"Trace v2 schema must reject states[] (reserved for v3)"
);
assert_eq!(
schema["properties"]["version"]["not"]["type"], "integer",
"Trace v2 schema must reject a numeric version (that selects the v3 path)"
);
}
}
+445
View File
@@ -0,0 +1,445 @@
//! Trace v3 — state-action-state log with `states[]` and step `state` refs.
//!
//! Wire format with top-level `version: 3`.
use schemars::JsonSchema;
use serde::{Deserialize, Deserializer, Serialize};
use super::interaction::KeyModifier;
use super::record_common::TraceEntry;
pub const TRACE_VERSION_V3: u32 = 3;
pub const VOM_FORMAT_VERSION: u32 = 1;
fn deserialize_trace_v3_version<'de, D>(deserializer: D) -> Result<u32, D::Error>
where
D: Deserializer<'de>,
{
let version = u32::deserialize(deserializer)?;
if version != TRACE_VERSION_V3 {
return Err(serde::de::Error::custom(format!(
"unsupported trace version {version} (expected {TRACE_VERSION_V3})"
)));
}
Ok(version)
}
fn trace_v3_version_schema(_: &mut schemars::r#gen::SchemaGenerator) -> schemars::schema::Schema {
schemars::schema::SchemaObject {
instance_type: Some(schemars::schema::InstanceType::Integer.into()),
const_value: Some(serde_json::json!(TRACE_VERSION_V3)),
..Default::default()
}
.into()
}
/// Stable semantic handle for an interacted element within a page observation.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
pub struct TargetDescriptorV3 {
#[serde(default, skip_serializing_if = "Option::is_none", rename = "ref")]
pub element_ref: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub role: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub name: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub ctx: Option<String>,
#[serde(default, skip_serializing_if = "std::ops::Not::not")]
pub unmatched: bool,
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
pub struct RecorderInfo {
pub bsk: String,
pub vom: u32,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[serde(rename_all = "snake_case")]
pub enum StopReason {
UserFinish,
CliStop,
}
/// Page observation dictionary entry — referenced by steps via `state` / `result.state`.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
pub struct TraceStateV3 {
pub id: String,
pub url: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub title: Option<String>,
/// Full page observation (front matter + VOM body + annotations).
pub body: String,
#[serde(default, skip_serializing_if = "std::ops::Not::not")]
pub truncated: bool,
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
pub struct StepResultV3 {
pub state: String,
}
/// Fields shared by every step variant (flattened in JSON).
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
pub struct StepCommonV3 {
pub id: u32,
/// Observation id immediately before this action.
pub state: String,
pub result: StepResultV3,
}
/// One selected option (`select` op).
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
pub struct SelectedOptionV3 {
pub value: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub label: Option<String>,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[serde(rename_all = "snake_case")]
pub enum NavigationCause {
UserTyped,
Link,
FormSubmit,
Reload,
History,
Script,
Browser,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, JsonSchema)]
#[serde(rename_all = "snake_case")]
pub enum FillCommit {
Enter,
Suggestion,
Blur,
}
/// One recorded user action — discriminated union by `op`.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
#[serde(tag = "op", rename_all = "snake_case")]
pub enum StepV3 {
Navigate {
#[serde(flatten)]
common: StepCommonV3,
to: String,
cause: NavigationCause,
},
SwitchTab {
#[serde(flatten)]
common: StepCommonV3,
},
Click {
#[serde(flatten)]
common: StepCommonV3,
target: TargetDescriptorV3,
},
Hover {
#[serde(flatten)]
common: StepCommonV3,
target: TargetDescriptorV3,
},
Fill {
#[serde(flatten)]
common: StepCommonV3,
target: TargetDescriptorV3,
value: String,
commit: FillCommit,
#[serde(default, skip_serializing_if = "std::ops::Not::not")]
redacted: bool,
},
Select {
#[serde(flatten)]
common: StepCommonV3,
target: TargetDescriptorV3,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
selection: Vec<SelectedOptionV3>,
},
Press {
#[serde(flatten)]
common: StepCommonV3,
key: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
modifiers: Option<Vec<KeyModifier>>,
#[serde(default, skip_serializing_if = "Option::is_none")]
target: Option<TargetDescriptorV3>,
},
Scroll {
#[serde(flatten)]
common: StepCommonV3,
},
}
/// Wire trace returned by `tool.record_stop` / `await`.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
#[serde(deny_unknown_fields)]
pub struct TraceV3 {
#[serde(deserialize_with = "deserialize_trace_v3_version")]
#[schemars(schema_with = "trace_v3_version_schema")]
pub version: u32,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub purpose: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub started_at: Option<String>,
pub recorded_at: String,
pub stopped_by: StopReason,
pub entry: TraceEntry,
pub recorder: RecorderInfo,
pub states: Vec<TraceStateV3>,
pub steps: Vec<StepV3>,
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
fn sample_common(id: u32, state: &str, result_state: &str) -> StepCommonV3 {
StepCommonV3 {
id,
state: state.into(),
result: StepResultV3 {
state: result_state.into(),
},
}
}
fn sample_target() -> TargetDescriptorV3 {
TargetDescriptorV3 {
element_ref: Some("e21".into()),
role: Some("button".into()),
name: Some("发布".into()),
ctx: Some("金桔柠檬 6 号".into()),
unmatched: false,
}
}
fn sample_trace() -> TraceV3 {
TraceV3 {
version: TRACE_VERSION_V3,
purpose: Some("把草稿商品发布上架".into()),
started_at: Some("2026-08-10T02:10:41.080Z".into()),
recorded_at: "2026-08-10T02:12:55.360Z".into(),
stopped_by: StopReason::UserFinish,
entry: TraceEntry {
start_url: "https://example.com/".into(),
},
recorder: RecorderInfo {
bsk: "0.1.10".into(),
vom: VOM_FORMAT_VERSION,
},
states: vec![
TraceStateV3 {
id: "s1".into(),
url: "https://example.com/".into(),
title: Some("Example Domain".into()),
body: "@vom 1\nRootWebArea \"Example Domain\"".into(),
truncated: false,
},
TraceStateV3 {
id: "s2".into(),
url: "https://shop.example.com/products?status=draft".into(),
title: Some("商品管理".into()),
body: "@vom 1\nRootWebArea \"商品管理\"".into(),
truncated: false,
},
],
steps: vec![
StepV3::Navigate {
common: sample_common(1, "s1", "s2"),
to: "https://shop.example.com/products?status=draft".into(),
cause: NavigationCause::UserTyped,
},
StepV3::Fill {
common: sample_common(2, "s2", "s2"),
target: TargetDescriptorV3 {
element_ref: Some("e12".into()),
role: Some("textbox".into()),
name: Some("搜索商品".into()),
ctx: None,
unmatched: false,
},
value: "金桔柠檬".into(),
commit: FillCommit::Enter,
redacted: false,
},
StepV3::Click {
common: sample_common(3, "s2", "s2"),
target: sample_target(),
},
],
}
}
#[test]
fn step_click_round_trips() {
let step = StepV3::Click {
common: sample_common(1, "s1", "s2"),
target: sample_target(),
};
let v = serde_json::to_value(&step).unwrap();
assert_eq!(v.get("op").and_then(|v| v.as_str()), Some("click"));
assert_eq!(v.get("state").and_then(|v| v.as_str()), Some("s1"));
assert_eq!(v["result"]["state"], "s2");
assert_eq!(v["target"]["ref"], "e21");
let round: StepV3 = serde_json::from_value(v).unwrap();
assert_eq!(round, step);
}
#[test]
fn step_fill_with_commit_round_trips() {
let step = StepV3::Fill {
common: sample_common(2, "s2", "s3"),
target: TargetDescriptorV3 {
element_ref: Some("e12".into()),
role: Some("textbox".into()),
name: Some("搜索商品".into()),
ctx: None,
unmatched: false,
},
value: "browser skill".into(),
commit: FillCommit::Enter,
redacted: false,
};
let v = serde_json::to_value(&step).unwrap();
assert_eq!(v["op"], "fill");
assert_eq!(v["commit"], "enter");
assert!(v.get("redacted").is_none());
let round: StepV3 = serde_json::from_value(v).unwrap();
assert_eq!(round, step);
}
#[test]
fn step_fill_password_is_redacted() {
let step = StepV3::Fill {
common: sample_common(1, "s1", "s1"),
target: TargetDescriptorV3 {
element_ref: Some("e3".into()),
role: Some("textbox".into()),
name: Some("密码".into()),
ctx: None,
unmatched: false,
},
value: "***".into(),
commit: FillCommit::Blur,
redacted: true,
};
let v = serde_json::to_value(&step).unwrap();
assert_eq!(v["value"], "***");
assert_eq!(v["redacted"], true);
}
#[test]
fn step_navigate_with_cause_round_trips() {
let step = StepV3::Navigate {
common: sample_common(1, "s1", "s2"),
to: "https://example.com".into(),
cause: NavigationCause::UserTyped,
};
let v = serde_json::to_value(&step).unwrap();
assert_eq!(v["op"], "navigate");
assert_eq!(v["cause"], "user_typed");
let round: StepV3 = serde_json::from_value(v).unwrap();
assert_eq!(round, step);
}
#[test]
fn step_switch_tab_round_trips() {
let step = StepV3::SwitchTab {
common: sample_common(2, "s2", "s5"),
};
let v = serde_json::to_value(&step).unwrap();
assert_eq!(v["op"], "switch_tab");
assert_eq!(v["state"], "s2");
assert_eq!(v["result"]["state"], "s5");
let round: StepV3 = serde_json::from_value(v).unwrap();
assert_eq!(round, step);
}
#[test]
fn trace_v3_round_trips() {
let trace = sample_trace();
let v = serde_json::to_value(&trace).unwrap();
assert_eq!(v.get("version").and_then(|v| v.as_u64()), Some(3));
assert!(v.get("pages").is_none());
assert_eq!(v["recorder"]["vom"], 1);
assert_eq!(v["states"].as_array().unwrap().len(), 2);
let round: TraceV3 = serde_json::from_value(v).unwrap();
assert_eq!(round, trace);
}
#[test]
fn default_fields_are_omitted() {
let step = StepV3::Click {
common: sample_common(1, "s1", "s1"),
target: TargetDescriptorV3 {
element_ref: Some("e1".into()),
role: Some("button".into()),
name: Some("OK".into()),
ctx: None,
unmatched: false,
},
};
let v = serde_json::to_value(&step).unwrap();
assert!(v.get("unmatched").is_none());
assert!(v["target"].get("ctx").is_none());
assert!(v["target"].get("unmatched").is_none());
}
#[test]
fn unmatched_target_serializes_flag() {
let step = StepV3::Click {
common: sample_common(1, "s1", "s2"),
target: TargetDescriptorV3 {
element_ref: None,
role: Some("button".into()),
name: Some("发布".into()),
ctx: None,
unmatched: true,
},
};
let v = serde_json::to_value(&step).unwrap();
assert_eq!(v["target"]["unmatched"], true);
assert!(v["target"].get("ref").is_none());
}
#[test]
fn extension_trace_deserializes() {
let v = json!({
"version": 3,
"recorded_at": "2026-07-21T08:00:00Z",
"started_at": "2026-07-21T07:59:00Z",
"purpose": "demo",
"stopped_by": "user_finish",
"entry": { "start_url": "https://example.com/editor" },
"recorder": { "bsk": "0.1.10", "vom": 1 },
"states": [
{ "id": "s1", "url": "https://example.com/editor", "body": "@vom 1" },
{ "id": "s2", "url": "https://example.com/p/99", "body": "@vom 1" }
],
"steps": [
{
"op": "fill",
"id": 1,
"state": "s1",
"result": { "state": "s1" },
"target": { "ref": "e1", "role": "textbox", "name": "标题" },
"value": "hello",
"commit": "blur"
},
{
"op": "click",
"id": 2,
"state": "s1",
"result": { "state": "s2" },
"target": { "ref": "e2", "role": "button", "name": "发布" }
}
]
});
let trace: TraceV3 = serde_json::from_value(v).unwrap();
assert_eq!(trace.version, 3);
assert_eq!(trace.states.len(), 2);
assert_eq!(trace.steps.len(), 2);
}
}
@@ -48,7 +48,7 @@
"record": {
"sectionTitle": "Action recording",
"cardDesc": "Record your actions for Agent reference",
"sectionHint": "Generates a recording instruction — send it to your Agent to start recording your actions; the Agent can then use the recording to automate similar tasks.",
"sectionHint": "Generates a recording instruction — send it to your Agent to start recording your actions; the Agent can then use the resulting trace bundle to automate similar tasks.",
"topicLabel": "Topic (optional)",
"topicPlaceholder": "e.g. Publish an article",
"startUrlLabel": "Starting URL (optional)",
@@ -56,7 +56,7 @@
"copyButton": "Copy recording instructions",
"copied": "Copied",
"hintDisconnected": "Available when connected",
"promptTemplate": "Use BrowserSkill's `bsk` CLI to record the browser actions I'm about to demonstrate.\n\nSteps:\n1. Run: {{command}}\n2. Once you confirm it's started, I'll operate in the opened window myself.\n3. After I click Finish, read the returned trace.json and summarize the actions (pages + steps); don't re-run or \"verify\"."
"promptTemplate": "Use BrowserSkill's `bsk` CLI to record the browser actions I'm about to demonstrate.\n\nSteps:\n1. Run: {{command}}\n2. Once you confirm it's started, I'll operate in the opened window myself.\n3. After I click Finish, read the written `./trace` bundle (`trace.json` + `states/`) and summarize what I did."
}
},
"controlOverlay": {
@@ -48,7 +48,7 @@
"record": {
"sectionTitle": "操作录制",
"cardDesc": "录制你的操作,供 Agent 参考",
"sectionHint": "生成一段录制指令,发给 Agent 后可以启动插件录制用户操作,之后 Agent 可以参考录制结果自动完成同类任务。",
"sectionHint": "生成一段录制指令,发给 Agent 后可以启动插件录制用户操作,之后 Agent 可以参考写出的 trace 目录包自动完成同类任务。",
"topicLabel": "操作主题(可选)",
"topicPlaceholder": "例如:发布一篇文章",
"startUrlLabel": "起始网址(可选)",
@@ -56,7 +56,7 @@
"copyButton": "复制录制指令",
"copied": "已复制",
"hintDisconnected": "连接后可用",
"promptTemplate": "用 BrowserSkill 的 `bsk` CLI 录制我接下来的浏览器操作。\n\n步骤:\n1. 执行:{{command}}\n2. 你确认已开始后,我会在打开的窗口里自己操作。\n3. 我点“结束”后,读取返回的 trace.json 并总结操作(pages + steps),不要重跑或“验证”。"
"promptTemplate": "用 BrowserSkill 的 `bsk` CLI 录制我接下来的浏览器操作。\n\n步骤:\n1. 执行:{{command}}\n2. 你确认已开始后,我会在打开的窗口里自己操作。\n3. 我点“结束”后,读取写出的 `./trace` 目录包(`trace.json` + `states/`),总结我做了什么。"
}
},
"controlOverlay": {
+14 -6
View File
@@ -265,19 +265,27 @@ retry — complete the task autonomously or stop gracefully.
### Recording — `bsk record`
Capture the user's own actions in the Agent Window to a `trace.json`, for later LLM-driven automation:
Capture the user's own actions in the Agent Window to a **trace bundle**, for later LLM-driven automation. New CLI builds request **trace v3** (page observations + action chain); older extensions may still return **trace v2** (actions only), which the CLI exports as a single `trace.json` without `states/`.
```bash
bsk record start --browser <instance-id-or-label> [--url https://…] [--purpose "publish a wiki doc"] [--output trace.json]
bsk record start --browser <instance-id-or-label> \
[--url https://…] [--purpose "publish a wiki doc"] \
[--max-page-tokens 3000] [--redact-values] \
[--output trace]
# `--url` is optional; default https://example.com/ when omitted (must be http(s)).
# Blocks until the user clicks Finish in the recording panel, then writes ./trace.json and closes the window.
# Blocks until the user clicks Finish in the recording panel, then writes:
# trace/trace.json — action chain (+ state index when v3)
# trace/states/ — v3 only: one `sN.txt` observe snapshot per settled page state
bsk record stop [--output trace.json] # terminal fallback if the browser panel is unavailable
bsk record stop [--output trace] # terminal fallback if the browser panel is unavailable
```
- The trace is a **record-only action log** (a `pages[]` dictionary + `navigate`/`click`/`fill`/`select`/`press` steps with `target` descriptors). It records *what the user did*; deciding which inputs are variable is left to the executing agent.
- **v3 bundle (preferred):** `--output` is a directory (default `./trace`) containing `trace.json` and `states/`. `trace.json` lists `states[]` (page observation ids) and `steps[]` (each step binds `state` = observe snapshot *before* the action and `result.state` = snapshot *after* settle). Page bodies live in `states/sN.txt` using the same VOM format as `bsk observe`.
- **v2 fallback:** when the connected extension is older, export may contain only legacy `trace.json` with `pages[]` action context and no `states/` observation files. Update the extension for full v3 bundles.
- Each `states[]` entry is one **settled page observation** (captured after recording start, navigation landing, or action settle — not on a timer).
- `target.ref` values like `@e12` exist **only inside** the bundle for disambiguation; do **not** copy `@eN` refs into SKILL.md or agent runbooks — use visible names from the observation text instead.
- `--purpose` is optional context metadata; it does **not** change what gets captured.
- There is **no** `bsk replay` — to redo a flow, read the trace and reuse the existing `session` / `snapshot` / `@eN` / `click` / `fill` tools. Follow **Stop when the goal is met**.
- `--redact-values` masks all form values in page files as `[filled]` / `[empty]`.
- Do **not** record on banking/SSO/password-manager pages; passwords are redacted but traces may still contain sensitive text.
## Error handling