Compare commits

...
14 Commits
Author SHA1 Message Date
BB-fat b4450fbb8c chore(release): bsk 0.1.10 (#73) 2026-08-08 23:22:03 +08:00
BB-fat 1268761611 fix(screenshot): hide overlays during capture so shots only contain page content (#64) 2026-08-08 22:48:50 +08:00
Zhang GH 37d3e2b22f Merge pull request #72 from Tencent/issue-69-cws-publishing
CI: add automate Chrome Web Store publishing with service account
2026-08-07 19:24:30 +08:00
drakezhang ce8bc6c870 add automate Chrome Web Store publishing with service account 2026-08-07 14:30:59 +08:00
Zhang GH 93df62a356 Merge pull request #42 from shnpd/feat/popup-function-icon
feat(extension): use function launcher icon
2026-08-06 16:47:52 +08:00
Zhang GH 20002e1ff2 Merge pull request #63 from Tencent/feat/record-overlay-timer
feat(extension): compact recording overlay with live timer
2026-08-06 16:47:11 +08:00
“haonan” 674c53339f fix(extension): keep record overlay timer across navigations 2026-08-06 16:28:21 +08:00
BB-fat d0bee1a09b feat(install-skill): add Kimi Code harness support (#65)
* feat(install-skill): add Kimi Code harness support

Add kimi-code to the install-skill harness list: installs the bundled SKILL.md into ~/.kimi/skills (Kimi Code CLI's brand-group user skills directory, highest priority per https://github.com/MoonshotAI/kimi-cli/blob/main/docs/en/customization/skills.md). Detection signals: ~/.kimi directory and kimi/kimi-cli on PATH. Aliases: kimi, kimi-code, kimi-cli, kimicode, kimi_cli.

* fix(install-skill): point kimi-code harness at ~/.kimi-code and honor KIMI_CODE_HOME

Kimi Code CLI's canonical repo is MoonshotAI/kimi-code: user-level skills live in $KIMI_CODE_HOME/skills (default ~/.kimi-code/skills), not kimi-cli's ~/.kimi/skills. Detection checks the kimi-code home dir (env-aware label) and the kimi binary on PATH. Serialize env-mutating kimi tests with a shared mutex to avoid a parallel-test race on KIMI_CODE_HOME.
2026-08-06 13:25:18 +08:00
“haonan” 66684c4ce3 feat(extension): compact recording overlay with live timer
Show the recording message for 10s, then switch to a fixed-width MM:SS
timer with the icon-only finish button.
2026-08-05 20:26:43 +08:00
BB-fat 0f264c5df1 feat(emulate): add mobile device emulation via CDP Emulation domain (#60)
* feat(emulate): add mobile device emulation via CDP Emulation domain

Add `bsk emulate` so agents can debug mobile page behaviour by
emulating a device environment on a session tab:

- Viewport metrics (Emulation.setDeviceMetricsOverride), User-Agent
  (Emulation.setUserAgentOverride, optional accept-language and UA
  client-hints metadata), and touch (Emulation.setTouchEmulationEnabled
  with maxTouchPoints).
- Seven CLI-side device presets (iphone-14, iphone-14-pro-max,
  iphone-se, pixel-7, galaxy-s23, ipad-mini, galaxy-tab-s8) via
  --device; manual flags (--width/--height/--dpr/--mobile/--ua/
  --accept-language/--touch/--max-touch-points) work standalone or
  override individual preset fields.
- `bsk emulate --off` clears every override (clearDeviceMetricsOverride
  + touch disabled + empty UA override) and restores the real
  environment.
- Overrides are per-tab (CDP per-target) and not inherited by new tabs;
  the result JSON carries a note reminding agents to re-apply.

Wire: new tool.emulate method end-to-end (bsk-protocol types + JSON
schemas + Method::ToolEmulate classified BrowserMutation, daemon
forwarding, CLI subcommand, extension tool handler + ChromiumCdp
Emulation wrappers, dispatcher registration), docs in both SKILL.md
copies, and unit tests on every layer.

* refine(emulate): merge per-tab state and align preset override semantics

- extension: remember the applied emulation state per tab and merge each new request onto it field by field before applying, so a later --width/--height no longer resets an earlier preset's dpr/mobile; --off clears the stored state along with the CDP overrides
- extension: cdp_failed errors now note that already-applied overrides were not rolled back and can be reset with --off
- cli: validate width/height pairing and metric requirements on the merged preset+flags result, so --device iphone-14 --width 500 keeps the preset height
- cli: add --no-mobile/--no-touch (conflicting with --mobile/--touch) to turn preset mobile/touch fields back off
- docs: describe the merge behaviour, --no-mobile/--no-touch and --off state clearing in both SKILL.md copies
2026-08-05 18:10:39 +08:00
BB-fat 49378d70de feat: support Agent Window sizing (session start --width/--height, window resize) (#52)
- `bsk session start --width <px> --height <px>` opens the Agent Window
  with the given outer size (chrome.windows.create)
- new `bsk window resize --session <id> --width <px> --height <px>`
  resizes the Agent Window at runtime (chrome.windows.update)
- both validated on CLI, daemon and extension sides (100..=7680 CSS px)
- protocol: new tool.window_resize method (BrowserMutation) and
  optional width/height on tool.session_start params; schemas regenerated
2026-08-05 11:19:10 +08:00
BB-fat d7f0f1e78c fix(install-skill): restrict --all to detected harnesses (#59)
`bsk install-skill --all` installed the bundled skill into every supported harness directory (detected or not), creating junk paths like ~/.claude/skills on machines without those harnesses. --all now targets detected harnesses only and errors with guidance when none are found; --harness remains the explicit override. The --yes non-interactive branch shares the same detected-only logic.
2026-08-04 21:35:01 +08:00
BB-fat 71255ae142 feat(request-help): add BSK_REQUEST_HELP=off to disable blocking help requests (#53)
* feat(request-help): add BSK_REQUEST_HELP=off to disable blocking help requests

* style(extension): format HelpOutcome union per biome

* docs(skill): condense BSK_REQUEST_HELP section
2026-07-31 23:54:30 +08:00
“haonan”andCursor aab3b7a8e5 feat(extension): use function launcher icon
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-23 15:44:11 +08:00
61 changed files with 4695 additions and 104 deletions
+199
View File
@@ -126,3 +126,202 @@ jobs:
make_latest: false
generate_release_notes: true
files: dist/*.zip
publish-cws:
needs: [resolve, build-extension]
runs-on: ubuntu-latest
environment: chrome-web-store
permissions:
contents: read
steps:
- name: Download build artifact
uses: actions/download-artifact@v8
with:
name: extension-zip
path: dist
- name: Validate Chrome Web Store configuration
shell: bash
env:
PUBLISHER_ID: ${{ vars.CHROME_WEBSTORE_PUBLISHER_ID }}
EXTENSION_ID: ${{ vars.CHROME_WEBSTORE_EXTENSION_ID }}
run: |
set -euo pipefail
if [ -z "$PUBLISHER_ID" ]; then
echo "::error::Repository variable CHROME_WEBSTORE_PUBLISHER_ID is not configured"
exit 1
fi
if [ -z "$EXTENSION_ID" ]; then
echo "::error::Repository variable CHROME_WEBSTORE_EXTENSION_ID is not configured"
exit 1
fi
ARCHIVE="$(find dist -maxdepth 1 -name '*.zip' -type f -print -quit)"
if [ -z "$ARCHIVE" ]; then
echo "::error::No extension zip found in the downloaded artifact"
exit 1
fi
echo "ARCHIVE=$ARCHIVE" >> "$GITHUB_ENV"
- name: Authenticate with service account
id: cws-auth
env:
SERVICE_ACCOUNT_JSON: ${{ secrets.CHROME_WEBSTORE_SERVICE_ACCOUNT_JSON }}
shell: bash
run: |
set -euo pipefail
node --input-type=module <<'NODE'
import { sign } from "node:crypto";
import { appendFileSync } from "node:fs";
const rawCredentials = process.env.SERVICE_ACCOUNT_JSON;
if (!rawCredentials) {
throw new Error(
"Repository secret CHROME_WEBSTORE_SERVICE_ACCOUNT_JSON is not configured",
);
}
let credentials;
try {
credentials = JSON.parse(rawCredentials);
} catch {
throw new Error(
"Repository secret CHROME_WEBSTORE_SERVICE_ACCOUNT_JSON is not valid JSON",
);
}
if (!credentials.client_email || !credentials.private_key) {
throw new Error(
"Service account JSON must contain client_email and private_key",
);
}
const encode = (value) =>
Buffer.from(JSON.stringify(value)).toString("base64url");
const issuedAt = Math.floor(Date.now() / 1000);
const header = {
alg: "RS256",
typ: "JWT",
kid: credentials.private_key_id,
};
const claim = {
iss: credentials.client_email,
scope: "https://www.googleapis.com/auth/chromewebstore",
aud: "https://oauth2.googleapis.com/token",
iat: issuedAt,
exp: issuedAt + 3600,
};
const unsignedJwt = `${encode(header)}.${encode(claim)}`;
const signature = sign(
"RSA-SHA256",
Buffer.from(unsignedJwt),
credentials.private_key,
).toString("base64url");
const assertion = `${unsignedJwt}.${signature}`;
const response = await fetch("https://oauth2.googleapis.com/token", {
method: "POST",
headers: { "Content-Type": "application/x-www-form-urlencoded" },
body: new URLSearchParams({
grant_type: "urn:ietf:params:oauth:grant-type:jwt-bearer",
assertion,
}),
});
const responseText = await response.text();
if (!response.ok) {
throw new Error(
`Service account token exchange failed (${response.status}): ${responseText}`,
);
}
const tokenResponse = JSON.parse(responseText);
if (!tokenResponse.access_token) {
throw new Error("Service account token response has no access_token");
}
console.log(`::add-mask::${tokenResponse.access_token}`);
appendFileSync(
process.env.GITHUB_OUTPUT,
`access_token=${tokenResponse.access_token}\n`,
);
NODE
- name: Upload and publish Chrome Web Store extension
shell: bash
env:
ACCESS_TOKEN: ${{ steps.cws-auth.outputs.access_token }}
PUBLISHER_ID: ${{ vars.CHROME_WEBSTORE_PUBLISHER_ID }}
EXTENSION_ID: ${{ vars.CHROME_WEBSTORE_EXTENSION_ID }}
run: |
set -euo pipefail
ITEM_NAME="publishers/${PUBLISHER_ID}/items/${EXTENSION_ID}"
API_ROOT="https://chromewebstore.googleapis.com"
UPLOAD_RESPONSE="$RUNNER_TEMP/cws-upload-response.json"
STATUS_RESPONSE="$RUNNER_TEMP/cws-status-response.json"
PUBLISH_RESPONSE="$RUNNER_TEMP/cws-publish-response.json"
HTTP_STATUS="$(curl --silent --show-error \
--output "$UPLOAD_RESPONSE" \
--write-out '%{http_code}' \
--request POST \
--header "Authorization: Bearer $ACCESS_TOKEN" \
--header "Content-Type: application/zip" \
--data-binary "@$ARCHIVE" \
"$API_ROOT/upload/v2/$ITEM_NAME:upload")"
if [ "$HTTP_STATUS" -lt 200 ] || [ "$HTTP_STATUS" -ge 300 ]; then
echo "::error::Chrome Web Store upload failed with HTTP $HTTP_STATUS"
jq . "$UPLOAD_RESPONSE" || cat "$UPLOAD_RESPONSE"
exit 1
fi
UPLOAD_STATE="$(jq -r '.uploadState // empty' "$UPLOAD_RESPONSE")"
echo "Chrome Web Store upload state: $UPLOAD_STATE"
if [ "$UPLOAD_STATE" = "IN_PROGRESS" ] || [ "$UPLOAD_STATE" = "UPLOAD_IN_PROGRESS" ]; then
for attempt in $(seq 1 30); do
sleep 5
HTTP_STATUS="$(curl --silent --show-error \
--output "$STATUS_RESPONSE" \
--write-out '%{http_code}' \
--header "Authorization: Bearer $ACCESS_TOKEN" \
"$API_ROOT/v2/$ITEM_NAME:fetchStatus")"
if [ "$HTTP_STATUS" -lt 200 ] || [ "$HTTP_STATUS" -ge 300 ]; then
echo "::error::Chrome Web Store status check failed with HTTP $HTTP_STATUS"
jq . "$STATUS_RESPONSE" || cat "$STATUS_RESPONSE"
exit 1
fi
UPLOAD_STATE="$(jq -r '.lastAsyncUploadState // empty' "$STATUS_RESPONSE")"
echo "Upload status check $attempt/30: $UPLOAD_STATE"
if [ "$UPLOAD_STATE" = "SUCCEEDED" ]; then
break
fi
if [ "$UPLOAD_STATE" != "IN_PROGRESS" ] && [ "$UPLOAD_STATE" != "UPLOAD_IN_PROGRESS" ]; then
echo "::error::Chrome Web Store upload did not succeed: $UPLOAD_STATE"
jq . "$STATUS_RESPONSE"
exit 1
fi
done
fi
if [ "$UPLOAD_STATE" != "SUCCEEDED" ]; then
echo "::error::Chrome Web Store upload did not complete: $UPLOAD_STATE"
jq . "$UPLOAD_RESPONSE"
exit 1
fi
HTTP_STATUS="$(curl --silent --show-error \
--output "$PUBLISH_RESPONSE" \
--write-out '%{http_code}' \
--request POST \
--header "Authorization: Bearer $ACCESS_TOKEN" \
--header "Content-Type: application/json" \
--data '{"publishType":"DEFAULT_PUBLISH"}' \
"$API_ROOT/v2/$ITEM_NAME:publish")"
if [ "$HTTP_STATUS" -lt 200 ] || [ "$HTTP_STATUS" -ge 300 ]; then
echo "::error::Chrome Web Store publish failed with HTTP $HTTP_STATUS"
jq . "$PUBLISH_RESPONSE" || cat "$PUBLISH_RESPONSE"
exit 1
fi
echo "Chrome Web Store submission created successfully:"
jq '{itemId, state, warningInfo}' "$PUBLISH_RESPONSE"
Generated
+2 -2
View File
@@ -134,7 +134,7 @@ dependencies = [
[[package]]
name = "bsk"
version = "0.1.9"
version = "0.1.10"
dependencies = [
"anyhow",
"bsk-protocol",
@@ -168,7 +168,7 @@ dependencies = [
[[package]]
name = "bsk-protocol"
version = "0.1.9"
version = "0.1.10"
dependencies = [
"schemars",
"semver",
+1 -1
View File
@@ -7,7 +7,7 @@ members = [
default-members = ["crates/bsk-cli"]
[workspace.package]
version = "0.1.9"
version = "0.1.10"
edition = "2024"
rust-version = "1.85"
license = "MIT"
+4
View File
@@ -0,0 +1,4 @@
<svg width="24" height="24" viewBox="0 0 24 24" fill="none" xmlns="http://www.w3.org/2000/svg">
<path d="M6.30589 17.9712C6.21652 18.4603 6.63298 18.8722 7.12749 18.7838C7.88158 18.6491 8.86635 18.4075 9.25454 18.0236C9.49544 17.7853 9.61164 17.4432 9.57757 17.0726C9.5435 16.702 9.36195 16.3331 9.07287 16.0472C8.78378 15.7613 8.41084 15.5818 8.03608 15.5481C7.66133 15.5144 7.31545 15.6293 7.07455 15.8676C6.6865 16.2513 6.4422 17.2253 6.30589 17.9712Z" fill="#E94500"/>
<path d="M7.77425 8.64281C10.7374 4.12636 15.4876 3.7066 17.9364 4.13499C18.3335 4.20461 18.6474 4.48261 18.7821 4.84105C20.6715 9.87191 18.4167 13.7056 16.5008 15.5334C16.1262 15.8909 15.9395 16.3745 16.0282 16.8215L16.12 17.2834C16.24 17.887 15.871 18.4819 15.2723 18.6496L12.7811 19.3469C12.1057 19.536 11.4124 19.1112 11.285 18.4289L11.036 17.0959C10.9844 16.82 10.8353 16.5712 10.6151 16.3948L8.12777 14.4035C7.90665 14.2266 7.62818 14.1345 7.34359 14.1428L5.72835 14.1897C4.9877 14.2112 4.40832 13.5657 4.51644 12.8401L4.89632 10.2912C4.98194 9.71734 5.47627 9.2895 6.06234 9.28148L6.69124 9.27171C7.11716 9.26594 7.52243 9.02649 7.77425 8.64281ZM14.4959 7.91429C13.8284 7.91436 13.2869 8.45568 13.2869 9.12327C13.2871 9.79078 13.8284 10.3322 14.4959 10.3323C15.1635 10.3323 15.7048 9.79082 15.7049 9.12327C15.7049 8.45564 15.1636 7.91429 14.4959 7.91429Z" fill="#E94500"/>
</svg>

After

Width:  |  Height:  |  Size: 1.3 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.9 KiB

@@ -85,6 +85,23 @@ export const CDP_PROTOCOL_VERSION = "1.3";
/** Per-tab monotonic sequence returned by [`ChromiumCdp.dialogCursor`]. */
export type DialogCursor = number;
/** CDP `Emulation.setDeviceMetricsOverride` payload. */
export interface DeviceMetricsOverride {
width: number;
height: number;
/** `0` asks Chrome to use the display's native factor. */
deviceScaleFactor: number;
mobile: boolean;
}
/** CDP `Emulation.setUserAgentOverride` payload. */
export interface UserAgentOverride {
userAgent: string;
acceptLanguage?: string;
/** CDP `Emulation.UserAgentMetadata` (already camelCase). */
userAgentMetadata?: Record<string, unknown>;
}
const MAX_DIALOG_BUFFER = 32;
const MAX_DIALOG_FIELD_LENGTH = 4096;
const MAX_CONSOLE_BUFFER = 200;
@@ -194,6 +211,33 @@ export class ChromiumCdp {
return this.dialogSequences.get(tabId) ?? 0;
}
/** `Emulation.setDeviceMetricsOverride` — pin the tab's viewport metrics. */
async setDeviceMetricsOverride(tabId: number, metrics: DeviceMetricsOverride): Promise<void> {
await this.send(tabId, "Emulation.setDeviceMetricsOverride", { ...metrics });
}
/** `Emulation.clearDeviceMetricsOverride` — restore native viewport metrics. */
async clearDeviceMetricsOverride(tabId: number): Promise<void> {
await this.send(tabId, "Emulation.clearDeviceMetricsOverride", {});
}
/** `Emulation.setUserAgentOverride` — an empty `userAgent` restores the real UA. */
async setUserAgentOverride(tabId: number, override: UserAgentOverride): Promise<void> {
await this.send(tabId, "Emulation.setUserAgentOverride", { ...override });
}
/** `Emulation.setTouchEmulationEnabled`. */
async setTouchEmulationEnabled(
tabId: number,
enabled: boolean,
maxTouchPoints?: number,
): Promise<void> {
await this.send(tabId, "Emulation.setTouchEmulationEnabled", {
enabled,
...(maxTouchPoints !== undefined ? { maxTouchPoints } : {}),
});
}
/** Dialogs observed on `tabId` with sequence strictly greater than `cursor`. */
dialogsSince(tabId: number, cursor: DialogCursor): JavaScriptDialogInfo[] {
const buf = this.dialogBuffers.get(tabId) ?? [];
+216 -43
View File
@@ -1,9 +1,21 @@
import { useTranslation } from "@browser-skill/i18n/react";
import { RiStopCircleLine } from "@remixicon/react";
import { useEffect, useState } from "react";
import {
type PointerEvent as ReactPointerEvent,
useCallback,
useEffect,
useRef,
useState,
} from "react";
import recordIconUrl from "../../assets/record-icon.png";
export interface RecordRequestData {
id: string;
/**
* Epoch ms when the recording began. Supplied by the background so the timer
* keeps counting the whole session across navigations; falls back to mount
* time when the background could not report it.
*/
startedAtMs?: number;
onFinish: () => void;
}
@@ -11,9 +23,51 @@ type Props = {
request: RecordRequestData | null;
};
const VIEWPORT_MARGIN = 16;
const DRAG_THRESHOLD_PX = 8;
const INTRO_DURATION_MS = 10_000;
const TIMER_REFRESH_MS = 250;
function formatElapsed(seconds: number): string {
const minutes = Math.floor(seconds / 60);
const remainingSeconds = seconds % 60;
return `${String(minutes).padStart(2, "0")}:${String(remainingSeconds).padStart(2, "0")}`;
}
function clamp(value: number, min: number, max: number): number {
return Math.max(min, Math.min(max, value));
}
function clampDragPos(
top: number,
left: number,
panelW: number,
panelH: number,
): { top: number; left: number } {
const maxLeft = window.innerWidth - panelW - VIEWPORT_MARGIN;
const maxTop = window.innerHeight - panelH - VIEWPORT_MARGIN;
return {
top: clamp(top, VIEWPORT_MARGIN, Math.max(VIEWPORT_MARGIN, maxTop)),
left: clamp(left, VIEWPORT_MARGIN, Math.max(VIEWPORT_MARGIN, maxLeft)),
};
}
export function RecordOverlay({ request }: Props) {
const { t } = useTranslation("extension");
const [show, setShow] = useState(false);
const [elapsedSeconds, setElapsedSeconds] = useState(0);
const [dragPos, setDragPos] = useState<{ top: number; left: number } | null>(null);
const [dragging, setDragging] = useState(false);
const pillRef = useRef<HTMLDivElement>(null);
const dragStartRef = useRef<{
pointerX: number;
pointerY: number;
originLeft: number;
originTop: number;
panelW: number;
panelH: number;
armed: boolean;
} | null>(null);
useEffect(() => {
if (request) {
@@ -23,8 +77,87 @@ export function RecordOverlay({ request }: Props) {
setShow(false);
}, [request]);
const requestStartedAtMs = request?.startedAtMs;
useEffect(() => {
setDragPos(null);
setDragging(false);
dragStartRef.current = null;
if (!request) {
setElapsedSeconds(0);
return;
}
const startedAt = requestStartedAtMs ?? Date.now();
const updateElapsed = () => {
setElapsedSeconds(Math.max(0, Math.floor((Date.now() - startedAt) / 1_000)));
};
updateElapsed();
const timer = window.setInterval(updateElapsed, TIMER_REFRESH_MS);
return () => window.clearInterval(timer);
}, [request?.id, requestStartedAtMs]);
const onPillPointerDown = useCallback((e: ReactPointerEvent<HTMLDivElement>) => {
if (e.button !== 0) return;
if (e.target instanceof Element && e.target.closest("button, a, input, textarea, select")) {
return;
}
const pill = pillRef.current;
if (!pill) return;
const rect = pill.getBoundingClientRect();
dragStartRef.current = {
pointerX: e.clientX,
pointerY: e.clientY,
originLeft: rect.left,
originTop: rect.top,
panelW: pill.offsetWidth || rect.width,
panelH: pill.offsetHeight || rect.height,
armed: false,
};
}, []);
useEffect(() => {
const onMove = (e: PointerEvent) => {
const start = dragStartRef.current;
if (!start) return;
const dx = e.clientX - start.pointerX;
const dy = e.clientY - start.pointerY;
if (!start.armed) {
if (Math.hypot(dx, dy) < DRAG_THRESHOLD_PX) return;
start.armed = true;
setDragging(true);
setDragPos({ top: start.originTop, left: start.originLeft });
}
setDragPos(
clampDragPos(start.originTop + dy, start.originLeft + dx, start.panelW, start.panelH),
);
};
const onUp = () => {
const start = dragStartRef.current;
if (!start) return;
if (start.armed) {
setDragging(false);
}
dragStartRef.current = null;
};
window.addEventListener("pointermove", onMove);
window.addEventListener("pointerup", onUp);
window.addEventListener("pointercancel", onUp);
return () => {
window.removeEventListener("pointermove", onMove);
window.removeEventListener("pointerup", onUp);
window.removeEventListener("pointercancel", onUp);
};
}, []);
if (!request) return null;
const introVisible = elapsedSeconds * 1_000 < INTRO_DURATION_MS;
const positioned = dragPos !== null;
const defaultTransform = show
? "translateX(-50%) translateY(0)"
: "translateX(-50%) translateY(8px)";
return (
<>
<style>{`
@@ -35,78 +168,118 @@ export function RecordOverlay({ request }: Props) {
`}</style>
<div
ref={pillRef}
data-slot="record-overlay-pill"
data-phase={introVisible ? "intro" : "timer"}
data-dragging={dragging ? "true" : "false"}
onPointerDown={onPillPointerDown}
style={{
position: "fixed",
bottom: 32,
left: "50%",
...(positioned
? { top: dragPos.top, left: dragPos.left, bottom: "auto" }
: { bottom: 32, left: "50%" }),
zIndex: 2147483647,
pointerEvents: "auto",
display: "flex",
alignItems: "center",
gap: 10,
gap: 5,
backgroundColor: "#fff",
borderRadius: 9999,
padding: "10px 10px 10px 20px",
boxShadow: "0 8px 32px rgba(15,23,42,0.16), 0 2px 8px rgba(0,0,0,0.1)",
height: 50,
boxSizing: "border-box",
padding: "6px 8px 6px 16px",
boxShadow: dragging
? "0 12px 40px rgba(15,23,42,0.22), 0 2px 8px rgba(0,0,0,0.12)"
: "0 8px 32px rgba(15,23,42,0.16), 0 2px 8px rgba(0,0,0,0.1)",
opacity: show ? 1 : 0,
transform: show ? "translateX(-50%) translateY(0)" : "translateX(-50%) translateY(8px)",
transition: "opacity 300ms ease-out, transform 300ms ease-out",
transform: positioned ? "none" : defaultTransform,
transition: dragging
? "box-shadow 150ms ease-out"
: "opacity 300ms ease-out, transform 300ms ease-out, box-shadow 150ms ease-out",
fontFamily:
'-apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Helvetica, Arial, sans-serif',
maxWidth: "min(420px, calc(100vw - 32px))",
cursor: dragging ? "grabbing" : "grab",
userSelect: "none",
touchAction: "none",
}}
>
<span
data-slot="record-overlay-indicator"
style={{
width: 9,
height: 9,
borderRadius: "50%",
backgroundColor: "#ef4444",
flexShrink: 0,
animation: "bsk-rec-pulse 1.4s ease-in-out infinite",
}}
/>
<span
<div
data-slot="record-overlay-status"
style={{
display: "flex",
alignItems: "center",
gap: 8,
minWidth: 0,
flex: 1,
fontSize: 16,
fontWeight: 500,
color: "#333",
whiteSpace: "nowrap",
userSelect: "none",
overflow: "hidden",
textOverflow: "ellipsis",
borderRadius: 9999,
outline: "none",
minHeight: 32,
paddingRight: 4,
}}
>
{t("recordOverlay.recording")}
</span>
<span
data-slot="record-overlay-indicator"
style={{
width: 9,
height: 9,
borderRadius: "50%",
backgroundColor: "#ef4444",
flexShrink: 0,
animation: "bsk-rec-pulse 1.4s ease-in-out infinite",
}}
/>
<span
data-slot="record-overlay-label"
style={{
display: "block",
flex: "0 1 auto",
fontSize: 16,
fontWeight: 500,
color: "#333",
whiteSpace: "nowrap",
userSelect: "none",
overflow: "hidden",
textOverflow: "ellipsis",
width: introVisible ? "auto" : 48,
maxWidth: introVisible ? 280 : 48,
fontVariantNumeric: introVisible ? undefined : "tabular-nums",
textAlign: introVisible ? "left" : "center",
margin: 0,
padding: 0,
minWidth: 0,
transition: "max-width 180ms ease-out",
}}
>
{introVisible ? t("recordOverlay.recording") : formatElapsed(elapsedSeconds)}
</span>
</div>
<button
type="button"
data-slot="record-overlay-finish"
aria-label={t("recordOverlay.finish")}
onClick={request.onFinish}
style={{
pointerEvents: "auto",
display: "flex",
alignItems: "center",
gap: 6,
display: "block",
width: 32,
height: 32,
border: "none",
borderRadius: 9999,
padding: "8px 20px 8px 16px",
fontSize: 15,
fontWeight: 600,
color: "#fff",
backgroundColor: "#f97316",
padding: 0,
backgroundColor: "transparent",
cursor: "pointer",
transition: "background-color 150ms ease-out, opacity 150ms ease-out",
whiteSpace: "nowrap",
lineHeight: 1,
flexShrink: 0,
}}
>
<RiStopCircleLine size={18} color="#fff" />
{t("recordOverlay.finish")}
<img
src={recordIconUrl}
alt=""
width={24}
height={24}
draggable={false}
style={{ display: "block", margin: "auto" }}
/>
</button>
</div>
</>
@@ -1,37 +1,181 @@
import { cleanup, fireEvent, render } from "@testing-library/react";
import { i18n } from "@browser-skill/i18n";
import { I18nextProvider } from "@browser-skill/i18n/react";
import { act, cleanup, fireEvent, render, screen } from "@testing-library/react";
import { createElement } from "react";
import { afterEach, describe, expect, it, vi } from "vitest";
import { RecordOverlay } from "../RecordOverlay";
function renderOverlay(request: { id: string; startedAtMs?: number; onFinish: () => void } | null) {
return render(
createElement(I18nextProvider, { i18n }, createElement(RecordOverlay, { request })),
);
}
function advanceRecording(ms: number) {
act(() => {
vi.advanceTimersByTime(ms);
});
}
describe("RecordOverlay", () => {
afterEach(() => {
cleanup();
vi.useRealTimers();
});
it("renders the bottom pill with orange finish CTA and no logo", () => {
it("renders the bottom pill with the icon-only finish button", () => {
const onFinish = vi.fn();
const { container } = render(<RecordOverlay request={{ id: "rec-1", onFinish }} />);
const { container } = renderOverlay({ id: "rec-1", onFinish });
const pill = container.querySelector("[data-slot='record-overlay-pill']");
expect(pill).toBeTruthy();
expect(pill?.querySelector("img")).toBeNull();
expect((pill as HTMLElement).style.borderRadius).toBe("9999px");
expect((pill as HTMLElement).style.backgroundColor).toBe("#fff");
expect((pill as HTMLElement).style.gap).toBe("5px");
expect((pill as HTMLElement).style.padding).toBe("6px 8px 6px 16px");
expect((pill as HTMLElement).style.height).toBe("50px");
expect((pill as HTMLElement).style.boxSizing).toBe("border-box");
expect(container.querySelector("[data-slot='record-overlay-collapse']")).toBeNull();
const finish = container.querySelector("[data-slot='record-overlay-finish']");
const status = container.querySelector("[data-slot='record-overlay-status']") as HTMLElement;
expect(status.style.minHeight).toBe("32px");
const finish = screen.getByRole("button", {
name: i18n.t("recordOverlay.finish", { ns: "extension" }),
});
expect(finish).toBeTruthy();
expect((finish as HTMLElement).style.backgroundColor).toBe("#f97316");
expect(finish.textContent).toBe("");
const icon = finish.querySelector("img");
expect(icon).toBeTruthy();
expect(icon?.getAttribute("src")).toContain("record-icon.png");
expect((finish as HTMLElement).style.width).toBe("32px");
expect((finish as HTMLElement).style.height).toBe("32px");
expect(icon?.getAttribute("width")).toBe("24");
expect(icon?.getAttribute("height")).toBe("24");
expect((finish as HTMLElement).style.borderRadius).toBe("9999px");
fireEvent.click(finish!);
fireEvent.click(finish);
expect(onFinish).toHaveBeenCalledTimes(1);
});
it("shows a pulsing recording indicator and no full-screen glow layer", () => {
const { container } = render(<RecordOverlay request={{ id: "rec-1", onFinish: vi.fn() }} />);
const { container } = renderOverlay({ id: "rec-1", onFinish: vi.fn() });
expect(container.querySelector("[data-slot='record-overlay']")).toBeNull();
const indicator = container.querySelector("[data-slot='record-overlay-indicator']");
expect(indicator).toBeTruthy();
expect((indicator as HTMLElement).style.animation).toContain("bsk-rec-pulse");
});
it("shows the recording message for ten seconds, then displays elapsed time", () => {
vi.useFakeTimers();
const { container } = renderOverlay({ id: "rec-1", onFinish: vi.fn() });
const pill = container.querySelector("[data-slot='record-overlay-pill']") as HTMLElement;
const label = container.querySelector("[data-slot='record-overlay-label']") as HTMLElement;
expect(pill.dataset.phase).toBe("intro");
expect(label.textContent).toBe(i18n.t("recordOverlay.recording", { ns: "extension" }));
advanceRecording(9999);
expect(pill.dataset.phase).toBe("intro");
advanceRecording(1);
expect(pill.dataset.phase).toBe("timer");
expect(label.textContent).toBe("00:10");
});
it("formats elapsed time as zero-padded minutes and seconds", () => {
vi.useFakeTimers();
const { container } = renderOverlay({ id: "rec-1", onFinish: vi.fn() });
const label = container.querySelector("[data-slot='record-overlay-label']") as HTMLElement;
advanceRecording(125_000);
expect(label.textContent).toBe("02:05");
});
it("keeps a fixed timer label width with tabular numerals", () => {
vi.useFakeTimers();
const { container } = renderOverlay({ id: "rec-1", onFinish: vi.fn() });
const label = container.querySelector("[data-slot='record-overlay-label']") as HTMLElement;
advanceRecording(10_000);
expect(label.style.width).toBe("48px");
expect(label.style.maxWidth).toBe("48px");
expect(label.style.fontVariantNumeric).toBe("tabular-nums");
expect(label.style.textAlign).toBe("center");
advanceRecording(1_000);
expect(label.textContent).toBe("00:11");
expect(label.style.width).toBe("48px");
});
it("resets the intro and elapsed time for a new request", () => {
vi.useFakeTimers();
const first = { id: "rec-1", onFinish: vi.fn() };
const second = { id: "rec-2", onFinish: vi.fn() };
const { container, rerender } = renderOverlay(first);
advanceRecording(65_000);
rerender(
createElement(I18nextProvider, { i18n }, createElement(RecordOverlay, { request: second })),
);
const pill = container.querySelector("[data-slot='record-overlay-pill']") as HTMLElement;
const label = container.querySelector("[data-slot='record-overlay-label']") as HTMLElement;
expect(pill.dataset.phase).toBe("intro");
expect(label.textContent).toBe(i18n.t("recordOverlay.recording", { ns: "extension" }));
advanceRecording(10_000);
expect(pill.dataset.phase).toBe("timer");
expect(label.textContent).toBe("00:10");
});
it("continues the session timer when the overlay remounts on a new page", () => {
vi.useFakeTimers();
const startedAtMs = Date.now() - 65_000;
const { container } = renderOverlay({ id: "rec-1", startedAtMs, onFinish: vi.fn() });
const pill = container.querySelector("[data-slot='record-overlay-pill']") as HTMLElement;
const label = container.querySelector("[data-slot='record-overlay-label']") as HTMLElement;
expect(pill.dataset.phase).toBe("timer");
expect(label.textContent).toBe("01:05");
advanceRecording(5_000);
expect(label.textContent).toBe("01:10");
});
it("does not start drag when pointer down on finish", () => {
const { container } = renderOverlay({ id: "rec-1", onFinish: vi.fn() });
const pill = container.querySelector("[data-slot='record-overlay-pill']") as HTMLElement;
const finish = container.querySelector("[data-slot='record-overlay-finish']") as HTMLElement;
fireEvent.pointerDown(finish, { button: 0, clientX: 10, clientY: 10, pointerId: 1 });
expect(pill.getAttribute("data-dragging")).toBe("false");
});
it("moves the pill when dragging the non-button area without toggling collapse", () => {
const { container } = renderOverlay({ id: "rec-1", onFinish: vi.fn() });
const pill = container.querySelector("[data-slot='record-overlay-pill']") as HTMLElement;
pill.getBoundingClientRect = () =>
({
top: 600,
left: 200,
width: 320,
height: 48,
right: 520,
bottom: 648,
}) as DOMRect;
Object.defineProperty(pill, "offsetWidth", { value: 320, configurable: true });
Object.defineProperty(pill, "offsetHeight", { value: 48, configurable: true });
fireEvent.pointerDown(pill, { button: 0, clientX: 300, clientY: 620, pointerId: 1 });
fireEvent(window, new PointerEvent("pointermove", { clientX: 360, clientY: 580 }));
fireEvent(window, new PointerEvent("pointerup", { pointerId: 1 }));
expect(pill.getAttribute("data-phase")).toBe("intro");
expect(pill.style.left).toBe("260px");
expect(pill.style.top).toBe("560px");
expect(pill.style.bottom).toBe("auto");
expect(pill.style.transform).toBe("none");
});
});
@@ -0,0 +1,125 @@
import { beforeEach, describe, expect, it, vi } from "vitest";
import { CAPTURE_SUPPRESS, type CaptureSuppressAck } from "@/lib/capture-suppress-bridge";
import { CAPTURE_HIDDEN_ATTR, createCaptureSuppressController } from "../capture-suppress";
/**
* rAF is stubbed with a manual queue so tests can drive the compositor
* frame-by-frame and assert the ack only lands after the second frame.
*/
describe("capture-suppress controller", () => {
let rafQueue: FrameRequestCallback[];
let host: HTMLElement;
beforeEach(() => {
rafQueue = [];
vi.stubGlobal("requestAnimationFrame", (cb: FrameRequestCallback) => {
rafQueue.push(cb);
return rafQueue.length;
});
vi.stubGlobal("cancelAnimationFrame", vi.fn());
document.body.innerHTML = "";
host = document.createElement("div");
document.body.append(host);
});
/** Run every queued rAF callback, draining microtasks between rounds. */
async function flushFrames(rounds = 4): Promise<void> {
for (let round = 0; round < rounds && rafQueue.length > 0; round += 1) {
const callbacks = rafQueue.splice(0);
for (const cb of callbacks) cb(0);
await Promise.resolve();
await Promise.resolve();
}
}
it("begin hides the host immediately but only acks after two frames", async () => {
const controller = createCaptureSuppressController(() => host);
const sendResponse = vi.fn();
const needsAsync = controller.handleMessage(
{ type: CAPTURE_SUPPRESS, phase: "begin" },
sendResponse,
);
expect(needsAsync).toBe(true);
expect(controller.pendingCount).toBe(1);
expect(host.hasAttribute(CAPTURE_HIDDEN_ATTR)).toBe(true);
expect(sendResponse).not.toHaveBeenCalled();
// First frame commits the style change; the ack must still be pending.
await flushFrames(1);
expect(sendResponse).not.toHaveBeenCalled();
// Second frame: the compositor has produced an overlay-free frame.
await flushFrames();
const expected: CaptureSuppressAck = { type: CAPTURE_SUPPRESS, ok: true };
expect(sendResponse).toHaveBeenCalledWith(expected);
});
it("end removes the attribute once the count returns to zero", async () => {
const controller = createCaptureSuppressController(() => host);
controller.handleMessage({ type: CAPTURE_SUPPRESS, phase: "begin" }, vi.fn());
await flushFrames();
expect(host.hasAttribute(CAPTURE_HIDDEN_ATTR)).toBe(true);
const sendResponse = vi.fn();
const needsAsync = controller.handleMessage(
{ type: CAPTURE_SUPPRESS, phase: "end" },
sendResponse,
);
expect(needsAsync).toBe(false);
expect(controller.pendingCount).toBe(0);
expect(host.hasAttribute(CAPTURE_HIDDEN_ATTR)).toBe(false);
expect(sendResponse).toHaveBeenCalledWith({ type: CAPTURE_SUPPRESS, ok: true });
});
it("reference-counts concurrent begins so one end does not unhide early", async () => {
const controller = createCaptureSuppressController(() => host);
controller.handleMessage({ type: CAPTURE_SUPPRESS, phase: "begin" }, vi.fn());
controller.handleMessage({ type: CAPTURE_SUPPRESS, phase: "begin" }, vi.fn());
await flushFrames();
expect(controller.pendingCount).toBe(2);
controller.handleMessage({ type: CAPTURE_SUPPRESS, phase: "end" }, vi.fn());
expect(controller.pendingCount).toBe(1);
expect(host.hasAttribute(CAPTURE_HIDDEN_ATTR)).toBe(true);
controller.handleMessage({ type: CAPTURE_SUPPRESS, phase: "end" }, vi.fn());
expect(controller.pendingCount).toBe(0);
expect(host.hasAttribute(CAPTURE_HIDDEN_ATTR)).toBe(false);
});
it("treats a stray end as a no-op", () => {
const controller = createCaptureSuppressController(() => host);
const sendResponse = vi.fn();
controller.handleMessage({ type: CAPTURE_SUPPRESS, phase: "end" }, sendResponse);
expect(controller.pendingCount).toBe(0);
expect(host.hasAttribute(CAPTURE_HIDDEN_ATTR)).toBe(false);
expect(sendResponse).toHaveBeenCalledWith({ type: CAPTURE_SUPPRESS, ok: true });
});
it("re-applies the attribute when the host is remounted mid-capture", () => {
let currentHost: HTMLElement | null = host;
const controller = createCaptureSuppressController(() => currentHost);
controller.handleMessage({ type: CAPTURE_SUPPRESS, phase: "begin" }, vi.fn());
// Host lost and rebuilt before `end` arrives.
currentHost = null;
const rebuilt = document.createElement("div");
controller.onHostMounted(rebuilt);
expect(rebuilt.hasAttribute(CAPTURE_HIDDEN_ATTR)).toBe(true);
});
it("leaves a remounted host visible once suppression has ended", () => {
const controller = createCaptureSuppressController(() => host);
const rebuilt = document.createElement("div");
controller.onHostMounted(rebuilt);
expect(rebuilt.hasAttribute(CAPTURE_HIDDEN_ATTR)).toBe(false);
});
});
@@ -0,0 +1,99 @@
/**
* Content-script side of the capture-suppress bridge: hides the overlay
* shadow host while the background captures a screenshot.
*
* `begin` sets `data-bsk-capture-hidden` on the host (overlay.css maps it
* to `display: none !important`, which removes the whole shadow subtree
* from compositing) and only acks after two animation frames — the first
* frame commits the style change, the second guarantees the compositor
* has produced a frame without the overlay, so the capture API cannot
* pick up a stale frame that still contains it. `end` is
* reference-counted so overlapping captures do not unhide the host early.
*/
import {
CAPTURE_SUPPRESS,
type CaptureSuppressAck,
type CaptureSuppressMessage,
} from "@/lib/capture-suppress-bridge";
export const CAPTURE_HIDDEN_ATTR = "data-bsk-capture-hidden";
/**
* Safety net for hidden/background tabs where rAF callbacks never fire:
* the attribute is already set synchronously, so any frame the compositor
* produces from now on excludes the overlay. Waiting longer would just
* deadlock the capture.
*/
const FRAME_FALLBACK_MS = 500;
function nextFrame(): Promise<void> {
return new Promise((resolve) => {
const raf = requestAnimationFrame(() => {
clearTimeout(timer);
resolve();
});
const timer = setTimeout(() => {
cancelAnimationFrame(raf);
resolve();
}, FRAME_FALLBACK_MS);
});
}
export interface CaptureSuppressController {
/**
* Handle one bridge message. Returns true when the ack is sent
* asynchronously and the caller must keep the message channel open.
*/
handleMessage(
message: CaptureSuppressMessage,
sendResponse: (ack: CaptureSuppressAck) => void,
): boolean;
/** Re-apply the hidden attribute when the overlay host is (re)mounted. */
onHostMounted(host: HTMLElement): void;
/** Number of in-flight `begin` phases (exposed for tests). */
readonly pendingCount: number;
}
export function createCaptureSuppressController(
getHost: () => HTMLElement | null,
): CaptureSuppressController {
let pending = 0;
const ack: CaptureSuppressAck = { type: CAPTURE_SUPPRESS, ok: true };
function setHidden(host: HTMLElement | null, hidden: boolean): void {
if (!host) return;
if (hidden) host.setAttribute(CAPTURE_HIDDEN_ATTR, "");
else host.removeAttribute(CAPTURE_HIDDEN_ATTR);
}
return {
get pendingCount() {
return pending;
},
handleMessage(message, sendResponse) {
if (message.phase === "begin") {
pending += 1;
setHidden(getHost(), true);
void (async () => {
await nextFrame();
await nextFrame();
sendResponse(ack);
})();
return true;
}
// `end` without a matching `begin` (e.g. a retried message) must not
// drive the counter negative or clear a concurrent capture's flag.
pending = Math.max(0, pending - 1);
if (pending === 0) setHidden(getHost(), false);
sendResponse(ack);
return false;
},
onHostMounted(host) {
// Host lost + rebuilt mid-capture: the fresh host must stay hidden
// until the last `end` arrives.
setHidden(host, pending > 0);
},
};
}
+7
View File
@@ -14,3 +14,10 @@
:host([data-bsk-overlay-surface][data-bsk-overlay-blocking]) {
pointer-events: auto !important;
}
/* Screenshot suppression: the host lives in the main DOM, so hiding it
removes the whole shadow subtree from compositing and captured frames
only contain page content. */
:host([data-bsk-capture-hidden]) {
display: none !important;
}
+2 -2
View File
@@ -468,7 +468,7 @@ export function handleRecordContentMessage(
capture: RecordCaptureController | null;
setActiveRequestId(id: string | null): void;
setCapture(capture: RecordCaptureController | null): void;
onStart(requestId: string): void;
onStart(requestId: string, startedAtMs?: number): void;
onStop(): void;
},
sendResponse?: (response: RecordStartAck | RecordStopAck) => void,
@@ -485,7 +485,7 @@ export function handleRecordContentMessage(
}),
);
state.setActiveRequestId(message.requestId);
state.onStart(message.requestId);
state.onStart(message.requestId, message.startedAtMs);
sendResponse?.({ ok: true });
return sendResponse !== undefined;
}
+22 -3
View File
@@ -5,6 +5,7 @@ import { flushSync } from "react-dom";
import ReactDOM from "react-dom/client";
import { BorrowConfirmationOverlay } from "@/content/BorrowConfirmationOverlay";
import { ControlOverlay } from "@/content/ControlOverlay";
import { createCaptureSuppressController } from "@/content/capture-suppress";
import { HelpRequestOverlay } from "@/content/HelpRequestOverlay";
import overlayCss from "@/content/overlay.css?inline";
import { OverlayController, shouldShowAgentControlOverlay } from "@/content/overlay-controller";
@@ -14,6 +15,11 @@ import {
isRecordContentMessage,
type RecordCaptureController,
} from "@/content/record-capture";
import {
type CaptureSuppressAck,
type CaptureSuppressMessage,
isCaptureSuppressMessage,
} from "@/lib/capture-suppress-bridge";
import {
HELP_ACK,
HELP_FINISH,
@@ -39,6 +45,7 @@ import { sendInterrupt } from "@/lib/overlay-interrupt-client";
import {
RECORD_FINISH,
RECORD_QUERY,
type RecordQueryResponse,
type RecordStartAck,
type RecordStopAck,
} from "@/lib/record-bridge";
@@ -69,6 +76,8 @@ export default defineContentScript({
let hostLossReported = false;
let remountInProgress = false;
const captureSuppress = createCaptureSuppressController(() => overlayHost);
const ui = await createShadowRootUi(ctx, {
name: "browser-skill-overlay",
position: "inline",
@@ -80,6 +89,8 @@ export default defineContentScript({
overlayHost = shadowHost;
overlayContainer = container;
hostLossReported = false;
// A host rebuilt mid-capture must stay hidden until `end` arrives.
captureSuppress.onHostMounted(shadowHost);
const app = document.createElement("div");
app.className = "bsk-overlay-root";
container.append(app);
@@ -237,12 +248,17 @@ export default defineContentScript({
| BorrowCancelMessage
| HelpRequestMessage
| HelpCancelMessage
| CaptureSuppressMessage
| OverlayAgentOverlayResetMessage
| OverlayAgentStateMessage
| OverlayAutomationBypassMessage,
_sender: chrome.runtime.MessageSender,
sendResponse: (response: BorrowResponseMessage | HelpAckMessage) => void,
sendResponse: (response: BorrowResponseMessage | HelpAckMessage | CaptureSuppressAck) => void,
) => {
if (isCaptureSuppressMessage(message)) {
return captureSuppress.handleMessage(message, sendResponse);
}
if (isRecordContentMessage(message)) {
const needsAsync = handleRecordContentMessage(
message,
@@ -255,9 +271,10 @@ export default defineContentScript({
setCapture: (capture) => {
recordCapture = capture;
},
onStart: (requestId) => {
onStart: (requestId, startedAtMs) => {
overlays.setAgentRecordRequest({
id: requestId,
...(typeof startedAtMs === "number" ? { startedAtMs } : {}),
onFinish: () => {
void chrome.runtime.sendMessage({
type: RECORD_FINISH,
@@ -415,15 +432,17 @@ export default defineContentScript({
try {
const recordQuery = (await chrome.runtime.sendMessage({
type: RECORD_QUERY,
})) as { active?: boolean; requestId?: string } | undefined;
})) as RecordQueryResponse | undefined;
if (
recordQuery?.active &&
typeof recordQuery.requestId === "string" &&
overlays.snapshot().activeRecord === null
) {
const requestId = recordQuery.requestId;
const startedAtMs = recordQuery.startedAtMs;
overlays.setAgentRecordRequest({
id: requestId,
...(typeof startedAtMs === "number" ? { startedAtMs } : {}),
onFinish: () => {
void chrome.runtime.sendMessage({
type: RECORD_FINISH,
+9 -9
View File
@@ -1,14 +1,9 @@
import { useTranslation } from "@browser-skill/i18n/react";
import { Badge, Button, cn, Input, Label } from "@browser-skill/ui";
import {
RiApps2Line,
RiArrowLeftLine,
RiArrowRightSLine,
RiCheckLine,
RiFileCopyLine,
} from "@remixicon/react";
import { RiArrowLeftLine, RiArrowRightSLine, RiCheckLine, RiFileCopyLine } from "@remixicon/react";
import { type ChangeEvent, useEffect, useState } from "react";
import { PROTOCOL_VERSION } from "@/transport/handshake";
import functionIconUrl from "../../../assets/function.svg";
import { ConnectionStatusIndicator } from "./connection-status-indicator";
import { POPUP_FEATURES, type PopupView } from "./features";
import { type PopupStatusState, useConnectionState } from "./use-connection-state";
@@ -156,12 +151,17 @@ export function App() {
type="button"
variant="ghost"
size="icon"
className="size-7 shrink-0 rounded-md"
className="group size-8 shrink-0 rounded-md"
aria-label={t("popup.launcher.title")}
onClick={() => setView("features")}
data-slot="popup-launcher"
>
<RiApps2Line className="size-4" aria-hidden />
<img
src={functionIconUrl}
alt=""
className="size-6 opacity-35 transition-opacity group-hover:opacity-100 group-focus-visible:opacity-100"
data-slot="popup-launcher-icon"
/>
</Button>
)}
</header>
@@ -0,0 +1,77 @@
import { describe, expect, it, vi } from "vitest";
import {
CAPTURE_SUPPRESS,
type CaptureSuppressMessage,
withOverlaysHiddenForCapture,
} from "../capture-suppress-bridge";
function recordingSendToTab(events: string[]) {
return vi.fn(async (_tabId: number, message: CaptureSuppressMessage) => {
events.push(message.phase);
return { type: CAPTURE_SUPPRESS, ok: true };
});
}
describe("withOverlaysHiddenForCapture", () => {
it("sends begin → fn → end and returns fn's result", async () => {
const events: string[] = [];
const sendToTab = recordingSendToTab(events);
const result = await withOverlaysHiddenForCapture(
7,
async () => {
events.push("capture");
return "shot";
},
sendToTab,
);
expect(result).toBe("shot");
expect(events).toEqual(["begin", "capture", "end"]);
expect(sendToTab).toHaveBeenCalledWith(7, { type: CAPTURE_SUPPRESS, phase: "begin" });
expect(sendToTab).toHaveBeenCalledWith(7, { type: CAPTURE_SUPPRESS, phase: "end" });
});
it("still sends end when fn throws, then rethrows", async () => {
const events: string[] = [];
const sendToTab = recordingSendToTab(events);
await expect(
withOverlaysHiddenForCapture(
7,
async () => {
events.push("capture");
throw new Error("capture exploded");
},
sendToTab,
),
).rejects.toThrow("capture exploded");
expect(events).toEqual(["begin", "capture", "end"]);
});
it("captures directly when the tab has no content script", async () => {
const sendToTab = vi.fn(async () => {
throw new Error("Could not establish connection. Receiving end does not exist.");
});
const fn = vi.fn(async () => "shot");
const result = await withOverlaysHiddenForCapture(7, fn, sendToTab);
expect(result).toBe("shot");
expect(fn).toHaveBeenCalledTimes(1);
// begin failed → no matching end may be sent.
expect(sendToTab).toHaveBeenCalledTimes(1);
});
it("swallows end failures so the capture result stands", async () => {
const sendToTab = vi.fn(async (_tabId: number, message: CaptureSuppressMessage) => {
if (message.phase === "end") throw new Error("tab navigated away");
return { type: CAPTURE_SUPPRESS, ok: true };
});
const result = await withOverlaysHiddenForCapture(7, async () => "shot", sendToTab);
expect(result).toBe("shot");
expect(sendToTab).toHaveBeenCalledTimes(2);
});
});
@@ -0,0 +1,71 @@
/**
* Wire protocol and background-side helper for hiding the extension's
* in-page overlay while a screenshot is captured, so captured frames only
* contain the page itself.
*
* The background wraps every capture in `withOverlaysHiddenForCapture`,
* which sends `begin` (the content script hides the overlay host and only
* acks once the compositor has produced an overlay-free frame) and always
* follows up with `end`, even when the capture throws. Tabs without the
* content script (chrome://, the Web Store, ...) host no overlay, so a
* failed `begin` falls through to capturing directly.
*/
export const CAPTURE_SUPPRESS = "bsk/capture-suppress";
export type CaptureSuppressPhase = "begin" | "end";
export interface CaptureSuppressMessage {
type: typeof CAPTURE_SUPPRESS;
phase: CaptureSuppressPhase;
}
export interface CaptureSuppressAck {
type: typeof CAPTURE_SUPPRESS;
ok: true;
}
export function isCaptureSuppressMessage(msg: unknown): msg is CaptureSuppressMessage {
if (typeof msg !== "object" || msg === null) return false;
const m = msg as Record<string, unknown>;
return m.type === CAPTURE_SUPPRESS && (m.phase === "begin" || m.phase === "end");
}
/** Minimal `chrome.tabs.sendMessage` surface so tests can inject a fake. */
export type CaptureSuppressSendToTab = (
tabId: number,
message: CaptureSuppressMessage,
) => Promise<unknown>;
const defaultSendToTab: CaptureSuppressSendToTab = (tabId, message) =>
chrome.tabs.sendMessage(tabId, message);
/**
* Run `fn` with the target tab's overlay hidden. `end` is sent from a
* `finally` so the overlay always reappears, and a missing content script
* (or any other `begin` failure) simply skips suppression — there is no
* overlay to hide in that tab.
*/
export async function withOverlaysHiddenForCapture<T>(
tabId: number,
fn: () => Promise<T>,
sendToTab: CaptureSuppressSendToTab = defaultSendToTab,
): Promise<T> {
let began = false;
try {
await sendToTab(tabId, { type: CAPTURE_SUPPRESS, phase: "begin" });
began = true;
} catch {
// No content script in the target tab → no overlay can leak into the
// capture; proceed without suppression.
}
try {
return await fn();
} finally {
if (began) {
await sendToTab(tabId, { type: CAPTURE_SUPPRESS, phase: "end" }).catch((err) => {
console.debug("[bsk capture] overlay restore failed", err);
});
}
}
}
+8
View File
@@ -25,11 +25,19 @@ export interface RecordQueryMessage {
export interface RecordQueryResponse {
active: boolean;
requestId?: string;
/** Epoch ms when the recording began; see `RecordStartMessage.startedAtMs`. */
startedAtMs?: number;
}
export interface RecordStartMessage {
type: typeof RECORD_START;
requestId: string;
/**
* Epoch ms when the whole recording began, not when this tab was armed.
* The overlay timer must span the session, so it survives navigations and
* content-script remounts instead of restarting per page.
*/
startedAtMs?: number;
}
export interface RecordStepPayload {
@@ -39,6 +39,13 @@ describe("SessionManager", () => {
expect(ctx.refStore.isEmpty()).toBe(true);
expect(ctx.borrowedTabs.size).toBe(0);
});
it("forwards an optional window size when starting a session", async () => {
const aw = fakeAgentWindow();
const sm = new SessionManager({ agentWindow: aw });
const ctx = await sm.start("aa11", { width: 1280, height: 800 });
expect(aw.createMock).toHaveBeenCalledWith("about:blank", { width: 1280, height: 800 });
expect(ctx.agentWindowId).toBe(100);
});
it("indexes the session by sessionId and agent window id", async () => {
const aw = fakeAgentWindow();
@@ -8,7 +8,7 @@
*/
export interface AgentWindowApi {
create(url: string): Promise<number>;
create(url: string, size?: { width: number; height: number }): Promise<number>;
remove(windowId: number): Promise<void>;
/**
* Guarantee the Agent Window has an active, CDP-navigable tab.
@@ -22,11 +22,12 @@ export interface AgentWindowApi {
export const AGENT_WINDOW_HOME = "about:blank";
export const chromeAgentWindowApi: AgentWindowApi = {
async create(url: string): Promise<number> {
async create(url: string, size?: { width: number; height: number }): Promise<number> {
const win = await chrome.windows.create({
type: "normal",
focused: true,
url,
...(size ? { width: size.width, height: size.height } : {}),
});
if (typeof win?.id !== "number") {
throw new Error("[bh] chrome.windows.create returned no window id");
@@ -128,11 +128,18 @@ export class SessionManager {
* Returns the created window id so callers can echo it back to the
* daemon in the `tool.session_start` reply.
*/
async start(sessionId: string): Promise<SessionContext> {
async start(
sessionId: string,
size?: { width: number; height: number },
): Promise<SessionContext> {
if (this.sessions.has(sessionId)) {
throw new Error(`[bh] session ${sessionId} already exists`);
}
const windowId = await this.agentWindow.create(AGENT_WINDOW_HOME);
// Only pass `size` when given so the no-size call shape (and its
// chrome.windows.create payload) stays exactly as before.
const windowId = size
? await this.agentWindow.create(AGENT_WINDOW_HOME, size)
: await this.agentWindow.create(AGENT_WINDOW_HOME);
await this.agentWindow.ensureActiveTab(windowId, AGENT_WINDOW_HOME);
const ctx: SessionContext = {
sessionId,
@@ -124,6 +124,10 @@ describe("ToolDispatcher", () => {
truncated: false,
}) satisfies ConsoleResult,
),
setDeviceMetricsOverride: vi.fn(async () => {}),
clearDeviceMetricsOverride: vi.fn(async () => {}),
setUserAgentOverride: vi.fn(async () => {}),
setTouchEmulationEnabled: vi.fn(async () => {}),
};
const dispatcher = new ToolDispatcher({ transport, sessions, cdp });
dispatcher.start();
@@ -163,6 +167,10 @@ describe("ToolDispatcher", () => {
next_since: 0,
truncated: false,
})),
setDeviceMetricsOverride: vi.fn(async () => {}),
clearDeviceMetricsOverride: vi.fn(async () => {}),
setUserAgentOverride: vi.fn(async () => {}),
setTouchEmulationEnabled: vi.fn(async () => {}),
};
const dispatcher = new ToolDispatcher({ transport, sessions, cdp });
dispatcher.start();
@@ -0,0 +1,416 @@
import { beforeEach, describe, expect, it, vi } from "vitest";
import type { DeviceMetricsOverride, UserAgentOverride } from "@/browser-driver/chromium-cdp";
import { SessionManager } from "@/session-manager/manager";
import type { EmulateOverrides, RpcError } from "@/transport/types";
import {
EMULATE_SCOPE_NOTE,
type EmulateCdpRunner,
handleEmulate,
resetEmulateStatesForTests,
toCdpUserAgentMetadata,
validateOverrides,
} from "../emulate";
function fakeAgentWindow(ids: number[]) {
let i = 0;
return {
create: vi.fn(async () => {
const id = ids[i++];
if (id === undefined) throw new Error("ran out of fake ids");
return id;
}),
remove: vi.fn(async () => {}),
ensureActiveTab: vi.fn(async () => {}),
};
}
interface CdpCalls {
metrics: Array<{ tabId: number; metrics: DeviceMetricsOverride }>;
clearedMetrics: number[];
ua: Array<{ tabId: number; override: UserAgentOverride }>;
touch: Array<{ tabId: number; enabled: boolean; maxTouchPoints?: number }>;
}
function makeDeps(opts: { throwOn?: string; tabWindowId?: number } = {}) {
const calls: CdpCalls = { metrics: [], clearedMetrics: [], ua: [], touch: [] };
const maybeThrow = (kind: string) => {
if (opts.throwOn === kind) throw new Error(`simulated ${kind} failure`);
};
const cdp: EmulateCdpRunner = {
setDeviceMetricsOverride: vi.fn(async (tabId, metrics) => {
maybeThrow("metrics");
calls.metrics.push({ tabId, metrics });
}),
clearDeviceMetricsOverride: vi.fn(async (tabId) => {
maybeThrow("metrics");
calls.clearedMetrics.push(tabId);
}),
setUserAgentOverride: vi.fn(async (tabId, override) => {
maybeThrow("ua");
calls.ua.push({ tabId, override });
}),
setTouchEmulationEnabled: vi.fn(async (tabId, enabled, maxTouchPoints) => {
maybeThrow("touch");
calls.touch.push({ tabId, enabled, maxTouchPoints });
}),
trackSessionTab: vi.fn(),
};
const tabsApi = {
get: vi.fn(
async (tabId: number) =>
({ id: tabId, windowId: opts.tabWindowId ?? 100, active: true }) as chrome.tabs.Tab,
),
query: vi.fn(async () => [
{ id: 7, windowId: opts.tabWindowId ?? 100, active: true } as chrome.tabs.Tab,
]),
};
return { cdp, tabsApi, calls };
}
async function makeManager(): Promise<SessionManager> {
const sm = new SessionManager({ agentWindow: fakeAgentWindow([100]) });
await sm.start("aa11");
return sm;
}
const fullOverrides: EmulateOverrides = {
width: 390,
height: 844,
device_scale_factor: 3,
mobile: true,
user_agent: "Mozilla/5.0 (iPhone)",
accept_language: "zh-CN",
touch: true,
max_touch_points: 5,
};
describe("handleEmulate", () => {
beforeEach(() => {
resetEmulateStatesForTests();
});
it("applies viewport, UA and touch overrides to the active tab", async () => {
const sm = await makeManager();
const deps = makeDeps();
const result = await handleEmulate(sm, { session_id: "aa11", overrides: fullOverrides }, deps);
expect(result).toEqual({
tab_id: 7,
cleared: false,
applied: fullOverrides,
note: EMULATE_SCOPE_NOTE,
});
expect(deps.calls.metrics).toEqual([
{ tabId: 7, metrics: { width: 390, height: 844, deviceScaleFactor: 3, mobile: true } },
]);
expect(deps.calls.ua).toEqual([
{
tabId: 7,
override: { userAgent: "Mozilla/5.0 (iPhone)", acceptLanguage: "zh-CN" },
},
]);
expect(deps.calls.touch).toEqual([{ tabId: 7, enabled: true, maxTouchPoints: 5 }]);
expect(deps.cdp.trackSessionTab).toHaveBeenCalledWith("aa11", 7);
});
it("defaults deviceScaleFactor to 0 and mobile to false when omitted", async () => {
const sm = await makeManager();
const deps = makeDeps();
const result = await handleEmulate(
sm,
{ session_id: "aa11", overrides: { width: 800, height: 1280 } },
deps,
);
expect(result).toMatchObject({ tab_id: 7, cleared: false });
expect(deps.calls.metrics).toEqual([
{ tabId: 7, metrics: { width: 800, height: 1280, deviceScaleFactor: 0, mobile: false } },
]);
expect(deps.calls.ua).toEqual([]);
expect(deps.calls.touch).toEqual([]);
});
it("applies a UA-only override", async () => {
const sm = await makeManager();
const deps = makeDeps();
const result = await handleEmulate(
sm,
{ session_id: "aa11", overrides: { user_agent: "custom-ua" } },
deps,
);
expect(result).toMatchObject({ tab_id: 7, cleared: false });
expect(deps.calls.metrics).toEqual([]);
expect(deps.calls.ua).toEqual([{ tabId: 7, override: { userAgent: "custom-ua" } }]);
});
it("converts user_agent_metadata to CDP camelCase", async () => {
const sm = await makeManager();
const deps = makeDeps();
const result = await handleEmulate(
sm,
{
session_id: "aa11",
overrides: {
user_agent: "custom-ua",
user_agent_metadata: {
brands: [{ brand: "Chromium", version: "120" }],
full_version: "120.0.0.0",
platform: "Android",
platform_version: "14",
mobile: true,
},
},
},
deps,
);
expect(result).toMatchObject({ tab_id: 7 });
expect(deps.calls.ua[0].override.userAgentMetadata).toEqual({
brands: [{ brand: "Chromium", version: "120" }],
fullVersion: "120.0.0.0",
platform: "Android",
platformVersion: "14",
mobile: true,
});
});
it("max_touch_points without touch enables touch emulation", async () => {
const sm = await makeManager();
const deps = makeDeps();
const result = await handleEmulate(
sm,
{ session_id: "aa11", overrides: { max_touch_points: 5 } },
deps,
);
expect(result).toMatchObject({ tab_id: 7, cleared: false });
expect(deps.calls.touch).toEqual([{ tabId: 7, enabled: true, maxTouchPoints: 5 }]);
});
it("touch: false only disables touch emulation", async () => {
const sm = await makeManager();
const deps = makeDeps();
const result = await handleEmulate(
sm,
{ session_id: "aa11", overrides: { touch: false } },
deps,
);
expect(result).toMatchObject({ tab_id: 7, cleared: false });
expect(deps.calls.touch).toEqual([{ tabId: 7, enabled: false, maxTouchPoints: undefined }]);
});
it("targets an explicit tab_id", async () => {
const sm = await makeManager();
const deps = makeDeps();
const result = await handleEmulate(
sm,
{ session_id: "aa11", tab_id: 9, overrides: { user_agent: "custom-ua" } },
deps,
);
expect(result).toMatchObject({ tab_id: 9 });
expect(deps.calls.ua).toEqual([{ tabId: 9, override: { userAgent: "custom-ua" } }]);
});
it("off clears metrics, touch and UA overrides", async () => {
const sm = await makeManager();
const deps = makeDeps();
const result = await handleEmulate(sm, { session_id: "aa11", off: true }, deps);
expect(result).toEqual({ tab_id: 7, cleared: true, note: EMULATE_SCOPE_NOTE });
expect(deps.calls.clearedMetrics).toEqual([7]);
expect(deps.calls.touch).toEqual([{ tabId: 7, enabled: false, maxTouchPoints: undefined }]);
expect(deps.calls.ua).toEqual([{ tabId: 7, override: { userAgent: "" } }]);
});
it("rejects off combined with overrides", async () => {
const sm = await makeManager();
const deps = makeDeps();
const result = await handleEmulate(
sm,
{ session_id: "aa11", off: true, overrides: { touch: true } },
deps,
);
expect(result).toMatchObject({ code: "invalid_params" });
expect(deps.calls.clearedMetrics).toEqual([]);
});
it("rejects missing overrides and missing off", async () => {
const sm = await makeManager();
const deps = makeDeps();
const result = await handleEmulate(sm, { session_id: "aa11" }, deps);
expect(result).toMatchObject({ code: "invalid_params" });
});
it("rejects an unknown session", async () => {
const sm = await makeManager();
const deps = makeDeps();
const result = await handleEmulate(sm, { session_id: "zz99", off: true }, deps);
expect(result).toMatchObject({ code: "not_found" });
});
it("rejects a tab outside the Agent Window", async () => {
const sm = await makeManager();
const deps = makeDeps({ tabWindowId: 200 });
const result = await handleEmulate(sm, { session_id: "aa11", tab_id: 9, off: true }, deps);
expect(result).toMatchObject({ code: "permission_denied" });
expect(deps.calls.clearedMetrics).toEqual([]);
});
it("merges onto the tab's stored state: later width/height keep the earlier dpr/mobile", async () => {
const sm = await makeManager();
const deps = makeDeps();
await handleEmulate(sm, { session_id: "aa11", overrides: fullOverrides }, deps);
const result = await handleEmulate(
sm,
{ session_id: "aa11", overrides: { width: 500, height: 900 } },
deps,
);
expect(result).toMatchObject({
tab_id: 7,
cleared: false,
applied: { width: 500, height: 900, device_scale_factor: 3, mobile: true },
});
expect(deps.calls.metrics).toEqual([
{ tabId: 7, metrics: { width: 390, height: 844, deviceScaleFactor: 3, mobile: true } },
{ tabId: 7, metrics: { width: 500, height: 900, deviceScaleFactor: 3, mobile: true } },
]);
// The merged state is applied as a whole: UA and touch are re-applied too.
expect(deps.calls.ua).toHaveLength(2);
expect(deps.calls.touch).toHaveLength(2);
});
it("keeps emulation state per tab", async () => {
const sm = await makeManager();
const deps = makeDeps();
await handleEmulate(sm, { session_id: "aa11", overrides: fullOverrides }, deps);
await handleEmulate(
sm,
{ session_id: "aa11", tab_id: 9, overrides: { width: 500, height: 900 } },
deps,
);
// Tab 9 has no stored state, so dpr/mobile fall back to the CDP defaults.
expect(deps.calls.metrics.at(-1)).toEqual({
tabId: 9,
metrics: { width: 500, height: 900, deviceScaleFactor: 0, mobile: false },
});
});
it("off clears the stored state: a later request starts from scratch", async () => {
const sm = await makeManager();
const deps = makeDeps();
await handleEmulate(sm, { session_id: "aa11", overrides: fullOverrides }, deps);
await handleEmulate(sm, { session_id: "aa11", off: true }, deps);
const result = await handleEmulate(
sm,
{ session_id: "aa11", overrides: { width: 500, height: 900 } },
deps,
);
expect(result).toEqual({
tab_id: 7,
cleared: false,
applied: { width: 500, height: 900 },
note: EMULATE_SCOPE_NOTE,
});
// dpr/mobile fell back to the CDP defaults instead of the cleared values,
// and the cleared UA/touch were not re-applied.
expect(deps.calls.metrics.at(-1)).toEqual({
tabId: 7,
metrics: { width: 500, height: 900, deviceScaleFactor: 0, mobile: false },
});
expect(deps.calls.ua).toHaveLength(2); // preset + the clearing empty UA
expect(deps.calls.touch).toHaveLength(2); // preset + the disabling call
});
it("maps CDP failures to cdp_failed and points at --off", async () => {
const sm = await makeManager();
const deps = makeDeps({ throwOn: "ua" });
const result = await handleEmulate(
sm,
{ session_id: "aa11", overrides: { user_agent: "custom-ua" } },
deps,
);
expect(result).toMatchObject({ code: "cdp_failed" });
const message = (result as RpcError).message;
expect(message).toContain("simulated ua failure");
expect(message).toContain("not rolled back");
expect(message).toContain("--off");
});
it("reports an off-branch CDP failure with the same not-rolled-back note", async () => {
const sm = await makeManager();
const deps = makeDeps({ throwOn: "metrics" });
const result = await handleEmulate(sm, { session_id: "aa11", off: true }, deps);
expect(result).toMatchObject({ code: "cdp_failed" });
const message = (result as RpcError).message;
expect(message).toContain("simulated metrics failure");
expect(message).toContain("not rolled back");
expect(message).toContain("--off");
});
});
describe("validateOverrides", () => {
it("accepts a full override set", () => {
expect(validateOverrides(fullOverrides)).toBeNull();
});
it("rejects width without height and vice versa", () => {
expect(validateOverrides({ width: 390 })).toMatchObject({ code: "invalid_params" });
expect(validateOverrides({ height: 844 })).toMatchObject({ code: "invalid_params" });
});
it("rejects non-positive or non-integer dimensions", () => {
expect(validateOverrides({ width: 0, height: 844 })).toMatchObject({
code: "invalid_params",
});
expect(validateOverrides({ width: 390.5, height: 844 })).toMatchObject({
code: "invalid_params",
});
});
it("rejects metrics extras without dimensions", () => {
expect(validateOverrides({ device_scale_factor: 3 })).toMatchObject({
code: "invalid_params",
});
expect(validateOverrides({ mobile: true })).toMatchObject({ code: "invalid_params" });
});
it("rejects a non-positive device_scale_factor", () => {
expect(validateOverrides({ width: 390, height: 844, device_scale_factor: 0 })).toMatchObject({
code: "invalid_params",
});
expect(
validateOverrides({ width: 390, height: 844, device_scale_factor: Number.NaN }),
).toMatchObject({ code: "invalid_params" });
});
it("rejects accept_language / user_agent_metadata without user_agent", () => {
expect(validateOverrides({ accept_language: "zh-CN" })).toMatchObject({
code: "invalid_params",
});
expect(validateOverrides({ user_agent_metadata: { platform: "Android" } })).toMatchObject({
code: "invalid_params",
});
});
it("rejects invalid max_touch_points", () => {
expect(validateOverrides({ touch: true, max_touch_points: 0 })).toMatchObject({
code: "invalid_params",
});
});
it("rejects an empty override set", () => {
expect(validateOverrides({})).toMatchObject({ code: "invalid_params" });
});
});
describe("toCdpUserAgentMetadata", () => {
it("maps snake_case fields to camelCase and drops absent ones", () => {
expect(
toCdpUserAgentMetadata({
brands: [{ brand: "Chromium", version: "120" }],
platform: "Android",
mobile: true,
}),
).toEqual({
brands: [{ brand: "Chromium", version: "120" }],
platform: "Android",
mobile: true,
});
expect(toCdpUserAgentMetadata({})).toEqual({});
});
});
@@ -1,5 +1,6 @@
import { renderVom } from "@browser-skill/vom";
import { describe, expect, it, vi } from "vitest";
import { CAPTURE_SUPPRESS, type CaptureSuppressMessage } from "@/lib/capture-suppress-bridge";
import { OVERLAY_HOST_MARKER_ATTR, OVERLAY_HOST_NAME } from "@/lib/overlay-bridge";
import { SessionManager } from "@/session-manager/manager";
import type { CdpRunner } from "@/tools/shared";
@@ -345,6 +346,98 @@ describe("handleScreenshot", () => {
});
});
describe("handleScreenshot overlay suppression", () => {
/**
* Fake background→content bridge that mirrors the content script's
* contract: `begin` hides the overlay, `end` restores it. Records the
* phase order so tests can assert begin → capture → end.
*/
function fakeSuppressBridge(events: string[]) {
let overlayHidden = false;
const sendToTab = vi.fn(async (_tabId: number, message: CaptureSuppressMessage) => {
events.push(message.phase);
overlayHidden = message.phase === "begin";
return { type: CAPTURE_SUPPRESS, ok: true };
});
return { sendToTab, isHidden: () => overlayHidden };
}
it("hides the overlay around a visible-tab capture", async () => {
const sm = new SessionManager({ agentWindow: fakeAgentWindow([100]) });
await sm.start("aa11");
const events: string[] = [];
const bridge = fakeSuppressBridge(events);
const capture = vi.fn(async (_w: number) => {
events.push("capture");
// The overlay must already be hidden when the capture runs.
expect(bridge.isHidden()).toBe(true);
return `data:image/png;base64,${TINY_PNG}`;
});
const res = await handleScreenshot(
sm,
{ session_id: "aa11" },
{ ...makeScreenshotDeps({ captureVisibleTab: capture }), sendToTab: bridge.sendToTab },
);
if ("code" in res) throw new Error(`unexpected error: ${JSON.stringify(res)}`);
expect(res.image_base64).toBe(TINY_PNG);
expect(events).toEqual(["begin", "capture", "end"]);
expect(bridge.isHidden()).toBe(false);
});
it("hides the overlay around a CDP element capture", async () => {
const sm = new SessionManager({ agentWindow: fakeAgentWindow([100]) });
const ctx = await sm.start("aa11");
ctx.refStore.set("e5", 999, { tabId: 7 });
const events: string[] = [];
const bridge = fakeSuppressBridge(events);
const { cdp } = makeFakeCdp({
"DOM.scrollIntoViewIfNeeded": () => ({}),
"DOM.getContentQuads": () => ({ quads: [[10, 20, 110, 20, 110, 60, 10, 60]] }),
"Page.captureScreenshot": () => {
events.push("capture");
expect(bridge.isHidden()).toBe(true);
return { data: TINY_PNG };
},
});
const res = await handleScreenshot(
sm,
{ session_id: "aa11", ref: "@e5", tab_id: 7 },
{
...makeScreenshotDeps({
cdp,
get: vi.fn(async () => ({ id: 7, windowId: 100, active: false }) as chrome.tabs.Tab),
query: vi.fn(),
captureVisibleTab: vi.fn(),
}),
sendToTab: bridge.sendToTab,
},
);
if ("code" in res) throw new Error(`unexpected error: ${JSON.stringify(res)}`);
expect(res.image_base64).toBe(TINY_PNG);
expect(events).toEqual(["begin", "capture", "end"]);
expect(bridge.isHidden()).toBe(false);
});
it("still captures when the tab has no content script to ack", async () => {
const sm = new SessionManager({ agentWindow: fakeAgentWindow([100]) });
await sm.start("aa11");
const sendToTab = vi.fn(async () => {
throw new Error("Could not establish connection. Receiving end does not exist.");
});
const capture = vi.fn(async (_w: number) => `data:image/png;base64,${TINY_PNG}`);
const res = await handleScreenshot(
sm,
{ session_id: "aa11" },
{ ...makeScreenshotDeps({ captureVisibleTab: capture }), sendToTab },
);
if ("code" in res) throw new Error(`unexpected error: ${JSON.stringify(res)}`);
expect(res.image_base64).toBe(TINY_PNG);
expect(capture).toHaveBeenCalledTimes(1);
// The failed begin must not be followed by an end.
expect(sendToTab).toHaveBeenCalledTimes(1);
});
});
// ---------------------------------------------------------------------------
// buildVomScene
// ---------------------------------------------------------------------------
+138
View File
@@ -0,0 +1,138 @@
import { describe, expect, it, vi } from "vitest";
import { SessionManager } from "@/session-manager/manager";
import { handleSessionStart } from "../session";
import { handleWindowResize, type WindowResizeApi } from "../window";
function fakeAgentWindow(ids: number[]) {
let i = 0;
const create = vi.fn(async () => {
const id = ids[i++];
if (id === undefined) throw new Error("ran out of fake ids");
return id;
});
const remove = vi.fn(async () => {});
const ensureActiveTab = vi.fn(async () => {});
return { create, remove, ensureActiveTab };
}
async function makeManager(): Promise<SessionManager> {
const sm = new SessionManager({ agentWindow: fakeAgentWindow([100]) });
await sm.start("aa11");
return sm;
}
function fakeUpdateApi(opts?: { throw?: boolean }) {
const calls: Array<{ windowId: number; width: number; height: number }> = [];
const api: WindowResizeApi = {
update: vi.fn(async (windowId: number, updateInfo: { width: number; height: number }) => {
if (opts?.throw) throw new Error("simulated chrome failure");
calls.push({ windowId, width: updateInfo.width, height: updateInfo.height });
return undefined;
}),
};
return { api, calls };
}
describe("handleWindowResize", () => {
it("resizes the session's Agent Window", async () => {
const sm = await makeManager();
const { api, calls } = fakeUpdateApi();
const result = await handleWindowResize(
sm,
{ session_id: "aa11", width: 1280, height: 800 },
api,
);
expect(result).toEqual({ window_id: 100, width: 1280, height: 800 });
expect(calls).toEqual([{ windowId: 100, width: 1280, height: 800 }]);
});
it("rejects an unknown session", async () => {
const sm = await makeManager();
const { api } = fakeUpdateApi();
const result = await handleWindowResize(
sm,
{ session_id: "zz99", width: 1280, height: 800 },
api,
);
expect(result).toMatchObject({ code: "not_found" });
});
it("rejects missing session_id", async () => {
const sm = await makeManager();
const { api } = fakeUpdateApi();
const result = await handleWindowResize(sm, { session_id: "", width: 1280, height: 800 }, api);
expect(result).toMatchObject({ code: "invalid_params" });
});
it("rejects missing dimensions", async () => {
const sm = await makeManager();
const { api, calls } = fakeUpdateApi();
const result = await handleWindowResize(
sm,
{ session_id: "aa11" } as unknown as { session_id: string; width: number; height: number },
api,
);
expect(result).toMatchObject({ code: "invalid_params" });
expect(calls).toEqual([]);
});
it.each([
[99, 800],
[100, 7681],
[1280.5, 800],
[Number.NaN, 800],
])("rejects out-of-range or non-integer dimensions (%s, %s)", async (width, height) => {
const sm = await makeManager();
const { api, calls } = fakeUpdateApi();
const result = await handleWindowResize(sm, { session_id: "aa11", width, height }, api);
expect(result).toMatchObject({ code: "invalid_params" });
expect(calls).toEqual([]);
});
it("maps chrome API failures to protocol_error", async () => {
const sm = await makeManager();
const { api } = fakeUpdateApi({ throw: true });
const result = await handleWindowResize(
sm,
{ session_id: "aa11", width: 1280, height: 800 },
api,
);
expect(result).toMatchObject({ code: "protocol_error", message: "simulated chrome failure" });
});
});
describe("handleSessionStart window size", () => {
it("passes width/height through to Agent Window creation", async () => {
const aw = fakeAgentWindow([100]);
const sm = new SessionManager({ agentWindow: aw });
const result = await handleSessionStart(sm, { session_id: "aa11", width: 1280, height: 800 });
expect(result).toEqual({ agent_window_id: 100 });
expect(aw.create).toHaveBeenCalledWith("about:blank", { width: 1280, height: 800 });
});
it("creates the window without size when width/height are omitted", async () => {
const aw = fakeAgentWindow([100]);
const sm = new SessionManager({ agentWindow: aw });
const result = await handleSessionStart(sm, { session_id: "aa11" });
expect(result).toEqual({ agent_window_id: 100 });
expect(aw.create).toHaveBeenCalledWith("about:blank");
});
it("rejects a lone width without height", async () => {
const sm = new SessionManager({ agentWindow: fakeAgentWindow([100]) });
const result = await handleSessionStart(sm, { session_id: "aa11", width: 1280 });
expect(result).toMatchObject({ code: "invalid_params" });
expect(sm.has("aa11")).toBe(false);
});
it.each([
[99, 800],
[1280, 7681],
[1280.5, 800],
])("rejects invalid dimensions (%s, %s)", async (width, height) => {
const sm = new SessionManager({ agentWindow: fakeAgentWindow([100]) });
const result = await handleSessionStart(sm, { session_id: "aa11", width, height });
expect(result).toMatchObject({ code: "invalid_params" });
expect(sm.has("aa11")).toBe(false);
});
});
+15 -1
View File
@@ -4,6 +4,7 @@ import type { Transport } from "@/transport/transport";
import type {
ClickParams,
ConsoleParams,
EmulateParams,
EvaluateParams,
FillParams,
GetHtmlParams,
@@ -29,6 +30,7 @@ import type {
} from "@/transport/types";
import { isRequestFrame } from "@/transport/types";
import { handleConsole } from "./console";
import { type EmulateCdpRunner, handleEmulate } from "./emulate";
import { handleEvaluate } from "./evaluate";
import { handleRequestHelp } from "./human-loop";
import { handleClick, handleFill, handlePress, handleSelect } from "./interaction";
@@ -71,9 +73,11 @@ import {
type TabSelectParams,
} from "./tabs";
import { handleWaitForNavigation } from "./waits";
import { handleWindowResize, type WindowResizeParams } from "./window";
type DispatcherCdpRunner = CdpRunner &
NetworkCdpRunner & {
NetworkCdpRunner &
EmulateCdpRunner & {
detachSession(sessionId: string): Promise<void>;
};
@@ -286,6 +290,14 @@ export class ToolDispatcher {
});
case "tool.tab_return":
return handleTabReturn(this.sessions, req.params as TabReturnParams);
case "tool.window_resize":
return handleWindowResize(this.sessions, req.params as WindowResizeParams);
case "tool.emulate":
return handleEmulate(
this.sessions,
req.params as EmulateParams,
this.cdp ? { cdp: this.cdp, tabsApi: chromeTabsApi } : undefined,
);
case "tool.screenshot":
return handleScreenshot(
this.sessions,
@@ -478,6 +490,8 @@ function sessionIdForBrowserControlMethod(req: RequestFrame): string | null {
case "tool.tab_select":
case "tool.tab_borrow":
case "tool.tab_return":
case "tool.window_resize":
case "tool.emulate":
case "tool.navigate":
case "tool.navigate_back":
case "tool.navigate_forward":
+285
View File
@@ -0,0 +1,285 @@
// `tool.emulate` — apply or clear CDP Emulation-domain overrides
// (viewport metrics, user agent, touch) on a session tab, so agents can
// debug mobile page behaviour. Overrides are per-target: new tabs do
// not inherit them, and `off` restores the tab's real environment.
//
// Device presets are resolved CLI-side; this handler only executes the
// concrete override parameters it receives. Each request is merged
// field by field onto the tab's remembered emulation state and the
// merged state is applied as a whole, so a partial request does not
// reset fields set by an earlier one.
import type { DeviceMetricsOverride, UserAgentOverride } from "@/browser-driver/chromium-cdp";
import { ChromiumCdp } from "@/browser-driver/chromium-cdp";
import type { SessionManager } from "@/session-manager/manager";
import type {
EmulateOverrides,
EmulateParams,
EmulateResult,
RpcError,
UserAgentMetadata,
} from "@/transport/types";
import {
type ChromeTabsApi,
chromeTabsApi,
enforceAgentWindow,
isRpcError,
lookupSession,
resolveTargetTab,
} from "./shared";
/**
* Emulation-domain surface the handler needs. Backed by `ChromiumCdp`
* in production; tests inject a fake. `trackSessionTab` is optional so
* test doubles need not implement it.
*/
export interface EmulateCdpRunner {
setDeviceMetricsOverride(tabId: number, metrics: DeviceMetricsOverride): Promise<void>;
clearDeviceMetricsOverride(tabId: number): Promise<void>;
setUserAgentOverride(tabId: number, override: UserAgentOverride): Promise<void>;
setTouchEmulationEnabled(tabId: number, enabled: boolean, maxTouchPoints?: number): Promise<void>;
trackSessionTab?(sessionId: string, tabId: number): void;
}
export interface EmulateDeps {
cdp: EmulateCdpRunner;
tabsApi: ChromeTabsApi;
}
function defaultEmulateDeps(): EmulateDeps {
return {
cdp: new ChromiumCdp(),
tabsApi: chromeTabsApi,
};
}
/**
* Result note explaining the per-target scope of CDP emulation
* overrides, echoed on every successful call so agents know to
* re-apply emulation after opening a new tab.
*/
export const EMULATE_SCOPE_NOTE =
"emulation overrides are per-tab (CDP target): new tabs do not inherit them — re-run `bsk emulate` on each new tab";
/** Convert the wire (snake_case) UA metadata to CDP's camelCase shape. */
export function toCdpUserAgentMetadata(meta: UserAgentMetadata): Record<string, unknown> {
const out: Record<string, unknown> = {};
if (meta.brands !== undefined) out.brands = meta.brands;
if (meta.full_version !== undefined) out.fullVersion = meta.full_version;
if (meta.platform !== undefined) out.platform = meta.platform;
if (meta.platform_version !== undefined) out.platformVersion = meta.platform_version;
if (meta.architecture !== undefined) out.architecture = meta.architecture;
if (meta.model !== undefined) out.model = meta.model;
if (meta.mobile !== undefined) out.mobile = meta.mobile;
return out;
}
function invalidParams(message: string): RpcError {
return { code: "invalid_params", message };
}
/**
* Per-tab record of the emulation state applied so far. New requests
* are merged onto it field by field (see `mergeEmulateOverrides`) and
* the merged state is applied as a whole, so e.g. a later
* `--width/--height` does not silently reset the dpr/mobile of an
* earlier `--device` preset.
*
* The record is best-effort: an entry left behind by a closed tab is
* simply overwritten by the next emulate call that targets the same tab
* id, and the whole map is lost when the extension service worker
* reloads — after a reload the next emulate is equivalent to a full
* (re)set of just the fields it carries.
*/
const tabEmulationStates = new Map<number, EmulateOverrides>();
/** Test hook: drop every per-tab emulation state record. */
export function resetEmulateStatesForTests(): void {
tabEmulationStates.clear();
}
/**
* Merge one validated request onto the tab's stored state: fields
* present in `overrides` overwrite the stored value, absent fields keep
* it. Returns a fresh object; `stored` is not mutated.
*/
export function mergeEmulateOverrides(
stored: EmulateOverrides | undefined,
overrides: EmulateOverrides,
): EmulateOverrides {
const merged: EmulateOverrides = { ...stored };
if (overrides.width !== undefined) merged.width = overrides.width;
if (overrides.height !== undefined) merged.height = overrides.height;
if (overrides.device_scale_factor !== undefined) {
merged.device_scale_factor = overrides.device_scale_factor;
}
if (overrides.mobile !== undefined) merged.mobile = overrides.mobile;
if (overrides.user_agent !== undefined) merged.user_agent = overrides.user_agent;
if (overrides.accept_language !== undefined) {
merged.accept_language = overrides.accept_language;
}
if (overrides.user_agent_metadata !== undefined) {
merged.user_agent_metadata = overrides.user_agent_metadata;
}
if (overrides.touch !== undefined) merged.touch = overrides.touch;
if (overrides.max_touch_points !== undefined) {
merged.max_touch_points = overrides.max_touch_points;
}
return merged;
}
/**
* CDP calls run in sequence without rollback: on a mid-way failure the
* overrides already applied stay in effect, so the error says how to
* reset them.
*/
function cdpFailed(err: unknown): RpcError {
const cause = err instanceof Error ? err.message : String(err);
return {
code: "cdp_failed",
message: `${cause} (overrides applied before the failure were not rolled back — use --off to reset)`,
};
}
/**
* Cross-field validation for one overrides set. The CLI performs the
* same checks; the extension re-validates because the wire format is
* public.
*/
export function validateOverrides(overrides: EmulateOverrides): RpcError | null {
const hasWidth = overrides.width !== undefined;
const hasHeight = overrides.height !== undefined;
if (hasWidth !== hasHeight) {
return invalidParams("emulate overrides require width and height together");
}
if (hasWidth) {
if (!Number.isSafeInteger(overrides.width) || (overrides.width as number) <= 0) {
return invalidParams("width must be a positive integer");
}
if (!Number.isSafeInteger(overrides.height) || (overrides.height as number) <= 0) {
return invalidParams("height must be a positive integer");
}
}
if (
(overrides.device_scale_factor !== undefined || overrides.mobile !== undefined) &&
!hasWidth
) {
return invalidParams("device_scale_factor and mobile require width and height");
}
if (
overrides.device_scale_factor !== undefined &&
(!Number.isFinite(overrides.device_scale_factor) || overrides.device_scale_factor <= 0)
) {
return invalidParams("device_scale_factor must be a positive finite number");
}
if (
(overrides.accept_language !== undefined || overrides.user_agent_metadata !== undefined) &&
overrides.user_agent === undefined
) {
return invalidParams("accept_language and user_agent_metadata require user_agent");
}
if (
overrides.max_touch_points !== undefined &&
(!Number.isSafeInteger(overrides.max_touch_points) || overrides.max_touch_points < 1)
) {
return invalidParams("max_touch_points must be a positive integer");
}
const anyOverride =
hasWidth ||
overrides.user_agent !== undefined ||
overrides.touch !== undefined ||
overrides.max_touch_points !== undefined;
if (!anyOverride) {
return invalidParams(
"emulate requires at least one override (viewport, user agent, or touch) or off: true",
);
}
return null;
}
/**
* Handler for `tool.emulate` (called by the daemon over WS).
*
* Merges the requested overrides onto the tab's remembered state and
* applies the merged state in a fixed order — viewport metrics, then
* UA, then touch — or clears everything (state included) when `off` is
* set. Raw CDP failures surface as `cdp_failed` (§4.5).
*/
export async function handleEmulate(
manager: SessionManager,
params: EmulateParams,
deps: EmulateDeps = defaultEmulateDeps(),
): Promise<EmulateResult | RpcError> {
const ctxOrErr = lookupSession(manager, params, "emulate");
if (isRpcError(ctxOrErr)) return ctxOrErr;
const ctx = ctxOrErr;
const target = await resolveTargetTab(manager, ctx, params?.tab_id, deps.tabsApi);
if (isRpcError(target)) return target;
const denied = enforceAgentWindow(ctx, target, "emulate");
if (denied) return denied;
if (params.off === true) {
if (params.overrides !== undefined) {
return invalidParams("emulate off cannot be combined with overrides");
}
try {
deps.cdp.trackSessionTab?.(ctx.sessionId, target.tabId);
await deps.cdp.clearDeviceMetricsOverride(target.tabId);
await deps.cdp.setTouchEmulationEnabled(target.tabId, false);
// An empty userAgent string clears the override (CDP convention).
await deps.cdp.setUserAgentOverride(target.tabId, { userAgent: "" });
} catch (err) {
return cdpFailed(err);
}
tabEmulationStates.delete(target.tabId);
return { tab_id: target.tabId, cleared: true, note: EMULATE_SCOPE_NOTE };
}
const overrides = params.overrides;
if (!overrides) {
return invalidParams("emulate requires overrides or off: true");
}
const invalid = validateOverrides(overrides);
if (invalid) return invalid;
// Fields absent from this request keep their previously applied
// values; the merged state is what gets applied (and echoed back).
const merged = mergeEmulateOverrides(tabEmulationStates.get(target.tabId), overrides);
try {
deps.cdp.trackSessionTab?.(ctx.sessionId, target.tabId);
if (merged.width !== undefined && merged.height !== undefined) {
await deps.cdp.setDeviceMetricsOverride(target.tabId, {
width: merged.width,
height: merged.height,
deviceScaleFactor: merged.device_scale_factor ?? 0,
mobile: merged.mobile ?? false,
});
}
if (merged.user_agent !== undefined) {
await deps.cdp.setUserAgentOverride(target.tabId, {
userAgent: merged.user_agent,
...(merged.accept_language !== undefined ? { acceptLanguage: merged.accept_language } : {}),
...(merged.user_agent_metadata !== undefined
? { userAgentMetadata: toCdpUserAgentMetadata(merged.user_agent_metadata) }
: {}),
});
}
if (merged.touch !== undefined || merged.max_touch_points !== undefined) {
await deps.cdp.setTouchEmulationEnabled(
target.tabId,
merged.touch ?? true,
merged.max_touch_points,
);
}
} catch (err) {
return cdpFailed(err);
}
// Record the merged state only once it was fully applied.
tabEmulationStates.set(target.tabId, merged);
return {
tab_id: target.tabId,
cleared: false,
applied: merged,
note: EMULATE_SCOPE_NOTE,
};
}
+21 -2
View File
@@ -11,6 +11,10 @@ import {
type VomScene,
} from "@browser-skill/vom";
import { ChromiumCdp } from "@/browser-driver/chromium-cdp";
import {
type CaptureSuppressSendToTab,
withOverlaysHiddenForCapture,
} from "@/lib/capture-suppress-bridge";
import type { SessionManager } from "@/session-manager/manager";
import type {
GetHtmlParams,
@@ -114,6 +118,12 @@ export interface ScreenshotDeps {
cdp?: SharedCdpRunner;
tabsApi: ChromeTabsApi;
captureApi: ChromeTabsCaptureApi;
/**
* Bridge used to hide the in-page overlay while a screenshot is taken,
* so captured frames only contain page content. Defaults to
* `chrome.tabs.sendMessage`; tests inject a fake.
*/
sendToTab?: CaptureSuppressSendToTab;
}
function defaultScreenshotDeps(): ScreenshotDeps {
@@ -192,7 +202,12 @@ export async function handleScreenshot(
if (isRpcError(node)) return node;
deps.cdp.trackSessionTab?.(ctx.sessionId, target.tabId);
await deps.cdp.ensureAttachedToUrl?.(target.tabId, target.url);
const captured = await captureElementScreenshot(deps.cdp, target.tabId, node.backendNodeId);
const cdp = deps.cdp;
const captured = await withOverlaysHiddenForCapture(
target.tabId,
() => captureElementScreenshot(cdp, target.tabId, node.backendNodeId),
deps.sendToTab,
);
if (isRpcError(captured)) return captured;
return withShotDialogs({
image_base64: captured.image_base64,
@@ -212,7 +227,11 @@ export async function handleScreenshot(
}
try {
const dataUrl = await deps.captureApi.captureVisibleTab(target.windowId, { format: "png" });
const dataUrl = await withOverlaysHiddenForCapture(
target.tabId,
() => deps.captureApi.captureVisibleTab(target.windowId, { format: "png" }),
deps.sendToTab,
);
const image_base64 = stripDataUrlPrefix(dataUrl);
const dims = parsePngDimensions(image_base64) ?? { width: 0, height: 0 };
return withShotDialogs({
+15 -4
View File
@@ -49,6 +49,7 @@ interface ActiveRecording {
purpose?: string;
steps: DraftTraceStep[];
startedAt: string;
startedAtMs: number;
finishPromise: Promise<Trace>;
resolveFinish: (trace: Trace) => void;
rejectFinish: (err: Error) => void;
@@ -373,7 +374,11 @@ async function rearmRecording(
// stuck with pointer-events:none (page usable, Interrupt dead). RecordOverlay
// already hides the control chrome while activeRecord is set.
for (let attempt = 0; attempt < RECORD_REARM_MAX_ATTEMPTS; attempt += 1) {
const startMsg: RecordStartMessage = { type: RECORD_START, requestId: recording.requestId };
const startMsg: RecordStartMessage = {
type: RECORD_START,
requestId: recording.requestId,
startedAtMs: recording.startedAtMs,
};
try {
await sendRecordStartWithAck(targetTabId, startMsg, deps.sendToTab);
recording.tabId = targetTabId;
@@ -501,7 +506,11 @@ export function attachRecordQueryListener(deps: RecordDeps = getDefaultDeps()):
return;
}
await rearmRecording(recording, tabId, deps);
sendResponse({ active: true, requestId: recording.requestId });
sendResponse({
active: true,
requestId: recording.requestId,
startedAtMs: recording.startedAtMs,
});
})();
return true;
};
@@ -572,6 +581,7 @@ export async function handleRecordStart(
rejectFinish = reject;
});
const navigateUrl = params.url ?? RECORD_DEFAULT_START_URL;
const startedAtMs = Date.now();
recordings.set(params.session_id, {
requestId,
tabId: target.tabId,
@@ -579,7 +589,8 @@ export async function handleRecordStart(
startUrl: navigateUrl,
...(params.purpose ? { purpose: params.purpose } : {}),
steps: [],
startedAt: new Date().toISOString(),
startedAt: new Date(startedAtMs).toISOString(),
startedAtMs,
finishPromise,
resolveFinish,
rejectFinish,
@@ -707,7 +718,7 @@ export async function handleRecordStart(
if (cancelled) return cancelled;
}
const startMsg: RecordStartMessage = { type: RECORD_START, requestId };
const startMsg: RecordStartMessage = { type: RECORD_START, requestId, startedAtMs };
try {
await sendRecordStartWithAck(target.tabId, startMsg, deps.sendToTab);
+47 -1
View File
@@ -1,11 +1,55 @@
import type { SessionManager } from "@/session-manager/manager";
import type { RpcError } from "@/transport/types";
import { clearRecordingForSession } from "./record";
import { isRpcError } from "./shared";
import { returnBorrowedTab, type TabManagementDeps } from "./tabs";
/** Valid range for Agent Window dimensions in CSS pixels. */
export const WINDOW_SIZE_MIN = 100;
export const WINDOW_SIZE_MAX = 7680;
/**
* Validate an optional width/height pair. Both must be given (or
* neither), be integers, and fall within 100..=7680. Returns the size
* to pass down, or an `invalid_params` RpcError.
*/
export function validateWindowSize(
width: unknown,
height: unknown,
): { width: number; height: number } | undefined | RpcError {
if (width === undefined && height === undefined) return undefined;
if (width === undefined || height === undefined) {
return {
code: "invalid_params",
message: "width and height must be given together",
};
}
for (const [name, value] of [
["width", width],
["height", height],
] as const) {
if (
typeof value !== "number" ||
!Number.isInteger(value) ||
value < WINDOW_SIZE_MIN ||
value > WINDOW_SIZE_MAX
) {
return {
code: "invalid_params",
message: `${name} must be an integer in 100..=7680`,
};
}
}
return { width: width as number, height: height as number };
}
export interface SessionStartParams {
session_id: string;
browser_instance_id?: string;
/** Optional Agent Window outer width in CSS pixels (100..=7680). */
width?: number;
/** Optional Agent Window outer height in CSS pixels (100..=7680). */
height?: number;
}
export interface SessionStartResult {
@@ -52,8 +96,10 @@ export async function handleSessionStart(
message: "session.start requires session_id",
};
}
const sizeOrErr = validateWindowSize(params.width, params.height);
if (isRpcError(sizeOrErr)) return sizeOrErr;
try {
const ctx = await manager.start(params.session_id);
const ctx = await manager.start(params.session_id, sizeOrErr);
return { agent_window_id: ctx.agentWindowId };
} catch (err) {
// chrome.windows.create / SessionManager failures are not CDP
+82
View File
@@ -0,0 +1,82 @@
// Window-tool handlers: `tool.window_resize` resizes the session's
// Agent Window via `chrome.windows.update`.
import type { SessionManager } from "@/session-manager/manager";
import type { RpcError } from "@/transport/types";
import { validateWindowSize } from "./session";
import { isRpcError, lookupSession } from "./shared";
/**
* Mirror of bsk-protocol `WindowResizeParams` /
* `WindowResizeResult` (see crates/bsk-protocol/src/tools/window.rs).
*/
export interface WindowResizeParams {
session_id: string;
width: number;
height: number;
}
export interface WindowResizeResult {
window_id: number;
width: number;
height: number;
}
/**
* Subset of `chrome.windows` we depend on, injectable so unit tests
* can fake the browser without monkey-patching the global `chrome`.
*/
export interface WindowResizeApi {
update(windowId: number, updateInfo: { width: number; height: number }): Promise<unknown>;
}
export const chromeWindowResizeApi: WindowResizeApi = {
async update(windowId, updateInfo) {
return chrome.windows.update(windowId, updateInfo);
},
};
/**
* Handler for `tool.window_resize` (called by the daemon over WS).
*
* Resizes the session's Agent Window to the given outer dimensions in
* CSS pixels. chrome API failures are surfaced as `protocol_error`
* (§4.5 reserves `cdp_failed` for raw CDP errors).
*/
export async function handleWindowResize(
manager: SessionManager,
params: WindowResizeParams,
api: WindowResizeApi = chromeWindowResizeApi,
): Promise<WindowResizeResult | RpcError> {
const ctxOrErr = lookupSession(manager, params, "window_resize");
if (isRpcError(ctxOrErr)) return ctxOrErr;
const ctx = ctxOrErr;
const sizeOrErr = validateWindowSize(params.width, params.height);
if (isRpcError(sizeOrErr)) return sizeOrErr;
if (!sizeOrErr) {
// validateWindowSize only returns undefined when both are absent,
// which is valid for session_start but not for an explicit resize.
return {
code: "invalid_params",
message: "window_resize requires width and height",
};
}
try {
await api.update(ctx.agentWindowId, {
width: sizeOrErr.width,
height: sizeOrErr.height,
});
} catch (err) {
return {
code: "protocol_error",
message: err instanceof Error ? err.message : String(err),
};
}
return {
window_id: ctx.agentWindowId,
width: sizeOrErr.width,
height: sizeOrErr.height,
};
}
+64 -1
View File
@@ -518,7 +518,13 @@ export interface HelpTarget {
selector?: string;
}
export type HelpOutcome = "continued" | "cancelled" | "timed_out" | "completed" | "navigated";
export type HelpOutcome =
| "continued"
| "cancelled"
| "timed_out"
| "completed"
| "navigated"
| "disabled";
export interface HelpCompletionCondition {
url_contains?: string;
@@ -559,6 +565,63 @@ export interface RequestHelpResult {
resolved_targets?: ResolvedTarget[];
}
// --------------------------------------------------------------------------
// Device-emulation payloads — tool.emulate (mirror bsk-protocol emulate.rs)
// --------------------------------------------------------------------------
export interface UserAgentBrandVersion {
brand: string;
version: string;
}
/** Mirror of CDP `Emulation.UserAgentMetadata`; all fields optional. */
export interface UserAgentMetadata {
brands?: UserAgentBrandVersion[];
full_version?: string;
platform?: string;
platform_version?: string;
architecture?: string;
model?: string;
mobile?: boolean;
}
/**
* Concrete emulation overrides for one tab. The extension merges each
* request field by field onto the tab's remembered emulation state:
* fields present here overwrite the stored value, absent fields keep
* it, and the merged state is applied as a whole.
*/
export interface EmulateOverrides {
width?: number;
height?: number;
device_scale_factor?: number;
mobile?: boolean;
user_agent?: string;
accept_language?: string;
user_agent_metadata?: UserAgentMetadata;
touch?: boolean;
max_touch_points?: number;
}
export interface EmulateParams {
session_id: string;
tab_id?: number;
/** Clear every emulation override on the tab. Exclusive with `overrides`. */
off?: boolean;
/** Overrides to apply. Required unless `off` is set. */
overrides?: EmulateOverrides;
}
export interface EmulateResult {
tab_id: number;
/** True when overrides were cleared (`off`); false when applied. */
cleared: boolean;
/** Echo of the overrides that were applied. Absent when cleared. */
applied?: EmulateOverrides;
/** Scope note: overrides are per-tab (CDP target), not inherited by new tabs. */
note?: string;
}
// --------------------------------------------------------------------------
// Semantic record payloads — mirror bsk-protocol record.rs (Trace v2)
// --------------------------------------------------------------------------
+28 -1
View File
@@ -123,11 +123,33 @@ Details and flags: **`bsk <cmd> --help`**
| Command | Summary |
|---------|---------|
| `bsk session start` | Open Agent Window; prints **4-letter session id** |
| `bsk session start` | Open Agent Window (`--width`/`--height` for initial size); prints **4-letter session id** |
| `bsk session stop <id>` | End session, close Agent Window, auto-return borrowed tabs |
| `bsk session stop --all` | Stop every active session |
| `bsk session list` | List active sessions |
### Window (require `--session <id>`)
| Command | Summary |
|---------|---------|
| `bsk window resize` | Resize the Agent Window (`--width`, `--height`; 100..=7680 CSS px) |
### Device emulation — `bsk emulate` (requires `--session <id>`)
Emulate a mobile device environment on the agent tab via CDP — viewport, User-Agent, and touch — to debug a page's mobile layout and behaviour:
```bash
bsk emulate --session <id> --device iphone-14
bsk emulate --session <id> --width 390 --height 844 --dpr 3 --mobile --ua "Mozilla/5.0 (iPhone…" --touch
bsk emulate --session <id> --off
```
- Presets (`--device`): `iphone-14`, `iphone-14-pro-max`, `iphone-se`, `pixel-7`, `galaxy-s23`, `ipad-mini`, `galaxy-tab-s8`. Manual flags (`--width`/`--height`/`--dpr`/`--mobile`/`--ua`/`--accept-language`/`--touch`/`--max-touch-points`) also work without a preset, or override individual preset fields; `--no-mobile`/`--no-touch` turn a preset's mobile viewport / touch emulation back off.
- Repeated runs merge field by field onto the tab's current emulation state — only the flags you pass change. E.g. after `--device iphone-14`, `bsk emulate --session <id> --width 390 --height 844` keeps the preset's dpr (`3`) and mobile viewport. (The extension remembers the state per tab; after an extension reload the next run applies only the fields it carries.)
- `--off` clears every override (viewport, UA, touch) and the remembered state, restoring the tab's real environment.
- Scope: overrides apply to **one tab only** (CDP per-target) and are **not inherited by new tabs** — re-run `bsk emulate` after opening or switching to another tab (default target is the session's active tab; `--tab-id` overrides).
- Emulation covers viewport/UA/touch only; it does not throttle the network, fake geolocation, or synthesise real touch-event streams.
### Tabs (require `--session <id>`)
| Command | Summary |
@@ -219,6 +241,11 @@ which refs/selectors matched a live element.
control. After a `continued` or `completed` result, issue a separate
observation tool call (usually `bsk snapshot --session <id>`) before using
new refs or reasoning about the post-help page state.
#### Disabling request-help (unattended mode)
Set `BSK_REQUEST_HELP=off` on unattended servers: `bsk request-help` then
returns immediately with `outcome="disabled"` (no overlay, no waiting,
exit 0). Any other value keeps it enabled. If you get `disabled`, do not
retry — complete the task autonomously or stop gracefully.
### Recording — `bsk record`
+728
View File
@@ -0,0 +1,728 @@
//! `bsk emulate` — emulate a mobile device environment on a tab via the
//! CDP Emulation domain (viewport metrics, user agent, touch), for
//! mobile page debugging.
//!
//! Device presets are resolved here, CLI-side: the extension only
//! executes the concrete override parameters it receives, so presets are
//! not duplicated in TypeScript.
use std::path::PathBuf;
use anyhow::Context;
use bsk_protocol::tools::{EmulateOverrides, EmulateParams, EmulateResult};
use bsk_protocol::{ErrorCode, Method, RpcError};
use clap::Args;
use crate::cli::TOOL_IPC_TIMEOUT;
use crate::cli::ensure_daemon::ensure_daemon;
use crate::cli::error::{CliError, Format};
/// One built-in device preset: viewport metrics + UA + touch applied in
/// a single `--device` invocation.
#[derive(Debug, Clone, Copy, PartialEq)]
pub struct DevicePreset {
pub name: &'static str,
pub width: u32,
pub height: u32,
pub device_scale_factor: f64,
pub mobile: bool,
pub user_agent: &'static str,
pub max_touch_points: u32,
}
const IPHONE_UA: &str = "Mozilla/5.0 (iPhone; CPU iPhone OS 17_0 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.0 Mobile/15E148 Safari/604.1";
const IPAD_UA: &str = "Mozilla/5.0 (iPad; CPU OS 17_0 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.0 Mobile/15E148 Safari/604.1";
/// Built-in device presets for `--device`. Keep names lowercase-kebab;
/// they are the public CLI identifiers.
pub const DEVICE_PRESETS: &[DevicePreset] = &[
DevicePreset {
name: "iphone-14",
width: 390,
height: 844,
device_scale_factor: 3.0,
mobile: true,
user_agent: IPHONE_UA,
max_touch_points: 5,
},
DevicePreset {
name: "iphone-14-pro-max",
width: 430,
height: 932,
device_scale_factor: 3.0,
mobile: true,
user_agent: IPHONE_UA,
max_touch_points: 5,
},
DevicePreset {
name: "iphone-se",
width: 375,
height: 667,
device_scale_factor: 2.0,
mobile: true,
user_agent: IPHONE_UA,
max_touch_points: 5,
},
DevicePreset {
name: "pixel-7",
width: 412,
height: 915,
device_scale_factor: 2.625,
mobile: true,
user_agent: "Mozilla/5.0 (Linux; Android 14; Pixel 7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.43 Mobile Safari/537.36",
max_touch_points: 5,
},
DevicePreset {
name: "galaxy-s23",
width: 360,
height: 780,
device_scale_factor: 3.0,
mobile: true,
user_agent: "Mozilla/5.0 (Linux; Android 14; SM-S911B) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.43 Mobile Safari/537.36",
max_touch_points: 5,
},
DevicePreset {
name: "ipad-mini",
width: 744,
height: 1133,
device_scale_factor: 2.0,
mobile: true,
user_agent: IPAD_UA,
max_touch_points: 5,
},
DevicePreset {
name: "galaxy-tab-s8",
width: 800,
height: 1280,
device_scale_factor: 2.0,
mobile: true,
user_agent: "Mozilla/5.0 (Linux; Android 14; SM-X700) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.6099.43 Safari/537.36",
max_touch_points: 5,
},
];
/// Comma-separated preset names for error messages.
pub fn preset_names() -> String {
DEVICE_PRESETS
.iter()
.map(|p| p.name)
.collect::<Vec<_>>()
.join(", ")
}
/// Look up a preset by its CLI name.
pub fn find_preset(name: &str) -> Option<&'static DevicePreset> {
DEVICE_PRESETS.iter().find(|p| p.name == name)
}
#[derive(Debug, Clone, Args)]
pub struct EmulateArgs {
/// Session id (must be active).
#[arg(long)]
pub session: String,
/// Target tab. Defaults to the Agent Window's active tab.
#[arg(long = "tab-id")]
pub tab_id: Option<i64>,
/// Built-in device preset (iphone-14, iphone-14-pro-max, iphone-se,
/// pixel-7, galaxy-s23, ipad-mini, galaxy-tab-s8). Manual flags
/// override individual preset fields.
#[arg(long)]
pub device: Option<String>,
/// Viewport width in CSS pixels (requires --height unless --device
/// supplies it).
#[arg(long, value_parser = viewport_dimension)]
pub width: Option<u32>,
/// Viewport height in CSS pixels (requires --width unless --device
/// supplies it).
#[arg(long, value_parser = viewport_dimension)]
pub height: Option<u32>,
/// Device pixel ratio (requires --width/--height without --device).
#[arg(long, value_parser = device_pixel_ratio)]
pub dpr: Option<f64>,
/// Emulate a mobile viewport (requires --width/--height without
/// --device).
#[arg(long)]
pub mobile: bool,
/// Turn a preset's mobile viewport flag back off.
#[arg(long, conflicts_with = "mobile")]
pub no_mobile: bool,
/// User-Agent override string.
#[arg(long)]
pub ua: Option<String>,
/// Accept-Language header override (requires --ua or --device).
#[arg(long = "accept-language")]
pub accept_language: Option<String>,
/// Enable touch emulation (combines with --device or manual flags).
#[arg(long)]
pub touch: bool,
/// Turn a preset's touch emulation back off.
#[arg(long, conflicts_with = "touch")]
pub no_touch: bool,
/// Touch points reported while touch emulation is enabled (implies
/// --touch).
#[arg(long = "max-touch-points", value_parser = touch_points)]
pub max_touch_points: Option<u32>,
/// Clear all emulation overrides and restore the tab's real
/// environment. Mutually exclusive with every other option.
#[arg(long)]
pub off: bool,
}
/// Parse a `--width` / `--height` viewport dimension (CSS pixels).
fn viewport_dimension(s: &str) -> Result<u32, String> {
let value: u32 = s
.parse()
.map_err(|_| format!("invalid viewport dimension {s:?}: expected a positive integer"))?;
if (1..=7680).contains(&value) {
Ok(value)
} else {
Err(format!(
"viewport dimension {value} out of range (1..=7680)"
))
}
}
/// Parse a `--dpr` device pixel ratio.
fn device_pixel_ratio(s: &str) -> Result<f64, String> {
let value: f64 = s
.parse()
.map_err(|_| format!("invalid device pixel ratio {s:?}: expected a number"))?;
if value.is_finite() && value > 0.0 {
Ok(value)
} else {
Err(format!(
"device pixel ratio {value} out of range (must be positive and finite)"
))
}
}
/// Parse a `--max-touch-points` value.
fn touch_points(s: &str) -> Result<u32, String> {
let value: u32 = s
.parse()
.map_err(|_| format!("invalid max touch points {s:?}: expected a positive integer"))?;
if value >= 1 {
Ok(value)
} else {
Err("max touch points must be at least 1".to_string())
}
}
fn invalid_params(message: &str) -> CliError {
CliError::from_rpc(RpcError {
code: ErrorCode::InvalidParams,
message: message.to_string(),
data: None,
})
}
/// Validate the argument combination and build the wire params. Pure so
/// it can be unit-tested without a daemon.
///
/// The preset (if any) forms the base, manual flags override individual
/// fields, and only the merged result is validated — so a preset can
/// supply the dimension a lone `--width`/`--height` lacks.
pub fn build_params(args: &EmulateArgs) -> Result<EmulateParams, CliError> {
let has_manual = args.width.is_some()
|| args.height.is_some()
|| args.dpr.is_some()
|| args.mobile
|| args.no_mobile
|| args.ua.is_some()
|| args.accept_language.is_some()
|| args.touch
|| args.no_touch
|| args.max_touch_points.is_some();
if args.off {
if args.device.is_some() || has_manual {
return Err(invalid_params(
"--off cannot be combined with --device or manual emulation options",
));
}
return Ok(EmulateParams {
session_id: args.session.clone(),
tab_id: args.tab_id,
off: Some(true),
overrides: None,
});
}
if args.device.is_none() && !has_manual {
return Err(invalid_params(&format!(
"nothing to do: pass --device <preset> ({}), manual options (--width/--height/--dpr/--mobile/--ua/--touch), or --off",
preset_names()
)));
}
let mut overrides = match args.device.as_deref() {
Some(name) => {
let preset = find_preset(name).ok_or_else(|| {
invalid_params(&format!(
"unknown device preset {name:?} (available: {})",
preset_names()
))
})?;
EmulateOverrides {
width: Some(preset.width),
height: Some(preset.height),
device_scale_factor: Some(preset.device_scale_factor),
mobile: Some(preset.mobile),
user_agent: Some(preset.user_agent.to_string()),
accept_language: None,
user_agent_metadata: None,
touch: Some(true),
max_touch_points: Some(preset.max_touch_points),
}
}
None => EmulateOverrides::default(),
};
if let Some(w) = args.width {
overrides.width = Some(w);
}
if let Some(h) = args.height {
overrides.height = Some(h);
}
if let Some(dpr) = args.dpr {
overrides.device_scale_factor = Some(dpr);
}
if args.mobile {
overrides.mobile = Some(true);
}
if args.no_mobile {
overrides.mobile = Some(false);
}
if let Some(ua) = &args.ua {
overrides.user_agent = Some(ua.clone());
}
if let Some(accept_language) = &args.accept_language {
overrides.accept_language = Some(accept_language.clone());
}
if args.touch || args.max_touch_points.is_some() {
overrides.touch = Some(true);
}
if args.no_touch {
overrides.touch = Some(false);
}
if let Some(max_touch_points) = args.max_touch_points {
overrides.max_touch_points = Some(max_touch_points);
}
// Cross-field checks run on the merged result.
match (overrides.width, overrides.height) {
(Some(_), None) | (None, Some(_)) => {
return Err(invalid_params(
"--width and --height must be provided together",
));
}
_ => {}
}
if overrides.width.is_none()
&& (overrides.device_scale_factor.is_some() || overrides.mobile.is_some())
{
return Err(invalid_params(
"--dpr/--mobile/--no-mobile require --width/--height (viewport metrics need dimensions)",
));
}
if overrides.accept_language.is_some() && overrides.user_agent.is_none() {
return Err(invalid_params("--accept-language requires --ua"));
}
Ok(EmulateParams {
session_id: args.session.clone(),
tab_id: args.tab_id,
off: None,
overrides: Some(overrides),
})
}
pub fn dispatch(args: EmulateArgs, format: Format) -> Result<(), CliError> {
let device = args.device.clone();
let params = build_params(&args)?;
let info = ensure_daemon().context("ensure daemon is running")?;
let reply = call(info.sock_path, params)?;
render(&reply, device.as_deref(), format)
}
fn call(sock: PathBuf, params: EmulateParams) -> Result<EmulateResult, CliError> {
crate::cli::business_rpc::call::<EmulateParams, EmulateResult>(
sock,
"emulate",
Method::ToolEmulate,
Some(params),
TOOL_IPC_TIMEOUT,
)
}
fn render(reply: &EmulateResult, device: Option<&str>, format: Format) -> Result<(), CliError> {
match format {
Format::Json => {
let json = serde_json::to_string_pretty(reply)
.map_err(|e| CliError::Local(anyhow::anyhow!(e)))?;
println!("{json}");
}
Format::Human => {
if reply.cleared {
println!("tab={} emulation cleared", reply.tab_id);
} else {
print!("tab={} emulation applied", reply.tab_id);
if let Some(device) = device {
print!(" device={device}");
}
if let Some(applied) = &reply.applied {
if let (Some(w), Some(h)) = (applied.width, applied.height) {
print!(" viewport={w}x{h}");
if let Some(dpr) = applied.device_scale_factor {
print!("@{dpr}x");
}
if applied.mobile == Some(true) {
print!(" mobile");
}
}
if applied.user_agent.is_some() {
print!(" ua");
}
if applied.touch == Some(true) {
print!(" touch");
}
}
println!();
}
if let Some(note) = &reply.note {
println!("note: {note}");
}
}
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
fn base_args() -> EmulateArgs {
EmulateArgs {
session: "s1".into(),
tab_id: None,
device: None,
width: None,
height: None,
dpr: None,
mobile: false,
no_mobile: false,
ua: None,
accept_language: None,
touch: false,
no_touch: false,
max_touch_points: None,
off: false,
}
}
fn build(args: &EmulateArgs) -> EmulateParams {
build_params(args).expect("build_params should succeed")
}
#[test]
fn preset_alone_applies_full_environment() {
let args = EmulateArgs {
device: Some("iphone-14".into()),
..base_args()
};
let params = build(&args);
let o = params.overrides.unwrap();
assert_eq!(o.width, Some(390));
assert_eq!(o.height, Some(844));
assert_eq!(o.device_scale_factor, Some(3.0));
assert_eq!(o.mobile, Some(true));
assert!(o.user_agent.unwrap().contains("iPhone"));
assert_eq!(o.touch, Some(true));
assert_eq!(o.max_touch_points, Some(5));
assert!(params.off.is_none());
}
#[test]
fn every_preset_is_applicable() {
assert_eq!(DEVICE_PRESETS.len(), 7);
for preset in DEVICE_PRESETS {
let args = EmulateArgs {
device: Some(preset.name.into()),
..base_args()
};
let o = build(&args).overrides.unwrap();
assert_eq!(o.width, Some(preset.width));
assert_eq!(o.height, Some(preset.height));
assert!(o.user_agent.is_some());
}
}
#[test]
fn manual_flags_override_preset_fields() {
let args = EmulateArgs {
device: Some("pixel-7".into()),
width: Some(500),
height: Some(900),
dpr: Some(2.0),
ua: Some("custom-ua".into()),
..base_args()
};
let o = build(&args).overrides.unwrap();
assert_eq!(o.width, Some(500));
assert_eq!(o.height, Some(900));
assert_eq!(o.device_scale_factor, Some(2.0));
assert_eq!(o.user_agent.as_deref(), Some("custom-ua"));
// Untouched preset fields survive.
assert_eq!(o.mobile, Some(true));
assert_eq!(o.touch, Some(true));
}
#[test]
fn manual_only_viewport_and_ua() {
let args = EmulateArgs {
width: Some(390),
height: Some(844),
dpr: Some(3.0),
mobile: true,
ua: Some("Mozilla/5.0 (iPhone…)".into()),
touch: true,
..base_args()
};
let o = build(&args).overrides.unwrap();
assert_eq!(o.width, Some(390));
assert_eq!(o.device_scale_factor, Some(3.0));
assert_eq!(o.mobile, Some(true));
assert_eq!(o.touch, Some(true));
assert_eq!(o.max_touch_points, None);
}
#[test]
fn touch_alone_is_valid() {
let args = EmulateArgs {
touch: true,
..base_args()
};
let o = build(&args).overrides.unwrap();
assert_eq!(o.touch, Some(true));
assert!(o.width.is_none());
}
#[test]
fn max_touch_points_implies_touch() {
let args = EmulateArgs {
max_touch_points: Some(5),
..base_args()
};
let o = build(&args).overrides.unwrap();
assert_eq!(o.touch, Some(true));
assert_eq!(o.max_touch_points, Some(5));
}
#[test]
fn off_builds_clear_params() {
let args = EmulateArgs {
off: true,
..base_args()
};
let params = build(&args);
assert_eq!(params.off, Some(true));
assert!(params.overrides.is_none());
}
#[test]
fn off_is_exclusive() {
for args in [
EmulateArgs {
off: true,
device: Some("iphone-14".into()),
..base_args()
},
EmulateArgs {
off: true,
touch: true,
..base_args()
},
EmulateArgs {
off: true,
width: Some(390),
height: Some(844),
..base_args()
},
] {
let err = build_params(&args).unwrap_err();
assert!(err.to_string().contains("--off"));
}
}
#[test]
fn empty_invocation_is_rejected_with_guidance() {
let err = build_params(&base_args()).unwrap_err();
let msg = err.to_string();
assert!(msg.contains("--device"));
assert!(msg.contains("--off"));
assert!(msg.contains("iphone-14"));
}
#[test]
fn unknown_preset_lists_available() {
let args = EmulateArgs {
device: Some("nokia-3310".into()),
..base_args()
};
let err = build_params(&args).unwrap_err();
let msg = err.to_string();
assert!(msg.contains("nokia-3310"));
assert!(msg.contains("iphone-14"));
}
#[test]
fn width_height_must_pair() {
for args in [
EmulateArgs {
width: Some(390),
..base_args()
},
EmulateArgs {
height: Some(844),
..base_args()
},
] {
let err = build_params(&args).unwrap_err();
assert!(err.to_string().contains("together"));
}
}
#[test]
fn dpr_and_mobile_require_dimensions_without_preset() {
for args in [
EmulateArgs {
dpr: Some(3.0),
..base_args()
},
EmulateArgs {
mobile: true,
..base_args()
},
] {
let err = build_params(&args).unwrap_err();
assert!(err.to_string().contains("--width/--height"));
}
}
#[test]
fn dpr_without_dimensions_is_fine_with_preset() {
let args = EmulateArgs {
device: Some("iphone-se".into()),
dpr: Some(2.5),
..base_args()
};
let o = build(&args).overrides.unwrap();
assert_eq!(o.device_scale_factor, Some(2.5));
assert_eq!(o.width, Some(375));
}
#[test]
fn preset_supplies_the_missing_dimension() {
// Pairing is validated on the merged result, so a lone --width
// keeps the preset's height instead of erroring out.
let args = EmulateArgs {
device: Some("iphone-14".into()),
width: Some(500),
..base_args()
};
let o = build(&args).overrides.unwrap();
assert_eq!(o.width, Some(500));
assert_eq!(o.height, Some(844));
// Same for a lone --height.
let args = EmulateArgs {
device: Some("iphone-14".into()),
height: Some(900),
..base_args()
};
let o = build(&args).overrides.unwrap();
assert_eq!(o.width, Some(390));
assert_eq!(o.height, Some(900));
}
#[test]
fn no_mobile_and_no_touch_override_preset_fields() {
let args = EmulateArgs {
device: Some("iphone-14".into()),
no_mobile: true,
no_touch: true,
..base_args()
};
let o = build(&args).overrides.unwrap();
assert_eq!(o.mobile, Some(false));
assert_eq!(o.touch, Some(false));
// Untouched preset fields survive.
assert_eq!(o.width, Some(390));
assert_eq!(o.max_touch_points, Some(5));
}
#[test]
fn no_mobile_without_dimensions_is_rejected() {
// --no-mobile still maps to the mobile field, which needs a
// viewport without a preset.
let args = EmulateArgs {
no_mobile: true,
..base_args()
};
let err = build_params(&args).unwrap_err();
assert!(err.to_string().contains("--width/--height"));
}
#[test]
fn no_touch_alone_is_valid() {
let args = EmulateArgs {
no_touch: true,
..base_args()
};
let o = build(&args).overrides.unwrap();
assert_eq!(o.touch, Some(false));
assert!(o.width.is_none());
}
#[test]
fn accept_language_requires_ua() {
let args = EmulateArgs {
accept_language: Some("zh-CN".into()),
..base_args()
};
let err = build_params(&args).unwrap_err();
assert!(err.to_string().contains("--ua"));
// …but pairs fine with a preset (which carries a UA).
let args = EmulateArgs {
device: Some("iphone-14".into()),
accept_language: Some("zh-CN".into()),
..base_args()
};
let o = build(&args).overrides.unwrap();
assert_eq!(o.accept_language.as_deref(), Some("zh-CN"));
}
#[test]
fn preset_names_are_unique_and_listed() {
let names = preset_names();
for preset in DEVICE_PRESETS {
assert!(names.contains(preset.name));
assert_eq!(find_preset(preset.name), Some(preset));
}
assert!(find_preset("IPHONE-14").is_none());
}
}
+53 -1
View File
@@ -8,7 +8,7 @@ use std::time::Duration;
use anyhow::Context;
use bsk_protocol::Method;
use bsk_protocol::tools::{
HelpCompletionCriteria, HelpTarget, RequestHelpParams, RequestHelpResult,
HelpCompletionCriteria, HelpOutcome, HelpTarget, RequestHelpParams, RequestHelpResult,
};
use clap::Args;
@@ -16,6 +16,31 @@ use crate::cli::ensure_daemon::ensure_daemon;
use crate::cli::error::{CliError, Format};
use crate::cli::navigate::parse_timeout_ms;
/// Environment variable that disables the blocking `request-help`
/// human-in-loop tool, for unattended / background-server deployments
/// where a blocking help overlay must never appear.
pub(crate) const REQUEST_HELP_ENV: &str = "BSK_REQUEST_HELP";
/// Note attached to the synthetic result returned when request-help is
/// disabled, so the agent knows why no overlay was shown.
pub(crate) const REQUEST_HELP_DISABLED_NOTE: &str =
"request-help disabled by BSK_REQUEST_HELP=off (unattended mode)";
/// Whether `request-help` is disabled via [`REQUEST_HELP_ENV`]. Only the
/// value `off` (case-insensitive, surrounding whitespace ignored)
/// disables it; unset or any other value keeps the default enabled
/// behaviour.
pub(crate) fn request_help_disabled() -> bool {
is_off_value(std::env::var(REQUEST_HELP_ENV).ok().as_deref())
}
/// Pure value check behind [`request_help_disabled`], kept separate so it
/// can be unit-tested without touching process env (parallel tests race
/// on `std::env::set_var`, which is also `unsafe` in edition 2024).
fn is_off_value(value: Option<&str>) -> bool {
value.is_some_and(|v| v.trim().eq_ignore_ascii_case("off"))
}
#[derive(Debug, Clone, Args)]
pub struct RequestHelpArgs {
#[arg(long)]
@@ -72,6 +97,18 @@ pub fn parse_target(raw: &str) -> HelpTarget {
}
pub fn dispatch(args: RequestHelpArgs, format: Format) -> Result<(), CliError> {
// Disabled (`BSK_REQUEST_HELP=off`): return a synthetic `disabled`
// result immediately — no daemon startup, no overlay, no waiting.
if request_help_disabled() {
let result = RequestHelpResult {
outcome: HelpOutcome::Disabled,
completed_by: None,
note: Some(REQUEST_HELP_DISABLED_NOTE.into()),
tab_id: args.tab_id.unwrap_or(0),
resolved_targets: None,
};
return render(&result, format);
}
let info = ensure_daemon().context("ensure daemon is running")?;
let targets: Vec<HelpTarget> = args.target.iter().map(|t| parse_target(t)).collect();
let completion_criteria = args
@@ -156,4 +193,19 @@ mod tests {
assert_eq!(parse_target("email").selector.as_deref(), Some("email"));
assert!(parse_target("#login").ref_.is_none());
}
#[test]
fn off_value_detection() {
assert!(is_off_value(Some("off")));
assert!(is_off_value(Some("OFF")));
assert!(is_off_value(Some("Off")));
assert!(is_off_value(Some(" off ")));
assert!(is_off_value(Some("\toff\n")));
assert!(!is_off_value(None));
assert!(!is_off_value(Some("")));
assert!(!is_off_value(Some("on")));
assert!(!is_off_value(Some("0")));
assert!(!is_off_value(Some("offx")));
assert!(!is_off_value(Some("of f")));
}
}
+73 -15
View File
@@ -22,7 +22,7 @@ pub struct InstallSkillArgs {
#[arg(long = "harness", short = 'H', value_name = "HARNESS")]
pub harness: Vec<String>,
/// Install into every harness on this machine (detected or not).
/// Install into every harness detected on this machine.
#[arg(long)]
pub all: bool,
@@ -96,10 +96,6 @@ fn resolve_targets(
args: &InstallSkillArgs,
reports: &[crate::skill_install::HarnessReport],
) -> Result<Vec<HarnessId>> {
if args.all {
return Ok(HarnessId::ALL.to_vec());
}
if !args.harness.is_empty() {
let mut ids = Vec::with_capacity(args.harness.len());
for raw in &args.harness {
@@ -110,27 +106,32 @@ fn resolve_targets(
return Ok(ids);
}
let stdin_tty = io::stdin().is_terminal();
let stderr_tty = io::stderr().is_terminal();
if stdin_tty && stderr_tty && !args.yes {
return run_interactive_prompt(reports);
}
if args.yes {
let detected: Vec<_> = reports
if args.all || args.yes {
// `--all` means "every harness present on this machine": restrict
// to detected harnesses so we never create skill directories for
// harnesses the user does not have installed.
let mut detected: Vec<_> = reports
.iter()
.filter(|r| r.detected)
.map(|r| r.id)
.collect();
detected.sort_by_key(|id| id.index());
detected.dedup();
if detected.is_empty() {
bail!(
"no harnesses detected; pass --harness <id> or use --all -y to install everywhere"
"no harnesses detected on this machine; pass --harness <id> to install into a specific harness, or use --list to inspect"
);
}
return Ok(detected);
}
let stdin_tty = io::stdin().is_terminal();
let stderr_tty = io::stderr().is_terminal();
if stdin_tty && stderr_tty {
return run_interactive_prompt(reports);
}
bail!(
"non-interactive mode requires --harness, --all, or --yes (with detected harnesses); use --list to inspect"
);
@@ -174,3 +175,60 @@ fn render_human(output: &crate::skill_install::InstallSkillOutput) {
eprintln!("× {} — {}", err.harness, err.message);
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::skill_install::HarnessReport;
fn args(all: bool, yes: bool) -> InstallSkillArgs {
InstallSkillArgs {
harness: Vec::new(),
all,
list: false,
yes,
source: None,
force: false,
}
}
fn report(id: HarnessId, detected: bool) -> HarnessReport {
HarnessReport {
id,
skills_dir: PathBuf::from("/tmp/skills"),
detected,
detection_detail: None,
installed: false,
}
}
#[test]
fn all_targets_detected_harnesses_only() {
let reports = vec![
report(HarnessId::Cursor, true),
report(HarnessId::Codex, false),
report(HarnessId::ClaudeCode, true),
];
let targets = resolve_targets(&args(true, false), &reports).unwrap();
// Sorted by HarnessId::ALL order: ClaudeCode before Cursor;
// undetected Codex must not be targeted.
assert_eq!(targets, vec![HarnessId::ClaudeCode, HarnessId::Cursor]);
}
#[test]
fn yes_targets_detected_harnesses_only() {
let reports = vec![
report(HarnessId::Cursor, true),
report(HarnessId::Codex, false),
];
let targets = resolve_targets(&args(false, true), &reports).unwrap();
assert_eq!(targets, vec![HarnessId::Cursor]);
}
#[test]
fn all_and_yes_error_when_nothing_detected() {
let reports = vec![report(HarnessId::Cursor, false)];
assert!(resolve_targets(&args(true, false), &reports).is_err());
assert!(resolve_targets(&args(false, true), &reports).is_err());
}
}
+10
View File
@@ -9,6 +9,7 @@ pub mod console;
pub mod daemon;
pub mod dialogs;
pub mod doctor;
pub mod emulate;
pub mod ensure_daemon;
pub mod error;
pub mod evaluate;
@@ -30,11 +31,13 @@ pub mod status;
pub mod tab;
pub mod update;
pub mod waits;
pub mod window;
use clap::{Args, Parser, Subcommand};
use crate::cli::console::ConsoleArgs;
use crate::cli::daemon::DaemonCmd;
use crate::cli::emulate::EmulateArgs;
use crate::cli::evaluate::EvaluateArgs;
use crate::cli::get_html::GetHtmlArgs;
use crate::cli::human_loop::RequestHelpArgs;
@@ -50,6 +53,7 @@ use crate::cli::snapshot::SnapshotArgs;
use crate::cli::tab::TabCmd;
use crate::cli::update::UpdateArgs;
use crate::cli::waits::{WaitForNavigationArgs, WaitMsArgs};
use crate::cli::window::WindowCmd;
/// Tool calls wait slightly longer than the daemon's 30s tool timeout so
/// callers receive the structured daemon timeout instead of dropping the
@@ -118,6 +122,12 @@ pub enum Command {
/// Tab management commands.
Tab(TabCmd),
/// Agent Window management commands.
Window(WindowCmd),
/// Emulate a mobile device environment (viewport, UA, touch) on a tab.
Emulate(EmulateArgs),
/// Capture a PNG of the active tab or a snapshot ref element.
Screenshot(ScreenshotArgs),
+1 -1
View File
@@ -85,7 +85,7 @@ fn dispatch_start(args: RecordStartArgs, format: Format) -> Result<(), CliError>
}
let info = ensure_daemon().context("ensure daemon is running")?;
let session = start_session(info.sock_path.clone(), args.browser)?;
let session = start_session(info.sock_path.clone(), args.browser, None, None)?;
let start_params = RecordStartParams {
session_id: session.session_id.clone(),
+42 -2
View File
@@ -54,6 +54,30 @@ pub struct SessionStartArgs {
/// are connected).
#[arg(long)]
pub browser: Option<String>,
/// Agent Window outer width in CSS pixels (100..=7680). Both
/// `--width` and `--height` must be given to take effect.
#[arg(long, value_parser = window_size)]
pub width: Option<u32>,
/// Agent Window outer height in CSS pixels (100..=7680). Both
/// `--width` and `--height` must be given to take effect.
#[arg(long, value_parser = window_size)]
pub height: Option<u32>,
}
/// Parse a `--width` / `--height` Agent Window dimension (CSS pixels).
fn window_size(s: &str) -> Result<u32, String> {
let value: u32 = s
.parse()
.map_err(|_| format!("invalid window dimension {s:?}: expected a positive integer"))?;
if (100..=7680).contains(&value) {
Ok(value)
} else {
Err(format!(
"window dimension {value} out of range (100..=7680)"
))
}
}
#[derive(Debug, Clone, Args)]
@@ -71,6 +95,10 @@ pub struct SessionStopArgs {
struct StartParams {
#[serde(skip_serializing_if = "Option::is_none")]
browser_instance_id: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
width: Option<u32>,
#[serde(skip_serializing_if = "Option::is_none")]
height: Option<u32>,
}
#[derive(Debug, Deserialize)]
@@ -124,6 +152,11 @@ pub fn dispatch(cmd: SessionCmd, format: Format) -> Result<(), CliError> {
}
fn run_start(sock: PathBuf, args: SessionStartArgs, format: Format) -> Result<(), CliError> {
if args.width.is_some() != args.height.is_some() {
return Err(CliError::Local(anyhow::anyhow!(
"--width and --height must be given together"
)));
}
// The daemon may hold `session.start` open for up to
// `EXTENSION_CONNECT_WAIT` while a just-woken service worker
// reconnects. Let the user know we are waiting rather than hung —
@@ -140,7 +173,7 @@ fn run_start(sock: PathBuf, args: SessionStartArgs, format: Format) -> Result<()
}
});
}
let result = start_session(sock, args.browser);
let result = start_session(sock, args.browser, args.width, args.height);
waited.store(true, Ordering::SeqCst);
match result {
Ok(reply) => match format {
@@ -165,12 +198,19 @@ fn run_start(sock: PathBuf, args: SessionStartArgs, format: Format) -> Result<()
}
/// Start a session and open the Agent Window. Used by `session start` and `record start`.
pub fn start_session(sock: PathBuf, browser: Option<String>) -> Result<StartReply, CliError> {
pub fn start_session(
sock: PathBuf,
browser: Option<String>,
width: Option<u32>,
height: Option<u32>,
) -> Result<StartReply, CliError> {
call(
sock,
Method::SessionStart,
Some(StartParams {
browser_instance_id: browser,
width,
height,
}),
SESSION_START_IPC_TIMEOUT,
)
+103
View File
@@ -0,0 +1,103 @@
//! `bsk window …` subcommands — Agent Window management.
use std::path::PathBuf;
use anyhow::Context;
use bsk_protocol::Method;
use bsk_protocol::tools::{WindowResizeParams, WindowResizeResult};
use clap::{Args, Subcommand};
use crate::cli::TOOL_IPC_TIMEOUT;
use crate::cli::ensure_daemon::ensure_daemon;
use crate::cli::error::{CliError, Format};
#[derive(Debug, Clone, Args)]
pub struct WindowCmd {
#[command(subcommand)]
pub sub: WindowSub,
}
#[derive(Debug, Clone, Subcommand)]
pub enum WindowSub {
/// Resize the session's Agent Window.
Resize(WindowResizeArgs),
}
#[derive(Debug, Clone, Args)]
pub struct WindowResizeArgs {
/// Session id (must be active).
#[arg(long)]
pub session: String,
/// New Agent Window outer width in CSS pixels (100..=7680).
#[arg(long, value_parser = window_size)]
pub width: u32,
/// New Agent Window outer height in CSS pixels (100..=7680).
#[arg(long, value_parser = window_size)]
pub height: u32,
}
/// Parse a `--width` / `--height` Agent Window dimension (CSS pixels).
fn window_size(s: &str) -> Result<u32, String> {
let value: u32 = s
.parse()
.map_err(|_| format!("invalid window dimension {s:?}: expected a positive integer"))?;
if (100..=7680).contains(&value) {
Ok(value)
} else {
Err(format!(
"window dimension {value} out of range (100..=7680)"
))
}
}
pub fn dispatch(cmd: WindowCmd, format: Format) -> Result<(), CliError> {
let info = ensure_daemon().context("ensure daemon is running")?;
match cmd.sub {
WindowSub::Resize(args) => run_resize(info.sock_path, args, format),
}
}
fn run_resize(sock: PathBuf, args: WindowResizeArgs, format: Format) -> Result<(), CliError> {
let params = WindowResizeParams {
session_id: args.session,
width: args.width,
height: args.height,
};
let reply: WindowResizeResult =
ipc_call("window-resize-1", Method::ToolWindowResize, sock, params)?;
match format {
Format::Json => {
let json = serde_json::to_string_pretty(&reply)
.map_err(|e| CliError::Local(anyhow::anyhow!(e)))?;
println!("{json}");
}
Format::Human => {
println!(
"resized window_id={} to {}x{}",
reply.window_id, reply.width, reply.height
);
}
}
Ok(())
}
fn ipc_call<P, R>(
rpc_id_prefix: &'static str,
method: Method,
sock: PathBuf,
params: P,
) -> Result<R, CliError>
where
P: serde::Serialize + Send + 'static,
R: serde::de::DeserializeOwned + Send + 'static,
{
crate::cli::business_rpc::call::<P, R>(
sock,
rpc_id_prefix,
method,
Some(params),
TOOL_IPC_TIMEOUT,
)
}
+38 -1
View File
@@ -26,7 +26,9 @@ use bsk_protocol::system::{
BrowserListParams, BrowserStatusEntry, SessionStatusEntry, StatusParams, StatusResult,
VersionSkewEntry,
};
use bsk_protocol::tools::{ReturnFailure, WaitMsParams, WaitMsResult};
use bsk_protocol::tools::{
HelpOutcome, RequestHelpResult, ReturnFailure, WaitMsParams, WaitMsResult,
};
use bsk_protocol::{
CancelParams, CancelResult, ErrorCode, Method, PingResult, ResponseBody, RpcError, RpcId,
};
@@ -227,6 +229,8 @@ pub fn full_handler(status: DaemonStatus, state: Arc<DaemonState>) -> RpcHandler
| Method::ToolTabSelect
| Method::ToolTabBorrow
| Method::ToolTabReturn
| Method::ToolWindowResize
| Method::ToolEmulate
| Method::ToolScreenshot
| Method::ToolConsole
| Method::ToolNetwork
@@ -280,6 +284,19 @@ async fn handle_tool_dispatch(
method: Method,
params: Value,
) -> ResponseBody {
// `BSK_REQUEST_HELP=off` (unattended mode): never forward the
// blocking human-in-loop call to the extension; answer immediately
// with a synthetic `disabled` result.
if method == Method::ToolRequestHelp && crate::cli::human_loop::request_help_disabled() {
let result = RequestHelpResult {
outcome: HelpOutcome::Disabled,
completed_by: None,
note: Some(crate::cli::human_loop::REQUEST_HELP_DISABLED_NOTE.into()),
tab_id: params.get("tab_id").and_then(Value::as_i64).unwrap_or(0),
resolved_targets: None,
};
return ResponseBody::Ok(serde_json::to_value(result).unwrap_or(Value::Null));
}
let session_id = match params.get("session_id").and_then(|v| v.as_str()) {
Some(s) if !s.is_empty() => SessionId(s.to_string()),
_ => {
@@ -523,6 +540,10 @@ fn tool_dispatch_timeout(params: &Value) -> Result<Duration, RpcError> {
struct CliSessionStartParams {
#[serde(default)]
pub browser_instance_id: Option<String>,
#[serde(default)]
pub width: Option<u32>,
#[serde(default)]
pub height: Option<u32>,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
@@ -614,6 +635,8 @@ async fn handle_session_start(state: &Arc<DaemonState>, params: Value) -> Result
let params: CliSessionStartParams = if params.is_null() {
CliSessionStartParams {
browser_instance_id: None,
width: None,
height: None,
}
} else {
serde_json::from_value(params).map_err(|err| RpcError {
@@ -622,11 +645,25 @@ async fn handle_session_start(state: &Arc<DaemonState>, params: Value) -> Result
data: None,
})?
};
// `--width` without `--height` (or vice versa) is rejected: the
// extension only accepts a complete size pair.
let window_size = match (params.width, params.height) {
(Some(width), Some(height)) => Some((width, height)),
(None, None) => None,
_ => {
return Err(RpcError {
code: ErrorCode::InvalidParams,
message: "width and height must be given together".into(),
data: None,
});
}
};
match start_session(
&state.browsers,
&state.sessions,
&state.tool_queues,
params.browser_instance_id.as_deref(),
window_size,
state.config.extension_connect_wait,
DEFAULT_RPC_TIMEOUT,
)
+4
View File
@@ -401,6 +401,8 @@ pub async fn start_session(
sessions: &Arc<SessionRegistry>,
queues: &Arc<ToolQueueRegistry>,
requested: Option<&str>,
// Optional Agent Window outer size as `(width, height)` CSS pixels.
window_size: Option<(u32, u32)>,
connect_wait: Duration,
timeout_dur: Duration,
) -> Result<Session, StartSessionError> {
@@ -427,6 +429,8 @@ pub async fn start_session(
let params = SessionStartParams {
session_id: session_id.0.clone(),
browser_instance_id: Some(client.id.0.clone()),
width: window_size.map(|(width, _)| width),
height: window_size.map(|(_, height)| height),
};
let rpc_id = next_rpc_id("sess-start");
let request = RequestFrame {
+2
View File
@@ -80,6 +80,8 @@ fn dispatch(cli: Cli, format: Format) -> Result<(), CliError> {
Command::Session(cmd) => cli::session::dispatch(cmd, format),
Command::Browsers => cli::browsers::dispatch(format),
Command::Tab(cmd) => cli::tab::dispatch(cmd, format),
Command::Window(cmd) => cli::window::dispatch(cmd, format),
Command::Emulate(args) => cli::emulate::dispatch(args, format),
Command::Screenshot(args) => cli::screenshot::dispatch(args, format),
Command::Snapshot(args) => cli::snapshot::dispatch(args, format),
Command::Observe(args) => cli::observe::dispatch(args, format),
@@ -21,6 +21,7 @@ pub enum HarnessId {
Workbuddy,
PiAgent,
Hermes,
KimiCode,
}
impl HarnessId {
@@ -36,6 +37,7 @@ impl HarnessId {
HarnessId::Workbuddy,
HarnessId::PiAgent,
HarnessId::Hermes,
HarnessId::KimiCode,
];
pub fn index(self) -> usize {
@@ -55,6 +57,7 @@ impl HarnessId {
HarnessId::Workbuddy => "workbuddy",
HarnessId::PiAgent => "pi",
HarnessId::Hermes => "hermes",
HarnessId::KimiCode => "kimi-code",
}
}
@@ -71,6 +74,7 @@ impl HarnessId {
HarnessId::Workbuddy => "WorkBuddy",
HarnessId::PiAgent => "Pi",
HarnessId::Hermes => "Hermes Agent",
HarnessId::KimiCode => "Kimi Code",
}
}
@@ -87,6 +91,10 @@ impl HarnessId {
HarnessId::Workbuddy => home.join(".workbuddy").join("skills"),
HarnessId::PiAgent => home.join(".pi").join("agent").join("skills"),
HarnessId::Hermes => hermes_home_for_user_home(home).join("skills"),
// Kimi Code CLI 用户级 skills 目录:`$KIMI_CODE_HOME/skills`,
// 默认 ~/.kimi-code/skills,见
// https://github.com/MoonshotAI/kimi-code/blob/main/docs/en/customization/skills.md
HarnessId::KimiCode => kimi_code_home_for_user_home(home).join("skills"),
}
}
@@ -215,6 +223,23 @@ impl HarnessId {
push("`hermes` on PATH");
}
}
HarnessId::KimiCode => {
let kimi_home = kimi_code_home_for_user_home(home);
if kimi_home.is_dir() {
let label = if std::env::var("KIMI_CODE_HOME")
.ok()
.is_some_and(|value| !value.trim().is_empty())
{
format!("$KIMI_CODE_HOME ({})", kimi_home.display())
} else {
"~/.kimi-code".to_string()
};
push(&label);
}
if command_exists("kimi") {
push("`kimi` on PATH");
}
}
}
let detected = !signals.is_empty();
@@ -265,6 +290,7 @@ pub fn parse_harness_id(raw: &str) -> Result<HarnessId> {
"workbuddy" | "work-buddy" => Ok(HarnessId::Workbuddy),
"pi" | "pi-agent" | "pi_agent" | "piagent" => Ok(HarnessId::PiAgent),
"hermes" | "hermes-agent" | "hermes_agent" | "hermesagent" => Ok(HarnessId::Hermes),
"kimi-code" | "kimi" | "kimicode" | "kimi-cli" | "kimi_cli" => Ok(HarnessId::KimiCode),
other => bail!(
"unknown harness '{other}'; expected one of: {}",
HarnessId::ALL
@@ -280,6 +306,18 @@ pub(crate) fn home_dir() -> Result<PathBuf> {
dirs::home_dir().ok_or_else(|| anyhow::anyhow!("could not resolve home directory"))
}
/// Kimi Code CLI 的数据根目录:`KIMI_CODE_HOME` 设置时用之,否则 `~/.kimi-code`
/// (见 kimi-code 官方 skills 文档:用户级 skills 在 `$KIMI_CODE_HOME/skills`)。
fn kimi_code_home_for_user_home(home: &Path) -> PathBuf {
if let Ok(override_home) = std::env::var("KIMI_CODE_HOME") {
let trimmed = override_home.trim();
if !trimmed.is_empty() {
return PathBuf::from(trimmed);
}
}
home.join(".kimi-code")
}
/// Mirrors [`get_hermes_home()`](https://github.com/NousResearch/hermes-agent/blob/main/hermes_constants.py):
/// `HERMES_HOME` when set, else `%LOCALAPPDATA%\\hermes` on native Windows, else `~/.hermes`.
fn hermes_home_for_user_home(home: &Path) -> PathBuf {
@@ -360,6 +398,12 @@ fn command_exists(name: &str) -> bool {
mod tests {
use super::*;
/// 读写 KIMI_CODE_HOME 环境变量的测试共用的互斥锁(Rust 测试默认并行)。
fn kimi_env_lock() -> &'static std::sync::Mutex<()> {
static LOCK: std::sync::Mutex<()> = std::sync::Mutex::new(());
&LOCK
}
#[test]
fn parse_aliases() {
assert_eq!(parse_harness_id("codebody").unwrap(), HarnessId::Codebuddy);
@@ -369,6 +413,9 @@ mod tests {
);
assert_eq!(parse_harness_id("pi-agent").unwrap(), HarnessId::PiAgent);
assert_eq!(parse_harness_id("hermes-agent").unwrap(), HarnessId::Hermes);
assert_eq!(parse_harness_id("kimi").unwrap(), HarnessId::KimiCode);
assert_eq!(parse_harness_id("kimi-code").unwrap(), HarnessId::KimiCode);
assert_eq!(parse_harness_id("kimi-cli").unwrap(), HarnessId::KimiCode);
}
#[test]
@@ -424,6 +471,52 @@ mod tests {
HarnessId::Hermes.skills_dir_for_home(home),
PathBuf::from("/home/user/.hermes/skills")
);
assert_eq!(
HarnessId::KimiCode.skills_dir_for_home(home),
PathBuf::from("/home/user/.kimi-code/skills")
);
}
#[test]
fn detects_kimi_code_from_home_layout() {
// 与改动 KIMI_CODE_HOME 的测试互斥,避免并行读写环境变量产生竞争。
let _guard = kimi_env_lock()
.lock()
.unwrap_or_else(|err| err.into_inner());
let tmp = tempfile::TempDir::new().unwrap();
let home = tmp.path();
std::fs::create_dir_all(home.join(".kimi-code")).unwrap();
let report = HarnessId::KimiCode.report_for_home(home);
assert!(report.detected);
assert_eq!(report.skills_dir, home.join(".kimi-code").join("skills"));
assert_eq!(HarnessId::KimiCode.cli_name(), "kimi-code");
assert_eq!(HarnessId::KimiCode.display_name(), "Kimi Code");
}
#[test]
fn kimi_code_skills_dir_honors_kimi_code_home_env() {
let _guard = kimi_env_lock()
.lock()
.unwrap_or_else(|err| err.into_inner());
let tmp = tempfile::TempDir::new().unwrap();
let custom = tmp.path().join("custom-kimi-code");
std::fs::create_dir_all(&custom).unwrap();
let previous = std::env::var("KIMI_CODE_HOME").ok();
// SAFETY: 持有 kimi_env_lock,覆盖所有读写 KIMI_CODE_HOME 的测试。
unsafe {
std::env::set_var("KIMI_CODE_HOME", &custom);
}
let home = Path::new("/home/user");
assert_eq!(
HarnessId::KimiCode.skills_dir_for_home(home),
custom.join("skills")
);
unsafe {
match previous {
Some(value) => std::env::set_var("KIMI_CODE_HOME", value),
None => std::env::remove_var("KIMI_CODE_HOME"),
}
}
}
#[test]
+255
View File
@@ -227,3 +227,258 @@ fn parses_record_start_without_url() {
assert_eq!(args.browser.as_deref(), Some("022ca8ac"));
assert!(args.url.is_none());
}
#[test]
fn parses_session_start_with_window_size() {
use bsk::cli::session::{SessionCmd, SessionSub};
let cli = parse(&[
"bsk", "session", "start", "--width", "1280", "--height", "800",
]);
let Command::Session(SessionCmd {
sub: SessionSub::Start(args),
}) = cli.command
else {
panic!("expected session start subcommand");
};
assert_eq!(args.width, Some(1280));
assert_eq!(args.height, Some(800));
}
#[test]
fn session_start_window_size_defaults_to_none() {
use bsk::cli::session::{SessionCmd, SessionSub};
let cli = parse(&["bsk", "session", "start"]);
let Command::Session(SessionCmd {
sub: SessionSub::Start(args),
}) = cli.command
else {
panic!("expected session start subcommand");
};
assert!(args.width.is_none());
assert!(args.height.is_none());
}
#[test]
fn rejects_out_of_range_session_start_window_size() {
assert!(Cli::try_parse_from(["bsk", "session", "start", "--width", "99"]).is_err());
assert!(Cli::try_parse_from(["bsk", "session", "start", "--height", "7681"]).is_err());
assert!(Cli::try_parse_from(["bsk", "session", "start", "--width", "abc"]).is_err());
}
#[test]
fn parses_window_resize() {
use bsk::cli::window::{WindowCmd, WindowSub};
let cli = parse(&[
"bsk",
"window",
"resize",
"--session",
"s1",
"--width",
"1280",
"--height",
"800",
]);
let Command::Window(WindowCmd {
sub: WindowSub::Resize(args),
}) = cli.command
else {
panic!("expected window resize subcommand");
};
assert_eq!(args.session, "s1");
assert_eq!(args.width, 1280);
assert_eq!(args.height, 800);
}
#[test]
fn rejects_invalid_window_resize_dimensions() {
assert!(
Cli::try_parse_from([
"bsk",
"window",
"resize",
"--session",
"s1",
"--width",
"99",
"--height",
"800"
])
.is_err()
);
assert!(
Cli::try_parse_from([
"bsk",
"window",
"resize",
"--session",
"s1",
"--width",
"1280",
"--height",
"7681"
])
.is_err()
);
// width/height are required for resize.
assert!(Cli::try_parse_from(["bsk", "window", "resize", "--session", "s1"]).is_err());
}
#[test]
fn parses_emulate_with_device_preset() {
use bsk::cli::emulate::EmulateArgs;
let cli = parse(&["bsk", "emulate", "--session", "s1", "--device", "iphone-14"]);
let Command::Emulate(EmulateArgs {
session,
device,
off,
..
}) = cli.command
else {
panic!("expected emulate subcommand");
};
assert_eq!(session, "s1");
assert_eq!(device.as_deref(), Some("iphone-14"));
assert!(!off);
}
#[test]
fn parses_emulate_manual_overrides() {
let cli = parse(&[
"bsk",
"emulate",
"--session",
"s1",
"--width",
"390",
"--height",
"844",
"--dpr",
"3",
"--mobile",
"--ua",
"Mozilla/5.0 (iPhone)",
"--accept-language",
"zh-CN",
"--touch",
"--max-touch-points",
"5",
"--tab-id",
"7",
]);
let Command::Emulate(args) = cli.command else {
panic!("expected emulate subcommand");
};
assert_eq!(args.width, Some(390));
assert_eq!(args.height, Some(844));
assert_eq!(args.dpr, Some(3.0));
assert!(args.mobile);
assert_eq!(args.ua.as_deref(), Some("Mozilla/5.0 (iPhone)"));
assert_eq!(args.accept_language.as_deref(), Some("zh-CN"));
assert!(args.touch);
assert_eq!(args.max_touch_points, Some(5));
assert_eq!(args.tab_id, Some(7));
}
#[test]
fn parses_emulate_off() {
let cli = parse(&["bsk", "emulate", "--session", "s1", "--off"]);
let Command::Emulate(args) = cli.command else {
panic!("expected emulate subcommand");
};
assert!(args.off);
}
#[test]
fn parses_emulate_no_mobile_no_touch() {
let cli = parse(&[
"bsk",
"emulate",
"--session",
"s1",
"--device",
"iphone-14",
"--no-mobile",
"--no-touch",
]);
let Command::Emulate(args) = cli.command else {
panic!("expected emulate subcommand");
};
assert!(args.no_mobile);
assert!(args.no_touch);
assert!(!args.mobile);
assert!(!args.touch);
}
#[test]
fn rejects_conflicting_emulate_flags() {
for extra in [
&["--mobile", "--no-mobile"][..],
&["--touch", "--no-touch"][..],
] {
let mut argv = vec!["bsk", "emulate", "--session", "s1", "--device", "iphone-14"];
argv.extend_from_slice(extra);
assert!(Cli::try_parse_from(argv).is_err());
}
}
#[test]
fn rejects_invalid_emulate_values() {
// Zero / out-of-range dimensions.
assert!(
Cli::try_parse_from([
"bsk",
"emulate",
"--session",
"s1",
"--width",
"0",
"--height",
"844"
])
.is_err()
);
// Non-positive dpr.
assert!(
Cli::try_parse_from([
"bsk",
"emulate",
"--session",
"s1",
"--width",
"390",
"--height",
"844",
"--dpr",
"0"
])
.is_err()
);
assert!(
Cli::try_parse_from([
"bsk",
"emulate",
"--session",
"s1",
"--width",
"390",
"--height",
"844",
"--dpr",
"abc"
])
.is_err()
);
// Zero touch points.
assert!(
Cli::try_parse_from([
"bsk",
"emulate",
"--session",
"s1",
"--max-touch-points",
"0"
])
.is_err()
);
}
@@ -0,0 +1,151 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "EmulateOverrides",
"description": "Concrete emulation overrides for one tab. Field presence drives the extension: only the overrides whose fields are set are touched, so a call may change just the UA, just the viewport, or any combination.",
"type": "object",
"properties": {
"accept_language": {
"description": "`Accept-Language` header override. Requires `user_agent`.",
"type": [
"string",
"null"
]
},
"device_scale_factor": {
"description": "Device scale factor (device pixel ratio). Only meaningful with `width`/`height`; the extension maps `None` to CDP's `0` (\"use the display's native factor\").",
"type": [
"number",
"null"
],
"format": "double"
},
"height": {
"description": "Viewport height in CSS pixels. Requires `width`.",
"type": [
"integer",
"null"
],
"format": "uint32",
"minimum": 0.0
},
"max_touch_points": {
"description": "Touch points reported while touch emulation is on (CDP defaults to 1 when omitted).",
"type": [
"integer",
"null"
],
"format": "uint32",
"minimum": 0.0
},
"mobile": {
"description": "Emulate a mobile viewport (meta-viewport handling, overlay scrollbars). Only meaningful with `width`/`height`.",
"type": [
"boolean",
"null"
]
},
"touch": {
"description": "Enable/disable touch emulation (`Emulation.setTouchEmulationEnabled`). When omitted while `max_touch_points` is set, touch is enabled.",
"type": [
"boolean",
"null"
]
},
"user_agent": {
"description": "UA string for `Emulation.setUserAgentOverride`.",
"type": [
"string",
"null"
]
},
"user_agent_metadata": {
"description": "UA client-hints metadata. Requires `user_agent`.",
"anyOf": [
{
"$ref": "#/definitions/UserAgentMetadata"
},
{
"type": "null"
}
]
},
"width": {
"description": "Viewport width in CSS pixels. Requires `height`.",
"type": [
"integer",
"null"
],
"format": "uint32",
"minimum": 0.0
}
},
"definitions": {
"UserAgentBrandVersion": {
"description": "One entry of CDP `Emulation.UserAgentBrandVersion` (UA client hints).",
"type": "object",
"required": [
"brand",
"version"
],
"properties": {
"brand": {
"type": "string"
},
"version": {
"type": "string"
}
}
},
"UserAgentMetadata": {
"description": "Mirror of CDP `Emulation.UserAgentMetadata`; every field is optional and only sent when present.",
"type": "object",
"properties": {
"architecture": {
"type": [
"string",
"null"
]
},
"brands": {
"type": [
"array",
"null"
],
"items": {
"$ref": "#/definitions/UserAgentBrandVersion"
}
},
"full_version": {
"type": [
"string",
"null"
]
},
"mobile": {
"type": [
"boolean",
"null"
]
},
"model": {
"type": [
"string",
"null"
]
},
"platform": {
"type": [
"string",
"null"
]
},
"platform_version": {
"type": [
"string",
"null"
]
}
}
}
}
}
@@ -0,0 +1,188 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "EmulateParams",
"type": "object",
"required": [
"session_id"
],
"properties": {
"off": {
"description": "Clear every emulation override on the tab and restore its real environment. Mutually exclusive with `overrides`.",
"type": [
"boolean",
"null"
]
},
"overrides": {
"description": "Overrides to apply. Required unless `off` is set.",
"anyOf": [
{
"$ref": "#/definitions/EmulateOverrides"
},
{
"type": "null"
}
]
},
"session_id": {
"type": "string"
},
"tab_id": {
"description": "Target tab. Defaults to the Agent Window's active tab.",
"type": [
"integer",
"null"
],
"format": "int64"
}
},
"definitions": {
"EmulateOverrides": {
"description": "Concrete emulation overrides for one tab. Field presence drives the extension: only the overrides whose fields are set are touched, so a call may change just the UA, just the viewport, or any combination.",
"type": "object",
"properties": {
"accept_language": {
"description": "`Accept-Language` header override. Requires `user_agent`.",
"type": [
"string",
"null"
]
},
"device_scale_factor": {
"description": "Device scale factor (device pixel ratio). Only meaningful with `width`/`height`; the extension maps `None` to CDP's `0` (\"use the display's native factor\").",
"type": [
"number",
"null"
],
"format": "double"
},
"height": {
"description": "Viewport height in CSS pixels. Requires `width`.",
"type": [
"integer",
"null"
],
"format": "uint32",
"minimum": 0.0
},
"max_touch_points": {
"description": "Touch points reported while touch emulation is on (CDP defaults to 1 when omitted).",
"type": [
"integer",
"null"
],
"format": "uint32",
"minimum": 0.0
},
"mobile": {
"description": "Emulate a mobile viewport (meta-viewport handling, overlay scrollbars). Only meaningful with `width`/`height`.",
"type": [
"boolean",
"null"
]
},
"touch": {
"description": "Enable/disable touch emulation (`Emulation.setTouchEmulationEnabled`). When omitted while `max_touch_points` is set, touch is enabled.",
"type": [
"boolean",
"null"
]
},
"user_agent": {
"description": "UA string for `Emulation.setUserAgentOverride`.",
"type": [
"string",
"null"
]
},
"user_agent_metadata": {
"description": "UA client-hints metadata. Requires `user_agent`.",
"anyOf": [
{
"$ref": "#/definitions/UserAgentMetadata"
},
{
"type": "null"
}
]
},
"width": {
"description": "Viewport width in CSS pixels. Requires `height`.",
"type": [
"integer",
"null"
],
"format": "uint32",
"minimum": 0.0
}
}
},
"UserAgentBrandVersion": {
"description": "One entry of CDP `Emulation.UserAgentBrandVersion` (UA client hints).",
"type": "object",
"required": [
"brand",
"version"
],
"properties": {
"brand": {
"type": "string"
},
"version": {
"type": "string"
}
}
},
"UserAgentMetadata": {
"description": "Mirror of CDP `Emulation.UserAgentMetadata`; every field is optional and only sent when present.",
"type": "object",
"properties": {
"architecture": {
"type": [
"string",
"null"
]
},
"brands": {
"type": [
"array",
"null"
],
"items": {
"$ref": "#/definitions/UserAgentBrandVersion"
}
},
"full_version": {
"type": [
"string",
"null"
]
},
"mobile": {
"type": [
"boolean",
"null"
]
},
"model": {
"type": [
"string",
"null"
]
},
"platform": {
"type": [
"string",
"null"
]
},
"platform_version": {
"type": [
"string",
"null"
]
}
}
}
}
}
@@ -0,0 +1,186 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "EmulateResult",
"type": "object",
"required": [
"cleared",
"tab_id"
],
"properties": {
"applied": {
"description": "Echo of the overrides that were applied. Absent when cleared.",
"anyOf": [
{
"$ref": "#/definitions/EmulateOverrides"
},
{
"type": "null"
}
]
},
"cleared": {
"description": "True when overrides were cleared (`off`); false when applied.",
"type": "boolean"
},
"note": {
"description": "Scope note: emulation overrides are per-tab (CDP target) and are not inherited by new tabs.",
"type": [
"string",
"null"
]
},
"tab_id": {
"type": "integer",
"format": "int64"
}
},
"definitions": {
"EmulateOverrides": {
"description": "Concrete emulation overrides for one tab. Field presence drives the extension: only the overrides whose fields are set are touched, so a call may change just the UA, just the viewport, or any combination.",
"type": "object",
"properties": {
"accept_language": {
"description": "`Accept-Language` header override. Requires `user_agent`.",
"type": [
"string",
"null"
]
},
"device_scale_factor": {
"description": "Device scale factor (device pixel ratio). Only meaningful with `width`/`height`; the extension maps `None` to CDP's `0` (\"use the display's native factor\").",
"type": [
"number",
"null"
],
"format": "double"
},
"height": {
"description": "Viewport height in CSS pixels. Requires `width`.",
"type": [
"integer",
"null"
],
"format": "uint32",
"minimum": 0.0
},
"max_touch_points": {
"description": "Touch points reported while touch emulation is on (CDP defaults to 1 when omitted).",
"type": [
"integer",
"null"
],
"format": "uint32",
"minimum": 0.0
},
"mobile": {
"description": "Emulate a mobile viewport (meta-viewport handling, overlay scrollbars). Only meaningful with `width`/`height`.",
"type": [
"boolean",
"null"
]
},
"touch": {
"description": "Enable/disable touch emulation (`Emulation.setTouchEmulationEnabled`). When omitted while `max_touch_points` is set, touch is enabled.",
"type": [
"boolean",
"null"
]
},
"user_agent": {
"description": "UA string for `Emulation.setUserAgentOverride`.",
"type": [
"string",
"null"
]
},
"user_agent_metadata": {
"description": "UA client-hints metadata. Requires `user_agent`.",
"anyOf": [
{
"$ref": "#/definitions/UserAgentMetadata"
},
{
"type": "null"
}
]
},
"width": {
"description": "Viewport width in CSS pixels. Requires `height`.",
"type": [
"integer",
"null"
],
"format": "uint32",
"minimum": 0.0
}
}
},
"UserAgentBrandVersion": {
"description": "One entry of CDP `Emulation.UserAgentBrandVersion` (UA client hints).",
"type": "object",
"required": [
"brand",
"version"
],
"properties": {
"brand": {
"type": "string"
},
"version": {
"type": "string"
}
}
},
"UserAgentMetadata": {
"description": "Mirror of CDP `Emulation.UserAgentMetadata`; every field is optional and only sent when present.",
"type": "object",
"properties": {
"architecture": {
"type": [
"string",
"null"
]
},
"brands": {
"type": [
"array",
"null"
],
"items": {
"$ref": "#/definitions/UserAgentBrandVersion"
}
},
"full_version": {
"type": [
"string",
"null"
]
},
"mobile": {
"type": [
"boolean",
"null"
]
},
"model": {
"type": [
"string",
"null"
]
},
"platform": {
"type": [
"string",
"null"
]
},
"platform_version": {
"type": [
"string",
"null"
]
}
}
}
}
}
@@ -0,0 +1,71 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "UserAgentMetadata",
"description": "Mirror of CDP `Emulation.UserAgentMetadata`; every field is optional and only sent when present.",
"type": "object",
"properties": {
"architecture": {
"type": [
"string",
"null"
]
},
"brands": {
"type": [
"array",
"null"
],
"items": {
"$ref": "#/definitions/UserAgentBrandVersion"
}
},
"full_version": {
"type": [
"string",
"null"
]
},
"mobile": {
"type": [
"boolean",
"null"
]
},
"model": {
"type": [
"string",
"null"
]
},
"platform": {
"type": [
"string",
"null"
]
},
"platform_version": {
"type": [
"string",
"null"
]
}
},
"definitions": {
"UserAgentBrandVersion": {
"description": "One entry of CDP `Emulation.UserAgentBrandVersion` (UA client hints).",
"type": "object",
"required": [
"brand",
"version"
],
"properties": {
"brand": {
"type": "string"
},
"version": {
"type": "string"
}
}
}
}
}
@@ -83,6 +83,13 @@
"enum": [
"navigated"
]
},
{
"description": "request-help is disabled by configuration (`BSK_REQUEST_HELP=off`): the call returns immediately without ever showing the overlay.",
"type": "string",
"enum": [
"disabled"
]
}
]
},
@@ -12,8 +12,26 @@
"null"
]
},
"height": {
"description": "Optional Agent Window outer height in CSS pixels (100..=7680).",
"type": [
"integer",
"null"
],
"format": "uint32",
"minimum": 0.0
},
"session_id": {
"type": "string"
},
"width": {
"description": "Optional Agent Window outer width in CSS pixels (100..=7680).",
"type": [
"integer",
"null"
],
"format": "uint32",
"minimum": 0.0
}
}
}
@@ -0,0 +1,27 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "WindowResizeParams",
"type": "object",
"required": [
"height",
"session_id",
"width"
],
"properties": {
"height": {
"description": "New Agent Window outer height in CSS pixels (100..=7680).",
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"session_id": {
"type": "string"
},
"width": {
"description": "New Agent Window outer width in CSS pixels (100..=7680).",
"type": "integer",
"format": "uint32",
"minimum": 0.0
}
}
}
@@ -0,0 +1,26 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "WindowResizeResult",
"type": "object",
"required": [
"height",
"width",
"window_id"
],
"properties": {
"height": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"width": {
"type": "integer",
"format": "uint32",
"minimum": 0.0
},
"window_id": {
"type": "integer",
"format": "int64"
}
}
}
@@ -45,6 +45,14 @@ fn main() {
dump!(SessionStopParams, "tool_session_stop_params");
dump!(SessionStopResult, "tool_session_stop_result");
dump!(WindowResizeParams, "tool_window_resize_params");
dump!(WindowResizeResult, "tool_window_resize_result");
dump!(EmulateParams, "tool_emulate_params");
dump!(EmulateResult, "tool_emulate_result");
dump!(EmulateOverrides, "tool_emulate_overrides");
dump!(UserAgentMetadata, "tool_emulate_user_agent_metadata");
dump!(TabListParams, "tool_tab_list_params");
dump!(TabListResult, "tool_tab_list_result");
dump!(TabCreateParams, "tool_tab_create_params");
+15
View File
@@ -44,6 +44,10 @@ pub enum Method {
ToolSessionStart,
#[serde(rename = "tool.session_stop")]
ToolSessionStop,
#[serde(rename = "tool.window_resize")]
ToolWindowResize,
#[serde(rename = "tool.emulate")]
ToolEmulate,
#[serde(rename = "tool.tab_list")]
ToolTabList,
#[serde(rename = "tool.tab_create")]
@@ -140,6 +144,8 @@ impl Method {
| Method::ToolTabBorrow
| Method::ToolTabReturn
| Method::ToolTabSelect
| Method::ToolWindowResize
| Method::ToolEmulate
| Method::ToolNavigate
| Method::ToolNavigateBack
| Method::ToolNavigateForward
@@ -231,6 +237,13 @@ mod tests {
assert_eq!(serde_json::to_value(method).unwrap(), json!("tool.network"));
}
#[test]
fn emulate_method_round_trips() {
let method: Method = serde_json::from_value(json!("tool.emulate")).unwrap();
assert_eq!(method, Method::ToolEmulate);
assert_eq!(serde_json::to_value(method).unwrap(), json!("tool.emulate"));
}
#[test]
fn cancel_params_and_result_round_trip() {
let params: CancelParams = serde_json::from_value(json!({ "rpc_id": "wait-1" })).unwrap();
@@ -272,6 +285,8 @@ mod tests {
assert!(Method::ToolSelect.is_mutating());
assert!(Method::ToolEvaluate.is_mutating());
assert!(Method::ToolRecordStart.is_mutating());
assert!(Method::ToolWindowResize.is_mutating());
assert!(Method::ToolEmulate.is_mutating());
}
#[test]
+218
View File
@@ -0,0 +1,218 @@
//! Device-emulation tool (`tool.emulate`).
//!
//! Applies CDP Emulation-domain overrides (viewport metrics, user
//! agent, touch) to a single tab so an agent can debug mobile page
//! behaviour. Overrides are per-target: they do not propagate to new
//! tabs, and `off` restores the tab's real environment
//! (`Emulation.clearDeviceMetricsOverride` + touch disabled + UA
//! override cleared).
//!
//! Device presets are resolved CLI-side; the wire only carries the
//! concrete overrides the extension should execute.
use schemars::JsonSchema;
use serde::{Deserialize, Serialize};
/// One entry of CDP `Emulation.UserAgentBrandVersion` (UA client hints).
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
pub struct UserAgentBrandVersion {
pub brand: String,
pub version: String,
}
/// Mirror of CDP `Emulation.UserAgentMetadata`; every field is optional
/// and only sent when present.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
pub struct UserAgentMetadata {
#[serde(default, skip_serializing_if = "Option::is_none")]
pub brands: Option<Vec<UserAgentBrandVersion>>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub full_version: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub platform: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub platform_version: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub architecture: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub model: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub mobile: Option<bool>,
}
/// Concrete emulation overrides for one tab. Field presence drives the
/// extension: only the overrides whose fields are set are touched, so a
/// call may change just the UA, just the viewport, or any combination.
#[derive(Debug, Clone, Default, PartialEq, Serialize, Deserialize, JsonSchema)]
pub struct EmulateOverrides {
/// Viewport width in CSS pixels. Requires `height`.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub width: Option<u32>,
/// Viewport height in CSS pixels. Requires `width`.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub height: Option<u32>,
/// Device scale factor (device pixel ratio). Only meaningful with
/// `width`/`height`; the extension maps `None` to CDP's `0`
/// ("use the display's native factor").
#[serde(default, skip_serializing_if = "Option::is_none")]
pub device_scale_factor: Option<f64>,
/// Emulate a mobile viewport (meta-viewport handling, overlay
/// scrollbars). Only meaningful with `width`/`height`.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub mobile: Option<bool>,
/// UA string for `Emulation.setUserAgentOverride`.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub user_agent: Option<String>,
/// `Accept-Language` header override. Requires `user_agent`.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub accept_language: Option<String>,
/// UA client-hints metadata. Requires `user_agent`.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub user_agent_metadata: Option<UserAgentMetadata>,
/// Enable/disable touch emulation
/// (`Emulation.setTouchEmulationEnabled`). When omitted while
/// `max_touch_points` is set, touch is enabled.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub touch: Option<bool>,
/// Touch points reported while touch emulation is on (CDP defaults
/// to 1 when omitted).
#[serde(default, skip_serializing_if = "Option::is_none")]
pub max_touch_points: Option<u32>,
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
pub struct EmulateParams {
pub session_id: String,
/// Target tab. Defaults to the Agent Window's active tab.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub tab_id: Option<i64>,
/// Clear every emulation override on the tab and restore its real
/// environment. Mutually exclusive with `overrides`.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub off: Option<bool>,
/// Overrides to apply. Required unless `off` is set.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub overrides: Option<EmulateOverrides>,
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
pub struct EmulateResult {
pub tab_id: i64,
/// True when overrides were cleared (`off`); false when applied.
pub cleared: bool,
/// Echo of the overrides that were applied. Absent when cleared.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub applied: Option<EmulateOverrides>,
/// Scope note: emulation overrides are per-tab (CDP target) and are
/// not inherited by new tabs.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub note: Option<String>,
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
#[test]
fn params_omit_optional_fields() {
let p = EmulateParams {
session_id: "abcd".into(),
tab_id: None,
off: None,
overrides: None,
};
let v = serde_json::to_value(&p).unwrap();
assert_eq!(v, json!({ "session_id": "abcd" }));
let round: EmulateParams = serde_json::from_value(v).unwrap();
assert_eq!(round, p);
}
#[test]
fn overrides_round_trip_with_all_fields() {
let o = EmulateOverrides {
width: Some(390),
height: Some(844),
device_scale_factor: Some(3.0),
mobile: Some(true),
user_agent: Some("Mozilla/5.0 (iPhone…)".into()),
accept_language: Some("zh-CN".into()),
user_agent_metadata: Some(UserAgentMetadata {
brands: Some(vec![UserAgentBrandVersion {
brand: "Safari".into(),
version: "17".into(),
}]),
full_version: None,
platform: Some("iOS".into()),
platform_version: None,
architecture: None,
model: Some("iPhone".into()),
mobile: Some(true),
}),
touch: Some(true),
max_touch_points: Some(5),
};
let v = serde_json::to_value(&o).unwrap();
assert_eq!(v["width"], json!(390));
assert_eq!(v["device_scale_factor"], json!(3.0));
assert_eq!(
v["user_agent_metadata"]["brands"][0]["brand"],
json!("Safari")
);
// Absent metadata fields stay absent.
assert!(v["user_agent_metadata"].get("full_version").is_none());
let round: EmulateOverrides = serde_json::from_value(v).unwrap();
assert_eq!(round, o);
}
#[test]
fn overrides_default_is_empty_object() {
let v = serde_json::to_value(EmulateOverrides::default()).unwrap();
assert_eq!(v, json!({}));
let round: EmulateOverrides = serde_json::from_value(json!({})).unwrap();
assert_eq!(round, EmulateOverrides::default());
}
#[test]
fn off_params_round_trip() {
let p = EmulateParams {
session_id: "abcd".into(),
tab_id: Some(7),
off: Some(true),
overrides: None,
};
let v = serde_json::to_value(&p).unwrap();
assert_eq!(v, json!({ "session_id": "abcd", "tab_id": 7, "off": true }));
let round: EmulateParams = serde_json::from_value(v).unwrap();
assert_eq!(round, p);
}
#[test]
fn result_round_trips() {
let r = EmulateResult {
tab_id: 7,
cleared: false,
applied: Some(EmulateOverrides {
width: Some(390),
height: Some(844),
..EmulateOverrides::default()
}),
note: Some("per-tab".into()),
};
let v = serde_json::to_value(&r).unwrap();
let round: EmulateResult = serde_json::from_value(v).unwrap();
assert_eq!(round, r);
}
#[test]
fn cleared_result_omits_applied() {
let r = EmulateResult {
tab_id: 7,
cleared: true,
applied: None,
note: None,
};
let v = serde_json::to_value(&r).unwrap();
assert_eq!(v, json!({ "tab_id": 7, "cleared": true }));
}
}
@@ -95,6 +95,9 @@ pub enum HelpOutcome {
/// The page navigated while waiting (full reload or SPA URL change).
/// Deprecated: navigation alone should not end a help request.
Navigated,
/// request-help is disabled by configuration (`BSK_REQUEST_HELP=off`):
/// the call returns immediately without ever showing the overlay.
Disabled,
}
impl HelpOutcome {
@@ -105,6 +108,7 @@ impl HelpOutcome {
Self::TimedOut => "timed_out",
Self::Completed => "completed",
Self::Navigated => "navigated",
Self::Disabled => "disabled",
}
}
}
@@ -202,6 +206,10 @@ mod tests {
serde_json::to_value(HelpOutcome::Navigated).unwrap(),
json!("navigated")
);
assert_eq!(
serde_json::to_value(HelpOutcome::Disabled).unwrap(),
json!("disabled")
);
}
#[test]
+4
View File
@@ -2,6 +2,7 @@
pub mod console;
pub mod dialog;
pub mod emulate;
pub mod human_loop;
pub mod interaction;
pub mod navigation;
@@ -12,9 +13,11 @@ pub mod script;
pub mod session;
pub mod tabs;
pub mod waits;
pub mod window;
pub use console::*;
pub use dialog::*;
pub use emulate::*;
pub use human_loop::*;
pub use interaction::*;
pub use navigation::*;
@@ -25,3 +28,4 @@ pub use script::*;
pub use session::*;
pub use tabs::*;
pub use waits::*;
pub use window::*;
+6
View File
@@ -10,6 +10,12 @@ pub struct SessionStartParams {
pub session_id: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub browser_instance_id: Option<String>,
/// Optional Agent Window outer width in CSS pixels (100..=7680).
#[serde(default, skip_serializing_if = "Option::is_none")]
pub width: Option<u32>,
/// Optional Agent Window outer height in CSS pixels (100..=7680).
#[serde(default, skip_serializing_if = "Option::is_none")]
pub height: Option<u32>,
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
+62
View File
@@ -0,0 +1,62 @@
//! Agent Window management tools (`tool.window_*`).
use schemars::JsonSchema;
use serde::{Deserialize, Serialize};
// ----- window_resize -----
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
pub struct WindowResizeParams {
pub session_id: String,
/// New Agent Window outer width in CSS pixels (100..=7680).
pub width: u32,
/// New Agent Window outer height in CSS pixels (100..=7680).
pub height: u32,
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
pub struct WindowResizeResult {
pub window_id: i64,
pub width: u32,
pub height: u32,
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
#[test]
fn window_resize_params_round_trip() {
let params: WindowResizeParams = serde_json::from_value(json!({
"session_id": "ab12",
"width": 1280,
"height": 800,
}))
.unwrap();
assert_eq!(params.session_id, "ab12");
assert_eq!(params.width, 1280);
assert_eq!(params.height, 800);
let encoded = serde_json::to_value(params).unwrap();
assert_eq!(
encoded,
json!({ "session_id": "ab12", "width": 1280, "height": 800 })
);
}
#[test]
fn window_resize_result_round_trip() {
let result: WindowResizeResult = serde_json::from_value(json!({
"window_id": 42,
"width": 1280,
"height": 800,
}))
.unwrap();
assert_eq!(result.window_id, 42);
let encoded = serde_json::to_value(result).unwrap();
assert_eq!(
encoded,
json!({ "window_id": 42, "width": 1280, "height": 800 })
);
}
}
@@ -85,7 +85,7 @@
"notificationTitle": "BrowserSkill:Agent 需要你的帮助"
},
"recordOverlay": {
"recording": "正在录制用户操作",
"recording": "正在录制你的操作",
"finish": "结束"
}
}
+28 -1
View File
@@ -123,11 +123,33 @@ Details and flags: **`bsk <cmd> --help`**
| Command | Summary |
|---------|---------|
| `bsk session start` | Open Agent Window; prints **4-letter session id** |
| `bsk session start` | Open Agent Window (`--width`/`--height` for initial size); prints **4-letter session id** |
| `bsk session stop <id>` | End session, close Agent Window, auto-return borrowed tabs |
| `bsk session stop --all` | Stop every active session |
| `bsk session list` | List active sessions |
### Window (require `--session <id>`)
| Command | Summary |
|---------|---------|
| `bsk window resize` | Resize the Agent Window (`--width`, `--height`; 100..=7680 CSS px) |
### Device emulation — `bsk emulate` (requires `--session <id>`)
Emulate a mobile device environment on the agent tab via CDP — viewport, User-Agent, and touch — to debug a page's mobile layout and behaviour:
```bash
bsk emulate --session <id> --device iphone-14
bsk emulate --session <id> --width 390 --height 844 --dpr 3 --mobile --ua "Mozilla/5.0 (iPhone…" --touch
bsk emulate --session <id> --off
```
- Presets (`--device`): `iphone-14`, `iphone-14-pro-max`, `iphone-se`, `pixel-7`, `galaxy-s23`, `ipad-mini`, `galaxy-tab-s8`. Manual flags (`--width`/`--height`/`--dpr`/`--mobile`/`--ua`/`--accept-language`/`--touch`/`--max-touch-points`) also work without a preset, or override individual preset fields; `--no-mobile`/`--no-touch` turn a preset's mobile viewport / touch emulation back off.
- Repeated runs merge field by field onto the tab's current emulation state — only the flags you pass change. E.g. after `--device iphone-14`, `bsk emulate --session <id> --width 390 --height 844` keeps the preset's dpr (`3`) and mobile viewport. (The extension remembers the state per tab; after an extension reload the next run applies only the fields it carries.)
- `--off` clears every override (viewport, UA, touch) and the remembered state, restoring the tab's real environment.
- Scope: overrides apply to **one tab only** (CDP per-target) and are **not inherited by new tabs** — re-run `bsk emulate` after opening or switching to another tab (default target is the session's active tab; `--tab-id` overrides).
- Emulation covers viewport/UA/touch only; it does not throttle the network, fake geolocation, or synthesise real touch-event streams.
### Tabs (require `--session <id>`)
| Command | Summary |
@@ -219,6 +241,11 @@ which refs/selectors matched a live element.
control. After a `continued` or `completed` result, issue a separate
observation tool call (usually `bsk snapshot --session <id>`) before using
new refs or reasoning about the post-help page state.
#### Disabling request-help (unattended mode)
Set `BSK_REQUEST_HELP=off` on unattended servers: `bsk request-help` then
returns immediately with `outcome="disabled"` (no overlay, no waiting,
exit 0). Any other value keeps it enabled. If you get `disabled`, do not
retry — complete the task autonomously or stop gracefully.
### Recording — `bsk record`