mirror of
https://github.com/Tencent/BrowserSkill.git
synced 2026-09-10 13:01:23 +08:00
feat: add bsk record for semantic user-action traces
Capture user actions in the Agent Window and export a record-only trace (pages[] + navigate/click/fill/select/press steps with target descriptors). Adds the record overlay UI aligned with the control overlay, and wires record start/stop/await through CLI, daemon, protocol, and extension.
This commit is contained in:
@@ -46,6 +46,21 @@ Optional: `bsk session start --browser <instance-id-or-label>` when multiple bro
|
||||
|
||||
Emergency cleanup: `bsk session stop --all` or the Agent Window overlay **Stop all**.
|
||||
|
||||
## Stop when the goal is met
|
||||
|
||||
Every task is a **bounded goal**, not open-ended browsing. The goal may come from the user's request, a recorded `trace.json`, or both.
|
||||
|
||||
1. **Define success first** — one concrete, observable condition derived from the user's words, `purpose`, or the last meaningful step in a trace (e.g. "form submitted", "item added to cart", "playback started").
|
||||
2. **Take the shortest path** — snapshot → act → at most one check. Do not wander, re-try unrelated actions, or stack exploratory steps.
|
||||
3. **Stop as soon as success is reached** — run `bsk session stop <id>` immediately unless the user explicitly asked to keep the session open (e.g. "don't close yet", "keep browsing").
|
||||
4. **No post-success work** — once the goal is met, do not click, refresh, navigate, re-search, switch tabs, or "double-check" that it worked. Further verification is a new task.
|
||||
5. **When blocked, pause — do not brute-force** — if the page requires human input (login, captcha, OTP, payment confirmation) or an action fails twice with no progress, call `bsk request-help` instead of retrying blindly. See **Ask the human for help** below.
|
||||
6. **When unsure** — at most one extra `bsk snapshot`. If success looks met, stop. If not, ask the user; do not keep clicking.
|
||||
|
||||
**With a trace:** replay steps in order using `target` role/name/tag and raw `value`/`selection` fields. After the last step (or when its `effect.navigated_to` / success hint is satisfied), apply rules 3–4 immediately. The trace guides execution; it does not extend control beyond the goal.
|
||||
|
||||
**Without a trace:** the user's request *is* the success condition. Satisfying it ends the task — same stop rules apply.
|
||||
|
||||
## Core interaction loop
|
||||
|
||||
Write operations only affect tabs in the **Agent Window** (or tabs you **borrowed** into it).
|
||||
@@ -191,6 +206,23 @@ acts. The result `outcome` is one of:
|
||||
`note` carries any text the user typed back. `resolved_targets` reports
|
||||
which refs/selectors matched a live element.
|
||||
|
||||
### Recording — `bsk record`
|
||||
|
||||
Capture the user's own actions in the Agent Window to a `trace.json`, for later LLM-driven automation:
|
||||
|
||||
```bash
|
||||
bsk record start --url https://… [--purpose "publish a wiki doc"] [--output trace.json]
|
||||
# `--url` is required on a fresh session (about:blank cannot host the content script).
|
||||
# Blocks until the user clicks Finish in the recording panel, then writes ./trace.json and closes the window.
|
||||
|
||||
bsk record stop [--output trace.json] # terminal fallback if the browser panel is unavailable
|
||||
```
|
||||
|
||||
- The trace is a **record-only action log** (a `pages[]` dictionary + `navigate`/`click`/`fill`/`select`/`press` steps with `target` descriptors). It records *what the user did*; deciding which inputs are variable is left to the executing agent.
|
||||
- `--purpose` is optional context metadata; it does **not** change what gets captured.
|
||||
- There is **no** `bsk replay` — to redo a flow, read the trace and reuse the existing `session` / `snapshot` / `@eN` / `click` / `fill` tools. Follow **Stop when the goal is met**.
|
||||
- Do **not** record on banking/SSO/password-manager pages; passwords are redacted but traces may still contain sensitive text.
|
||||
|
||||
## Error handling
|
||||
|
||||
### Exit codes (`echo $?` after `bsk …`)
|
||||
@@ -221,8 +253,9 @@ Always **`bsk session stop <id>`** in a `finally`-style path so the Agent Window
|
||||
1. **No token theft** — do not `bsk evaluate` on sensitive sites to read `localStorage`, cookies, or auth headers for exfiltration.
|
||||
2. **No long borrow** — do not leave a user's personal tab in the Agent Window across unrelated tasks.
|
||||
3. **No skip stop** — always `bsk session stop <id>`; never assume idle timeout will clean up.
|
||||
4. **No observe escalation before snapshot** — use `bsk snapshot` first; only use `bsk get-html` or `bsk screenshot` when the snapshot is insufficient. Element screenshots (`--ref @eN`) still require a fresh snapshot ref — never skip snapshot just to grab a visual.
|
||||
5. **`evaluate` is powerful and risky** — use only when snapshot + click/fill/select cannot suffice; never on credential surfaces.
|
||||
4. **No post-success control** — once the user’s goal (or last trace step) is met, do not keep operating the page; stop the session unless they asked to keep it open.
|
||||
5. **No observe escalation before snapshot** — use `bsk snapshot` first; only use `bsk get-html` or `bsk screenshot` when the snapshot is insufficient. Element screenshots (`--ref @eN`) still require a fresh snapshot ref — never skip snapshot just to grab a visual.
|
||||
6. **`evaluate` is powerful and risky** — use only when snapshot + click/fill/select cannot suffice; never on credential surfaces.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -19,6 +19,8 @@ pub mod interaction;
|
||||
pub mod logs;
|
||||
pub mod navigate;
|
||||
pub mod network;
|
||||
pub mod record;
|
||||
pub mod record_state;
|
||||
pub mod render_error;
|
||||
pub mod screenshot;
|
||||
pub mod session;
|
||||
@@ -39,6 +41,7 @@ use crate::cli::install_skill::InstallSkillArgs;
|
||||
use crate::cli::interaction::{ClickArgs, FillArgs, PressArgs, SelectArgs};
|
||||
use crate::cli::navigate::{NavigateCommand, NavigateHistoryArgs, ReloadArgs};
|
||||
use crate::cli::network::NetworkArgs;
|
||||
use crate::cli::record::RecordCmd;
|
||||
use crate::cli::screenshot::ScreenshotArgs;
|
||||
use crate::cli::session::SessionCmd;
|
||||
use crate::cli::snapshot::SnapshotArgs;
|
||||
@@ -169,6 +172,9 @@ pub enum Command {
|
||||
/// Ask the human to complete an in-page step (captcha / login / confirm).
|
||||
#[command(name = "request-help")]
|
||||
RequestHelp(RequestHelpArgs),
|
||||
|
||||
/// Record user actions into a semantic `trace.json` textbook for LLMs.
|
||||
Record(RecordCmd),
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Args, Default)]
|
||||
|
||||
@@ -0,0 +1,256 @@
|
||||
//! `bsk record start|stop` — capture user actions in the Agent Window.
|
||||
|
||||
use std::fs;
|
||||
use std::path::PathBuf;
|
||||
use std::time::Duration;
|
||||
|
||||
use anyhow::Context;
|
||||
use bsk_protocol::Method;
|
||||
use bsk_protocol::tools::{
|
||||
RecordAwaitParams, RecordAwaitResult, RecordStartParams, RecordStartResult, RecordStopParams,
|
||||
RecordStopResult, Trace,
|
||||
};
|
||||
use clap::{Args, Subcommand};
|
||||
|
||||
use crate::cli::TOOL_IPC_TIMEOUT;
|
||||
use crate::cli::business_rpc;
|
||||
use crate::cli::ensure_daemon::ensure_daemon;
|
||||
use crate::cli::error::{CliError, Format};
|
||||
use crate::cli::record_state;
|
||||
use crate::cli::session::{start_session, stop_session};
|
||||
|
||||
/// Max wait for the user to click 结束 in the browser (24 hours).
|
||||
const RECORD_AWAIT_TIMEOUT_MS: u32 = 86_400_000;
|
||||
|
||||
/// `bsk record …` subcommand tree.
|
||||
#[derive(Debug, Clone, Args)]
|
||||
pub struct RecordCmd {
|
||||
#[command(subcommand)]
|
||||
pub sub: RecordSub,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Subcommand)]
|
||||
pub enum RecordSub {
|
||||
/// Open the Agent Window, record user actions, and block until finished in the browser.
|
||||
Start(RecordStartArgs),
|
||||
/// Stop recording from the terminal (fallback), write trace JSON, and close the window.
|
||||
/// Works even while `record start` is blocked in `record_await` (daemon forwards
|
||||
/// `tool.record_stop` without the per-session busy gate).
|
||||
Stop(RecordStopArgs),
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Args)]
|
||||
pub struct RecordStartArgs {
|
||||
#[arg(long = "tab-id")]
|
||||
pub tab_id: Option<i64>,
|
||||
|
||||
/// Navigate to this http(s) URL before recording (required for a fresh
|
||||
/// session — Agent Window starts on about:blank, which cannot host the
|
||||
/// content script).
|
||||
#[arg(long)]
|
||||
pub url: Option<String>,
|
||||
|
||||
/// Optional goal text stored on the exported trace for LLM context.
|
||||
#[arg(long)]
|
||||
pub purpose: Option<String>,
|
||||
|
||||
/// Output path for the trace JSON (default `./trace.json`).
|
||||
#[arg(long, default_value = "trace.json")]
|
||||
pub output: PathBuf,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Args)]
|
||||
pub struct RecordStopArgs {
|
||||
/// Output path for the trace JSON (default `./trace.json`).
|
||||
#[arg(long, default_value = "trace.json")]
|
||||
pub output: PathBuf,
|
||||
}
|
||||
|
||||
pub fn dispatch(cmd: RecordCmd, format: Format) -> Result<(), CliError> {
|
||||
match cmd.sub {
|
||||
RecordSub::Start(args) => dispatch_start(args, format),
|
||||
RecordSub::Stop(args) => dispatch_stop(args, format),
|
||||
}
|
||||
}
|
||||
|
||||
fn dispatch_start(args: RecordStartArgs, format: Format) -> Result<(), CliError> {
|
||||
if record_state::read().is_ok() {
|
||||
return Err(CliError::Local(anyhow::anyhow!(
|
||||
"a recording is already in progress; run `bsk record stop` first"
|
||||
)));
|
||||
}
|
||||
|
||||
let info = ensure_daemon().context("ensure daemon is running")?;
|
||||
let session = start_session(info.sock_path.clone(), None)?;
|
||||
|
||||
let start_params = RecordStartParams {
|
||||
session_id: session.session_id.clone(),
|
||||
tab_id: args.tab_id,
|
||||
url: args.url,
|
||||
purpose: args.purpose.clone(),
|
||||
};
|
||||
let start_result = business_rpc::call::<RecordStartParams, RecordStartResult>(
|
||||
info.sock_path.clone(),
|
||||
"record-start",
|
||||
Method::ToolRecordStart,
|
||||
Some(start_params),
|
||||
TOOL_IPC_TIMEOUT,
|
||||
);
|
||||
|
||||
let start_result = match start_result {
|
||||
Ok(result) => result,
|
||||
Err(err) => {
|
||||
let _ = stop_session(info.sock_path, &session.session_id);
|
||||
return Err(err);
|
||||
}
|
||||
};
|
||||
|
||||
if let Err(err) = record_state::write(&session.session_id) {
|
||||
let _ = stop_session(info.sock_path.clone(), &session.session_id);
|
||||
return Err(CliError::Local(err));
|
||||
}
|
||||
|
||||
if format == Format::Human {
|
||||
println!(
|
||||
"recording on tab={} — click 结束 in the browser when done",
|
||||
start_result.tab_id
|
||||
);
|
||||
}
|
||||
|
||||
let await_params = RecordAwaitParams {
|
||||
session_id: session.session_id.clone(),
|
||||
timeout_ms: Some(RECORD_AWAIT_TIMEOUT_MS),
|
||||
};
|
||||
let await_result = business_rpc::call::<RecordAwaitParams, RecordAwaitResult>(
|
||||
info.sock_path.clone(),
|
||||
"record-await",
|
||||
Method::ToolRecordAwait,
|
||||
Some(await_params),
|
||||
record_await_ipc_timeout(RECORD_AWAIT_TIMEOUT_MS),
|
||||
);
|
||||
|
||||
// Keep write/render inside a Result so `?` cannot skip session teardown.
|
||||
let run_result: Result<(), CliError> = match await_result {
|
||||
Ok(result) => (|| {
|
||||
write_trace_file(&args.output, &result.trace)?;
|
||||
render_finish(&result.trace, &args.output, format)
|
||||
})(),
|
||||
Err(err) => Err(err),
|
||||
};
|
||||
|
||||
let session_stop_result = stop_session(info.sock_path, &session.session_id);
|
||||
record_state::clear();
|
||||
|
||||
run_result?;
|
||||
session_stop_result?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn dispatch_stop(args: RecordStopArgs, format: Format) -> Result<(), CliError> {
|
||||
let state = record_state::read().map_err(CliError::Local)?;
|
||||
let info = ensure_daemon().context("ensure daemon is running")?;
|
||||
let session_id = state.session_id.clone();
|
||||
|
||||
let run_result: Result<(), CliError> = (|| {
|
||||
let params = RecordStopParams {
|
||||
session_id: session_id.clone(),
|
||||
};
|
||||
let result = business_rpc::call::<RecordStopParams, RecordStopResult>(
|
||||
info.sock_path.clone(),
|
||||
"record-stop",
|
||||
Method::ToolRecordStop,
|
||||
Some(params),
|
||||
TOOL_IPC_TIMEOUT,
|
||||
)?;
|
||||
write_trace_file(&args.output, &result.trace)?;
|
||||
render_stop(&result, &args.output, format)
|
||||
})();
|
||||
|
||||
let session_stop_result = stop_session(info.sock_path, &session_id);
|
||||
record_state::clear();
|
||||
|
||||
run_result?;
|
||||
session_stop_result?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn record_await_ipc_timeout(timeout_ms: u32) -> Duration {
|
||||
Duration::from_millis(u64::from(timeout_ms))
|
||||
.checked_add(Duration::from_secs(15))
|
||||
.unwrap_or(Duration::from_secs(u64::from(timeout_ms / 1_000) + 15))
|
||||
}
|
||||
|
||||
fn write_trace_file(output: &PathBuf, trace: &Trace) -> Result<(), CliError> {
|
||||
let json = serde_json::to_string_pretty(trace)
|
||||
.context("serialize trace JSON")
|
||||
.map_err(CliError::Local)?;
|
||||
if let Some(parent) = output.parent() {
|
||||
if !parent.as_os_str().is_empty() {
|
||||
fs::create_dir_all(parent)
|
||||
.with_context(|| format!("create output directory {}", parent.display()))
|
||||
.map_err(CliError::Local)?;
|
||||
}
|
||||
}
|
||||
fs::write(output, format!("{json}\n"))
|
||||
.with_context(|| format!("write trace to {}", output.display()))
|
||||
.map_err(CliError::Local)?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn render_finish(trace: &Trace, output: &PathBuf, format: Format) -> Result<(), CliError> {
|
||||
match format {
|
||||
Format::Json => {
|
||||
println!(
|
||||
"{}",
|
||||
serde_json::to_string(&serde_json::json!({
|
||||
"output": output,
|
||||
"trace": trace,
|
||||
"window_closed": true,
|
||||
}))
|
||||
.map_err(|e| CliError::Local(anyhow::anyhow!(e)))?
|
||||
);
|
||||
}
|
||||
Format::Human => {
|
||||
println!("saved {} steps to {}", trace.steps.len(), output.display());
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn render_stop(
|
||||
result: &RecordStopResult,
|
||||
output: &PathBuf,
|
||||
format: Format,
|
||||
) -> Result<(), CliError> {
|
||||
render_finish(&result.trace, output, format)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn default_output_is_trace_json() {
|
||||
let args = RecordStopArgs {
|
||||
output: PathBuf::from("trace.json"),
|
||||
};
|
||||
assert_eq!(args.output, PathBuf::from("trace.json"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn start_args_default_output_is_trace_json() {
|
||||
let args = RecordStartArgs {
|
||||
tab_id: None,
|
||||
url: None,
|
||||
purpose: None,
|
||||
output: PathBuf::from("trace.json"),
|
||||
};
|
||||
assert_eq!(args.output, PathBuf::from("trace.json"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn record_await_ipc_timeout_covers_long_wait() {
|
||||
let got = record_await_ipc_timeout(RECORD_AWAIT_TIMEOUT_MS);
|
||||
assert!(got >= Duration::from_secs(86_400));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,117 @@
|
||||
//! Persist the session id between `bsk record start` and `bsk record stop`.
|
||||
|
||||
use std::fs;
|
||||
use std::time::{SystemTime, UNIX_EPOCH};
|
||||
|
||||
use anyhow::{Context, Result};
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
use crate::daemon::paths;
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct RecordSessionState {
|
||||
pub session_id: String,
|
||||
pub started_at: String,
|
||||
}
|
||||
|
||||
pub fn write(session_id: &str) -> Result<()> {
|
||||
let path = paths::record_session_path()?;
|
||||
paths::ensure_bsk_home()?;
|
||||
if path.exists() {
|
||||
return Err(anyhow::anyhow!(
|
||||
"a recording is already in progress; run `bsk record stop` first"
|
||||
));
|
||||
}
|
||||
let state = RecordSessionState {
|
||||
session_id: session_id.to_string(),
|
||||
started_at: started_at_unix_ms(),
|
||||
};
|
||||
let json = serde_json::to_string_pretty(&state).context("serialize record session state")?;
|
||||
fs::write(&path, format!("{json}\n")).with_context(|| format!("write {}", path.display()))?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub fn read() -> Result<RecordSessionState> {
|
||||
let path = paths::record_session_path()?;
|
||||
if !path.exists() {
|
||||
return Err(anyhow::anyhow!(
|
||||
"no recording in progress; run `bsk record start` first"
|
||||
));
|
||||
}
|
||||
let raw = fs::read_to_string(&path).with_context(|| format!("read {}", path.display()))?;
|
||||
serde_json::from_str(&raw).with_context(|| format!("parse {}", path.display()))
|
||||
}
|
||||
|
||||
pub fn clear() {
|
||||
if let Ok(path) = paths::record_session_path() {
|
||||
let _ = fs::remove_file(path);
|
||||
}
|
||||
}
|
||||
|
||||
fn started_at_unix_ms() -> String {
|
||||
let ms = SystemTime::now()
|
||||
.duration_since(UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.as_millis();
|
||||
format!("{ms}")
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use std::sync::Mutex;
|
||||
|
||||
fn env_guard() -> &'static Mutex<()> {
|
||||
static GUARD: Mutex<()> = Mutex::new(());
|
||||
&GUARD
|
||||
}
|
||||
|
||||
fn with_temp_home<F: FnOnce()>(f: F) {
|
||||
let _lock = env_guard().lock().unwrap_or_else(|e| e.into_inner());
|
||||
let tmp = tempfile::tempdir().unwrap();
|
||||
unsafe {
|
||||
std::env::set_var(crate::daemon::paths::BSK_HOME_ENV, tmp.path());
|
||||
}
|
||||
f();
|
||||
clear();
|
||||
unsafe {
|
||||
std::env::remove_var(crate::daemon::paths::BSK_HOME_ENV);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn write_read_round_trips() {
|
||||
with_temp_home(|| {
|
||||
write("abcd").unwrap();
|
||||
let state = read().unwrap();
|
||||
assert_eq!(state.session_id, "abcd");
|
||||
assert!(!state.started_at.is_empty());
|
||||
});
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn write_rejects_duplicate() {
|
||||
with_temp_home(|| {
|
||||
write("abcd").unwrap();
|
||||
let err = write("efgh").unwrap_err();
|
||||
assert!(err.to_string().contains("already in progress"));
|
||||
});
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn read_errors_when_missing() {
|
||||
with_temp_home(|| {
|
||||
let err = read().unwrap_err();
|
||||
assert!(err.to_string().contains("no recording in progress"));
|
||||
});
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn clear_removes_state_file() {
|
||||
with_temp_home(|| {
|
||||
write("abcd").unwrap();
|
||||
clear();
|
||||
assert!(read().is_err());
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -63,11 +63,11 @@ struct StartParams {
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize)]
|
||||
struct StartReply {
|
||||
session_id: String,
|
||||
browser_instance_id: String,
|
||||
pub struct StartReply {
|
||||
pub session_id: String,
|
||||
pub browser_instance_id: String,
|
||||
#[serde(default)]
|
||||
agent_window_id: Option<i64>,
|
||||
pub agent_window_id: Option<i64>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Serialize)]
|
||||
@@ -78,21 +78,21 @@ struct StopParams {
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize)]
|
||||
struct StopReply {
|
||||
stopped: Vec<String>,
|
||||
pub struct StopReply {
|
||||
pub stopped: Vec<String>,
|
||||
#[serde(default)]
|
||||
failed: Vec<StopFailure>,
|
||||
pub failed: Vec<StopFailure>,
|
||||
#[serde(default)]
|
||||
returned_tab_ids: Vec<i64>,
|
||||
pub returned_tab_ids: Vec<i64>,
|
||||
#[serde(default)]
|
||||
return_failures: Vec<ReturnFailure>,
|
||||
pub return_failures: Vec<ReturnFailure>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize, Serialize)]
|
||||
struct StopFailure {
|
||||
session_id: String,
|
||||
code: ErrorCode,
|
||||
message: String,
|
||||
pub struct StopFailure {
|
||||
pub session_id: String,
|
||||
pub code: ErrorCode,
|
||||
pub message: String,
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize)]
|
||||
@@ -113,14 +113,7 @@ pub fn dispatch(cmd: SessionCmd, format: Format) -> Result<(), CliError> {
|
||||
}
|
||||
|
||||
fn run_start(sock: PathBuf, args: SessionStartArgs, format: Format) -> Result<(), CliError> {
|
||||
let result: Result<StartReply, CliError> = call(
|
||||
sock,
|
||||
Method::SessionStart,
|
||||
Some(StartParams {
|
||||
browser_instance_id: args.browser,
|
||||
}),
|
||||
Duration::from_secs(30),
|
||||
);
|
||||
let result = start_session(sock, args.browser);
|
||||
match result {
|
||||
Ok(reply) => match format {
|
||||
Format::Json => {
|
||||
@@ -143,6 +136,31 @@ fn run_start(sock: PathBuf, args: SessionStartArgs, format: Format) -> Result<()
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Start a session and open the Agent Window. Used by `session start` and `record start`.
|
||||
pub fn start_session(sock: PathBuf, browser: Option<String>) -> Result<StartReply, CliError> {
|
||||
call(
|
||||
sock,
|
||||
Method::SessionStart,
|
||||
Some(StartParams {
|
||||
browser_instance_id: browser,
|
||||
}),
|
||||
Duration::from_secs(30),
|
||||
)
|
||||
}
|
||||
|
||||
/// Stop a single session by id.
|
||||
pub fn stop_session(sock: PathBuf, session_id: &str) -> Result<StopReply, CliError> {
|
||||
call(
|
||||
sock,
|
||||
Method::SessionStop,
|
||||
Some(StopParams {
|
||||
session_id: Some(session_id.to_string()),
|
||||
all: false,
|
||||
}),
|
||||
SESSION_STOP_IPC_TIMEOUT,
|
||||
)
|
||||
}
|
||||
|
||||
/// Render a `session.start` failure (review I3).
|
||||
///
|
||||
/// The previous implementation hand-rolled `eprintln!("error: ...")`
|
||||
|
||||
@@ -242,7 +242,10 @@ pub fn full_handler(status: DaemonStatus, state: Arc<DaemonState>) -> RpcHandler
|
||||
| Method::ToolSelect
|
||||
| Method::ToolEvaluate
|
||||
| Method::ToolWaitForNavigation
|
||||
| Method::ToolRequestHelp => {
|
||||
| Method::ToolRequestHelp
|
||||
| Method::ToolRecordStart
|
||||
| Method::ToolRecordStop
|
||||
| Method::ToolRecordAwait => {
|
||||
handle_tool_dispatch(&state, rpc_id, method, params).await
|
||||
}
|
||||
Method::ToolWaitMs => handle_wait_ms(&state.abort_registry, rpc_id, params).await,
|
||||
@@ -319,10 +322,19 @@ async fn handle_tool_dispatch(
|
||||
}
|
||||
};
|
||||
let entry = inflight_guard.entry();
|
||||
let outcome = state
|
||||
.tool_queues
|
||||
.dispatch(&session_id, method, params, timeout, Some(entry))
|
||||
.await;
|
||||
// `record_stop` must reach the extension while `record_await` holds the
|
||||
// serial busy lock — finishing the recording unblocks await.
|
||||
let outcome = if method == Method::ToolRecordStop {
|
||||
state
|
||||
.tool_queues
|
||||
.dispatch_unlocked(&session_id, method, params, timeout, Some(entry))
|
||||
.await
|
||||
} else {
|
||||
state
|
||||
.tool_queues
|
||||
.dispatch(&session_id, method, params, timeout, Some(entry))
|
||||
.await
|
||||
};
|
||||
drop(inflight_guard);
|
||||
match outcome {
|
||||
Ok(v) => ResponseBody::Ok(v),
|
||||
|
||||
@@ -95,6 +95,11 @@ pub fn sock_path() -> Result<PathBuf> {
|
||||
Ok(bsk_home()?.join("run").join("daemon.sock"))
|
||||
}
|
||||
|
||||
/// Path to the in-progress recording session state (`record-session.json`).
|
||||
pub fn record_session_path() -> Result<PathBuf> {
|
||||
Ok(bsk_home()?.join("record-session.json"))
|
||||
}
|
||||
|
||||
/// Windows named-pipe name. Include the resolved `BSK_HOME` path in the
|
||||
/// token so test homes and custom installs do not share a predictable
|
||||
/// per-username pipe.
|
||||
@@ -163,6 +168,10 @@ mod tests {
|
||||
assert_eq!(log_path().unwrap(), home.join("daemon.log"));
|
||||
assert_eq!(update_check_path().unwrap(), home.join("update-check.json"));
|
||||
assert_eq!(sock_path().unwrap(), home.join("run").join("daemon.sock"));
|
||||
assert_eq!(
|
||||
record_session_path().unwrap(),
|
||||
home.join("record-session.json")
|
||||
);
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
@@ -221,6 +221,13 @@ impl ToolQueueRegistry {
|
||||
guard.get(sid).is_some_and(|entry| entry.accepting)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
fn force_busy_for_tests(&self, sid: &SessionId, busy: bool) {
|
||||
let guard = self.queues.lock().expect("tool queue registry poisoned");
|
||||
let entry = guard.get(sid).expect("queue must exist");
|
||||
entry.state.lock().expect("queue state poisoned").busy = busy;
|
||||
}
|
||||
|
||||
/// Spawn the worker task for a session id. Idempotent: spawning the
|
||||
/// same id twice replaces the previous queue (the previous worker
|
||||
/// closes when its sender drops).
|
||||
@@ -320,6 +327,42 @@ impl ToolQueueRegistry {
|
||||
outcome
|
||||
}
|
||||
|
||||
/// Forward a tool RPC to the extension without taking the
|
||||
/// per-session busy lock. Used by `tool.record_stop` so a second
|
||||
/// CLI can finish an in-flight `tool.record_await` (extension
|
||||
/// dispatch already runs concurrent `void dispatch(msg)` handlers).
|
||||
pub async fn dispatch_unlocked(
|
||||
&self,
|
||||
sid: &SessionId,
|
||||
method: Method,
|
||||
params: Value,
|
||||
timeout: Duration,
|
||||
inflight: Option<Arc<ToolInflightEntry>>,
|
||||
) -> Result<Value, DispatchError> {
|
||||
{
|
||||
let guard = self.queues.lock().expect("tool queue registry poisoned");
|
||||
let Some(entry) = guard.get(sid) else {
|
||||
return Err(DispatchError::SessionNotFound);
|
||||
};
|
||||
if !entry.accepting {
|
||||
return Err(DispatchError::SessionStopping);
|
||||
}
|
||||
}
|
||||
let (respond_tx, _respond_rx) = oneshot::channel();
|
||||
let job = ToolJob {
|
||||
method,
|
||||
params,
|
||||
timeout,
|
||||
respond: respond_tx,
|
||||
inflight,
|
||||
session_id: sid.clone(),
|
||||
};
|
||||
match forward_one(sid, &self.browsers, &self.sessions, &job).await {
|
||||
Ok(v) => Ok(v),
|
||||
Err(err) => Err(DispatchError::Rpc(err)),
|
||||
}
|
||||
}
|
||||
|
||||
/// Stop accepting new jobs for `sid`, enqueue one final control RPC,
|
||||
/// and wait for it behind any already-queued tools. Used by
|
||||
/// `session.stop` so extension teardown cannot race an in-flight
|
||||
@@ -789,3 +832,50 @@ mod tool_job_session_id_tests {
|
||||
assert_eq!(job.session_id.0, "sess-A");
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod dispatch_unlocked_tests {
|
||||
use super::*;
|
||||
use crate::daemon::browsers::BrowserRegistry;
|
||||
use crate::daemon::sessions::{SessionId, SessionRegistry};
|
||||
use bsk_protocol::Method;
|
||||
use serde_json::json;
|
||||
use std::sync::Arc;
|
||||
|
||||
#[tokio::test]
|
||||
async fn unlocked_dispatch_skips_session_busy_gate() {
|
||||
let browsers = Arc::new(BrowserRegistry::new());
|
||||
let sessions = Arc::new(SessionRegistry::new());
|
||||
let queues = ToolQueueRegistry::new(browsers, sessions);
|
||||
let sid = SessionId("s-record".into());
|
||||
queues.spawn(sid.clone());
|
||||
queues.force_busy_for_tests(&sid, true);
|
||||
|
||||
let busy = queues
|
||||
.dispatch(
|
||||
&sid,
|
||||
Method::ToolTabList,
|
||||
json!({ "session_id": "s-record" }),
|
||||
Duration::from_secs(1),
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
assert!(matches!(busy, Err(DispatchError::SessionBusy)));
|
||||
|
||||
let unlocked = queues
|
||||
.dispatch_unlocked(
|
||||
&sid,
|
||||
Method::ToolRecordStop,
|
||||
json!({ "session_id": "s-record" }),
|
||||
Duration::from_secs(1),
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
assert!(
|
||||
!matches!(unlocked, Err(DispatchError::SessionBusy)),
|
||||
"record_stop must not be rejected as SessionBusy; got {unlocked:?}"
|
||||
);
|
||||
// No session row → forward_one returns NotFound as Rpc.
|
||||
assert!(matches!(unlocked, Err(DispatchError::Rpc(_))));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -97,6 +97,7 @@ fn dispatch(cli: Cli, format: Format) -> Result<(), CliError> {
|
||||
Command::WaitForNavigation(args) => cli::waits::dispatch_wait_for_navigation(args, format),
|
||||
Command::WaitMs(args) => cli::waits::dispatch_wait_ms(args, format),
|
||||
Command::RequestHelp(args) => cli::human_loop::dispatch(args, format),
|
||||
Command::Record(cmd) => cli::record::dispatch(cmd, format),
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,21 @@
|
||||
{
|
||||
"$schema": "http://json-schema.org/draft-07/schema#",
|
||||
"title": "RecordAwaitParams",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"session_id"
|
||||
],
|
||||
"properties": {
|
||||
"session_id": {
|
||||
"type": "string"
|
||||
},
|
||||
"timeout_ms": {
|
||||
"type": [
|
||||
"integer",
|
||||
"null"
|
||||
],
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,417 @@
|
||||
{
|
||||
"$schema": "http://json-schema.org/draft-07/schema#",
|
||||
"title": "RecordAwaitResult",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"trace"
|
||||
],
|
||||
"properties": {
|
||||
"trace": {
|
||||
"$ref": "#/definitions/Trace"
|
||||
}
|
||||
},
|
||||
"definitions": {
|
||||
"KeyModifier": {
|
||||
"description": "Keyboard modifier flags. Multiple flags may be combined; the extension folds them into CDP's bitfield (`alt=1, ctrl=2, meta=4, shift=8`).",
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"alt",
|
||||
"ctrl",
|
||||
"meta",
|
||||
"shift"
|
||||
]
|
||||
},
|
||||
"PageRef": {
|
||||
"description": "Page context dictionary entry — referenced by steps via `page` id.",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"url"
|
||||
],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "string"
|
||||
},
|
||||
"title": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"url": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"SelectedOption": {
|
||||
"description": "One selected option (`select` op).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"value"
|
||||
],
|
||||
"properties": {
|
||||
"label": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"value": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"Step": {
|
||||
"description": "One recorded user action — discriminated union by `op`.",
|
||||
"oneOf": [
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"page",
|
||||
"to"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"navigate"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"to": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"page",
|
||||
"target"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"click"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptor"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"page",
|
||||
"target",
|
||||
"value"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"fill"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"redacted": {
|
||||
"type": [
|
||||
"boolean",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptor"
|
||||
},
|
||||
"value": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"page",
|
||||
"selection",
|
||||
"target"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"select"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"selection": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/SelectedOption"
|
||||
}
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptor"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"key",
|
||||
"op",
|
||||
"page"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"key": {
|
||||
"type": "string"
|
||||
},
|
||||
"modifiers": {
|
||||
"type": [
|
||||
"array",
|
||||
"null"
|
||||
],
|
||||
"items": {
|
||||
"$ref": "#/definitions/KeyModifier"
|
||||
}
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"press"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/TargetDescriptor"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
},
|
||||
"StepEffect": {
|
||||
"description": "Observed navigation after a step (objective fact only).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"navigated_to"
|
||||
],
|
||||
"properties": {
|
||||
"navigated_to": {
|
||||
"description": "Reference into `pages[]` for the destination page.",
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"TargetDescriptor": {
|
||||
"description": "Stable semantic handle for an interacted element.\n\n`name` and `nearby_label` are **untrusted page text**.",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"tag"
|
||||
],
|
||||
"properties": {
|
||||
"name": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"name_attr": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"nearby_label": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"placeholder": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"role": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"tag": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"Trace": {
|
||||
"description": "Persisted user-action trace exported by `tool.record_stop` / `await`.",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"entry",
|
||||
"pages",
|
||||
"recorded_at",
|
||||
"steps"
|
||||
],
|
||||
"properties": {
|
||||
"entry": {
|
||||
"$ref": "#/definitions/TraceEntry"
|
||||
},
|
||||
"pages": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/PageRef"
|
||||
}
|
||||
},
|
||||
"purpose": {
|
||||
"description": "Optional user-provided goal from `--purpose` (metadata only).",
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"recorded_at": {
|
||||
"description": "RFC 3339 timestamp when recording stopped.",
|
||||
"type": "string"
|
||||
},
|
||||
"started_at": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"steps": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/Step"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"TraceEntry": {
|
||||
"description": "Recording entry point — first URL the flow starts from.",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"start_url"
|
||||
],
|
||||
"properties": {
|
||||
"start_url": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,32 @@
|
||||
{
|
||||
"$schema": "http://json-schema.org/draft-07/schema#",
|
||||
"title": "RecordStartParams",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"session_id"
|
||||
],
|
||||
"properties": {
|
||||
"purpose": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"session_id": {
|
||||
"type": "string"
|
||||
},
|
||||
"tab_id": {
|
||||
"type": [
|
||||
"integer",
|
||||
"null"
|
||||
],
|
||||
"format": "int64"
|
||||
},
|
||||
"url": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,18 @@
|
||||
{
|
||||
"$schema": "http://json-schema.org/draft-07/schema#",
|
||||
"title": "RecordStartResult",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"recording",
|
||||
"tab_id"
|
||||
],
|
||||
"properties": {
|
||||
"recording": {
|
||||
"type": "boolean"
|
||||
},
|
||||
"tab_id": {
|
||||
"type": "integer",
|
||||
"format": "int64"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,13 @@
|
||||
{
|
||||
"$schema": "http://json-schema.org/draft-07/schema#",
|
||||
"title": "RecordStopParams",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"session_id"
|
||||
],
|
||||
"properties": {
|
||||
"session_id": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,417 @@
|
||||
{
|
||||
"$schema": "http://json-schema.org/draft-07/schema#",
|
||||
"title": "RecordStopResult",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"trace"
|
||||
],
|
||||
"properties": {
|
||||
"trace": {
|
||||
"$ref": "#/definitions/Trace"
|
||||
}
|
||||
},
|
||||
"definitions": {
|
||||
"KeyModifier": {
|
||||
"description": "Keyboard modifier flags. Multiple flags may be combined; the extension folds them into CDP's bitfield (`alt=1, ctrl=2, meta=4, shift=8`).",
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"alt",
|
||||
"ctrl",
|
||||
"meta",
|
||||
"shift"
|
||||
]
|
||||
},
|
||||
"PageRef": {
|
||||
"description": "Page context dictionary entry — referenced by steps via `page` id.",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"url"
|
||||
],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "string"
|
||||
},
|
||||
"title": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"url": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"SelectedOption": {
|
||||
"description": "One selected option (`select` op).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"value"
|
||||
],
|
||||
"properties": {
|
||||
"label": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"value": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"Step": {
|
||||
"description": "One recorded user action — discriminated union by `op`.",
|
||||
"oneOf": [
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"page",
|
||||
"to"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"navigate"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"to": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"page",
|
||||
"target"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"click"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptor"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"page",
|
||||
"target",
|
||||
"value"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"fill"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"redacted": {
|
||||
"type": [
|
||||
"boolean",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptor"
|
||||
},
|
||||
"value": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"page",
|
||||
"selection",
|
||||
"target"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"select"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"selection": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/SelectedOption"
|
||||
}
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptor"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"key",
|
||||
"op",
|
||||
"page"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"key": {
|
||||
"type": "string"
|
||||
},
|
||||
"modifiers": {
|
||||
"type": [
|
||||
"array",
|
||||
"null"
|
||||
],
|
||||
"items": {
|
||||
"$ref": "#/definitions/KeyModifier"
|
||||
}
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"press"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/TargetDescriptor"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
},
|
||||
"StepEffect": {
|
||||
"description": "Observed navigation after a step (objective fact only).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"navigated_to"
|
||||
],
|
||||
"properties": {
|
||||
"navigated_to": {
|
||||
"description": "Reference into `pages[]` for the destination page.",
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"TargetDescriptor": {
|
||||
"description": "Stable semantic handle for an interacted element.\n\n`name` and `nearby_label` are **untrusted page text**.",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"tag"
|
||||
],
|
||||
"properties": {
|
||||
"name": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"name_attr": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"nearby_label": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"placeholder": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"role": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"tag": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"Trace": {
|
||||
"description": "Persisted user-action trace exported by `tool.record_stop` / `await`.",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"entry",
|
||||
"pages",
|
||||
"recorded_at",
|
||||
"steps"
|
||||
],
|
||||
"properties": {
|
||||
"entry": {
|
||||
"$ref": "#/definitions/TraceEntry"
|
||||
},
|
||||
"pages": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/PageRef"
|
||||
}
|
||||
},
|
||||
"purpose": {
|
||||
"description": "Optional user-provided goal from `--purpose` (metadata only).",
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"recorded_at": {
|
||||
"description": "RFC 3339 timestamp when recording stopped.",
|
||||
"type": "string"
|
||||
},
|
||||
"started_at": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"steps": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/Step"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"TraceEntry": {
|
||||
"description": "Recording entry point — first URL the flow starts from.",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"start_url"
|
||||
],
|
||||
"properties": {
|
||||
"start_url": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,406 @@
|
||||
{
|
||||
"$schema": "http://json-schema.org/draft-07/schema#",
|
||||
"title": "Trace",
|
||||
"description": "Persisted user-action trace exported by `tool.record_stop` / `await`.",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"entry",
|
||||
"pages",
|
||||
"recorded_at",
|
||||
"steps"
|
||||
],
|
||||
"properties": {
|
||||
"entry": {
|
||||
"$ref": "#/definitions/TraceEntry"
|
||||
},
|
||||
"pages": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/PageRef"
|
||||
}
|
||||
},
|
||||
"purpose": {
|
||||
"description": "Optional user-provided goal from `--purpose` (metadata only).",
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"recorded_at": {
|
||||
"description": "RFC 3339 timestamp when recording stopped.",
|
||||
"type": "string"
|
||||
},
|
||||
"started_at": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"steps": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/Step"
|
||||
}
|
||||
}
|
||||
},
|
||||
"definitions": {
|
||||
"KeyModifier": {
|
||||
"description": "Keyboard modifier flags. Multiple flags may be combined; the extension folds them into CDP's bitfield (`alt=1, ctrl=2, meta=4, shift=8`).",
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"alt",
|
||||
"ctrl",
|
||||
"meta",
|
||||
"shift"
|
||||
]
|
||||
},
|
||||
"PageRef": {
|
||||
"description": "Page context dictionary entry — referenced by steps via `page` id.",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"url"
|
||||
],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "string"
|
||||
},
|
||||
"title": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"url": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"SelectedOption": {
|
||||
"description": "One selected option (`select` op).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"value"
|
||||
],
|
||||
"properties": {
|
||||
"label": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"value": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"Step": {
|
||||
"description": "One recorded user action — discriminated union by `op`.",
|
||||
"oneOf": [
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"page",
|
||||
"to"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"navigate"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"to": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"page",
|
||||
"target"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"click"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptor"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"page",
|
||||
"target",
|
||||
"value"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"fill"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"redacted": {
|
||||
"type": [
|
||||
"boolean",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptor"
|
||||
},
|
||||
"value": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"page",
|
||||
"selection",
|
||||
"target"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"select"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"selection": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/SelectedOption"
|
||||
}
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptor"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"key",
|
||||
"op",
|
||||
"page"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"key": {
|
||||
"type": "string"
|
||||
},
|
||||
"modifiers": {
|
||||
"type": [
|
||||
"array",
|
||||
"null"
|
||||
],
|
||||
"items": {
|
||||
"$ref": "#/definitions/KeyModifier"
|
||||
}
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"press"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/TargetDescriptor"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
},
|
||||
"StepEffect": {
|
||||
"description": "Observed navigation after a step (objective fact only).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"navigated_to"
|
||||
],
|
||||
"properties": {
|
||||
"navigated_to": {
|
||||
"description": "Reference into `pages[]` for the destination page.",
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"TargetDescriptor": {
|
||||
"description": "Stable semantic handle for an interacted element.\n\n`name` and `nearby_label` are **untrusted page text**.",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"tag"
|
||||
],
|
||||
"properties": {
|
||||
"name": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"name_attr": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"nearby_label": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"placeholder": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"role": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"tag": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"TraceEntry": {
|
||||
"description": "Recording entry point — first URL the flow starts from.",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"start_url"
|
||||
],
|
||||
"properties": {
|
||||
"start_url": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,328 @@
|
||||
{
|
||||
"$schema": "http://json-schema.org/draft-07/schema#",
|
||||
"title": "Step",
|
||||
"description": "One recorded user action — discriminated union by `op`.",
|
||||
"oneOf": [
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"page",
|
||||
"to"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"navigate"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"to": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"page",
|
||||
"target"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"click"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptor"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"page",
|
||||
"target",
|
||||
"value"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"fill"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"redacted": {
|
||||
"type": [
|
||||
"boolean",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptor"
|
||||
},
|
||||
"value": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"op",
|
||||
"page",
|
||||
"selection",
|
||||
"target"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"select"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"selection": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/definitions/SelectedOption"
|
||||
}
|
||||
},
|
||||
"target": {
|
||||
"$ref": "#/definitions/TargetDescriptor"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"description": "Fields shared by every step variant (flattened in JSON).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"key",
|
||||
"op",
|
||||
"page"
|
||||
],
|
||||
"properties": {
|
||||
"effect": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/StepEffect"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": {
|
||||
"type": "integer",
|
||||
"format": "uint32",
|
||||
"minimum": 0.0
|
||||
},
|
||||
"key": {
|
||||
"type": "string"
|
||||
},
|
||||
"modifiers": {
|
||||
"type": [
|
||||
"array",
|
||||
"null"
|
||||
],
|
||||
"items": {
|
||||
"$ref": "#/definitions/KeyModifier"
|
||||
}
|
||||
},
|
||||
"op": {
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"press"
|
||||
]
|
||||
},
|
||||
"page": {
|
||||
"description": "Reference into `pages[]`.",
|
||||
"type": "string"
|
||||
},
|
||||
"target": {
|
||||
"anyOf": [
|
||||
{
|
||||
"$ref": "#/definitions/TargetDescriptor"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"definitions": {
|
||||
"KeyModifier": {
|
||||
"description": "Keyboard modifier flags. Multiple flags may be combined; the extension folds them into CDP's bitfield (`alt=1, ctrl=2, meta=4, shift=8`).",
|
||||
"type": "string",
|
||||
"enum": [
|
||||
"alt",
|
||||
"ctrl",
|
||||
"meta",
|
||||
"shift"
|
||||
]
|
||||
},
|
||||
"SelectedOption": {
|
||||
"description": "One selected option (`select` op).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"value"
|
||||
],
|
||||
"properties": {
|
||||
"label": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"value": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"StepEffect": {
|
||||
"description": "Observed navigation after a step (objective fact only).",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"navigated_to"
|
||||
],
|
||||
"properties": {
|
||||
"navigated_to": {
|
||||
"description": "Reference into `pages[]` for the destination page.",
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"TargetDescriptor": {
|
||||
"description": "Stable semantic handle for an interacted element.\n\n`name` and `nearby_label` are **untrusted page text**.",
|
||||
"type": "object",
|
||||
"required": [
|
||||
"tag"
|
||||
],
|
||||
"properties": {
|
||||
"name": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"name_attr": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"nearby_label": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"placeholder": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"role": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"tag": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -100,4 +100,13 @@ fn main() {
|
||||
dump!(WaitMsResult, "tool_wait_ms_result");
|
||||
dump!(RequestHelpParams, "tool_request_help_params");
|
||||
dump!(RequestHelpResult, "tool_request_help_result");
|
||||
|
||||
dump!(Trace, "trace");
|
||||
dump!(Step, "trace_step");
|
||||
dump!(RecordStartParams, "tool_record_start_params");
|
||||
dump!(RecordStartResult, "tool_record_start_result");
|
||||
dump!(RecordStopParams, "tool_record_stop_params");
|
||||
dump!(RecordStopResult, "tool_record_stop_result");
|
||||
dump!(RecordAwaitParams, "tool_record_await_params");
|
||||
dump!(RecordAwaitResult, "tool_record_await_result");
|
||||
}
|
||||
|
||||
@@ -74,6 +74,12 @@ pub enum Method {
|
||||
ToolWaitMs,
|
||||
#[serde(rename = "tool.request_help")]
|
||||
ToolRequestHelp,
|
||||
#[serde(rename = "tool.record_start")]
|
||||
ToolRecordStart,
|
||||
#[serde(rename = "tool.record_stop")]
|
||||
ToolRecordStop,
|
||||
#[serde(rename = "tool.record_await")]
|
||||
ToolRecordAwait,
|
||||
|
||||
#[serde(rename = "cancel")]
|
||||
Cancel,
|
||||
@@ -121,9 +127,16 @@ impl Method {
|
||||
| Method::ToolFill
|
||||
| Method::ToolPress
|
||||
| Method::ToolSelect
|
||||
| Method::ToolEvaluate => true,
|
||||
| Method::ToolEvaluate
|
||||
// May navigate via optional `url` and changes Agent Window
|
||||
// chrome; gate behind pending-interrupt like other writes.
|
||||
| Method::ToolRecordStart => true,
|
||||
|
||||
// Read-only tool calls — transparent.
|
||||
//
|
||||
// `record_stop` / `record_await` observe / finish a recording
|
||||
// without driving new automation gestures, so they stay
|
||||
// ungated (teardown after interrupt must still work).
|
||||
Method::ToolTabList
|
||||
| Method::ToolSnapshot
|
||||
| Method::ToolGetHtml
|
||||
@@ -132,7 +145,9 @@ impl Method {
|
||||
| Method::ToolNetwork
|
||||
| Method::ToolWaitForNavigation
|
||||
| Method::ToolWaitMs
|
||||
| Method::ToolRequestHelp => false,
|
||||
| Method::ToolRequestHelp
|
||||
| Method::ToolRecordStop
|
||||
| Method::ToolRecordAwait => false,
|
||||
|
||||
// Session lifecycle — not gated.
|
||||
Method::SessionStart
|
||||
@@ -218,6 +233,13 @@ mod tests {
|
||||
assert!(Method::ToolPress.is_mutating());
|
||||
assert!(Method::ToolSelect.is_mutating());
|
||||
assert!(Method::ToolEvaluate.is_mutating());
|
||||
assert!(Method::ToolRecordStart.is_mutating());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn is_mutating_classifies_record_stop_await_as_non_mutating() {
|
||||
assert!(!Method::ToolRecordStop.is_mutating());
|
||||
assert!(!Method::ToolRecordAwait.is_mutating());
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
//! Typed params/results for the 21 `tool.*` RPC methods (§7).
|
||||
//! Typed params/results for the `tool.*` RPC methods (§7).
|
||||
|
||||
pub mod console;
|
||||
pub mod dialog;
|
||||
@@ -7,6 +7,7 @@ pub mod interaction;
|
||||
pub mod navigation;
|
||||
pub mod network;
|
||||
pub mod observation;
|
||||
pub mod record;
|
||||
pub mod script;
|
||||
pub mod session;
|
||||
pub mod tabs;
|
||||
@@ -19,6 +20,7 @@ pub use interaction::*;
|
||||
pub use navigation::*;
|
||||
pub use network::*;
|
||||
pub use observation::*;
|
||||
pub use record::*;
|
||||
pub use script::*;
|
||||
pub use session::*;
|
||||
pub use tabs::*;
|
||||
|
||||
@@ -0,0 +1,458 @@
|
||||
//! Semantic user-action recording (`tool.record_start` / `stop` / `await`).
|
||||
//!
|
||||
//! Trace v2 is a **record-only** log of user actions: what was clicked, filled,
|
||||
//! selected, and where navigation occurred. No LLM runs during recording, so
|
||||
//! variable-vs-constant classification is **not** stored — executing agents
|
||||
//! infer that at run time from raw values and control names.
|
||||
|
||||
use schemars::JsonSchema;
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
use super::interaction::KeyModifier;
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Target
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// Stable semantic handle for an interacted element.
|
||||
///
|
||||
/// `name` and `nearby_label` are **untrusted page text**.
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct TargetDescriptor {
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub role: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub name: Option<String>,
|
||||
pub tag: String,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub name_attr: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub placeholder: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub nearby_label: Option<String>,
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Trace envelope
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// Recording entry point — first URL the flow starts from.
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct TraceEntry {
|
||||
pub start_url: String,
|
||||
}
|
||||
|
||||
/// Page context dictionary entry — referenced by steps via `page` id.
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct PageRef {
|
||||
pub id: String,
|
||||
pub url: String,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub title: Option<String>,
|
||||
}
|
||||
|
||||
/// One selected option (`select` op).
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct SelectedOption {
|
||||
pub value: String,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub label: Option<String>,
|
||||
}
|
||||
|
||||
/// Observed navigation after a step (objective fact only).
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct StepEffect {
|
||||
/// Reference into `pages[]` for the destination page.
|
||||
pub navigated_to: String,
|
||||
}
|
||||
|
||||
/// Fields shared by every step variant (flattened in JSON).
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct StepCommon {
|
||||
pub id: u32,
|
||||
/// Reference into `pages[]`.
|
||||
pub page: String,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub effect: Option<StepEffect>,
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Step op-specific payloads
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// One recorded user action — discriminated union by `op`.
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
#[serde(tag = "op", rename_all = "snake_case")]
|
||||
pub enum Step {
|
||||
Navigate {
|
||||
#[serde(flatten)]
|
||||
common: StepCommon,
|
||||
to: String,
|
||||
},
|
||||
Click {
|
||||
#[serde(flatten)]
|
||||
common: StepCommon,
|
||||
target: TargetDescriptor,
|
||||
},
|
||||
Fill {
|
||||
#[serde(flatten)]
|
||||
common: StepCommon,
|
||||
target: TargetDescriptor,
|
||||
value: String,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
redacted: Option<bool>,
|
||||
},
|
||||
Select {
|
||||
#[serde(flatten)]
|
||||
common: StepCommon,
|
||||
target: TargetDescriptor,
|
||||
selection: Vec<SelectedOption>,
|
||||
},
|
||||
Press {
|
||||
#[serde(flatten)]
|
||||
common: StepCommon,
|
||||
key: String,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
modifiers: Option<Vec<KeyModifier>>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
target: Option<TargetDescriptor>,
|
||||
},
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Trace root
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// Persisted user-action trace exported by `tool.record_stop` / `await`.
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct Trace {
|
||||
/// RFC 3339 timestamp when recording stopped.
|
||||
pub recorded_at: String,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub started_at: Option<String>,
|
||||
/// Optional user-provided goal from `--purpose` (metadata only).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub purpose: Option<String>,
|
||||
pub entry: TraceEntry,
|
||||
pub pages: Vec<PageRef>,
|
||||
pub steps: Vec<Step>,
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// RPC params / results
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct RecordStartParams {
|
||||
pub session_id: String,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub tab_id: Option<i64>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub url: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub purpose: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct RecordStartResult {
|
||||
pub tab_id: i64,
|
||||
pub recording: bool,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct RecordStopParams {
|
||||
pub session_id: String,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct RecordStopResult {
|
||||
pub trace: Trace,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct RecordAwaitParams {
|
||||
pub session_id: String,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub timeout_ms: Option<u32>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, JsonSchema)]
|
||||
pub struct RecordAwaitResult {
|
||||
pub trace: Trace,
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Tests
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use serde_json::json;
|
||||
|
||||
fn sample_common(id: u32) -> StepCommon {
|
||||
StepCommon {
|
||||
id,
|
||||
page: "p1".into(),
|
||||
effect: None,
|
||||
}
|
||||
}
|
||||
|
||||
fn sample_target() -> TargetDescriptor {
|
||||
TargetDescriptor {
|
||||
role: Some("button".into()),
|
||||
name: Some("发布".into()),
|
||||
tag: "button".into(),
|
||||
name_attr: None,
|
||||
placeholder: None,
|
||||
nearby_label: None,
|
||||
}
|
||||
}
|
||||
|
||||
fn sample_trace() -> Trace {
|
||||
Trace {
|
||||
recorded_at: "2026-07-17T09:01:10Z".into(),
|
||||
started_at: Some("2026-07-17T09:00:00Z".into()),
|
||||
purpose: Some("发布一篇文章".into()),
|
||||
entry: TraceEntry {
|
||||
start_url: "https://x.com/editor".into(),
|
||||
},
|
||||
pages: vec![
|
||||
PageRef {
|
||||
id: "p1".into(),
|
||||
url: "https://x.com/editor".into(),
|
||||
title: Some("写文章".into()),
|
||||
},
|
||||
PageRef {
|
||||
id: "p2".into(),
|
||||
url: "https://x.com/p/99".into(),
|
||||
title: None,
|
||||
},
|
||||
],
|
||||
steps: vec![
|
||||
Step::Fill {
|
||||
common: sample_common(1),
|
||||
target: TargetDescriptor {
|
||||
role: Some("textbox".into()),
|
||||
name: Some("标题".into()),
|
||||
tag: "input".into(),
|
||||
name_attr: None,
|
||||
placeholder: None,
|
||||
nearby_label: None,
|
||||
},
|
||||
value: "我的第一篇文章".into(),
|
||||
redacted: None,
|
||||
},
|
||||
Step::Select {
|
||||
common: sample_common(3),
|
||||
target: TargetDescriptor {
|
||||
role: Some("combobox".into()),
|
||||
name: Some("分类".into()),
|
||||
tag: "select".into(),
|
||||
name_attr: None,
|
||||
placeholder: None,
|
||||
nearby_label: None,
|
||||
},
|
||||
selection: vec![SelectedOption {
|
||||
value: "tech".into(),
|
||||
label: Some("技术分享".into()),
|
||||
}],
|
||||
},
|
||||
Step::Click {
|
||||
common: StepCommon {
|
||||
effect: Some(StepEffect {
|
||||
navigated_to: "p2".into(),
|
||||
}),
|
||||
..sample_common(4)
|
||||
},
|
||||
target: sample_target(),
|
||||
},
|
||||
],
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn step_click_round_trips() {
|
||||
let step = Step::Click {
|
||||
common: sample_common(1),
|
||||
target: sample_target(),
|
||||
};
|
||||
let v = serde_json::to_value(&step).unwrap();
|
||||
assert_eq!(v.get("op").and_then(|v| v.as_str()), Some("click"));
|
||||
assert_eq!(v.get("page").and_then(|v| v.as_str()), Some("p1"));
|
||||
assert!(v.get("key").is_none());
|
||||
let round: Step = serde_json::from_value(v).unwrap();
|
||||
assert_eq!(round, step);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn step_fill_with_raw_value_round_trips() {
|
||||
let step = Step::Fill {
|
||||
common: sample_common(2),
|
||||
target: TargetDescriptor {
|
||||
role: Some("textbox".into()),
|
||||
name: Some("服务名称".into()),
|
||||
tag: "input".into(),
|
||||
name_attr: Some("serviceName".into()),
|
||||
placeholder: None,
|
||||
nearby_label: None,
|
||||
},
|
||||
value: "my-svc".into(),
|
||||
redacted: None,
|
||||
};
|
||||
let v = serde_json::to_value(&step).unwrap();
|
||||
assert_eq!(v["op"], "fill");
|
||||
assert_eq!(v["value"], "my-svc");
|
||||
let round: Step = serde_json::from_value(v).unwrap();
|
||||
assert_eq!(round, step);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn step_fill_password_is_redacted() {
|
||||
let step = Step::Fill {
|
||||
common: sample_common(1),
|
||||
target: TargetDescriptor {
|
||||
role: Some("textbox".into()),
|
||||
name: Some("密码".into()),
|
||||
tag: "input".into(),
|
||||
name_attr: None,
|
||||
placeholder: None,
|
||||
nearby_label: None,
|
||||
},
|
||||
value: "***".into(),
|
||||
redacted: Some(true),
|
||||
};
|
||||
let v = serde_json::to_value(&step).unwrap();
|
||||
assert_eq!(v["value"], "***");
|
||||
assert_eq!(v["redacted"], true);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn step_select_uses_object_array() {
|
||||
let step = Step::Select {
|
||||
common: sample_common(3),
|
||||
target: TargetDescriptor {
|
||||
role: Some("combobox".into()),
|
||||
name: Some("分类".into()),
|
||||
tag: "select".into(),
|
||||
name_attr: None,
|
||||
placeholder: None,
|
||||
nearby_label: None,
|
||||
},
|
||||
selection: vec![SelectedOption {
|
||||
value: "tech".into(),
|
||||
label: Some("技术分享".into()),
|
||||
}],
|
||||
};
|
||||
let v = serde_json::to_value(&step).unwrap();
|
||||
assert_eq!(v["selection"][0]["value"], "tech");
|
||||
assert_eq!(v["selection"][0]["label"], "技术分享");
|
||||
let round: Step = serde_json::from_value(v).unwrap();
|
||||
assert_eq!(round, step);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn step_navigate_round_trips() {
|
||||
let step = Step::Navigate {
|
||||
common: sample_common(1),
|
||||
to: "https://example.com".into(),
|
||||
};
|
||||
let v = serde_json::to_value(&step).unwrap();
|
||||
assert_eq!(v["op"], "navigate");
|
||||
assert_eq!(v["to"], "https://example.com");
|
||||
let round: Step = serde_json::from_value(v).unwrap();
|
||||
assert_eq!(round, step);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn step_press_with_modifiers_round_trips() {
|
||||
let step = Step::Press {
|
||||
common: StepCommon {
|
||||
effect: Some(StepEffect {
|
||||
navigated_to: "p2".into(),
|
||||
}),
|
||||
..sample_common(2)
|
||||
},
|
||||
key: "Enter".into(),
|
||||
modifiers: Some(vec![KeyModifier::Ctrl, KeyModifier::Shift]),
|
||||
target: Some(sample_target()),
|
||||
};
|
||||
let v = serde_json::to_value(&step).unwrap();
|
||||
assert_eq!(v["key"], "Enter");
|
||||
assert_eq!(v["modifiers"], json!(["ctrl", "shift"]));
|
||||
assert_eq!(v["effect"]["navigated_to"], "p2");
|
||||
let round: Step = serde_json::from_value(v).unwrap();
|
||||
assert_eq!(round, step);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn trace_record_only_round_trips() {
|
||||
let trace = sample_trace();
|
||||
let v = serde_json::to_value(&trace).unwrap();
|
||||
assert!(v.get("version").is_none());
|
||||
assert_eq!(
|
||||
v.get("purpose").and_then(|v| v.as_str()),
|
||||
Some("发布一篇文章")
|
||||
);
|
||||
assert!(v.get("parameters").is_none());
|
||||
assert!(v.get("site").is_none());
|
||||
assert!(v.get("goal").is_none());
|
||||
assert_eq!(
|
||||
v.get("entry")
|
||||
.and_then(|e| e.get("start_url"))
|
||||
.and_then(|u| u.as_str()),
|
||||
Some("https://x.com/editor")
|
||||
);
|
||||
let round: Trace = serde_json::from_value(v).unwrap();
|
||||
assert_eq!(round, trace);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn discriminated_union_click_ignores_extra_key_field() {
|
||||
let bad = json!({
|
||||
"op": "click",
|
||||
"id": 1,
|
||||
"page": "p1",
|
||||
"target": { "tag": "button", "name": "OK" },
|
||||
"key": "Enter"
|
||||
});
|
||||
let step: Step = serde_json::from_value(bad).unwrap();
|
||||
assert!(matches!(step, Step::Click { .. }));
|
||||
let v = serde_json::to_value(&step).unwrap();
|
||||
assert!(v.get("key").is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn extension_trace_deserializes() {
|
||||
let v = json!({
|
||||
"recorded_at": "2026-07-21T08:00:00Z",
|
||||
"started_at": "2026-07-21T07:59:00Z",
|
||||
"purpose": "demo",
|
||||
"entry": { "start_url": "https://example.com/editor" },
|
||||
"pages": [
|
||||
{ "id": "p1", "url": "https://example.com/editor" },
|
||||
{ "id": "p2", "url": "https://example.com/p/99" }
|
||||
],
|
||||
"steps": [
|
||||
{
|
||||
"op": "fill",
|
||||
"id": 1,
|
||||
"page": "p1",
|
||||
"target": { "tag": "input", "role": "textbox", "name": "标题" },
|
||||
"value": "hello"
|
||||
},
|
||||
{
|
||||
"op": "click",
|
||||
"id": 2,
|
||||
"page": "p1",
|
||||
"target": { "tag": "button", "role": "button", "name": "发布" },
|
||||
"effect": { "navigated_to": "p2" }
|
||||
}
|
||||
]
|
||||
});
|
||||
let trace: Trace = serde_json::from_value(v).unwrap();
|
||||
assert_eq!(trace.entry.start_url, "https://example.com/editor");
|
||||
assert_eq!(trace.pages.len(), 2);
|
||||
assert_eq!(trace.steps.len(), 2);
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user