源码快照 @ 2026-09-01 (source-only, 剔除非源码素材)
Publish Claude Code Binaries / Stage and publish binaries (push) Canceled after 0s
Publish Tutti App Runtime / Build darwin-arm64 (push) Canceled after 0s
Publish Tutti App Runtime / Build darwin-amd64 (push) Canceled after 0s
Publish Tutti App Runtime / Build linux-amd64 (push) Canceled after 0s
Publish Tutti App Runtime / Build windows-amd64 (push) Canceled after 0s
Sync Tutti Agent Skills / sync (push) Canceled after 0s
Windows Agent Adapters / Test Windows agent adapters (push) Canceled after 0s
Windows Daemon Adapters / Test Windows daemon adapters (push) Canceled after 0s
Publish Tutti App Runtime / Publish runtime catalog (push) Canceled after 0s
Publish Claude Code Binaries / Stage and publish binaries (push) Canceled after 0s
Publish Tutti App Runtime / Build darwin-arm64 (push) Canceled after 0s
Publish Tutti App Runtime / Build darwin-amd64 (push) Canceled after 0s
Publish Tutti App Runtime / Build linux-amd64 (push) Canceled after 0s
Publish Tutti App Runtime / Build windows-amd64 (push) Canceled after 0s
Sync Tutti Agent Skills / sync (push) Canceled after 0s
Windows Agent Adapters / Test Windows agent adapters (push) Canceled after 0s
Windows Daemon Adapters / Test Windows daemon adapters (push) Canceled after 0s
Publish Tutti App Runtime / Publish runtime catalog (push) Canceled after 0s
This commit is contained in:
commit
2ed3415fb7
8018 files changed
+1936476
No files matched your search
@@ -0,0 +1,192 @@
|
||||
---
|
||||
name: analyze-performance-traces
|
||||
description: Analyze Chrome, Chromium, Electron, React DevTools, or Perfetto-compatible JSON traces and audit user-reported profiling findings without loading large artifacts into context; prove trigger-to-render/layout chains, separate measured facts from source inference, find exact code choke points, classify forced layout and render fanout, implement semantically safe fixes, and verify behavior plus repository budgets. Use for trace files, reported profiling durations or call chains, dropped frames, long tasks, resize or scroll jank, render storms, layout thrashing, selector hot paths, interaction latency, or requests to locate exact source-level bottlenecks.
|
||||
---
|
||||
|
||||
# Analyze Performance Traces
|
||||
|
||||
End with an evidence chain from trigger to exact source. Fix the earliest proven cause. Never turn an API name or an inclusive duration into a causal claim without checking execution context.
|
||||
|
||||
## Choose the evidence mode
|
||||
|
||||
State which mode applies before making findings:
|
||||
|
||||
1. **Trace-backed**: a trace is supplied or discoverable. Durations, processes, threads, and event ordering may be reported from it.
|
||||
2. **Reported-finding audit**: the user supplies durations or a chain but not the artifact. Treat those details as leads. Confirm current source paths and trigger conditions; label the durations, thread, and invalidation scope unverified.
|
||||
3. **Source-only**: no trace-derived lead exists. Report hypotheses, not measured bottlenecks.
|
||||
|
||||
Do not block a reported-finding audit merely because the original trace file is absent. If the user asks for safe implementation after the source chain is proven, proceed within authorization; require a comparable post-change trace before claiming millisecond or frame-rate improvement.
|
||||
|
||||
If neither a trace nor a concrete lead exists, ask for the smallest reproducible capture unless the user explicitly wants source-only analysis.
|
||||
|
||||
## Start safely
|
||||
|
||||
1. Read repository instructions, nearest area instructions, relevant architecture docs, current diff, and validation policy. Record Git baseline before editing.
|
||||
2. Discover trace artifacts without printing them wholesale. For each candidate:
|
||||
|
||||
```sh
|
||||
ls -lh TRACE.json
|
||||
head -c 512 TRACE.json
|
||||
tail -c 512 TRACE.json
|
||||
```
|
||||
|
||||
3. Run the bundled bounded-memory summarizer:
|
||||
|
||||
```sh
|
||||
node <skill-dir>/scripts/summarize-trace TRACE.json --top 40 --min-ms 16
|
||||
```
|
||||
|
||||
4. Record trace revision/build, production versus development mode, profiling hooks, source maps, screenshots, React tracks, renderer count, and capture window. Separate profiler startup and instrumentation overhead from product work.
|
||||
5. Prefer a repository-owned performance runner when it reproduces the same interaction. List scenarios first; a convenient but different scenario is not proof. In this repository, inspect `docs/conventions/testing.md` and use `pnpm perf:agent-gui -- --list-scenarios` when relevant.
|
||||
|
||||
### Tutti manual capture
|
||||
|
||||
When a new Desktop trace is required:
|
||||
|
||||
```sh
|
||||
TUTTI_ELECTRON_REMOTE_DEBUGGING_PORT=9223 \
|
||||
TUTTI_ELECTRON_JS_FLAGS=--max-old-space-size=8192 \
|
||||
VITE_TUTTI_WHY_DID_YOU_RENDER=0 \
|
||||
make dev-gui
|
||||
```
|
||||
|
||||
Then capture from another terminal:
|
||||
|
||||
```sh
|
||||
pnpm trace:desktop -- --duration 15
|
||||
```
|
||||
|
||||
## Build one evidence chain
|
||||
|
||||
Work in this order:
|
||||
|
||||
1. **Select process/thread**: identify browser main, renderer main, compositor, workers, and GPU threads. Never sum unrelated threads.
|
||||
2. **Select the window**: locate the interaction or burst containing the symptom. Quantify long tasks, layout/style, scripting, paint, frame signals, and repeated events only inside that window.
|
||||
3. **Correlate timestamps**: connect input, timer, stream, resize, or observer delivery to state updates, React commits, DOM mutation, style/layout, paint, and missed frames.
|
||||
4. **Measure fanout**: count repeated components, selectors, entities, DOM nodes, or geometry reads. Express multiplicative patterns such as `34 sections × 67 updates = 2,278 renders`.
|
||||
5. **Map to source**: use stack URLs, source maps, named React tracks, event handlers, class names, and unique strings. Follow symbols with `rg`. Verify current source still matches the traced revision.
|
||||
6. **Inspect cardinality**: use read-only DB/query inspection when scale explains the cost. Remove local paths, secrets, and personal data from durable output.
|
||||
7. **State the chain**:
|
||||
|
||||
```text
|
||||
trigger → state/DOM write → churn or invalidation → render/layout fanout → paint/frame impact
|
||||
```
|
||||
|
||||
Keep measured facts, source-confirmed facts, and inference visibly separate. Do not rank nested events only by inclusive time; use self time when available or label duration inclusive.
|
||||
|
||||
## Audit forced layout precisely
|
||||
|
||||
For every suspected geometry hotspot, record:
|
||||
|
||||
| Field | Question |
|
||||
| -------------- | ------------------------------------------------------------------------------------------------- |
|
||||
| Trigger | What invokes it, and how often? |
|
||||
| Invalidation | Which DOM/style write may have dirtied layout first? |
|
||||
| Read/operation | `scrollHeight`, `client*`, `offset*`, rect/style read, `scrollIntoView`, virtualizer measurement? |
|
||||
| Phase | render, ref commit, layout effect, effect, observer, event, or animation frame? |
|
||||
| Scope | Which scroll/layout subtree must become current? |
|
||||
| Semantics | Why is the read needed: selection, bottom lock, prepend anchor, tooltip, placement? |
|
||||
| Evidence | Trace-backed, source-confirmed path, or inference? |
|
||||
|
||||
Apply these rules:
|
||||
|
||||
- A geometry API can force layout only when relevant layout is dirty. Its presence alone is not proof.
|
||||
- A layout effect after a DOM-heavy commit is high risk because the read occurs before paint while invalidation is pending.
|
||||
- `ResizeObserver` runs after layout calculation; a read-first callback usually consumes current geometry. Writes earlier in the same observer delivery can dirty layout again, so keep observer callbacks read-first, then write.
|
||||
- `requestAnimationFrame`, scroll events, and ordinary effects are not automatically layout-clean.
|
||||
- `scrollIntoView` performs synchronous visibility/scroll calculation. An explicit low-frequency reveal may still be correct and cheaper than changing interaction timing.
|
||||
- Replacing `scrollIntoView` with `offsetTop`, rect reads, or computed style does not inherently remove forced layout.
|
||||
- Distinguish **natural layout work** from **synchronously forced scheduling**. Observer-driven code may coalesce layout without eliminating the layout itself.
|
||||
- Do not claim “whole page/tree layout” without trace scope, layout-object evidence, or containment analysis.
|
||||
|
||||
## Recognize root-cause families
|
||||
|
||||
- Reducer recreates every entity when derived values are unchanged.
|
||||
- Memoized child receives fresh callbacks, arrays, objects, or context projections.
|
||||
- Selector scans/sorts all entities once per consumer, producing `O(S × (T + I))` work.
|
||||
- Streaming or resize commits trigger unconditional scroll-container geometry reads.
|
||||
- A measurement effect reads, writes state/style, then causes another render/layout pass.
|
||||
- The same resize is covered by both `ResizeObserver` and a global resize listener.
|
||||
- Virtualizer DOM writes are followed by parent scroll geometry reads in the same commit.
|
||||
- Development profiling, extensions, or source-map instrumentation dominate the apparent frame.
|
||||
|
||||
Fix the earliest owner. Do not hide producer churn with broad leaf memoization or replace a proven geometry problem with timing heuristics.
|
||||
|
||||
## Choose a fix without semantic drift
|
||||
|
||||
### Data and render churn
|
||||
|
||||
- Reuse references when every rendered/derived value is exactly equal.
|
||||
- Stabilize event-time callbacks without freezing stale reads.
|
||||
- Replace repeated scans with one-pass grouping while preserving filters, orphan rules, stable ordering, and tie-breakers.
|
||||
- Keep exact comparators in named helpers.
|
||||
- Avoid persistent indexes unless evidence proves ephemeral grouping insufficient.
|
||||
|
||||
### DOM geometry and scrolling
|
||||
|
||||
- Separate **semantic pre-paint commands** from **high-frequency content growth**. Conversation switches, explicit submit-to-bottom, focus, and prepend restoration may require layout-phase correction; ordinary streaming growth should not automatically share that path.
|
||||
- Before reading geometry, gate on the semantic branches that genuinely need it. A hot update that requires no scroll side effect should return before any geometry read.
|
||||
- For continuing content/viewport growth, prefer observing the actual content box and viewport. Consume geometry after layout while preserving bottom lock, user scroll-away, prepend anchors, and explicit reveal identity.
|
||||
- Expose a narrow content ref through the owning primitive when needed; avoid brittle queries into private DOM structure.
|
||||
- Remove a global resize listener only when element/parent observation covers the same geometry changes.
|
||||
- Preserve initial observer delivery and text/content-change remeasurement; observing a fixed outer box alone may miss intrinsic overflow changes unless observation is re-established or the measured node resizes.
|
||||
- Avoid CSS containment, `content-visibility`, or size containment around virtualized/dynamic-height content unless measurement, sticky, focus, overlay, and clipping behavior are proven equivalent.
|
||||
- Reuse an existing lifecycle effect/observer owner when repository budgets constrain effect count. Do not raise an architecture baseline to land an optimization.
|
||||
|
||||
### Visibility classification
|
||||
|
||||
Treat a change as strictly behavior-preserving only when rendered values, ordering, scroll/focus timing, lock ownership, mounted state, and side effects remain equivalent.
|
||||
|
||||
Usually safe after exact tests:
|
||||
|
||||
- return before an unused geometry read;
|
||||
- consume observer-delivered geometry while preserving the same lock and anchor transitions;
|
||||
- remove a redundant event source with equivalent observation coverage;
|
||||
- reuse references or replace equivalent scans.
|
||||
|
||||
Potentially user-observable; exclude or request direction unless explicitly authorized:
|
||||
|
||||
- debounce, throttle, or move behavior to a later animation frame/effect;
|
||||
- IntersectionObserver visibility gating;
|
||||
- delayed reveal or autofocus;
|
||||
- always showing a tooltip instead of measuring overflow;
|
||||
- virtualization/unmounting, stale caching, event dropping, or reduced animation rate.
|
||||
|
||||
## Implement with direct performance tests
|
||||
|
||||
1. Preserve architecture and local ownership. Do not add compatibility, fallback, or timing layers without evidence.
|
||||
2. Add tests that expose both semantics and the expensive operation:
|
||||
- spy on geometry getters; after stable mount, a hot streaming rerender should perform zero synchronous reads when no semantic scroll command exists;
|
||||
- mock `ResizeObserver`, deliver it explicitly, and verify bottom lock plus user scroll-away;
|
||||
- verify prepend, selection, focus, reveal revision, tooltip overflow, and native/custom primitive variants affected by the change;
|
||||
- add structural-sharing identity tests for render-churn fixes.
|
||||
3. Scan sibling callsites by mechanism, not API string alone. Classify each as confirmed same trigger, structurally similar but different frequency/semantics, or unverified.
|
||||
4. Run formatter and focused tests while iterating. Inspect the repository's changed-aware dry-run before the final gate.
|
||||
5. Run architecture budgets/boundaries. If a metric increases, redesign or merge ownership; never raise the baseline merely to pass.
|
||||
6. Run the repository-selected final validation once the diff settles. Perform the documentation-impact and durable-lesson check.
|
||||
|
||||
## Validate and report
|
||||
|
||||
Validate at three levels:
|
||||
|
||||
- **Semantic**: identical values, ordering, filtering, focus, scroll lock, reveal, prepend, and event behavior.
|
||||
- **Structural**: stable references, bounded render fanout, and no new duplicate effect/observer owner.
|
||||
- **Performance**: prevented hot-path reads/renders proven by tests; comparable trace when practical.
|
||||
|
||||
Report:
|
||||
|
||||
1. evidence mode and process/thread/window facts;
|
||||
2. exact source chain and trigger frequency;
|
||||
3. measured facts versus source inference;
|
||||
4. fixes grouped by behavior-preserving versus timing-sensitive;
|
||||
5. implementation-plan changes caused by gates or counter-evidence;
|
||||
6. tests, typechecks, boundaries, and changed-aware result;
|
||||
7. same-mechanism sweep: included, excluded, and why;
|
||||
8. documentation impact and line-count distribution when code changed;
|
||||
9. anything unverified, especially missing post-change trace.
|
||||
|
||||
Never claim millisecond, frame-rate, or layout-scope improvement without a comparable post-change capture. It is valid to claim a proven reduction such as “ordinary streaming rerender performs zero synchronous `scrollHeight` reads.”
|
||||
|
||||
## Bundled script
|
||||
|
||||
`scripts/summarize-trace` streams `traceEvents`, keeps bounded top-event state, and outputs JSON with thread metadata, event totals, long tasks, frame signals, and source hints. Use it first for large traces; write narrow follow-up scripts only after a specific hypothesis exists.
|
||||
@@ -0,0 +1,4 @@
|
||||
interface:
|
||||
display_name: "Analyze Performance Traces"
|
||||
short_description: "Trace renderer jank to exact source paths"
|
||||
default_prompt: "Use $analyze-performance-traces to audit this trace or reported performance chain, locate exact source bottlenecks, and propose semantically safe fixes."
|
||||
@@ -0,0 +1,313 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
import { createReadStream, statSync } from "node:fs";
|
||||
import { basename, resolve } from "node:path";
|
||||
|
||||
const { tracePath, top, minMs } = parseArguments(process.argv.slice(2));
|
||||
const absoluteTracePath = resolve(tracePath);
|
||||
const minimumDurationUs = minMs * 1_000;
|
||||
const threadNames = new Map();
|
||||
const nameStats = new Map();
|
||||
const threadStats = new Map();
|
||||
const frameSignals = new Map();
|
||||
const longestEvents = [];
|
||||
let eventCount = 0;
|
||||
let completeEventCount = 0;
|
||||
let longEventCount = 0;
|
||||
let earliestTimestampUs = Number.POSITIVE_INFINITY;
|
||||
let latestTimestampUs = Number.NEGATIVE_INFINITY;
|
||||
|
||||
await streamTraceEvents(absoluteTracePath, processEvent);
|
||||
|
||||
const summary = {
|
||||
file: basename(absoluteTracePath),
|
||||
fileBytes: statSync(absoluteTracePath).size,
|
||||
traceDurationMs:
|
||||
Number.isFinite(earliestTimestampUs) && Number.isFinite(latestTimestampUs)
|
||||
? round((latestTimestampUs - earliestTimestampUs) / 1_000)
|
||||
: null,
|
||||
eventCount,
|
||||
completeEventCount,
|
||||
longEventThresholdMs: minMs,
|
||||
longEventCount,
|
||||
threads: [...threadNames.entries()]
|
||||
.map(([key, name]) => ({ key, name }))
|
||||
.sort((left, right) => left.key.localeCompare(right.key)),
|
||||
topThreadsByInclusiveDuration: topEntries(threadStats, top).map(
|
||||
([key, stats]) => ({
|
||||
key,
|
||||
name: threadNames.get(key) ?? null,
|
||||
count: stats.count,
|
||||
totalMs: round(stats.totalUs / 1_000),
|
||||
maxMs: round(stats.maxUs / 1_000)
|
||||
})
|
||||
),
|
||||
topEventNamesByInclusiveDuration: topEntries(nameStats, top).map(
|
||||
([name, stats]) => ({
|
||||
name,
|
||||
count: stats.count,
|
||||
totalMs: round(stats.totalUs / 1_000),
|
||||
maxMs: round(stats.maxUs / 1_000)
|
||||
})
|
||||
),
|
||||
frameSignals: [...frameSignals.entries()]
|
||||
.map(([name, count]) => ({ name, count }))
|
||||
.sort((left, right) => right.count - left.count),
|
||||
longestEvents: longestEvents.map((event) => ({
|
||||
...event,
|
||||
threadName: threadNames.get(event.thread) ?? null
|
||||
})),
|
||||
cautions: [
|
||||
"Durations are inclusive and nested events may overlap.",
|
||||
"Development profiler and extension overhead may not represent production.",
|
||||
"Use timestamps and stacks to prove causality before editing code."
|
||||
]
|
||||
};
|
||||
|
||||
process.stdout.write(`${JSON.stringify(summary, null, 2)}\n`);
|
||||
|
||||
function processEvent(event) {
|
||||
eventCount += 1;
|
||||
const thread = `${event.pid ?? "?"}:${event.tid ?? "?"}`;
|
||||
if (
|
||||
event.ph === "M" &&
|
||||
event.name === "thread_name" &&
|
||||
typeof event.args?.name === "string"
|
||||
) {
|
||||
threadNames.set(thread, event.args.name);
|
||||
}
|
||||
|
||||
const timestampUs = finiteNumber(event.ts);
|
||||
const durationUs = finiteNumber(event.dur) ?? 0;
|
||||
const name = typeof event.name === "string" ? event.name : "<unnamed>";
|
||||
if (/frame/i.test(name)) {
|
||||
frameSignals.set(name, (frameSignals.get(name) ?? 0) + 1);
|
||||
}
|
||||
if (event.ph !== "X" || durationUs <= 0) return;
|
||||
|
||||
completeEventCount += 1;
|
||||
if (timestampUs !== null) {
|
||||
earliestTimestampUs = Math.min(earliestTimestampUs, timestampUs);
|
||||
latestTimestampUs = Math.max(latestTimestampUs, timestampUs + durationUs);
|
||||
}
|
||||
updateDurationStats(nameStats, name, durationUs);
|
||||
updateDurationStats(threadStats, thread, durationUs);
|
||||
if (durationUs < minimumDurationUs) return;
|
||||
|
||||
longEventCount += 1;
|
||||
longestEvents.push({
|
||||
name,
|
||||
category: typeof event.cat === "string" ? event.cat : null,
|
||||
durationMs: round(durationUs / 1_000),
|
||||
timestampMs: timestampUs === null ? null : round(timestampUs / 1_000),
|
||||
thread,
|
||||
details: collectEventDetails(event.args),
|
||||
sourceHints: collectSourceHints(event.args)
|
||||
});
|
||||
longestEvents.sort((left, right) => right.durationMs - left.durationMs);
|
||||
if (longestEvents.length > top) longestEvents.length = top;
|
||||
}
|
||||
|
||||
function collectEventDetails(value) {
|
||||
const allowedKeys = new Set([
|
||||
"className",
|
||||
"dirtyObjects",
|
||||
"eventType",
|
||||
"frame",
|
||||
"functionName",
|
||||
"id",
|
||||
"nodeId",
|
||||
"nodeName",
|
||||
"reason",
|
||||
"scriptName",
|
||||
"totalObjects",
|
||||
"type",
|
||||
"url"
|
||||
]);
|
||||
const details = {};
|
||||
visit(value, "args", 0);
|
||||
return details;
|
||||
|
||||
function visit(entry, path, depth) {
|
||||
if (
|
||||
depth > 7 ||
|
||||
entry === null ||
|
||||
entry === undefined ||
|
||||
Object.keys(details).length >= 16
|
||||
) {
|
||||
return;
|
||||
}
|
||||
if (Array.isArray(entry)) {
|
||||
for (let index = 0; index < entry.length; index += 1) {
|
||||
visit(entry[index], `${path}[${index}]`, depth + 1);
|
||||
}
|
||||
return;
|
||||
}
|
||||
if (typeof entry !== "object") return;
|
||||
for (const [key, child] of Object.entries(entry)) {
|
||||
const childPath = `${path}.${key}`;
|
||||
if (
|
||||
allowedKeys.has(key) &&
|
||||
(typeof child === "string" || typeof child === "number")
|
||||
) {
|
||||
details[childPath] =
|
||||
typeof child === "string" ? child.slice(0, 300) : child;
|
||||
}
|
||||
visit(child, childPath, depth + 1);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function updateDurationStats(target, key, durationUs) {
|
||||
const stats = target.get(key) ?? { count: 0, totalUs: 0, maxUs: 0 };
|
||||
stats.count += 1;
|
||||
stats.totalUs += durationUs;
|
||||
stats.maxUs = Math.max(stats.maxUs, durationUs);
|
||||
target.set(key, stats);
|
||||
}
|
||||
|
||||
function topEntries(stats, limit) {
|
||||
return [...stats.entries()]
|
||||
.sort((left, right) => right[1].totalUs - left[1].totalUs)
|
||||
.slice(0, limit);
|
||||
}
|
||||
|
||||
function collectSourceHints(value) {
|
||||
const hints = new Set();
|
||||
visit(value, 0);
|
||||
return [...hints].slice(0, 8);
|
||||
|
||||
function visit(entry, depth) {
|
||||
if (depth > 7 || entry === null || entry === undefined || hints.size >= 8) {
|
||||
return;
|
||||
}
|
||||
if (Array.isArray(entry)) {
|
||||
for (const item of entry) visit(item, depth + 1);
|
||||
return;
|
||||
}
|
||||
if (typeof entry !== "object") return;
|
||||
|
||||
const functionName = textValue(entry.functionName ?? entry.name);
|
||||
const url = textValue(entry.url ?? entry.scriptName);
|
||||
const line = finiteNumber(entry.lineNumber);
|
||||
const column = finiteNumber(entry.columnNumber);
|
||||
if (url) {
|
||||
hints.add(
|
||||
`${functionName ? `${functionName} ` : ""}${url}${
|
||||
line === null ? "" : `:${line + 1}`
|
||||
}${column === null ? "" : `:${column + 1}`}`
|
||||
);
|
||||
}
|
||||
for (const child of Object.values(entry)) visit(child, depth + 1);
|
||||
}
|
||||
}
|
||||
|
||||
async function streamTraceEvents(path, onEvent) {
|
||||
const marker = '"traceEvents"';
|
||||
let state = "search-marker";
|
||||
let searchTail = "";
|
||||
let objectText = "";
|
||||
let depth = 0;
|
||||
let inString = false;
|
||||
let escaped = false;
|
||||
|
||||
for await (const chunk of createReadStream(path, { encoding: "utf8" })) {
|
||||
consume(chunk);
|
||||
if (state === "done") break;
|
||||
}
|
||||
if (state === "search-marker" || state === "wait-array") {
|
||||
throw new Error('Trace JSON does not contain a "traceEvents" array');
|
||||
}
|
||||
if (objectText) throw new Error("Trace ended inside a trace event");
|
||||
|
||||
function consume(chunk) {
|
||||
let text = chunk;
|
||||
if (state === "search-marker") {
|
||||
const candidate = searchTail + text;
|
||||
const markerIndex = candidate.indexOf(marker);
|
||||
if (markerIndex < 0) {
|
||||
searchTail = candidate.slice(-marker.length);
|
||||
return;
|
||||
}
|
||||
text = candidate.slice(markerIndex + marker.length);
|
||||
searchTail = "";
|
||||
state = "wait-array";
|
||||
}
|
||||
if (state === "wait-array") {
|
||||
const arrayIndex = text.indexOf("[");
|
||||
if (arrayIndex < 0) return;
|
||||
text = text.slice(arrayIndex + 1);
|
||||
state = "read-array";
|
||||
}
|
||||
if (state !== "read-array") return;
|
||||
|
||||
for (const character of text) {
|
||||
if (!objectText) {
|
||||
if (character === "{") {
|
||||
objectText = character;
|
||||
depth = 1;
|
||||
inString = false;
|
||||
escaped = false;
|
||||
} else if (character === "]") {
|
||||
state = "done";
|
||||
return;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
objectText += character;
|
||||
if (inString) {
|
||||
if (escaped) escaped = false;
|
||||
else if (character === "\\") escaped = true;
|
||||
else if (character === '"') inString = false;
|
||||
continue;
|
||||
}
|
||||
if (character === '"') inString = true;
|
||||
else if (character === "{" || character === "[") depth += 1;
|
||||
else if (character === "}" || character === "]") depth -= 1;
|
||||
|
||||
if (depth === 0) {
|
||||
onEvent(JSON.parse(objectText));
|
||||
objectText = "";
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function parseArguments(args) {
|
||||
if (args.length === 0 || args.includes("--help")) {
|
||||
process.stderr.write(
|
||||
"Usage: summarize-trace TRACE.json [--top N] [--min-ms N]\n"
|
||||
);
|
||||
process.exit(args.includes("--help") ? 0 : 1);
|
||||
}
|
||||
let top = 30;
|
||||
let minMs = 16;
|
||||
for (let index = 1; index < args.length; index += 1) {
|
||||
if (args[index] === "--top") top = positiveNumber(args[++index], "--top");
|
||||
else if (args[index] === "--min-ms") {
|
||||
minMs = positiveNumber(args[++index], "--min-ms");
|
||||
} else throw new Error(`Unknown argument: ${args[index]}`);
|
||||
}
|
||||
return { tracePath: args[0], top, minMs };
|
||||
}
|
||||
|
||||
function positiveNumber(value, flag) {
|
||||
const number = Number(value);
|
||||
if (!Number.isFinite(number) || number <= 0) {
|
||||
throw new Error(`${flag} requires a positive number`);
|
||||
}
|
||||
return number;
|
||||
}
|
||||
|
||||
function finiteNumber(value) {
|
||||
return typeof value === "number" && Number.isFinite(value) ? value : null;
|
||||
}
|
||||
|
||||
function textValue(value) {
|
||||
return typeof value === "string" && value.trim() ? value.trim() : null;
|
||||
}
|
||||
|
||||
function round(value) {
|
||||
return Math.round(value * 1_000) / 1_000;
|
||||
}
|
||||
@@ -0,0 +1,299 @@
|
||||
---
|
||||
name: tutti-app-release
|
||||
description: "Set up, review, run, or debug external repositories that publish a Tutti workspace app through the reusable Tutti App Release GitHub Actions workflow. Use for caller workflows, tutti.app.json manifests, @tutti-os/app-release-tools, S3/CloudFront release hosting, latest.json, versions.json, catalog.json, catalog-only repairs, and App Center visibility issues."
|
||||
---
|
||||
|
||||
# Tutti App Release
|
||||
|
||||
Use this skill when an external app repository publishes a remote Tutti workspace app into App Center release metadata.
|
||||
|
||||
The external repository calls the reusable workflow from the Tutti/Tutti repository:
|
||||
|
||||
```yaml
|
||||
uses: tutti-os/tutti/.github/workflows/publish-tutti-app-release.yml@main
|
||||
```
|
||||
|
||||
That workflow builds one app package, runs `@tutti-os/app-release-tools`, uploads immutable release files plus mutable `latest.json` and `versions.json`, and can optionally merge that app into the shared `catalog.json`.
|
||||
|
||||
## Operating Rules
|
||||
|
||||
Inspect before acting:
|
||||
|
||||
- Read the caller workflow, `tutti.app.json`, package script, and recent GitHub Actions runs.
|
||||
- Confirm whether the user wants a new app release, a catalog refresh, or CI debugging. Do not rerun a release just to repair catalog state.
|
||||
|
||||
For catalog publication:
|
||||
|
||||
- There are three valid modes. Explain them explicitly when helping a user choose.
|
||||
- Release only: `publish_catalog: false`. This uploads a new app version and updates `apps/<appId>/latest.json` and `versions.json`; App Center will see it the next time `catalog.json` is published.
|
||||
- Release and catalog: `publish_catalog: true`. This uploads the app release, then immediately merges that release into `catalog.json`.
|
||||
- Catalog only: `catalog_only: true`. Use this after a release already succeeded when the user forgot to publish catalog, wants to validate first, or wants to refresh catalog without bumping or uploading a new version.
|
||||
- Do not rerun a full app release just to repair catalog state.
|
||||
- If the app caller workflow exposes `catalog_only`, use that. If it does not, use the Tutti catalog workflow directly or add the caller input when the user wants that repo to support catalog-only dispatch.
|
||||
|
||||
For versioning:
|
||||
|
||||
- Production releases should be workflow-driven. Run the production caller workflow with `release_bump` (`patch`, `minor`, or `major`); the reusable workflow calculates the next version from the greater of the packaged manifest version and existing release tags, then creates the tag after the S3 release verifies.
|
||||
- The workflow never bumps or commits app manifest versions. Do not add caller-side release PRs or manifest bump commits to work around protected branches.
|
||||
- Staging releases should leave `release_bump` empty. The workflow uses `manifest.version+<short git sha>` from the packaged manifest and does not create a release tag.
|
||||
|
||||
For reusable workflow changes:
|
||||
|
||||
- Keep long-lived release behavior in the Tutti reusable workflows and app release tools.
|
||||
- Keep caller workflows small and stable: app id, package command/dir, runner/tool versions when needed, and environment-specific AWS/CDN values.
|
||||
- Require callers to declare `min_tutti_version` explicitly. Use `0.0.0` only for releases that are safe for every legacy Tutti client.
|
||||
- Avoid requiring app repositories to change for catalog repair behavior. Use an existing catalog-only path for that.
|
||||
|
||||
## Release Contract
|
||||
|
||||
The caller repository must be compatible with pnpm. The reusable workflow runs `pnpm install --frozen-lockfile` before `package_command`, so the repository should commit `pnpm-lock.yaml` and define the package script used by `package_command`.
|
||||
|
||||
The caller repository must commit a source manifest:
|
||||
|
||||
```text
|
||||
tutti.app.json
|
||||
```
|
||||
|
||||
The generated package directory must contain:
|
||||
|
||||
- `tutti.app.json`
|
||||
- `bootstrap.sh`
|
||||
- `AGENTS.md`
|
||||
- the manifest icon asset, such as `icon.png` or `icon.svg`
|
||||
- all runtime files and assets
|
||||
|
||||
The source and package manifests must use `schemaVersion: "tutti.app.manifest.v1"`. `appId` must match the workflow `app_id` input.
|
||||
|
||||
The workflow writes release objects under:
|
||||
|
||||
```text
|
||||
apps/<appId>/<version>/
|
||||
apps/<appId>/latest.json
|
||||
apps/<appId>/versions.json
|
||||
```
|
||||
|
||||
For production, the workflow derives the next release version by fetching
|
||||
existing stable semver tags with the configured `release_tag_prefix` (default
|
||||
`<appId>-v`), reading the packaged manifest version, and applying
|
||||
`release_bump` to the greater version. It creates the annotated release tag after
|
||||
the S3 release has been uploaded and verified. If `release_bump` is empty, the
|
||||
workflow uses `manifest.version+<short git sha>` from the packaged manifest,
|
||||
which is intended for staging.
|
||||
|
||||
## Reference Caller Workflow
|
||||
|
||||
Use this as the default single-app production caller. Production releases should
|
||||
be workflow-driven so publishing does not need to write version bump commits back
|
||||
to the protected source branch. Keep new app repositories as close to this
|
||||
reference as their package command allows.
|
||||
|
||||
```yaml
|
||||
name: Publish Tutti App Production
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
release_bump:
|
||||
description: Semver bump to publish.
|
||||
required: true
|
||||
type: choice
|
||||
default: patch
|
||||
options:
|
||||
- patch
|
||||
- minor
|
||||
- major
|
||||
publish_catalog:
|
||||
description: Whether to publish the production App Center catalog after this release.
|
||||
required: false
|
||||
type: boolean
|
||||
default: true
|
||||
catalog_only:
|
||||
description: Whether to skip app release upload and only publish the existing latest release to catalog.
|
||||
required: false
|
||||
type: boolean
|
||||
default: false
|
||||
permissions:
|
||||
contents: write
|
||||
id-token: write
|
||||
|
||||
jobs:
|
||||
publish:
|
||||
uses: tutti-os/tutti/.github/workflows/publish-tutti-app-release.yml@main
|
||||
with:
|
||||
app_id: your-app-id
|
||||
min_tutti_version: "REQUIRED_MIN_TUTTI_VERSION"
|
||||
package_command: pnpm package:tutti
|
||||
package_dir: build/tutti-app/package
|
||||
icon_path: build/tutti-app/package/icon.png
|
||||
release_tag_prefix: your-app-id-v
|
||||
release_bump: ${{ inputs.release_bump }}
|
||||
create_release_tag: ${{ !inputs.catalog_only }}
|
||||
publish_catalog: ${{ inputs.publish_catalog }}
|
||||
catalog_only: ${{ inputs.catalog_only }}
|
||||
aws_region: ${{ vars.TUTTI_APP_RELEASES_PRODUCTION_AWS_REGION || vars.TUTTI_APP_RELEASES_AWS_REGION }}
|
||||
aws_role_arn: ${{ vars.TUTTI_APP_RELEASES_PRODUCTION_AWS_ROLE_ARN || vars.TUTTI_APP_RELEASES_AWS_ROLE_ARN }}
|
||||
s3_bucket: ${{ vars.TUTTI_APP_RELEASES_PRODUCTION_S3_BUCKET || vars.TUTTI_APP_RELEASES_S3_BUCKET }}
|
||||
s3_prefix: ${{ vars.TUTTI_APP_RELEASES_PRODUCTION_S3_PREFIX || vars.TUTTI_APP_RELEASES_S3_PREFIX }}
|
||||
release_assets_base_url: ${{ vars.TUTTI_APP_RELEASES_PRODUCTION_BASE_URL || vars.TUTTI_APP_RELEASES_BASE_URL }}
|
||||
catalog_cloudfront_distribution_id: ${{ vars.TUTTI_APP_RELEASES_PRODUCTION_CLOUDFRONT_DISTRIBUTION_ID || vars.TUTTI_APP_RELEASES_CLOUDFRONT_DISTRIBUTION_ID || '' }}
|
||||
```
|
||||
|
||||
Use only `workflow_dispatch` for staging unless every merge should publish
|
||||
staging too. Keep staging and production on separate S3 prefixes and base URLs.
|
||||
|
||||
Pin the reusable workflow ref only when the caller needs reproducible release behavior:
|
||||
|
||||
```yaml
|
||||
uses: tutti-os/tutti/.github/workflows/publish-tutti-app-release.yml@<tag-or-commit-sha>
|
||||
```
|
||||
|
||||
For monorepos that publish multiple apps, keep the caller shape the same but
|
||||
resolve a target matrix from repository config. Each matrix target should expose
|
||||
the same reusable workflow inputs:
|
||||
|
||||
```yaml
|
||||
app_id: ${{ matrix.target.app_id }}
|
||||
min_tutti_version: ${{ matrix.target.min_tutti_version }}
|
||||
package_command: ${{ matrix.target.package_command }}
|
||||
package_dir: ${{ matrix.target.package_dir }}
|
||||
icon_path: ${{ matrix.target.icon_path }}
|
||||
release_tag_prefix: ${{ matrix.target.release_tag_prefix }}
|
||||
release_bump: ${{ inputs.release_bump }}
|
||||
create_release_tag: ${{ !inputs.catalog_only }}
|
||||
```
|
||||
|
||||
## Reusable Workflow Inputs
|
||||
|
||||
Required release inputs:
|
||||
|
||||
- `app_id`: app id, matching the source and package `tutti.app.json`.
|
||||
- `min_tutti_version`: minimum compatible Tutti SemVer for a normal release. It has no permissive default; catalog-only repairs read the stored rule from `versions.json`.
|
||||
- `aws_region`: AWS region for the release bucket.
|
||||
- `aws_role_arn`: IAM role assumed through GitHub OIDC.
|
||||
- `s3_bucket`: bucket receiving release files.
|
||||
|
||||
Conditionally required inputs:
|
||||
|
||||
- `package_command`: builds or copies the final package. Required unless `catalog_only` is true.
|
||||
- `package_dir`: package directory produced by `package_command`. Required unless `catalog_only` is true.
|
||||
- `release_assets_base_url`: public base URL for `s3_bucket` plus `s3_prefix`. Required unless `catalog_only` is true.
|
||||
|
||||
Optional release/version inputs:
|
||||
|
||||
- `icon_path`: package-local icon path override. Use when the manifest icon should be resolved from a generated package path.
|
||||
- `release_tag_prefix`: release tag prefix used when calculating and creating production release tags. Defaults to `<appId>-v`.
|
||||
- `release_bump`: production semver bump. Valid values are `patch`, `minor`, and `major`. Leave empty for staging.
|
||||
- `create_release_tag`: create the annotated release tag after the S3 release verifies. Production callers should set this to true; staging callers should leave it false.
|
||||
|
||||
Optional catalog inputs:
|
||||
|
||||
- `publish_catalog`: after uploading the app release, merge that release into `catalog.json`. Default: `false`.
|
||||
- `catalog_only`: skip package build, release metadata generation, and app release upload; rebuild the app from `apps/<appId>/versions.json` into `catalog.json`. Default: `false`.
|
||||
- `catalog_cloudfront_distribution_id`: CloudFront distribution id for invalidating `/<s3_prefix>/catalog.json` after catalog upload. Default: empty. Caller workflows normally read this from `TUTTI_APP_RELEASES_PRODUCTION_CLOUDFRONT_DISTRIBUTION_ID` or `TUTTI_APP_RELEASES_CLOUDFRONT_DISTRIBUTION_ID`; prefer storing the shared value as an organization variable with selected repository access, and use repository variables only for overrides or when organization variables are unavailable. If neither variable is configured, invalidation is skipped and readers rely on the catalog cache TTL.
|
||||
|
||||
Optional runtime/tooling inputs:
|
||||
|
||||
- `runner`: GitHub runner label. Default: `ubuntu-latest`.
|
||||
- `node_version`: Node.js version. Default: `24`.
|
||||
- `pnpm_version`: pnpm version. Default: `10.11.0`.
|
||||
- `release_tools_package`: app release tools package spec. Default: `@tutti-os/app-release-tools@latest`. Pin this only when debugging or when the reusable workflow depends on a version not yet intended for general use.
|
||||
|
||||
## Catalog Publication
|
||||
|
||||
An app release writes `apps/<appId>/latest.json` and updates `apps/<appId>/versions.json`. App Center sees it only after the shared catalog includes that app.
|
||||
|
||||
Publishing modes:
|
||||
|
||||
- Release only: run the app release workflow with `publish_catalog: false` and `catalog_only: false`. This updates `apps/<appId>/latest.json` and `versions.json`; catalog changes wait until a later catalog publish.
|
||||
- Release and catalog: run the app release workflow with `publish_catalog: true`. This publishes the app and then updates `catalog.json` in the same run.
|
||||
- Catalog only: run with `catalog_only: true` after a release already exists. This reads the existing `apps/<appId>/versions.json` and updates `catalog.json` without rebuilding, uploading, or bumping a new app version.
|
||||
|
||||
Catalog merge semantics:
|
||||
|
||||
- App repository release workflows are scoped to their current `app_id`. `publish_catalog: true` and `catalog_only: true` merge only that app's `apps/<appId>/versions.json` into `catalog.json`.
|
||||
- A release-only app that is not already in `catalog.json` will not be picked up by another app's later `publish_catalog: true` run.
|
||||
- The Tutti catalog workflows can merge one or more explicitly selected app ids through `app_ids`. Use that path when refreshing multiple apps, or run each app's catalog-only dispatch separately.
|
||||
- Neither path scans S3 for every published app index. Every app that should be added or refreshed must be the current caller app id or be explicitly listed in `app_ids`.
|
||||
|
||||
`catalog.json` keeps schema `tutti.app.catalog.v1`. Its `apps[]` array contains the highest active `minTuttiVersion: 0.0.0` release for old clients. New clients read the additive compact `compatibility.apps` frontier. Mutable metadata writes use S3 ETag preconditions because GitHub concurrency groups do not serialize workflows across different repositories.
|
||||
|
||||
Catalog-only can be exposed by the app repository caller workflow, or run from the Tutti catalog workflows:
|
||||
|
||||
- Production: <https://github.com/tutti-os/tutti/actions/workflows/publish-tutti-app-catalog.yml>
|
||||
- Staging: <https://github.com/tutti-os/tutti/actions/workflows/publish-tutti-app-catalog-staging.yml>
|
||||
|
||||
Recommended inputs for refresh/repair:
|
||||
|
||||
- `catalog_mode`: `merge`
|
||||
- `app_ids`: the released remote app id, such as `vibe-design`
|
||||
- Leave AWS, S3, prefix, and CloudFront inputs empty unless overriding organization or repository variables.
|
||||
|
||||
Use `replace` only for deliberate full catalog replacement. Built-in app ids such as `automation` are not published through the remote catalog workflow.
|
||||
|
||||
Catalog writes:
|
||||
|
||||
```text
|
||||
s3://<s3_bucket>/<s3_prefix>/catalog.json
|
||||
```
|
||||
|
||||
If `catalog.json` changed but App Center still shows old metadata, check CloudFront invalidation and confirm `TUTTI_APP_CATALOG_URL` points at the expected staging or production URL.
|
||||
|
||||
## AWS Requirements
|
||||
|
||||
The caller repository needs a GitHub OIDC role that can write release files:
|
||||
|
||||
```text
|
||||
s3://<s3_bucket>/<s3_prefix>/apps/<appId>/<version>/*
|
||||
s3://<s3_bucket>/<s3_prefix>/apps/<appId>/latest.json
|
||||
s3://<s3_bucket>/<s3_prefix>/apps/<appId>/versions.json
|
||||
```
|
||||
|
||||
When `publish_catalog` or `catalog_only` is used in the app release workflow, that role also needs catalog read/write access:
|
||||
|
||||
```text
|
||||
s3://<s3_bucket>/<s3_prefix>/catalog.json
|
||||
```
|
||||
|
||||
CloudFront invalidation requires permission for the matching distribution id. Store shared non-secret configuration such as the CloudFront distribution id in GitHub organization variables with selected repository access when possible; use repository variables only for repository-specific overrides.
|
||||
|
||||
## Local Validation
|
||||
|
||||
Before relying on GitHub Actions, validate the package locally:
|
||||
|
||||
```sh
|
||||
pnpm --package @tutti-os/app-release-tools@latest dlx build-tutti-app-release \
|
||||
--app-id your-app-id \
|
||||
--package-dir build/tutti-app/package \
|
||||
--output-dir /tmp/tutti-app-release \
|
||||
--base-url https://cdn.example.com/tutti-app-releases \
|
||||
--version 0.1.0+local \
|
||||
--git-sha local
|
||||
```
|
||||
|
||||
Expected output:
|
||||
|
||||
- `/tmp/tutti-app-release/apps/<appId>/<version>/<appId>-<version>.zip`
|
||||
- `/tmp/tutti-app-release/apps/<appId>/<version>/release.json`
|
||||
- `/tmp/tutti-app-release/apps/<appId>/latest.json`
|
||||
|
||||
Normal publishing also creates or updates the remote `apps/<appId>/versions.json` compatibility index.
|
||||
|
||||
## Completion Checklist
|
||||
|
||||
- Caller workflow uses `tutti-os/tutti/.github/workflows/publish-tutti-app-release.yml`.
|
||||
- `package_command` produces `package_dir`.
|
||||
- `package_dir/tutti.app.json` is valid JSON and `appId` matches workflow `app_id`.
|
||||
- Package contains `bootstrap.sh`, `AGENTS.md`, icon asset, and runtime files.
|
||||
- `s3_prefix` and `release_assets_base_url` point to the same public release root.
|
||||
- Staging and production use separate prefixes.
|
||||
- Catalog refresh after an existing successful release uses `catalog_only` or the Tutti catalog workflow instead of a new release.
|
||||
|
||||
## Common Failures
|
||||
|
||||
- `manifest appId must match app id`: align workflow `app_id` and manifest `appId`.
|
||||
- `missing tutti.app.json`: fix package generation so the package directory contains a manifest.
|
||||
- `release_bump must be one of major, minor, or patch`: production callers must pass a supported bump type.
|
||||
- `create_release_tag requires release_bump`: release tags are only created for workflow-driven production bumps.
|
||||
- `manifest icon asset missing`: include the asset inside the package or pass `icon_path`.
|
||||
- AWS `AccessDenied`: check OIDC trust policy, role ARN, bucket policy, region, prefix, and catalog/CloudFront permissions.
|
||||
- Release succeeded but App Center does not show it: publish or refresh the catalog with `catalog_only` or the Tutti catalog workflow in merge mode.
|
||||
@@ -0,0 +1,7 @@
|
||||
interface:
|
||||
display_name: "Tutti App Release"
|
||||
short_description: "Publish external Tutti workspace apps"
|
||||
default_prompt: "Use $tutti-app-release to set up or review an external repository's Tutti App Release GitHub Actions workflow."
|
||||
|
||||
policy:
|
||||
allow_implicit_invocation: true
|
||||
@@ -0,0 +1,169 @@
|
||||
---
|
||||
name: tutti-architecture-review
|
||||
description: Review tutti git diffs for project structure, layering, module ownership, and duplicate event-center infrastructure by planning focused architecture review tasks, then having the main agent orchestrate sub-agents for only the changed areas.
|
||||
---
|
||||
|
||||
# Tutti Architecture Review
|
||||
|
||||
Use this skill when reviewing a `tutti` change or a named module for repository structure, module ownership, or layering compliance. This is a focused architecture review, not a general bug hunt.
|
||||
|
||||
## Vocabulary
|
||||
|
||||
Use the architecture vocabulary consistently:
|
||||
|
||||
- **Module**: anything with an interface and an implementation.
|
||||
- **Interface**: everything a caller must know to use the module correctly.
|
||||
- **Implementation**: the code inside a module.
|
||||
- **Depth**: leverage at the interface; deep modules hide useful behavior behind a small interface.
|
||||
- **Seam**: where an interface lives.
|
||||
- **Adapter**: a concrete thing satisfying an interface at a seam.
|
||||
- **Leverage**: what callers get from depth.
|
||||
- **Locality**: what maintainers get from depth.
|
||||
|
||||
Prefer these words in findings. Avoid vague substitutes such as "component", "service", "utility", or "boundary" when a vocabulary term fits.
|
||||
|
||||
## Workflow
|
||||
|
||||
1. Resolve the user's review intent:
|
||||
- plain `git diff` review for the current change
|
||||
- module-focused diff review when the user names a module inside the current change
|
||||
- static module review when the user wants a named module inspected even without current diff overlap
|
||||
|
||||
Use light natural-language guidance when the request is ambiguous. Do not force the user through a fixed mode menu.
|
||||
|
||||
2. When the user names a module, let the main agent infer a few candidate path keywords and gather candidate paths. Then normalize those paths into a scope file:
|
||||
|
||||
```bash
|
||||
node ./.codex/skills/tutti-architecture-review/scripts/build-review-scope.mjs \
|
||||
--input /tmp/tutti-review-candidates.json \
|
||||
--output /tmp/tutti-review-scope.json
|
||||
```
|
||||
|
||||
Candidate input is agent-produced. The script does not invent module keywords or search the repository itself; it only normalizes candidate paths into a stable scope contract for the planner.
|
||||
|
||||
Read `references/scope-contract.md` when changing or consuming the candidate input, normalized scope output, or planner scope metadata.
|
||||
|
||||
3. Run the review planner from the repository root:
|
||||
|
||||
```bash
|
||||
pnpm review:architecture:package
|
||||
```
|
||||
|
||||
For module-focused review, pass the generated scope file:
|
||||
|
||||
```bash
|
||||
node ./.codex/skills/tutti-architecture-review/scripts/plan-review.mjs \
|
||||
--scope-file /tmp/tutti-review-scope.json \
|
||||
--format json \
|
||||
--output-temp
|
||||
```
|
||||
|
||||
The planner remains `git diff` first. With `--scope-file`, it reviews `scope ∩ diff` when there is overlap, and falls back to scoped-file review only when there is no diff overlap.
|
||||
|
||||
For explicit static module review, force scope-only planning:
|
||||
|
||||
```bash
|
||||
node ./.codex/skills/tutti-architecture-review/scripts/plan-review.mjs \
|
||||
--scope-file /tmp/tutti-review-scope.json \
|
||||
--scope-mode static-only \
|
||||
--format json \
|
||||
--output-temp
|
||||
```
|
||||
|
||||
4. Read the generated JSON task package from `workflowEntry.packagePath`. Each task includes `riskLevel`, `spawnRecommendation`, `summaryForMainAgent`, `matchedFiles`, `preflightSignals`, and a ready-to-use `prompt`.
|
||||
|
||||
5. Spawn `explorer` sub-agents according to `spawnRecommendation`:
|
||||
- `required`: spawn unless the user explicitly asked for a narrower review
|
||||
- `recommended`: spawn when the review is not trivially small
|
||||
- `optional`: the main agent may review locally
|
||||
|
||||
Do not ask sub-agents to edit files. Their job is to inspect the relevant diff and report architecture findings.
|
||||
|
||||
6. Continue local work while sub-agents run only if there is non-overlapping review or summarization work. Do not duplicate a sub-agent's assigned scope.
|
||||
|
||||
7. Merge sub-agent reports into a code-review style answer:
|
||||
- findings first, ordered by severity
|
||||
- cite file paths and line numbers when possible
|
||||
- explain the violated rule and why it matters for locality, leverage, or dependency direction
|
||||
- include "No architecture findings" when a reviewer found no issues
|
||||
|
||||
## Reviewer Expectations
|
||||
|
||||
Every sub-agent should:
|
||||
|
||||
- read `AGENTS.md` and the closest area `AGENTS.md` for its changed files
|
||||
- read only the reference files listed in the task package, plus files needed to understand the diff
|
||||
- inspect the relevant git diff directly instead of relying only on file names
|
||||
- report only actionable architecture issues, not taste preferences
|
||||
- distinguish hard rule violations from speculative deepening opportunities
|
||||
- avoid proposing new interfaces unless the changed code already creates pressure for a real seam
|
||||
- when eventing, pub-sub, or bidirectional coordination appears, check whether the shared business event stream's `global`, `desktop`, or `workspace` scope modules already own the seam before accepting new event-center infrastructure
|
||||
|
||||
## Task Planner
|
||||
|
||||
The planner is deterministic and repository-local:
|
||||
|
||||
- `scripts/plan-review.mjs` reads `git diff`, optional untracked files, and an optional normalized scope file
|
||||
- `scripts/build-review-scope.mjs` normalizes agent-produced candidate paths into a stable scope contract
|
||||
- `references/review-rules.json` declares reviewer tasks, path rules, and regex-style preflight signals
|
||||
- the script maps changed paths to architecture reviewer tasks
|
||||
- it adds lightweight preflight signals for suspicious imports, generated-contract drift, possible hardcoded copy, and cross-area seams
|
||||
- it assigns task risk and spawn recommendations for the main agent
|
||||
- it emits JSON or Markdown for the main agent to orchestrate
|
||||
- it never starts sub-agents itself
|
||||
|
||||
Useful options:
|
||||
|
||||
```bash
|
||||
pnpm review:architecture
|
||||
pnpm review:architecture:package
|
||||
pnpm review:architecture:test
|
||||
node ./.codex/skills/tutti-architecture-review/scripts/plan-review.mjs --format markdown
|
||||
node ./.codex/skills/tutti-architecture-review/scripts/plan-review.mjs --format summary
|
||||
node ./.codex/skills/tutti-architecture-review/scripts/plan-review.mjs --base origin/main
|
||||
node ./.codex/skills/tutti-architecture-review/scripts/plan-review.mjs --staged
|
||||
node ./.codex/skills/tutti-architecture-review/scripts/plan-review.mjs --no-untracked
|
||||
node ./.codex/skills/tutti-architecture-review/scripts/plan-review.mjs --scope-file /tmp/tutti-review-scope.json --format summary
|
||||
node ./.codex/skills/tutti-architecture-review/scripts/plan-review.mjs --scope-file /tmp/tutti-review-scope.json --scope-mode static-only --format summary
|
||||
node ./.codex/skills/tutti-architecture-review/scripts/plan-review.mjs --task desktop-layering --format markdown
|
||||
node ./.codex/skills/tutti-architecture-review/scripts/plan-review.mjs --output /tmp/tutti-review-tasks.json
|
||||
node ./.codex/skills/tutti-architecture-review/scripts/plan-review.mjs --output-temp
|
||||
node ./.codex/skills/tutti-architecture-review/scripts/plan-review.mjs --from-package /tmp/tutti-review-tasks.json --format markdown
|
||||
node ./.codex/skills/tutti-architecture-review/scripts/plan-review.mjs --from-package /tmp/tutti-review-tasks.json --task desktop-layering --format summary
|
||||
```
|
||||
|
||||
## Task Package Entry
|
||||
|
||||
A task package file is a stable workflow entrypoint. Prefer creating one before spawning sub-agents, especially for large reviews:
|
||||
|
||||
```bash
|
||||
node ./.codex/skills/tutti-architecture-review/scripts/plan-review.mjs --format json --output-temp
|
||||
```
|
||||
|
||||
Then use `workflowEntry.packagePath` as the source of truth for the review. If the conversation resumes later, reload the same package instead of recomputing the plan:
|
||||
|
||||
```bash
|
||||
node ./.codex/skills/tutti-architecture-review/scripts/plan-review.mjs --from-package /tmp/tutti-architecture-review-YYYYMMDDTHHMMSSZ.json --format markdown
|
||||
```
|
||||
|
||||
Use `--format summary` before spawning agents when you need a compact orchestration view. Use `--task <id>` to inspect or rerender one reviewer from either the current diff or an existing task package.
|
||||
|
||||
Run the planner self-test after changing task matching, risk rules, output formats, or task-package behavior:
|
||||
|
||||
```bash
|
||||
node --test ./.codex/skills/tutti-architecture-review/scripts/plan-review.test.mjs ./.codex/skills/tutti-architecture-review/scripts/build-review-scope.test.mjs
|
||||
```
|
||||
|
||||
Read `references/tutti-layering.md` when a task needs the compact project rules.
|
||||
Read `references/scope-contract.md` when changing the scope-file contract or the main-agent handoff around module review.
|
||||
Read `docs/architecture/business-event-stream.md` when reviewing event-center modules, typed pub-sub, or WebSocket-based product coordination. Do not use the architecture review to enforce generated event-protocol drift; that belongs to `pnpm check:event-protocol-generated`.
|
||||
|
||||
Read or edit `references/review-rules.json` when changing:
|
||||
|
||||
- reviewer task titles, focus, references, or path matching
|
||||
- simple preflight regex rules
|
||||
- task assignment for preflight signals
|
||||
|
||||
Keep combination logic in `scripts/plan-review.mjs`, such as cross-cutting trigger reasons, generated-source pairing, risk calculation, spawn recommendations, and output rendering.
|
||||
|
||||
Keep module-phrase interpretation in the main agent, not in `build-review-scope.mjs` or `plan-review.mjs`. Those scripts should stay deterministic and repository-local.
|
||||
@@ -0,0 +1,7 @@
|
||||
interface:
|
||||
display_name: "Tutti Architecture Review"
|
||||
short_description: "Review tutti layering and event-center ownership"
|
||||
default_prompt: "Use $tutti-architecture-review to review the current git diff, or focus the review on a user-named tutti module when the request calls for it."
|
||||
|
||||
policy:
|
||||
allow_implicit_invocation: true
|
||||
@@ -0,0 +1,294 @@
|
||||
{
|
||||
"version": 1,
|
||||
"tasks": [
|
||||
{
|
||||
"id": "repo-structure",
|
||||
"title": "Repo Structure Reviewer",
|
||||
"area": "repository",
|
||||
"priority": 10,
|
||||
"pathRules": [
|
||||
{ "type": "exact", "value": "AGENTS.md" },
|
||||
{ "type": "exact", "value": "pnpm-workspace.yaml" },
|
||||
{ "type": "exact", "value": "package.json" },
|
||||
{ "type": "prefix", "value": ".github/" },
|
||||
{ "type": "prefix", "value": "docs/architecture/" },
|
||||
{ "type": "prefix", "value": "docs/conventions/" },
|
||||
{ "type": "newTopLevelArea" },
|
||||
{ "type": "packageManifest" }
|
||||
],
|
||||
"referenceFiles": [
|
||||
"AGENTS.md",
|
||||
"docs/architecture/project-structure.md",
|
||||
"docs/conventions/README.md",
|
||||
".codex/skills/tutti-architecture-review/references/tutti-layering.md"
|
||||
],
|
||||
"focus": [
|
||||
"Top-level ownership across apps, services, packages, config, tools, and docs.",
|
||||
"Whether new directories represent durable ownership rather than speculative structure.",
|
||||
"Whether a new or changed package has a real shared seam and a narrow interface.",
|
||||
"Whether business logic is moving away from services/tuttid into an app or generic package."
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "desktop-layering",
|
||||
"title": "Desktop Layering Reviewer",
|
||||
"area": "apps/desktop",
|
||||
"priority": 20,
|
||||
"pathRules": [{ "type": "prefix", "value": "apps/desktop/" }],
|
||||
"referenceFiles": [
|
||||
"AGENTS.md",
|
||||
"apps/desktop/AGENTS.md",
|
||||
"docs/conventions/desktop-layering.md",
|
||||
"docs/conventions/desktop-visual-language.md",
|
||||
"docs/conventions/ui-system.md",
|
||||
".codex/skills/tutti-architecture-review/references/tutti-layering.md"
|
||||
],
|
||||
"focus": [
|
||||
"main/preload/renderer/shared responsibility direction.",
|
||||
"Whether apps/desktop is becoming a second business core.",
|
||||
"Renderer feature service ownership and services/internal privacy.",
|
||||
"Typed preload API usage instead of direct Electron, daemon client, or transport access from renderer.",
|
||||
"User-visible copy and UI-system ownership when UI files change."
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "tuttid-layering",
|
||||
"title": "Tuttid Layering Reviewer",
|
||||
"area": "services/tuttid",
|
||||
"priority": 30,
|
||||
"pathRules": [{ "type": "prefix", "value": "services/tuttid/" }],
|
||||
"referenceFiles": [
|
||||
"AGENTS.md",
|
||||
"services/tuttid/AGENTS.md",
|
||||
"docs/conventions/tuttid-layering.md",
|
||||
"docs/conventions/api-contracts.md",
|
||||
"docs/conventions/local-state-storage.md",
|
||||
".codex/skills/tutti-architecture-review/references/tutti-layering.md"
|
||||
],
|
||||
"focus": [
|
||||
"api/service/biz/data ownership and dependency direction.",
|
||||
"Whether business workflows, persistence, and transport concerns are placed in the owning layer.",
|
||||
"Whether the OpenAPI source changes before daemon HTTP contract implementation changes.",
|
||||
"Whether tests live at the layer they validate.",
|
||||
"Whether a new seam is real or only a shallow pass-through."
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "package-boundaries",
|
||||
"title": "Package Boundary Reviewer",
|
||||
"area": "packages",
|
||||
"priority": 40,
|
||||
"pathRules": [{ "type": "prefix", "value": "packages/" }],
|
||||
"referenceFiles": [
|
||||
"AGENTS.md",
|
||||
"packages/AGENTS.md",
|
||||
"docs/architecture/project-structure.md",
|
||||
"docs/conventions/ui-system.md",
|
||||
".codex/skills/tutti-architecture-review/references/tutti-layering.md"
|
||||
],
|
||||
"focus": [
|
||||
"Whether package changes represent a real multi-consumer seam.",
|
||||
"Whether package names are narrow and responsibility-based.",
|
||||
"Whether UI package changes preserve root exports and avoid exposing internal layout.",
|
||||
"Whether business workflows are leaking into shared packages."
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "business-event-stream",
|
||||
"title": "Business Event Stream Reviewer",
|
||||
"area": "events",
|
||||
"priority": 45,
|
||||
"pathRules": [
|
||||
{ "type": "prefix", "value": "packages/events/" },
|
||||
{ "type": "prefix", "value": "services/tuttid/api/events/" },
|
||||
{ "type": "prefix", "value": "services/tuttid/service/eventstream/" },
|
||||
{ "type": "exact", "value": "services/tuttid/api/routes.go" },
|
||||
{
|
||||
"type": "exact",
|
||||
"value": "packages/clients/tuttid-ts/src/eventStreamClient.ts"
|
||||
},
|
||||
{
|
||||
"type": "exact",
|
||||
"value": "docs/architecture/business-event-stream.md"
|
||||
}
|
||||
],
|
||||
"referenceFiles": [
|
||||
"AGENTS.md",
|
||||
"packages/AGENTS.md",
|
||||
"services/tuttid/AGENTS.md",
|
||||
"docs/architecture/business-event-stream.md",
|
||||
".codex/skills/tutti-architecture-review/references/tutti-layering.md"
|
||||
],
|
||||
"focus": [
|
||||
"Whether new event-center, event-bus, pub-sub, or WebSocket coordination infrastructure duplicates the shared business event stream.",
|
||||
"Whether the existing `global`, `desktop`, and `workspace` scope modules are considered before adding a new event-center seam.",
|
||||
"Whether tuttid remains the business authority for domain-state events while desktop publishes only typed intents."
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "contracts-and-generated-sources",
|
||||
"title": "Contracts And Generated Sources Reviewer",
|
||||
"area": "contracts",
|
||||
"priority": 50,
|
||||
"pathRules": [
|
||||
{ "type": "exact", "value": "config/tutti.defaults.json" },
|
||||
{ "type": "prefix", "value": "config/" },
|
||||
{ "type": "contains", "value": "/generated/" },
|
||||
{
|
||||
"type": "exact",
|
||||
"value": "services/tuttid/api/openapi/tuttid.v1.yaml"
|
||||
},
|
||||
{ "type": "prefix", "value": "services/tuttid/api/openapi/" },
|
||||
{ "type": "suffix", "value": "pnpm-lock.yaml" },
|
||||
{ "type": "suffix", "value": "package.json" },
|
||||
{ "type": "contains", "value": "/i18n/" },
|
||||
{ "type": "prefix", "value": "tools/scripts/" },
|
||||
{ "type": "prefix", "value": "docs/conventions/" }
|
||||
],
|
||||
"referenceFiles": [
|
||||
"AGENTS.md",
|
||||
"docs/conventions/api-contracts.md",
|
||||
"docs/conventions/runtime-overrides.md",
|
||||
"docs/conventions/local-git-hooks.md",
|
||||
"docs/conventions/static-analysis.md",
|
||||
".codex/skills/tutti-architecture-review/references/tutti-layering.md"
|
||||
],
|
||||
"focus": [
|
||||
"Whether source-of-truth files changed before generated outputs or contract consumers.",
|
||||
"Whether environment override growth is documented.",
|
||||
"Whether i18n and user-visible copy rules are preserved.",
|
||||
"Whether repository-managed checks update durable convention docs when the check behavior changes."
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": "cross-cutting-architecture",
|
||||
"title": "Cross-Cutting Architecture Reviewer",
|
||||
"area": "cross-cutting",
|
||||
"priority": 60,
|
||||
"requiresCrossCuttingTrigger": true,
|
||||
"pathRules": [
|
||||
{
|
||||
"type": "exact",
|
||||
"value": "services/tuttid/api/openapi/tuttid.v1.yaml"
|
||||
},
|
||||
{ "type": "prefix", "value": "services/tuttid/api/generated/" },
|
||||
{ "type": "prefix", "value": "services/tuttid/api/routes" },
|
||||
{ "type": "prefix", "value": "services/tuttid/api/" },
|
||||
{ "type": "prefix", "value": "services/tuttid/service/" },
|
||||
{ "type": "prefix", "value": "apps/desktop/src/shared/contracts/" },
|
||||
{ "type": "prefix", "value": "apps/desktop/src/preload/" },
|
||||
{ "type": "prefix", "value": "apps/desktop/src/main/ipc/" },
|
||||
{
|
||||
"type": "prefix",
|
||||
"value": "packages/clients/tuttid-ts/src/generated/"
|
||||
},
|
||||
{ "type": "suffix", "value": "package.json" },
|
||||
{ "type": "packageManifest" }
|
||||
],
|
||||
"referenceFiles": [
|
||||
"AGENTS.md",
|
||||
"docs/architecture/project-structure.md",
|
||||
"docs/conventions/desktop-layering.md",
|
||||
"docs/conventions/tuttid-layering.md",
|
||||
".codex/skills/tutti-architecture-review/references/tutti-layering.md"
|
||||
],
|
||||
"focus": [
|
||||
"Whether changes spanning areas preserve a single business core in services/tuttid.",
|
||||
"Whether desktop and daemon communicate through contracts instead of duplicated domain behavior.",
|
||||
"Whether shared abstractions improve locality and leverage rather than spreading shallow modules.",
|
||||
"Whether the changed seams are named after durable domain concepts."
|
||||
]
|
||||
}
|
||||
],
|
||||
"signals": [
|
||||
{
|
||||
"id": "renderer-direct-platform-import",
|
||||
"severity": "high",
|
||||
"taskIds": ["desktop-layering", "cross-cutting-architecture"],
|
||||
"pathRules": [
|
||||
{ "type": "prefix", "value": "apps/desktop/src/renderer/" }
|
||||
],
|
||||
"lineRegex": "from\\s+[\"'](?:electron|node:|@electron/)|require\\([\"']electron",
|
||||
"message": "Renderer code appears to import Electron or Node directly."
|
||||
},
|
||||
{
|
||||
"id": "renderer-imports-daemon-client-package",
|
||||
"severity": "medium",
|
||||
"taskIds": ["desktop-layering", "cross-cutting-architecture"],
|
||||
"pathRules": [
|
||||
{ "type": "prefix", "value": "apps/desktop/src/renderer/" }
|
||||
],
|
||||
"lineRegex": "from\\s+[\"']@tutti/client-tuttid-ts|[\"']@tutti/client-tuttid-ts[\"']",
|
||||
"message": "Renderer code imports the daemon client package. Verify this is contract/type usage and not daemon client construction."
|
||||
},
|
||||
{
|
||||
"id": "renderer-feature-internal-import",
|
||||
"severity": "medium",
|
||||
"taskIds": ["desktop-layering"],
|
||||
"pathRules": [
|
||||
{ "type": "prefix", "value": "apps/desktop/src/renderer/" }
|
||||
],
|
||||
"excludePathContains": ["/services/internal/"],
|
||||
"lineRegex": "services/internal",
|
||||
"message": "Renderer code references a feature services/internal path outside that internal implementation."
|
||||
},
|
||||
{
|
||||
"id": "data-imports-api",
|
||||
"severity": "high",
|
||||
"taskIds": ["tuttid-layering"],
|
||||
"pathRules": [{ "type": "prefix", "value": "services/tuttid/data/" }],
|
||||
"lineRegex": "services/tuttid/api|/api/|from\\s+[\"'].*api",
|
||||
"message": "Data layer appears to depend on the API layer."
|
||||
},
|
||||
{
|
||||
"id": "biz-imports-outer-layer",
|
||||
"severity": "high",
|
||||
"taskIds": ["tuttid-layering"],
|
||||
"pathRules": [{ "type": "prefix", "value": "services/tuttid/biz/" }],
|
||||
"lineRegex": "services/tuttid/(?:api|service|data)|/(?:api|service|data)/",
|
||||
"message": "Biz layer appears to depend on api, service, or data."
|
||||
},
|
||||
{
|
||||
"id": "possible-hardcoded-user-copy",
|
||||
"severity": "low",
|
||||
"taskIds": ["desktop-layering", "contracts-and-generated-sources"],
|
||||
"pathRules": [{ "type": "prefix", "value": "apps/desktop/" }],
|
||||
"excludePathContains": ["/i18n/"],
|
||||
"fileExtensions": [".ts", ".tsx"],
|
||||
"lineRegex": "\\b(?:throw new Error|title|label|message|description|placeholder)\\s*[:=]\\s*[\"'][A-Za-z][^\"']{8,}[\"']",
|
||||
"message": "Possible user-visible string outside the desktop i18n layer. Verify whether it is product copy."
|
||||
},
|
||||
{
|
||||
"id": "possible-duplicate-event-infra",
|
||||
"severity": "medium",
|
||||
"taskIds": [
|
||||
"desktop-layering",
|
||||
"tuttid-layering",
|
||||
"package-boundaries",
|
||||
"business-event-stream",
|
||||
"cross-cutting-architecture"
|
||||
],
|
||||
"pathRules": [
|
||||
{ "type": "prefix", "value": "apps/" },
|
||||
{ "type": "prefix", "value": "services/" },
|
||||
{ "type": "prefix", "value": "packages/" }
|
||||
],
|
||||
"excludePathContains": [
|
||||
"packages/events/protocol/",
|
||||
"services/tuttid/api/events/",
|
||||
"services/tuttid/service/eventstream/",
|
||||
"packages/clients/tuttid-ts/src/eventStreamClient"
|
||||
],
|
||||
"fileExtensions": [".ts", ".tsx", ".js", ".jsx", ".mjs", ".go"],
|
||||
"lineRegex": "\\b(?:EventBus|EventCenter|EventHub|PubSub|createEventStream|eventSocket)\\b|event bus|event center|pub-sub|pubsub",
|
||||
"message": "New event-center or pub-sub infrastructure may duplicate the shared business event stream. Verify whether the existing global/desktop/workspace scope modules should be reused."
|
||||
},
|
||||
{
|
||||
"id": "cross-area-import",
|
||||
"severity": "medium",
|
||||
"taskIds": ["repo-structure", "cross-cutting-architecture"],
|
||||
"kind": "crossAreaImport",
|
||||
"message": "Import appears to cross a top-level product area. Verify this seam is intentional."
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,107 @@
|
||||
# Review Scope Contract
|
||||
|
||||
`build-review-scope.mjs` and `plan-review.mjs --scope-file` communicate through a stable JSON contract.
|
||||
|
||||
Keep this contract small and deterministic:
|
||||
|
||||
- the main agent owns module-phrase interpretation and repository search
|
||||
- `build-review-scope.mjs` owns normalization, deduplication, and scope shrinking
|
||||
- `plan-review.mjs` owns review planning inside the supplied scope
|
||||
|
||||
## Candidate Input
|
||||
|
||||
The main agent may gather candidate paths however it wants, then write a candidate file for `build-review-scope.mjs`.
|
||||
|
||||
Required shape:
|
||||
|
||||
```json
|
||||
{
|
||||
"version": 1,
|
||||
"query": "workspace module",
|
||||
"keywords": ["workspace", "workspaces"],
|
||||
"candidates": [
|
||||
{
|
||||
"path": "apps/desktop/src/renderer/features/workspaces/index.ts",
|
||||
"reason": "filename matched workspaces"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- `version` must be `1`
|
||||
- `query` is the user-facing module phrase or review request
|
||||
- `keywords` is an optional list of agent-expanded path keywords
|
||||
- `candidates[].path` is repository-relative and uses `/`
|
||||
- directory candidates should end with `/`
|
||||
- `reason` should be short and explain why the candidate was included
|
||||
|
||||
## Normalized Scope Output
|
||||
|
||||
`build-review-scope.mjs` emits the normalized scope consumed by `plan-review.mjs --scope-file`.
|
||||
|
||||
```json
|
||||
{
|
||||
"version": 1,
|
||||
"query": "workspace module",
|
||||
"keywords": ["workspace", "workspaces"],
|
||||
"strategy": "agent-expanded-path-candidates",
|
||||
"scopes": [
|
||||
{
|
||||
"path": "apps/desktop/src/renderer/features/workspaces/",
|
||||
"kind": "directory",
|
||||
"reason": "filename matched workspaces",
|
||||
"sourcePaths": [
|
||||
"apps/desktop/src/renderer/features/workspaces/index.ts",
|
||||
"apps/desktop/src/renderer/features/workspaces/services/internal/load.ts"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- `version` must be `1`
|
||||
- `strategy` describes how the scope was produced, not how the planner should behave
|
||||
- `scopes[].kind` is `directory` or `file`
|
||||
- `scopes[].path` is repository-relative and normalized
|
||||
- `directory` scope paths end with `/`
|
||||
- `sourcePaths` records the candidate paths that were collapsed into the scope
|
||||
|
||||
## Planner Output Metadata
|
||||
|
||||
When `plan-review.mjs` consumes a scope file, the generated task package exposes a structured `reviewScope` block plus compact workflow hints:
|
||||
|
||||
```json
|
||||
{
|
||||
"workflowEntry": {
|
||||
"scopeFile": "/tmp/tutti-review-scope.json",
|
||||
"scopeMode": "auto",
|
||||
"scopeSelectionMode": "diff-intersection",
|
||||
"scopeSummary": "workspace module: 1 scope, mode auto, selection diff-intersection"
|
||||
},
|
||||
"reviewScope": {
|
||||
"query": "workspace module",
|
||||
"keywords": ["workspace", "workspaces"],
|
||||
"strategy": "agent-expanded-path-candidates",
|
||||
"scopeMode": "auto",
|
||||
"selectionMode": "diff-intersection",
|
||||
"scopeCount": 1,
|
||||
"scopes": [
|
||||
{
|
||||
"path": "apps/desktop/src/renderer/features/workspaces/",
|
||||
"kind": "directory"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Interpretation:
|
||||
|
||||
- `scopeMode` is the planner mode requested by the caller, such as `auto` or `static-only`
|
||||
- `selectionMode` is the actual planner choice, such as `diff-intersection`, `scope-fallback`, or `static-only`
|
||||
- `scopeSummary` is only a convenience string for the main agent
|
||||
- `reviewScope` is the structured source of truth
|
||||
@@ -0,0 +1,130 @@
|
||||
# Tutti Layering Reference
|
||||
|
||||
This is a compact reference for architecture reviewers. The durable source of truth remains the repository docs under `docs/architecture` and `docs/conventions`.
|
||||
|
||||
## Repository Shape
|
||||
|
||||
- `apps/desktop` owns Electron shell, renderer UI, preload bridge, OS integration, and daemon supervision.
|
||||
- `services/tuttid` owns business rules, durable local state, domain workflows, and persistence.
|
||||
- `packages/*` exists only for real shared seams with narrow names.
|
||||
- `config` contains machine-consumable repository defaults, not user settings or documentation.
|
||||
- `tools` contains repository support scripts, not permanent product behavior.
|
||||
- Do not create vague packages such as `shared`, `common`, `utils`, or `client-sdk`.
|
||||
|
||||
Review questions:
|
||||
|
||||
- Does the change put business logic in `services/tuttid` rather than `apps/desktop`?
|
||||
- Does a new package have multiple real consumers and a narrow interface?
|
||||
- Does a new directory represent durable ownership, or is it speculative structure?
|
||||
- Does a helper module have depth, or is it a shallow pass-through?
|
||||
- When a workspace package includes frontend orchestration, is it still host-reusable rather than coupled to one concrete product integration?
|
||||
|
||||
## Desktop Layering
|
||||
|
||||
Authoritative docs:
|
||||
|
||||
- `docs/conventions/desktop-layering.md`
|
||||
- `apps/desktop/AGENTS.md`
|
||||
- `docs/conventions/desktop-visual-language.md`
|
||||
- `docs/conventions/ui-system.md`
|
||||
|
||||
Allowed ownership:
|
||||
|
||||
- `src/main`: Electron-specific capabilities, app bootstrap, windows, IPC registration, transport endpoint resolution, daemon supervision, updates, logging.
|
||||
- `src/preload`: renderer-facing typed desktop SDK; hides IPC channel names.
|
||||
- `src/renderer`: consumes typed preload APIs; owns React UI and renderer-local feature services.
|
||||
- `src/shared`: narrow desktop-local bridge contracts and i18n resources.
|
||||
|
||||
Hard checks:
|
||||
|
||||
- `main` must not implement business workflows or durable domain state.
|
||||
- `preload` must not expose a generic `invoke(channel, payload)` surface.
|
||||
- `renderer` must not import Electron APIs, construct daemon clients, or resolve transport endpoints.
|
||||
- renderer UI must not import another feature's `services/internal/**`.
|
||||
- renderer must use `@tutti-os/ui-system` instead of recreating design tokens, icons, or primitives locally.
|
||||
- user-visible copy belongs in the i18n layer.
|
||||
|
||||
Preferred renderer feature shape:
|
||||
|
||||
```text
|
||||
renderer/src/features/<feature>/
|
||||
index.ts
|
||||
services/
|
||||
<feature>Service.interface.ts
|
||||
<feature>Types.ts
|
||||
register<Feature>Services.ts
|
||||
internal/
|
||||
<feature>Service.ts
|
||||
<feature>Store.ts
|
||||
<feature>Model.ts
|
||||
adapters/
|
||||
ui/
|
||||
```
|
||||
|
||||
## Tuttid Layering
|
||||
|
||||
Authoritative docs:
|
||||
|
||||
- `docs/conventions/tuttid-layering.md`
|
||||
- `services/tuttid/AGENTS.md`
|
||||
- `docs/conventions/api-contracts.md`
|
||||
- `docs/conventions/local-state-storage.md`
|
||||
|
||||
Ownership:
|
||||
|
||||
- `main.go`: process bootstrap only.
|
||||
- `wiring.go`: composition root; may know concrete implementations.
|
||||
- `app`: process lifecycle around HTTP server.
|
||||
- `server`: HTTP server assembly and middleware.
|
||||
- `api`: request decoding, response encoding, HTTP status selection, route dispatch, generated DTOs.
|
||||
- `service`: use-case orchestration, domain validation, DTO translation, collaborator calls.
|
||||
- `biz`: small transport-agnostic domain models/rules shared across layers.
|
||||
- `data`: concrete persistence adapters, SQLite, migrations, file-backed repositories.
|
||||
- `integration`: cross-layer black-box tests.
|
||||
- `types`: cross-domain support only.
|
||||
|
||||
Dependency direction:
|
||||
|
||||
```text
|
||||
main -> wiring -> app/server/api -> service -> data -> biz
|
||||
```
|
||||
|
||||
Hard checks:
|
||||
|
||||
- `data` must not depend on `api`.
|
||||
- `biz` must not depend on `api`, `service`, or `data`.
|
||||
- `api` should not know concrete persistence details except narrow sentinel error mapping.
|
||||
- `service` should not perform direct SQL or HTTP response writing.
|
||||
- simple behavior should not be split into a full domain slice before the seam is real.
|
||||
|
||||
Important exception:
|
||||
|
||||
- `WorkbenchSnapshot*` is a repository-owned shared contract synchronized from
|
||||
`packages/workbench/snapshot`.
|
||||
- Reusing the synchronized Go snapshot contract in `service/workspace` for
|
||||
validation and canonicalization is allowed when it avoids a parallel
|
||||
hand-maintained mirror.
|
||||
- The exception does not move HTTP decoding, status-code mapping, or route-local
|
||||
validation ownership out of `api`.
|
||||
|
||||
## Contract And Generated Source Rules
|
||||
|
||||
- Change `services/tuttid/api/openapi/tuttid.v1.yaml` before changing daemon HTTP request or response contracts.
|
||||
- `WorkbenchSnapshot*` is the main exception: change
|
||||
`packages/workbench/snapshot/src/schema.json` first, then sync the OpenAPI and
|
||||
Go generated artifacts that depend on it.
|
||||
- Before adding local event buses, event centers, product pub-sub routes, or
|
||||
duplicate WebSocket coordination, check whether the shared business event
|
||||
stream and its `global`, `desktop`, or `workspace` scope modules already own
|
||||
the seam.
|
||||
- Shared default-source changes under `config/tutti.defaults.json` require generated outputs to be refreshed.
|
||||
- Supported environment override growth must be documented in `docs/conventions/runtime-overrides.md`.
|
||||
- User-visible copy and locale-resource changes should go through the relevant i18n layer.
|
||||
- Repository-managed checks and hooks should update the durable convention docs when their rules change.
|
||||
|
||||
## Review Severity
|
||||
|
||||
- `P0`: architecture break that blocks the change from working or shipping.
|
||||
- `P1`: hard layering or contract violation that will spread business logic or break callers.
|
||||
- `P2`: real maintainability issue: shallow module, misplaced ownership, test locality loss, or likely future duplication.
|
||||
- `P3`: speculative deepening opportunity; useful but not required for this change.
|
||||
@@ -0,0 +1,317 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
import { mkdirSync, readFileSync, writeFileSync } from "node:fs";
|
||||
import { dirname, resolve } from "node:path";
|
||||
|
||||
const USAGE = `Usage: node ./.codex/skills/tutti-architecture-review/scripts/build-review-scope.mjs [options]
|
||||
|
||||
Options:
|
||||
--input <path> Read candidate paths from a JSON file.
|
||||
--output <path> Write the normalized scope JSON to a file instead of stdout.
|
||||
--help Show this help.
|
||||
`;
|
||||
|
||||
function main() {
|
||||
const options = parseArgs(process.argv.slice(2));
|
||||
if (options.help) {
|
||||
process.stdout.write(USAGE);
|
||||
return;
|
||||
}
|
||||
|
||||
const candidateInput = readCandidateInput(options.input);
|
||||
const scope = buildScope(candidateInput);
|
||||
const output = `${JSON.stringify(scope, null, 2)}\n`;
|
||||
|
||||
if (!options.output) {
|
||||
process.stdout.write(output);
|
||||
return;
|
||||
}
|
||||
|
||||
const outputPath = resolve(options.output);
|
||||
mkdirSync(dirname(outputPath), { recursive: true });
|
||||
writeFileSync(outputPath, output);
|
||||
process.stderr.write(`Wrote review scope to ${outputPath}\n`);
|
||||
}
|
||||
|
||||
function parseArgs(args) {
|
||||
const options = {
|
||||
input: null,
|
||||
output: null,
|
||||
help: false
|
||||
};
|
||||
|
||||
for (let index = 0; index < args.length; index += 1) {
|
||||
const arg = args[index];
|
||||
if (arg === "--help" || arg === "-h") {
|
||||
options.help = true;
|
||||
} else if (arg === "--input") {
|
||||
options.input = requireValue(args, (index += 1), "--input");
|
||||
} else if (arg === "--output") {
|
||||
options.output = requireValue(args, (index += 1), "--output");
|
||||
} else {
|
||||
throw new Error(`Unknown option: ${arg}\n\n${USAGE}`);
|
||||
}
|
||||
}
|
||||
|
||||
if (!options.help && !options.input) {
|
||||
throw new Error("--input is required");
|
||||
}
|
||||
|
||||
return options;
|
||||
}
|
||||
|
||||
function requireValue(args, index, flag) {
|
||||
const value = args[index];
|
||||
if (!value || value.startsWith("--")) {
|
||||
throw new Error(`${flag} requires a value`);
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
function readCandidateInput(path) {
|
||||
const inputPath = resolve(path);
|
||||
const parsed = JSON.parse(readFileSync(inputPath, "utf8"));
|
||||
|
||||
if (parsed.version !== 1) {
|
||||
throw new Error(`Unsupported candidate input version: ${parsed.version}`);
|
||||
}
|
||||
if (!Array.isArray(parsed.candidates)) {
|
||||
throw new Error("Candidate input must define a candidates array");
|
||||
}
|
||||
|
||||
return {
|
||||
version: parsed.version,
|
||||
query: String(parsed.query ?? "").trim(),
|
||||
keywords: Array.isArray(parsed.keywords)
|
||||
? parsed.keywords.map((keyword) => String(keyword).trim()).filter(Boolean)
|
||||
: [],
|
||||
candidates: parsed.candidates.map(normalizeCandidate).filter(Boolean)
|
||||
};
|
||||
}
|
||||
|
||||
function normalizeCandidate(candidate) {
|
||||
if (!candidate || typeof candidate !== "object") return null;
|
||||
|
||||
const normalizedPath = normalizePath(candidate.path);
|
||||
if (!normalizedPath) return null;
|
||||
|
||||
return {
|
||||
path: normalizedPath,
|
||||
reason: String(candidate.reason ?? "").trim()
|
||||
};
|
||||
}
|
||||
|
||||
function buildScope(candidateInput) {
|
||||
const candidateMap = new Map();
|
||||
|
||||
for (const candidate of candidateInput.candidates) {
|
||||
const existing = candidateMap.get(candidate.path);
|
||||
if (existing) {
|
||||
if (candidate.reason && !existing.reason.includes(candidate.reason)) {
|
||||
existing.reason = [existing.reason, candidate.reason]
|
||||
.filter(Boolean)
|
||||
.join("; ");
|
||||
}
|
||||
continue;
|
||||
}
|
||||
candidateMap.set(candidate.path, { ...candidate });
|
||||
}
|
||||
|
||||
const normalizedCandidates = [...candidateMap.values()];
|
||||
const directoryGroups = new Map();
|
||||
const fileCandidates = [];
|
||||
|
||||
for (const candidate of normalizedCandidates) {
|
||||
if (isDirectoryPath(candidate.path)) {
|
||||
const directoryPath = ensureDirectoryPath(candidate.path);
|
||||
addDirectoryCandidate(directoryGroups, directoryPath, candidate);
|
||||
continue;
|
||||
}
|
||||
fileCandidates.push(candidate);
|
||||
}
|
||||
|
||||
const ancestorCounts = countDirectoryAncestors(fileCandidates);
|
||||
|
||||
for (const candidate of fileCandidates) {
|
||||
const directoryPath =
|
||||
findDeepestContainingDirectory(
|
||||
[...directoryGroups.keys()],
|
||||
candidate.path
|
||||
) ??
|
||||
findDeepestKeywordScopedAncestor(
|
||||
candidate.path,
|
||||
ancestorCounts,
|
||||
candidateInput.keywords
|
||||
);
|
||||
|
||||
if (!directoryPath) continue;
|
||||
addDirectoryCandidate(directoryGroups, directoryPath, candidate);
|
||||
}
|
||||
|
||||
const scopes = [];
|
||||
|
||||
for (const group of directoryGroups.values()) {
|
||||
if (
|
||||
group.sourcePaths.size === 1 &&
|
||||
!isDirectoryCandidateSource(group.path, group.sourcePaths)
|
||||
) {
|
||||
const onlyPath = [...group.sourcePaths][0];
|
||||
const [reason] = [...group.reasons];
|
||||
scopes.push({
|
||||
path: onlyPath,
|
||||
kind: "file",
|
||||
reason: reason ?? `candidate path matched under ${group.path}`,
|
||||
sourcePaths: [onlyPath]
|
||||
});
|
||||
continue;
|
||||
}
|
||||
|
||||
scopes.push({
|
||||
path: group.path,
|
||||
kind: "directory",
|
||||
reason:
|
||||
[...group.reasons][0] ??
|
||||
`multiple candidate paths matched in ${group.path}`,
|
||||
sourcePaths: [...group.sourcePaths].sort()
|
||||
});
|
||||
}
|
||||
|
||||
const coveredPaths = new Set(
|
||||
[...directoryGroups.values()].flatMap((group) => [...group.sourcePaths])
|
||||
);
|
||||
|
||||
for (const candidate of fileCandidates) {
|
||||
if (coveredPaths.has(candidate.path)) continue;
|
||||
scopes.push({
|
||||
path: candidate.path,
|
||||
kind: "file",
|
||||
reason: candidate.reason || "candidate path matched",
|
||||
sourcePaths: [candidate.path]
|
||||
});
|
||||
}
|
||||
|
||||
const prunedScopes = pruneNestedScopes(scopes)
|
||||
.sort((left, right) => left.path.localeCompare(right.path))
|
||||
.slice(0, 8);
|
||||
|
||||
return {
|
||||
version: 1,
|
||||
query: candidateInput.query,
|
||||
keywords: candidateInput.keywords,
|
||||
strategy: "agent-expanded-path-candidates",
|
||||
scopes: prunedScopes
|
||||
};
|
||||
}
|
||||
|
||||
function addDirectoryCandidate(directoryGroups, directoryPath, candidate) {
|
||||
const existing = directoryGroups.get(directoryPath) ?? {
|
||||
kind: "directory",
|
||||
path: directoryPath,
|
||||
reasons: new Set(),
|
||||
sourcePaths: new Set()
|
||||
};
|
||||
if (candidate.reason) existing.reasons.add(candidate.reason);
|
||||
existing.sourcePaths.add(candidate.path);
|
||||
directoryGroups.set(directoryPath, existing);
|
||||
}
|
||||
|
||||
function countDirectoryAncestors(fileCandidates) {
|
||||
const counts = new Map();
|
||||
|
||||
for (const candidate of fileCandidates) {
|
||||
for (const ancestor of ancestorDirectories(candidate.path)) {
|
||||
counts.set(ancestor, (counts.get(ancestor) ?? 0) + 1);
|
||||
}
|
||||
}
|
||||
|
||||
return counts;
|
||||
}
|
||||
|
||||
function ancestorDirectories(path) {
|
||||
const directories = [];
|
||||
let current = parentDirectoryPath(path);
|
||||
|
||||
while (current) {
|
||||
directories.push(current);
|
||||
current = parentDirectoryPath(current.slice(0, -1));
|
||||
}
|
||||
|
||||
return directories;
|
||||
}
|
||||
|
||||
function findDeepestContainingDirectory(directoryPaths, candidatePath) {
|
||||
return [...directoryPaths]
|
||||
.filter((directoryPath) => candidatePath.startsWith(directoryPath))
|
||||
.sort((left, right) => right.length - left.length)[0];
|
||||
}
|
||||
|
||||
function findDeepestKeywordScopedAncestor(path, ancestorCounts, keywords) {
|
||||
return (
|
||||
ancestorDirectories(path).find(
|
||||
(ancestor) =>
|
||||
(ancestorCounts.get(ancestor) ?? 0) > 1 &&
|
||||
pathSegmentMatchesKeywords(ancestor, keywords)
|
||||
) ?? null
|
||||
);
|
||||
}
|
||||
|
||||
function pathSegmentMatchesKeywords(path, keywords) {
|
||||
if (!Array.isArray(keywords) || keywords.length === 0) return false;
|
||||
|
||||
const segment = path.replace(/\/+$/, "").split("/").at(-1)?.toLowerCase();
|
||||
if (!segment) return false;
|
||||
|
||||
return keywords.some((keyword) => {
|
||||
const normalizedKeyword = String(keyword).trim().toLowerCase();
|
||||
return normalizedKeyword && segment.includes(normalizedKeyword);
|
||||
});
|
||||
}
|
||||
|
||||
function pruneNestedScopes(scopes) {
|
||||
const directories = scopes.filter((scope) => scope.kind === "directory");
|
||||
|
||||
return scopes.filter((scope) => {
|
||||
if (scope.kind === "directory") return true;
|
||||
return !directories.some((directoryScope) =>
|
||||
scope.path.startsWith(directoryScope.path)
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
function isDirectoryCandidateSource(path, sourcePaths) {
|
||||
return [...sourcePaths].some((sourcePath) => sourcePath === path);
|
||||
}
|
||||
|
||||
function parentDirectoryPath(path) {
|
||||
const trimmed = path.replace(/\/+$/, "");
|
||||
const lastSlash = trimmed.lastIndexOf("/");
|
||||
if (lastSlash === -1) return null;
|
||||
return `${trimmed.slice(0, lastSlash + 1)}`;
|
||||
}
|
||||
|
||||
function normalizePath(path) {
|
||||
if (typeof path !== "string") return "";
|
||||
const normalized = path
|
||||
.trim()
|
||||
.replaceAll("\\", "/")
|
||||
.replace(/^\.\/+/, "");
|
||||
if (!normalized) return "";
|
||||
return normalized.endsWith("/")
|
||||
? ensureDirectoryPath(normalized)
|
||||
: normalized;
|
||||
}
|
||||
|
||||
function ensureDirectoryPath(path) {
|
||||
return path.endsWith("/") ? path : `${path}/`;
|
||||
}
|
||||
|
||||
function isDirectoryPath(path) {
|
||||
return path.endsWith("/");
|
||||
}
|
||||
|
||||
try {
|
||||
main();
|
||||
} catch (error) {
|
||||
process.stderr.write(`${error.message}\n`);
|
||||
process.exitCode = 1;
|
||||
}
|
||||
@@ -0,0 +1,202 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { spawnSync } from "node:child_process";
|
||||
import { mkdir, mkdtemp, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { dirname, join, resolve } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import test from "node:test";
|
||||
|
||||
const scriptPath = resolve(
|
||||
dirname(fileURLToPath(import.meta.url)),
|
||||
"build-review-scope.mjs"
|
||||
);
|
||||
|
||||
test("collapses sibling file candidates into directory scopes", async () => {
|
||||
const workspaceRoot = await mkdtemp(join(tmpdir(), "tutti-review-scope-"));
|
||||
const inputPath = join(workspaceRoot, "candidates.json");
|
||||
|
||||
await writeWorkspaceFile(
|
||||
workspaceRoot,
|
||||
"apps/desktop/src/renderer/features/workspaces/index.ts",
|
||||
"export const workspaces = true;\n"
|
||||
);
|
||||
await writeWorkspaceFile(
|
||||
workspaceRoot,
|
||||
"apps/desktop/src/renderer/features/workspaces/services/internal/load.ts",
|
||||
"export const load = true;\n"
|
||||
);
|
||||
await writeWorkspaceFile(
|
||||
workspaceRoot,
|
||||
"services/tuttid/service/workspaces/manager.go",
|
||||
"package workspaces\n"
|
||||
);
|
||||
|
||||
await writeFile(
|
||||
inputPath,
|
||||
JSON.stringify(
|
||||
{
|
||||
version: 1,
|
||||
query: "workspace module",
|
||||
keywords: ["workspace", "workspaces"],
|
||||
candidates: [
|
||||
{
|
||||
path: "apps/desktop/src/renderer/features/workspaces/index.ts",
|
||||
reason: "filename matched workspaces"
|
||||
},
|
||||
{
|
||||
path: "apps/desktop/src/renderer/features/workspaces/services/internal/load.ts",
|
||||
reason: "path segment matched workspaces"
|
||||
},
|
||||
{
|
||||
path: "services/tuttid/service/workspaces/manager.go",
|
||||
reason: "path segment matched workspaces"
|
||||
}
|
||||
]
|
||||
},
|
||||
null,
|
||||
2
|
||||
),
|
||||
"utf8"
|
||||
);
|
||||
|
||||
const result = spawnSync(
|
||||
process.execPath,
|
||||
[scriptPath, "--input", inputPath],
|
||||
{
|
||||
cwd: workspaceRoot,
|
||||
encoding: "utf8"
|
||||
}
|
||||
);
|
||||
|
||||
assert.equal(result.status, 0, result.stderr);
|
||||
|
||||
const scope = JSON.parse(result.stdout);
|
||||
assert.equal(scope.strategy, "agent-expanded-path-candidates");
|
||||
assert.deepEqual(
|
||||
scope.scopes.map((entry) => entry.path),
|
||||
[
|
||||
"apps/desktop/src/renderer/features/workspaces/",
|
||||
"services/tuttid/service/workspaces/manager.go"
|
||||
]
|
||||
);
|
||||
assert.equal(scope.scopes[0].kind, "directory");
|
||||
assert.equal(scope.scopes[0].sourcePaths.length, 2);
|
||||
assert.equal(scope.scopes[1].kind, "file");
|
||||
});
|
||||
|
||||
test("removes nested file scopes when a directory scope already covers them", async () => {
|
||||
const workspaceRoot = await mkdtemp(join(tmpdir(), "tutti-review-scope-"));
|
||||
const inputPath = join(workspaceRoot, "candidates.json");
|
||||
|
||||
await writeWorkspaceFile(
|
||||
workspaceRoot,
|
||||
"packages/workspace/file-manager/src/index.ts",
|
||||
"export * from './ui';\n"
|
||||
);
|
||||
|
||||
await writeFile(
|
||||
inputPath,
|
||||
JSON.stringify(
|
||||
{
|
||||
version: 1,
|
||||
query: "file manager",
|
||||
keywords: ["file-manager"],
|
||||
candidates: [
|
||||
{
|
||||
path: "packages/workspace/file-manager/src/",
|
||||
reason: "directory matched file-manager"
|
||||
},
|
||||
{
|
||||
path: "packages/workspace/file-manager/src/index.ts",
|
||||
reason: "file matched file-manager"
|
||||
}
|
||||
]
|
||||
},
|
||||
null,
|
||||
2
|
||||
),
|
||||
"utf8"
|
||||
);
|
||||
|
||||
const result = spawnSync(
|
||||
process.execPath,
|
||||
[scriptPath, "--input", inputPath],
|
||||
{
|
||||
cwd: workspaceRoot,
|
||||
encoding: "utf8"
|
||||
}
|
||||
);
|
||||
|
||||
assert.equal(result.status, 0, result.stderr);
|
||||
|
||||
const scope = JSON.parse(result.stdout);
|
||||
assert.equal(scope.scopes.length, 1);
|
||||
assert.equal(scope.scopes[0].path, "packages/workspace/file-manager/src/");
|
||||
assert.equal(scope.scopes[0].kind, "directory");
|
||||
});
|
||||
|
||||
test("does not collapse to a broad parent when only the parent is shared", async () => {
|
||||
const workspaceRoot = await mkdtemp(join(tmpdir(), "tutti-review-scope-"));
|
||||
const inputPath = join(workspaceRoot, "candidates.json");
|
||||
|
||||
await writeWorkspaceFile(
|
||||
workspaceRoot,
|
||||
"apps/desktop/src/renderer/features/workspaces/index.ts",
|
||||
"export const workspaces = true;\n"
|
||||
);
|
||||
await writeWorkspaceFile(
|
||||
workspaceRoot,
|
||||
"apps/desktop/src/renderer/features/settings/index.ts",
|
||||
"export const settings = true;\n"
|
||||
);
|
||||
|
||||
await writeFile(
|
||||
inputPath,
|
||||
JSON.stringify(
|
||||
{
|
||||
version: 1,
|
||||
query: "workspace module",
|
||||
keywords: ["workspace", "workspaces"],
|
||||
candidates: [
|
||||
{
|
||||
path: "apps/desktop/src/renderer/features/workspaces/index.ts",
|
||||
reason: "filename matched workspaces"
|
||||
},
|
||||
{
|
||||
path: "apps/desktop/src/renderer/features/settings/index.ts",
|
||||
reason: "neighbor feature candidate"
|
||||
}
|
||||
]
|
||||
},
|
||||
null,
|
||||
2
|
||||
),
|
||||
"utf8"
|
||||
);
|
||||
|
||||
const result = spawnSync(
|
||||
process.execPath,
|
||||
[scriptPath, "--input", inputPath],
|
||||
{
|
||||
cwd: workspaceRoot,
|
||||
encoding: "utf8"
|
||||
}
|
||||
);
|
||||
|
||||
assert.equal(result.status, 0, result.stderr);
|
||||
|
||||
const scope = JSON.parse(result.stdout);
|
||||
assert.deepEqual(
|
||||
scope.scopes.map((entry) => entry.path),
|
||||
[
|
||||
"apps/desktop/src/renderer/features/settings/index.ts",
|
||||
"apps/desktop/src/renderer/features/workspaces/index.ts"
|
||||
]
|
||||
);
|
||||
});
|
||||
|
||||
async function writeWorkspaceFile(workspaceRoot, path, content) {
|
||||
const absolutePath = join(workspaceRoot, path);
|
||||
await mkdir(dirname(absolutePath), { recursive: true });
|
||||
await writeFile(absolutePath, content, "utf8");
|
||||
}
|
||||
+1103
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,490 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { spawnSync } from "node:child_process";
|
||||
import { mkdir, mkdtemp, realpath, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { dirname, join, resolve } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import test from "node:test";
|
||||
|
||||
import { createIsolatedGitEnvironment } from "../../../../tools/scripts/git-environment.mjs";
|
||||
|
||||
const scriptPath = resolve(
|
||||
dirname(fileURLToPath(import.meta.url)),
|
||||
"plan-review.mjs"
|
||||
);
|
||||
|
||||
test("plans tasks, risk, and preflight signals from a real git diff", async () => {
|
||||
const workspaceRoot = await createFixtureRepo();
|
||||
|
||||
await writeWorkspaceFile(
|
||||
workspaceRoot,
|
||||
"apps/desktop/src/renderer/src/App.tsx",
|
||||
'import { app } from "electron";\nconsole.log(app);\n'
|
||||
);
|
||||
await writeWorkspaceFile(
|
||||
workspaceRoot,
|
||||
"services/tuttid/api/generated/types.gen.go",
|
||||
"package generated\n\ntype WorkbenchSnapshot struct{}\n"
|
||||
);
|
||||
await writeWorkspaceFile(
|
||||
workspaceRoot,
|
||||
"packages/workbench/surface/package.json",
|
||||
'{ "name": "@tutti/workbench-surface", "version": "0.0.1" }\n'
|
||||
);
|
||||
|
||||
const result = runPlanner(workspaceRoot, [
|
||||
"--format",
|
||||
"json",
|
||||
"--no-untracked"
|
||||
]);
|
||||
|
||||
assert.equal(result.status, 0, result.stderr);
|
||||
|
||||
const taskPackage = JSON.parse(result.stdout);
|
||||
const taskIds = taskPackage.tasks.map((task) => task.id);
|
||||
|
||||
assert.ok(taskIds.includes("desktop-layering"));
|
||||
assert.ok(taskIds.includes("contracts-and-generated-sources"));
|
||||
assert.ok(taskIds.includes("cross-cutting-architecture"));
|
||||
assert.ok(
|
||||
taskPackage.preflightSignals.some(
|
||||
(signal) => signal.id === "renderer-direct-platform-import"
|
||||
)
|
||||
);
|
||||
assert.ok(
|
||||
taskPackage.preflightSignals.some(
|
||||
(signal) => signal.id === "generated-without-source"
|
||||
)
|
||||
);
|
||||
|
||||
const desktopTask = taskPackage.tasks.find(
|
||||
(task) => task.id === "desktop-layering"
|
||||
);
|
||||
assert.equal(desktopTask.riskLevel, "high");
|
||||
assert.equal(desktopTask.spawnRecommendation, "required");
|
||||
});
|
||||
|
||||
test("flags possible duplicate event infrastructure in owning reviewers", async () => {
|
||||
const workspaceRoot = await createFixtureRepo();
|
||||
|
||||
await writeWorkspaceFile(
|
||||
workspaceRoot,
|
||||
"apps/desktop/src/renderer/src/App.tsx",
|
||||
"export class EventBus {}\n"
|
||||
);
|
||||
|
||||
const result = runPlanner(workspaceRoot, [
|
||||
"--format",
|
||||
"json",
|
||||
"--no-untracked"
|
||||
]);
|
||||
|
||||
assert.equal(result.status, 0, result.stderr);
|
||||
|
||||
const taskPackage = JSON.parse(result.stdout);
|
||||
const desktopTask = taskPackage.tasks.find(
|
||||
(task) => task.id === "desktop-layering"
|
||||
);
|
||||
|
||||
assert.ok(
|
||||
taskPackage.preflightSignals.some(
|
||||
(signal) => signal.id === "possible-duplicate-event-infra"
|
||||
)
|
||||
);
|
||||
assert.ok(
|
||||
desktopTask.preflightSignals.some(
|
||||
(signal) => signal.id === "possible-duplicate-event-infra"
|
||||
)
|
||||
);
|
||||
});
|
||||
|
||||
test("--task filters generated packages to one reviewer", async () => {
|
||||
const workspaceRoot = await createFixtureRepo();
|
||||
|
||||
await writeWorkspaceFile(
|
||||
workspaceRoot,
|
||||
"apps/desktop/src/main/ipc/workspace.ts",
|
||||
"export const workspaceIpc = true;\n"
|
||||
);
|
||||
|
||||
const result = runPlanner(workspaceRoot, [
|
||||
"--format",
|
||||
"json",
|
||||
"--no-untracked",
|
||||
"--task",
|
||||
"desktop-layering"
|
||||
]);
|
||||
|
||||
assert.equal(result.status, 0, result.stderr);
|
||||
|
||||
const taskPackage = JSON.parse(result.stdout);
|
||||
assert.equal(taskPackage.tasks.length, 1);
|
||||
assert.equal(taskPackage.tasks[0].id, "desktop-layering");
|
||||
assert.equal(taskPackage.workflowEntry.taskFilter, "desktop-layering");
|
||||
});
|
||||
|
||||
test("--from-package with --format summary does not require a git repository", async () => {
|
||||
const workspaceRoot = await mkdtemp(join(tmpdir(), "tutti-review-package-"));
|
||||
const packagePath = join(workspaceRoot, "task-package.json");
|
||||
|
||||
await writeFile(
|
||||
packagePath,
|
||||
JSON.stringify(
|
||||
{
|
||||
version: 1,
|
||||
generatedAt: "2026-05-21T00:00:00.000Z",
|
||||
repoRoot: "/example/tutti",
|
||||
mode: "worktree",
|
||||
changedFiles: [],
|
||||
preflightSignals: [],
|
||||
crossCuttingReasons: [],
|
||||
workflowEntry: {
|
||||
packagePath
|
||||
},
|
||||
tasks: [sampleTask("desktop-layering"), sampleTask("tuttid-layering")],
|
||||
empty: false
|
||||
},
|
||||
null,
|
||||
2
|
||||
),
|
||||
"utf8"
|
||||
);
|
||||
|
||||
const result = runPlanner(workspaceRoot, [
|
||||
"--from-package",
|
||||
packagePath,
|
||||
"--task",
|
||||
"desktop-layering",
|
||||
"--format",
|
||||
"summary"
|
||||
]);
|
||||
|
||||
assert.equal(result.status, 0, result.stderr);
|
||||
assert.match(result.stdout, /Tutti Architecture Review Summary/);
|
||||
assert.match(result.stdout, /\| desktop-layering \|/);
|
||||
assert.doesNotMatch(result.stdout, /\| tuttid-layering \|/);
|
||||
assert.match(result.stdout, /Task filter: desktop-layering/);
|
||||
});
|
||||
|
||||
test("--scope-file narrows diff review to matching scoped paths", async () => {
|
||||
const workspaceRoot = await createFixtureRepo();
|
||||
const scopePath = join(workspaceRoot, "review-scope.json");
|
||||
|
||||
await writeWorkspaceFile(
|
||||
workspaceRoot,
|
||||
"apps/desktop/src/main/ipc/workspace.ts",
|
||||
"export const workspaceIpc = true;\n"
|
||||
);
|
||||
await writeWorkspaceFile(
|
||||
workspaceRoot,
|
||||
"packages/workbench/surface/package.json",
|
||||
'{ "name": "@tutti/workbench-surface", "version": "0.0.2" }\n'
|
||||
);
|
||||
await writeFile(
|
||||
scopePath,
|
||||
JSON.stringify(
|
||||
{
|
||||
version: 1,
|
||||
query: "workspace module",
|
||||
scopes: [
|
||||
{
|
||||
path: "apps/desktop/src/main/ipc/",
|
||||
kind: "directory"
|
||||
}
|
||||
]
|
||||
},
|
||||
null,
|
||||
2
|
||||
),
|
||||
"utf8"
|
||||
);
|
||||
|
||||
const result = runPlanner(workspaceRoot, [
|
||||
"--format",
|
||||
"json",
|
||||
"--no-untracked",
|
||||
"--scope-file",
|
||||
scopePath
|
||||
]);
|
||||
|
||||
assert.equal(result.status, 0, result.stderr);
|
||||
|
||||
const taskPackage = JSON.parse(result.stdout);
|
||||
assert.equal(
|
||||
taskPackage.workflowEntry.scopeSelectionMode,
|
||||
"diff-intersection"
|
||||
);
|
||||
assert.equal(taskPackage.workflowEntry.scopeMode, "auto");
|
||||
assert.match(taskPackage.workflowEntry.scopeSummary, /workspace module/);
|
||||
assert.deepEqual(taskPackage.reviewScope, {
|
||||
query: "workspace module",
|
||||
keywords: [],
|
||||
strategy: "scope-file",
|
||||
scopeMode: "auto",
|
||||
selectionMode: "diff-intersection",
|
||||
scopeCount: 1,
|
||||
scopes: [
|
||||
{
|
||||
path: "apps/desktop/src/main/ipc/",
|
||||
kind: "directory"
|
||||
}
|
||||
]
|
||||
});
|
||||
assert.deepEqual(
|
||||
taskPackage.changedFiles.map((file) => file.path),
|
||||
["apps/desktop/src/main/ipc/workspace.ts"]
|
||||
);
|
||||
assert.deepEqual(
|
||||
taskPackage.tasks.map((task) => task.id),
|
||||
["desktop-layering"]
|
||||
);
|
||||
});
|
||||
|
||||
test("--scope-file falls back to scoped files when diff overlap is empty", async () => {
|
||||
const workspaceRoot = await createFixtureRepo();
|
||||
const scopePath = join(workspaceRoot, "review-scope.json");
|
||||
|
||||
await writeWorkspaceFile(
|
||||
workspaceRoot,
|
||||
"apps/desktop/src/renderer/src/App.tsx",
|
||||
'import { app } from "electron";\nconsole.log(app);\n'
|
||||
);
|
||||
runGit(workspaceRoot, ["add", "apps/desktop/src/renderer/src/App.tsx"]);
|
||||
runGit(workspaceRoot, [
|
||||
"-c",
|
||||
"user.name=Tutti Test",
|
||||
"-c",
|
||||
"user.email=tutti-test@example.com",
|
||||
"commit",
|
||||
"-m",
|
||||
"update renderer app"
|
||||
]);
|
||||
|
||||
await writeFile(
|
||||
scopePath,
|
||||
JSON.stringify(
|
||||
{
|
||||
version: 1,
|
||||
query: "renderer app",
|
||||
scopes: [
|
||||
{
|
||||
path: "apps/desktop/src/renderer/src/App.tsx",
|
||||
kind: "file"
|
||||
}
|
||||
]
|
||||
},
|
||||
null,
|
||||
2
|
||||
),
|
||||
"utf8"
|
||||
);
|
||||
|
||||
const result = runPlanner(workspaceRoot, [
|
||||
"--format",
|
||||
"json",
|
||||
"--no-untracked",
|
||||
"--scope-file",
|
||||
scopePath
|
||||
]);
|
||||
|
||||
assert.equal(result.status, 0, result.stderr);
|
||||
|
||||
const taskPackage = JSON.parse(result.stdout);
|
||||
assert.equal(taskPackage.workflowEntry.scopeSelectionMode, "scope-fallback");
|
||||
assert.equal(taskPackage.reviewScope.selectionMode, "scope-fallback");
|
||||
assert.deepEqual(
|
||||
taskPackage.changedFiles.map((file) => file.statusText),
|
||||
["scoped"]
|
||||
);
|
||||
assert.ok(
|
||||
taskPackage.preflightSignals.some(
|
||||
(signal) => signal.id === "renderer-direct-platform-import"
|
||||
)
|
||||
);
|
||||
});
|
||||
|
||||
test("--scope-mode static-only reviews the whole scoped area even with diff overlap", async () => {
|
||||
const workspaceRoot = await createFixtureRepo();
|
||||
const scopePath = join(workspaceRoot, "review-scope.json");
|
||||
|
||||
await writeWorkspaceFile(
|
||||
workspaceRoot,
|
||||
"apps/desktop/src/main/ipc/workspace.ts",
|
||||
"export const workspaceIpc = true;\n"
|
||||
);
|
||||
await writeWorkspaceFile(
|
||||
workspaceRoot,
|
||||
"apps/desktop/src/main/ipc/untracked.ts",
|
||||
"export const untracked = true;\n"
|
||||
);
|
||||
await writeFile(
|
||||
scopePath,
|
||||
JSON.stringify(
|
||||
{
|
||||
version: 1,
|
||||
query: "workspace ipc",
|
||||
scopes: [
|
||||
{
|
||||
path: "apps/desktop/src/main/ipc/",
|
||||
kind: "directory"
|
||||
}
|
||||
]
|
||||
},
|
||||
null,
|
||||
2
|
||||
),
|
||||
"utf8"
|
||||
);
|
||||
|
||||
const result = runPlanner(workspaceRoot, [
|
||||
"--format",
|
||||
"json",
|
||||
"--no-untracked",
|
||||
"--scope-file",
|
||||
scopePath,
|
||||
"--scope-mode",
|
||||
"static-only"
|
||||
]);
|
||||
|
||||
assert.equal(result.status, 0, result.stderr);
|
||||
|
||||
const taskPackage = JSON.parse(result.stdout);
|
||||
assert.equal(taskPackage.workflowEntry.scopeMode, "static-only");
|
||||
assert.equal(taskPackage.workflowEntry.scopeSelectionMode, "static-only");
|
||||
assert.match(taskPackage.workflowEntry.scopeSummary, /selection static-only/);
|
||||
assert.equal(taskPackage.reviewScope.scopeMode, "static-only");
|
||||
assert.deepEqual(
|
||||
taskPackage.changedFiles.map((file) => file.path),
|
||||
[
|
||||
"apps/desktop/src/main/ipc/secondary.ts",
|
||||
"apps/desktop/src/main/ipc/workspace.ts"
|
||||
]
|
||||
);
|
||||
assert.deepEqual(
|
||||
taskPackage.changedFiles.map((file) => file.statusText),
|
||||
["scoped", "scoped"]
|
||||
);
|
||||
});
|
||||
|
||||
test("rejects unknown task filters with available task ids", async () => {
|
||||
const workspaceRoot = await createFixtureRepo();
|
||||
|
||||
await writeWorkspaceFile(
|
||||
workspaceRoot,
|
||||
"apps/desktop/src/main/ipc/workspace.ts",
|
||||
"export const workspaceIpc = true;\n"
|
||||
);
|
||||
|
||||
const result = runPlanner(workspaceRoot, [
|
||||
"--format",
|
||||
"summary",
|
||||
"--no-untracked",
|
||||
"--task",
|
||||
"missing-task"
|
||||
]);
|
||||
|
||||
assert.notEqual(result.status, 0);
|
||||
assert.match(result.stderr, /Unknown --task missing-task/);
|
||||
assert.match(result.stderr, /desktop-layering/);
|
||||
});
|
||||
|
||||
async function createFixtureRepo() {
|
||||
const workspaceRoot = await mkdtemp(join(tmpdir(), "tutti-review-"));
|
||||
|
||||
await writeWorkspaceFile(
|
||||
workspaceRoot,
|
||||
"apps/desktop/src/renderer/src/App.tsx",
|
||||
"console.log('renderer');\n"
|
||||
);
|
||||
await writeWorkspaceFile(
|
||||
workspaceRoot,
|
||||
"apps/desktop/src/main/ipc/workspace.ts",
|
||||
"export const workspaceIpc = false;\n"
|
||||
);
|
||||
await writeWorkspaceFile(
|
||||
workspaceRoot,
|
||||
"apps/desktop/src/main/ipc/secondary.ts",
|
||||
"export const secondaryIpc = false;\n"
|
||||
);
|
||||
await writeWorkspaceFile(
|
||||
workspaceRoot,
|
||||
"services/tuttid/api/generated/types.gen.go",
|
||||
"package generated\n"
|
||||
);
|
||||
await writeWorkspaceFile(
|
||||
workspaceRoot,
|
||||
"packages/workbench/surface/package.json",
|
||||
'{ "name": "@tutti/workbench-surface" }\n'
|
||||
);
|
||||
|
||||
runGit(workspaceRoot, ["init"]);
|
||||
await assertFixtureGitRoot(workspaceRoot);
|
||||
runGit(workspaceRoot, ["add", "."]);
|
||||
runGit(workspaceRoot, [
|
||||
"-c",
|
||||
"user.name=Tutti Test",
|
||||
"-c",
|
||||
"user.email=tutti-test@example.com",
|
||||
"commit",
|
||||
"-m",
|
||||
"fixture"
|
||||
]);
|
||||
|
||||
return workspaceRoot;
|
||||
}
|
||||
|
||||
async function writeWorkspaceFile(workspaceRoot, path, content) {
|
||||
const absolutePath = join(workspaceRoot, path);
|
||||
await mkdir(dirname(absolutePath), { recursive: true });
|
||||
await writeFile(absolutePath, content, "utf8");
|
||||
}
|
||||
|
||||
function sampleTask(id) {
|
||||
return {
|
||||
id,
|
||||
title: `${id} reviewer`,
|
||||
area: id,
|
||||
priority: 10,
|
||||
riskLevel: "low",
|
||||
spawnRecommendation: "optional",
|
||||
triggerReasons: [],
|
||||
summaryForMainAgent: `${id}: 1 file, low risk, no preflight signals; spawn is optional.`,
|
||||
matchedFiles: [
|
||||
{
|
||||
path: "example.ts",
|
||||
statusText: "modified"
|
||||
}
|
||||
],
|
||||
preflightSignals: [],
|
||||
referenceFiles: [],
|
||||
reviewFocus: [],
|
||||
prompt: "Review example.ts"
|
||||
};
|
||||
}
|
||||
|
||||
function runPlanner(workspaceRoot, args) {
|
||||
return spawnSync(process.execPath, [scriptPath, ...args], {
|
||||
cwd: workspaceRoot,
|
||||
encoding: "utf8",
|
||||
env: createIsolatedGitEnvironment(workspaceRoot)
|
||||
});
|
||||
}
|
||||
|
||||
function runGit(workspaceRoot, args) {
|
||||
const result = spawnSync("git", args, {
|
||||
cwd: workspaceRoot,
|
||||
encoding: "utf8",
|
||||
env: createIsolatedGitEnvironment(workspaceRoot)
|
||||
});
|
||||
|
||||
assert.equal(result.status, 0, result.stderr || result.stdout);
|
||||
return result;
|
||||
}
|
||||
|
||||
async function assertFixtureGitRoot(workspaceRoot) {
|
||||
const result = runGit(workspaceRoot, ["rev-parse", "--absolute-git-dir"]);
|
||||
assert.equal(
|
||||
await realpath(result.stdout.trim()),
|
||||
await realpath(join(workspaceRoot, ".git"))
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,342 @@
|
||||
---
|
||||
name: tutti-record-agent-session-replay
|
||||
description: >-
|
||||
From a Tutti checkout, run, audit, freshly replay, publish, or diagnose
|
||||
Session Replay cassettes that are driven by case-repository scenario scripts
|
||||
(CDP), not by interactive UI recording. Use for real-Provider capture while a
|
||||
scenario.mjs executes, cassette transport or semantic-state mismatches, fresh
|
||||
replay qualification, AgentGUI replay evidence, and product-side support for a
|
||||
new Agent Target beyond local:codex / local:claude-code. Do not use to author
|
||||
Case metadata or scenario scripts (case repository write-replay-case), or for
|
||||
executionKind "ui" Cases.
|
||||
---
|
||||
|
||||
# Qualify Tutti Agent Session Replay Cassettes
|
||||
|
||||
Work from the Tutti checkout. Keep product implementation and the generic
|
||||
runner in Tutti; keep Case metadata, **scenario scripts**, fixtures, qualified
|
||||
Cassettes, and evidence in the external case repository (`tutti-os/tutti-replay`).
|
||||
|
||||
Mental model (script-first, not UI recording):
|
||||
|
||||
1. Humans/agents **write** a deterministic `scenarios/*.mjs` (prepare / drive /
|
||||
assert) in the case repository — that script is the recording plan.
|
||||
2. **Record** means: Tutti runner launches Desktop, CDP-executes that script
|
||||
against a **live** Provider, and captures the Cassette. There is no separate
|
||||
click-to-record UI workflow for Session Replay.
|
||||
3. Day-to-day Record/Replay is usually triggered from the case repository QA
|
||||
console; this skill is for Tutti-side CLI qualification, diagnosis, runner
|
||||
or Replay product defects, and new Provider capture support.
|
||||
|
||||
Prove qualification in this order:
|
||||
|
||||
`existing scenario script -> live Record (script + Provider) -> structural audit -> fresh Replay -> optional publication`
|
||||
|
||||
Never call a Cassette qualified until Record, audit, and a fresh isolated
|
||||
Replay have all passed. If the scenario script itself is missing or wrong,
|
||||
stop and use the case repository `write-replay-case` skill — do not invent
|
||||
Cases inside Tutti.
|
||||
|
||||
Qualification is assertion-specific, not command-specific. A `replay passed`
|
||||
exit proves transport and semantic playback, but does not prove that every
|
||||
Case action ran during Replay. Build a four-column matrix for each core
|
||||
assertion: `Record proof | Cassette proof | Replay proof | Evidence`. Stop and
|
||||
repair the scenario when any required Replay cell is empty.
|
||||
|
||||
## Start the QA console (case repository)
|
||||
|
||||
Browsing Cases, Test Plans, and one-click Record/Replay (which run the same
|
||||
scenario scripts) live in the **case repository** (sibling checkout, commonly
|
||||
`../tutti-replay`; GitHub: `tutti-os/tutti-replay`).
|
||||
|
||||
From the case repository root:
|
||||
|
||||
```bash
|
||||
pnpm install
|
||||
pnpm dev
|
||||
```
|
||||
|
||||
Open only `http://127.0.0.1:3333` (never the API port `:3334`). In the UI,
|
||||
set the Tutti checkout absolute path, create a Test Plan, then Record or
|
||||
Replay. First-time machine setup: that repository's `SETUP.md`. Authoring or
|
||||
mirroring scenario scripts: `.agents/skills/write-replay-case/`.
|
||||
|
||||
For a long-lived LAN service on macOS use `pnpm replay:service install`
|
||||
(port `2333`); do not run `pnpm dev` and the stable service at the same time.
|
||||
|
||||
## Establish scope
|
||||
|
||||
1. Read the Tutti root and closest `AGENTS.md` files.
|
||||
2. Read `docs/architecture/agent-session-replay.md` (Provider support: developer
|
||||
recording currently accepts `local:codex` and `local:claude-code` only).
|
||||
3. For AgentGUI behavior, also read `docs/architecture/agent-gui-node.md` and
|
||||
`packages/agent/gui/AGENTS.md`.
|
||||
4. For Session, Turn, Goal, or runtime-operation lifecycle behavior, read
|
||||
`packages/agent/host/README.md`; lifecycle semantics remain in Host.
|
||||
5. Inspect `git status --short` in both repositories and preserve pre-existing
|
||||
work.
|
||||
6. Resolve the case repository from the user-provided path, the configured
|
||||
cases path, or the sibling `../tutti-replay` checkout. Do not guess another
|
||||
location if none exists.
|
||||
|
||||
Read the selected Case before planning:
|
||||
|
||||
- `cases/<case-id>/case.json`
|
||||
- every `cases/<case-id>/scenarios/*.mjs` except `*.impl.mjs`
|
||||
- referenced shared scenario helpers and `runtime-fixtures/`
|
||||
- existing `cassettes/`, `evidence/`, and relevant Run artifacts
|
||||
- the case repository's `README.md` and `CONTEXT.md` when publication or Case
|
||||
lifecycle is involved
|
||||
|
||||
If `case.json` declares `executionKind: "ui"`, stop this Session Replay
|
||||
workflow. Pure UI Cases are still script-driven (`defineUiScenario` + CDP),
|
||||
but they use **ui-drive** and publish `ui/` screenshots — they do not Record
|
||||
Provider Cassettes. Author and run them via the case repository
|
||||
`write-replay-case` skill and the QA console; do not use this skill's
|
||||
`--record` / Cassette audit path for them.
|
||||
|
||||
Use CDP through Tutti's repository runner. Do not use Computer Use unless the
|
||||
user explicitly requests it.
|
||||
|
||||
## Maintain the ownership boundary
|
||||
|
||||
- Add or update Case scenarios only under
|
||||
`cases/<case-id>/scenarios/*.mjs` in the case repository.
|
||||
- Put reusable scenario helpers in that repository's `scenario-runtime/`.
|
||||
- Do not add Case registries, Case-specific scenarios, fixtures, or qualified
|
||||
Cassettes to Tutti.
|
||||
- Change Tutti only for generic product, runner, protocol, or Replay defects.
|
||||
- Fix root causes. Do not relax transport matching, semantic verification,
|
||||
terminal assertions, or checkpoint requirements to accept a broken Case.
|
||||
|
||||
## Add a new Provider (product vs case repository)
|
||||
|
||||
Today Replay recording targets are `local:codex` and `local:claude-code`. The
|
||||
shared Session Replay core is provider-neutral, but each new Agent Target still
|
||||
needs Tutti capture + fail-closed playback before any Case work is useful.
|
||||
|
||||
Split work explicitly:
|
||||
|
||||
| Layer | Where | Who | Scope |
|
||||
| ------- | --------------- | ------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Product | Tutti | experienced / mentored | Adapter capture, projected tape, portability, structural audit, outbound verification, input-unit barriers, isolated Provider home, deterministic fail-closed Replay for `local:<provider>` |
|
||||
| Cases | case repository | can hand to intern after product gate | `providerProfiles`, `KNOWN_PROVIDERS`, `defineMirroredRecordScenario` mirrors, Record via console, publish cassettes |
|
||||
|
||||
Do **not** start by writing Cases for an unsupported Provider. Product must
|
||||
accept `--agent-target-id local:<provider>` for Record and Replay first.
|
||||
|
||||
### Tutti checklist (this repository)
|
||||
|
||||
Copy and tick:
|
||||
|
||||
```
|
||||
- [ ] 1. Provider adapter can Record real traffic into a Cassette
|
||||
- [ ] 2. Projected tape + portability (paths/homes) match session-replay contract
|
||||
- [ ] 3. Structural audit passes (manifest, frames, activity causality)
|
||||
- [ ] 4. Fresh isolated Replay is fail-closed (no live Provider fallback)
|
||||
- [ ] 5. Runner accepts --agent-target-id local:<provider>
|
||||
- [ ] 6. docs/architecture/agent-session-replay.md Provider support updated
|
||||
- [ ] 7. Hand off to case repository write-replay-case for mirrors + console Record
|
||||
```
|
||||
|
||||
Prove with one smoke scenario from the case repository (or a temporary
|
||||
scenario file) using the Tutti runner Record → audit → fresh Replay loop below.
|
||||
Keep account secrets out of logs and reports.
|
||||
|
||||
### Case repository handoff
|
||||
|
||||
After the product gate is green, follow that repository's
|
||||
`write-replay-case` skill section on multi-provider mirrors. Typical touch
|
||||
points there (not in Tutti): `scenario-runtime/shared.mjs` `providerProfiles`,
|
||||
`src/shared/agent-target.ts` `KNOWN_PROVIDERS`, mirrored `*.impl.mjs` + variant
|
||||
Case dirs, then console Record/Replay publication.
|
||||
|
||||
## Scenario scripts (owned by the case repository)
|
||||
|
||||
Session Replay drive logic is **authored as scripts**, not captured from
|
||||
manual UI interaction. Prefer the case repository's `write-replay-case` skill
|
||||
and `defineMirroredRecordScenario` / `defineRecordScenario` helpers.
|
||||
|
||||
When diagnosing or qualifying from Tutti, still require that the loaded
|
||||
scenario:
|
||||
|
||||
- exports `prepare`, `drive`, and `assert`;
|
||||
- sets Provider via profile/helper (not a hard-coded Codex-only identity);
|
||||
- sets every behavior-affecting composer default explicitly;
|
||||
- uses one stable prompt per intended Turn, with exact markers / final tokens;
|
||||
- uses accessible labels, test IDs, or semantic DOM state instead of coordinates;
|
||||
- waits before each interaction; answers each approval/question/plan once;
|
||||
- asserts a terminal state with no enabled stale controls;
|
||||
- declares `expectedRecordingMode` when continuing an existing Session.
|
||||
|
||||
Know the runner hook boundary before accepting the scenario:
|
||||
|
||||
- Record calls `prepare`, `drive`, `assert(phase="terminal")`, and
|
||||
`assert(phase="recorded")`.
|
||||
- Fresh Replay does **not** call `scenario.assert`; it replays Cassette traffic
|
||||
and calls `settleForScreenshot` at eligible checkpoints.
|
||||
- `captureEvidence` proves a Record state only. `captureFrame` inside
|
||||
`settleForScreenshot` proves a Replay checkpoint state.
|
||||
|
||||
For each action under test, require one Replay mechanism:
|
||||
|
||||
1. **Cassette-native**: the action appears in `activity-events.jsonl` as the
|
||||
expected intent/effect or direct stimulus and is consumed by a checkpoint.
|
||||
2. **Replay driver**: when the action is not recorded (for example a local UI
|
||||
action after Turn completion), `settleForScreenshot` performs it through
|
||||
the real product UI/API at one exact checkpoint, asserts before/after state,
|
||||
and captures transient evidence.
|
||||
|
||||
If neither exists, classify the assertion as Record-only. Do not call the Case
|
||||
Replay-qualified or infer coverage from the final UI looking correct.
|
||||
|
||||
For question cards, the script must trigger the Provider's real user-input
|
||||
request. For plan Cases, wait for the completed plan and implementation
|
||||
decision before driving the real action.
|
||||
|
||||
If the script needs to change, edit it in the case repository — then re-Record.
|
||||
|
||||
## Record = execute the scenario script with a live Provider
|
||||
|
||||
From the Tutti root, run the repository runner so it CDP-drives the scenario
|
||||
file and captures a Cassette. Derive scenario ID, Cassette name, and Agent
|
||||
Target from the loaded scenario:
|
||||
|
||||
```bash
|
||||
pnpm e2e:agent-gui -- \
|
||||
--record .tmp/cassettes/<cassette-name> \
|
||||
--scenario <scenario-id> \
|
||||
--scenario-file <case-repository>/cases/<case-id>/scenarios/<scenario-id>.mjs \
|
||||
--agent-target-id <agent-target-id> \
|
||||
--keep-runtime \
|
||||
--timeout-ms 300000 \
|
||||
--stall-timeout-ms 60000
|
||||
```
|
||||
|
||||
Prefer the case console Test Plan「录制」for routine work; use this CLI when
|
||||
debugging runner/Replay behavior or when the console is unavailable.
|
||||
|
||||
Omit `--headless` while debugging; add it for unattended execution. Keep the
|
||||
runtime only long enough to inspect or collect its artifacts.
|
||||
|
||||
Inspect failures from the smallest relevant evidence set:
|
||||
|
||||
- record screenshots and checkpoint screenshots;
|
||||
- `logs/desktop.log`;
|
||||
- `state/logs/tuttid.log`;
|
||||
- `state/tuttid.db` through targeted queries;
|
||||
- the incomplete Cassette and a small decoded Provider-frame window.
|
||||
|
||||
Do not dump an entire Provider stream or expose account data.
|
||||
|
||||
## Audit each Cassette
|
||||
|
||||
Run the bundled structural audit from the Tutti root:
|
||||
|
||||
```bash
|
||||
node .codex/skills/tutti-record-agent-session-replay/scripts/audit-cassette.mjs \
|
||||
.tmp/cassettes/<cassette-name>
|
||||
```
|
||||
|
||||
Require:
|
||||
|
||||
- Cassette inventory, hashes, and size policy verify;
|
||||
- the Provider manifest is complete;
|
||||
- global and per-connection frame sequences are continuous;
|
||||
- Activity sequences are continuous;
|
||||
- intent-to-effect causality satisfies
|
||||
`packages/agent/session-replay/activity-contract.json`;
|
||||
- expected interactions, plan decisions, tools, exits, terminal Turns, and
|
||||
final response state match `case.json` and the scenario assertions.
|
||||
- every Case-critical action expected to replay is present in
|
||||
`activity-events.jsonl` / `checkpoint-plan.json`, or is explicitly owned by
|
||||
a checkpoint Replay driver.
|
||||
|
||||
The bundled audit proves structural invariants and emits a semantic summary.
|
||||
It does not replace Case-specific assertions or a fresh Replay.
|
||||
|
||||
Before Replay, compare the audit output and raw checkpoint plan with the Case's
|
||||
coverage matrix. Counts alone are insufficient: for example, an Activity
|
||||
stream containing only `activation/requested` + `session/activate` cannot prove
|
||||
a later Undo/Reapply click.
|
||||
|
||||
## Run a fresh Replay
|
||||
|
||||
Replay from a fresh isolated Tutti runtime and pass the scenario so checkpoint
|
||||
screenshot settling runs. Remember that `scenario.assert` does not run here:
|
||||
|
||||
```bash
|
||||
pnpm e2e:agent-gui -- \
|
||||
--replay .tmp/cassettes/<cassette-name> \
|
||||
--scenario <scenario-id> \
|
||||
--scenario-file <case-repository>/cases/<case-id>/scenarios/<scenario-id>.mjs \
|
||||
--screenshot-checkpoints \
|
||||
--keep-runtime \
|
||||
--timeout-ms 300000 \
|
||||
--stall-timeout-ms 60000
|
||||
```
|
||||
|
||||
Require the runner's `replay passed` result, all planned checkpoints, the
|
||||
expected AgentGUI terminal state, and a fully drained Provider transport.
|
||||
Provider transport remains fail-closed; only repository-declared observer-only
|
||||
probes may yield to causal traffic.
|
||||
|
||||
Then prove each target assertion through its selected Replay mechanism:
|
||||
|
||||
- Cassette-native: show the matching Activity/direct stimulus and resulting
|
||||
checkpoint state.
|
||||
- Replay driver: show its before/after assertions completed and inspect the
|
||||
named `captureFrame` screenshots, including transient states rather than
|
||||
only the final stable UI.
|
||||
- External filesystem/process side effects: prove how the fresh Replay project
|
||||
receives the minimal deterministic pre-action state. Cassette semantic state
|
||||
does not by itself recreate arbitrary Provider filesystem mutations.
|
||||
|
||||
Never report “Record & Replay cover the behavior” when only Record executed
|
||||
the Case action. Report transport/semantic Replay separately from
|
||||
assertion-level behavior coverage.
|
||||
|
||||
When one Case owns multiple scenarios, Record and audit every resulting
|
||||
Cassette, then qualify them together through one Replay Workspace. Let the
|
||||
case repository workflow generate the workspace manifest; do not invent a
|
||||
second Case registry in Tutti.
|
||||
|
||||
## Publish only qualified artifacts
|
||||
|
||||
Prefer the case repository's publication workflow because it records the Run,
|
||||
archives prior artifacts, and publishes `cassettes/` plus `evidence/` only
|
||||
after qualification.
|
||||
|
||||
If the user explicitly requests manual publication:
|
||||
|
||||
1. Stage all new Cassettes and evidence without touching the qualified copies.
|
||||
2. Verify every staged Cassette completed Record, audit, and fresh Replay.
|
||||
3. Replace `cases/<case-id>/cassettes/` and `evidence/` as one Case operation,
|
||||
preserving the previous qualified artifacts in the case repository's
|
||||
archive convention.
|
||||
4. Re-read every published manifest and evidence directory.
|
||||
|
||||
Do not infer Case lifecycle from a successful command. Run status, manual
|
||||
acceptance, and `case.json` lifecycle status are distinct. Do not set
|
||||
`status: "confirmed"` or `acceptedAt` unless the case repository's acceptance
|
||||
requirements have been satisfied.
|
||||
|
||||
## Finish
|
||||
|
||||
- If Tutti implementation changed, run the validation selected by
|
||||
`docs/conventions/testing.md` plus any closest-area checks.
|
||||
- If case metadata or scenarios changed, run the case repository's metadata,
|
||||
scenario, and type checks.
|
||||
- Recheck both worktrees and separate pre-existing changes from this task.
|
||||
- Perform the Tutti documentation-impact check.
|
||||
|
||||
Report:
|
||||
|
||||
- Case and Cassette names;
|
||||
- Record, audit, and fresh Replay results;
|
||||
- Provider-frame, Activity, interaction, tool, Turn, and final-state summaries;
|
||||
- the assertion coverage matrix and exact Replay mechanism for each core action;
|
||||
- Tutti implementation changes and case repository artifact changes;
|
||||
- changed-line distribution by functional area, excluding pre-existing work;
|
||||
- documentation impact;
|
||||
- failed gates and unimplemented scope.
|
||||
@@ -0,0 +1,4 @@
|
||||
interface:
|
||||
display_name: "Tutti Session Replay"
|
||||
short_description: "Qualify Session Replay cassettes from scenario scripts"
|
||||
default_prompt: "Use $tutti-record-agent-session-replay to run the case scenario script with live Record, audit, and fresh Replay from this Tutti checkout (script-driven capture, not interactive UI recording)."
|
||||
@@ -0,0 +1,270 @@
|
||||
#!/usr/bin/env node
|
||||
import { access, readFile } from "node:fs/promises";
|
||||
import { dirname, join, resolve } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { verifyCassette } from "../../../../tools/scripts/agent-session-replay-runner/cassette.mjs";
|
||||
|
||||
const directory = resolve(process.argv[2] ?? "");
|
||||
if (!process.argv[2]) {
|
||||
throw new Error("usage: audit-cassette.mjs <cassette-directory>");
|
||||
}
|
||||
|
||||
const repoRoot = await resolveRepoRoot();
|
||||
const activityContract = await readJSON(
|
||||
join(
|
||||
repoRoot,
|
||||
"packages",
|
||||
"agent",
|
||||
"session-replay",
|
||||
"activity-contract.json"
|
||||
)
|
||||
);
|
||||
if (activityContract.schemaVersion !== 1) {
|
||||
throw new Error(
|
||||
`Unsupported activity contract schemaVersion ${activityContract.schemaVersion}, want 1`
|
||||
);
|
||||
}
|
||||
|
||||
const manifest = await verifyCassette(directory);
|
||||
const [providerManifest, frameText, activityText, expectedState] =
|
||||
await Promise.all([
|
||||
readJSON(join(directory, "provider", "manifest.json")),
|
||||
readFile(join(directory, "provider", "frames.jsonl"), "utf8"),
|
||||
readFile(join(directory, "activity-events.jsonl"), "utf8"),
|
||||
readJSON(join(directory, "expected-state.json"))
|
||||
]);
|
||||
|
||||
const frames = parseJSONLines(frameText);
|
||||
const activities = parseJSONLines(activityText);
|
||||
assertContinuous(
|
||||
frames.map((frame) => frame.globalSeq),
|
||||
"Provider global sequence"
|
||||
);
|
||||
assertContinuous(
|
||||
activities.map((event) => event.sequence),
|
||||
"Activity sequence"
|
||||
);
|
||||
const causalityViolations = auditActivityCausality(
|
||||
activities,
|
||||
activityContract
|
||||
);
|
||||
if (causalityViolations.length > 0) {
|
||||
throw new Error(
|
||||
`Activity causality audit failed with ${causalityViolations.length} violation(s):\n` +
|
||||
causalityViolations.map((violation) => `- ${violation}`).join("\n")
|
||||
);
|
||||
}
|
||||
for (const connection of providerManifest.connections ?? []) {
|
||||
assertContinuous(
|
||||
frames
|
||||
.filter((frame) => frame.connectionId === connection.connectionId)
|
||||
.map((frame) => frame.chunkSeq),
|
||||
`Provider ${connection.connectionId} sequence`
|
||||
);
|
||||
}
|
||||
|
||||
const sessions = expectedState.agent?.sessions ?? [];
|
||||
const result = {
|
||||
cassette: {
|
||||
id: manifest.id,
|
||||
name: manifest.name,
|
||||
providerTarget: manifest.agentTargetId,
|
||||
mode: manifest.mode,
|
||||
schemaVersion: manifest.schemaVersion,
|
||||
totalBytes: manifest.totalBytes
|
||||
},
|
||||
provider: {
|
||||
status: providerManifest.status,
|
||||
frameCount: providerManifest.frameCount,
|
||||
actualFrameCount: frames.length,
|
||||
framesSha256: providerManifest.framesSha256,
|
||||
connections: (providerManifest.connections ?? []).map((connection) => ({
|
||||
connectionId: connection.connectionId,
|
||||
provider: connection.provider,
|
||||
launchOrdinal: connection.launchOrdinal,
|
||||
captureOrigin: connection.captureOrigin
|
||||
}))
|
||||
},
|
||||
activities: activities.map((event) => ({
|
||||
sequence: event.sequence,
|
||||
kind: event.kind,
|
||||
type: event.type,
|
||||
optionId: event.payload?.optionId ?? null,
|
||||
action: event.payload?.action ?? null,
|
||||
requestId: event.payload?.requestId ?? null
|
||||
})),
|
||||
causality: {
|
||||
contractSchemaVersion: activityContract.schemaVersion,
|
||||
intentCount: activities.filter((event) => event.kind === "intent").length,
|
||||
effectCount: activities.filter((event) => event.kind === "effect").length,
|
||||
directStimulusCount: activities.filter(
|
||||
(event) => event.kind === "direct-stimulus"
|
||||
).length
|
||||
},
|
||||
state: {
|
||||
sessions: sessions.map((session) => ({
|
||||
id: session.id,
|
||||
turns: (session.turns ?? []).map((turn) => ({
|
||||
id: turn.id,
|
||||
phase: turn.phase,
|
||||
outcome: turn.outcome ?? null
|
||||
})),
|
||||
interactions: (session.interactions ?? []).map((interaction) => ({
|
||||
requestId: interaction.requestId,
|
||||
kind: interaction.kind,
|
||||
status: interaction.status,
|
||||
optionId: interaction.output?.optionId ?? null,
|
||||
action: interaction.output?.action ?? null
|
||||
})),
|
||||
tools: (session.messages ?? [])
|
||||
.filter((message) => message.kind === "tool_call")
|
||||
.map((message) => ({
|
||||
status: message.status,
|
||||
exitCode: message.payload?.output?.exitCode ?? null
|
||||
})),
|
||||
finalAssistantText:
|
||||
[...(session.messages ?? [])]
|
||||
.reverse()
|
||||
.find((message) => message.kind === "text")?.payload?.text ?? null
|
||||
}))
|
||||
}
|
||||
};
|
||||
|
||||
if (
|
||||
providerManifest.status !== "complete" ||
|
||||
providerManifest.frameCount !== frames.length
|
||||
) {
|
||||
throw new Error("Provider cassette is incomplete");
|
||||
}
|
||||
process.stdout.write(`${JSON.stringify(result, null, 2)}\n`);
|
||||
|
||||
async function readJSON(path) {
|
||||
return JSON.parse(await readFile(path, "utf8"));
|
||||
}
|
||||
|
||||
function parseJSONLines(text) {
|
||||
return text
|
||||
.split(/\r?\n/u)
|
||||
.filter((line) => line.trim())
|
||||
.map((line) => JSON.parse(line));
|
||||
}
|
||||
|
||||
async function resolveRepoRoot() {
|
||||
const scriptDirectory = dirname(fileURLToPath(import.meta.url));
|
||||
const root = resolve(scriptDirectory, "../../../..");
|
||||
for (const marker of ["pnpm-workspace.yaml", "package.json"]) {
|
||||
try {
|
||||
await access(join(root, marker));
|
||||
return root;
|
||||
} catch {
|
||||
// try the next marker
|
||||
}
|
||||
}
|
||||
throw new Error(
|
||||
`Repository root not found at ${root} (missing pnpm-workspace.yaml/package.json)`
|
||||
);
|
||||
}
|
||||
|
||||
function auditActivityCausality(activities, contract) {
|
||||
const violations = [];
|
||||
const intents = contract.intents ?? {};
|
||||
const eventsById = new Map(activities.map((event) => [event.eventId, event]));
|
||||
const referencedIntentIds = new Set();
|
||||
|
||||
for (const event of activities) {
|
||||
if (event.kind === "direct-stimulus") {
|
||||
if (event.causedByEventId != null) {
|
||||
violations.push(
|
||||
`${describeEvent(event)}: direct-stimulus must not have causedByEventId ${event.causedByEventId}`
|
||||
);
|
||||
}
|
||||
continue;
|
||||
}
|
||||
if (event.kind === "intent") {
|
||||
if (!intents[event.type]) {
|
||||
violations.push(
|
||||
`${describeEvent(event)}: intent type is not declared in activity-contract.json`
|
||||
);
|
||||
}
|
||||
continue;
|
||||
}
|
||||
if (event.kind !== "effect") {
|
||||
continue;
|
||||
}
|
||||
if (event.causedByEventId == null) {
|
||||
violations.push(
|
||||
`${describeEvent(event)}: effect is missing causedByEventId`
|
||||
);
|
||||
continue;
|
||||
}
|
||||
const cause = eventsById.get(event.causedByEventId);
|
||||
if (!cause) {
|
||||
violations.push(
|
||||
`${describeEvent(event)}: causedByEventId ${event.causedByEventId} does not exist`
|
||||
);
|
||||
continue;
|
||||
}
|
||||
if (cause.kind !== "intent") {
|
||||
violations.push(
|
||||
`${describeEvent(event)}: causedByEventId ${event.causedByEventId} is kind ${cause.kind}, want intent`
|
||||
);
|
||||
continue;
|
||||
}
|
||||
referencedIntentIds.add(cause.eventId);
|
||||
if (cause.sequence >= event.sequence) {
|
||||
violations.push(
|
||||
`${describeEvent(event)}: cause sequence ${cause.sequence} is not earlier than effect`
|
||||
);
|
||||
}
|
||||
if (
|
||||
event.correlationId != null &&
|
||||
cause.correlationId != null &&
|
||||
event.correlationId !== cause.correlationId
|
||||
) {
|
||||
violations.push(
|
||||
`${describeEvent(event)}: correlationId conflicts with cause correlationId ${cause.correlationId}`
|
||||
);
|
||||
}
|
||||
const declaredEffects = intents[cause.type]?.effects;
|
||||
if (declaredEffects && !declaredEffects.includes(event.type)) {
|
||||
violations.push(
|
||||
`${describeEvent(event)}: effect type is not declared for intent ${cause.type} (allowed: ${declaredEffects.join(", ") || "none"})`
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
for (const event of activities) {
|
||||
if (event.kind !== "intent") {
|
||||
continue;
|
||||
}
|
||||
if (
|
||||
intents[event.type]?.requiresEffect &&
|
||||
!referencedIntentIds.has(event.eventId)
|
||||
) {
|
||||
violations.push(
|
||||
`${describeEvent(event)}: intent requires at least one effect but none references it`
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
return violations;
|
||||
}
|
||||
|
||||
function describeEvent(event) {
|
||||
return (
|
||||
`sequence=${event.sequence ?? "?"} eventId=${event.eventId ?? "?"} ` +
|
||||
`kind=${event.kind ?? "?"} type=${event.type ?? "?"} ` +
|
||||
`correlationId=${event.correlationId ?? "-"}`
|
||||
);
|
||||
}
|
||||
|
||||
function assertContinuous(values, label) {
|
||||
values.forEach((value, index) => {
|
||||
const expected = index + 1;
|
||||
if (value !== expected) {
|
||||
throw new Error(
|
||||
`${label} is ${value} at index ${index}, want ${expected}`
|
||||
);
|
||||
}
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,165 @@
|
||||
---
|
||||
name: tutti-test-audit
|
||||
description: Audit, design, write, materially revise, or remove Tutti tests. Use whenever Codex changes or reviews unit, component, conformance, integration, regression, platform, or repository tests; enforce a protected product contract, credible failure, correct owning boundary, negative-control evidence, overlap review, deterministic setup, and an executing CI lane.
|
||||
---
|
||||
|
||||
# Tutti Test Audit
|
||||
|
||||
Make every changed test earn its maintenance cost. A green test, a coverage
|
||||
increase, or a larger test count is not a quality verdict.
|
||||
|
||||
Read the root and closest scoped `AGENTS.md`, then read
|
||||
`docs/conventions/unit-testing.md` before a non-trivial test change or review.
|
||||
Follow `docs/conventions/testing.md#validation-selection` for commands and
|
||||
validation scope.
|
||||
|
||||
## Establish the evidence map
|
||||
|
||||
Before editing, inspect:
|
||||
|
||||
1. the production owner, public entry point, relevant caller and callee;
|
||||
2. sibling implementations that share the same invariant;
|
||||
3. existing tests, fixtures, conformance suites, and repository checks;
|
||||
4. the changed-aware and platform lane that will select the evidence;
|
||||
5. relevant history for a regression or a suspicious existing test.
|
||||
|
||||
For Agent Host lifecycle semantics, also read
|
||||
`packages/agent/host/README.md` and start with the conformance contract. Do not
|
||||
reimplement lifecycle tests in an adapter.
|
||||
|
||||
## Pass the authoring gate
|
||||
|
||||
Answer all six questions before adding or materially rewriting a test:
|
||||
|
||||
1. **Protected contract:** What observable behavior, invariant, compatibility
|
||||
promise, or prior failure matters?
|
||||
2. **Credible failure:** What plausible faulty implementation must make the
|
||||
test fail?
|
||||
3. **Coverage gap:** Why would existing evidence not catch that failure? Can an
|
||||
existing table, scenario, or fixture express it without duplication?
|
||||
4. **Owner and observer:** Which module owns the decision, and what is the
|
||||
lowest boundary that can observe the risk without mocking it away?
|
||||
5. **Production seam:** Does the test require an export, flag, wrapper, global,
|
||||
or injection hook that no production caller needs? If so, move to the real
|
||||
boundary or redesign the owner.
|
||||
6. **Execution:** Which blocking lane selects the test, and which native OS must
|
||||
execute it?
|
||||
|
||||
A missing answer means the test is not ready. “No new test” is a valid outcome
|
||||
when stronger existing evidence already protects the contract or the proposed
|
||||
assertion only restates a type, literal, implementation detail, or presentation
|
||||
choice.
|
||||
|
||||
## Choose the proof shape
|
||||
|
||||
- Extend an existing table or conformance scenario when it owns the same
|
||||
contract and fixture.
|
||||
- Add a new scenario when the behavior, failure mechanism, setup, or execution
|
||||
lane is independently meaningful.
|
||||
- Move upward to a component, narrow integration, conformance, or E2E boundary
|
||||
when a unit double would replace the semantics under test.
|
||||
- Use a repository check, not a unit test, when source structure, imports,
|
||||
generated artifacts, manifests, or architecture are the actual contract.
|
||||
- Keep visual styling in visual evidence unless styling changes native behavior
|
||||
such as pointer ownership or window dragging.
|
||||
|
||||
Prefer one vertical proof through the owning boundary plus narrow tests for
|
||||
important decision partitions over many stub-backed branch tests.
|
||||
|
||||
## Build sensitive evidence
|
||||
|
||||
For a confirmed bug, capture the failing reproduction before the fix whenever
|
||||
possible. The regression test must fail on pre-fix behavior for the intended
|
||||
reason and pass after the owner-boundary repair. If the old revision cannot be
|
||||
run safely, temporarily inject or restore the faulty decision as a local
|
||||
negative control and report that limitation; never commit the mutation.
|
||||
|
||||
For new behavior, first demonstrate that the focused test fails because the
|
||||
contract is absent, not because setup, imports, or fixtures are broken.
|
||||
|
||||
For a read-only audit, do not edit production code merely to manufacture a
|
||||
negative control. Name the smallest plausible faulty implementation, determine
|
||||
whether the current assertions would detect it, and label that sensitivity
|
||||
unverified when it cannot be executed safely.
|
||||
|
||||
Then:
|
||||
|
||||
- assert stable outcomes and important non-effects;
|
||||
- include neighboring valid, rejection, cleanup, retry, or preservation cases
|
||||
only when they address a credible risk;
|
||||
- use real isolated SQLite, filesystem, Git, HTTP, IPC, parser, or process
|
||||
boundaries when their semantics are the subject;
|
||||
- keep external doubles narrow and fail closed on unexpected calls;
|
||||
- coordinate concurrency with events, channels, barriers, or latches rather
|
||||
than sleeps or lucky elapsed-time bounds;
|
||||
- control clocks, IDs, randomness, ports, user state, credentials, globals,
|
||||
listeners, timers, processes, and temporary resources;
|
||||
- assert call counts only when count is itself a contract such as idempotency,
|
||||
single-flight, or at-most-once dispatch.
|
||||
|
||||
## Apply Tutti routing
|
||||
|
||||
- Session, Turn, Goal, runtime-operation, and recovery semantics start in
|
||||
`packages/agent/host/conformance`; consumers provide drivers.
|
||||
- Adapter tests own transport, DTO, authorization, query, presentation, and
|
||||
product-policy behavior only.
|
||||
- SQLite, filesystem, path, shell, executable, process, signal, permission, and
|
||||
native API behavior cross the real isolated receiver boundary.
|
||||
- UI behavior drives accessible keyboard, pointer, focus, and state transitions
|
||||
and observes callbacks or user-visible state.
|
||||
- HTTP, IPC, and provider protocol tests cross the real parser, middleware, or
|
||||
codec boundary when receiver behavior is the risk.
|
||||
- Platform-dependent behavior runs on the native platform. A serialized
|
||||
request shape or a skipped Linux test is not Windows proof.
|
||||
|
||||
## Audit existing tests
|
||||
|
||||
Treat these as review candidates, not automatic deletions:
|
||||
|
||||
- no meaningful assertion, self-comparison, or only “does not throw”;
|
||||
- copied production algorithm, catalog, defaults, or current inventory;
|
||||
- source strings or regexes presented as runtime behavior proof;
|
||||
- private call-shape or mocked call-graph assertions;
|
||||
- static render, class, style, SVG path, or incidental DOM assertions;
|
||||
- duplicate coverage weaker than an existing boundary scenario;
|
||||
- sleeps, polling delays, pass-on-retry, silent skips, or zero-selected lanes;
|
||||
- test-only production exports, wrappers, flags, globals, or dead code.
|
||||
|
||||
Before weakening or removing a test, identify the failure it can actually catch,
|
||||
its history, and the stronger evidence that remains. Preserve independent
|
||||
public API, protocol, config, migration, storage, security, platform, generated,
|
||||
release, and architecture contracts even when they are static or slow.
|
||||
|
||||
Classify each candidate as:
|
||||
|
||||
- **keep** — protects an independent contract;
|
||||
- **strengthen or move** — right risk, wrong oracle or boundary;
|
||||
- **consolidate** — useful contract already has a canonical owner;
|
||||
- **delete** — no credible failure remains protected, including any test-only
|
||||
production seam it kept alive.
|
||||
|
||||
## Validate and hand off
|
||||
|
||||
1. Run the smallest owner and sibling tests that prove the contract.
|
||||
2. For authored behavior changes, demonstrate the pre-fix failure or another
|
||||
credible negative control. For a read-only audit, report the failure model
|
||||
without mutating the checkout.
|
||||
3. Apply the repository validation-selection policy; verify the intended lane
|
||||
selected and ran non-zero evidence.
|
||||
4. Run native-platform proof when required, or name the remaining manual gate.
|
||||
5. Inspect the final diff for duplicated setup, weakened assertions, snapshot
|
||||
churn, leaked resources, and unnecessary production seams.
|
||||
|
||||
Report this compact evidence for non-trivial test changes:
|
||||
|
||||
```md
|
||||
Test evidence
|
||||
|
||||
- Protected contract or prior failure:
|
||||
- Credible faulty implementation / negative control:
|
||||
- Existing coverage gap:
|
||||
- Owning seam and test level:
|
||||
- Observable outcomes and important non-effects:
|
||||
- Blocking lane and native OS, when relevant:
|
||||
- Residual risk left to integration, E2E, visual, or manual proof:
|
||||
```
|
||||
@@ -0,0 +1,4 @@
|
||||
interface:
|
||||
display_name: "Tutti Test Audit"
|
||||
short_description: "Design and audit behavior-protecting Tutti tests"
|
||||
default_prompt: "Use $tutti-test-audit to design or review tests that protect this change’s real behavior."
|
||||
Reference in new issue
Block a user