源码快照 @ 2026-09-01 (source-only, 剔除非源码素材)

docs/conventions/unit-testing.md

@@ -0,0 +1,237 @@
+# Unit Testing
+
+This document contains Tutti's durable test-design rules. The executable
+authoring and review workflow lives in `$tutti-test-audit`; test discovery,
+commands, changed-aware validation, performance scenarios, and Session Replay
+live in [Testing](./testing.md).
+
+Apply these rules to new and materially rewritten tests. They do not require a
+mass rewrite of legacy suites, but a touched test must not preserve a known
+low-value pattern without reason.
+
+## Quality bar
+
+A test earns its place when it protects a product contract that a plausible
+implementation mistake could violate. It must execute the production owner and
+observe a stable result at a boundary capable of exposing that mistake.
+
+Start with this statement:
+
+```text
+When ACTION occurs under CONDITION, the owner must produce OUTCOME while
+preserving or avoiding NON_EFFECT. A faulty implementation that REGRESSION
+must make this test fail.
+```
+
+A merge-worthy test has all of these properties:
+
+- **Contract:** it protects observable behavior, an invariant, compatibility,
+  or a previously observed failure that matters.
+- **Sensitivity:** a credible wrong implementation, including the old bug for a
+  regression, fails an assertion for the intended reason.
+- **Fidelity:** it invokes the production owner through the lowest credible
+  boundary instead of reproducing or mocking away the behavior.
+- **Oracle:** it asserts stable outcomes and important non-effects rather than
+  incidental structure.
+- **Determinism:** time, concurrency, randomness, state, and external resources
+  produce the same verdict on a busy runner.
+- **Execution:** the blocking repository lane discovers the test, and every
+  platform-specific arm runs on its native platform.
+
+Adding no test is preferable to adding evidence that only restates a constant,
+type, mock arrangement, or presentation detail. Test count, assertion count,
+and line coverage are diagnostic metrics, not quality verdicts.
+
+## Choose the lowest credible boundary
+
+“Unit” describes isolation, not importance. Use the lowest level that can
+observe the named risk without replacing it.
+
+| Level                 | Appropriate risk                                                           | Boundary that remains real                |
+| --------------------- | -------------------------------------------------------------------------- | ----------------------------------------- |
+| Pure unit             | Parsing, normalization, projection, policy, deterministic algorithms       | None                                      |
+| Behavioral component  | One controller, store, service, or UI interaction                          | Internal production wiring                |
+| Conformance           | A contract shared by implementations or consumers                          | Public contract and driver                |
+| Narrow integration    | SQLite, filesystem, Git, HTTP, IPC, parser, subprocess, or OS semantics    | The boundary named by the risk            |
+| End to end            | A journey through a composition root                                       | Public entry point to user-visible result |
+| Visual or performance | Pixels, layout, animation, responsiveness, or timing budgets               | Real renderer or trace                    |
+| Repository check      | Imports, generated artifacts, manifests, ownership, or source architecture | Repository files themselves               |
+
+A test named E2E that bypasses the composition root and replaces all internal
+collaborators is a component test. A colocated test that launches a real
+subprocess is an integration test.
+
+Prefer one vertical proof through the owning boundary plus narrow unit cases for
+important decisions. A helper-only unit test cannot prove that production
+callers still invoke the helper. Tutti has encountered this exact failure with
+remote-image materialization; see the
+[troubleshooting entry](./troubleshooting/agent-session-lifecycle.md#remote-agent-image-reaches-the-provider-as-an-unsupported-url).
+
+### Tutti routing rules
+
+- New Session, Turn, Goal, runtime-operation, or recovery semantics start as a
+  scenario in `packages/agent/host/conformance`. Consumer adapters implement
+  drivers; they do not clone lifecycle semantics.
+- Adapter tests own transport, DTO, authorization, query, presentation, and
+  product-policy behavior.
+- SQLite, filesystem, paths, Git, command quoting, executable discovery,
+  process lifecycle, IPC, and native API behavior require a real isolated
+  instance of the relevant boundary.
+- UI behavior belongs in a behavioral component test that drives accessible
+  keyboard, pointer, focus, or state transitions. Pixels and layout belong in
+  visual evidence.
+- HTTP and provider protocol behavior should cross the real parser, middleware,
+  codec, or receiver when that behavior is the risk.
+- Host filesystem, shell, process, registry, signal, permission, and executable
+  behavior must run on the native OS. A request-shape assertion cannot replace
+  the receiving Windows API or process boundary.
+
+## Design from risk
+
+Identify the module that owns the decision and the boundary where a caller can
+observe it. Stable observations include:
+
+- returned domain values and typed errors;
+- canonical or persisted state read through the owning store;
+- emitted events and observable ordering;
+- the final provider, HTTP, IPC, parser, or process request;
+- filesystem contents, permissions, and executable behavior;
+- accessible UI state, callbacks, enabled actions, and focus;
+- absence of a write, dispatch, retry, duplicate, secret, or leaked resource.
+
+Select only input partitions that can change the decision or represent a real
+failure mode: missing and malformed input, exact boundaries, identity reuse,
+ordering, partial failure, response loss, restart, cancellation, competing
+writers, preservation of unrelated state, and native-platform differences.
+
+One test may use several assertions when they jointly prove one behavior. Test
+names should state cause and observable effect, such as `retries transient
+catalog failures but not permanent failures`, rather than `works` or a private
+method name.
+
+### Prove regression sensitivity
+
+Capture a confirmed bug's failing reproduction before the fix whenever safe.
+The resulting regression test must fail on pre-fix behavior for the intended
+reason. If the old revision cannot run, temporarily restore or inject the faulty
+decision as a local negative control; do not commit that mutation.
+
+For new behavior, first show that the focused test fails because the contract is
+missing, not because test setup, imports, or fixtures are broken. When
+sensitivity is not obvious, include the negative-control evidence in the PR.
+
+## Assert contracts, not implementation shape
+
+Prefer observations in this order:
+
+1. externally visible result or typed error;
+2. durable state and preservation of unrelated state;
+3. event or protocol sequence at an owned boundary;
+4. side-effect count when count is itself the contract;
+5. a narrow internal observation only when no stable boundary exists.
+
+Failure-path tests should normally assert both what happened and what did not:
+no provider request, no canonical row, no rollback of an earlier commit, no
+stale output, no secret in diagnostics, or no new identity during a retry.
+`does not throw` or `err == nil` alone is rarely a sufficient oracle.
+
+### Prefer invariants to change detectors
+
+A change detector freezes data expected to evolve: an exact catalog size,
+current model name, schema-version literal, copied defaults object, snapshot of
+a registry, or duplicated lookup table. Protect the relationship consumers rely
+on instead: every selectable model has required metadata, migrations end at the
+declared current version, identities stay unique, or forbidden sets do not
+overlap.
+
+Use snapshots only for deliberately stable public serialization, generated
+contracts, or normalized CLI output. Keep them small, normalize paths, IDs,
+time, width, and platform noise, and pair them with semantic assertions.
+
+### Keep source contracts out of runtime unit tests
+
+A runtime test must not read production `.go`, `.ts`, or `.tsx` text and use a
+string, regex, or `indexOf` as proof that a call or option is wired correctly.
+Execute the runtime seam or extract the real decision into its owner.
+
+When source shape is intentionally the contract—imports, generated files,
+manifests, architecture, or security policy—implement a named repository check
+under `tools/scripts` and register it with the repository-check infrastructure.
+That check is valuable, but it is not runtime unit coverage.
+
+## Use real owned resources and narrow doubles
+
+Use temporary real implementations when their semantics are part of the risk:
+SQLite for transactions and recovery, a filesystem for paths and atomic writes,
+a local HTTP server for redirects and streaming, a temporary Git repository for
+worktree behavior, and a subprocess or parser for quoting and wire contracts.
+
+Use a double when the collaborator is externally owned, nondeterministic,
+unsafe, or irrelevant to the current decision. A double should implement the
+smallest owning port, return explicit configured outcomes, fail immediately on
+unexpected calls, and capture only contract observations. Never make every
+method silently succeed. A test that stubs every layer and verifies that the
+stubs called one another proves its arrangement, not the product.
+
+Isolate state before production imports can capture it. Tests must not read or
+write real Tutti state, credentials, provider configuration, user Git config,
+or developer services. Always restore environment variables, clocks, globals,
+listeners, timers, stores, sockets, subprocesses, and temporary resources,
+including on failure.
+
+## Make concurrency deterministic
+
+Express the required happens-before relationship with signals:
+
+1. operation A reaches a named gate;
+2. operation B is submitted while A is held;
+3. the test observes the contract in that state;
+4. the gate is released;
+5. both operations settle and final state is asserted.
+
+Use channels, barriers, latches, deferred values, or domain events. A timeout is
+a generous deadlock guard around an awaited signal, not readiness evidence or
+the behavioral oracle. “Nothing happened for 20 ms” does not prove that nothing
+can happen.
+
+Inject clocks, timers, deadline errors, random sources, IDs, or schedulers when
+the decision depends on them. Test immediately before, at, and after thresholds.
+Retries may expose a flake, but they do not repair it; a pass-on-retry must remain
+visible as a defect.
+
+## Preserve suite and CI integrity
+
+- Keep small suites beside their owner. Split large lifecycle, recovery,
+  concurrency, migration, rendering, or platform scenarios by behavior rather
+  than creating one file per private helper or one giant suite.
+- Keep tests order-independent. Parallelize only when they do not share mutable
+  globals, environment, ports, clocks, files, or fixtures.
+- Every test must be discovered by its package command and the appropriate root
+  changed-aware lane. A selected suite that collects zero tests must fail.
+- Empty files, committed `.only`, required missing fixtures, and suites whose
+  meaningful cases all skip are defects.
+- Platform markers, build tags, and filename filters must map to a blocking
+  native CI lane.
+- Keep unit tests offline. Live-provider qualification belongs in an explicit,
+  isolated lane or sanitized record/replay workflow.
+
+Coverage can identify unvisited branches, but a repository-wide percentage
+rewards shallow render tests, copied constants, and mocked call graphs. For
+critical lifecycle, authorization, persistence, recovery, and security logic,
+targeted fault injection or mutation sampling is a stronger audit.
+
+## Repository examples
+
+Use these as patterns for their specific risks, not as templates to copy:
+
+- [Agent Host conformance](../../packages/agent/host/conformance/scenarios.go)
+  expresses lifecycle behavior once for multiple consumers.
+- [Managed runtime tests](../../services/tuttid/service/managedruntime/runtime_test.go)
+  distinguish transient and permanent failures, use a local HTTP server, and
+  assert retries, cancellation, and response cleanup.
+- [Combobox behavior](../../packages/ui/system/src/components/combobox/combobox.spec.tsx)
+  drives keyboard and pointer actions and observes selection, disabled behavior,
+  filtering, accessibility, and closure.
+- [Windows Codex contract](../../packages/agent/daemon/runtime/codex_appserver_windows_contract_test.go)
+  checks behavior against the real Windows receiver instead of trusting a
+  serialized request shape.