# Computer Use Troubleshooting Use this guide for recurring cua-driver and `tutti computer` failures. Keep stable CLI and authorization contracts in [Tutti CLI Contract](../tutti-cli-contract.md); keep symptom-driven diagnosis and recovery here. ## Computer use keeps reopening setup after installation or authorization ### Symptom The Desktop settings row reports that computer use is installed and authorized, but selecting `/computer` opens setup again. On macOS, the first visit to a Privacy & Security pane may not list CuaDriver, or Screen Recording may remain granted but not effective until the driver is restarted. ### Root cause and fix Desktop permission status and Agent Composer options refresh independently. A fresh host probe may therefore be authorized while an older Composer snapshot still reports computer use as unavailable. Treat the fresh installed and authorized host status as sufficient for the Composer interaction; the daemon still performs its authoritative readiness clamp when the session launches. On macOS, start the user-initiated permission grant before opening System Settings so TCC has registered CuaDriver on the first visit. After both Accessibility and Screen Recording are granted but capture is not yet effective, restart CuaDriver through its app bundle and then read status again; plain polling cannot clear the process-cached permission state. ### Validation - An uninstalled driver still routes `/computer` to setup. - An installed and authorized driver inserts and submits `/computer` without reopening setup, even if Composer options were loaded before authorization. - CuaDriver appears on the first user-initiated visit to each macOS permission pane. - Returning from macOS System Settings reconciles granted permissions and capture readiness without restarting Tutti. - Windows continues to use `doctor --json`; macOS-only reconciliation never runs on Windows. ## CuaDriver permissions are granted but every input tool is denied ### Symptom macOS Accessibility and Screen Recording are both granted. Window listing, screenshots, and the AX tree work, but native `click`, `move_cursor`, `press_key`, `type_text`, and `scroll` appear with `allowed: false`. Their denial reason is `tool has denied or unknown capabilities: input.delivery_mode`. ### Root cause and fix CuaDriver 0.20 adds `input.delivery_mode` to tools that support its explicit background/foreground input-delivery ladder. Tutti's native-tool policy is fail-closed and previously did not recognize that capability, so it rejected the complete tool whenever the otherwise authorized input action advertised the new metadata. The same catalog also declares `input.pointer.move` on `move_cursor`; that capability must accompany the existing `agent_cursor.move` authorization. This is not an observation-only computer session and is not a stale macOS TCC grant. Authorize `input.delivery_mode` and `input.pointer.move` alongside the existing keyboard, pointer, and `window.activate` capabilities. Delivery mode does not grant a new class of input by itself; it selects delivery posture for an input tool already allowed by Tutti. Keep unknown capabilities denied. ### Validation - With CuaDriver 0.20, `tutti computer tool list --json` marks the input tools allowed while a tool carrying an unrelated future capability remains denied. - Background delivery remains the default; foreground delivery is explicit and uses the live native schema. - Restart the updated tuttid process so it loads the new policy. Existing Agent conversations do not need to be recreated because native tools are listed and authorized live on every call. ## Screen Recording is enabled but Tutti says it has not taken effect ### Symptom macOS System Settings shows CuaDriver enabled for both Accessibility and Screen Recording, but Tutti reports Screen Recording as authorized but not effective. `cua-driver permissions status --json` may report both TCC grants as `true` while returning `screen_recording_capturable: null` and `direct_capture_status: "not_checked"`. ### Root cause and fix CuaDriver 0.20 keeps its read-only permission status content-free and does not run the prompt-capable direct-capture probe. A nullable `screen_recording_capturable` therefore means the probe was not run, not that capture failed. Tutti treats the two states separately: an explicit `false` remains not ready, while `null` is accepted when Accessibility and Screen Recording are both granted. Actual computer actions still cross CuaDriver's own permission enforcement boundary. Do not repeatedly toggle the macOS permission or reinstall CuaDriver solely because the nullable probe field is present. Use the exact daemon-attributed status payload to distinguish an unprobed result from an explicit capture failure. ### Validation - `cua-driver permissions status --json` attributes the response to `com.trycua.driver` and reports Accessibility and Screen Recording as `true`. - A nullable capture probe no longer keeps Tutti in the authorization wizard. - An explicit `screen_recording_capturable: false` still shows the recovery path and blocks computer-use readiness. - The daemon and desktop apply the same nullable-versus-false interpretation. ## A computer click reports success but the UI does not change ### Symptom A stable or native computer click reports that it was posted, but a fresh screenshot shows no UI change. This commonly affects Electron-based apps. The agent-cursor overlay may also appear offset from the intended point after a display-configuration change. ### Quick checks 1. Capture a fresh target-window screenshot with the same explicit `pid` and `window-id` used for the action. 2. Treat a posted-click response as dispatch confirmation only. Do not use the agent-cursor overlay as evidence that the event landed at the displayed point. 3. Inspect the screenshot structured content for the target's `element_token`. If no suitable element exists, confirm that pixel coordinates came from the latest screenshot of the same window. 4. Use `computer tool describe --name click --json` before native escalation so the live cua-driver schema remains the source of truth. ### Root cause Pixel clicks use background CGEvent delivery. cua-driver can confirm that the event was dispatched, but it cannot read back the UI effect. Focus-sensitive or Electron surfaces may silently discard a background synthetic click. The agent-cursor overlay is rendered through a separate visual channel and may have a stale display offset, so its apparent position does not validate event delivery. ### Fix 1. Prefer the native `click` element-token path from the latest screenshot when an actionable element is available. 2. Verify the result with another fresh screenshot. 3. If a background pixel click had no effect, do not repeat the same click. Follow the live native schema and retry once with `delivery_mode: "foreground"`, which briefly fronts the target window. 4. If foreground delivery still has no visible effect, re-snapshot and re-resolve the target instead of reusing stale coordinates or tokens. ### Validation - The post-action screenshot shows the expected state change. - The action used the same `pid` and `window-id` as its source screenshot. - Element-token retries use a token from the latest snapshot; stale-token errors trigger a new snapshot. - Pixel escalation follows background, verify, foreground, verify rather than repeated unverified clicks. ### References - [Tutti CLI Contract: Computer command surfaces](../tutti-cli-contract.md#computer-command-surfaces) - [Injected Computer Use skill](../../../packages/agent/runtimeprep/skill_templates/computer-use.md) ## A Windows native tool call rejects valid-looking JSON ### Symptom `computer tool call` reports that `arguments-json` is not a JSON object, or a nested `powershell.exe -Command` invocation changes quotes and backslashes. ### Root cause and fix JSON embedded in a command string crosses both the outer launcher and Windows shell parsing boundaries. Serialize the object with the shell's JSON library and pipe it through `--arguments-json -`; do not launch a second PowerShell process around the Tutti CLI command. `--arguments-json` remains supported for simple existing scripts, but the two input sources cannot be combined. When using Windows PowerShell 5.1, temporarily set `$OutputEncoding` to `[Text.UTF8Encoding]::new($false)` around the pipeline and restore it in a `finally` block; its default ASCII encoding cannot preserve non-ASCII JSON. If the driver returns `isError=true` together with Windows success status `0x00000000`, treat it as an inconsistent driver result rather than successful delivery. If it rejects automation of its own authorization process, choose a different application target; never automate the authorization UI. ### Validation - Stdin JSON round-trips nested values, backslashes, spaces, and non-ASCII text. - Every state-changing action is followed by a fresh screenshot. - An inconsistent success-code error and a protected authorization target have explicit diagnostic messages. ## A Windows CUA doctor check is slow or reports UIA fallback ### Symptom Windows computer-use status takes several seconds, or the driver doctor reports that optional UI Automation support is unhealthy and is falling back to Win32-only window tools. A status response may still be authorized because the installed driver can perform the core Win32 actions. ### Root cause and fix The Windows doctor probes both the optional UIA adapter and the native Win32 path. A broken accessibility provider can make the optional probe slow or degraded even when native input remains usable. The desktop and daemon bound the doctor process at 10 seconds and classify the explicit UIA-to-Win32 fallback as a degraded-but-usable result. CuaDriver 0.18 may emit that fallback as an ANSI-colored stderr warning before the JSON document rather than inside a JSON diagnostic field. Both adapters therefore inspect the complete output, but only after validating that it contains a parseable doctor JSON result. The desktop preserves the diagnostic with reason `driver-doctor-failed`, while the daemon keeps host-managed computer tools available. A real timeout, malformed response, or missing driver remains an unknown/not-ready result; do not silently treat those as healthy. ### Validation - Run the exact installed `cua-driver doctor --json` executable and record whether the response is healthy, degraded, timed out, or unparseable. - Confirm degraded output keeps Win32 computer actions available while the diagnostic warns that UIA-specific window operations may not work. - Cover both a fallback stored inside the JSON document and an ANSI/stderr fallback prefix followed by an otherwise valid JSON document. - Confirm a real doctor timeout returns within 10 seconds and does not leave a child process running.