openwork/docs/remote-chat-over-mcp-architecture.md

# Remote Chat over MCP — sessions on another surface via the OpenWork MCP Gateway

Status: proposal
Owner: TBD
Related: `docs/marketplace-capabilities-architecture.md`, `ee/apps/den-api/src/mcp/README.md`

Covers two execution targets behind the same gateway capabilities:
`target: "cloud"` (session on the member's OpenWork Web cloud worker) and
`target: "desktop"` (session on one of the member's signed-in desktop apps,
dispatched over the existing Desktop Automation runner channel).

## Problem

OpenWork Desktop and OpenWork Web are both live chat surfaces, but they cannot
talk to each other. A member working in a desktop chat has no way to start or
continue a conversation that runs on their OpenWork Web cloud worker — for
example to hand long-running work to the cloud, to make a session visible to
teammates in the browser, or to keep work running after the laptop closes.

At the same time, the MCP gateway (`/mcp/agent`) is the sanctioned way to
extend what agents can do org-wide, and desktop already attaches it to every
workspace engine. Anything we build as a gateway capability works from desktop
**and** from every external MCP client (Claude Code, Codex, Cursor) for free.

## Goal

From a desktop chat (or any MCP client connected to the gateway), the agent
can:

1. Create a chat session on the member's OpenWork Web cloud worker.
2. Send prompts to it.
3. Read back status and transcript.
4. Hand the member a link/card to open that session live in OpenWork Web.

Non-goals (v1):

- Streaming token-by-token output back over MCP (poll/read instead).
- Cross-member or cross-org sessions. Everything is scoped to the signed-in
  member's own worker.
- A new chat UI. OpenWork Web renders the session with its existing UI because
  the session is a native openwork-server session — no sync layer needed.

## What exists today (reused, not rebuilt)

| Piece | Where | Role in this design |
| --- | --- | --- |
| MCP gateway | `ee/apps/den-api/src/mcp/agent.ts` | Hosts `search_capabilities` / `execute_capability`; new work plugs in behind them |
| Capability registry | `ee/apps/den-api/src/mcp/capability-registry.ts` | Fan-out to capability sources — **the extension point** |
| Worker resolution | den-api `/v1/cloud/gateway/resolve` (used by `ee/apps/den-gateway/src/app.ts`) | Maps a member to their cloud openwork-server instance + auth |
| openwork-server session API | `apps/server` (`POST /workspace/:id/opencode/session`, native prompt/transcript/status routes) | The actual chat runtime on the worker |
| Programmatic session client | `packages/headless-threads` | Typed client for driving native sessions from code |
| Desktop → gateway attach | `apps/app/src/react-app/domains/connections/cloud-mcp-reconciler.ts` + `apps/server/src/routes/cloud-mcp.ts` (token mint: `POST /v1/mcp/token`) | Desktop engines already have `/mcp/agent`; zero desktop changes required for v1 |
| MCP App cards | `ee/apps/den-api/src/mcp/connection-action-app.ts` pattern + `packages/mcp-apps` | Render an "Open in OpenWork Web" card |

## Architecture

```
Desktop chat (engine)                den-api (/mcp/agent)              Cloud worker (openwork-server)
─────────────────────                ────────────────────              ──────────────────────────────
agent calls                          capability-registry
search_capabilities ───────────────▶  └─ remote-session source
execute_capability                       │ resolve member worker
 "remoteSession.create" ───────────▶     │  (cloud/gateway/resolve) ──▶ POST /workspace/:id/opencode/session
 "remoteSession.send" ─────────────▶     │                        ───▶ POST /workspace/:id/opencode/session/:id/prompt_async
 "remoteSession.read" ─────────────▶     │                        ───▶ GET  native session/message/todo/status routes
                                         └ returns result + MCP App
                                           card linking to OpenWork Web
                                                                        OpenWork Web (apps/app via
                                                                        den-gateway) shows the session
                                                                        live — same server, no sync
```

Key property: the "web MCP" is not a second MCP server. The gateway is the
single MCP surface; OpenWork Web sees the session because the session lives on
the same openwork-server the web UI already reads through den-gateway.

### 1. New capability source: `remote-session-capabilities.ts`

New file in `ee/apps/den-api/src/mcp/`, registered in
`capability-registry.ts` alongside `builtin-skills.ts` /
`native-capabilities.ts`. It contributes capabilities to
`search_capabilities` and handles their execution — **not** new top-level MCP
tools, keeping the gateway's small-constant-tool-list contract intact.

Capabilities (names indicative):

| Capability | Args | Behavior |
| --- | --- | --- |
| `remoteSession.create` | `{ target: "cloud" \| "desktop", title?, workspaceId?, instructions?, runnerId? }` | Cloud: resolve member worker, create session. Desktop: enqueue a claimable task for the member's runner (§6). Returns `{ sessionId \| taskId, target, webUrl? }` |
| `remoteSession.send` | `{ sessionId, prompt }` | Cloud: `prompt_async` to the session. Desktop: append a follow-up task bound to the same local session. Returns a receipt `{ messageId, state: "accepted" }` |
| `remoteSession.read` | `{ sessionId, since? }` | Session status + transcript slice (bounded, paginated). Cloud: read from worker. Desktop: read from Den-side task events (§6) |
| `remoteSession.list` | `{ target? }` | Member's remote sessions; for desktop also lists runner presence per machine |

Execution path inside the source:

1. Member identity comes from the already-verified MCP auth context
   (`auth.ts`) — never from arguments.
2. Resolve the worker exactly as den-gateway does (`/v1/cloud/gateway/resolve`)
   via a shared internal helper; do not duplicate resolution logic.
3. Call openwork-server using the `packages/headless-threads` client (or its
   underlying HTTP shape) with the resolved worker URL + token.
4. Map worker errors to gateway error vocabulary: no worker provisioned →
   actionable "needs cloud worker" result (relay the human step, mirroring
   `needs_admin_setup` / `needs_signin` conventions), never a fake success.

### 2. Async model

`prompt_async` returns immediately; the agent polls with
`remoteSession.read`. This matches MCP's request/response grain and avoids
holding gateway connections open. If later we want push, the gateway can add
MCP resource subscriptions on a `remote-session://` resource (pattern already
exists: `automation-index.ts`, `resource.ts`) — explicitly out of scope for v1.

### 3. MCP App card

`remoteSession.create` and `remoteSession.send` results include a standard
MCP Apps `ui://` card (pattern: `connection-action-app.ts` +
`packages/mcp-apps` renderer): session title, state, last activity, and an
**Open in OpenWork Web** link (`https://web.openworklabs.com/...` deep link,
resolved from runtime config — same origin den-web's "Web tab" uses). Clients
without MCP Apps get text fallback with the same URL.

### 4. Policy and scopes

- Gate the whole source behind a scope in `ee/apps/den-api/src/mcp/scopes.ts`
  (e.g. `remote-session`), included in first-party desktop tokens
  (`/v1/mcp/token`) and public OAuth tokens by default; org policy can turn it
  off via the existing exposure allowlist mechanics (`policy.ts` conventions).
- **Org capability flag**: Cloud is default-off per organization
  (`metadata.capabilities.cloud`, `cloud-rollout.ts`). When the flag is off,
  the source is invisible in `search_capabilities` and execute reports
  `unknown_capability` — mirroring the external-MCP rollout pattern, so
  members of a flag-off org never see an action they cannot take. The runtime
  re-checks the flag live at execute time (`cloud_not_available`, an
  admin-facing action) as defense in depth against mid-session flag flips;
  `needs_cloud_setup` is reserved for the member-facing "open OpenWork Cloud
  once to provision" case.
- Member-scoped only: capabilities operate on the caller's resolved worker.
  `sessionId` from another member's worker must 404, not 403-leak.
- Prompts transit the gateway but are stored only on the worker (same trust
  domain as existing web chat). No prompt content in gateway logs; log
  receipts/ids only.

### 5. Desktop side (v1: zero code)

Desktop engines already carry the `openwork-cloud` MCP entry. The agent
discovers `remoteSession.*` via `search_capabilities` naturally. Optional
polish, in order:

1. A built-in remote skill ("send this to my web chat") teaching the flow —
   `builtin-skills.ts`, no client release needed.
2. Desktop UI affordance ("Continue in Web") that pre-fills the prompt —
   later, once the capability path is proven.

### 6. Desktop execution target — reuse the Automation runner channel

Web → desktop communication already exists: Desktop-placed Automations. Den
never reaches into a desktop; the signed-in desktop app pulls work over an
authenticated channel and reports back. That channel is task-agnostic in
everything but naming, and `target: "desktop"` reuses it instead of inventing
a second dispatch path.

What ships today (all reused):

- **Runner client:** `apps/desktop/electron/automation-runner.mjs` polls
  `GET /v1/automation-runner/work` every 60 s (wake-on-resume via
  `powerMonitor`), claims runs, heartbeats every 10 s (heartbeat response
  carries `cancelRequested` + lease validity), posts ordered events, and
  completes with a durable receipt.
- **Runner identity/auth:** per-machine `runnerId` (persisted UUID) +
  HMAC runner token (12 h TTL, audience-bound to the minting Den base URL),
  minted by the renderer with member auth via `POST /v1/automation-runners/token`
  (`automation-runner-bridge.tsx`). The mint route is `"x-mcp": false` —
  MCP callers can never obtain runner credentials. This stays true here:
  the gateway capability only **enqueues tasks**; it never touches runner auth.
- **Execution shape:** the assignment is already `{instructions, model,
  timeoutMs}` and `executeDesktopAutomation` runs it as a **normal visible
  local session** (`POST /workspace/:id/opencode/session`), returning `sessionId` /
  `workspaceId` into the Den receipt. Exactly the semantics we want: remote
  work on desktop is a real thread the human can watch and take over.
- **Built-in extension points:** runner registration declares
  `protocolVersion`, `capabilities`, and `supportedExecutionTargets`
  (`packages/types/src/automations.ts`), so a new work kind is a versioned,
  negotiated addition — old runners simply never see it.

Flow:

```
Any MCP client            den-api                          Desktop app (Electron main)
──────────────            ───────                          ───────────────────────────
execute_capability
 remoteSession.create ──▶ enqueue remote-session task
   {target:"desktop"}     (claimable, deadline-bound,
                           same tables/protocol as
                           desktop automation runs)
                                                           GET /v1/automation-runner/work
                                                             ← task {kind:"remote_session", …}
                                                           claim → lease → create visible
                                                           local session → heartbeat →
                                                           POST events (assistant text,
                                                             sessionId, usage, terminal)
 remoteSession.read ────▶ read task events from Den ◀──────POST complete {receipt}
                          (Den never contacts desktop)
```

**Reuse boundary — promote the channel, don't squat on it.** The runner
*protocol* (claim/lease/heartbeat/events/complete), runner *identity/auth*,
and *presence* are task-agnostic and shared. But automation **run tables, run
history, receipts, and schedule-native lifecycle states** (missed occurrence,
claim deadline bounded by next occurrence) are automation semantics —
`remote_session` tasks must NOT be stored as automation runs or appear in
`listAutomationRuns`. The channel is generalized into a desktop task dispatch
seam on which automations are one work kind, not the landlord.

Deltas required (small, additive):

1. **Work kind on a generalized seam, not a new channel**: extract the
   claim/lease/heartbeat/events/complete protocol + runner auth/presence as a
   shared layer. Server-first migration: introduce generic runner endpoints
   (e.g. `/v1/desktop-runner/*`) and keep `/v1/automation-runner/*` as
   compatible aliases — the runner ships inside Electron, so old clients live
   for a long time. `remote_session` tasks get their own table and lifecycle
   vocabulary (`queued/claimed/running/interrupted/done` — no "missed
   occurrence"). Runner token capabilities gain a `remote_session` grant next
   to automation grants so org policy can allow one without the other. The
   work kind is negotiated through the existing `capabilities` /
   `protocolVersion` handshake.
2. **Runner handler:** in `automation-runner.mjs`, a `remote_session` task
   executes exactly like an automation run (visible session titled from the
   request), except follow-up `remoteSession.send` tasks resume the same
   local `sessionId` instead of creating a new one.
3. **Latency:** the 60 s work poll is fine for fire-and-forget handoff but
   sluggish for conversation. The low-latency seam is already built and
   tested server-side: the SSE stream `GET /v1/automation-runners/events`
   (`automation_work_available` notifications) — the runner just doesn't
   consume it yet. Consuming `eventsPath` is the single change that makes
   desktop dispatch near-real-time, and it benefits Automations too.
4. **Machine selection:** a member may have several desktops. Default to the
   most-recently-seen runner within the presence window
   (`GET /v1/automation-runners/presence`, 10 min window); allow explicit
   `runnerId` (from `remoteSession.list`) to pin a machine. No runner in the
   presence window → actionable "open OpenWork Desktop and sign in" result,
   never a silent queue-forever.
5. **Transcript reads:** Den cannot query the desktop, so `remoteSession.read`
   for desktop targets reads the ordered task events the runner posted
   (assistant text, state, `sessionId`) — same durable store automation run
   receipts use. Full-fidelity transcript stays on the desktop; events carry
   the summary + final output.

Security invariants preserved from the automation channel:

- Runner tokens: first-party mint only, audience-bound, owner-revocable,
  instantly dead if the member leaves the org. MCP can enqueue, never run.
- Same-member only: tasks are scoped `{organizationId, ownerMemberId}`; a
  gateway caller can only dispatch to their **own** desktops. Cross-member
  dispatch is a non-goal and must 404.
- Desktop execution stays **visible** (a real thread), consistent with the
  deliberate automation-runner decision — no headless remote control of a
  member's machine.
- Model access is re-checked at claim time, same as automation claims.

### 7. Extending the gateway generally (the reusable recipe)

This feature doubles as the template for future gateway extensions:

- **New discoverable behavior** → new capability source file registered in
  `capability-registry.ts`, gated by a scope, surfaced through
  `search_capabilities` / `execute_capability`.
- **New rich result UI** → `registerAgentXxxApp` helper + renderer in
  `packages/mcp-apps`.
- **New top-level MCP tool** → only when the tool must exist without
  discovery (rare; requires updating every client's mental model). Avoid.

## Offline and delivery semantics

The two targets are asymmetric: cloud runtimes can be booted on demand;
desktops cannot.

### Cloud target — auto-wake, bounded wait, retryable state

"Web not running" usually means the member's cloud worker is idle-stopped
(Den stops healthy workers after 30 min idle, `stopIdleCloudWorkers`), not
that anything is missing — the browser being closed never matters, since the
session runtime is the worker and the web UI is just a viewer.

- Existing behavior reused: resolving a stopped worker fire-and-forget wakes
  it and reports `"waking"` (`resolveCloudInstance` →
  `wakeCloudWorker`, provision deadline 15 min, silent reprovision if the
  sandbox is gone). Cloud automation runs additionally await the wake and
  poll readiness at 1 s intervals up to 120 s (`cloud-agent-executor.ts`).
- `remoteSession.create` / `send {target:"cloud"}`: trigger the wake, poll
  readiness within a short in-call budget (~20–30 s to keep the MCP call
  responsive), otherwise return a **retryable `waking` state**; the agent
  retries the same idempotent call. No durable queue — the wake is already
  in flight server-side.
- No worker provisioned at all (vs. stopped) → actionable needs-setup
  result, same as the failure-modes table.

### Desktop target — fail fast by default, opt-in store-and-forward

Den cannot power on a machine and never contacts the desktop; the runner
pulls. Therefore:

- **Default: fail fast.** No runner inside the 10-min presence window →
  `remoteSession.create {target:"desktop"}` returns the exact human action
  ("open OpenWork Desktop on <machine> and sign in"). Conversations are not
  silently queued against machines that may be off for days.
- **Sleeping ≠ offline.** A running app on a sleeping laptop wakes and polls
  immediately on `powerMonitor` resume/unlock, so delivery after lid-open is
  seconds, not the 60 s poll interval.
- **Opt-in queueing:** an explicit `deliverBy` timestamp on the task enables
  store-and-forward for handoff-style work ("run on my desktop whenever it's
  online today"). A task unclaimed past `deliverBy` transitions to `expired`
  with a clear state readable via `remoteSession.read` — analogous to missed
  desktop automation runs, but expressed in the remote-session lifecycle
  vocabulary, never silently dropped.

## Failure modes

| Failure | Behavior |
| --- | --- |
| Member has no cloud worker | `remoteSession.create` returns an actionable needs-setup result naming the exact human step (provision worker in Den dashboard) |
| Worker unreachable / asleep | Retryable error with `referenceId`; no partial session creation |
| Session deleted on web | `remoteSession.send`/`read` return `unknown_session`, prompting `remoteSession.list` |
| Token expiry mid-flow | Standard gateway 401 semantics; desktop reconciler already auto-refreshes |
| No desktop runner in presence window | `remoteSession.create {target:"desktop"}` returns actionable "open OpenWork Desktop and sign in on <machine>" — never queues forever |
| Desktop task unclaimed past deadline | Marked missed with cause, same as missed desktop automation runs; surfaced via `remoteSession.read` |
| Desktop goes offline mid-task | Lease expires (60 s heartbeat loss) → task marked interrupted with last posted events preserved |

## Rollout

1. **Phase 1** — capability source + worker routing + text results. Proof:
   testkit spec in `evals/specs/` driving desktop engine → gateway → mock
   worker (witness for openwork-server, per `build-a-witness`), asserting
   session creation, prompt receipt, transcript read, and member-scoping
   (cross-member 404).
2. **Phase 2** — MCP App card + built-in skill. Proof: MCP App render spec
   (pattern: existing `mcp-app-render-test`), plus e2e where the session
   created from desktop appears in headless web UI (`worlds/dev-headless.ts`).
3. **Phase 3** — `target: "desktop"`: generalize the runner seam server-first
   (generic `/v1/desktop-runner/*` endpoints with `/v1/automation-runner/*`
   aliases, dedicated `remote_session` task table), then add the
   `remote_session` work kind + runner handler in `automation-runner.mjs`,
   negotiated via `protocolVersion`/`capabilities`. Proof: testkit spec with
   a fake runner exercising claim/heartbeat/events/complete for the new kind,
   member-scoping, no-runner-present, and an assertion that remote-session
   tasks never appear in `listAutomationRuns`.
4. **Phase 4** — runner consumes the existing SSE stream
   (`/v1/automation-runners/events`) for near-real-time dispatch (shared win
   with Automations); external clients doc page (Claude Code/Codex get both
   targets with no extra work); optional MCP resource subscriptions for live
   transcript updates.

## Alternatives considered

- **Desktop talks to the worker directly (connect-link handoff), skipping the
  gateway.** Fewer hops, but only works from OpenWork Desktop, duplicates
  worker resolution/auth in the client, and bypasses org policy. Rejected.
- **New top-level MCP tools (`create_remote_chat`, `send_to_session`).**
  Breaks the gateway's deliberate two-tool discovery contract and bloats
  every connected client. Rejected in favor of a capability source.
- **A new dedicated cloud→desktop channel (WebSocket/push) for desktop
  targets.** The automation runner channel already provides claim/lease/
  heartbeat/receipts with hardened runner auth, and its SSE wake-up stream is
  already built server-side. A parallel channel would duplicate all of it and
  double the desktop's attack surface. Rejected — extend the work-kind schema
  instead.
- **Letting MCP callers mint or hold runner tokens to talk to desktops
  "directly".** Violates the deliberate `x-mcp: false` boundary on
  `/v1/automation-runners/token`; the gateway must only enqueue tasks.
  Rejected.
- **A durable-execution engine (Temporal, Inngest) for dispatch and
  delivery.** The genuine unlock (long-lived multi-step sagas with timers and
  signals) does not exist in this design: every operation is either
  request-scoped (cloud wake-and-poll, <2 min) or edge-pull (the desktop
  runner cannot be an engine worker behind NAT — the claim/lease protocol
  stays bespoke regardless). Den's durability model is already crash-safe
  state machines in Postgres (worker status as cross-replica mutex, claimable
  rows + deadline sweeps, durable receipts), so an engine would be a
  re-platform, not an enhancement — and it adds a second source of truth
  beside the receipt tables users read, plus heavy self-host burden for OSS
  deployers (own cluster/datastore for Temporal; second runtime for
  Inngest) and Temporal's determinism/versioning tax. Rejected for this
  design. Escalation path if pressure appears: (1) Postgres-native queue
  (pg-boss / graphile-worker) — no new vendor, ships with the existing DB;
  (2) only if the Workflows product ever needs true sagas, an internal
  orchestration seam with Postgres as the default implementation and an
  engine as an optional adapter, so self-deploy stays vanilla Postgres.
- **A separate MCP server hosted by OpenWork Web.** There is no web MCP
  server today (den-web is dashboard-only); building one duplicates auth,
  policy, and exposure logic the gateway already owns. Rejected.