2026-09-02 01:05:41 +08:00
commit 04bac1f6e2
2200 changed files with 489415 additions and 0 deletions
@@ -0,0 +1,36 @@
# MCP OAuth conformance proof
Historical record: these screenshots were captured by the legacy `mcp-oauth-conformance` flow on
July 14, 2026 (America/Los_Angeles). The flow uses an isolated Den database and
OpenWork's local OAuth MCP conformance server; it does not use a third-party
service or pre-seeded connection.
The passing journey proves that:
1. URL-only requirements discovery reports OAuth, registration, scope, and
   administrator requirements without creating a connection or registering a
   client.
2. DCR identifies OpenWork as a web client, PKCE authorization completes, and
   the provider returns through the deployment-wide callback.
3. The normalized connection contract reports a connected dynamic client with
   the shared callback and selected scope.
4. The authorized tool is visible in the dashboard and works through
   `search_capabilities` and `execute_capability`.
5. At a 600px-tall viewport, the add-connection dialog remains fully bounded,
   switches to internal scrolling, and keeps the final action reachable. The
   run measured a 550px client height, 745px scroll height, `overflow-y: auto`,
   and dialog bounds from 24px to 576px.
The machine-readable assertions and voice-over coverage are recorded in
[report.md](./report.md). The durable end-to-end flow lives in
`evals/flows/mcp-oauth-conformance.flow.mjs`.
## Frames
![Requirements discovery](./mcp-oauth-conformance-01-requirements-discovered.png)
![Shared callback success](./mcp-oauth-conformance-02-shared-callback-connected.png)
![Normalized shared-callback connection](./mcp-oauth-conformance-03-dashboard-shared-callback.png)
![Authorized tool catalog](./mcp-oauth-conformance-04-authorized-tool-catalog.png)
@@ -0,0 +1,41 @@
# Eval run 2026-07-15T02-36-45-026Z
- Started: 2026-07-15T02:36:45.026Z
- CDP: http://127.0.0.1:39723
- Result: PASSED (1 passed, 0 failed, 0 skipped)
- legacy report: fraimz.html
## ✅ mcp-oauth-conformance — URL discovery, shared callback OAuth, and MCP tool use succeed end to end
Kind: user-facing flow demo
Spec: docs/external-mcp-oauth.md
- ✅ Prepare the isolated Den and OAuth MCP server (1851ms)
  - Assertion: Visible text includes "Add a connection" (passed)
- ✅ Requirements discovery is visible and side-effect free (662ms)
  - Assertion: Visible text includes "OAuth authentication is required." (passed)
  - Assertion: Visible text includes "Registration: dynamic registration." (passed)
  - Assertion: Visible text includes "Administrator action required" (passed)
  - Assertion: Visible text includes "mcp:read" (passed)
  - Frame: mcp-oauth-conformance-01-requirements-discovered.png (passed)
    - Voiceover: An administrator enters only the MCP URL and asks OpenWork to discover requirements. Before anything is saved, the dialog explains that OAuth is required, recommends dynamic registration, shows the available scopes, and calls out the administrator checks that standards metadata cannot prove.
- ✅ DCR and PKCE return through the shared callback (479ms)
  - Assertion: Visible text includes "OAuth conformance mrlgygve" (passed)
  - Assertion: Visible text does not include "Connection failed" (passed)
  - Frame: mcp-oauth-conformance-02-shared-callback-connected.png (passed)
    - Voiceover: After the administrator chooses the least-privilege read scope, OpenWork completes dynamic registration and PKCE authorization in a real browser window. The provider returns to the deployment-wide callback and Den shows the successful connection page.
- ✅ The normalized connection contract reports the shared callback (1841ms)
  - Frame: mcp-oauth-conformance-03-dashboard-shared-callback.png (passed)
    - Voiceover: Back on the Connections screen, the normalized connection card reports Connected, Shared callback, and dynamic registration. The callback shown is deployment-wide and contains no connection identifier.
- ✅ The authorized tool is usable from UI and the agent MCP surface (526ms)
  - Assertion: Visible text includes "Tools available to your agents" (passed)
  - Assertion: Visible text includes "mock_echo" (passed)
  - Frame: mcp-oauth-conformance-04-authorized-tool-catalog.png (passed)
    - Voiceover: The connected server exposes its real Mock Echo tool in the dashboard catalog. The same connection is immediately usable through search_capabilities and execute_capability, proving the OAuth result reaches the agent-facing MCP surface end to end.
- ✅ Clean up the proof connection and mock server (17ms)
- ✅ Voice-over script coverage (0ms)
  - Assertion: Script frame 1 narrated: "An administrator enters only the MCP URL and asks OpenWork to discover requirements. Bef" (passed)
  - Assertion: Script frame 2 narrated: "After the administrator chooses the least-privilege read scope, OpenWork completes dynam" (passed)
  - Assertion: Script frame 3 narrated: "Back on the Connections screen, the normalized connection card reports Connected, Shared" (passed)
  - Assertion: Script frame 4 narrated: "The connected server exposes its real Mock Echo tool in the dashboard catalog. The same " (passed)
Screenshots: mcp-oauth-conformance-01-requirements-discovered.png, mcp-oauth-conformance-02-shared-callback-connected.png, mcp-oauth-conformance-03-dashboard-shared-callback.png, mcp-oauth-conformance-04-authorized-tool-catalog.png