wikiskill/ui_design.md

# WikiSkill — Web UI Design Document

Companion to `design.md`. Describes the interactive web application that lets users
run the WikiSkill evolution loop, watch it live, inspect every produced artifact, and
evaluate the evolved skills — a visual, testable harness for the paper's Algorithm 1.

## 1. Goals

- **Run**: configure a WikiSkill evolution run (dataset, iterations, LLM backend) and
  start/stop it from the browser.
- **Watch**: stream the evolution live (logs, ReAct proposer steps, gating decisions).
- **Inspect**: browse the complete three-layer workspace (`raw/`, `wiki/`, `skills/`)
  produced by the run.
- **Evaluate**: compare final evolved skills vs. no-skill baseline on the test split
  with paired-bootstrap significance.
- **Understand**: an "How it works" view that maps the UI to the paper's architecture
  (Figure 2 / Algorithm 1).

Non-goals: skill editing, wiki pruning, multi-user auth. This is a local study tool.

## 2. User workflows

1. **Explore** — open `http://127.0.0.1:5000`, read How-it-works, browse an existing
   workspace from disk.
2. **Run** — Configure → Start → watch live events → switch to Dashboard/Workspace.
3. **Analyze** — inspect the evolution curve, accepted/rejected proposals, wiki
   patterns, raw traces, then Evaluate on the test split.
4. **Repeat** — tweak parameters (iterations, ReAct turns), start a new run in a fresh
   workspace, compare.

## 3. Information architecture (single page, six tabs)

| Tab | Purpose |
|-----|---------|
| **Run** | Configuration form + live console + status pill; start/cancel |
| **Dashboard** | R_best curve, per-iteration acceptance bars, history table |
| **Evaluation** | Test-split comparison (skilled vs baseline), bootstrap p-value |
| **Workspace** | File tree + viewer for raw/wiki/skills/run_state.json |
| **Knowledge** | Wiki index, pattern pages, evolution log, skill-impact tracker, skills |
| **How it works** | Layer diagram + Algorithm 1 pseudocode + glossary |

## 4. Screen details

### 4.1 Run tab
- **Config form**: LLM backend (`mock`|`openai`), model, base URL, API key (password
  field), temperature, max iterations (K), max ReAct turns, workspace root (empty =
  auto `runs/run_<timestamp>`).
### 4.3 Evaluation tab
- Big-number cards: Skilled accuracy, Baseline accuracy, Δ, bootstrap p-value.
- Horizontal bar comparison + verdict ("Skilled is significantly better", etc.).
- Re-evaluate button (re-runs `evaluate_test()` for the current workspace).

### 4.4 Workspace tab
- Left: collapsible file tree starting at the workspace root.
- Right: content viewer — JSON pretty-printed, markdown shown in raw + rendered form,
  diffs monospace.

### 4.5 Knowledge tab
- Sub-tabs: Skills (cards with SKILL.md + PURPOSE.md), Wiki patterns (list + pages),
  Evolution log, Skill-impact history.

### 4.6 How-it-works tab
- Static annotated pipeline of the three layers and four components, Algorithm 1
  pseudocode, and definitions table (S_k, W_k, P_k, T_train, R_best…).

## 5. Design system

- Dark-on-light ("paper" feel): background `#f7f7f5`, panels white `#fff`, ink `#1a1a1a`.
- Accent: indigo `#3b5bdb` (brand), success `#099268`, danger `#e03131`, warn `#f59f00`.
- Type: system stack; headings 600, mono stack for code/traces.
- Badges/pills for Accept/Reject and status; cards with 1px border, 10px radius.
- Charts rendered as inline SVG (zero JS chart libraries → works offline).

## 6. Backend contract

| Method/Path | Body/Query | Returns |
|---|---|---|
| `GET /` | — | SPA HTML |
| `GET /api/health` | — | `{ok, version}` |
| `POST /api/run` | config JSON | `202 {run_id}`; `400` validation; `409` busy |
| `GET /api/status` | — | `{active, state:{status, events[], result, error}}` |
| `GET /api/stream` | — | Server-Sent Events (`log`/`status`/`react`/`done`/`error`) |
| `POST /api/cancel` | — | `200 {cancelled}` |
| `GET /api/workspace?root=` | query | `{root, tree, files}` (text files) |
| `POST /api/evaluate` | `{workspace_root, llm_backend, …}` | evaluation report JSON |

Validation rules: workspace_root must be a *relative* path (no abs, no `..`); run
rejected with 409 if another run is active; run rejected if root already contains
evolution history (choose a fresh root to avoid clobbering the immutable raw layer).

### SSE protocol
Events: `{type:"log", message}`, `{type:"react", turn, kind, path?, found?}`,
`{type:"status", phase}`, `{type:"done", result}`, `{type:"error", error}`.
Heartbeat comments every 15 s; stream closes after `done`/`error`.

## 7. State model

Server (single flask app):
- `RunManager` — one active run; holds `RunState {run_id, status, events, result, error}`
  and an SSE queue; background daemon thread executes the `EvolutionOrchestrator`
  (MockLLM or OpenAICompatLLM) and wires `log` + proposer `on_step` to the queue;
  `cancel_event` lets the orchestrator stop between iterations.

Client (vanilla JS):
- `state = {config, status, events[], result, evaluation}`; tab rendering is
  pure functions over `state`; `EventSource('/api/stream')` drives updates.

## 8. Error handling & resilience

- JSON payloads only; unknown/invalid fields → 400 with message.
- Proposal failures / malformed LLM JSON are absorbed by the orchestrator (logged).
- Run exceptions → `type:"error"` event + status `error` (traceback stored, exposed
  via status).
- EventSource auto-reconnect on transient drops.
- All file reads limited to 200 kB per file and whitelisted extension set.

## 9. Accessibility & usability

- Semantic HTML (tabs as buttons with aria), keyboard navigation for tree, focus
  states, sufficient contrast, no motion-only information.

## 10. Testing strategy

- Backend: Flask test client (mock backend) — run lifecycle, status, workspace tree,
  evaluation, single-flight 409, validation 400s, cancel.
- Runner: unit test RunManager concurrency (double-start raises).
- E2E smoke: boot `wikiskill serve` on a port; fetch `/`, `/api/health`, run
  `/api/run` with mock, poll status to completion, verify workspace artifacts.
- Regression: full `pytest` suite (core + UI) must stay green.

## 11. Non-functional targets

- Zero CDN dependencies (offline usable).
- Startup < 1 s; mock run completes in seconds; UI remains responsive during runs
  (threaded Flask).
- Flask is the single new runtime dependency, grouped under `[project.optional-dependencies] ui`.
- **Actions**: Start, Cancel (appears while running). Status pill: idle | starting |
  running | evaluating | done | error.
- **Live console**: appended log lines; SSE-driven; auto-scrolls; monospace.
- **ReAct event log**: marks `read_file` calls (path + found/not-found) and final
  proposal, rendered inline in the console with icons.

### 4.2 Dashboard tab
- **R_best curve**: SVG polyline (x = iteration incl. baseline, y = R_best value).
- **Validation bars**: one bar per evaluated proposal, colored green=Accepted /
  red=Rejected; dashed baseline marker.
- **History table**: iteration, action, target skill, validation score, outcome.