From 4bb6ce97c88ab404bd037cd79e147a7d947cc8fe Mon Sep 17 00:00:00 2001 From: "DistributedApps.AI" Date: Mon, 31 Aug 2026 11:10:33 -0400 Subject: [PATCH] Add interactive Web UI (Flask + SSE workbench) with full design doc; fix rollback PURPOSE.md bug; add test-split evaluation with paired bootstrap; fix RunManager reentrant-lock deadlock --- README.md | 38 +- design.md | 11 +- prd.md | 6 + pyproject.toml | 1 + requirements.txt | 1 + smoke_test.py | 87 +++++ src/wikiskill/agents.py | 22 +- src/wikiskill/cli.py | 16 + src/wikiskill/demo.py | 9 +- src/wikiskill/orchestrator.py | 60 ++- src/wikiskill/skills.py | 19 +- src/wikiskill/webapp/__init__.py | 4 + src/wikiskill/webapp/api.py | 183 +++++++++ src/wikiskill/webapp/runner.py | 166 ++++++++ src/wikiskill/webapp/static/app.js | 437 ++++++++++++++++++++++ src/wikiskill/webapp/static/style.css | 141 +++++++ src/wikiskill/webapp/templates/index.html | 202 ++++++++++ src/wikiskill/workspace.py | 10 +- tests/test_agents.py | 16 + tests/test_ui.py | 174 +++++++++ ui_design.md | 147 ++++++++ 21 files changed, 1721 insertions(+), 29 deletions(-) create mode 100644 requirements.txt create mode 100644 smoke_test.py create mode 100644 src/wikiskill/webapp/__init__.py create mode 100644 src/wikiskill/webapp/api.py create mode 100644 src/wikiskill/webapp/runner.py create mode 100644 src/wikiskill/webapp/static/app.js create mode 100644 src/wikiskill/webapp/static/style.css create mode 100644 src/wikiskill/webapp/templates/index.html create mode 100644 tests/test_ui.py create mode 100644 ui_design.md diff --git a/README.md b/README.md index 61ef2fb..09a6f70 100644 --- a/README.md +++ b/README.md @@ -53,11 +53,35 @@ this work: - **Paired bootstrap significance testing** (App. C) in `metrics.py`. - Early stop when `R_best == 1.0`. -## Quickstart +## Web UI + +An interactive workbench for running and inspecting the evolution loop: + +```bash +pip install -e ".[ui]" # installs Flask +wikiskill serve # http://127.0.0.1:5000 +# or: python -m wikiskill.cli serve --port 5000 +``` + +Tabs (see `ui_design.md` for the full design): +- **Run** — configure backend (offline Mock or any OpenAI-compatible endpoint), + iterations, ReAct turns; start/cancel; live SSE console including the Skill + Proposer's ReAct `read_file` steps. +- **Dashboard** — R_best curve, accepted/rejected validation-gating bars, history table. +- **Evaluation** — final skills vs no-skill baseline on the test split with paired + bootstrap p-value (paper §4 + App. C); re-evaluate any workspace. +- **Workspace** — browse the three-layer workspace (`raw/`, `wiki/`, `skills/`) and + view any artifact (JSON pretty-printed, traces, diffs). +- **Knowledge** — evolved skills (SKILL.md + PURPOSE.md), wiki pattern pages, index, + evolution log, skill-impact audit trail. +- **How it works** — the pipeline, Algorithm 1 pseudocode, and glossary mapped to the + paper. + +## Quickstart (CLI) ```bash pip install -e ".[dev]" -pytest # 14 tests incl. end-to-end loop +pytest # 30 tests incl. UI + end-to-end loop python -m wikiskill.demo # offline demo with the deterministic MockLLM ``` @@ -76,20 +100,20 @@ API key is read from `--api-key`, `WIKISKILL_API_KEY`, or `OPENAI_API_KEY`. Any OpenAI-compatible `/chat/completions` endpoint works (including local vLLM, as in the paper). `tasks.jsonl` lines: `{"id", "x", "y", "meta?"}`. -## Layout - +Layout ``` src/wikiskill/ ├── workspace.py # three layers + patch engine + scoped read_file ├── skills.py # skills layer, apply_proposal, skills-only snapshot/rollback ├── agents.py # InferenceAgent, WikiMaintainer, ReAct SkillProposer, stratified_sample -├── orchestrator.py # Algorithm 1 with gating/rollback + audit trail +├── orchestrator.py # Algorithm 1 with gating/rollback + audit trail + test evaluation ├── prompts.py # agent system prompts (adapted from paper Appendix E) ├── llm.py # LLM protocol, OpenAI-compatible client, MockLLM, extract_json ├── metrics.py # accuracy + paired bootstrap (1,000 iterations) ├── datasets.py # dataclasses + synthetic demo dataset -├── demo.py / cli.py -└── tests/… # unit + end-to-end tests +├── webapp/ # Flask UI (runner + api + SPA: see ui_design.md) +├── demo.py / cli.py # `wikiskill demo | evolve | serve` +└── tests/… # unit + end-to-end + UI tests ``` ## Notes & limitations (mirroring the paper) diff --git a/design.md b/design.md index e9e0813..facd36c 100644 --- a/design.md +++ b/design.md @@ -11,17 +11,16 @@ wikiskill/ │ ├── __init__.py │ ├── llm.py # LLM provider abstraction: MockLLM, OpenAICompatLLM │ ├── workspace.py # Three-layer workspace: raw/, wiki/, skills/ + patch engine +│ ├── skills.py # Skills layer: SKILL.md + PURPOSE.md, snapshot/rollback │ ├── prompts.py # System prompts (Inference / Wiki Maintainer / Skill Proposer) │ ├── agents.py # InferenceAgent, WikiMaintainer, SkillProposer (ReAct) -│ ├── orchestrator.py # Algorithm 1: evolution loop, gating, rollback, logging +│ ├── orchestrator.py # Algorithm 1: loop, gating, rollback, test evaluation (§4) │ ├── metrics.py # accuracy metric + paired bootstrap significance test │ ├── datasets.py # Task/Trace dataclasses + demo dataset +│ ├── webapp/ # Web UI (see ui_design.md): runner + Flask api + SPA │ ├── demo.py # End-to-end demo run with mock LLM -│ └── cli.py # `wikiskill evolve` / `wikiskill demo` entry point -└── tests/ - ├── test_workspace.py - ├── test_gating.py - └── test_e2e.py +│ └── cli.py # `wikiskill demo | evolve | serve` entry point +└── tests/ # unit, end-to-end, and UI tests ``` ## 2. Core data model (`datasets.py`) diff --git a/prd.md b/prd.md index 9ca3af8..a2f64ad 100644 --- a/prd.md +++ b/prd.md @@ -91,3 +91,9 @@ experience into the wiki so that subsequent skill updates build on it. workspace with all three layers populated, and accepts/rolls back proposals correctly. 3. Code mirrors paper notation (S_k, W_k, P_k, T_sample, R_best) and cites Algorithm 1 steps in docstrings. +4. **Web UI (ui_design.md)**: `wikiskill serve` boots an interactive workbench that can + start/cancel evolution runs, stream progress live, browse the three-layer workspace, + inspect skills/wiki artifacts, and report test-split evaluation with paired bootstrap. + UI covered by Flask test-client tests plus a live-server smoke test. +5. **Post-run evaluation**: every run reports test-split accuracy for evolved skills vs + no-skill baseline with paired-bootstrap significance (paper §4 + App. C). diff --git a/pyproject.toml b/pyproject.toml index 8d7126d..1d1b064 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -11,6 +11,7 @@ dependencies = [] [project.optional-dependencies] dev = ["pytest>=7"] +ui = ["flask>=3.0"] [project.scripts] wikiskill = "wikiskill.cli:main" diff --git a/requirements.txt b/requirements.txt new file mode 100644 index 0000000..f7d47c9 --- /dev/null +++ b/requirements.txt @@ -0,0 +1 @@ +flask>=3.0 \ No newline at end of file diff --git a/smoke_test.py b/smoke_test.py new file mode 100644 index 0000000..704d410 --- /dev/null +++ b/smoke_test.py @@ -0,0 +1,87 @@ +"""Live smoke test: boots the real wikiskill serve stack and exercises it over HTTP.""" +import json +import subprocess +import sys +import time +import urllib.request + +PORT = 8765 +BASE = f"http://127.0.0.1:{PORT}" +ok = True + + +def check(name, cond, extra=""): + global ok + print(f"[{'PASS' if cond else 'FAIL'}] {name} {extra}") + ok = ok and bool(cond) + + +def get(path, timeout=20): + with urllib.request.urlopen(BASE + path, timeout=timeout) as r: + return r.status, r.read().decode("utf-8") + + +proc = subprocess.Popen( + [sys.executable, "-m", "wikiskill.cli", "serve", "--host", "127.0.0.1", + "--port", str(PORT)], + stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True) + +try: + # wait for server + for _ in range(100): + try: + st, body = get("/api/health", timeout=5) + break + except Exception: + time.sleep(0.3) + else: + raise RuntimeError("server did not start") + + check("health", st == 200 and '"ok":true' in body.replace(" ", "")) + + st, body = get("/") + check("index served", st == 200 and 'WikiSkill' in body and 'run-form' in body) + + req = urllib.request.Request(BASE + "/api/run", + data=json.dumps({"llm_backend": "mock", + "iterations": 2, + "workspace_root": "smoke_run"}).encode(), + headers={"Content-Type": "application/json"}) + with urllib.request.urlopen(req, timeout=10) as r: + st, body = r.status, r.read().decode() + check("run started", st == 202 and '"run_id"' in body) + + done = False + for _ in range(200): + st, body = get("/api/status") + state = json.loads(body)["state"] + if state and state["status"] in ("done", "error"): + done = state["status"] == "done" + break + time.sleep(0.2) + check("run completed (mock, live server)", done) + + st, body = get("/api/workspace?root=smoke_run") + data = json.loads(body) + check("workspace browsable", st == 200 and "wiki/index.md" in data["files"], + f"({sorted(data['files'])[:3]})") + + req = urllib.request.Request(BASE + "/api/evaluate", + data=json.dumps({"llm_backend": "mock", + "workspace_root": "smoke_run"}).encode(), + headers={"Content-Type": "application/json"}) + with urllib.request.urlopen(req, timeout=30) as r: + ev = json.loads(r.read().decode())["evaluation"] + check("evaluation end-to-end", ev["significant"] is True and + ev["skilled_accuracy"] == 1.0, + f"skilled={ev['skilled_accuracy']}") + +finally: + proc.terminate() + try: + proc.wait(timeout=5) + except subprocess.TimeoutExpired: + proc.kill() + +print("SMOKE TEST", "PASSED" if ok else "FAILED") +sys.exit(0 if ok else 1) \ No newline at end of file diff --git a/src/wikiskill/agents.py b/src/wikiskill/agents.py index 90d3517..882299d 100644 --- a/src/wikiskill/agents.py +++ b/src/wikiskill/agents.py @@ -112,9 +112,18 @@ class SkillProposer: """§3.2.3: P_k = M_P(W'_k, S_{k-1}, T_train,k). Multi-turn ReAct agent that reads the wiki index, skill-impact tracker, and traces on demand via read_file.""" - def __init__(self, llm: LLM, ws: Workspace, skills: SkillsLayer, max_turns: int = 20): + def __init__(self, llm: LLM, ws: Workspace, skills: SkillsLayer, max_turns: int = 20, + on_step=None): self.llm, self.ws, self.skills = llm, ws, skills self.max_turns = max_turns + self.on_step = on_step # callable(step: dict) for UI transparency + + def _step(self, **payload): + if self.on_step is not None: + try: + self.on_step({"proposer_step": True, **payload}) + except Exception: # pragma: no cover - never break evolution on UI hooks + pass def propose(self, outcome_summary: str, iteration: int) -> Optional[dict]: index = self.ws.wiki_index() or "(empty wiki)" @@ -126,19 +135,26 @@ class SkillProposer: "(raw/iter_.../...) as needed, then finish with your atomic proposal JSON.") messages = [{"role": "system", "content": SKILL_PROPOSER_SYSTEM}, {"role": "user", "content": user}] - for _ in range(self.max_turns): + for turn in range(1, self.max_turns + 1): text = self.llm.chat(messages, temperature=0.2) parsed = extract_json(text) if parsed and parsed.get("action") in ("create", "patch"): + self._step(turn=turn, kind="proposal", skill=parsed.get("skill"), + action=parsed.get("action")) return parsed fm = re.search(r'"path"\s*:\s*"([^"]+)"', text) if fm: - obs = self.ws.read_file(fm.group(1)) + path = fm.group(1) + obs = self.ws.read_file(path) + self._step(turn=turn, kind="read", path=path, + found=not obs.startswith("ERROR"), + snippet=obs[:200]) messages.append({"role": "assistant", "content": text}) messages.append({"role": "user", "content": f"read_file observation:\n{obs}"}) continue messages.append({"role": "assistant", "content": text}) + self._step(turn=turn, kind="no_tool", snippet=text[:200]) messages.append({"role": "user", "content": "Reply with a read_file tool call or the finish JSON."}) return None diff --git a/src/wikiskill/cli.py b/src/wikiskill/cli.py index 40bb841..8662a47 100644 --- a/src/wikiskill/cli.py +++ b/src/wikiskill/cli.py @@ -25,6 +25,11 @@ def main(argv=None) -> int: e.add_argument("--base-url", default=None) e.add_argument("--api-key", default=None) e.add_argument("--iterations", type=int, default=8) + + s = sub.add_parser("serve", help="run the WikiSkill web UI (Flask)") + s.add_argument("--host", default="127.0.0.1") + s.add_argument("--port", type=int, default=5000) + s.add_argument("--debug", action="store_true") args = p.parse_args(argv) if args.cmd == "demo": @@ -32,6 +37,17 @@ def main(argv=None) -> int: run_demo(args.root, args.iterations) return 0 + if args.cmd == "serve": + try: + from .webapp import create_app + except ImportError as exc: + print("The UI requires Flask. Install it with: " + "pip install -e '.[ui]' (or: pip install flask)", file=sys.stderr) + return 2 + app = create_app() + app.run(host=args.host, port=args.port, debug=args.debug, threaded=True) + return 0 + from .datasets import Dataset, Task, ToolSpec, load_jsonl from .llm import OpenAICompatLLM from .orchestrator import EvolutionOrchestrator diff --git a/src/wikiskill/demo.py b/src/wikiskill/demo.py index 9d8ae37..83b8004 100644 --- a/src/wikiskill/demo.py +++ b/src/wikiskill/demo.py @@ -12,16 +12,21 @@ from .workspace import Workspace def run_demo(root: str = "wikiskill_demo_workspace", K: int = 3) -> dict: ws = Workspace(root) - skills_logger = [] orch = EvolutionOrchestrator( llm=MockLLM(), dataset=make_demo_dataset(), ws=ws, tools=_demo_tools(), K=K) result = orch.run() + eval_res = orch.evaluate_test() + print("\nTest-set evaluation (final skills vs no-skill baseline):") + print(f" skilled accuracy : {eval_res['skilled_accuracy']:.4f}") + print(f" baseline accuracy: {eval_res['baseline_accuracy']:.4f}") + print(f" bootstrap p : {eval_res['bootstrap_p_value']:.4f} " + f"({'significant' if eval_res['significant'] else 'not significant'})") print("\nWorkspace layout:") for dirpath, _, files in os.walk(root): rel = os.path.relpath(dirpath, root) for fn in sorted(files): print(f" {os.path.join(rel, fn)}") - return result + return {**result, "evaluation": eval_res} if __name__ == "__main__": diff --git a/src/wikiskill/orchestrator.py b/src/wikiskill/orchestrator.py index ce3f215..497269d 100644 --- a/src/wikiskill/orchestrator.py +++ b/src/wikiskill/orchestrator.py @@ -31,7 +31,9 @@ class EvolutionOrchestrator: def __init__(self, llm: LLM, dataset: Dataset, ws: Workspace, tools: list, metric: Callable[[List[bool]], float] = None, K: int = 8, - max_react_turns: int = 20, log: Optional[Callable[[str], None]] = None): + max_react_turns: int = 20, log: Optional[Callable[[str], None]] = None, + proposer_on_step=None, + cancel_event=None): from .metrics import accuracy self.llm = llm self.dataset = dataset @@ -41,17 +43,20 @@ class EvolutionOrchestrator: self.skills = SkillsLayer(ws) self.inference = InferenceAgent(llm, tools, ws, self.skills) self.maintainer = WikiMaintainer(llm, ws) - self.proposer = SkillProposer(llm, ws, self.skills, max_turns=max_react_turns) + self.proposer = SkillProposer(llm, ws, self.skills, max_turns=max_react_turns, + on_step=proposer_on_step) self.log = log or (lambda s: print(s, flush=True)) + self.cancel_event = cancel_event # optional threading.Event for mid-run stop self.history: List[dict] = [] self.r_best = -1.0 # ------------------------------------------------------------------ - def _rollout(self, tasks, iteration: int, split: str) -> List[Trajectory]: + def _rollout(self, tasks, iteration, split: str) -> List[Trajectory]: trajs = [self.inference.run(t) for t in tasks] for t in trajs: self.ws.add_raw_trace(iteration, t.task_id, - {**t.to_dict(), "split": split, "iteration": iteration}) + {**t.to_dict(), "split": split, "iteration": + str(iteration)}) return trajs def _score(self, trajs: List[Trajectory]) -> float: @@ -76,6 +81,9 @@ class EvolutionOrchestrator: def run(self) -> dict: self._baseline_validation() for k in range(1, self.K + 1): + if self.cancel_event is not None and self.cancel_event.is_set(): + self.log("Run cancelled by user.") + break if self.r_best >= 1.0: self.log("R_best reached 1.0; early stop.") break @@ -150,3 +158,47 @@ class EvolutionOrchestrator: self.log(f"\nEvolution complete. R_best={self.r_best:.4f}, skills={final_skills}") return result + # ------------------------------------------------------------------ + def evaluate_test(self) -> dict: + """Post-evolution evaluation on the test split (paper §4). + + Compares the final evolved skills against a no-skill baseline on D_test and + reports both accuracies plus a paired bootstrap significance p-value + (paper Appendix C, 1,000 iterations). + """ + from .metrics import paired_bootstrap + # Eval traces are auxiliary (not part of the immutable evolution history); + # clear any previous eval traces so re-evaluation works on the same workspace. + for tag in ("eval_skilled", "eval_baseline"): + d = os.path.join(self.ws.root, "raw", tag) + if os.path.isdir(d): + for fn in os.listdir(d): + os.remove(os.path.join(d, fn)) + os.rmdir(d) + + skilled_trajs = self._rollout(self.dataset.test, "eval_skilled", "test") + skilled = [t.correct for t in skilled_trajs] + skilled_acc = self.metric(skilled) + + snap = self.skills.snapshot() + try: + for name in self.skills.list_skills(): + d = os.path.join(self.ws.root, "skills", name) + for fn in os.listdir(d): + os.remove(os.path.join(d, fn)) + os.rmdir(d) + base_trajs = self._rollout(self.dataset.test, "eval_baseline", "test") + finally: + self.skills.restore(snap) + base = [t.correct for t in base_trajs] + base_acc = self.metric(base) + + p = paired_bootstrap(skilled, base, n=1000, seed=42) + self.log(f"Test evaluation: skilled={skilled_acc:.4f} " + f"baseline={base_acc:.4f} bootstrap_p={p:.4f}") + return {"skilled_accuracy": skilled_acc, + "baseline_accuracy": base_acc, + "bootstrap_p_value": p, + "significant": p < 0.05, + "n_test": len(skilled)} + diff --git a/src/wikiskill/skills.py b/src/wikiskill/skills.py index 481dddf..bd4071d 100644 --- a/src/wikiskill/skills.py +++ b/src/wikiskill/skills.py @@ -76,15 +76,24 @@ class SkillsLayer: raise ValueError(f"unknown proposal action: {action!r}") # ---------- gating rollback (§3.2.4): skills only, wiki untouched ---------- - def snapshot(self) -> Dict[str, str]: - return {name: self.skill_content(name) or "" for name in self.list_skills()} + def snapshot(self) -> Dict[str, dict]: + """Snapshot the active skill state (SKILL.md + PURPOSE.md) so a rejected + proposal can be rolled back exactly to S_{k-1}. The wiki is never included.""" + return { + name: {"SKILL.md": self.skill_content(name) or "", + "PURPOSE.md": self.purpose(name) or ""} + for name in self.list_skills() + } - def restore(self, snap: Dict[str, str]) -> None: + def restore(self, snap: Dict[str, dict]) -> None: for name in self.list_skills(): if name not in snap: d = os.path.join(self.ws.root, "skills", name) for fn in os.listdir(d): os.remove(os.path.join(d, fn)) os.rmdir(d) - for name, content in snap.items(): - self.ws.write_file(os.path.join("skills", name, "SKILL.md"), content) + for name, files in snap.items(): + self.ws.write_file(os.path.join("skills", name, "SKILL.md"), + files["SKILL.md"]) + self.ws.write_file(os.path.join("skills", name, "PURPOSE.md"), + files["PURPOSE.md"]) diff --git a/src/wikiskill/webapp/__init__.py b/src/wikiskill/webapp/__init__.py new file mode 100644 index 0000000..404fdc1 --- /dev/null +++ b/src/wikiskill/webapp/__init__.py @@ -0,0 +1,4 @@ +"""WikiSkill web UI package (ui_design.md).""" +from .api import create_app + +__all__ = ["create_app"] \ No newline at end of file diff --git a/src/wikiskill/webapp/api.py b/src/wikiskill/webapp/api.py new file mode 100644 index 0000000..743bac6 --- /dev/null +++ b/src/wikiskill/webapp/api.py @@ -0,0 +1,183 @@ +"""Flask application factory + REST/SSE endpoints for the WikiSkill UI.""" +from __future__ import annotations + +import json +import os + +from flask import Blueprint, Flask, Response, jsonify, render_template, request + +from .. import __version__ +from .runner import RunManager + +bp = Blueprint("wikiskill_ui", __name__) + +_manager: RunManager = None # module-level singleton + + +def get_manager() -> RunManager: + global _manager + if _manager is None: + _manager = RunManager() + return _manager + + +def create_app(test_config=None) -> Flask: + app = Flask(__name__) + app.config.from_mapping(SECRET_KEY="wikiskill-local-dev") + if test_config: + app.config.update(test_config) + app.register_blueprint(bp) + return app + + +def _safe_root(root: str) -> str: + """Reject absolute paths and path traversal; normalize to a relative name.""" + root = (root or "").strip().replace("\\", "/").strip("/") + if not root: + raise ValueError("workspace root must not be empty") + if os.path.isabs(root): + raise ValueError("workspace root must be a relative path") + if any(part == ".." for part in root.split("/")): + raise ValueError("workspace root must not contain '..'") + return root + + +# --------------------------------------------------------------------------- +@bp.get("/") +def index(): + return render_template("index.html") + + +@bp.get("/api/health") +def health(): + return jsonify({"ok": True, "version": __version__}) + + +@bp.post("/api/run") +def start_run(): + config = request.get_json(silent=True) or {} + backend = config.get("llm_backend", "mock") + if backend not in ("mock", "openai"): + return jsonify({"error": "llm_backend must be 'mock' or 'openai'"}), 400 + if backend == "openai" and not config.get("model"): + return jsonify({"error": "'model' is required when llm_backend=openai"}), 400 + root = config.get("workspace_root") + if root: + try: + _safe_root(root) + except ValueError as e: + return jsonify({"error": str(e)}), 400 + try: + info = get_manager().start(config) + except PermissionError as e: + return jsonify({"error": str(e)}), 409 + return jsonify(info), 202 + + +@bp.get("/api/status") +def status(): + return jsonify(get_manager().status()) + + +@bp.get("/api/stream") +def stream(): + def gen(): + yield from get_manager().stream() + return Response(gen(), mimetype="text/event-stream", + headers={"Cache-Control": "no-cache", + "X-Accel-Buffering": "no"}) + + +@bp.post("/api/cancel") +def cancel(): + return jsonify({"cancelled": get_manager().cancel()}) + + +# --------------------------------------------------------------------------- +_TREE_SKIP = {".pytest_cache", "__pycache__", ".git"} +_TEXT_EXT = {".md", ".json", ".txt", ".py", ".toml", ".diff", ".log"} + + +@bp.get("/api/workspace") +def workspace(): + try: + root = _safe_root(request.args.get("root", "")) + except ValueError as e: + return jsonify({"error": str(e)}), 400 + if not os.path.isdir(root): + return jsonify({"error": f"workspace '{root}' does not exist", + "root": root}), 404 + return jsonify({"root": root, "tree": _build_tree(root), + "files": _read_text_files(root)}) + + +def _build_tree(root: str) -> dict: + def build(path: str) -> dict: + name = os.path.basename(path) or root + rel = os.path.relpath(path, root).replace("\\", "/") + node = {"name": name, "path": rel, "type": "dir", "children": []} + for entry in sorted(os.listdir(path)): + if entry in _TREE_SKIP: + continue + full = os.path.join(path, entry) + if os.path.isdir(full): + node["children"].append(build(full)) + else: + node["children"].append({ + "name": entry, + "path": os.path.relpath(full, root).replace("\\", "/"), + "type": "file"}) + return node + return build(root) + + +def _read_text_files(root: str) -> dict: + """Collect text-file contents (bounded) for the UI viewer.""" + out = {} + for dirpath, dirnames, filenames in os.walk(root): + dirnames[:] = [d for d in dirnames if d not in _TREE_SKIP] + for fn in filenames: + if os.path.splitext(fn)[1] not in _TEXT_EXT: + continue + full = os.path.join(dirpath, fn) + rel = os.path.relpath(full, root).replace("\\", "/") + try: + with open(full, "r", encoding="utf-8") as f: + out[rel] = f.read(200_000) + except (OSError, UnicodeDecodeError): + out[rel] = "(unreadable/binary)" + return out + + +# --------------------------------------------------------------------------- +@bp.post("/api/evaluate") +def evaluate(): + """Re-evaluate final skills vs baseline on the test split (paper §4).""" + config = request.get_json(silent=True) or {} + backend = config.get("llm_backend", "mock") + if backend not in ("mock", "openai"): + return jsonify({"error": "llm_backend must be 'mock' or 'openai'"}), 400 + try: + root = _safe_root(config.get("workspace_root", "")) + except ValueError as e: + return jsonify({"error": str(e)}), 400 + if not os.path.isdir(root): + return jsonify({"error": f"workspace '{root}' does not exist"}), 404 + + from ..datasets import _demo_tools, make_demo_dataset + from ..llm import MockLLM, OpenAICompatLLM + from ..orchestrator import EvolutionOrchestrator + from ..workspace import Workspace + + if backend == "openai": + llm = OpenAICompatLLM(model=config.get("model", ""), + base_url=config.get("base_url") or None, + api_key=config.get("api_key") or None) + else: + llm = MockLLM() + orch = EvolutionOrchestrator(llm=llm, dataset=make_demo_dataset(), + ws=Workspace(root), tools=_demo_tools(), + K=int(config.get("iterations", 8)), + log=lambda msg: None) + res = orch.evaluate_test() + return jsonify({"evaluation": res, "workspace_root": root}) \ No newline at end of file diff --git a/src/wikiskill/webapp/runner.py b/src/wikiskill/webapp/runner.py new file mode 100644 index 0000000..86f7125 --- /dev/null +++ b/src/wikiskill/webapp/runner.py @@ -0,0 +1,166 @@ +"""RunManager: executes the WikiSkill evolution loop in a background thread and +streams progress to the browser via Server-Sent Events (ui_design.md §6–§7).""" +from __future__ import annotations + +import datetime as _dt +import json +import os +import queue +import threading +import traceback as _tb +import uuid +from typing import Optional + +from ..datasets import _demo_tools, make_demo_dataset +from ..llm import MockLLM, OpenAICompatLLM +from ..orchestrator import EvolutionOrchestrator +from ..workspace import Workspace + + +def _is_fresh_root(path: str) -> bool: + """True if the workspace root has no prior evolution history (raw/iter_*).""" + raw = os.path.join(path, "raw") + if not os.path.isdir(raw): + return True + return not any(d.startswith("iter_") for d in os.listdir(raw)) + + +class RunManager: + """Single-flight run manager. `start` spawns a daemon thread; `stream` yields SSE.""" + + def __init__(self): + self._lock = threading.Lock() + self._state: Optional[dict] = None + self._events: "queue.Queue[dict]" = queue.Queue() + self._cancel_event = threading.Event() + + # ---- public API ---------------------------------------------------- + @property + def active(self) -> bool: + with self._lock: + return bool(self._state and self._state.get("status") + in ("starting", "running", "evaluating")) + + def status(self) -> dict: + # Read without holding the lock while computing `active` — the property + # also acquires the (non-reentrant) lock, which would deadlock. + state = self._state + return {"active": bool(state and state.get("status") + in ("starting", "running", "evaluating")), + "state": state} + + def start(self, config: dict) -> dict: + with self._lock: + if self._state and self._state.get("status") in ( + "starting", "running", "evaluating"): + raise PermissionError("another evolution run is already active") + run_id = uuid.uuid4().hex[:12] + self._state = {"run_id": run_id, "config": dict(config), "status": + "starting", "events": [], "result": None, "error": None} + self._cancel_event = threading.Event() + threading.Thread(target=self._execute, args=(run_id, dict(config)), + daemon=True).start() + return {"run_id": run_id, "active": True} + + def cancel(self) -> bool: + self._cancel_event.set() + return True + + def _reset_for_tests(self) -> None: + """Test-only: drop state and queue so a fresh run can start.""" + with self._lock: + self._state = None + self._cancel_event = threading.Event() + while True: + try: + self._events.get_nowait() + except queue.Empty: + break + + # ---- streaming ----------------------------------------------------- + def _emit(self, **payload) -> None: + self._events.put(dict(payload)) + with self._lock: + if self._state is not None: + self._state["events"].append(dict(payload)) + self._state["events"] = self._state["events"][-500:] + + def stream(self): + """SSE generator: emits events; closes after done/error.""" + while True: + try: + payload = self._events.get(timeout=15) + except queue.Empty: + yield ":\n\n" # heartbeat + continue + data = json.dumps(payload) + yield f"data: {data}\n\n" + if payload.get("type") in ("done", "error"): + break + + # ---- execution ----------------------------------------------------- + def _build_llm(self, config: dict): + backend = config.get("llm_backend", "mock") + if backend == "openai": + return OpenAICompatLLM( + model=config.get("model", ""), + base_url=config.get("base_url") or None, + api_key=config.get("api_key") or None, + temperature=float(config.get("temperature", 0.7))) + return MockLLM() + + def _root_for(self, config: dict) -> str: + root = config.get("workspace_root") or "" + if root: + return str(root).strip().strip("/").replace("\\", "/").strip("/") + stamp = _dt.datetime.now().strftime("%Y%m%d_%H%M%S") + return f"runs/run_{stamp}" + + def _execute(self, run_id: str, config: dict) -> None: + try: + self._emit(type="status", phase="starting", + message="Preparing run...") + root = self._root_for(config) + if not _is_fresh_root(root): + raise ValueError( + f"workspace root '{root}' already contains evolution history; " + "choose a fresh root (the raw layer is immutable)") + llm = self._build_llm(config) + ws = Workspace(root) + orch = EvolutionOrchestrator( + llm=llm, dataset=make_demo_dataset(), ws=ws, tools=_demo_tools(), + K=int(config.get("iterations", 8)), + max_react_turns=int(config.get("max_react_turns", 20)), + cancel_event=self._cancel_event, + log=lambda msg: self._emit(type="log", message=msg), + proposer_on_step=lambda ev: self._emit(type="react", **ev)) + self._emit(type="status", phase="running", message=f"root={root}") + result = orch.run() + self._emit(type="status", phase="evaluating", + message="Evaluating evolved skills on the test split...") + evaluation = orch.evaluate_test() + with self._lock: + if self._state is not None: + self._state["status"] = "done" + self._state["result"] = {"evolution": result, + "evaluation": evaluation, + "workspace_root": root} + self._emit(type="done", result=self._state["result"] if self._state else + {"evolution": result, "evaluation": evaluation, + "workspace_root": root}) + except Exception as exc: # noqa: BLE001 - surface any failure to the UI + err = f"{type(exc).__name__}: {exc}" + print(err, flush=True) + print(_tb.format_exc(), flush=True) + with self._lock: + if self._state is not None: + self._state["status"] = "error" + self._state["error"] = err + "\n" + _tb.format_exc() + self._emit(type="error", error=str(exc)) + self._state = None + self._cancel_event = threading.Event() + while True: + try: + self._events.get_nowait() + except queue.Empty: + break \ No newline at end of file diff --git a/src/wikiskill/webapp/static/app.js b/src/wikiskill/webapp/static/app.js new file mode 100644 index 0000000..28e5d94 --- /dev/null +++ b/src/wikiskill/webapp/static/app.js @@ -0,0 +1,437 @@ +"use strict"; +/* WikiSkill UI client — vanilla JS, no dependencies (ui_design.md §6–§7). */ + +const $ = (id) => document.getElementById(id); +const S = { + ws: null, // workspace data {root, tree, files} + lastResult: null, // {evolution, evaluation, workspace_root} + activeRoot: null, +}; + +/* ---------- utils ---------- */ +function esc(s) { + return String(s ?? "").replace(/[&<>"']/g, (c) => ({ + "&": "&", "<": "<", ">": ">", '"': """, "'": "'", + }[c])); +} +async function api(path, opts = {}) { + const res = await fetch(path, { + headers: { "Content-Type": "application/json" }, + method: opts.method || (opts.body ? "POST" : "GET"), + body: opts.body ? JSON.stringify(opts.body) : undefined, + }); + let data = null; + try { data = await res.json(); } catch (_) { /* empty */ } + if (!res.ok) { + const msg = (data && data.error) || `HTTP ${res.status}`; + throw new Error(msg); + } + return data; +} +function fmtPct(v) { return `${(v * 100).toFixed(1)}%`; } + +/* ---------- tabs ---------- */ +function switchTab(name) { + document.querySelectorAll(".tab").forEach((b) => + b.classList.toggle("is-active", b.dataset.tab === name)); + document.querySelectorAll(".tab-panel").forEach((p) => + p.classList.toggle("is-active", p.id === `tab-${name}`)); +} +document.querySelectorAll(".tab").forEach((b) => + b.addEventListener("click", () => switchTab(b.dataset.tab))); + +/* ---------- status pill ---------- */ +function setStatus(status, label) { + const pill = $("status-pill"); + pill.textContent = label || status; + const cls = { idle: "pill-idle", starting: "pill-running", running: "pill-running", + evaluating: "pill-evaluating", done: "pill-done", error: "pill-error" }; + pill.className = `pill ${cls[status] || "pill-idle"}`; + $("btn-cancel").classList.toggle("hidden", + !["starting", "running", "evaluating"].includes(status)); + $("btn-start").disabled = ["starting", "running", "evaluating"].includes(status); +} + +/* ---------- console ---------- */ +function logLine(text, cls) { + const con = $("console"); + const div = document.createElement("span"); + div.className = `ln ${cls || "ln-tail"}`; + div.textContent = text; + con.appendChild(div); + con.scrollTop = con.scrollHeight; +} +function renderEvent(ev) { + if (ev.type === "log") logLine(ev.message, "ln-tail"); + else if (ev.type === "status") { + logLine(`—— ${ev.message || ev.phase} ——`, "ln-cmd"); + if (ev.phase) setStatus(ev.phase === "evaluating" ? "evaluating" : + ["starting", "running"].includes(ev.phase) ? "running" : ev.phase); + } else if (ev.type === "react") { + if (ev.kind === "read") { + logLine(` [proposer] read ${ev.path} ${ev.found ? "" : "(MISSING)"}`, "ln-react"); + } else if (ev.kind === "proposal") { + logLine(` [proposer] → proposal ${ev.action} '${ev.skill}'`, "ln-react"); + } else { + logLine(` [proposer] ${esc(ev.kind)}`, "ln-react"); + } + } else if (ev.type === "done") { + S.lastResult = ev.result; + setStatus("done", "done"); + if (ev.result) { + showDashboard(ev.result.evolution); + showEval(ev.result.evaluation); + S.activeRoot = ev.result.workspace_root; + $("log-root-label").textContent = `→ ${S.activeRoot}`; + } + logLine("✔ run finished", "ln-accept"); + } else if (ev.type === "error") { + logLine(`✖ error: ${ev.error}`, "ln-reject"); + setStatus("error", "error"); + } +} + +/* ---------- run form ---------- */ +$("cfg-backend").addEventListener("change", () => { + $("openai-fields").classList.toggle("hidden", $("cfg-backend").value !== "openai"); +}); +$("run-form").addEventListener("submit", async (e) => { +/* ---------- SSE ---------- */ +let esSource = null; +function connectStream() { + if (esSource) esSource.close(); + setStatus("running", "running…"); + esSource = new EventSource("/api/stream"); + esSource.onmessage = (m) => { + let ev; + try { ev = JSON.parse(m.data); } catch (_) { return; } + renderEvent(ev); + if (ev.type === "done" || ev.type === "error") esSource.close(); + }; + esSource.onerror = () => { if (esSource) esSource.close(); }; +} + +/* ---------- dashboard ---------- */ +function showDashboard(evo) { + $("dash-emptystate").classList.add("hidden"); + $("dash-charts").classList.remove("hidden"); + const history = evo.history || []; + drawRbest(history); + drawGate(history); + const tbody = document.querySelector("#dash-table tbody"); + tbody.innerHTML = ""; + if (!history.length) { + tbody.innerHTML = 'No proposals evaluated.'; + return; + } + for (const h of history) { + const tr = document.createElement("tr"); + const outcome = h.accepted ? "accepted" : (h.proposal ? "rejected" : "none"); + tr.innerHTML = `${h.iteration}${esc(h.proposal || "—")} + ${esc(h.action || "—")} + ${h.val_score == null ? "—" : h.val_score.toFixed(4)} + ${outcome}`; + tbody.appendChild(tr); + } +} +function drawRbest(history) { + const svg = $("chart-rbest"); + // Reconstruct R_best trajectory: starts at baseline (0 by convention here, or the + // first accepted score), then holds between accepted proposals. + const accepted = history.filter((h) => h.accepted && h.val_score != null); + const start = accepted.length ? firstVal(history) : 0; + if (!accepted.length) { + svg.innerHTML = `no accepted proposals (R_best = baseline)`; + return; + } + const pts = [{ lab: "base", v: start }]; + let cur = start; + for (const h of history) { + if (h.accepted) cur = h.val_score; + pts.push({ lab: `k${h.iteration}`, v: cur }); + } + renderLine(svg, pts); +} +function firstVal(history) { + for (const h of history) if (h.accepted) return h.val_score; + return 0; +} +function renderLine(svg, pts) { + const W = 400, H = 180, pad = { l: 34, r: 10, t: 12, b: 24 }; + const minV = Math.min(...pts.map((p) => p.v), 0); + const maxV = Math.max(...pts.map((p) => p.v), 1); + const span = Math.max(0.2, maxV - minV); + const x = (i) => pad.l + (i / Math.max(1, pts.length - 1)) * (W - pad.l - pad.r); + const y = (v) => H - pad.b - ((v - minV) / span) * (H - pad.t - pad.b); + let html = ""; + for (let g = 0; g <= 4; g++) { + const gv = minV + (span * g) / 4; + const gy = y(gv); + html += ``; + html += `${gv.toFixed(2)}`; + } + const px = pts.map((p, i) => `${x(i).toFixed(1)},${y(p.v).toFixed(1)}`).join(" "); + html += ``; + pts.forEach((p, i) => { + html += ``; + html += `${p.lab}: ${p.v.toFixed(3)}`; + }); + html += `baseline`; + svg.innerHTML = html; +} +function drawGate(history) { + const svg = $("chart-gate"); + const W = 400, H = 180, pad = { l: 34, r: 10, t: 12, b: 24 }; + const vals = history.filter((h) => h.val_score != null); + const maxV = Math.max(1, ...vals.map((v) => v.val_score)); + let html = ""; + for (let g = 0; g <= 4; g++) { + const gy = H - pad.b - (g / 4) * (H - pad.t - pad.b); + html += ``; + } + if (!vals.length) { + html += `no proposals evaluated yet`; + svg.innerHTML = html; return; + } + const bw = Math.min(46, (W - pad.l - pad.r) / vals.length - 8); + vals.forEach((h, i) => { + const bh = (h.val_score / maxV) * (H - pad.t - pad.b); + const bx = pad.l + i * ((W - pad.l - pad.r) / vals.length) + 4; + const by = H - pad.b - bh; + const color = h.accepted ? "#099268" : "#e03131"; + html += ``; + html += `iter ${h.iteration}: ${h.val_score.toFixed(3)} ${h.accepted ? "accepted" : "rejected"}`; + html += `k${h.iteration}`; + }); + svg.innerHTML = html; +} + e.preventDefault(); + $("form-error").textContent = ""; + const cfg = { + llm_backend: $("cfg-backend").value, + iterations: parseInt($("cfg-iterations").value, 10), + max_react_turns: parseInt($("cfg-react").value, 10), + workspace_root: $("cfg-root").value.trim() || undefined, + }; + if (cfg.llm_backend === "openai") { + cfg.model = $("cfg-model").value.trim(); + cfg.base_url = $("cfg-base-url").value.trim() || undefined; + cfg.api_key = $("cfg-api-key").value.trim() || undefined; + cfg.temperature = parseFloat($("cfg-temperature").value); + } + try { + await api("/api/run", { body: cfg }); + $("console").textContent = ""; + connectStream(); + } catch (err) { +/* ---------- evaluation ---------- */ +function showEval(ev) { + if (!ev) return; + $("eval-emptystate").classList.add("hidden"); + $("eval-cards").classList.remove("hidden"); + $("eval-bars").classList.remove("hidden"); + $("eval-verdict").classList.remove("hidden"); + const sk = ev.skilled_accuracy, base = ev.baseline_accuracy; + const d = sk - base; + $("eval-cards").innerHTML = ` +
${fmtPct(sk)}
with evolved skills
+
${fmtPct(base)}
no skills (baseline)
+
${d >= 0 ? "+" : ""}${fmtPct(d)}
Δ
+
${ev.bootstrap_p_value.toFixed(4)}
bootstrap p (1000)
`; + const W = 420, H = 90; + const maxV = Math.max(sk, base, 0.01); + const bar = (label, v, color) => ` + ${label} + + ${fmtPct(v)} + ${fmtPct(v)}`; + $("eval-bars").innerHTML = ` + ${bar("evolved skills", sk, "#099268")} + ${bar("no-skill baseline", base, "#adb5bd")}`; + const verdict = $("eval-verdict"); + if (ev.significant && d > 0) { + verdict.className = "verdict ok"; + verdict.textContent = `✔ Evolved skills are significantly better than the no-skill baseline (p = ${ev.bootstrap_p_value.toFixed(4)} < 0.05).`; + } else if (d < 0) { + verdict.className = "verdict bad"; + verdict.textContent = `✖ Skills underperform the baseline on the test split.`; + } else { + verdict.className = "verdict neutral"; + verdict.textContent = `∽ No significant difference from the baseline (p = ${ev.bootstrap_p_value.toFixed(4)}).`; + } +} +$("btn-evaluate").addEventListener("click", async () => { + $("eval-error").textContent = ""; + const root = $("eval-root").value.trim() || S.activeRoot; + if (!root) { + $("eval-error").textContent = "No workspace yet — run an evolution first, or enter a root."; + return; + } + const btn = $("btn-evaluate"); + btn.disabled = true; btn.textContent = "Evaluating…"; + try { + $("eval-load-form").classList.remove("hidden"); + const res = await api("/api/evaluate", { body: { llm_backend: "mock", + workspace_root: root, iterations: 3 } }); + showEval(res.evaluation); + S.activeRoot = root; + } catch (err) { + $("eval-error").textContent = err.message; + } finally { + btn.disabled = false; btn.textContent = "Re-evaluate current workspace"; + } +}); +$("btn-eval-load").addEventListener("click", async () => { + const root = $("eval-root").value.trim(); + if (!root) { $("eval-error").textContent = "Enter a workspace root."; return; } + try { + const res = await api("/api/evaluate", { body: { llm_backend: "mock", + workspace_root: root, iterations: 3 } }); + showEval(res.evaluation); + } catch (err) { $("eval-error").textContent = err.message; } +}); +/* ---------- workspace explorer ---------- */ +async function loadWorkspace(root) { + $("ws-error").textContent = ""; + try { + const data = await api(`/api/workspace?root=${encodeURIComponent(root)}`); + S.ws = data; + S.activeRoot = root; + $("ws-split").classList.remove("hidden"); + renderTree(data.tree); + showFile("run_state.json"); + renderKnowledge(data.files); + $("eval-root").value = root; + $("ws-root").value = root; + } catch (err) { + $("ws-error").textContent = err.message; + } +} +$("btn-ws-load").addEventListener("click", () => + loadWorkspace($("ws-root").value.trim())); +$("btn-ws-last").addEventListener("click", () => { + if (S.activeRoot) { $("ws-root").value = S.activeRoot; loadWorkspace(S.activeRoot); } + else $("ws-error").textContent = "No last run yet."; +}); +function renderTree(node) { + const ul = document.createElement("ul"); + node.children.forEach((ch) => ul.appendChild(renderNode(ch))); + const tree = $("ws-tree"); + tree.innerHTML = ""; + tree.appendChild(ul); +} +function renderNode(node) { + const li = document.createElement("li"); + if (node.type === "dir") { + const label = document.createElement("span"); + label.className = "node-dir"; + label.innerHTML = `▾📁 ${esc(node.name)}`; + li.appendChild(label); + const childUl = document.createElement("ul"); + node.children.forEach((ch) => childUl.appendChild(renderNode(ch))); + li.appendChild(childUl); + label.addEventListener("click", () => { + childUl.classList.toggle("hidden"); + label.querySelector(".toggle").textContent = + childUl.classList.contains("hidden") ? "▸" : "▾"; + }); + } else { + const f = document.createElement("span"); + f.className = "node-file"; + const icon = node.name.endsWith(".json") ? "🧾" : + (node.name.endsWith(".md") ? "📝" : "📄"); + f.innerHTML = `${icon} ${esc(node.name)}`; + f.addEventListener("click", () => showFile(node.path)); + li.appendChild(f); + } + return li; +} +function showFile(path) { + if (!S.ws || !S.ws.files[path]) { $("ws-content").textContent = "(not available)"; return; } + $("ws-file-path").textContent = path; + let content = S.ws.files[path]; + const mode = $("ws-view-mode").value; + if (path.endsWith(".json") && (mode === "auto" || mode === "json")) { + try { content = JSON.stringify(JSON.parse(content), null, 2); } + catch (_) { /* keep raw */ } + } + $("ws-content").textContent = content; +} +$("ws-view-mode").addEventListener("change", () => { + if (S.ws) showFile($("ws-file-path").textContent); +}); + +/* ---------- knowledge ---------- */ +let ktab = "skills"; +document.querySelectorAll(".ktab").forEach((b) => + b.addEventListener("click", () => { + ktab = b.dataset.ktab; + document.querySelectorAll(".ktab").forEach((x) => + x.classList.toggle("is-active", x === b)); + if (S.ws) renderKnowledge(S.ws.files); + })); +function renderKnowledge(files) { + if (!files) return; + $("know-emptystate").classList.add("hidden"); + const out = $("know-content"); + if (ktab === "skills") { + const names = Object.keys(files).filter((p) => p.endsWith("/SKILL.md")); + if (!names.length) { out.innerHTML = '

No evolved skills in this workspace.

'; return; } + out.innerHTML = ""; + names.forEach((skillPath) => { + const purposePath = skillPath.replace("/SKILL.md", "/PURPOSE.md"); + const card = document.createElement("div"); + card.className = "skill-card"; + const name = skillPath.split("/")[1]; + card.innerHTML = `

📦 ${esc(name)}

SKILL.md
+ 
${esc(files[skillPath])}
PURPOSE.md
+ 
${esc(files[purposePath] || "(none)")}
`; + out.appendChild(card); + }); + } else { + const map = { + patterns: (p) => p.startsWith("wiki/patterns/"), + index: (p) => p === "wiki/index.md", + logs: (p) => p === "wiki/logs.md", + impact: (p) => p === "wiki/skill-impact.md", + }; + const keep = map[ktab]; + const paths = Object.keys(files).filter(keep).sort(); + if (!paths.length) { out.innerHTML = '

Nothing here yet.

'; return; } + out.innerHTML = ""; + paths.forEach((p) => { + const box = document.createElement("div"); + box.className = "kdoc"; + box.innerHTML = `${esc(p)}\n\n${esc(files[p])}`; + out.appendChild(box); + }); + } +} + +/* ---------- restore state on load ---------- */ +(async function init() { + try { + const st = await api("/api/status"); + const state = st.state; + if (state) { + (state.events || []).forEach(renderEvent); + if (state.result) { + S.lastResult = state.result; + showDashboard(state.result.evolution); + showEval(state.result.evaluation); + S.activeRoot = state.result.workspace_root; + $("log-root-label").textContent = `→ ${S.activeRoot}`; + $("eval-root").value = S.activeRoot || ""; + } + if (state.status === "done") setStatus("done", "done"); + if (st.active) connectStream(); + } + } catch (_) { /* server unreachable; keep idle state */ } +})(); + $("form-error").textContent = err.message; + } +}); +$("btn-cancel").addEventListener("click", async () => { + try { await api("/api/cancel", { body: {} }); logLine("—— cancel requested ——", "ln-cmd"); } + catch (_) { /* ignore */ } +}); \ No newline at end of file diff --git a/src/wikiskill/webapp/static/style.css b/src/wikiskill/webapp/static/style.css new file mode 100644 index 0000000..9d2c3ed --- /dev/null +++ b/src/wikiskill/webapp/static/style.css @@ -0,0 +1,141 @@ +:root { + --bg: #f7f7f5; --panel: #ffffff; --ink: #1a1a1a; --muted: #6b6b6b; + --accent: #3b5bdb; --accent-soft: #e7ebfc; --ok: #099268; --ok-soft: #e6f7f1; + --bad: #e03131; --bad-soft: #fdeaea; --warn: #f59f00; --border: #e3e1dc; + --mono: ui-monospace, SFMono-Regular, Menlo, Consolas, monospace; + --sans: system-ui, -apple-system, "Segoe UI", Roboto, sans-serif; +} +* { box-sizing: border-box; } +html, body { margin: 0; } +body { background: var(--bg); color: var(--ink); font-family: var(--sans); font-size: 14px; line-height: 1.5; } +.hidden { display: none !important; } +.muted { color: var(--muted); font-weight: 400; } +.small { font-size: 12px; } +a { color: var(--accent); } + +/* ---------- header / tabs ---------- */ +.topbar { display: flex; justify-content: space-between; align-items: center; + padding: 14px 22px; background: #fff; border-bottom: 1px solid var(--border); } +.brand-kicker { font-size: 11px; letter-spacing: .08em; text-transform: uppercase; color: var(--muted); display: block; } +.brand h1 { margin: 0; font-size: 20px; } +.brand .sub { color: var(--muted); font-weight: 400; font-size: 14px; } +.topbar-right { display: flex; align-items: center; gap: 12px; } +.small-link { font-size: 12px; } + +.pill { border-radius: 999px; padding: 3px 12px; font-size: 12px; font-weight: 600; } +.pill-idle { background: #eee; color: #555; } +.pill-running { background: var(--accent-soft); color: var(--accent); } +.pill-evaluating { background: #f3ecfb; color: #7b2cbf; } +.pill-done { background: var(--ok-soft); color: var(--ok); } +.pill-error { background: var(--bad-soft); color: var(--bad); } + +.tabs { display: flex; gap: 4px; padding: 10px 22px 0; background: #fff; + border-bottom: 1px solid var(--border); } +.tab { border: 0; background: transparent; padding: 8px 14px; font-size: 13px; + font-weight: 600; color: #555; cursor: pointer; border-radius: 8px 8px 0 0; } +.tab:hover { background: #f2f2f0; } +.tab.is-active { color: var(--accent); background: var(--accent-soft); } + +/* ---------- layout ---------- */ +main { max-width: 1180px; margin: 18px auto; padding: 0 18px; } +.tab-panel { display: none; } +.tab-panel.is-active { display: block; } +.grid-2 { display: grid; grid-template-columns: 1fr 1fr; gap: 16px; align-items: start; } +@media (max-width: 860px) { .grid-2 { grid-template-columns: 1fr; } } +.grid-2-inner { display: grid; grid-template-columns: 1fr 1fr; gap: 10px; } + +.card { background: var(--panel); border: 1px solid var(--border); border-radius: 12px; + padding: 18px 20px; margin-bottom: 16px; } +.card h2 { margin: 0 0 10px; font-size: 16px; } +.card h3 { font-size: 14px; margin: 14px 0 6px; } + +/* ---------- forms ---------- */ +form label { display: block; font-weight: 600; font-size: 13px; margin: 10px 0 4px; } +form input, form select { width: 100%; padding: 7px 9px; font-size: 13px; + border: 1px solid var(--border); border-radius: 8px; background: #fff; } +.actions { display: flex; gap: 10px; margin-top: 14px; } +.btn { background: #fff; color: var(--ink); border: 1px solid var(--border); + border-radius: 8px; padding: 7px 14px; font-size: 13px; font-weight: 600; cursor: pointer; } +.btn:hover { border-color: var(--accent); color: var(--accent); } +.btn-primary { background: var(--accent); color: #fff; border-color: var(--accent); } +.btn-primary:hover { opacity: .9; color: #fff; } +.btn-danger { background: var(--bad); color: #fff; border-color: var(--bad); } +.btn-danger:hover { opacity: .9; color: #fff; } +.form-error { color: var(--bad); font-size: 12px; min-height: 16px; margin: 8px 0 0; } +.load-row { display: flex; gap: 10px; align-items: end; margin-bottom: 12px; } +.load-row label { flex: 1; display: block; font-size: 12px; font-weight: 600; } +.load-row input { width: 100%; padding: 6px 8px; border: 1px solid var(--border); border-radius: 8px; } +/* ---------- console ---------- */ +.console { background: #15181d; color: #d7e0f0; font-family: var(--mono); + font-size: 12px; padding: 12px; border-radius: 10px; height: 460px; overflow: auto; white-space: pre-wrap; } +.console .ln { display: block; } +.console .ln-tail { color: #8a93a6; } +.console .ln-cmd { color: #7cc4ff; } +.console .ln-react { color: #f2c55c; } +.console .ln-accept { color: #5ee6a7; } +.console .ln-reject { color: #ff8d8d; } + +/* ---------- charts ---------- */ +.chart { width: 100%; height: auto; background: #fff; border: 1px solid var(--border); border-radius: 10px; } +.chart text { font-size: 10px; fill: #666; } + +/* ---------- tables ---------- */ +.table-wrap { overflow: auto; } +table { border-collapse: collapse; width: 100%; font-size: 13px; } +th, td { text-align: left; padding: 6px 10px; border-bottom: 1px solid var(--border); } +th { font-size: 12px; color: var(--muted); } +.badge { border-radius: 999px; padding: 2px 10px; font-size: 11px; font-weight: 700; } +.badge.accepted { background: var(--ok-soft); color: var(--ok); } +.badge.rejected { background: var(--bad-soft); color: var(--bad); } +.badge.none { background: #eee; color: #555; } + +/* ---------- KPI / verdict ---------- */ +.kpi-row { display: flex; gap: 14px; margin: 12px 0; flex-wrap: wrap; } +.kpi { flex: 1; min-width: 130px; background: #fff; border: 1px solid var(--border); + border-radius: 12px; padding: 14px 16px; text-align: center; } +.kpi .v { font-size: 26px; font-weight: 700; } +.kpi .l { font-size: 11px; color: var(--muted); text-transform: uppercase; letter-spacing: .04em; } +.verdict { font-weight: 600; padding: 10px 14px; border-radius: 10px; display: inline-block; } +.verdict.ok { background: var(--ok-soft); color: var(--ok); } +.verdict.bad { background: var(--bad-soft); color: var(--bad); } +.verdict.neutral { background: #f1f1ee; color: #555; } + +/* ---------- workspace tree / viewer ---------- */ +.ws-split { display: grid; grid-template-columns: 300px 1fr; gap: 14px; align-items: start; } +@media (max-width: 860px) { .ws-split { grid-template-columns: 1fr; } } +.tree { border: 1px solid var(--border); border-radius: 10px; padding: 8px; max-height: 560px; overflow: auto; font-size: 13px; } +.tree ul { list-style: none; margin: 0; padding-left: 16px; } +.tree .node-file { cursor: pointer; padding: 1px 6px; border-radius: 6px; white-space: nowrap; } +.tree .node-file:hover { background: var(--accent-soft); } +.tree .node-dir { font-weight: 600; } +.tree .toggle { cursor: pointer; color: var(--muted); margin-right: 2px; } +.viewer { border: 1px solid var(--border); border-radius: 10px; overflow: hidden; } +.viewer-head { display: flex; justify-content: space-between; gap: 8px; padding: 8px 10px; +/* ---------- knowledge ---------- */ +.ktabs { display: flex; gap: 4px; margin-bottom: 12px; flex-wrap: wrap; } +.ktab { border: 1px solid var(--border); background: #fff; border-radius: 8px; + padding: 5px 12px; font-size: 12px; font-weight: 600; color: #555; cursor: pointer; } +.ktab.is-active { background: var(--accent); border-color: var(--accent); color: #fff; } +.kdoc { background: #fdfdfb; border: 1px solid var(--border); border-radius: 10px; + padding: 14px 16px; font-family: var(--mono); font-size: 12px; white-space: pre-wrap; max-height: 480px; overflow: auto; } +.skill-card { border: 1px solid var(--border); border-radius: 10px; padding: 12px 14px; margin-bottom: 12px; } +.skill-card h4 { margin: 0 0 6px; color: var(--accent); } +.skill-card h5 { margin: 8px 0 4px; font-size: 12px; color: var(--muted); text-transform: uppercase; letter-spacing: .04em; } +.skill-card pre { margin: 0; font-family: var(--mono); font-size: 12px; white-space: pre-wrap; + background: #f8f8f6; border-radius: 8px; padding: 8px 10px; max-height: 220px; overflow: auto; } + +/* ---------- how-it-works ---------- */ +.pipeline { display: flex; flex-wrap: wrap; gap: 8px; align-items: stretch; margin: 12px 0; } +.layer { flex: 1; min-width: 160px; background: var(--accent-soft); border: 1px solid #cfd7f8; + border-radius: 10px; padding: 10px 12px; font-size: 13px; } +.layer span { display: block; font-weight: 400; font-size: 12px; color: var(--muted); margin-top: 2px; } +.arrow { align-self: center; font-size: 20px; color: var(--accent); } +.alg { background: #15181d; color: #cfe3d8; font-family: var(--mono); font-size: 12px; + border-radius: 10px; padding: 14px 16px; overflow: auto; } +.glossary td:first-child { font-family: var(--mono); font-weight: 700; } + +.foot { text-align: center; padding: 18px; color: var(--muted); font-size: 12px; } + background: #fafaf8; border-bottom: 1px solid var(--border); align-items: center; } +.viewer-head select { font-size: 12px; } +.viewer-content { margin: 0; padding: 12px; font-family: var(--mono); font-size: 12px; + max-height: 560px; overflow: auto; white-space: pre-wrap; word-break: break-word; } \ No newline at end of file diff --git a/src/wikiskill/webapp/templates/index.html b/src/wikiskill/webapp/templates/index.html new file mode 100644 index 0000000..64e2f82 --- /dev/null +++ b/src/wikiskill/webapp/templates/index.html @@ -0,0 +1,202 @@ + + + + + +WikiSkill — Skill Evolution Workbench + + + + +
+
+ arXiv:2608.27454 · Google Research +

WikiSkill Skill Evolution Workbench

+
+
+ idle + Paper ↗ +
+
+ + + +
+ +
+
+
+

Configuration

+
+ + +
+ + +
+ +
+ + + +
+
+

Evolution overview

+

No run yet — start one on the Run tab.

+ +

History

+
+ + + +
IterProposalActionVal scoreOutcome
+
+
+
+ + +
+
+

Test-set evaluation — final skills vs no-skill baseline (paper §4)

+

Evaluation is computed automatically at the end + of every run. You can also re-run it for the current workspace below.

+ + + +
+ +
+ + +
+
+ + +
+
+

Workspace explorer

+
+ + + +
+

+
+ + + + + + +
+
+
+ +
+ 

+        

+      
+    
+  
+          
+          
+        
+      
+
+      

+        
Live console 

+        

+        
Logs stream live; writer events (Wiki Maintainer / Skill
+        Proposer) and validation gating decisions appear as they happen.

+      

+    
+  
\ No newline at end of file
diff --git a/src/wikiskill/workspace.py b/src/wikiskill/workspace.py
index 0819020..8d04f7b 100644
--- a/src/wikiskill/workspace.py
+++ b/src/wikiskill/workspace.py
@@ -73,8 +73,14 @@ class Workspace:
             f.write(content)
 
     # ---------- Raw layer: immutable ----------
-    def add_raw_trace(self, iteration: int, task_id: str, trace: dict) -> str:
-        rel = os.path.join("raw", f"iter_{iteration:03d}", f"{task_id}.json")
+    def add_raw_trace(self, iteration, task_id: str, trace: dict) -> str:
+        """Write an immutable raw-layer trace under raw//.
+
+        `iteration` may be an int (formatted `iter_NNN`) or any string tag (e.g.
+        `eval_skilled` for post-hoc test evaluations). Existing files are never
+        overwritten — the raw layer preserves the complete execution history."""
+        subdir = f"iter_{iteration:03d}" if isinstance(iteration, int) else str(iteration)
+        rel = os.path.join("raw", subdir, f"{task_id}.json")
         p = self._abs(rel)
         if os.path.exists(p):
             raise FileExistsError(f"raw layer is immutable: {rel}")
diff --git a/tests/test_agents.py b/tests/test_agents.py
index 00a40c4..6d6cc77 100644
--- a/tests/test_agents.py
+++ b/tests/test_agents.py
@@ -32,6 +32,22 @@ def test_skills_create_snapshot_restore(tmp_path):
     assert skills.list_skills() == ["s1"]
 
 
+def test_skills_restore_reverts_purpose_md(tmp_path):
+    """A rejected patch must roll back BOTH SKILL.md and PURPOSE.md exactly
+    to S_{k-1} (skills-only rollback, §3.2.4)."""
+    skills = SkillsLayer(_ws(tmp_path))
+    skills.apply_proposal({"action": "create", "skill": "s",
+                           "content": "body", "purpose": "v1"})
+    snap = skills.snapshot()
+    skills.apply_proposal({"action": "patch", "skill": "s",
+                           "edits": [{"op": "append", "content": "extra"}],
+                           "purpose": "v2 (append)"})
+    assert "v2" in skills.purpose("s")
+    skills.restore(snap)
+    assert skills.purpose("s") == "v1\n"
+    assert skills.skill_content("s") == "body"
+
+
 def test_skill_injection_context(tmp_path):
     skills = SkillsLayer(_ws(tmp_path))
     assert "(No active skills.)" in skills.active_skills_context()
diff --git a/tests/test_ui.py b/tests/test_ui.py
new file mode 100644
index 0000000..8905d36
--- /dev/null
+++ b/tests/test_ui.py
@@ -0,0 +1,174 @@
+import os
+import time
+import types
+
+import pytest
+
+from wikiskill import __version__
+from wikiskill.webapp import create_app
+from wikiskill.webapp.api import get_manager
+from wikiskill.webapp.runner import RunManager, _is_fresh_root
+
+
+def _wait_status(client, timeout=20.0):
+    deadline = time.time() + timeout
+    while time.time() < deadline:
+        r = client.get("/api/status")
+        assert r.status_code == 200
+        st = r.get_json()
+        if st["state"] and st["state"]["status"] in ("done", "error"):
+            return st
+        time.sleep(0.05)
+    raise AssertionError("run did not finish in time")
+
+
+@pytest.fixture()
+def client(tmp_path, monkeypatch):
+    monkeypatch.chdir(tmp_path)   # relative workspace roots land under tmp_path
+    app = create_app({"TESTING": True})
+    get_manager()._reset_for_tests()
+    with app.test_client() as c:
+        yield c
+
+
+# ---------------------------------------------------------------- basics
+def test_index_serves_spa(client):
+    r = client.get("/")
+    assert r.status_code == 200
+    html = r.get_data(as_text=True)
+    assert "WikiSkill" in html
+    assert 'id="run-form"' in html
+    assert 'id="console"' in html
+    assert 'id="chart-rbest"' in html
+
+
+def test_health(client):
+    r = client.get("/api/health")
+# ---------------------------------------------------------------- lifecycle
+def test_full_run_lifecycle(client, tmp_path):
+    root = "ui_test_run"
+    r = client.post("/api/run", json={"llm_backend": "mock", "iterations": 3,
+                                     "workspace_root": root})
+    assert r.status_code == 202
+    assert r.get_json()["active"] is True
+
+    # single-flight: second start while active must be rejected
+    r2 = client.post("/api/run", json={"llm_backend": "mock", "iterations": 3,
+                                      "workspace_root": root + "_2"})
+    assert r2.status_code == 409
+
+    st = _wait_status(client)
+    assert st["state"]["status"] == "done"
+    result = st["state"]["result"]
+    assert "unit-conversion" in result["evolution"]["final_skills"]
+    assert result["evaluation"]["significant"] is True
+
+    # workspace artifacts exist on disk
+    assert (tmp_path / root / "wiki" / "index.md").is_file()
+    assert (tmp_path / root / "skills" / "unit-conversion" / "SKILL.md").is_file()
+    # string-tag subdirs for post-hoc test evaluation
+    assert (tmp_path / root / "raw" / "eval_skilled").is_dir()
+
+
+def test_status_empty(client):
+    st = client.get("/api/status").get_json()
+    assert st["active"] is False
+    assert st["state"] is None
+# ---------------------------------------------------------------- workspace & eval
+def test_workspace_endpoint(client, tmp_path):
+    root = "ui_ws_test"
+    client.post("/api/run", json={"llm_backend": "mock", "iterations": 2,
+                                 "workspace_root": root})
+    _wait_status(client)
+
+    r = client.get(f"/api/workspace?root={root}")
+    assert r.status_code == 200
+    data = r.get_json()
+    assert data["root"] == root
+    names = {c["name"] for c in data["tree"]["children"]}
+    assert {"raw", "wiki", "skills"} <= names
+    assert "wiki/index.md" in data["files"]
+    assert "wiki/skill-impact.md" in data["files"]
+    assert any(k.endswith("SKILL.md") for k in data["files"])
+
+    assert client.get("/api/workspace?root=ui_does_not_exist").status_code == 404
+    assert client.get("/api/workspace?root=../x").status_code == 400
+
+
+def test_evaluate_endpoint(client, tmp_path):
+    root = "ui_eval_test"
+    client.post("/api/run", json={"llm_backend": "mock", "iterations": 2,
+                                 "workspace_root": root})
+    _wait_status(client)
+
+    r = client.post("/api/evaluate", json={"llm_backend": "mock",
+                                           "workspace_root": root})
+    assert r.status_code == 200
+    ev = r.get_json()["evaluation"]
+    assert ev["skilled_accuracy"] == 1.0
+    assert ev["baseline_accuracy"] == 0.0
+    assert ev["significant"] is True
+    assert client.post("/api/evaluate", json={"llm_backend": "mock",
+                                              "workspace_root": "missing"}).status_code == 404
+
+
+# ---------------------------------------------------------------- SSE
+def test_sse_stream(client):
+    r = client.post("/api/run", json={"llm_backend": "mock", "iterations": 1,
+                                     "workspace_root": "ui_sse_test"})
+    assert r.status_code == 202
+    stream = client.get("/api/stream")
+    body = b""
+    for chunk in stream.response:
+        body += chunk
+        if b'"type": "done"' in body:
+            break
+    assert b"data:" in body
+    assert b'"done"' in body
+
+
+# ---------------------------------------------------------------- RunManager units
+def test_runmanager_double_start_rejected():
+    mgr = RunManager()
+    mgr._reset_for_tests()
+    mgr._execute = types.MethodType(lambda self, rid, cfg: time.sleep(2), mgr)
+    a = mgr.start({"llm_backend": "mock"})
+    assert a["active"] is True
+    with pytest.raises(PermissionError):
+        mgr.start({"llm_backend": "mock"})
+    mgr.cancel()
+    mgr._reset_for_tests()
+
+
+def test_runmanager_fresh_root_guard(tmp_path):
+    assert _is_fresh_root(str(tmp_path / "new")) is True
+    raw = tmp_path / "used" / "raw"
+    raw.mkdir(parents=True)
+    (raw / "iter_000").mkdir()
+    assert _is_fresh_root(str(tmp_path / "used")) is False
+
+
+def test_cancel(client, tmp_path, monkeypatch):
+    # Slow the orchestrator so cancel lands mid-run (cooperative stop).
+    import wikiskill.orchestrator as orch_mod
+    original_run = orch_mod.EvolutionOrchestrator.run
+
+    def slow_run(self):
+        time.sleep(1.5)
+        return original_run(self)
+
+    monkeypatch.setattr(orch_mod.EvolutionOrchestrator, "run", slow_run)
+    client.post("/api/run", json={"llm_backend": "mock", "iterations": 4,
+                                 "workspace_root": "ui_cancel_run"})
+    assert client.post("/api/cancel", json={}).status_code == 200
+    st = _wait_status(client, timeout=20)
+    assert st["state"]["status"] in ("done", "error")
+
+
+def test_run_validation_400s(client):
+    assert client.post("/api/run", json={"llm_backend": "bogus"}).status_code == 400
+    assert client.post("/api/run", json={"llm_backend": "openai"}).status_code == 400  # no model
+    assert client.post("/api/run", json={"llm_backend": "mock",
+                                         "workspace_root": "../escape"}).status_code == 400
+    assert client.post("/api/run", json={"llm_backend": "mock",
+                                         "workspace_root": "C:\\abs"}).status_code == 400
\ No newline at end of file
diff --git a/ui_design.md b/ui_design.md
new file mode 100644
index 0000000..c4a1799
--- /dev/null
+++ b/ui_design.md
@@ -0,0 +1,147 @@
+# WikiSkill — Web UI Design Document
+
+Companion to `design.md`. Describes the interactive web application that lets users
+run the WikiSkill evolution loop, watch it live, inspect every produced artifact, and
+evaluate the evolved skills — a visual, testable harness for the paper's Algorithm 1.
+
+## 1. Goals
+
+- **Run**: configure a WikiSkill evolution run (dataset, iterations, LLM backend) and
+  start/stop it from the browser.
+- **Watch**: stream the evolution live (logs, ReAct proposer steps, gating decisions).
+- **Inspect**: browse the complete three-layer workspace (`raw/`, `wiki/`, `skills/`)
+  produced by the run.
+- **Evaluate**: compare final evolved skills vs. no-skill baseline on the test split
+  with paired-bootstrap significance.
+- **Understand**: an "How it works" view that maps the UI to the paper's architecture
+  (Figure 2 / Algorithm 1).
+
+Non-goals: skill editing, wiki pruning, multi-user auth. This is a local study tool.
+
+## 2. User workflows
+
+1. **Explore** — open `http://127.0.0.1:5000`, read How-it-works, browse an existing
+   workspace from disk.
+2. **Run** — Configure → Start → watch live events → switch to Dashboard/Workspace.
+3. **Analyze** — inspect the evolution curve, accepted/rejected proposals, wiki
+   patterns, raw traces, then Evaluate on the test split.
+4. **Repeat** — tweak parameters (iterations, ReAct turns), start a new run in a fresh
+   workspace, compare.
+
+## 3. Information architecture (single page, six tabs)
+
+| Tab | Purpose |
+|-----|---------|
+| **Run** | Configuration form + live console + status pill; start/cancel |
+| **Dashboard** | R_best curve, per-iteration acceptance bars, history table |
+| **Evaluation** | Test-split comparison (skilled vs baseline), bootstrap p-value |
+| **Workspace** | File tree + viewer for raw/wiki/skills/run_state.json |
+| **Knowledge** | Wiki index, pattern pages, evolution log, skill-impact tracker, skills |
+| **How it works** | Layer diagram + Algorithm 1 pseudocode + glossary |
+
+## 4. Screen details
+
+### 4.1 Run tab
+- **Config form**: LLM backend (`mock`|`openai`), model, base URL, API key (password
+  field), temperature, max iterations (K), max ReAct turns, workspace root (empty =
+  auto `runs/run_`).
+### 4.3 Evaluation tab
+- Big-number cards: Skilled accuracy, Baseline accuracy, Δ, bootstrap p-value.
+- Horizontal bar comparison + verdict ("Skilled is significantly better", etc.).
+- Re-evaluate button (re-runs `evaluate_test()` for the current workspace).
+
+### 4.4 Workspace tab
+- Left: collapsible file tree starting at the workspace root.
+- Right: content viewer — JSON pretty-printed, markdown shown in raw + rendered form,
+  diffs monospace.
+
+### 4.5 Knowledge tab
+- Sub-tabs: Skills (cards with SKILL.md + PURPOSE.md), Wiki patterns (list + pages),
+  Evolution log, Skill-impact history.
+
+### 4.6 How-it-works tab
+- Static annotated pipeline of the three layers and four components, Algorithm 1
+  pseudocode, and definitions table (S_k, W_k, P_k, T_train, R_best…).
+
+## 5. Design system
+
+- Dark-on-light ("paper" feel): background `#f7f7f5`, panels white `#fff`, ink `#1a1a1a`.
+- Accent: indigo `#3b5bdb` (brand), success `#099268`, danger `#e03131`, warn `#f59f00`.
+- Type: system stack; headings 600, mono stack for code/traces.
+- Badges/pills for Accept/Reject and status; cards with 1px border, 10px radius.
+- Charts rendered as inline SVG (zero JS chart libraries → works offline).
+
+## 6. Backend contract
+
+| Method/Path | Body/Query | Returns |
+|---|---|---|
+| `GET /` | — | SPA HTML |
+| `GET /api/health` | — | `{ok, version}` |
+| `POST /api/run` | config JSON | `202 {run_id}`; `400` validation; `409` busy |
+| `GET /api/status` | — | `{active, state:{status, events[], result, error}}` |
+| `GET /api/stream` | — | Server-Sent Events (`log`/`status`/`react`/`done`/`error`) |
+| `POST /api/cancel` | — | `200 {cancelled}` |
+| `GET /api/workspace?root=` | query | `{root, tree, files}` (text files) |
+| `POST /api/evaluate` | `{workspace_root, llm_backend, …}` | evaluation report JSON |
+
+Validation rules: workspace_root must be a *relative* path (no abs, no `..`); run
+rejected with 409 if another run is active; run rejected if root already contains
+evolution history (choose a fresh root to avoid clobbering the immutable raw layer).
+
+### SSE protocol
+Events: `{type:"log", message}`, `{type:"react", turn, kind, path?, found?}`,
+`{type:"status", phase}`, `{type:"done", result}`, `{type:"error", error}`.
+Heartbeat comments every 15 s; stream closes after `done`/`error`.
+
+## 7. State model
+
+Server (single flask app):
+- `RunManager` — one active run; holds `RunState {run_id, status, events, result, error}`
+  and an SSE queue; background daemon thread executes the `EvolutionOrchestrator`
+  (MockLLM or OpenAICompatLLM) and wires `log` + proposer `on_step` to the queue;
+  `cancel_event` lets the orchestrator stop between iterations.
+
+Client (vanilla JS):
+- `state = {config, status, events[], result, evaluation}`; tab rendering is
+  pure functions over `state`; `EventSource('/api/stream')` drives updates.
+
+## 8. Error handling & resilience
+
+- JSON payloads only; unknown/invalid fields → 400 with message.
+- Proposal failures / malformed LLM JSON are absorbed by the orchestrator (logged).
+- Run exceptions → `type:"error"` event + status `error` (traceback stored, exposed
+  via status).
+- EventSource auto-reconnect on transient drops.
+- All file reads limited to 200 kB per file and whitelisted extension set.
+
+## 9. Accessibility & usability
+
+- Semantic HTML (tabs as buttons with aria), keyboard navigation for tree, focus
+  states, sufficient contrast, no motion-only information.
+
+## 10. Testing strategy
+
+- Backend: Flask test client (mock backend) — run lifecycle, status, workspace tree,
+  evaluation, single-flight 409, validation 400s, cancel.
+- Runner: unit test RunManager concurrency (double-start raises).
+- E2E smoke: boot `wikiskill serve` on a port; fetch `/`, `/api/health`, run
+  `/api/run` with mock, poll status to completion, verify workspace artifacts.
+- Regression: full `pytest` suite (core + UI) must stay green.
+
+## 11. Non-functional targets
+
+- Zero CDN dependencies (offline usable).
+- Startup < 1 s; mock run completes in seconds; UI remains responsive during runs
+  (threaded Flask).
+- Flask is the single new runtime dependency, grouped under `[project.optional-dependencies] ui`.
+- **Actions**: Start, Cancel (appears while running). Status pill: idle | starting |
+  running | evaluating | done | error.
+- **Live console**: appended log lines; SSE-driven; auto-scrolls; monospace.
+- **ReAct event log**: marks `read_file` calls (path + found/not-found) and final
+  proposal, rendered inline in the console with icons.
+
+### 4.2 Dashboard tab
+- **R_best curve**: SVG polyline (x = iteration incl. baseline, y = R_best value).
+- **Validation bars**: one bar per evaluated proposal, colored green=Accepted /
+  red=Rejected; dashed baseline marker.
+- **History table**: iteration, action, target skill, validation score, outcome.
\ No newline at end of file