# WikiSkill — Implementation

Python implementation of **"WikiSkill: Compiling Agent Experience into Persistent
Knowledge for Skill Evolution"** (arXiv:2608.27454v1, Tang et al., Google Research).

Docs: see [`prd.md`](prd.md) (requirements), [`design.md`](design.md) (architecture),
[`plan.md`](plan.md) (build plan).

## Credits & Attribution

This project is an independent, open-source reimplementation of the research paper:

> **WikiSkill: Compiling Agent Experience into Persistent Knowledge for Skill Evolution**
> Liyan Tang, Cyrus Rashtchian, Chun-Sung Ferng, Andrew Tomkins, Da-Cheng Juan, Tu Vu —
> **Google Research** (and Virginia Tech).
> arXiv:2608.27454v1 [cs.AI], 27 Aug 2026. License: CC BY 4.0.
> Paper: https://arxiv.org/abs/2608.27454

All credit for the WikiSkill **method, algorithm (Algorithm 1), three-layer knowledge
architecture, agent designs, experimental findings, and agent prompts (Appendix E)**
belongs to the Google Research authors. This repository reimplements the framework in
code for study/experimentation purposes and is **not** an official Google product.
Agent system prompts in [`src/wikiskill/prompts.py`](src/wikiskill/prompts.py) are
condensed adaptations of those published in the paper (CC BY 4.0); adaptations in this
repo are released under the same spirit — please cite the original paper if you use
this work:

```bibtex
@article{tang2026wikiskill,
  title   = {WikiSkill: Compiling Agent Experience into Persistent Knowledge
             for Skill Evolution},
  author  = {Tang, Liyan and Rashtchian, Cyrus and Ferng, Chun-Sung and
             Tomkins, Andrew and Juan, Da-Cheng and Vu, Tu},
  journal = {arXiv preprint arXiv:2608.27454},
  year    = {2026},
  note    = {Google Research}
}
```


## What it implements

- **Three-layer workspace** (paper §3.1): immutable `raw/` traces, persistent `wiki/`
  (patterns + index + logs + skill-impact tracker), active `skills/` (SKILL.md + PURPOSE.md).
- **Evolutionary loop** (paper Algorithm 1, §3.2): Inference Agent rollouts with skill
  injection but **no wiki access**; Wiki Maintainer consolidation (create/update pattern
  pages via `append`/`replace`/`insert_after` patches); a **ReAct Skill Proposer** that
  reads the wiki and raw traces on demand and emits one atomic skill proposal.
- **Gating & rollback** (§3.2.4): accept iff validation score strictly improves `R_best`;
  otherwise roll back **skills only** — the wiki is never rolled back. The harness
  programmatically appends proposal + unified diff + outcome to `wiki/skill-impact.md`.
- **Stratified trace sampling** (App. C): ≤5 failing + ≤3 passing traces, 15,000-char cap.
- **Paired bootstrap significance testing** (App. C) in `metrics.py`.
- Early stop when `R_best == 1.0`.

## Quickstart

```bash
pip install -e ".[dev]"
pytest                                  # 14 tests incl. end-to-end loop
python -m wikiskill.demo                # offline demo with the deterministic MockLLM
```

Demo output: a `wikiskill_demo_workspace/` with `raw/`, `wiki/`, `skills/` populated,
accepted/rejected proposals logged in `wiki/skill-impact.md`, and `run_state.json`.

## Using a real LLM

```bash
wikiskill evolve --tasks tasks.jsonl \
  --train-ids train.txt --val-ids val.txt --test-ids test.txt \
  --model <model-name> --base-url https://your-endpoint/v1
```

API key is read from `--api-key`, `WIKISKILL_API_KEY`, or `OPENAI_API_KEY`.
Any OpenAI-compatible `/chat/completions` endpoint works (including local vLLM, as in
the paper). `tasks.jsonl` lines: `{"id", "x", "y", "meta?"}`.

## Layout

```
src/wikiskill/
├── workspace.py    # three layers + patch engine + scoped read_file
├── skills.py       # skills layer, apply_proposal, skills-only snapshot/rollback
├── agents.py       # InferenceAgent, WikiMaintainer, ReAct SkillProposer, stratified_sample
├── orchestrator.py # Algorithm 1 with gating/rollback + audit trail
├── prompts.py      # agent system prompts (adapted from paper Appendix E)
├── llm.py          # LLM protocol, OpenAI-compatible client, MockLLM, extract_json
├── metrics.py      # accuracy + paired bootstrap (1,000 iterations)
├── datasets.py     # dataclasses + synthetic demo dataset
├── demo.py / cli.py
└── tests/…         # unit + end-to-end tests
```

## Notes & limitations (mirroring the paper)

- Skills are injected directly into the prompt (no retrieval), matching the paper's setup.
- Gating requires strict improvement; neutral proposals are rejected (paper Limitations).
- The wiki accumulates without pruning (paper Limitations).