Publish hermes-skills-devops via gitea-publish skill
This commit is contained in:
figmar
@@ -0,0 +1,183 @@
|
||||
---
|
||||
name: hermes-troubleshooting
|
||||
description: "Use when Hermes gateway, auth, or platform issues arise."
|
||||
version: 1.0.0
|
||||
category: devops
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [hermes, troubleshooting, gateway, auth, profile, feishu]
|
||||
---
|
||||
|
||||
# Hermes Agent Troubleshooting
|
||||
|
||||
Diagnose and fix common Hermes Agent issues: provider authentication failures, gateway crashes, profile isolation, and messaging platform problems.
|
||||
|
||||
## Profile Isolation — Always Check the Right Profile
|
||||
|
||||
Hermes profiles have **separate** configs, logs, and credentials. A critical first step:
|
||||
|
||||
```bash
|
||||
hermes profile list # See all profiles and which gateway is running
|
||||
hermes status # Check current profile's config
|
||||
```
|
||||
|
||||
**Key insight:** When a messaging platform (Feishu, Telegram, etc.) reports an error, the gateway is running under a **different profile** than the TUI session you're currently in. The TUI session's logs won't contain the platform's errors.
|
||||
|
||||
### Log Locations Per Profile
|
||||
|
||||
| Profile | Logs Directory |
|
||||
|---------|---------------|
|
||||
| default | `/opt/data/logs/` |
|
||||
| fey | `/opt/data/profiles/fey/logs/` |
|
||||
| other | `/opt/data/profiles/<name>/logs/` |
|
||||
|
||||
Each profile has `agent.log`, `errors.log`, and `gateway.log` (if gateway is running).
|
||||
|
||||
## Provider Authentication Failed
|
||||
|
||||
**Symptom:** User on a messaging platform sees "Provider authentication failed. Check the configured credentials; raw provider details are in the gateway logs."
|
||||
|
||||
**Root cause:** This almost always means the **model API key** (in `config.yaml` → `model.api_key`) is invalid/expired, NOT the platform credentials (App ID, App Secret, etc.).
|
||||
|
||||
### Diagnostic Steps
|
||||
|
||||
1. **Find which profile the gateway runs under:**
|
||||
```bash
|
||||
hermes profile list
|
||||
# Look for the profile with "running" gateway
|
||||
```
|
||||
|
||||
2. **Check that profile's agent.log for the real error:**
|
||||
```bash
|
||||
grep -i "401\|auth.*fail\|invalid.*key\|user_key" /opt/data/logs/agent.log | tail -20
|
||||
```
|
||||
(Adjust path for non-default profiles.)
|
||||
|
||||
3. **Check the model config:**
|
||||
```bash
|
||||
grep "api_key\|base_url\|model:" /opt/data/config.yaml | head -10
|
||||
```
|
||||
|
||||
4. **Check platform config (to confirm platform creds are fine):**
|
||||
```bash
|
||||
grep -i "feishu\|telegram\|discord" /opt/data/.env | head -10
|
||||
```
|
||||
|
||||
### Common Causes
|
||||
|
||||
- **Custom proxy endpoint key expired:** If `model.base_url` points to a proxy (e.g., `http://113.249.102.8:18080/...`), the proxy's API key may have expired. The proxy returns `401: Authentication failed: invalid user_key`.
|
||||
- **API key rotated elsewhere:** The key was changed in the provider's dashboard but not in Hermes `.env`/`config.yaml`.
|
||||
- **Wrong profile's key:** The key in the gateway profile doesn't match the one you think is active.
|
||||
- **Key format mismatch when switching providers:** Different providers use different key formats. A key valid for one provider (e.g., `sk-sp-...` for a custom proxy) will **not** work for another (e.g., DashScope expects `sk-ant-...`, DeepSeek expects `sk-...`). When changing `model.base_url`, you must also provide a key from the new provider. Verify the key with a quick curl test before restarting the gateway:
|
||||
```bash
|
||||
curl -s -o /dev/null -w "%{http_code}" https://api.deepseek.com/v1/chat/completions \
|
||||
-H "Authorization: Bearer YOUR_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"model":"deepseek-chat","messages":[{"role":"user","content":"hi"}],"max_tokens":1}'
|
||||
# Expect 200, not 401
|
||||
```
|
||||
|
||||
### Fix
|
||||
|
||||
```bash
|
||||
# Update the API key in the correct profile's config
|
||||
hermes -p <profile_name> config set model.api_key 'new-key'
|
||||
|
||||
# If also changing the provider endpoint:
|
||||
hermes -p <profile_name> config set model.base_url 'https://api.provider.com/v1'
|
||||
hermes -p <profile_name> config set model.default 'model-name'
|
||||
|
||||
# Restart gateway (may need s6 kill pattern — see references/s6-gateway-restart.md)
|
||||
hermes gateway restart
|
||||
```
|
||||
|
||||
### `hermes` CLI Not on PATH
|
||||
|
||||
In containerized setups, `hermes` may not be on the default PATH. Use the full path:
|
||||
```bash
|
||||
/opt/hermes/.venv/bin/hermes -p default config set model.api_key 'new-key'
|
||||
```
|
||||
|
||||
### WebSocket 1011 and "session not found"
|
||||
|
||||
These are **symptoms**, not root causes. When the model API returns 401 repeatedly:
|
||||
- Gateway's conversation loop fails → internal error → WebSocket close code 1011
|
||||
- Session cache expires during the outage → "session not found"
|
||||
- Gateway may disconnect/reconnect the platform adapter
|
||||
|
||||
Fix the auth issue first; these resolve automatically.
|
||||
|
||||
## Gateway Not Running
|
||||
|
||||
**Symptom:** `hermes gateway status` shows "stopped" or messaging platforms show "not configured".
|
||||
|
||||
### Check
|
||||
|
||||
```bash
|
||||
hermes gateway status
|
||||
hermes profile list # See which profiles have running gateways
|
||||
```
|
||||
|
||||
### Fix
|
||||
|
||||
```bash
|
||||
hermes gateway run # Foreground (for testing)
|
||||
hermes gateway install # Install as background service
|
||||
hermes gateway start # Start the service
|
||||
```
|
||||
|
||||
If gateway is installed but stopped (e.g., in a container with s6):
|
||||
```bash
|
||||
# Check if it's a container environment
|
||||
hermes status | grep "Manager"
|
||||
# If s6: the service may need manual start or container restart
|
||||
```
|
||||
|
||||
**Cannot restart from inside gateway?** See `references/s6-gateway-restart.md` for the kill + s6 auto-restart pattern.
|
||||
|
||||
## Gateway Crash Loop
|
||||
|
||||
**Symptom:** Gateway keeps dying and restarting.
|
||||
|
||||
### Check crash logs
|
||||
|
||||
```bash
|
||||
# TUI gateway crash log (thread stacks at crash)
|
||||
tail -100 ~/.hermes/logs/tui_gateway_crash.log
|
||||
|
||||
# Systemd service failures
|
||||
systemctl --user status hermes-gateway
|
||||
journalctl --user -u hermes-gateway --since "1 hour ago"
|
||||
```
|
||||
|
||||
### Common causes
|
||||
|
||||
- **SSH logout kills gateway:** Enable linger: `sudo loginctl enable-linger $USER`
|
||||
- **WSL2 close kills gateway:** Requires `systemd=true` in `/etc/wsl.conf`
|
||||
- **Reset failed state:** `systemctl --user reset-failed hermes-gateway`
|
||||
|
||||
## Feishu-Specific Issues
|
||||
|
||||
### Feishu shows "not configured" in one profile but works in another
|
||||
|
||||
Feishu credentials are per-profile. Check the profile that runs the gateway:
|
||||
|
||||
```bash
|
||||
# In the gateway's profile:
|
||||
grep "FEISHU" ~/.hermes/.env
|
||||
# Should have: FEISHU_APP_ID, FEISHU_APP_SECRET, FEISHU_CONNECTION_MODE
|
||||
```
|
||||
|
||||
### Feishu WebSocket disconnects
|
||||
|
||||
Normal disconnects (code 1000 "bye") happen during gateway restart. Abnormal disconnects (code 1011) indicate internal errors — usually model auth failures.
|
||||
|
||||
## Quick Diagnostic Checklist
|
||||
|
||||
When something is broken:
|
||||
|
||||
1. `hermes profile list` — Which profile is running?
|
||||
2. `hermes status` — Is gateway running? Are platforms configured?
|
||||
3. `hermes doctor` — Detailed diagnostics
|
||||
4. Check the **right profile's** logs — not the TUI session's logs
|
||||
5. `grep "401\|auth\|fail" agent.log` — Find the real error behind generic messages
|
||||
@@ -0,0 +1,59 @@
|
||||
# Gateway Restart in s6 Container
|
||||
|
||||
## Problem
|
||||
|
||||
`hermes gateway restart` refuses to run from inside the gateway process (prevents restart loops):
|
||||
|
||||
```
|
||||
✗ Refusing to restart the gateway from inside the gateway process.
|
||||
This command was blocked to prevent restart loops.
|
||||
Use `hermes gateway restart` from a shell outside the running gateway.
|
||||
```
|
||||
|
||||
## Solution: Kill + S6 Auto-Restart
|
||||
|
||||
The gateway runs under s6 supervision in a Docker container. s6 auto-restarts services on exit.
|
||||
|
||||
### Step 1: Find the gateway PID
|
||||
|
||||
```bash
|
||||
ps aux | grep "hermes.*gateway\|gateway.*hermes" | grep -v grep
|
||||
# Look for: hermes gateway run --replace
|
||||
```
|
||||
|
||||
### Step 2: Send SIGHUP (graceful) or SIGKILL (force)
|
||||
|
||||
```bash
|
||||
kill -HUP <pid> # Graceful — s6 will restart it
|
||||
# or
|
||||
kill -9 <pid> # Force — use if HUP hangs
|
||||
```
|
||||
|
||||
### Step 3: Verify restart
|
||||
|
||||
```bash
|
||||
sleep 3
|
||||
ps aux | grep "hermes.*gateway" | grep -v grep
|
||||
# Should show a NEW PID (s6 restarted it)
|
||||
```
|
||||
|
||||
### Step 4: Check logs for clean startup
|
||||
|
||||
```bash
|
||||
tail -20 /opt/data/logs/gateway.log
|
||||
# Look for: "Starting Hermes Gateway...", platform connections
|
||||
```
|
||||
|
||||
## When to Use This
|
||||
|
||||
- After updating `config.yaml` or `.env` for the default profile
|
||||
- After `hermes -p <profile> config set ...` changes that need gateway reload
|
||||
- When gateway is in a bad state (OOM, stuck, 401 loop)
|
||||
- When running from inside the gateway process (TUI session, cron job, etc.)
|
||||
|
||||
## Important Notes
|
||||
|
||||
- **Do NOT use `nohup` or `&`** — s6 manages the lifecycle
|
||||
- **PID will change** — s6 spawns a new process; don't track the old PID
|
||||
- **Config reload is automatic** — gateway reads config.yaml on startup
|
||||
- **Platform reconnection** — Feishu, Weixin, etc. reconnect within ~5 seconds
|
||||
@@ -0,0 +1,189 @@
|
||||
---
|
||||
name: kanban-orchestrator
|
||||
description: Decomposition playbook + anti-temptation rules for an orchestrator profile routing work through Kanban. The "don't do the work yourself" rule and the basic lifecycle are auto-injected into every kanban worker's system prompt; this skill is the deeper playbook when you're specifically playing the orchestrator role.
|
||||
version: 3.0.0
|
||||
platforms: [linux, macos, windows]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [kanban, multi-agent, orchestration, routing]
|
||||
related_skills: [kanban-worker]
|
||||
---
|
||||
|
||||
# Kanban Orchestrator — Decomposition Playbook
|
||||
|
||||
> The **core worker lifecycle** (including the `kanban_create` fan-out pattern and the "decompose, don't execute" rule) is auto-injected into every kanban process via the `KANBAN_GUIDANCE` system-prompt block. This skill is the deeper playbook when you're an orchestrator profile whose whole job is routing.
|
||||
|
||||
## Profiles are user-configured — not a fixed roster
|
||||
|
||||
Hermes setups vary widely. Some users run a single profile that does everything; some run a small fleet (`docker-worker`, `cron-worker`); some run a curated specialist team they've named themselves. There is **no default specialist roster** — the orchestrator skill does not know what profiles exist on this machine.
|
||||
|
||||
Before fanning out, you must ground the decomposition in the profiles that actually exist. The dispatcher silently fails to spawn unknown assignee names — it doesn't autocorrect, doesn't suggest, doesn't fall back. So a card assigned to `researcher` on a setup that only has `docker-worker` just sits in `ready` forever.
|
||||
|
||||
**Step 0: discover available profiles before planning.**
|
||||
|
||||
Use one of these:
|
||||
|
||||
- `hermes profile list` — prints the table of profiles configured on this machine. Run it through your terminal tool if you have one; otherwise ask the user.
|
||||
- `kanban_list(assignee="<some-name>")` — sanity-check a single name. Returns an empty list (rather than an error) for an unknown assignee, so this only confirms a name you're already considering.
|
||||
- **Just ask the user.** "What profiles do you have set up?" is a fine first turn when the goal needs more than one specialist.
|
||||
|
||||
Cache the result in your working memory for the rest of the conversation. Re-asking every turn wastes a tool call.
|
||||
|
||||
## When to use the board (vs. just doing the work)
|
||||
|
||||
Create Kanban tasks when any of these are true:
|
||||
|
||||
1. **Multiple specialists are needed.** Research + analysis + writing is three profiles.
|
||||
2. **The work should survive a crash or restart.** Long-running, recurring, or important.
|
||||
3. **The user might want to interject.** Human-in-the-loop at any step.
|
||||
4. **Multiple subtasks can run in parallel.** Fan-out for speed.
|
||||
5. **Review / iteration is expected.** A reviewer profile loops on drafter output.
|
||||
6. **The audit trail matters.** Board rows persist in SQLite forever.
|
||||
|
||||
If *none* of those apply — it's a small one-shot reasoning task — use `delegate_task` instead or answer the user directly.
|
||||
|
||||
## The anti-temptation rules
|
||||
|
||||
Your job description says "route, don't execute." The rules that enforce that:
|
||||
|
||||
- **Do not execute the work yourself.** Your restricted toolset usually doesn't even include terminal/file/code/web for implementation. If you find yourself "just fixing this quickly" — stop and create a task for the right specialist.
|
||||
- **For any concrete task, create a Kanban task and assign it.** Every single time.
|
||||
- **Split multi-lane requests before creating cards.** A user prompt can contain several independent workstreams. Extract those lanes first, then create one card per lane instead of bundling unrelated work into a single implementer card.
|
||||
- **Run independent lanes in parallel.** If two cards do not need each other's output, leave them unlinked so the dispatcher can fan them out. Link only true data dependencies.
|
||||
- **Never create dependent work as independent ready cards.** If a card must wait for another card, pass `parents=[...]` in the original `kanban_create` call. Do not create it first and link it later, and do not rely on prose like "wait for T1" inside the body.
|
||||
- **If no specialist fits the available profiles, ask the user which profile to create or which existing profile to use.** Do not invent profile names; the dispatcher will silently drop unknown assignees.
|
||||
- **Decompose, route, and summarize — that's the whole job.**
|
||||
|
||||
## Decomposition playbook
|
||||
|
||||
### Step 1 — Understand the goal
|
||||
|
||||
Ask clarifying questions if the goal is ambiguous. Cheap to ask; expensive to spawn the wrong fleet.
|
||||
|
||||
### Step 2 — Sketch the task graph
|
||||
|
||||
Before creating anything, draft the graph out loud (in your response to the user). Treat every concrete workstream as a candidate card:
|
||||
|
||||
1. Extract the lanes from the request.
|
||||
2. Map each lane to one of the profiles you discovered in Step 0. If a lane doesn't fit any existing profile, ask the user which to use or create.
|
||||
3. Decide whether each lane is independent or gated by another lane.
|
||||
4. Create independent lanes as parallel cards with no parent links.
|
||||
5. Create synthesis/review/integration cards with parent links to the lanes they depend on. A child created with unfinished parents starts in `todo`; the dispatcher promotes it to `ready` only after every parent is done.
|
||||
|
||||
Examples of prompts that should fan out (using placeholder profile names — substitute whatever exists on the user's setup):
|
||||
|
||||
- "Build an app" → one card to a design-oriented profile for product/UI direction, one or two cards to engineering profiles for implementation, plus a later integration/review card if the user has a reviewer profile.
|
||||
- "Fix blockers and check model variants" → one implementation card for the blocker fixes plus one discovery/research card for config/source verification. A final reviewer card can depend on both.
|
||||
- "Research docs and implement" → a docs-research card can run in parallel with a codebase-discovery card; implementation waits only if it truly needs those findings.
|
||||
- "Analyze this screenshot and find the related code" → one card to a vision-capable profile for the visual analysis while another searches the codebase.
|
||||
|
||||
Words like "also," "finally," or "and" do not automatically imply a dependency. They often mean "make sure this is covered before reporting back." Only link tasks when one card cannot start until another card's output exists.
|
||||
|
||||
Show the graph to the user before creating cards. Let them correct it — including which actual profile name should own each lane.
|
||||
|
||||
### Step 3 — Create tasks and link
|
||||
|
||||
Use the profile names from Step 0. The example below uses placeholders `<profile-A>`, `<profile-B>`, `<profile-C>` — replace them with what the user actually has.
|
||||
|
||||
```python
|
||||
t1 = kanban_create(
|
||||
title="research: Postgres cost vs current",
|
||||
assignee="<profile-A>", # whichever profile handles research on this setup
|
||||
body="Compare estimated infrastructure costs, migration costs, and ongoing ops costs over a 3-year window. Sources: AWS/GCP pricing, team time estimates, current Postgres bills from peers.",
|
||||
tenant=os.environ.get("HERMES_TENANT"),
|
||||
)["task_id"]
|
||||
|
||||
t2 = kanban_create(
|
||||
title="research: Postgres performance vs current",
|
||||
assignee="<profile-A>", # same profile, run in parallel
|
||||
body="Compare query latency, throughput, and scaling characteristics at our expected data volume (~500GB, 10k QPS peak). Sources: benchmark papers, public case studies, pgbench results if easy.",
|
||||
)["task_id"]
|
||||
|
||||
t3 = kanban_create(
|
||||
title="synthesize migration recommendation",
|
||||
assignee="<profile-B>", # whichever profile does synthesis/analysis
|
||||
body="Read the findings from T1 (cost) and T2 (performance). Produce a 1-page recommendation with explicit trade-offs and a go/no-go call.",
|
||||
parents=[t1, t2],
|
||||
)["task_id"]
|
||||
|
||||
t4 = kanban_create(
|
||||
title="draft decision memo",
|
||||
assignee="<profile-C>", # whichever profile drafts user-facing prose
|
||||
body="Turn the analyst's recommendation into a 2-page memo for the CTO. Match the tone of previous decision memos in the team's knowledge base.",
|
||||
parents=[t3],
|
||||
)["task_id"]
|
||||
```
|
||||
|
||||
`parents=[...]` gates promotion — children stay in `todo` until every parent reaches `done`, then auto-promote to `ready`. No manual coordination needed; the dispatcher and dependency engine handle it.
|
||||
|
||||
If the task graph has dependencies, create the parent cards first, capture their returned ids, and include those ids in the child card's `parents` list during the child `kanban_create` call. Avoid creating all cards in parallel and linking them afterward; that creates a window where the dispatcher can claim a child before its inputs exist.
|
||||
|
||||
### Step 4 — Complete your own task
|
||||
|
||||
If you were spawned as a task yourself (e.g. a planner profile was assigned `T0: "investigate Postgres migration"`), mark it done with a summary of what you created:
|
||||
|
||||
```python
|
||||
kanban_complete(
|
||||
summary="decomposed into T1-T4: 2 research lanes in parallel, 1 synthesis on their outputs, 1 prose draft on the recommendation",
|
||||
metadata={
|
||||
"task_graph": {
|
||||
"T1": {"assignee": "<profile-A>", "parents": []},
|
||||
"T2": {"assignee": "<profile-A>", "parents": []},
|
||||
"T3": {"assignee": "<profile-B>", "parents": ["T1", "T2"]},
|
||||
"T4": {"assignee": "<profile-C>", "parents": ["T3"]},
|
||||
},
|
||||
},
|
||||
)
|
||||
```
|
||||
|
||||
### Step 5 — Report back to the user
|
||||
|
||||
Tell them what you created in plain prose, naming the actual profiles you used:
|
||||
|
||||
> I've queued 4 tasks:
|
||||
> - **T1** (`<profile-A>`): cost comparison
|
||||
> - **T2** (`<profile-A>`): performance comparison, in parallel with T1
|
||||
> - **T3** (`<profile-B>`): synthesizes T1 + T2 into a recommendation
|
||||
> - **T4** (`<profile-C>`): turns T3 into a CTO memo
|
||||
>
|
||||
> The dispatcher will pick up T1 and T2 now. T3 starts when both finish. You'll get a gateway ping when T4 completes. Use the dashboard or `hermes kanban tail <id>` to follow along.
|
||||
|
||||
## Common patterns
|
||||
|
||||
**Fan-out + fan-in (research → synthesize):** N research-style cards with no parents, one synthesis card with all of them as parents.
|
||||
|
||||
**Parallel implementation + validation:** one implementer card makes the change while one explorer/researcher card verifies config, docs, or source mapping. A reviewer card can depend on both. Do not make the implementer own unrelated verification just because the user mentioned both in one sentence.
|
||||
|
||||
**Pipeline with gates:** `planner → implementer → reviewer`. Each stage's `parents=[previous_task]`. Reviewer blocks or completes; if reviewer blocks, the operator unblocks with feedback and respawns.
|
||||
|
||||
**Same-profile queue:** N tasks, all assigned to the same profile, no dependencies between them. Dispatcher serializes — that profile processes them in priority order, accumulating experience in its own memory.
|
||||
|
||||
**Human-in-the-loop:** Any task can `kanban_block()` to wait for input. Dispatcher respawns after `/unblock`. The comment thread carries the full context.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
**Inventing profile names that don't exist.** The dispatcher silently fails to spawn unknown assignees — the card just sits in `ready` forever. Always assign to a profile from your Step 0 discovery; ask the user if you're unsure.
|
||||
|
||||
**Bundling independent lanes into one card.** If the user asks for two independent outcomes, create two cards. Example: "fix blockers and check model variants" is not one fixer task; create a fixer/engineer card for the fixes and an explorer/researcher card for the variant check, then optionally gate review on both.
|
||||
|
||||
**Over-linking because of wording.** "Finally check X" may still be parallel with implementation if X is static config, docs, or source discovery. Link it after implementation only when the check depends on the implementation result.
|
||||
|
||||
**Forgetting dependency links.** If the task graph says `research -> implement -> review`, do not create all tasks as independent ready cards. Use parent links so implement/review cannot run before their inputs exist.
|
||||
|
||||
**Reassignment vs. new task.** If a reviewer blocks with "needs changes," create a NEW task linked from the reviewer's task — don't re-run the same task with a stern look. The new task is assigned to the original implementer profile.
|
||||
|
||||
**Argument order for links.** `kanban_link(parent_id=..., child_id=...)` — parent first. Mixing them up demotes the wrong task to `todo`.
|
||||
|
||||
**Don't pre-create the whole graph if the shape depends on intermediate findings.** If T3's structure depends on what T1 and T2 find, let T3 exist as a "synthesize findings" task whose own first step is to read parent handoffs and plan the rest. Orchestrators can spawn orchestrators.
|
||||
|
||||
**Tenant inheritance.** If `HERMES_TENANT` is set in your env, pass `tenant=os.environ.get("HERMES_TENANT")` on every `kanban_create` call so child tasks stay in the same namespace.
|
||||
|
||||
## Recovering stuck workers
|
||||
|
||||
When a worker profile keeps crashing, hallucinating, or getting blocked by its own mistakes (usually: wrong model, missing skill, broken credential), the kanban dashboard flags the task with a ⚠ badge and opens a **Recovery** section in the drawer. Three primary actions:
|
||||
|
||||
1. **Reclaim** (or `hermes kanban reclaim <task_id>`) — abort the running worker immediately and reset the task to `ready`. The existing claim TTL is ~15 min; this is the fast path out.
|
||||
2. **Reassign** (or `hermes kanban reassign <task_id> <new-profile> --reclaim`) — switch the task to a different profile (one that exists on this setup) and let the dispatcher pick it up with a fresh worker.
|
||||
3. **Change profile model** — the dashboard prints a copy-paste hint for `hermes -p <profile> model` since profile config lives on disk; edit it in a terminal, then Reclaim to retry with the new model.
|
||||
|
||||
Hallucination warnings appear on tasks where a worker's `kanban_complete(created_cards=[...])` claim included card ids that don't exist or weren't created by the worker's profile (the gate blocks the completion), or where the free-form summary references `t_<hex>` ids that don't resolve (advisory prose scan, non-blocking). Both produce audit events that persist even after recovery actions — the trail stays for debugging.
|
||||
@@ -0,0 +1,184 @@
|
||||
---
|
||||
name: kanban-worker
|
||||
description: Pitfalls, examples, and edge cases for Hermes Kanban workers. The lifecycle itself is auto-injected into every worker's system prompt as KANBAN_GUIDANCE (from agent/prompt_builder.py); this skill is what you load when you want deeper detail on specific scenarios.
|
||||
version: 2.0.0
|
||||
platforms: [linux, macos, windows]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [kanban, multi-agent, collaboration, workflow, pitfalls]
|
||||
related_skills: [kanban-orchestrator]
|
||||
---
|
||||
|
||||
# Kanban Worker — Pitfalls and Examples
|
||||
|
||||
> You're seeing this skill because the Hermes Kanban dispatcher spawned you as a worker with `--skills kanban-worker` — it's loaded automatically for every dispatched worker. The **lifecycle** (6 steps: orient → work → heartbeat → block/complete) also lives in the `KANBAN_GUIDANCE` block that's auto-injected into your system prompt. This skill is the deeper detail: good handoff shapes, retry diagnostics, edge cases.
|
||||
|
||||
## Workspace handling
|
||||
|
||||
Your workspace kind determines how you should behave inside `$HERMES_KANBAN_WORKSPACE`:
|
||||
|
||||
| Kind | What it is | How to work |
|
||||
|---|---|---|
|
||||
| `scratch` | Fresh tmp dir, yours alone | Read/write freely; it gets GC'd when the task is archived. |
|
||||
| `dir:<path>` | Shared persistent directory | Other runs will read what you write. Treat it like long-lived state. Path is guaranteed absolute (the kernel rejects relative paths). |
|
||||
| `worktree` | Git worktree at the resolved path | If `.git` doesn't exist, run `git worktree add <path> <branch>` from the main repo first, then cd and work normally. Commit work here. |
|
||||
|
||||
## Tenant isolation
|
||||
|
||||
If `$HERMES_TENANT` is set, the task belongs to a tenant namespace. When reading or writing persistent memory, prefix memory entries with the tenant so context doesn't leak across tenants:
|
||||
|
||||
- Good: `business-a: Acme is our biggest customer`
|
||||
- Bad (leaks): `Acme is our biggest customer`
|
||||
|
||||
## Good summary + metadata shapes
|
||||
|
||||
The `kanban_complete(summary=..., metadata=...)` handoff is how downstream workers read what you did. Patterns that work:
|
||||
|
||||
**Coding task:**
|
||||
```python
|
||||
kanban_complete(
|
||||
summary="shipped rate limiter — token bucket, keys on user_id with IP fallback, 14 tests pass",
|
||||
metadata={
|
||||
"changed_files": ["rate_limiter.py", "tests/test_rate_limiter.py"],
|
||||
"tests_run": 14,
|
||||
"tests_passed": 14,
|
||||
"decisions": ["user_id primary, IP fallback for unauthenticated requests"],
|
||||
},
|
||||
)
|
||||
```
|
||||
|
||||
**Coding task that needs human review (review-required):**
|
||||
|
||||
For most code-changing tasks, the work isn't truly *done* until a human reviewer has eyes on it. Block instead of complete, with `reason` prefixed `review-required: ` so the dashboard surfaces the row as needing review. Drop the structured metadata (changed files, test counts, diff/PR url) into a comment first, since `kanban_block` only carries the human-readable reason — comments are the durable annotation channel. Reviewer either approves and runs `hermes kanban unblock <id>` (which re-spawns you with the comment thread for any follow-ups) or asks for changes via another comment.
|
||||
|
||||
```python
|
||||
import json
|
||||
|
||||
kanban_comment(
|
||||
body="review-required handoff:\n" + json.dumps({
|
||||
"changed_files": ["rate_limiter.py", "tests/test_rate_limiter.py"],
|
||||
"tests_run": 14,
|
||||
"tests_passed": 14,
|
||||
"diff_path": "/path/to/worktree", # or PR url if pushed
|
||||
"decisions": ["user_id primary, IP fallback for unauthenticated requests"],
|
||||
}, indent=2),
|
||||
)
|
||||
kanban_block(
|
||||
reason="review-required: rate limiter shipped, 14/14 tests pass — needs eyes on the user_id/IP fallback choice before merging",
|
||||
)
|
||||
```
|
||||
|
||||
Use `kanban_complete` only when the task is genuinely terminal — e.g. a one-line typo fix, a docs change with no functional consequences, or a research task where the artifact IS the writeup itself.
|
||||
|
||||
**Research task:**
|
||||
```python
|
||||
kanban_complete(
|
||||
summary="3 competing libraries reviewed; vLLM wins on throughput, SGLang on latency, Tensorrt-LLM on memory efficiency",
|
||||
metadata={
|
||||
"sources_read": 12,
|
||||
"recommendation": "vLLM",
|
||||
"benchmarks": {"vllm": 1.0, "sglang": 0.87, "trtllm": 0.72},
|
||||
},
|
||||
)
|
||||
```
|
||||
|
||||
**Review task:**
|
||||
```python
|
||||
kanban_complete(
|
||||
summary="reviewed PR #123; 2 blocking issues found (SQL injection in /search, missing CSRF on /settings)",
|
||||
metadata={
|
||||
"pr_number": 123,
|
||||
"findings": [
|
||||
{"severity": "critical", "file": "api/search.py", "line": 42, "issue": "raw SQL concat"},
|
||||
{"severity": "high", "file": "api/settings.py", "issue": "missing CSRF middleware"},
|
||||
],
|
||||
"approved": False,
|
||||
},
|
||||
)
|
||||
```
|
||||
|
||||
Shape `metadata` so downstream parsers (reviewers, aggregators, schedulers) can use it without re-reading your prose.
|
||||
|
||||
## Claiming cards you actually created
|
||||
|
||||
If your run produced new kanban tasks (via `kanban_create`), pass the ids in `created_cards` on `kanban_complete`. The kernel verifies each id exists and was created by your profile; any phantom id blocks the completion with an error listing what went wrong, and the rejected attempt is permanently recorded on the task's event log. **Only list ids you captured from a successful `kanban_create` return value — never invent ids from prose, never paste ids from earlier runs, never claim cards another worker created.**
|
||||
|
||||
```python
|
||||
# GOOD — capture return values, then claim them.
|
||||
c1 = kanban_create(title="remediate SQL injection", assignee="security-worker")
|
||||
c2 = kanban_create(title="fix CSRF middleware", assignee="web-worker")
|
||||
|
||||
kanban_complete(
|
||||
summary="Review done; spawned remediations for both findings.",
|
||||
metadata={"pr_number": 123, "approved": False},
|
||||
created_cards=[c1["task_id"], c2["task_id"]],
|
||||
)
|
||||
```
|
||||
|
||||
```python
|
||||
# BAD — claiming ids you don't have captured return values for.
|
||||
kanban_complete(
|
||||
summary="Created remediation cards t_a1b2c3d4, t_deadbeef", # hallucinated
|
||||
created_cards=["t_a1b2c3d4", "t_deadbeef"], # → gate rejects
|
||||
)
|
||||
```
|
||||
|
||||
If a `kanban_create` call fails (exception, tool_error), the card was NOT created — do not include a phantom id for it. Retry the create, or omit the id and mention the failure in your summary. The prose-scan pass also catches `t_<hex>` references in your free-form summary that don't resolve; these don't block the completion but show up as advisory warnings on the task in the dashboard.
|
||||
|
||||
## Block reasons that get answered fast
|
||||
|
||||
Bad: `"stuck"` — the human has no context.
|
||||
|
||||
Good: one sentence naming the specific decision you need. Leave longer context as a comment instead.
|
||||
|
||||
```python
|
||||
kanban_comment(
|
||||
task_id=os.environ["HERMES_KANBAN_TASK"],
|
||||
body="Full context: I have user IPs from Cloudflare headers but some users are behind NATs with thousands of peers. Keying on IP alone causes false positives.",
|
||||
)
|
||||
kanban_block(reason="Rate limit key choice: IP (simple, NAT-unsafe) or user_id (requires auth, skips anonymous endpoints)?")
|
||||
```
|
||||
|
||||
The block message is what appears in the dashboard / gateway notifier. The comment is the deeper context a human reads when they open the task.
|
||||
|
||||
## Heartbeats worth sending
|
||||
|
||||
Good heartbeats name progress: `"epoch 12/50, loss 0.31"`, `"scanned 1.2M/2.4M rows"`, `"uploaded 47/120 videos"`.
|
||||
|
||||
Bad heartbeats: `"still working"`, empty notes, sub-second intervals. Every few minutes max; skip entirely for tasks under ~2 minutes.
|
||||
|
||||
## Retry scenarios
|
||||
|
||||
If you open the task and `kanban_show` returns `runs: [...]` with one or more closed runs, you're a retry. The prior runs' `outcome` / `summary` / `error` tell you what didn't work. Don't repeat that path. Typical retry diagnostics:
|
||||
|
||||
- `outcome: "timed_out"` — the previous attempt hit `max_runtime_seconds`. You may need to chunk the work or shorten it.
|
||||
- `outcome: "crashed"` — OOM or segfault. Reduce memory footprint.
|
||||
- `outcome: "spawn_failed"` + `error: "..."` — usually a profile config issue (missing credential, bad PATH). Ask the human via `kanban_block` instead of retrying blindly.
|
||||
- `outcome: "reclaimed"` + `summary: "task archived..."` — operator archived the task out from under the previous run; you probably shouldn't be running at all, check status carefully.
|
||||
- `outcome: "blocked"` — a previous attempt blocked; the unblock comment should be in the thread by now.
|
||||
|
||||
## Do NOT
|
||||
|
||||
- Call `delegate_task` as a substitute for `kanban_create`. `delegate_task` is for short reasoning subtasks inside YOUR run; `kanban_create` is for cross-agent handoffs that outlive one API loop.
|
||||
- Modify files outside `$HERMES_KANBAN_WORKSPACE` unless the task body says to.
|
||||
- Create follow-up tasks assigned to yourself — assign to the right specialist.
|
||||
- Complete a task you didn't actually finish. Block it instead.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
**Task state can change between dispatch and your startup.** Between when the dispatcher claimed and when your process actually booted, the task may have been blocked, reassigned, or archived. Always `kanban_show` first. If it reports `blocked` or `archived`, stop — you shouldn't be running.
|
||||
|
||||
**Workspace may have stale artifacts.** Especially `dir:` and `worktree` workspaces can have files from previous runs. Read the comment thread — it usually explains why you're running again and what state the workspace is in.
|
||||
|
||||
**Don't rely on the CLI when the guidance is available.** The `kanban_*` tools work across all terminal backends (Docker, Modal, SSH). `hermes kanban <verb>` from your terminal tool will fail in containerized backends because the CLI isn't installed there. When in doubt, use the tool.
|
||||
|
||||
## CLI fallback (for scripting)
|
||||
|
||||
Every tool has a CLI equivalent for human operators and scripts:
|
||||
- `kanban_show` ↔ `hermes kanban show <id> --json`
|
||||
- `kanban_complete` ↔ `hermes kanban complete <id> --summary "..." --metadata '{...}'`
|
||||
- `kanban_block` ↔ `hermes kanban block <id> "reason"`
|
||||
- `kanban_create` ↔ `hermes kanban create "title" --assignee <profile> [--parent <id>]`
|
||||
- etc.
|
||||
|
||||
Use the tools from inside an agent; the CLI exists for the human at the terminal.
|
||||
@@ -0,0 +1,204 @@
|
||||
---
|
||||
name: webhook-subscriptions
|
||||
description: "Webhook subscriptions: event-driven agent runs."
|
||||
version: 1.1.0
|
||||
platforms: [linux, macos, windows]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [webhook, events, automation, integrations, notifications, push]
|
||||
---
|
||||
|
||||
# Webhook Subscriptions
|
||||
|
||||
Create dynamic webhook subscriptions so external services (GitHub, GitLab, Stripe, CI/CD, IoT sensors, monitoring tools) can trigger Hermes agent runs by POSTing events to a URL.
|
||||
|
||||
## Setup (Required First)
|
||||
|
||||
The webhook platform must be enabled before subscriptions can be created. Check with:
|
||||
```bash
|
||||
hermes webhook list
|
||||
```
|
||||
|
||||
If it says "Webhook platform is not enabled", set it up:
|
||||
|
||||
### Option 1: Setup wizard
|
||||
```bash
|
||||
hermes gateway setup
|
||||
```
|
||||
Follow the prompts to enable webhooks, set the port, and set a global HMAC secret.
|
||||
|
||||
### Option 2: Manual config
|
||||
Add to `~/.hermes/config.yaml`:
|
||||
```yaml
|
||||
platforms:
|
||||
webhook:
|
||||
enabled: true
|
||||
extra:
|
||||
host: "0.0.0.0"
|
||||
port: 8644
|
||||
secret: "generate-a-strong-secret-here"
|
||||
```
|
||||
|
||||
### Option 3: Environment variables
|
||||
Add to `~/.hermes/.env`:
|
||||
```bash
|
||||
WEBHOOK_ENABLED=true
|
||||
WEBHOOK_PORT=8644
|
||||
WEBHOOK_SECRET=generate-a-strong-secret-here
|
||||
```
|
||||
|
||||
After configuration, start (or restart) the gateway:
|
||||
```bash
|
||||
hermes gateway run
|
||||
# Or if using systemd:
|
||||
systemctl --user restart hermes-gateway
|
||||
```
|
||||
|
||||
Verify it's running:
|
||||
```bash
|
||||
curl http://localhost:8644/health
|
||||
```
|
||||
|
||||
## Commands
|
||||
|
||||
All management is via the `hermes webhook` CLI command:
|
||||
|
||||
### Create a subscription
|
||||
```bash
|
||||
hermes webhook subscribe <name> \
|
||||
--prompt "Prompt template with {payload.fields}" \
|
||||
--events "event1,event2" \
|
||||
--description "What this does" \
|
||||
--skills "skill1,skill2" \
|
||||
--deliver telegram \
|
||||
--deliver-chat-id "12345" \
|
||||
--secret "optional-custom-secret"
|
||||
```
|
||||
|
||||
Returns the webhook URL and HMAC secret. The user configures their service to POST to that URL.
|
||||
|
||||
### List subscriptions
|
||||
```bash
|
||||
hermes webhook list
|
||||
```
|
||||
|
||||
### Remove a subscription
|
||||
```bash
|
||||
hermes webhook remove <name>
|
||||
```
|
||||
|
||||
### Test a subscription
|
||||
```bash
|
||||
hermes webhook test <name>
|
||||
hermes webhook test <name> --payload '{"key": "value"}'
|
||||
```
|
||||
|
||||
## Prompt Templates
|
||||
|
||||
Prompts support `{dot.notation}` for accessing nested payload fields:
|
||||
|
||||
- `{issue.title}` — GitHub issue title
|
||||
- `{pull_request.user.login}` — PR author
|
||||
- `{data.object.amount}` — Stripe payment amount
|
||||
- `{sensor.temperature}` — IoT sensor reading
|
||||
|
||||
If no prompt is specified, the full JSON payload is dumped into the agent prompt.
|
||||
|
||||
## Common Patterns
|
||||
|
||||
### GitHub: new issues
|
||||
```bash
|
||||
hermes webhook subscribe github-issues \
|
||||
--events "issues" \
|
||||
--prompt "New GitHub issue #{issue.number}: {issue.title}\n\nAction: {action}\nAuthor: {issue.user.login}\nBody:\n{issue.body}\n\nPlease triage this issue." \
|
||||
--deliver telegram \
|
||||
--deliver-chat-id "-100123456789"
|
||||
```
|
||||
|
||||
Then in GitHub repo Settings → Webhooks → Add webhook:
|
||||
- Payload URL: the returned webhook_url
|
||||
- Content type: application/json
|
||||
- Secret: the returned secret
|
||||
- Events: "Issues"
|
||||
|
||||
### GitHub: PR reviews
|
||||
```bash
|
||||
hermes webhook subscribe github-prs \
|
||||
--events "pull_request" \
|
||||
--prompt "PR #{pull_request.number} {action}: {pull_request.title}\nBy: {pull_request.user.login}\nBranch: {pull_request.head.ref}\n\n{pull_request.body}" \
|
||||
--skills "github-code-review" \
|
||||
--deliver github_comment
|
||||
```
|
||||
|
||||
### Stripe: payment events
|
||||
```bash
|
||||
hermes webhook subscribe stripe-payments \
|
||||
--events "payment_intent.succeeded,payment_intent.payment_failed" \
|
||||
--prompt "Payment {data.object.status}: {data.object.amount} cents from {data.object.receipt_email}" \
|
||||
--deliver telegram \
|
||||
--deliver-chat-id "-100123456789"
|
||||
```
|
||||
|
||||
### CI/CD: build notifications
|
||||
```bash
|
||||
hermes webhook subscribe ci-builds \
|
||||
--events "pipeline" \
|
||||
--prompt "Build {object_attributes.status} on {project.name} branch {object_attributes.ref}\nCommit: {commit.message}" \
|
||||
--deliver discord \
|
||||
--deliver-chat-id "1234567890"
|
||||
```
|
||||
|
||||
### Generic monitoring alert
|
||||
```bash
|
||||
hermes webhook subscribe alerts \
|
||||
--prompt "Alert: {alert.name}\nSeverity: {alert.severity}\nMessage: {alert.message}\n\nPlease investigate and suggest remediation." \
|
||||
--deliver origin
|
||||
```
|
||||
|
||||
### Direct delivery (no agent, zero LLM cost)
|
||||
|
||||
For use cases where you just want to push a notification through to a user's chat — no reasoning, no agent loop — add `--deliver-only`. The rendered `--prompt` template becomes the literal message body and is dispatched directly to the target adapter.
|
||||
|
||||
Use this for:
|
||||
- External service push notifications (Supabase/Firebase webhooks → Telegram)
|
||||
- Monitoring alerts that should forward verbatim
|
||||
- Inter-agent pings where one agent is telling another agent's user something
|
||||
- Any webhook where an LLM round trip would be wasted effort
|
||||
|
||||
```bash
|
||||
hermes webhook subscribe antenna-matches \
|
||||
--deliver telegram \
|
||||
--deliver-chat-id "123456789" \
|
||||
--deliver-only \
|
||||
--prompt "🎉 New match: {match.user_name} matched with you!" \
|
||||
--description "Antenna match notifications"
|
||||
```
|
||||
|
||||
The POST returns `200 OK` on successful delivery, `502` on target failure — so upstream services can retry intelligently. HMAC auth, rate limits, and idempotency still apply.
|
||||
|
||||
Requires `--deliver` to be a real target (telegram, discord, slack, github_comment, etc.) — `--deliver log` is rejected because log-only direct delivery is pointless.
|
||||
|
||||
## Security
|
||||
|
||||
- Each subscription gets an auto-generated HMAC-SHA256 secret (or provide your own with `--secret`)
|
||||
- The webhook adapter validates signatures on every incoming POST
|
||||
- Static routes from config.yaml cannot be overwritten by dynamic subscriptions
|
||||
- Subscriptions persist to `~/.hermes/webhook_subscriptions.json`
|
||||
|
||||
## How It Works
|
||||
|
||||
1. `hermes webhook subscribe` writes to `~/.hermes/webhook_subscriptions.json`
|
||||
2. The webhook adapter hot-reloads this file on each incoming request (mtime-gated, negligible overhead)
|
||||
3. When a POST arrives matching a route, the adapter formats the prompt and triggers an agent run
|
||||
4. The agent's response is delivered to the configured target (Telegram, Discord, GitHub comment, etc.)
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
If webhooks aren't working:
|
||||
|
||||
1. **Is the gateway running?** Check with `systemctl --user status hermes-gateway` or `ps aux | grep gateway`
|
||||
2. **Is the webhook server listening?** `curl http://localhost:8644/health` should return `{"status": "ok"}`
|
||||
3. **Check gateway logs:** `grep webhook ~/.hermes/logs/gateway.log | tail -20`
|
||||
4. **Signature mismatch?** Verify the secret in your service matches the one from `hermes webhook list`. GitHub sends `X-Hub-Signature-256`, GitLab sends `X-Gitlab-Token`.
|
||||
5. **Firewall/NAT?** The webhook URL must be reachable from the service. For local development, use a tunnel (ngrok, cloudflared).
|
||||
6. **Wrong event type?** Check `--events` filter matches what the service sends. Use `hermes webhook test <name>` to verify the route works.
|
||||
Reference in New Issue
Block a user