Agent Harness (Experimental)
Both features on this page are experimental and off by default. They may change or be removed in a future release. With their flags off, nothing on this page affects your runs.
A harness is everything around the model: the loop that calls it, the context it sees, the tools it may use, and the instructions, skills and checks it gets. Shep has two experimental features here:
| Query-aware harness | Self-learning harness | |
|---|---|---|
| What it is | Shep’s own agent loop, offered as the Shep Harness agent | Learns per-repository instructions, skills, scripts and checks from your feature runs |
| Works with | SDK backends: OpenRouter, Together AI, Ollama, LLM Proxy | Every agent |
| Flag | queryAwareHarness | selfLearning |
| Web | /harness, the feature Context tab, Settings → Agent Harness | /learning, Settings → Self-learning |
| CLI | shep harness … | shep learn …, shep feat new --explore-harness |
Query-aware harness (Shep Harness)
Turn it on
shep settings flags enable queryAwareHarnessOr Settings → Feature flags → Query-aware agent harness. This adds the Shep Harness agent to the agent pickers, the Harness page, the feature Context tab, and the shep harness commands (hidden from shep --help while the flag is off).
Then choose its backend in Settings → Agent Harness: an SDK agent (OpenRouter, Together AI, Ollama or LLM Proxy) and, optionally, a model. The backend’s key comes from that agent’s token or its environment variable (OPENROUTER_API_KEY, TOGETHER_API_KEY, …).
What’s different
A regular agent keeps a growing transcript. The Shep Harness instead stores tool output, file reads and prompt sections as separate pieces and, before every model call, decides how much of each piece this step needs: hidden, a short view, a longer view, or the full text. That decision (the context plan) is saved, so you can see in the feature’s Context tab exactly what the model saw on each turn and why.
Before a tool runs, the harness checks it against permission rules (deny wins over ask, ask over allow). An “ask” waits for you in the web UI or the terminal; in non-interactive runs it counts as a deny. You can grant once, for a task, or for the session.
On small tasks it can use slightly more input tokens than a plain transcript. It saves the most on long tasks with large tool output.
Commands
shep harness init # set the repository up (writes only under .shep/harness/)
shep harness run "Fix the flaky date test" --non-interactive
shep harness ls # sessions
shep harness inspect session <id> # tasks, turns and permission log
shep harness inspect context <id> # what the model saw on one turn (a plan id, or a task id with --turn)
shep harness apply <id> # commit the result to a branch (nothing is pushed)
shep harness promote <id> # continue the result as a regular feature
shep harness discard <id>shep harness run takes --repo, --mode, --budget, --max-turns, --model, --test-command, --non-interactive and --json. Other subcommands: resume, stop, view, explain, permissions ls|allow|deny, capabilities, policies, and eval run|ls|report|save to compare the query-aware loop with a plain baseline on the same tasks. Run shep harness <command> --help for details.
Self-learning harness
The self-learning harness watches how features go in a repository and proposes improvements to what the agents get there: always-on instructions and guardrails, project memory, skills, scripts, subagents and (for Claude Code) hooks. Every proposal is a version you can review, adopt, compare and roll back.
Turn it on
It’s opt-in at every level:
| Level | Default | Change it |
|---|---|---|
selfLearning flag | off | shep settings flags enable selfLearning, or Settings → Feature flags |
| Learning mode | Off | Settings → Self-learning: Off, Reflect or Explore |
| Explore for one feature | off | shep feat new --explore-harness, the Explore harness variants option in the create drawer, or the new-feature view in shep tui |
| Adoption policy | Ask me every time | Settings → Self-learning |
With the flag on and the mode Off, nothing is learned automatically, but versions you already adopted still apply and you can still run shep learn reflect by hand.
Modes and cost
- Reflect runs one review after each feature run, completed or failed, and proposes changes. No extra implementation attempts.
- Explore has the implement step try several harness variants, each in its own throwaway worktree checked with your test command, then keeps the best result without knowing which variant produced it. With k = 1 that’s 4 implementation attempts per feature, about 4× the cost of one implement step. k goes up to 5. A cost cap (default $20) stops new attempts once reached and keeps the best one so far.
shep feat new "Add CSV export" --explore-harness # k = 1
shep feat new "Add CSV export" --explore-harness --explore-k 2Safety
- Scripts and hooks are never adopted automatically, whatever the policy.
- A proposal that does anything risky (network fetches, destructive commands, reading credentials, …) is never auto-adopted, and the findings are shown when you review it.
- Learned files are installed with a
shep-learned-prefix or under.shep/learned/, and listed in.git/info/exclude, so they’re never committed to your repository. - A version that does worse than its parent over enough runs is flagged Regressed.
Commands
shep learn status # the learned harness this repository runs on
shep learn history -n 10 # versions with their outcomes
shep learn show <version>
shep learn diff <a> <b>
shep learn adopt [<version>] # or --from-dir <dir> to import a hand-written harness
shep learn reject <version> --feedback "Too broad; keep the lint rule only"
shep learn rollback --reason "Slower CI fixes since v3"
shep learn reflect <feature-id> # review one finished run now, even while the mode is OffEach takes -r, --repo <path> (default: the folder you’re in) and most take --json. Version ids can be shortened to a unique prefix. While the flag is off, shep learn prints how to turn it on.
You can also review proposals on the Learning page, in the agent-questions inbox, and in shep tui (press d).
What’s next
- Feature flags — every flag and its default.
- Agents — the SDK backends the Shep Harness runs on.
- Usage metrics & privacy — the content-free
harness.*events.