Skip to Content
GuidesAgent Harness (Experimental)

Agent Harness (Experimental)

Both features on this page are experimental and off by default. They may change or be removed in a future release. With their flags off, nothing on this page affects your runs.

A harness is everything around the model: the loop that calls it, the context it sees, the tools it may use, and the instructions, skills and checks it gets. Shep has two experimental features here:

Query-aware harnessSelf-learning harness
What it isShep’s own agent loop, offered as the Shep Harness agentLearns per-repository instructions, skills, scripts and checks from your feature runs
Works withSDK backends: OpenRouter, Together AI, Ollama, LLM ProxyEvery agent
FlagqueryAwareHarnessselfLearning
Web/harness, the feature Context tab, Settings → Agent Harness/learning, Settings → Self-learning
CLIshep harness …shep learn …, shep feat new --explore-harness

Query-aware harness (Shep Harness)

Turn it on

shep settings flags enable queryAwareHarness

Or Settings → Feature flags → Query-aware agent harness. This adds the Shep Harness agent to the agent pickers, the Harness page, the feature Context tab, and the shep harness commands (hidden from shep --help while the flag is off).

Then choose its backend in Settings → Agent Harness: an SDK agent (OpenRouter, Together AI, Ollama or LLM Proxy) and, optionally, a model. The backend’s key comes from that agent’s token or its environment variable (OPENROUTER_API_KEY, TOGETHER_API_KEY, …).

What’s different

A regular agent keeps a growing transcript. The Shep Harness instead stores tool output, file reads and prompt sections as separate pieces and, before every model call, decides how much of each piece this step needs: hidden, a short view, a longer view, or the full text. That decision (the context plan) is saved, so you can see in the feature’s Context tab exactly what the model saw on each turn and why.

Before a tool runs, the harness checks it against permission rules (deny wins over ask, ask over allow). An “ask” waits for you in the web UI or the terminal; in non-interactive runs it counts as a deny. You can grant once, for a task, or for the session.

On small tasks it can use slightly more input tokens than a plain transcript. It saves the most on long tasks with large tool output.

Commands

shep harness init # set the repository up (writes only under .shep/harness/) shep harness run "Fix the flaky date test" --non-interactive shep harness ls # sessions shep harness inspect session <id> # tasks, turns and permission log shep harness inspect context <id> # what the model saw on one turn (a plan id, or a task id with --turn) shep harness apply <id> # commit the result to a branch (nothing is pushed) shep harness promote <id> # continue the result as a regular feature shep harness discard <id>

shep harness run takes --repo, --mode, --budget, --max-turns, --model, --test-command, --non-interactive and --json. Other subcommands: resume, stop, view, explain, permissions ls|allow|deny, capabilities, policies, and eval run|ls|report|save to compare the query-aware loop with a plain baseline on the same tasks. Run shep harness <command> --help for details.

Self-learning harness

The self-learning harness watches how features go in a repository and proposes improvements to what the agents get there: always-on instructions and guardrails, project memory, skills, scripts, subagents and (for Claude Code) hooks. Every proposal is a version you can review, adopt, compare and roll back.

Turn it on

It’s opt-in at every level:

LevelDefaultChange it
selfLearning flagoffshep settings flags enable selfLearning, or Settings → Feature flags
Learning modeOffSettings → Self-learning: Off, Reflect or Explore
Explore for one featureoffshep feat new --explore-harness, the Explore harness variants option in the create drawer, or the new-feature view in shep tui
Adoption policyAsk me every timeSettings → Self-learning

With the flag on and the mode Off, nothing is learned automatically, but versions you already adopted still apply and you can still run shep learn reflect by hand.

Modes and cost

  • Reflect runs one review after each feature run, completed or failed, and proposes changes. No extra implementation attempts.
  • Explore has the implement step try several harness variants, each in its own throwaway worktree checked with your test command, then keeps the best result without knowing which variant produced it. With k = 1 that’s 4 implementation attempts per feature, about 4× the cost of one implement step. k goes up to 5. A cost cap (default $20) stops new attempts once reached and keeps the best one so far.
shep feat new "Add CSV export" --explore-harness # k = 1 shep feat new "Add CSV export" --explore-harness --explore-k 2

Safety

  • Scripts and hooks are never adopted automatically, whatever the policy.
  • A proposal that does anything risky (network fetches, destructive commands, reading credentials, …) is never auto-adopted, and the findings are shown when you review it.
  • Learned files are installed with a shep-learned- prefix or under .shep/learned/, and listed in .git/info/exclude, so they’re never committed to your repository.
  • A version that does worse than its parent over enough runs is flagged Regressed.

Commands

shep learn status # the learned harness this repository runs on shep learn history -n 10 # versions with their outcomes shep learn show <version> shep learn diff <a> <b> shep learn adopt [<version>] # or --from-dir <dir> to import a hand-written harness shep learn reject <version> --feedback "Too broad; keep the lint rule only" shep learn rollback --reason "Slower CI fixes since v3" shep learn reflect <feature-id> # review one finished run now, even while the mode is Off

Each takes -r, --repo <path> (default: the folder you’re in) and most take --json. Version ids can be shortened to a unique prefix. While the flag is off, shep learn prints how to turn it on.

You can also review proposals on the Learning page, in the agent-questions inbox, and in shep tui (press d).

What’s next