Updated 8 October 2026

How to make a coding agent keep going until the tests pass

Run the agent headless, once per attempt, in a shell loop. After each attempt run your test command, stop when it exits 0, and stop anyway after a fixed number of attempts so a bad prompt cannot run all night. Each attempt starts a fresh session, so the files on disk are the agent's only memory.

This page gives you that loop in bash, then says what hard caps and a review step add on top of it, then shows where Agency does those parts for you. Disclosure: Agency is ours, and this site is its home.

Why a loop works, and where it goes wrong

An interactive agent session does not exit when the task is done: it idles at its prompt, waiting for you. Most agent CLIs also have a headless one-shot mode that does the work and exits (for example claude -p "<prompt>" and codex exec "<prompt>"). A loop calls that mode again and again with the same prompt, and the agent picks up where the last attempt stopped by reading the repository.

Two things decide whether this is useful. The first is the check: a command that exits 0 only when the work is really done, such as your test suite, a linter or a build. An agent saying it is finished is not a check. The second is a limit, because a loop that never passes its check spends money until you notice.

The loop by hand

Do this on a branch or in a git worktree, not in a checkout with work you care about: the loop commits after every attempt. Put the task in a file called PROMPT.md, and tell the agent to read and update a progress note, because a fresh session remembers nothing else.

Write one prompt that works on a fresh session

The same text is sent every attempt, so it has to say where to look. A shape that works:

Read PROGRESS.md if it exists. Pick the single most important unfinished
piece of: make the failing tests in tests/ pass. Implement it, run the tests,
commit with a clear message, and append one line to PROGRESS.md describing
what you did.

Loop with an attempt cap and timeouts

Save this as loop.sh. It runs at most 10 attempts, gives the agent 30 minutes per attempt and the tests 10 minutes per check, and commits whatever the agent left behind so each attempt is a visible step in git log. Swap npm test for your own check. The --permission-mode acceptEdits flag lets Claude Code edit files without asking; read the permissions section below before you copy it.
```
#!/usr/bin/env bash
set -u
max=10
for attempt in $(seq 1 "$max"); do
  echo "attempt $attempt of $max"
  timeout 1800 claude -p "$(cat PROMPT.md)" --permission-mode acceptEdits \
    || echo "agent exited nonzero"
  if [ -n "$(git status --porcelain)" ]; then
    git add -A && git commit -qm "wip: loop attempt $attempt"
  fi
  if timeout 600 npm test; then
    echo "tests pass on attempt $attempt"
    exit 0
  fi
done
echo "gave up after $max attempts, tests still failing"
exit 1
```
timeout is part of GNU coreutils. It is on Linux already; on macOS install coreutils with Homebrew and use gtimeout.
Run it, then read the diff
```
bash loop.sh
git log --oneline main..HEAD
git diff main...HEAD
```
A green check means the tests pass, not that the change is good. An agent can make a test pass by weakening the test. Read the diff before you merge.

What the bash version does not give you

A cap on total time or spend. Ten attempts of 30 minutes is five hours, and the script has no idea how many tokens they used
A reason when it stops. "Gave up" could mean the prompt is wrong, the tests are flaky, or the agent crashed on a bad flag ten times in a row
Crash detection. If the agent exits nonzero every time, the loop burns all its attempts on a problem the agent never got to touch
A way to see progress without staring at a terminal, or an alert when it ends
A review step. The loop's output is a branch, and merging it is still manual

Which limits to set

Set all three if you can: attempts, wall-clock time and tokens. Attempts alone is not enough, because an attempt has no fixed size. One can take ten minutes, another three hours. Whichever limit you use, check it between attempts rather than killing an attempt midway: a half-finished edit is worse than a late stop. Ten attempts is a reasonable starting point for a task you would otherwise leave running overnight; if it needs more, the prompt usually needs to be better first.

Permissions: a loop has no one to approve prompts

A headless attempt cannot stop and ask you. If the agent hits a permission prompt it cannot answer, the attempt dies or hangs, so loops need the agent's own non-interactive permission flags, such as --permission-mode acceptEdits for Claude Code. Choose them deliberately, and run the loop in a worktree or branch so the damage a wrong edit can do stays in one place.

Doing the same with Agency

Agency is a free, open-source desktop app for macOS and Linux. Its Loop agent dialog takes a prompt, an agent, a "done when" command and the limits, and runs the loop above in a run's own worktree and branch. As of October 2026 the numbers are:

Part of the loop	By hand	In Agency
Each attempt	`claude -p` in a loop	A fresh headless session in the run's worktree, same prompt every time; output streams into the run's terminal
The check	`timeout 600 npm test`	Your "done when" command, run through your shell in the worktree; exit 0 ends the loop. It times out after 600 seconds, and a timeout counts as a failed check
Attempt cap	`max=10`	Default 10, between 1 and 100
Time cap	Per-attempt `timeout` only	Optional, in minutes, from 1 minute to 1 week; off unless you set it
Token cap	None	Optional, from 1,000 to 1 billion tokens, read from the agent's own transcript; off unless you set it
Crashes	Not detected	Three nonzero agent exits in a row stall the loop, and crashed attempts do not use up the attempt count
Commits between attempts	`git commit` in the script	Leftover changes are committed as `wip: loop attempt N`
Progress	Terminal output	A strip showing `attempt 3/10 · running` or `checking`, a Stop loop button that keeps the worktree and its commits, and a desktop notification when the loop completes or stalls
Review and merge	`git diff`, `git merge`	The loop ends on a branch; the diff review and merge are the same as for any run, and nothing merges automatically

The four reasons a loop stalls

A stalled loop says why, so you know whether to raise a cap, fix a flag or rewrite the prompt:

Attempt cap: the check was still failing after the last allowed attempt. Usually a prompt or task problem
Agent failed repeatedly: three crashes in a row, which usually means a bad flag, an expired login or a rate limit
Time cap hit: the wall-clock limit you set ran out
Token cap hit: the token total reached the limit you set

Caps are checked only when the loop would start another attempt, so a running attempt is never killed, and a cap can trip no earlier than the next attempt boundary. A finished attempt always gets its check: if the tests pass, the loop completes even if a cap has just been passed. The token cap needs a transcript Agency can read; for an agent whose transcript it cannot read, the token cap never trips rather than guessing.

The permission gate stays visible

Agency does not turn on autonomy for you behind your back. The headless recipe for each agent is a profile setting you can read and change under Settings, as "Loop arguments". As of October 2026 the built-in recipes carry these permission flags: Claude Code gets --permission-mode acceptEdits, Codex gets --full-auto and GitHub Copilot CLI gets --allow-all-tools. The others run with their plain headless command. Those flags are what let an unattended attempt run at all, and each attempt runs in the run's own worktree, so a bad edit lands on a branch you review rather than on your main checkout.

Who should not use it: anyone on Windows or an Intel Mac, which Agency does not support yet, and anyone whose loop already works as the bash script above and who does not need a time or token cap. Agents without a headless one-shot mode cannot loop in Agency at all.

Questions

What if I do not have a test command?

Leave the "done when" field empty and Agency runs exactly the number of attempts you set, without checking anything. It labels this as fixed iterations, which suits tasks such as "find and fix one lint warning". It is not verified completion.

Why a fresh session for each attempt?

A long session fills its context with old output and starts to lose the thread. A fresh session per attempt avoids that, and the repository, the git log and your progress file carry the state instead.

Can I loop an agent and also race several agents?

Yes, as separate runs. A race sends one prompt to several agents so you can keep the best branch; a loop pushes one agent toward a passing check. See how to run coding agents in parallel for the worktree side.

Download Agency for macOS and Linux What Agency does