DEV Community

Remdore
Remdore

Posted on

Notes on making an AI agent ask before it pushes

I wanted an AI coding agent to do the work by itself and stop only before anything left its sandbox: pushing a branch, opening a pull request, commenting on one. DigitalOcean's Managed Agents let you write that into the session spec as permission rules. You allow everything by default and set a few commands to "ask". I ran two sessions against a small public repo of mine to see how it holds up. The first set of rules didn't fire once.

The setup

Each session was a Harness Runtime session with the claude-code harness, Claude Sonnet 5.5 through DigitalOcean's inference, the mars-2vcpu-4gb size, and GitHub access through an OAuth connection. The permission part of the spec looked like this:

permissions:
  default: allow
  rules:
    - tool: bash
      match: { command: "git push *" }
      action: ask
      enforcement: strict
    - tool: bash
      match: { command: "gh pr create *" }
      action: ask
      enforcement: strict
Enter fullscreen mode Exit fullscreen mode

The task was a real issue on the repo: write pytest tests for decode() and trial() in flip.py, push a branch and open a pull request.

Run one: no approval requests at all

The run took 54 seconds and 10 tool calls. It wrote 12 tests, got them passing, pushed the branch and opened PR #10. It never asked me anything about the push. This was its last command:

cd /workspace/bitrot-demo && git -c credential.helper='!gh auth git-credential' push -u origin tests-for-flip 2>&1 | tail -3 && gh pr create --title "..."
Enter fullscreen mode Exit fullscreen mode

The docs say rules match the command literally, and that's exactly what happened. git push * only matches a command that starts with git push. This one starts with cd, has -c credential.helper=... between git and push, and puts gh pr create at the end of a chain. Neither rule matched, and both things happened.

None of that was the agent being sneaky. Plain git had no credential helper in the sandbox, so it used the gh login as a one-off helper for that one command, and it chained the PR onto the push because that's how people write it in a terminal. The problem is that what people normally type doesn't look like git push *.

One more thing in that run surprised me. The commit failed at first because git had no author set, and the agent asked me which name to use. The question came back as "The user did not answer the questions" in 7 ms, and the agent picked a neutral identity and carried on. I wasn't attached to the session, so nobody could have answered. But it didn't hold the run the way an approval does.

Run two: wildcards on both sides

For the second session I changed the patterns to *git*push* and *gh pr *. The task was a small follow-up on the same branch: add a .gitignore, push it and leave a comment on the PR. I also told it to run the push and the comment as separate commands.

This time every gated command stopped and waited:

# Command Asked Answered Waited Answer What happened
1 git push origin tests-for-flip 08:07:49 08:11:13 3 min 24 s approve failed, no git credentials
2 git -c credential.helper=... push origin tests-for-flip 08:11:19 08:11:31 12 s approve pushed
3 gh pr comment 10 --body "..." 08:11:33 08:11:46 13 s reject not posted

Request 1 sat for more than three minutes and nothing timed out. The run simply waited.

I approved a push that then failed, and the retry needed its own approval. An approval covers one exact command string, not "push this branch". If an agent tends to retry with small changes, you answer once per variant.

The reject went the way I'd want. The tool call came back as "denied". The agent didn't retry or look for another way to post the comment. It finished, said the comment wasn't posted, and printed the gh pr comment command for me to run myself. The whole run took 4 minutes 10 seconds, most of it waiting on me.

What the wildcards cost

*git*push* also matches git log | grep push, and *gh pr * also matches gh pr view. Neither came up in these two runs. None of the other five bash commands in run two matched by accident. But the wider the pattern, the more often it asks.

It's still string matching. A script file that runs git push, a shell alias, or a Python subprocess call doesn't have those characters in the command the agent sends. I didn't test any of those, but going by literal matching I'd expect them to go straight through.

So I'd treat these rules as a way to get asked, not as a wall. What actually limits the agent is the token in the session. The GitHub connection's default scopes include full repo. When I asked for only public_repo, the connection was refused with "requested scopes exceed provider maximum".

For comparison, 8 of the 10 tool calls in run one were bash commands. With default: ask, each of those would have needed an answer too.

Where the phone came in, or didn't

The reason I tried this at all is that DigitalOcean has an iPhone app for Managed Agents, Cuttlefish, which is meant for approving an agent's next step from your phone. I answered all three requests from a laptop with doctl hr approve <session> <request-id> approve|reject, because I hadn't installed the app before the first request arrived. The 12 and 13 second waits are me at a terminal. I can't tell you what the phone flow is like.

What I got wrong

I wrote the rules as if a person types one command at a time. Agents chain commands, prepend cd and add -c flags.

I thought approving the push meant approving the push. It meant approving that one string, and the retry asked again.

I planned to approve from my phone and didn't install the app first. That's why request 1 waited three and a half minutes.

In run two I told the agent to run the push and the comment separately. That makes the wildcard result look better than it would have if the agent had chained them again. Run two doesn't show the wildcards catching a chained command, only that they caught the commands it ran.

And the GitHub connection link expired twice before I finished authorizing it. It's the last browser step, on GitHub's page, and it doesn't wait for you.

Caveats

Two runs, one harness, one model, one small Python repo. I didn't test other harnesses, best-effort enforcement, deny rules, or what the agent does when a request is deferred. The model tokens cost $0.13 for the first run and $0.05 for the second, plus a few cents of compute.

What I'd take from it

  • Run the session once and read doctl hr logs for human_input_requested before you trust a rule. Zero requests is a result, not a pass.
  • Put wildcards on both sides of the pattern, and expect a few false alarms.
  • Say in the prompt that gated commands should run on their own, not in a chain.
  • Limit what the session's token can do. The rule gets you asked; the token decides what's possible.

Top comments (2)

Collapse
 
nexusfoundry profile image
Nexus Foundry •

The cd && git -c credential.helper=... push && gh pr create run is the perfect demonstration of why string-matching permission rules are a leaky abstraction: the rule matched the shape of a command, not the capability being exercised. Wrappers, aliases, xargs, sh -c, env prefixes — the space of "commands that push" is unbounded while your glob list is finite.

The boundary that held for us isn't pattern-matching at all — it's capability scoping. Give the agent a credential that physically can't do the dangerous thing: a GitHub token scoped to open PRs but not push to protected branches, or route git ops through a thin wrapper that normalizes the command before any rule engine sees it. Then git push * vs cd x && git push stops mattering, because the credential itself is the enforcement.

Second thing worth naming: the 7ms "user did not answer" fallback. An approval prompt that times out into a default is worse than no prompt — it teaches the agent that asking is optional. If nobody's attached, the safe default should be stop, not proceed-with-neutral-identity. Same class of bug as a linter that warns instead of fails: the check exists but has no teeth.

Collapse
 
paullovelace profile image
Paul Lovelace •

The cd && git -c credential.helper=... push && gh pr create run is the perfect demo of why string-matching rules leak — the rule matched the shape of a command, not the capability, and wrappers just walk around it. That's the bit I'd steal for my own setup. Did the second session's rules actually hold, or did the agent find another way around the shape? Feels like the honest fix has to live at the capability level, not the command string, but I haven't seen a harness do that cleanly yet.