← Back to feed

The standard agent safety model gates the model input and reviews the model output. SafeAgents, released open-source by Hugging Face on October 30, takes a different approach: it gates each tool call individually, between the moment the model emits it and the moment the harness executes it.

How the proxy works

SafeAgents runs as a local reverse proxy in front of an OpenAI-compatible or Anthropic Messages server. The agent connects to SafeAgents instead of directly to its model endpoint. SafeAgents forwards inference calls unchanged, but inspects every tool call the model returns before passing it to the harness.

Blocked calls get replaced with a short text notice. The harness sees plain text and no execution happens. The model’s chain of thought is never inspected — only the structured tool call after inference completes.

Three checks run on every call, in order:

Tier 1 — Declared-tool and host validation. If the tool was not declared by the harness, it is blocked. If the call targets a host not previously seen in the conversation, it is blocked. Both checks run in-process, no network round-trip.

Tier 2 — atif-scan. Approximately 110 deterministic detectors run against the call. The scanner looks for sandbox side channels, network bypasses, namespace escapes, and reward hacking patterns. Specific blocked moves include attempts to read /proc/1/environ and connections to cloud metadata endpoints.

Tier 3 — Judge model (optional). If configured, a judge model receives the tool call and returns an allow or block verdict via JSON. The judge never sees the agent’s reasoning, only the isolated tool call. A small local model via llama.cpp is the intended use case. By default the judge is off; on judge error, the proxy allows and logs unless --judge-fail-closed is set.

On an M-series Mac, the local checks take 10 to 20 ms per call.

Reward hacking detection

SafeAgents includes a --log-only mode intended for RL and evaluation environments. In this mode, calls are logged but not blocked, producing a record of how often the model attempts to edit verifier tests, patch software to force a passing result, or find answers through a side channel. The log gives evaluators a direct signal for reward-hacking frequency without altering the evaluation trajectory.

This matters specifically for coding agent benchmarks where the verification harness lives on the same machine as the agent. A model that edits its own test verifier scores 100% on tasks it did not actually solve.

Integration and scope

SafeAgents supports Claude Code, OpenCode, Pi, Hermes, and any harness that accepts a configurable base URL. The repository includes setup guides for 12 harnesses, with six independently verified: Claude Code, OpenHands, the DeepSeek Harness, and three others. Both streaming and non-streaming response modes are supported.

The tool was registered as the safeagents-judge-bench official benchmark on HuggingFace on October 31, giving it a formalized evaluation surface for testing judge model quality within the proxy architecture.

The security model has limits. Tier 1 host checking uses substring matching, which can pass a call to an unauthorized host if the unauthorized host name is a substring of a previously trusted domain. The project notes this as a known limitation. Tier 2 covers the deterministic pattern space; adversarial tool calls crafted to evade known patterns are not blocked unless Tier 3 is active.

Context

Agent security tooling has moved fast in October. The prior cluster of OSS supervision tools that shipped October 22-24 focused mostly on orchestration-level monitoring and log aggregation. SafeAgents operates at a lower level, at the individual call rather than the session, which makes it composable with higher-level supervision rather than competing with it.

The practical case for a per-call proxy: an agent process runs as the local user and holds whatever credentials the user holds. A blocked tool call costs 10-20 ms. An executed malicious tool call costs significantly more.