Rule catalog · Metadata transparency

Existing tool descriptions are unchanged since the previous scan

tool_descriptions_unchanged_since_last_scanmediumweight 4Post-handshake

Authored by Stanley Hong · AgentReserve (founder).

For every tool whose name appears in both this scan and the most recent prior scan, the `description` field is byte-identical. A description rewrite with no name change is the canonical rug-pull camouflage: the existing approval still routes calls to the same tool name, but the documented behavior — and therefore the model's interpretation of the call — has shifted underneath. Silent on first scans.

When this rule runs

Requires a successful MCP `initialize` / `tools/list`. Skipped on perimeter-only scans where the server refused or failed the MCP handshake.

Why it matters

Tool descriptions reach the model verbatim on every call. A description that quietly grows an exfiltration directive or an identity-hijack phrase changes what the agent does without the operator approving the new copy. Naming the changed tools gives the operator a tight diff to review rather than 'something drifted'.

Pass condition

Every tool present in both scans has a byte-identical `description` (or both have no description).

Fail condition

At least one tool present in both scans has a different `description` between scans.

Evidence examples

When the rule fails, the report records evidence in roughly this shape:

  • {"changedDescriptions": [{"toolName": "summarize"}]}

Remediation

Treat description rewrites as a re-review event. Diff the prior and current text, validate the change against an operator-known release, and re-run the description-injection / tool-poisoning checks against the new copy before re-trusting the surface.

Methodology

This rule belongs to the Metadata transparency dimension. Whether the server identifies itself and documents its tools — and whether the advertised identity matches the wire identity (cert CN/SAN, hostname). Operators need a stable name, a version, and an internally consistent identity claim to perform any kind of audit.

Read the full methodology for how rules are aggregated into a score, how verdicts are decided, and how hard-fail rules override the aggregate.