Skip to content
AI-generated research idea🟡Fairly easy💡Moderate InnovationNo close match found

Automated Tool-Schema Inference for Zero-Config Typed-Identity Egress Guards

Infer tool side-effect categories and target fields automatically from tool schemas so structural prompt-injection guards generalize to new agent ecosystems without manual rules.

Time

several weeks

Compute level

Single GPU or less

Resources

One standard or high-VRAM GPU for local LLM schema analysis, OpenAI API access, and public MCP Security Bench (MSB) and AgentDyn benchmarks.

What the paper used

One NVIDIA RTX PRO 6000 Blackwell GPU for local Ollama models (qwen3:8b, qwen3-coder:30b, gemma4:31b Q4_K_M, gemma4:12b 4-bit) plus OpenAI API for gpt-4o-mini-2024-07-18; evaluated across 949 AgentDojo v1.2.2 pairs per backbone, 560 AgentDyn pairs, 665 ChatInject pairs, 389+288 AutoDojo pairs, and 949 PISmith pairs.

Prerequisites

Program analysis / AST inspection, JSON Schema and Model Context Protocol specifications, LLM tool-use security.

Scope

modify existing code

The Gap

In Section 3.1, Section 7, and Appendix B, SPILLWAY's typed-identity index I and effect classification depend on a manually curated per-benchmark role lexicon comprising fixed verb prefixes (11 read-only, 22 writing, 10 destructive, and 11 granting prefixes) and explicit target-argument names (19 for AgentDojo plus 6 for AgentDyn).

“Target roles come from a lexicon written once per benchmark, so a new tool set needs its roles declared.”
From the paper, Section 7, page 10. Matched against the PDF text.

Limitation

Deploying the defense onto arbitrary real-world tool ecosystems such as diverse Model Context Protocol (MCP) servers requires manual security engineering for each new tool set; misclassified side-effecting reads or unannotated nested target arguments silently bypass the floor or block legitimate workflows.

Proposed Solution

Develop an offline, one-time tool-registration analyzer that inspects JSON schemas, OpenAPI/MCP tool definitions, docstrings, and (where source is available) Python AST or HTTP method semantics using structured static analysis backed by a schema-verification LLM with formal consistency checks. Automatically synthesize the read/write/destructive/grant classification and nested JSONPath target/principal extractors at server onboarding time, failing closed on ambiguous tools until verified.

Why This Matters

Automating role lexicon generation would make typed-identity egress enforcement practical for dynamic, heterogeneous MCP deployments without per-app manual configuration.

How to Get Started

How to test it: Test whether automated schema and AST-assisted lexicon synthesis matches 100% of the ground-truth security guarantees (0.000 in-scope ASR) and achieves within 1% of the manual-lexicon utility across held-out AgentDyn and MCP Security Bench tool sets. Baseline is SPILLWAY's hand-written lexicon and naive verb-prefix matching; stop if automated classification mislabels any destructive or granting tool as read-only on held-out MCP servers.

  1. Evaluate SPILLWAY's existing AgentDojo verb-prefix lexicon zero-shot (without the 6 added AgentDyn argument rules) on AgentDyn and MCP Security Bench to quantify coverage gaps.
  2. Build a static schema and docstring extractor for MCP/JSON tool definitions that identifies nested identity types, HTTP/side-effect verbs, and target/principal parameters.
  3. Augment static extraction with a one-time, fail-closed LLM verification pass at tool-registration time and compile the output into SPILLWAY's lexicon format.
  4. Measure classification precision/recall against the manual AgentDojo/AgentDyn lexicons and run end-to-end UAU and ASR evaluations on MCP Security Bench.

Suggested by AI. Adapt it to your setup.

Has This Been Done?

No close match in 10 results

Papers such as Model Context Protocol Threat Modeling and Enterprise-Grade Security for the Model Context Protocol analyze security threats and mitigations within ecosystems like MCP. They do not propose an offline, one-time tool-registration analyzer that uses static analysis and schema-verification LLMs to automatically synthesize JSONPath targets and security classifications.

We searched OpenAlex and arXiv for “Model Context Protocol tool schema static analysis security policy generation” and “Automated Tool-Schema Inference for Zero-Config Typed-Identity Egress Guards” and an AI compared the idea with the top 10 results. A small search can miss relevant work.

Source Paper

SPILLWAY: Separating Target Admission from Sanitization to Defend Tool-Using Agents against Prompt Injection

Woosang Lim, Yeseul Chang, Jaewoo Lee · 2026

This paper introduces SPILLWAY, a runtime defense against indirect prompt injection for tool-using LLM agents that separates target admission from content sanitization. Rather than invoking a language model on every step or requiring a per-task planner, it enforces a deterministic egress floor using typed tool return schemas while using a rule-based ingress sanitizer to strip instructions and constrain a local recovery judge. Across three backbones and seven suites on AgentDojo and AgentDyn, SPILLWAY achieves zero in-scope attack success under standard and forged-template attacks and 0.005 under a reinforcement-learning attacker, with higher utility and far fewer model calls than existing defenses.

Related Ideas

Is this a good idea?

Share onXLinkedIn