Cross-Repository Micro-Skill Generalization via Project Manifest Embeddings
Implement a hierarchical skill bank combining global ecosystem skills with repository-specific overrides by embedding project setup manifests from public GitHub repositories.
Time
Compute level
Resources
What the paper used
Prerequisites
Scope
The Gap
In Section 4.3 and Appendix C, micro-skill injection was completely disabled for the security auditing benchmarks (CredData and ReposVul) because tasks did not share a recurring repository. Similarly, in SWE-Bench (Section 3.1), micro-skill state Si is strictly isolated per repository, preventing any transfer of operational conventions across codebases.
“TokenEconomizer begins each repository without prior traces, so its benefit may take several tasks to emerge.”
Limitation
Micro-skills cannot transfer across projects. High-utility operational facts (such as framework-standard test commands, virtual environment activations, or linter behaviors) must be relearned from scratch in every new repository, preventing any cost savings during the first 4-9 tasks of a stream.
Proposed Solution
Implement a hierarchical skill bank combining global ecosystem-level skills with repo-level overrides. Represent project environments by embedding setup manifests (e.g., pyproject.toml, package.json, setup.cfg, and Dockerfiles) using a dense code embedding model. Before task 1 of an unseen codebase, retrieve top-k candidate micro-skills from structurally similar repositories and evaluate their scope detectors in cold-start dry runs or initial environment probing steps.
The paper’s authors list this direction as future work, so they may already be pursuing it.
Why This Matters
Enabling transfer learning across codebases eliminates the warm-up tax observed in Figure 5b, where TokenEconomizer is initially more expensive than default execution for the first 5 tasks, thereby broadening applicability to single-task or multi-repo development pipelines.
How to Get Started
How to test it: Hypothesis: retrieving candidate micro-skills via embedded project manifests (pyproject.toml, package.json, setup.cfg) and validating their scope in cold-start runs reduces early-stream token costs without degrading accuracy compared to cold-start TokenEconomizer and default execution. Measure cumulative cache-aware cost over tasks 1-5 and full-stream mean cost alongside Pass@1 on SWE-Bench (and F1 on CredData/ReposVul); stop if early-stream or overall cost savings fail to beat standard TokenEconomizer or if Pass@1 drops by more than 1.5 percentage points.
- Scrape setup manifests including pyproject.toml, package.json, and Dockerfiles from a diverse set of public GitHub repositories.
- Embed all collected manifests using a standard dense code embedding model such as CodeBERT or OpenAI text-embedding-3-small.
- Construct a hierarchical skill indexing structure mapping ecosystem-level skills to repository clusters via vector similarity.
- Evaluate cold-start micro-skill retrieval performance and scope detector accuracy on unseen benchmarks like SWE-Bench Lite.
Code linked to this paper on Hugging Face: github.com/microsoft/TokenEconomizer
Suggested by AI. Adapt it to your setup.
Has This Been Done?
None of the listed papers cover cross-repository micro-skill generalization using project manifest embeddings and hierarchical skill banks. Papers like Dynamic Agent Skills: A Lifecycle Survey and Taxonomy of Evolving Skill Libraries and RESOURCE2SKILL discuss agent skill extraction and lifecycles, but they do not address retrieving skills from structurally similar codebases via environment manifests.
- Dynamic Agent Skills: A Lifecycle Survey and Taxonomy of Evolving Skill Libraries · 2026 · 9 citations
- RESOURCE2SKILL: Distilling Executable Agent Skills from Human-Created Multimodal Resources · 2026 · 5 citations
We searched OpenAlex and arXiv for “Implement a hierarchical skill bank combining global ecosystem skills with repository-specific overrides by embedding project setup manifests from public GitHub repositories.” and “Cross-Repository Micro-Skill Generalization via Project Manifest Embeddings” and an AI compared the idea with the top 10 results. A small search can miss relevant work.
Source Paper
TokenEconomizer: Experience-Driven Black-Box Cost Control for Coding AgentsAaron J. Li, Weijia Xu, Chandan Singh +8 more · 2026
This paper addresses the excessive token costs incurred by frontier coding agents when executing sequential software engineering tasks within shared environments. The authors introduce TokenEconomizer, an external, black-box controller that monitors execution traces to adaptively inject environment-specific micro-skills, prune unused tool definitions from agent contexts, and interrupt unproductive failure loops. Evaluated across Claude Code, Codex, and GitHub Copilot CLI on benchmarks including SWE-Bench, Web-Bench, and security auditing datasets, TokenEconomizer achieves an 18.4% to 30.9% token cost reduction while preserving or improving task success rates.
Related Ideas
- Evaluation on Repository-Scale Benchmarks (SWE-bench)AutoDev: Automated AI-Driven Development🟡Fairly easy
- Subgraph-Decomposed Curriculum RL for Hard Repository-Level Software EngineeringSage S2.1 Flash Technical Report🔴Hard
- Hermetic Harness-Agent Environment Isolation Against Dependency-Mediated Answer LeakageSage S2.1 Flash Technical Report🟡Fairly easy
- Persistent Agentic Heuristics through Post-Task ReflectionDeepCode: Open Agentic Coding🟡Fairly easy
- Retrieval-Augmented Cross-Kernel Experience Transfer to Mitigate Agent Harness SensitivityAscendGYM: A Cost-Aware Kernel Optimization Benchmark for Real-World Workloads on Ascend NPUs🟡Fairly easy