Universal Prompt Optimization via Multi-Model Adversarial Feedback
Optimize prompts for multiple language models simultaneously using an adversarial minimax objective that maximizes the minimum win-rate across an ensemble.
Time
Compute level
Resources
Prerequisites
Scope
The Gap
The 'Limitations' section notes that SPO currently optimizes prompts for a single model. The ablation study (Table 2) shows that prompts optimized on one model (e.g., Claude-3.5) do not always transfer perfectly to others (e.g., Llama-3-70B).
Limitation
The current optimization creates prompts that may exploit specific quirks or 'latent probabilistic biases' of the target model, resulting in brittle prompts that fail when model versions change or when deployed on cheaper, smaller models.
Proposed Solution
Modify the execution function φexe to generate outputs from an ensemble of models (M1, M2, M3) simultaneously. The evaluation function φeval would then aggregate pairwise win-rates across all models. The optimizer φopt would be conditioned to maximize the 'minimum win-rate' across the ensemble, acting as a minimax object to find universally robust prompts.
Why This Matters
This solves the 'model lock-in' problem, creating robust prompts that allow developers to switch backend LLMs (e.g., from GPT-4 to Llama-3) without re-optimizing their entire prompt library.
How to Get Started
- Establish an API evaluation harness supporting multiple open and closed-source large language models.
- Implement a multi-model execution function that generates parallel outputs for candidate prompts.
- Define an aggregation metric focusing on the worst-case win-rate across the ensemble.
- Run optimization loops using the multi-model feedback and evaluate cross-model prompt transferability.
Code linked to this paper on Hugging Face: github.com/geekan/metagpt
Suggested by AI. Adapt it to your setup.
Has This Been Done?
Works like Universal and Transferable Adversarial Attacks on Aligned Language Models and Fight Back Against Jailbreaking via Prompt Adversarial Tuning discuss adversarial prompts and robustness. However, none of the listed papers implement a minimax optimization framework using ensemble win-rates across multiple models for universal prompt optimization.
- Universal and Transferable Adversarial Attacks on Aligned Language Models · 2023 · 4834 citations
- LinkPrompt: Natural and Universal Adversarial Attacks on Prompt-based Language Models · 2024 · 25 citations
- Fight Back Against Jailbreaking via Prompt Adversarial Tuning · 2024 · 109 citations
We searched OpenAlex for “Optimize prompts for multiple language models simultaneously using an adversarial minimax objective that maximizes the minimum win-rate across an ensemble.” and “Universal Prompt Optimization via Multi-Model Adversarial Feedback” and an AI compared the idea with the top 10 results. A small search can miss relevant work.
Source Paper
Self-Supervised Prompt OptimizationJinyu Xiang, Jiayi Zhang, Zhaoyang Yu +6 more · 2025
The paper introduces Self-Supervised Prompt Optimization (SPO), a framework designed to automate the improvement of prompts for Large Language Models (LLMs) without relying on ground truth data or human feedback. SPO leverages the inherent capability of LLMs to distinguish better answers through pairwise comparisons (Output vs. Output) to guide the optimization process, achieving state-of-the-art performance at a fraction of the cost of existing methods.
Related Ideas
- Safety-Tuned Process Reward Models for Infeasible Task RefusalEnterpriseOps-Gym: Environments and Evaluations for Stateful Agentic Planning and Tool Use in Enterprise Settings🟠Medium
- Adaptive Budget Allocation for Evolutionary CrossoverGEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning🟡Fairly easy
- Closed-Loop Adaptive Red-Teaming and Adversarially Trained Span-Veto SanitizationSPILLWAY: Separating Target Admission from Sanitization to Defend Tool-Using Agents against Prompt Injection🔴Hard
- Automated Hyperparameter Optimization (AutoML) ModuleLlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models🟡Fairly easy
- Heterogeneous Multi-Reviewer Ensemble Routing for Autonomous Paper RefinementARIS: Autonomous Research via Adversarial Multi-Agent Collaboration🟡Fairly easy