Skip to content

Source paper

EnterpriseOps-Gym: Environments and Evaluations for Stateful Agentic Planning and Tool Use in Enterprise Settings

Shiva Krishna Reddy Malay, Shravan Nayak, Jishnu Sethumadhavan Nair and 6 more

Shiva Krishna Reddy Malay, Shravan Nayak, Jishnu Sethumadhavan Nair, Sagar Davasam, Aman Tiwari, Sathwik Tejaswi Madhusudhan, Sridhar Krishna Nemala, Srinivas Sunkara, Sai Rajeswar

arXiv 2603.13594 · 2026129 upvotes on Hugging Face

Research ideas from this paper (3)

Generated by AI. The paper’s authors did not write these.

AI summary

This paper introduces ENTERPRISEOPS-GYM, a highly complex, sandboxed benchmark featuring 164 database tables and 512 tools designed to evaluate LLM agents in realistic, policy-constrained enterprise workflows. The study reveals that current frontier models struggle significantly with long-horizon strategic planning, cascading state dependencies, and safely refusing infeasible tasks, indicating they are not yet ready for autonomous enterprise deployment.

How it works (AI summary)

The authors developed a fully containerized simulation covering eight enterprise domains (e.g., HR, ITSM, CSM). They evaluated 14 frontier models using a ReAct-style reasoning and tool-execution loop in an oracle-tool setting, measuring success via expert-authored, outcome-based SQL verifiers that enforce goal completion, state integrity, and strict policy compliance.