Empirical Production Serving Benchmarking of MLA-DSA Attention with IndexShare
A benchmark and serving engine that evaluates combined MLA, DSA, and IndexShare architectures under realistic dynamic batching and serving traffic on physical accelerators.
Time
Compute level
Resources
What the paper used
Prerequisites
Scope
The Gap
Section 2.2 evaluates the inference efficiency of MLA, MLA+DSA, and MLA+DSA+IndexShare variants on Ascend 910C using prefill/decode disaggregation at 256K context length, relying primarily on Huawei's Zhanlu simulation framework.
“Since simulation does not fully capture all production-serving overheads, the reported results may be more optimistic than full end-to-end deployment measurements.”
Limitation
The architectural evaluation relies on simulation modeling rather than end-to-end physical serving deployment, omitting real-world serving overheads such as dynamic batching, KV cache fragmentation, request arrival queuing, and disaggregated network transfer jitter.
Proposed Solution
Build an end-to-end physical serving implementation of MLA+DSA+IndexShare within modern serving frameworks such as vLLM or SGLang on physical hardware. Evaluate both time-to-first-token (TTFT) and time-per-output-token (TPOT) across varying request arrival distributions, concurrent load levels, and context lengths up to 256K tokens, measuring the impact of real KV cache memory management, indexer compute scheduling, and prefill/decode network transfers.
Why This Matters
Empirical verification will reveal whether the theoretical and simulated speedups (e.g., 4.68x prefill throughput) translate into real serving gains or are degraded by systems overheads under production traffic.
How to Get Started
How to test it: Deploy the open-sourced Xing4-29B model on physical hardware with continuous batching and compare measured TTFT, TPOT, and request throughput to simulated numbers across different request arrival rates, stopping if DSA indexer computation under dynamic batching completely negates the memory bandwidth savings of sparse KV retrieval.
- Deploy the open-source Xing4.0-29B-A4B checkpoint on a multi-accelerator serving node using SGLang or MindIE.
- Implement tracing for TTFT, TPOT, indexer execution time, and KV cache allocation memory overhead under static batching.
- Replay real multi-turn conversation traces with dynamic continuous batching at varying concurrency levels.
- Compare empirical serving throughput and latency curves against the simulated baselines reported in Table 2.
Suggested by AI. Adapt it to your setup.
Has This Been Done?
None of the listed papers provide an end-to-end production serving benchmark specifically integrating MLA, DSA, and IndexShare with evaluations of TTFT and TPOT up to 256K contexts. While works like 'DeepSeek-V4' and 'Multi-layer Indexer' discuss architectural components and sparse attention, they do not present this specific empirical serving benchmarking framework.
- Multi-layer Indexer: Overlapping Selection and Attention in DeepSeek Sparse Attention · 2026 · 0 citations
- FlashMemory-DeepSeek-V4: Lightning Index Ultra-Long Context via Lookahead Sparse Attention · 2026 · 4 citations
- DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence · 2026 · 1614 citations
We searched OpenAlex and arXiv for “deepseek sparse attention serving inference production benchmark” and “Empirical Production Serving Benchmarking of MLA-DSA Attention with IndexShare” and an AI compared the idea with the top 10 results. A small search can miss relevant work.
Source Paper
TeleChat to Xing4: Advancing Frontier Agentic Models on Domestic Chips and FrameworksXingchen AGI Lab, Baotian Hu, Bingkai Yang +136 more · 2026
This technical report presents Xing 4, a family of large language models scaling up to hundreds of billions of parameters trained natively on Ascend 910C NPUs using MindSpore HyperParallel. The report establishes design principles combining sparse attention, wider-and-shallower MoE topologies, and speculative decoding, supported by systems optimizations including hybrid sharded data parallelism, owner-based Muon execution, and CPU core isolation. It also introduces TeleGym, a unified execution and reinforcement learning platform for long-horizon agent and reasoning workloads.
Related Ideas
- Dynamic Network-Compute Latency ArbitratorLMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inference🟡Fairly easy
- Multi-Shape Generalization and End-to-End Workload Evaluation for NPU Kernel OptimizationAscendGYM: A Cost-Aware Kernel Optimization Benchmark for Real-World Workloads on Ascend NPUs🟠Medium
- Execution-Time and Hardware-Cost-Aware Scoring Beyond Token Budgets in Agentic Kernel OptimizationAscendGYM: A Cost-Aware Kernel Optimization Benchmark for Real-World Workloads on Ascend NPUs🟡Fairly easy
- Dynamic Bucket Resizing via Control TheoryPyTorch Distributed: Experiences on Accelerating Data Parallel Training🟡Fairly easy
- Dynamic Block Sizing for Multi-Architecture ServingEfficient Memory Management for Large Language Model Serving with PagedAttention🔴Hard