Skip to content
AI-generated research idea🔴Hard🔬Significant ResearchNo close match found

Empirical Production Serving Benchmarking of MLA-DSA Attention with IndexShare

A benchmark and serving engine that evaluates combined MLA, DSA, and IndexShare architectures under realistic dynamic batching and serving traffic on physical accelerators.

Time

a couple of months

Compute level

Multiple GPUs

Resources

Multi-accelerator inference server (e.g., 8-32 Ascend 910C cards or NVIDIA H100 GPUs) and production traffic traces

What the paper used

Simulated on 32 Ascend 910C cards using Huawei Zhanlu simulation framework for a 285B model at 256K context length.

Prerequisites

LLM inference serving systems, custom attention kernels, continuous batching

Scope

modify existing code

The Gap

Section 2.2 evaluates the inference efficiency of MLA, MLA+DSA, and MLA+DSA+IndexShare variants on Ascend 910C using prefill/decode disaggregation at 256K context length, relying primarily on Huawei's Zhanlu simulation framework.

“Since simulation does not fully capture all production-serving overheads, the reported results may be more optimistic than full end-to-end deployment measurements.”
From the paper, Section 2.2, page 6. Matched against the PDF text.

Limitation

The architectural evaluation relies on simulation modeling rather than end-to-end physical serving deployment, omitting real-world serving overheads such as dynamic batching, KV cache fragmentation, request arrival queuing, and disaggregated network transfer jitter.

Proposed Solution

Build an end-to-end physical serving implementation of MLA+DSA+IndexShare within modern serving frameworks such as vLLM or SGLang on physical hardware. Evaluate both time-to-first-token (TTFT) and time-per-output-token (TPOT) across varying request arrival distributions, concurrent load levels, and context lengths up to 256K tokens, measuring the impact of real KV cache memory management, indexer compute scheduling, and prefill/decode network transfers.

Why This Matters

Empirical verification will reveal whether the theoretical and simulated speedups (e.g., 4.68x prefill throughput) translate into real serving gains or are degraded by systems overheads under production traffic.

How to Get Started

How to test it: Deploy the open-sourced Xing4-29B model on physical hardware with continuous batching and compare measured TTFT, TPOT, and request throughput to simulated numbers across different request arrival rates, stopping if DSA indexer computation under dynamic batching completely negates the memory bandwidth savings of sparse KV retrieval.

  1. Deploy the open-source Xing4.0-29B-A4B checkpoint on a multi-accelerator serving node using SGLang or MindIE.
  2. Implement tracing for TTFT, TPOT, indexer execution time, and KV cache allocation memory overhead under static batching.
  3. Replay real multi-turn conversation traces with dynamic continuous batching at varying concurrency levels.
  4. Compare empirical serving throughput and latency curves against the simulated baselines reported in Table 2.

Suggested by AI. Adapt it to your setup.

Has This Been Done?

No close match in 10 results

None of the listed papers provide an end-to-end production serving benchmark specifically integrating MLA, DSA, and IndexShare with evaluations of TTFT and TPOT up to 256K contexts. While works like 'DeepSeek-V4' and 'Multi-layer Indexer' discuss architectural components and sparse attention, they do not present this specific empirical serving benchmarking framework.

We searched OpenAlex and arXiv for “deepseek sparse attention serving inference production benchmark” and “Empirical Production Serving Benchmarking of MLA-DSA Attention with IndexShare” and an AI compared the idea with the top 10 results. A small search can miss relevant work.

Source Paper

TeleChat to Xing4: Advancing Frontier Agentic Models on Domestic Chips and Frameworks

Xingchen AGI Lab, Baotian Hu, Bingkai Yang +136 more · 2026

This technical report presents Xing 4, a family of large language models scaling up to hundreds of billions of parameters trained natively on Ascend 910C NPUs using MindSpore HyperParallel. The report establishes design principles combining sparse attention, wider-and-shallower MoE topologies, and speculative decoding, supported by systems optimizations including hybrid sharded data parallelism, owner-based Muon execution, and CPU core isolation. It also introduces TeleGym, a unified execution and reinforcement learning platform for long-horizon agent and reasoning workloads.

Related Ideas

Is this a good idea?

Share onXLinkedIn