POWERSLIDER demonstrates how dynamic power caps change the LLM serving task: now, it is necessary to manage not only latency and throughput but also the instantaneous power envelope. The key here is not in “reducing power overall,” but in spending every watt where it costs less for goodput.
Modern inference clusters are constrained not only by energy but also by instantaneous network limitations. For demand response, this means a specific runtime contract: the cluster must remain below the time-varying power cap pmax(t) at every moment, rather than just saving energy over the long term. Here, conventional approaches begin to fail. Static energy optimization does not solve the dynamic problem, and rigid priority tiers quickly burn through goodput when the cap shifts deeper.
The issue is that the LLM pipeline is not a uniform load. The prefill phase is computationally intensive and almost linearly loses throughput when the GPU frequency decreases. Answer decode, on the other hand, is often memory-bandwidth-bound and can operate at significantly lower frequencies without a sharp drop. Reasoning workloads add a thinking phase, where the KV-cache becomes a separate constraint for scheduling. This is the architectural fracture: the same watt at different stages contributes differently to performance.
This leads to an important trade-off. Universal throttling of all GPUs uniformly cuts prefill where losses are maximal, even though part of the decode path could give up power almost painlessly. POWERSLIDER takes a different approach. It introduces a Flex SLO contract and turns bounded user slack into a constraint for optimization. The user intent of this contract is simple: part of the traffic can tolerate controlled latency degradation, but not indefinitely and not chaotically.
Technically, the solution is built around Prefill–Think–Answer disaggregation. This expands the classic prefill–decode scheme and provides the runtime with finer levers: per-stage GPU allocation, stage-aware DVFS, and KV-cache partitioning. Such a design does not eliminate complexity but makes it manageable. Instead of one coarse knob, the system receives several independent loops among which power deficits can be distributed.
Next comes the most interesting part. The configuration space quickly becomes too large for offline profiling. In the source, it is described as a combination of stage, SLO class, GPU allocation, frequency, KV chunk size, and model/TP setting. Therefore, POWERSLIDER uses an online solver derived from the Karush–Kuhn–Tucker conditions. This is a pragmatic choice: not to search for configurations through brute force but to solve the relaxation based on the current cap state.
An important detail is how the system makes decisions. The solver does not simply reduce frequency everywhere. It ranks stage-class groups based on how much goodput is lost for each watt saved. In memory-bound stages, reducing frequency saves power more cheaply because throughput there is less dependent on f. In compute-bound prefill, the picture is the opposite: each frequency reduction is costly. Therefore, degradation occurs not “top-down,” but based on the cost of performance loss per unit of power.
The implementation is also designed as an engineering compromise between response speed and switching costs. PSOpt recalculates the solution quickly, within 7.7 ms, allowing it to respond to cap changes in online mode. PSSched applies reallocation on a slower cycle, with drain-before-reassign, to avoid recalculating KV state. For DVFS, a vote-commit scheme is used with a period of 50–100 ms and one NVML call per GPU. This is not an ideal instantaneous response, but it aligns with the real limitations of the hardware interface.
There is also a separate mechanism to protect against deep caps. When DVFS hits the lower boundary and there is almost nothing left to cut power, the system power-gates drained instances and consolidates load onto fewer GPUs. This is an honest acknowledgment of the hardware floor: below static power, one cannot go only by frequency. Thus, the runtime must change not only frequency but also the number of active GPUs.
The most vulnerable area of such systems is reasoning traffic. It breaks prediction-based routing because output lengths are heavy-tailed. The source explicitly states that the length predictor gives a 37.1% misprediction rate after fine-tuning on reasoning traces. For an overloaded cluster, this turns into a queuing cascade. POWERSLIDER circumvents the problem through observation-based admission and stage isolation, rather than through a more “intelligent” length prediction.
Results show that this architectural bet is justified. On SGLang with production traces, POWERSLIDER maintains 78.3% online goodput with a 30% reduction, while the best baseline is 47.6%. On a replayed CAISO grid-emergency day with a trough down to 0.41×, the system holds 92% mean goodput and 54% at the deepest point, while all baselines drop below 7%. Latency tails also remain closer to nominal: LC TTFAT and TTLT are maintained within 1.3×, while the baselines show significantly worse growth.
Information source
arXiv is the largest open preprint repository (since 1991, under the auspices of Cornell), where researchers quickly post working versions of papers; the materials are publicly accessible but do not undergo full peer review, so results should be considered preliminary and, where possible, checked against updated versions or peer‑reviewed journals. arxiv.org