CLASP demonstrates why auto-scaling for stateful serverless stream processing cannot be reduced to just parallelism and slots. Chained requests, operator placement, and state migration are also important.
Stateful serverless environments increasingly look like a practical runtime for stream processing. But once operators start exchanging data through chained requests, a simple capacity model quickly becomes ineffective. The result is familiar to any architect: the cluster appears underloaded on paper, but saturates in reality, or conversely receives too many workers and pays for it in latency.
The core problem is not only how much work each operator does. It is also how much capacity a worker spends on scheduling and dispatching chained requests between workers. Existing approaches often estimate worker load through executor slots or execution workload only, which misses the overhead of remote requests. When too few workers are assigned, the cluster cannot keep up with the input rate. When too many workers are used, more chained requests cross worker boundaries, and end-to-end latency grows.
CLASP takes a more complete view. It models worker load as a combination of execution cost and chained-request cost. At runtime, it estimates both from observed metrics, then uses the model to decide how many workers are needed and how operators should be placed. The trade-off is clear: CLASP spends more logic in the planner, but it gains a better approximation of what the cluster can actually sustain.
The placement strategy is built around a simple goal: use the fewest workers that can still sustain the target input rate. To do that, CLASP packs operators onto workers while accounting for both local and remote chained requests. It also places operators in reverse-topological order, starting from sinks, so that communication cost is known when each decision is made. This is a pragmatic heuristic, not an exact solver, but the paper frames the original operator-placement problem as NP-hard, so heuristics are the right engineering choice here.
Implementation matters because stateful serverless systems break easily during rescaling. If operator instances move but their state stays behind, the new placement immediately creates remote state access and latency rises. CLASP therefore migrates state together with operator instances and pending requests. The migration is decentralized. The planner computes the plan and broadcasts it, but workers transfer data directly between themselves and resume processing locally once their own incoming transfers are complete.
The paper’s runtime model is also notable. CLASP monitors queue length, request latency, chained-request rates, and the number of distinct remote workers reached by chained requests. It then estimates three coefficients for local chained-request dispatch, remote dispatch, and fan-out. These coefficients are learned online from saturated workers using recursive least squares. That is a useful design choice because the overheads are deployment-specific, and hardcoding them would make the model brittle.
There are a few important implementation trade-offs. CLASP assumes a homogeneous cluster. It also relies on a warm-up phase and on runtime observations from saturated workers. So the model is adaptive, but not magical: if the workload never reaches saturation, the estimator has less signal. The method also uses a coarse tolerance when worker count is found by binary search once the cluster limit is reached. That is a sensible engineering compromise, but it means the system optimizes for operational stability rather than mathematical exactness.
The evaluation shows that this extra modeling pays off. Compared with the state-of-the-art baselines in the paper, CLASP improves throughput by up to 3.3× and reduces median end-to-end latency by up to 76%. The migration step is also short. Planner-side migration is typically 10–20 ms, and workers usually resume within 10 ms. The paper does not give a single universal latency number for all cases, but it does show that coordinated state migration avoids the long recovery path that incremental rebalancing would create.
The broader lesson is straightforward. In stateful serverless stream processing, scaling is not just capacity planning. It is a joint problem of operator placement, communication cost, and state locality. CLASP is an evolutionary improvement because it treats these three layers as one control loop instead of separate subsystems.
Information source
arXiv is the largest open preprint repository (since 1991, under the auspices of Cornell), where researchers quickly post working versions of papers; the materials are publicly accessible but do not undergo full peer review, so results should be considered preliminary and, where possible, checked against updated versions or peer‑reviewed journals. arxiv.org