CLASP: Chained-Request-Aware Scaling and Operator Placement for Serverless Stream Processing

A runtime scaling and scheduling strategy for stateful serverless stream processing that accounts for chained-request overhead to improve throughput and latency.

Tianyu Qi; Maria A. Rodriguez; Rajkumar Buyya· 2026· DOI 10.48550/arXiv.2608.29103

The core problem

Stateful serverless (Function-as-a-Service) environments, where workers host state servers, are increasingly adopted for stream processing. A stream application is structured as a pipeline of operators, each forwarding intermediate data downstream via chained requests. As input rates fluctuate, the system must dynamically adjust operator parallelism and place instances across workers to sustain the incoming rate. Existing scaling approaches often neglect the overhead of chained requests, leading to misestimation of required workers. Under-provisioning results in inability to keep up with input rates, while over-provisioning increases cross-worker chained requests, raising end-to-end latency. CLASP addresses this gap by explicitly modeling both execution and chained-request costs.

Experimental evaluation demonstrates that CLASP outperforms state-of-the-art scaling strategies. Specifically, CLASP improves throughput by up to 3.3x and reduces median end-to-end latency by up to 76%. These gains are attributed to its accurate cost estimation and efficient operator placement, which avoid both under- and over-provisioning. The results highlight the importance of considering chained-request overhead in serverless stream processing environments.

Stateful serverless (Function-as-a-Service) environments, where workers host state servers, are increasingly adopted for stream processing. A stream application is structured as a pipeline of operators, each forwarding intermediate data downstream via chained requests. As input rates fluctuate, the system must dynamically adjust operator parallelism and place instances across workers to sustain the incoming rate. Existing scaling approaches often neglect the overhead of chained requests, leading to misestimation of required workers. Under-provisioning results in inability to keep up with input rates, while over-provisioning increases cross-worker chained requests, raising end-to-end latency. CLASP addresses this gap by explicitly modeling both execution and chained-request costs.

CLASP operates at runtime, estimating execution cost and chained-request cost from observed metrics. It employs a capacity model that incorporates both costs to determine the optimal operator parallelism and placement. The model ensures operators are packed onto the fewest workers that can sustain the target input rate. Once a scaling decision is made, CLASP migrates each operator's state along with its instances to minimize execution pause time. The capacity model can be formalized as follows: the total cost per worker is the sum of execution cost C_{e x ec} and chained-request cost C_{c hain}, subject to the constraint that the total processing capacity meets or exceeds the target input rate R_{t a r g e t} . The optimization problem is to minimize the number of workers N such that i = 1 \sum N (C_{e x ec, i} + C_{c hain, i}) \geq R_{t a r g e t} . CLASP solves this dynamically as metrics evolve.

The key insight of CLASP is that chained-request overhead significantly impacts the performance of serverless stream processing. By incorporating this cost into the scaling decision, CLASP achieves a more efficient use of resources. The state migration mechanism further reduces pause time, ensuring smooth transitions during scaling. While CLASP shows promising results, future work could explore adaptation to heterogeneous workloads and integration with other serverless platforms. The taxonomy candidates (Architecture, Cybersecurity, Network, Cryptography) suggest potential intersections with security and network optimization, though the current work focuses on performance.

CS practitioners and researchers

Opening member content…