Ilmu Komputer & AI editorial
Gutenberg: Taming Latency-Critical Cloud Services with Near-Data-Processing
The core problem
Latency-critical cloud services are increasingly memory-bound, and they must simultaneously deliver isolation, fairness, and predictable quality of service (QoS). Near-data processing (NDP) offers a promising path: by executing requests close to memory, it reduces data movement and can improve locality through caching and replication. However, two stubborn obstacles remain. First, writes make replica maintenance expensive, because keeping replicas coherent typically requires eager full-page synchronization. Second, uneven compute and memory traffic can overload a few NDP units, inflating tail latency. Existing throughput-oriented schedulers were not designed for these challenges in co-located cloud services.
Gutenberg addresses these gaps with a CPU+NDP architecture for mutable, latency-critical cloud services. Its design rests on three pillars: (1) a CPU-resident delta buffer that stages subpage updates so hot writable pages can remain replicated without eager full-page synchronization; (2) CPU helper cores that assist request execution when NDP execution or replica maintenance becomes costly; and (3) an online controller that jointly decides page placement, replication, CPU/NDP
Innovation
The authors evaluate Gutenberg on TailBench using ZSim with Ramulator-calibrated memory timing. Across the evaluated services, Gutenberg outperforms prior systems, reducing average latency by up to 80.4% and p99 latency by up to 85.8%. These are substantial improvements for latency-critical cloud services, where tail latency often determines user-perceived performance and service-level objective (SLO) compliance.
The reported gains span both average and tail metrics, indicating that Gutenberg's benefits are not limited to the common case. The delta buffer reduces write-induced replica maintenance overhead, while helper cores and the online controller prevent a few NDP units from becoming bottlenecks. Together, these mechanisms address the two obstacles identified in the introduction: expensive replica maintenance under writes and overload of a few NDP units.
Beyond latency, Gutenberg also improves isolation and fairness while adapting to changing workload behaviors. Isolation and fairness are critical in multi-tenant cloud environments, where one service's behavior should not degrade another's QoS. The online controller's use of access patterns, queue pressure, and feedback from
Why it matters
Gutenberg's design reflects a broader insight: for mutable, latency-critical cloud services, the bottleneck is not raw memory bandwidth alone but the interaction between write-induced replica maintenance, uneven compute and memory traffic, and the need for isolation and fairness. Prior NDP systems improved locality through caching and replication, but writes made replica maintenance expensive, and throughput-oriented schedulers did not fully address these challenges for co-located services.
The delta buffer is the key enabler for mutable workloads. By staging subpage updates in CPU-resident memory, Gutenberg avoids eager full-page synchronization, allowing hot writable pages to remain replicated. This changes the economics of replication: the system pays a small delta cost rather than a full-page synchronization cost on every write. The trade-off is added complexity in the coordination protocol, which the authors address through model-checking.
CPU helper cores provide a second form of elasticity. When NDP execution or replica maintenance becomes costly, helper cores assist request execution, smoothing bursts and reducing tail latency. This hybrid execution model is a pragmatic response to the reality that NDP units may be unevenly loaded, and that a purely NDP-based design can suffer from hotspots.
The online controller ties these mechanisms together. By jointly deciding page placement, replication, CPU/NDP execution, and routing using access patterns, queue pressure, and feedback from prior decisions, it adapts to changing workload behaviors. This is important because cloud services rarely exhibit static behavior; traffic mixes, access patterns, and tenant demands shift over time. The controller also enforces isolation and fair resource allocation, which are non-negotiable in multi-tenant settings.
Several limitations and open questions remain. The evaluation is simulation-based (ZSim with Ramulator-calibrated timing), so real-hardware effects such as thermal behavior, interconnect contention, and NDP unit heterogeneity may differ. The model-checking covers the coordination protocol, but the online controller's heuristics are not formally verified. Finally, the taxonomy candidates for this work—Architecture, Cybersecurity, Network, and Cryptography—suggest potential intersections: isolation and fairness have security implications, and the CPU–NDP coordination protocol may be relevant to secure multi-tenant execution. Future work could explore these dimensions, as well as deployment in production cloud environments.
In summary, Gutenberg demonstrates that near-data processing can be made practical for mutable, latency-critical cloud services when writes are handled via delta buffering, execution is made elastic via helper cores, and control is centralized in an online, feedback-driven controller. The reported reductions of up to 80.4% in average latency and 85.8% in p99 latency, together with improved isolation and fairness, make a compelling case for this CPU+NDP hybrid approach.
Who should read this
Opening member content…