CIDERS: Cloud-Edge LLM Collaborative Learning via Accelerating Personalized Bilevel Optimization

A personalized bilevel optimization framework and efficient solver for consensus-guided continuous personalization in cloud-edge LLM systems

Victor H. Chen; Hairui Yu; Stella K. Chung; Hong Yan· 2026· DOI 10.48550/arXiv.2609.15664

The core problem

The rapid advancement of physical-world intelligence has positioned cloud-edge collaborative large language models (LLMs) as a promising roadmap for practical LLM deployment. However, existing cloud-edge paradigms struggle to balance global consensus with local personalization. They fail to satisfy the dual need for a unified knowledge foundation on the cloud and domain-specific adaptation at the edge. This work introduces, for the first time, a personalized bilevel optimization framework that formalizes cloud-edge LLM collaboration as a dual structure: the upper level optimizes edge-side personalization, while the lower level governs cloud-side knowledge transfer, enabling cloud and edge to evolve in coordination. To solve this efficiently, the authors propose CIDERS, an efficient solver that decomposes the model into a learnable backbone and a messenger. The cloud performs knowledge transfer to the learnable backbone, while the key innovation lies in embedding global trajectories into each local personalization step via consensus-variate correction to reconcile personalization with consensus. The paper provides a comprehensive theoretical analysis, including a geometric character

Innovation

Extensive experiments demonstrate that CIDERS consistently outperforms competitive baselines on the compressed edge path. Key quantitative results include: - **Mathematical reasoning**: 3.1x gain over baselines. - **Code generation**: 1.7x gain over baselines. - **Instruction metrics**: 10% relative gain. These gains are achieved while maintaining efficient cloud-edge collaboration. The compressed edge path refers to the scenario where edge devices have limited computational resources, making the efficiency of CIDERS particularly valuable. The results indicate that CIDERS effectively balances global consensus and local personalization, leading to significant improvements across diverse tasks.

The rapid advancement of physical-world intelligence has positioned cloud-edge collaborative large language models (LLMs) as a promising roadmap for practical LLM deployment. However, existing cloud-edge paradigms struggle to balance global consensus with local personalization. They fail to satisfy the dual need for a unified knowledge foundation on the cloud and domain-specific adaptation at the edge. This work introduces, for the first time, a personalized bilevel optimization framework that formalizes cloud-edge LLM collaboration as a dual structure: the upper level optimizes edge-side personalization, while the lower level governs cloud-side knowledge transfer, enabling cloud and edge to evolve in coordination. To solve this efficiently, the authors propose CIDERS, an efficient solver that decomposes the model into a learnable backbone and a messenger. The cloud performs knowledge transfer to the learnable backbone, while the key innovation lies in embedding global trajectories into each local personalization step via consensus-variate correction to reconcile personalization with consensus. The paper provides a comprehensive theoretical analysis, including a geometric characterization of the local trajectory and a full convergence guarantee, revealing an explicit trade-off structure between personalization and global convergence. Extensive experiments demonstrate that CIDERS consistently outperforms competitive baselines on the compressed edge path, with 3.1x and 1.7x gains on mathematical reasoning and code generation, respectively, and a 10% relative gain on instruction metrics. Mechanism experiments attribute these gains to early consensus-corrected coordination and task-aware distillation. Overall, CIDERS offers a viable path toward consensus-guided continuous personalization in cloud-edge LLM systems.

The proposed framework formalizes cloud-edge LLM collaboration as a personalized bilevel optimization problem. Let θ denote the cloud-side global model parameters and ϕ_{i} denote the edge-side personalized parameters for client i . The bilevel problem is:

Why it matters

Mechanism experiments attribute the observed gains to two key factors: early consensus-corrected coordination and task-aware distillation. Early consensus-corrected coordination ensures that global information is embedded into local personalization from the beginning, preventing divergence and promoting alignment. Task-aware distillation allows the cloud to transfer knowledge that is specifically relevant to the edge tasks, enhancing personalization without sacrificing global consensus. The theoretical analysis provides a geometric characterization of the local trajectory, revealing an explicit trade-off structure between personalization and global convergence. This trade-off is managed by the consensus-variate correction, which adjusts local updates to balance the two objectives. The full convergence guarantee ensures that the algorithm converges to a stationary point, providing a solid theoretical foundation for practical deployment. Overall, CIDERS offers a viable path toward consensus-guided continuous personalization in cloud-edge LLM systems. It addresses the critical challenge of balancing global consensus with local personalization, enabling efficient and effective deployment of LLMs in edge environments. Future work may explore extending CIDERS to other collaborative learning scenarios and further optimizing the trade-off between personalization and consensus.

Who should read this

CS practitioners and researchers

Opening member content…