Ilmu Komputer & AI editorial
WiCi: Wireless GPU Computing Infrastructure
The core problem
Large language model (LLM) inference applications are gaining significant traction, and demand for inference is growing exponentially. Notably, the GPU usage of inference is increasingly surpassing that of training. However, due to the mobility penalty, edge-side inference fails to deliver satisfactory performance. As a result, most inference service providers currently rely on cloud-based inference, which incurs substantial and not sustainable costs for enterprises, and these costs are even increasing in the agentic paradigm.
Motivated by this gap, the authors set a clear goal: enable powerful computing capabilities comparable to server-grade GPUs on mobile devices. They propose **Wireless GPU Computing Infrastructure (WiCi)**. Through WiCi, mobile devices can wirelessly access server-grade GPUs, running inference tasks on mobile clients while offloading GPU-related computations to a nearby GPU via WiFi. WiCi introduces a series of designs to ensure the infrastructure is scalable across different applications, compatible with different mobile devices, and capable of performance comparable to running on a physical GPU.
Innovation
The authors test WiCi from mobile devices and report substantial improvements over local mobile inference. Specifically, WiCi can reduce time to first token by up to **90%**, improve the token rate by approximately **39x** compared to local inference on mobile devices for the same model, and support much larger models. In addition, WiCi achieves up to nearly **80%** of the native performance of the server-grade GPU across different applications.
These results indicate that wireless offloading can bridge a significant portion of the performance gap between mobile devices and server-grade GPUs. The reported gains span both responsiveness (time to first token) and sustained throughput (token rate), while also expanding the feasible model size on mobile clients.
Why it matters
The findings suggest that WiCi addresses the mobility penalty that limits edge-side inference by relocating GPU-heavy computation to a nearby server-grade GPU over WiFi. This approach contrasts with cloud-based inference, which incurs substantial and not sustainable costs for enterprises, especially as agentic paradigms increase inference demand.
By achieving up to nearly 80% of native server-grade GPU performance across different applications, WiCi demonstrates that wireless offloading can be a practical middle ground between local mobile inference and fully cloud-based inference. The design emphasis on scalability across applications, compatibility across mobile devices, and performance comparable to a physical GPU positions WiCi as a candidate infrastructure for cost-sensitive, latency-sensitive LLM inference on mobile devices. The taxonomy candidates for this work include Architecture, Cybersecurity, Network, and Cryptography, reflecting its systems and networking foundations.
Who should read this
Opening member content…