Ion Stoica: Memory is AI's Primary Bottleneck, Infrastructure Innovation Crucial Across All Layers
Baca dalam 60 detik
- Ion Stoica mengungkapkan bahwa memori masih menjadi bottleneck krusial dalam pengembangan AI karena penyimpanan satu bit data tetap membutuhkan satu transistor.
- Efisiensi di setiap lapisan tumpukan teknologi dinilai wajib untuk mengimbangi lonjakan biaya pelatihan dan inferensi yang terus meningkat eksponensial.
- Pendekatan human-in-the-loop dan formal methods dipercaya dapat menjembatani kesenjangan antara intent pengguna dan kebutuhan sistem di era agentic AI.
Ion Stoica, a UC Berkeley Professor and co-founder of Databricks and Anyscale, stated that memory has become the biggest bottleneck in accelerating AI adoption in the agentic era. During a discussion titled The Next Endeavor, organized by Imagination in Action in collaboration with Stanford HAI on September 14–15 at Google Bay View, Mountain View, California, Stoica emphasized that memory limitations remain a fundamental unresolved issue.
Stoica explained that to this day, storing one bit of information still requires one transistor. "You can make transistors smaller, but this one-to-one mapping doesn't change," he said. According to him, computing capacity will grow faster than memory capacity, and bandwidth is also predicted to increase more rapidly because it's easier to scale. Consequently, memory is increasingly becoming a bottleneck, necessitating layer decomposition and system modularity for maximum efficiency.
In a session moderated by Daniela Rus, Director of MIT CSAIL, Stoica also highlighted the high costs of training and inference, which continue to grow exponentially. "Efficiency becomes incredibly important, and to get it, you almost have to innovate at every layer of the stack," he asserted. He added that many key innovations, including algorithmic evolution, are actually driven from the system side.
Stoica also discussed the challenges of alignment and the risks of recursive self-improvement. According to him, the primary goal must be to build reliable AI applications and systems that are aligned with user intent. "This is very difficult because there's a fundamental gap between user intent and system requirements, as well as between the real world and the environmental models we have," he explained. This gap, he continued, becomes increasingly difficult to close in an open and constantly changing world.
Responding to Rus's question about workload distribution between cloud and local devices, Stoica believes that local AI on edge devices can help address privacy concerns. He also linked this development to the big data era, where Hadoop and traditional queries were gradually replaced by much faster methods. Stoica himself was involved in the development of Spark, a SQL-related processing technology that now appears to be becoming obsolete in the AI era.
"As long as you're building this for humans, humans will be part of the loop; because you need to fulfill human intent... humans can be a bottleneck. We have to make sure that we use humans in the most effective way, to provide feedback, and to keep improving the system."
— Ion Stoica
Rus then raised the issue of physical AI, envisioning 100,000 robots operating to perform tasks. According to her, such an armada of robots with physical intelligence would require robust infrastructure. Stoica countered by highlighting the Hugging Face incident and the threats of reward hacking and hallucinations. He suggested that AI operations should, in many cases, be isolated to prevent unintended risks.
Stoica concluded by emphasizing the importance of strengthening formal methods. "You're going to try to close the gap and capture, in formal specifications, more of human intent and more of the real world," he concluded. This call for clarity is deemed crucial as the adoption of agentic AI accelerates.
Context for Indonesia
For Indonesia, Stoica's spotlight on memory bottlenecks and AI infrastructure efficiency reflects the challenges faced. Dependence on foreign data centers and limited local computing resources make national AI development vulnerable to cost surges. However, the opportunity for AI on edge devices—which can reduce latency and maintain privacy—could be a solution for remote areas with limited connectivity. The government and industry players need to encourage research at the system layer, not just applications, to avoid perpetually being technology consumers.
Looking ahead, the question is: can Indonesia build a robust AI ecosystem by prioritizing innovation at the infrastructure level, or will it continue to rely on breakthroughs from abroad?
