Apple Looks to Cram Massive Gemini Model into iPhone for AI-Powered Siri

Apple is working to bring a version of Google's multi-trillion parameter Gemini model to the iPhone to power a new generation of Siri, according to a report from The Information. The effort involves distilling an enormous frontier model down to something that can run on a smartphone—with significant help from both Google and Nvidia in the cloud.

The on-device AI challenge

Apple has long positioned on-device AI as a privacy differentiator. The company's Neural Engine and recent A-series chips include dedicated AI acceleration, but even the most capable smartphone hardware struggles with the memory and compute demands of frontier-scale models.

"The GPUs in most phones can process more AI tokens than the AI-focused NPUs," Ars Technica notes. "Even if phones had faster AI processing, they lack the RAM to keep enormous models in memory."

Cloud dependency remains

Despite Apple's privacy-focused rhetoric, the Gemini-infused Siri will reportedly run both on-device and in the cloud. This represents an apparent reversal of the company's stated preference for local-only AI. The cloud component leans heavily on Google's TPU infrastructure and Nvidia's GPU clusters.

Implications for self-hosted AI

Apple's struggle to bring frontier AI to a single device mirrors what self-hosters deal with daily: the trade-off between model capability and hardware constraints. While Apple has nearly unlimited engineering resources, they still cannot run a multi-trillion parameter model on a phone without cloud fallback.

For the self-hosted community, this validates the pragmatic approach we’ve long advocated: run small capable models locally for common tasks, and accept that the largest models will need remote access—whether that's a cloud API or a beefy local server.

Our guide to Best Hardware for Self-Hosted AI covers the practical realities of matching model size to available compute, and the Private AI vs Cloud AI comparison helps frame when each approach makes sense.

The distillation angle

Apple's approach relies heavily on model distillation—training a smaller "student" model to mimic a much larger "teacher." This technique is well-known in the self-hosted community, where quantised and distilled models like Phi-4, Gemma 3, and Llama 3 instruct-tuned variants deliver surprisingly good results on consumer hardware.

For teams interested in running Google's open-weight Gemma models locally, see our guide to Gemma 3 Local Setup.

Source

**Ars Technica:** https://arstechnica.com/ai/2026/05/apple-reportedly-trying-to-distill-googles-multi-trillion-parameter-gemini-ai-to-run-on-iphone/