How Apple Puts Synthetic AI in Older iPhones

According to Android Authority , Apple's research paper details a solution for running large language models (LLMs) on devices with limited RAM. The paper reveals how the company can store "model parameters" and load a portion of them into the device's RAM when needed, instead of loading the entire model into RAM.

Apple is looking to help older iPhones with less RAM run general AI

The paper claims that this method allows running models that require twice the RAM that an iPhone can have while still ensuring 4-5x and 20-25x inference speeds compared to simple loading methods on the CPU and GPU respectively.

Deploying synthetic AI on a device with more RAM would be a huge benefit because it would allow for faster read/write speeds. Speed ​​is important for on-device AI, allowing for much faster inference times because users don’t necessarily have to wait tens of seconds (or more) for a response or final result. All of this means an on-device AI assistant could potentially run at conversational speeds, generate images/text much faster, summarize articles faster, etc. But Apple’s solution means users don’t necessarily need a lot of RAM to speed up on-device AI task responsiveness.

Apple's approach could allow iPhones old and new to offer synthetic AI features right on their devices. That's important because Apple's iPhones typically offer less RAM than high-end Android phones. The iPhone 11 series, for example, only offers 4GB of RAM, while even the regular iPhone 15 has just 6GB.

Apple isn’t the only mobile company working on shrinking LLM. Qualcomm and MediaTek’s recent flagship chips both support INT4 precision to shrink these models. Either way, companies are trying to find new ways to reduce the system requirements for on-device AI, allowing even low-end phones to offer the feature.

Source link: https://thanhnien.vn/cach-apple-dua-ai-tong-hop-vao-iphone-doi-cu-185231222160500661.htm