On-Device AI Craves RAM
Ron Amadeo, writing for Ars Technica on a purported leak of a trio of Pixel 9 phones:
Rozetked says (through translation) that the phone is “similar in
size to the iPhone 15 Pro.” It runs a Tensor G4 SoC, of
course, and — here’s a noteworthy spec — has a whopping 16GB of
RAM according to the bootloader screen. The Pixel 8 Pro tops out
at 12GB.
Anything could change between prototype and product, especially
for RAM, which is usually scaled up and down in various phone
tiers. A jump in RAM is something we were expecting though. As
part of Google’s new AI-focused era, it wants generative AI models
turned on 24/7 for some use cases. Google said as much in a recent
in-house podcast, pointing to some features like a new
version of Smart Reply built right into the keyboard, which
“requires the models to be RAM-resident” — in other words, loaded
all the time. Google’s desire to keep generative AI models in
memory means less RAM for your operating system to actually do
operating system things, and one solution to that is to just add
more RAM. So how much RAM is enough? At one point Google said the
smaller Pixel 8’s 8GB of RAM was too much of a “hardware
limitation” for this approach. Google PR also recently
told us the company still hasn’t enabled generative AI smart reply
on Pixel 8 Pro by default with its 12GB of RAM, so expect these
RAM numbers to start shooting up.
That last link is to a story positing that Google’s Gemini Nano runs on the Pixel 8 Pro but not the regular Pixel because the Pro has more RAM (12 vs. 8 GB).
Comparing iPhone RAM to Android RAM has never been apples-to-apples (same goes for battery capacity), but still, it’s hard not to wonder whether Apple’s on-device AI plans are hamstrung by the relatively stingy amounts of RAM on iPhones. Here’s a list from 9to5Mac showing the RAM in each iPhone going back to the original (which had just 128 MB!). iOS 17 supports models dating back to 2018’s iPhone XS and XR (4 and 3 GB of RAM, respectively). If iOS 18 drops those models, the new baseline will be the iPhones 11 and 11 Pro, which all sport 4 GB. The most RAM on any iPhones to date is the 8 GB in the 15 Pro models, but 8 GB is what Google deemed insufficient for the Pixel 8 to run Gemini Nano.
Might some iOS 18 on-device AI features be limited to newer models with more RAM?
★
Ron Amadeo, writing for Ars Technica on a purported leak of a trio of Pixel 9 phones:
Rozetked says (through translation) that the phone is “similar in
size to the iPhone 15 Pro.” It runs a Tensor G4 SoC, of
course, and — here’s a noteworthy spec — has a whopping 16GB of
RAM according to the bootloader screen. The Pixel 8 Pro tops out
at 12GB.
Anything could change between prototype and product, especially
for RAM, which is usually scaled up and down in various phone
tiers. A jump in RAM is something we were expecting though. As
part of Google’s new AI-focused era, it wants generative AI models
turned on 24/7 for some use cases. Google said as much in a recent
in-house podcast, pointing to some features like a new
version of Smart Reply built right into the keyboard, which
“requires the models to be RAM-resident” — in other words, loaded
all the time. Google’s desire to keep generative AI models in
memory means less RAM for your operating system to actually do
operating system things, and one solution to that is to just add
more RAM. So how much RAM is enough? At one point Google said the
smaller Pixel 8’s 8GB of RAM was too much of a “hardware
limitation” for this approach. Google PR also recently
told us the company still hasn’t enabled generative AI smart reply
on Pixel 8 Pro by default with its 12GB of RAM, so expect these
RAM numbers to start shooting up.
That last link is to a story positing that Google’s Gemini Nano runs on the Pixel 8 Pro but not the regular Pixel because the Pro has more RAM (12 vs. 8 GB).
Comparing iPhone RAM to Android RAM has never been apples-to-apples (same goes for battery capacity), but still, it’s hard not to wonder whether Apple’s on-device AI plans are hamstrung by the relatively stingy amounts of RAM on iPhones. Here’s a list from 9to5Mac showing the RAM in each iPhone going back to the original (which had just 128 MB!). iOS 17 supports models dating back to 2018’s iPhone XS and XR (4 and 3 GB of RAM, respectively). If iOS 18 drops those models, the new baseline will be the iPhones 11 and 11 Pro, which all sport 4 GB. The most RAM on any iPhones to date is the 8 GB in the 15 Pro models, but 8 GB is what Google deemed insufficient for the Pixel 8 to run Gemini Nano.
Might some iOS 18 on-device AI features be limited to newer models with more RAM?