AI-powered smartphones – smarter or snoopier?
The AI-powered smartphones of 2023 and 2024 all claim amazing AI features. Unfortunately, very little of the market was ready to pay for AI or embrace more than a few of its parlour tricks.
Since the beginning of 2024, CyberShack has written over 150 reviews and articles on AI-equipped tech. We were inundated by readers who didn’t want AI in anything – computers, fridges, TVs, etc., and we don’t blame them. AI may bring benefits but at the expense of privacy and other liberties. YouGov research states that over 60% of US adults believe AI is just another data harvester, which is fundamentally correct.
Although some things do work better, like robot vacuum/mops that can see and avoid ‘pet poo’ and security cameras that can recognise people, animals and cars – useful AI features. But it is not really AI. It is machine learning intelligence with programmed responses. Passive AI is good – something that is smarter than us and gives a better outcome than we could achieve. Active AI, well, it could be bad!
The two areas that have gotten way out of control are computers and smartphones (essentially pocket computers).
When you add Google, Microsoft, Amazon, and Apple to the AI cake mix, you get Gemini, Copilot, Alexa, and a smarter Siri underpinned by massive AI clouds that do much of the work. And let’s not forget Intel, AMD, Qualcomm, and other chip makers that see AI as a new revenue source.
At the risk of sounding like a Luddite (and as a tech journalist, I am not), most readers still think AI is a gimmick they don’t need and don’t want to pay for. Ergo, a phone is a phone until killer AI apps actually provide killer solutions. YouGov found that 77% strongly agree that phone reception strength is the #1 reason to buy. We test for that – see Best Android phones 11/2024.
More reading:
- What is AI (Artificial Intelligence), how will it affect me?
- AI PCs are coming. What does that really mean?
- Gemini – Google Assistant’s split personality
- What is Microsoft Copilot – the unthinking person’s guide
- Microsoft 365 price increase – do you want to pay for AI?
AI-powered smartphones
Google’s Pixel 8 in 2023 was the first real AI phone with a dedicated Neural Processing Unit (NPU) capable of processing AI faster than its CPU or GPU could. Samsung followed with its Galaxy S23 using Google’s Gemini AI.
In 2024, we saw Google Pixel 9 and Samsung S24 Ultra battling for AI supremacy. Recently, OPPO Find X8 Pro – at last, flagship competition took the AI-power crown with a new processor and NPU.
In 2025, the battle will extend to any Android 14 or 15 phone, with an NPU and Google Gemini Nano performing most of the on-device AI, such as photo enhancement, translation, and text manipulation. Heavy-lifting AI is cloud-based and needs an internet connection over Wi-Fi or mobile data. Google says Gemini Nano will run on Android 10 or later with as little as 2GB of RAM. Our experience is that you need much more power for on-device processing.
Now, Apple Intelligence is on its new devices, creating a digital iPhone divide—it will only run on new Apple silicone. Apple is well behind on AI features but is selling more new and expensive iPhones based on its AI promise.
That got us thinking about what enables AI-powered smartphones
Since early 2024, our phone reviews have attempted to measure AI power so readers can decide what they need.
Initially, we used the industry standard TOPS (Trillion Operations per Second), but strangely, it has disappeared from use precisely because it could be used as an accurate benchmark. It is better for AI phone makers to obscure AI performance and rely on marketing hype.
The real answer is that it depends on the size of the AI model you run on the device. The smaller the model, the faster it runs—more jiggery-pokery!
So, we gathered up some AI phones to test using different synthetic benchmarks to see if there was any pattern.
Test sample phones
All were at out-of-the-box defaults, usually a balanced battery and connected to the charger, as our experience is that Performance mode is not that much better.
- 2024 Motorola G75 (8GB Qualcomm SD6 Gen 3). This is more to test the lower end.
- 2023 Google Pixel 8 Pro with a Tensor G3 processor (12GB based loosely on the Samsung Exynos 2100)
- 2024 Google Pixel Pro 9 XL with a Tensor G4 processor (16GB based loosely on the Samsung Exynos 2400). We also verified this against other Pixel 9 models with the same processor.
- 2024 Xiaomi POCO F6 Pro (12GB Qualcomm SD8 Gen 2)
- 2024 Samsung S24 FE (8GB Samsung Exynos 2400e)
- 2024 Samsung Galaxy S24 Ultra (12GB Qualcomm Snapdragon8 Gen 3)
- 2024 Samsung Galaxy Z Fold 6 (12GB Qualcomm Snapdragon8 Gen 3)
- 2024 OPPO Find X8 Pro (16GB MediaTek Dimensity 9400)
Test apps
- Geekbench AI measures: CPU, GPU, NNAPI (NPU).
- AiTuTu: Loads several large language modules but gives some inconsistent results.
- Ai Bench 6: Ditto, but not as comprehensive.
- GFLOPS: Giga Floating-Point Operations per Second and is Integer Operations Per Second, measurement of Arithmetic Logic Unit
- GINOPS: Giga Integer Operations Per Second, measurement of Arithmetic Logic Unit
Phones correspond to the numbers above
| Phone | GB CPU | GB GPU | GB NPU | AiTuTu | Ai Bench | GFLOPS | GNOPS |
| 1 SD6 3 | 1304 | 245 | 136 | 52,172 | 1014 | 13.52 | 14.51 |
| 2 G3 | 1426 | 589 | 4749 | 92,185 | 832 | 14.96 | 21.19 |
| 3 G4 | 3110 | 869 | 7063 | 102,019 | 1045 | 14.45 | 19.10 |
| 4 SD8 2 | 1878 | 953 | 561 | 1,056,264 | 2439 | 14.24 | 18.53 |
| 5 EX2400 | 3522 | 1364 | 1016 | 40,315 | 5810 | 21.27 | 26.14 |
| 6 SD8 3 | 3008 | 1362 | 1010 | 1,469,605 | 8355 | 18.43 | 28.18 |
| 7 SD8 3 | 3758 | 1278 | 12,713 | 1,481,101 | 8161 | 21.4 | 26.15 |
| 8 MT9400 | 2287 | 1059 | 11,905 | 215,900 | 11,147 | 14.06 | 19.85 |
What does this mean for AI-powered smartphones?
Some Samsung results are so outlandish that you could be excused for thinking the manufacturer was gaming the results. Reddit et al. still report that Samsung appears to be ‘gaming’ Geekbench results using its Game Optimising Service (GOS) and engaging super high-speed mode during these recognised tests. Samsung blames Qualcomm.
I suspect we can only rely on Tensor and MediaTek results until we see Qualcomm SD 8 Gen 4 (Elite) results. Interestingly, Google is stepping away from using Samsung’s Exynos and manufacturing as its base and moving to Taiwan’s TSMC using a 3nm die (like the MediaTek Dimenstiy 2400).
It shows that 12GB of RAM is the minimum, and 16GB is better. In fact, Google’s Pixel 9 Pro XL has 16GB of RAM and quarantines 4GB solely for AI use, which will help to future-proof this phone.
Summary: In 2024, Qualcomm SD 8 Gen 3 or later, Tensor G3 or later, and MediaTek Dimensity 9400 or later will be the only AI-capable phones.
Raw power
So, we know now what processor you need for a better AI on-device experience. However, we wondered if there was any correlation between other synthetic benchmarks.
We compared more standard synthetic results from
- Geekbench (GB) single/multi-core
- OpenCL – GPU
- Vulkan – GPU
- GIPS (Giga Instructions Per Second)
| GB Single | GB Multi | OpenCL | Vulkan | GIPS | |
| 1 SD6 3 | 963 | 2485 | 1755 | 2257 | 231,363 |
| 2 G3 | 1776 | 4606 | 5286 | 5963 | 290,498 |
| 3 G4 | 1888 | 4294 | 6553 | 6982 | 332,671 |
| 4 SD8 2 | 1995 | 5750 | 7056 | 7495 | 338,542 |
| 5 EX2400 | 1941 | 6064 | 15,544 | 15,726 | 393,107 |
| 6 SD8 3 | 2198 | 6728 | 12,316 | 14,339 | 404,359 |
| 7 SD8 3 | 2230 | 6830 | 13,290 | 14,016 | 384,028 |
| 8 MT9400 | 2750 | 8344 | 20,667 | 24.893 | 400,759 |
What does this mean?
Simply, it denotes raw processing power, and higher is better.
Interestingly, the Tensor G4 is slightly slower (GB Multi-core) than the G3 it replaced but has higher GIPS. The results are also more consistent with the Qualcomm SoCs.
Summary: It is not just about the NPU but the CPU’s and GPU’s raw power, which handles phone features, photography post-processing computational things and screen resolution/refresh. Again, it is a small sample, but GIPS over 300,000 (higher the better) is the best base indicator for AI-powered smartphones.
Brought to you by CyberShack.com.au
