Apple Executives discuss Apple’s Chip Philosophy
In a new report posted this morning in India, Tim Millet, vice president of Platform Architecture (Hardware Technologies) at Apple, is clear about what differentiates the company’s silicon strategy and why it is able to outperform the competition continuously.
“We are not a merchant silicon company. We do not build chips and sell them to other people. We do not make money that way. Folks who do make money that way have the burden of adding margin on top of whatever they buy,” he said.
Speaking to The Indian Express soon after the announcement of the new Apple M4 Pro and M4 Max processors, which Apple claims “feature the world’s fastest CPU core, delivering the industry’s best single-threaded performance”, Millet highlighted that merchant silicon companies “can’t just go to the latest cutting edge technology like the second generation, three nanometer…but we (Apple) benefit from it in a way that we believe it is worth it. It delivers for us and our products and our customers… we are trying to leave nothing on the table.”
Tom Boger, Vice President of Mac Product Marketing at Apple, who joined Millet on the call, underlined that it was unusual to see this “pace of innovation year after year after year” when it comes to silicon. The first M-series chip was released just four years ago. “That is the promise. That is a commitment we make to our teams to deliver innovations as they are available to us,” he said.
Apple’s secret weapon
Explaining what gives Apple silicon an edge now, Millet said, “We take advantage of the three major components, the architecture, the design, and the process technology. Our fourth tool, really our secret weapon, I think, is our ability to co-design these amazing chips with the system teams and the product designers as they are imagining possibilities.”
Millet cited the new Mac mini as a perfect example of that. “The opportunity was for us and for the design team to be able to come together and build this incredible new platform… there is no way that machine could have come to life without that collaboration. And that is really what Apple is all about,” he said.
This also manifests in a device like our new Mac mini which can for all practical purposes replace an Apple Studio computer used for video and other heavy workloads. Boger is clear this is a testament to Apple silicon. “It is a superpower in performance per watt. And we don’t make a bunch of chips and then decide where we are going to put them. We design our chips from the ground up for our products, and that is a tremendous strategic advantage that we have,” Boger said.
Always designed for AI
With Cupertino starting to roll out Apple Intelligence features in different parts of the world, Boger clarified that they have “always had intelligent features in our Mac”. “One of the things that we announced is the fact that Apple Intelligence runs on any Mac with an M series chip, right? So from the first M1 chip, we have included the Neural Engine, and we have a great architecture for AI. And we also have developers taking advantage of Apple silicon to offer our customers intelligent features. So the M Series chips were always built for AI,” he said.
But what made Apple prepare for this AI era in advance, especially when the real focus started picking up only after OpenAI announced ChatGPT in 2022?
“How did we know? I think we have to go back to 2017 when we introduced the Neural Engine in our iOS products. And really this was inspired by our recognition of the importance of computational photography,” explained Millet. “So we were seeing the amazing research that folks up in the University of Toronto were demonstrating… these new neural networks were capable of doing image recognition beyond the capacity of humans, or at least matching, and they were headed on a trajectory that was clear. And so we pounced on the opportunity to build that embedded capability into our camera processors for the phone,” he added.
But the folks at Apple realized this trend was just starting. “We knew we wanted to take our Neural Engine, centralize it, and make it a first-class module within the SOC. We build these SOCs every year and we have the opportunity to pick up the latest research, the latest direction,” Millet added.
He recalled how an “interesting paper” was published in 2017 that led to the invention of the transformer network, which then became the foundation for the LLMs that exploded in 2022. “It took about five years. Folks on my team recognized that these papers had the potential to be very interesting and might have a huge impact and made big architectural changes to the Neural Engine, just in case. We introduced the first transformer-capable SOCs in 2020 when we introduced the M1 and this all lined up perfectly,” he said.
Millet accepts that it is a “little bit of good fortune that this investment turned out to pay off. On the other hand, it shows you the diligence that we spend all our time trying to figure out where the ball is moving. We try to make sure we are there before it gets there.”
The difference between M4 Max and M4 Pro
Millet explained that when it comes to running LLMs, the big difference between the M4 Max and the M4 Pro is the memory system. “M4 Max has effectively about twice the memory bandwidth of M4 Pro. The responsiveness will come from the Neural Engine which is present in both so the compute side of it will be balanced and equal. The memory bandwidth side of it will help M4 Max for some of the bigger models to hit higher token rates. So the more complex models, actually could, in theory, run a little bit faster,” he elaborated.
But it is not that simple. “Now, truth be told… these systems are so fast it is hard to tell the difference. But for someone who was really pushing the edge for a very, very large model, they will definitely benefit from that wider memory system on the Max,” Millet said.