Transforming AI Experiences at the Edge with a System-Level Approach
The rapid evolution of AI on edge devices like laptops isn’t just transforming everyday life; it’s changing the way we think about how developers can exploit rapidly changing models to continue delivering amazing, world-changing experiences.
Developers want to land their AI-based innovations quickly at the highest possible performance. This requires a range of heterogenous capabilities delivered in a compute platform.
The criticality of CPUs
The CPU is highly accessible and easy to program, making it ideal for developing applications. And continuous investments in the Arm architecture—the foundation of the Arm CPU—have led to the development of critical features for accelerating AI computation. These efforts include dot product instructions, support for the BFloat16 format, matrix multiply instructions for Int8 and BFloat16, and the powerful SVE2 vector extension.
These advancements extend the capabilities of the latest Armv9 Cortex CPUs for applications requiring more compute performance for data processing and advanced workloads. This relentless focus on fast-paced architectural evolution prepares the developer ecosystem for ever-changing compute demands in the age of AI.
GPUs accelerate a broad array of parallelizable computations and for developers the AI benefits are accessed via APIs. Both the CPU and GPU can be seamlessly integrated with neural processing units (NPUs) in a heterogeneous system on chip (SoC) environment.
NPUs are tailored to efficiently process the operations commonly used in neural networks, such as matrix multiplications and convolutions. This specialization as an offload engine allows them to perform these tasks quickly to deliver enhanced AI performance.
However, compared to CPUs and GPUs, NPUs are complex to program and integrate into SoCs, and have limited flexibility since they are designed for specific AI tasks.
Enabling the growing developer ecosystem
One area benefitting from AI innovation is the current PC market, which is witnessing a rapid expansion of AI-based applications and features, enhancing a range of productivity and creativity tasks. AI-powered smart assistants, generative AI, and advanced AI workloads are transforming the imaging pipeline, a fundamental feature for PC video conferencing. This evolution is laying the foundation for the future growth of AI PCs.
Meanwhile, AI workloads will run more efficiently and faster within the memory bandwidth of PC designs via more mature and smaller LLMs, like Meta’s Llama3 and Microsoft’s Phi-3. Arm is already demonstrating how these new models can run entirely at the edge on the Arm CPU, with the impressive performance enabled by existing CPU instructions for AI and dedicated software optimizations.
Supporting Arm-based PCs’ strong hardware and software capabilities is a growing developer ecosystem. Thanks to the expanded end-to-end Windows Arm-native developer toolchain, more Windows developers are creating Arm-native builds of their applications. This is being driven by vendors of leading applications, including Adobe Photoshop, Dropbox, Firefox, Microsoft Edge, Office, Teams, Visual Studio, Netflix, Spotify, WhatsApp, and Zoom, all providing Arm-native builds. These investments translate into tangible application improvements, such as improved startup times and smoother performance.
Arm CPU at the forefront of AI
As the AI landscape continues to evolve, the importance of computing efficiency and design flexibility will only grow. The Arm CPU, with its continued advancements in AI capabilities, strong ecosystem support, and inherent advantages in latency and memory locality, is well-positioned to remain at the forefront of AI enablement in consumer devices.
By adopting a system-level approach that leverages the strengths of the CPU alongside other specialized processors, the industry can continue to push the boundaries of AI innovation, while delivering faster, more responsive and more secure AI experiences to users at the edge.
The rise of AI-enabled edge devices and PCs represents an exciting new era for the market, where really the only gating factor to transformative innovation is imagination. Arm is continuously evolving technologies that enable AI wherever computing happens.
For more information, please see our AI solutions page.
Any re-use permitted for informational and non-commercial or personal use only.
Editorial Contact
Latest on X
;
Frederique Olivier relies on Arm to capture the beauty of the natural world. 🌎
With the help of Arm, she can shoot images with better quality and source power in extreme conditions.
Discover the world from Frederique's eyes, made possible #onArm: https://newsroom.arm.com/blog/purpose-on-arm-frederique-olivier-photographer?utm_source=twitter&utm_medium=social-organic&utm_content=blog&utm_campaign=mk08_brand_na
🗓️ Join us on Wednesday November 6 for our next financial results conference call.
After market close, we'll report our earnings for the second quarter of fiscal year 2025, followed by a conference call to review our results and business outlook: https://newsroom.arm.com/news/arm-announces-earnings-release-date-for-second-quarter-fiscal-year-ended-2025?utm_source=twitter&utm_medium=social-organic&utm_content=newsroom&utm_campaign=mk29_exec-comms_na
6G is on the horizon, promising ultra-fast speeds, low latency, and seamless digital and physical experiences.
Learn what it takes to bring new AI capabilities to life on 6G: https://newsroom.arm.com/blog/arm-and-alphawave-semi-unite-on-6g-chiplets?utm_source=twitter&utm_medium=social-organic&utm_content=blog&utm_campaign=mk03_infrastructure_na
Experience new levels of performance and efficiency with the general availability of @GoogleCloud C4A VMs!
C4A features the Arm Neoverse V2-powered Google Axion CPU to support the most demanding AI & general purpose workloads: https://cloud.google.com/blog/products/compute/try-c4a-the-first-google-axion-processor
Congrats @GoogleCloud! 👏
Catch the full conversation to hear why suggesting AI is overhyped "couldn't be further from the truth": https://www.bloomberg.com/news/videos/2024-10-22/arm-ceo-dismisses-ai-bubble-fears-video
#BloombergTech
"AI is going to run on Arm. Period."
Arm is the only company on the planet that can run workloads of any size. There's plenty of opportunity to grow and drive innovation in the space, and we're leading this.
Thanks to #BloombergTech for having us!
At #GitHubUniverse or watching the live stream?
➡️Learn how to maximize performance with GitHub Copilot and GitHub Actions. Catch the team on the Main Stage at 3:45pm PT!
➡️Attending in person? Join our AI roundtable later today - spaces are limited! https://www.arm.com/resources/registration/github-universe-roundtable
Get ready to streamline development with the Arm extension for @GitHub Copilot!
Coming 🔜, the extension will empower developers to build and test software quickly and efficiently on the Arm compute platform, supporting seamless cloud-to-edge deployment: https://newsroom.arm.com/news/ai-driven-tools-on-github?utm_source=twitter&utm_medium=social-organic&utm_content=newsroom&utm_campaign=mk24_developer_na
Ready to future-proof your workloads?
Boost performance and cut costs by migrating to Arm. 💪
Companies like @HoneycombIo and @FusionAuth saw major gains with AWS Graviton and so can you. Explore how and access our migration guide to get started: https://newsroom.arm.com/blog/why-developers-are-migrating-to-arm?utm_source=twitter&utm_medium=social-organic&utm_content=blog&utm_campaign=mk24_developer_cloud-dev
Panasonic Automotive Systems joined us to discuss how AI is revolutionizing the in-vehicle experience in our latest Viewpoints podcast episode.
Discover how AI is shaping the future of driving, making it safer, more intuitive and more comfortable: https://newsroom.arm.com/podcasts/ai-in-the-car?utm_source=twitter&utm_medium=social-organic&utm_content=podcast&utm_campaign=mk12_automotive_na
The Armv9 Cortex-X925 CPU is setting new standards with a 15% IPC improvement!
Get ready for unparalleled performance and efficiency across flagship AI smartphones and PCs. More details: https://newsroom.arm.com/blog/armv9-cortex-x925-cpu-performance?utm_source=twitter&utm_medium=social-organic&utm_content=blog&utm_campaign=mk04_client_na
For @NVIDIA, transparency is the key to success. 🔑
To build one of the largest companies in the world, NVIDIA CEO Jensen Huang sought to build a porous organization that empowers people to experiment and improve.
Listen to the full Tech Unheard podcast: https://newsroom.arm.com/podcasts/tech-unheard-episode-one-jensen-huang?utm_source=twitter&utm_medium=social-organic&utm_content=podcast&utm_campaign=mk08_brand_na
The Arm CPU architecture is enabling quicker AI experiences with enhanced security, unlocking new possibilities for AI workloads at the edge.
Find out why the CPU is front and center in the future of AI on The @Verge.
Why AI starts on the CPU
The rise of AI inference means more AI workloads are being processed at the edge.
www.theverge.com
It's a big day for small models!
With @AIatMeta's new quantized Llama models and the Beta release of ExecuTorch, it's now quicker for 20 million Arm developers to create and deploy performant, intelligent gen AI at the edge.
👇We've put it to the test: https://newsroom.arm.com/news/accelerating-edge-ai-with-executorch?utm_source=twitter&utm_medium=social-organic&utm_content=newsroom&utm_campaign=mk13_tech-strat_na
We're excited to see the new quantized Llama 3.2 1B and 3B models available on the Arm compute platform.
👉 Developers, get ready to seamlessly integrate these new models into your applications #onArm with no additional modifications or optimizations, saving time and resources
Our collaboration with @vivo_Global and @MediaTek is redefining the AI smartphone. 📱
🤝 Together, we've delivered advanced performance and efficiency to vivo's X200 series with MediaTek's Dimensity 9400 with the latest Armv9 CPUs and Immortalis GPU: https://newsroom.arm.com/news/powering-next-generation-ai-smartphones-mediatek-vivo?utm_source=twitter&utm_medium=social-organic&utm_content=newsroom&utm_campaign=mk04_client_na
Collaboration is key to enabling the AI-enabled vehicle. 🚗
At #REAutoUSA, Dipti Vachani, our SVP and GM for Automotive shared how we're working across the stack to deliver solutions that enable software development from day 1, enabled by standards driven by SOAFEE.
Thank you, #GITEXGlobal, for including us to speak on this moment in technology where we can truly make a difference. Tamika Curry Smith was on the ground to share our commitments around #DEI and #AI.
The future is bright and built #onArm.
It's almost time! ⏰
In just a few hours, Arm CEO Rene Haas joins #BloombergTech to discuss everything from Arm's story so far to the opportunity in AI and what's next for the semiconductor industry.
👇 Watch the stream at 2:35 pm BST on @BloombergLive: https://x.com/i/broadcasts/1OyKAZovbvLGb
🏆 We've won the 2024 @Reuters Automotive D.R.I.V.E Honours Award for Innovation!
This award recognizes our commitment to advancing automotive design and production with our automotive technologies and CSS roadmap, launched back in March: https://1.reutersevents.com/LP=37549 #DRIVEHonours
We're transforming the automotive industry.
Our compute platform is key to developing SDVs with enhanced capabilities and safety features that will help enhance driving experiences. 🚗
Discover more on our blog ➡️: https://newsroom.arm.com/blog/software-defined-vehicles-future-automotive-industry?utm_source=twitter&utm_medium=social-organic&utm_content=blog&utm_campaign=mk12_automotive_na
Thanks for having us, #OCPSummit24!
With demos, daily speaker sessions, & new chiplet announcements with Arm Neoverse CSS, we showed how to tackle the scalability, design & development challenges of AI on the Arm compute platform.
Catch our highlights: https://newsroom.arm.com/blog/key-takeaways-from-ocp-global-summit-2024?utm_source=twitter&utm_medium=social-organic&utm_content=blog&utm_campaign=mk03_infrastructure_na
Join Rene Haas next week at #BloombergTech to hear how we're building the future of computing #onArm.
🔴🎥 Catch Rene in person on October 22 or tune in to the live stream via @BloombergLive at 2:35 pm BST. https://events.bloomberglive.com/bloomberg-technology-summit-london-2024/agenda/speakers/3407898
✅ Seamless compute scalability
✅ Trusted safety and security
✅ Accelerated software readiness
The Arm compute platform places us at the forefront of technology to deliver scalable, efficient, and innovative automotive solutions.
More in our blog 👉: https://newsroom.arm.com/blog/accelerating-adas-adoption?utm_source=twitter&utm_medium=social-organic&utm_content=blog&utm_campaign=mk12_automotive_na
