Deep Learning on Mobile Devices:  What's New in 2022?

08:00 Pacific Time   ┈   Andrey Ignatov   ┈   AI Benchmark Project Lead, ETH Zurich

Biography: Andrey Ignatov is the project lead of the AI Benchmark initiative targeted at performance evaluation of mobile, IoT and desktop hardware at ETH Zurich, Switzerland. His PhD research was aimed at designing efficient image processing models for smartphone NPUs / DSPs and developing the next-generation deep learning based mobile camera ISP solution. He is the lecturer on Deep Learning for Smartphones course at ETH Zurich and the main author of the AI Benchmark papers describing the current state of deep learning and AI hardware acceleration on mobile devices. He is a co-founder of the AI Witchlabs and co-organizer of the NTIRE and AIM events. His main line of research is focused on image restoration and automatic image quality enhancement, adaptation of AI applications for mobile devices and benchmarking machine learning hardware.

Abstract: In this tutorial, we will first recall all basic concepts, steps and optimizations required for efficient AI inference on mobile devices. Next, we will go into more detail about the latest mobile platforms from Qualcomm, MediaTek, Google, Samsung, Unisoc and Apple released during the past year, and will compare their performance in real-world computer vision AI tasks. We will also review the recent Android AI software stack updates, and will compare the deployment of TensorFlow Lite models on Android and iOS. Finally, we will talk about the power efficiency of mobile chipsets and their NPUs, and will analyze energy consumption of all popular Android SoCs.

Mobile AI Trend and The Power Performance Metric

09:15 Pacific Time   ┈   Dr. Allen Lu   ┈   Senior Director, Computing and AI Technology Group at MediaTek

09:45 Pacific Time   ┈   Cheng-Ming Chiang   ┈   Technical Manager, MediaTek Inc.

Biography: Dr. Lu is the Senior Director of Computing and Artificial intelligence Technology Group at MediaTek. He is responsible for the ML/DL algorithm/tool development for mobile phone, camera, tablet, and TV products. Prior to MediaTek, Dr. Lu served as the General Manager of Video IoT (iVoT) Business Unit at Novatek. He was responsible for business and technology planning and execution. Prior to Novatek, Dr. Lu funded Afatek, a fabless company developing RF, digital modulator, and demodulator chips. Afatek was acquired by iTE in 2008.

Dr. Lu worked in Silicon Valley from 1997-2002. He joined the Excess Bandwidth Corp. specializing in signal processing. Excess Bandwidth Corp. was acquired by Virata Corp (now Conexant) in 2000. Dr. Lu was a member of technical staff at Hewlett Packard prior to Excess Bandwidth.

Dr. Lu received his M.S. and Ph.D. in Electrical Engineering from Stanford University. He published more than 30 papers in the optical communication and photonic switching fields.

Cheng-Ming Chiang is the Technical Manager of Computing and Artificial intelligence Technology Group at MediaTek. He acts as an AI algorithm architect and is responsible for Algorithm-Software-Hardware co-design for AI accelerator. He co-defines next generation AI accelerator with hardware architects from algorithm’s point of view. He also guides customers and ecosystem to deploy their deep learning models on MediaTek APU.

Cheng-Ming Chiang is the program committee of NTIRE and AIM Workshops. He also co-organizes challenges in Mobile AI Workshops. He received both his M.S. and B.S. in Electronic Engineering from National Chiao Tung University in Taiwan in 2013 and 2011, respectively.

Abstract: Mobile AI is evolving from short-burst applications like photos to sustained ones such as videos, games, and VR/MR. The model complexity increases dramatically over time. To enable new mobile AI applications, a combination of process advances, hardware architecture improvements, and computation complexity reduction is required. Running bigger models with higher-resolution data under a limited power budget is the most critical challenge for mobile AI applications. The current performance metric based on the minimum inference time, peak performance metric, is a good measurement for short-burst AI applications, but not the appropriate metric for sustained AI applications because it does not consider the most critical power budget constraint. We suggest a power-performance curve (power curve) to derive the sustained performance metric at a fixed power budget and to address the mobile AI’s trend from peak performance to sustained performance. We will set up a video super resolution challenge to promote the sustained performance metric.

A Practical Guide to Getting the DNN Accuracy You Need and the Performance You Deserve

10:00 Pacific Time   ┈   Felix Baum   ┈   Director Product Management, Qualcomm Technologies

Biography: Felix Baum is responsible for AI Software Products at Qualcomm Technologies Inc. (QTI). Felix has spent 20+ years in the embedded industry, both as an embedded developer and as a product manager. He previously led QTI product management for the Hexagon Software supporting DSPs with scalar, vector and tensor accelerators for camera, video, machine learning and audio verticals. Prior to that, he led marketing and product management efforts for various real-time operating system technologies. His career began at NASA's Jet Propulsion Laboratory at the California Institute of Technology, designing flight software for various spacecraft and managing integration for a launch campaign of the GRACE mission. Felix holds a master's degree in computer science from the California State University at Northridge and a Master of Business Administration from the University of California at Los Angeles.

Abstract: Every day, developers struggle to take DNN workloads that were originally developed on workstations and migrate them to run on edge devices. Whether the application is in mobile, compute, IoT, XR or automotive, most AI developers start their algorithm development in the cloud or on a workstation and later migrate to on-device as an afterthought. Qualcomm is helping these developers on multiple fronts-democratizing AI at the edge by supporting frameworks and data types that developers are most familiar with, and at the same time building a set of tools to assist sophisticated developers who are taking extra steps to extract the best performance and power efficiency. In this session, we present the workflow and steps for effectively migrating DNN workloads to the edge. We discuss quantization issues, explore how the accuracy of models affects performance and power and outline the Qualcomm tools that help developers successfully launch new use cases on mobile and other edge devices.

Optimizing TensorFlow on Intel® Core platforms

10:45 Pacific Time   ┈   Vivek Kumar   ┈   Senior AI Software Architect at Intel

11:00 Pacific Time   ┈   Manuj Sabharwal   ┈   Senior AI Software Engineer at Intel

11:15 Pacific Time   ┈   Ramesh AG   ┈   Principal Engineer at Intel

Biography: Expert in embedded solutions for telecommunications, video processing and artificial intelligence. Co-founded start-ups and lead teams of more than 70 engineers distributed worldwide. 20 years of experience leading global teams and organizations, Executive MBA from London Business School and MEng in Telecommunications.

Abstract: Intel optimization with TensorFlow on Hybrid architecture Intel is collaborating with Google for TensorFlow optimizations enabled via Intel oneDNN to accelerate key performance-intensive operations such as convolution, matrix multiplication, and batch normalization using the AI acceleration instructions Intel(R) AVX2 and Intel(R) DL Boost. Intel oneDNN is an open-source, cross-platform performance library for DL applications. Additional Intel optimizations include aggressive op fusions for optimal cache performance. In this tutorial, we will show basic concepts, steps and optimizations required to improve performance on Intel Hybrid core processor AlderLake. This session also covers quantization techniques and tools to boost model inference performance using low precision, sparsity and quantization to leverage the 8-bit acceleration instructions on Intel CPUs.

SyNAP framework: Optimizing the AI model inference on the Edge

11:30 Pacific Time   ┈   Abdel Younes   ┈   Technical Director, Smart Home Solutions Architecture at Synaptics

Biography: Abdel Younes is currently leading the AI/ML system architecture team at Synaptics and the development of the SyNAP framework that supports AI on Synaptics edge SoC. From Aerospace Research to AI, going through OpenSource, Deeply Embedded SW development and Multimedia, Abdel Younes has 20+ years of experience leading projects and teams in various industries. He received his MS in Telecommunications and Electronics and his Ph.D. in Signal Processing and Telecommunication from the French Civil Aviation Academy (ENAC).

Abstract: AI on the edge presents multiple technical challenges like the availability of quantized model and the integration into real-world application; this presentation will focus on the runtime performance of Machine Learning models execution in terms of speed and memory. We will covers tools and techniques used to optimize model inference to get the most of the NPU embedded on the Synaptics VS680 System-On-Chip, including layer by layer profiling and compilation caching. and general and detect bottlenecks in runtime execution of AI models. We also describe the SyNAP runtime software stack, starting from the Android NNAPI down to the secure TrustZone driver of the NPU.

Performance Test of AI Server Systems

12:30 Pacific Time   ┈   Cao Xiaoqi   ┈   Senior Engineer, Huawei

Biography:

Abstract: This talk provides an overview to an IEEE standard "P2937 Standard for performance benchmarking for AI server systems" under development. The standard provides a set of metrics, workloads and rules for not only benchmarking but also finding the possible bottlenecks of systems' performance for optimization. Hard issues, such as energy cost isolation, in performance test paradigm are also included in this talk.

Denoising as a Building Block for Imaging, Inverse Problems, and Machine Learning

13:00 Pacific Time   ┈   Peyman Milanfar   ┈   Principal Scientist / Director at Google Research

Biography: Peyman is a Principal Scientist / Director at Google Research, where he leads the Computational Imaging team. Prior to this, he was a Professor of Electrical Engineering at UC Santa Cruz from 1999-2014. He was Associate Dean for Research at the School of Engineering from 2010-12. From 2012-2014 he was on leave at Google-x, where he helped develop the imaging pipeline for Google Glass. Most recently, Peyman's team at Google developed the digital zoom pipeline for the Pixel phones, which includes the multi-frame super-resolution (Super Res Zoom) pipeline, and the RAISR upscaling algorithm.

Peyman received his undergraduate education in electrical engineering and mathematics from the University of California, Berkeley, and the MS and PhD degrees in electrical engineering from the Massachusetts Institute of Technology. He holds 20 patents. He founded MotionDSP, which was acquired by Cubic Inc.

Peyman has been keynote speaker at numerous technical conferences including Picture Coding Symposium (PCS), SIAM Imaging Sciences, SPIE, and the International Conference on Multimedia (ICME). Along with his students, he has won several best paper awards from the IEEE Signal Processing Society, including the 2021 Best paper award for "Kernel Regression in Image Processing and Reconstruction".

He is a Distinguished Lecturer of the IEEE Signal Processing Society, and a Fellow of the IEEE "for contributions to inverse problems and super-resolution in imaging."

Abstract: Denoising is one of the oldest problems in imaging. In the last decade, the quality of denoising algorithms has reached phenomenal levels – almost as good as we can ever hope. There are thousands of papers on this topic, and their scope is vast and approaches so diverse that putting them in some order (as I will do) is both useful and challenging. I'll describe what we can say about this general class of operators, and what makes them so special. I will argue that denoising is increasingly important, not just as a process for removing noise, but especially now as a core engine and building block for much more complex tasks in imaging, inverse problems, and ML.

CLIP4Hashing: Unsupervised Deep Hashing for Cross-Modal Video-Text Retrieval

13:30 Pacific Time   ┈   Yikang Li   ┈   Research Scientist at OPPO

Biography:

Abstract: With explosively increasing amount of data on social media platforms like Twitter and TikTok, fast and accurate cross-modal retrieval on mobile devices is gaining much attention. Compared with the traditional cross-modal retrieval algorithms that work in the continuous feature space, hashing-based algorithms can retrieve data faster and more efficiently. However, most existing algorithms have difficulties in seeking or constructing a well-defined joint semantic space. In this paper, an unsupervised deep cross-modal video-text hashing approach, CLIP4Hashing, is proposed, which mitigates the difficulties in bridging between different modalities in the Hamming space through building a single hashing net by employing the pre-trained CLIP model. The approach is enhanced by two novel techniques, the dynamic weighting strategy and the design of the min-max hashing layer, which are found to be the main sources of the performance gain. With evaluation using three challenging video-text benchmark datasets, we demonstrate that CLIP4Hashing is able to significantly outperform existing state-of-the-art hashing algorithms. Additionally, with larger bit sizes (e.g., 2048 bits), CLIP4Hashing can even deliver competitive performance compared with the results based on non-hashing features.