On-Device Processing and AI Go Hand-in-Hand

Whether operating an autonomous vehicle, using facial recognition to access your bank account, or keeping your device safe from fast-changing security threats, artificial intelligence (AI) is playing a bigger role in how we use our mobile devices. In the past the power of the cloud was required for this processing, but mobile devices are now equipped to run compute-intensive AI operations. In fact, in many cases, the mobile device is the preferred platform for running AI-powered applications.

The reality is that more AI applications are emerging today than most of us realize. When a mobile device is equipped with AI, it can vastly expand and improve our lives, whether by capturing sharper images and videos, communicating with us more naturally, or perceiving the environment and autonomously navigating us to our destination safely.

“AI is an umbrella term designed to encompass anything that helps a device replicate the human brain,” says Gary Brotman, director of product management for Qualcomm. “Machine learning (ML) is a broad class of techniques and algorithms to solve the problems that make AI possible. The class we focus on is deep learning, deep neural networks, and recurrent neural networks, running on the actual device.”

The significant improvement of AI algorithms and on-device processing, two crucial ingredients for making AI ubiquitous, are leading to more seamless and compelling user experiences. This is particularly true as AI-based functionality moves to vehicles, household devices, and Internet of Things (IoT) sensors. All the functions that fall under the AI umbrella, such as ML, deep learning, and reinforcement learning, support the enhanced perceptive and cognitive capabilities on modern mobile devices.

For example, mobile AI can improve image recognition and advanced image processing, such as producing bokeh effects (a soft out-of-focus background) and style transfers. AI-equipped devices can also learn to recognize keywords and voices, improving their response to the consumer and aiding in foreign language translation.

In addition, AI can help mobile devices and apps become more aware of user preferences and surroundings, understand intent, and respond in contextually relevant ways. “AI on your device results in a more contextually rich experience,” says Brotman. “And over time your device will be able to predict and have a deeper understanding of what you’re going to do next.”

AI in Your Hand

On-device AI has several significant benefits. “The first is performance. Processing on the device is just faster—no roundtrip to the cloud,” says Brotman. “Privacy is next. People are comfortable sharing some personal data, but not all of it. And the third is reliability. Mobile networks are pervasive, but there’s no guarantee you’ll always have a connection.”

• Performance: Running AI algorithms on the device—independent of the cloud—can greatly improve response time and efficiency, as data doesn’t need to be transferred between the cloud and the device. This is important because mobile AI capabilities tend to be time-sensitive for user experience and decision making.

“AI apps tend to be real-time and mission-critical,” says Jeff Gehlhaar, vice president of technology for Qualcomm. “Many AI-use cases that enhance an experience can’t afford latency.” 

An autonomous vehicle for example that needs to apply its brakes, for example, can’t afford even a millisecond of latency that might result from cloud processing. Decisions must be made in a split second for the vehicle to operate safely.

In terms of user experience, a natural voice user interface can only tolerate so much latency. Users are accustomed to immediate responses when using a natural language processing speech interface, and the repercussions of network delays will lead to poor experiences.

• Privacy and security: Keeping your data on the device ensures privacy, and AI is also used for biometric authentication using voice, fingerprint, iris, and facial recognition. “Using your face to unlock a device is becoming commonplace,” says Brotman. “And 3D facial recognition is emerging to provide a higher degree of authenticity for enabling mobile payments.”

Mobile devices that process AI applications can also increase both device and data security by maintaining a watchful eye for aberrant behavior. “AI can help detect malware and anomalous behavior,” says Gehlhaar. “We can train the neural network to see how bad actors behave. And it can detect those bad behaviors, like asking, ‘Why is my camera opening my contact database?’

• Reliability: Even in the most advanced areas of the world, mobile networks can fail. When it comes to AI-driven mobile capabilities, however, there’s no room for error. Autonomous vehicles simply can’t afford to experience a dropped wireless signal, such as might occur upon entering a tunnel or parking garage. On-device processing, in addition to other redundancy features, will always be a requirement for mission-critical usages like autonomous driving.

Making AI Mobile

While these AI functions can now run on the device, the cloud still has a role, particularly as a complement to on-device processing. AI apps still rely on cloud platforms to manage big data and to “train” the neural network models that drive AI inference.

Mobile devices themselves must also be equipped to effectively run AI workloads. For example, the processing must happen within the platform’s constraints, including power consumption and thermal limits. Application processors with diverse compute engines are particularly well-suited for efficiently running AI tasks. The Qualcomm Snapdragon Mobile Platform, for example, is equipped with three separate processing engines—a central processing unit (CPU), graphics processing unit (GPU), and digital signal processor (DSP) with vector processing capabilities—all of which play key roles in on-device AI.

“With heterogenous computing, there are a variety of different engines within the chip to most efficiently process a given task,” says Pat Lawlor, technical marketing staff manager at Qualcomm. “The CPU, GPU, and DSP have different strengths and weaknesses, and they can work together or separately, depending on the AI task. They complement each other, and the AI tasks run on the appropriate engines for high performance at low power.”

The increased processing power built into the chipsets of modern mobile devices help them handle the intense processing. For example, the Qualcomm Hexagon 685 DSP, Adreno 630 GPU, and Kryo 385 CPU in the Snapdragon 845 can deliver up-to two to three times faster AI processing over the previous generation. The Hexagon DSP, for example, was originally designed for vector math-intensive workloads like audio processing and continues to be enhanced to address AI workloads. This accelerates running neural networks that are part of AI inference.

What’s Next for Mobile AI?

Mobile AI is a rapidly growing market. With continual advancements in neural networks, deep learning algorithms, and hardware design, we will see vast improvements in accuracy and speed, plus new, immersive user experiences.

In the broader universe of mobility, 5G wireless networks are also on the horizon. “AI will improve and augment 5G and vice versa,” says Brotman. “5G will enable devices to more freely communicate with each other to share data and share context.” With this development, we will experience a fully connected universe of intelligent mobile devices, facilitating more personalized, real-time user experiences.

Our lives today are made richer by the capabilities of our mobile devices, and our future will increasingly be improved by the advancements being made in AI. The convergence of these two powerful trends is already shaping how we will experience our personal and business lives in the very near future.

To learn more about on-device AI, visit qualcomm.com/artificial-intelligence.