Over the years there have been many headlines regarding teens having been killed, maimed or severely injured due to wearing earbuds and not paying attention to their surroundings, especially near crosswalks. One report pointed to the Toronto Police mentioning Apple's AirPods as a problem, though it's more of generic earphones issue in general. The issue is a global issue, with another major case in  the UK in 2020.

While Apple has focused on advancing health sensors to their AirPods, Huawei, Apple's competitor in China, has filed for a patent wherein their future audio earwear devices may integrate world cameras and use AI to detect intersections and objects quickly approaching the wearer.  

Huawei notes that computer vision technology is one of Artificial Intelligence (AI) technologies used for image recognition, motion tracking and image data extraction. It allows a computing system to automatically identify objects in individual pictures or frames of video, or to recognize human activity over a series of video frames. It provides high-level understanding of the real world via computer vision tasks such as scene understanding, 3D perception, gesture recognition, etc. from images captured by a camera (e.g., RGB camera, Event camera, IR camera). For example, the computer vision can be used for determining the position of a person’s face in a digital image. The computer vision will define the next innovation on wearables. 

Earwear designed with integrated cameras can be used for pattern recognition, gesture recognition, 3D recognition of goods and objects, 3D scanning and 3D photography.

It is an object of this patent to provide a camera-enabled earwear with systems and methods that extends the user’s view.

Another object of the present disclosure is to perform real-time and on-board processing of several computer vision tasks that understand the user and their environment.

Still another object of the patent is to provide user-and-earwear interaction/feedback based on the user and environment understanding if necessary.

A camera mounted on the earwear may have a limited field-of-view. Therefore, it is possible to effectively capture images by configuring the sensor to capture the image such that the captured image does not overlap another image captured by another ear-wearing type electronic device.

More specifically, the ear-wearing type electronic device may interact with the user by alerting them when there’s a nearby car detected by a computer vision task as noted in Huawei's patent FIG. 7 below; in FIG. 14(a), the field of view of the second world-facing camera #301′ and the field of view of the second world-facing camera #302′ may have some partial overlap.

Huawei's patent FIG. 15 above is a diagram for explaining a method of user i Another usage scenario may be smart walking and cycling. For example, as shown in FIG. 15, the earwear may interact with the user (e.g., a cyclist wearing the earwear #100) by alerting him/her when there’s a nearby car detected by the computer vision task. FIG. 15 is a diagram for explaining example methods of user interaction/feedback. As shown in FIG. 15, the first world-facing camera #301 and the second world-facing camera #301′ may alert a cyclist wearing the earwear if nearby cars are detected and recognized by the earwear nteraction/feedback. 

Huawei's patent FIG. 3 below depicts an example system architecture of the earwear; as shown in FIG. 13(b), the user’s field of view and the field of view of the first world-facing camera #301 and the second world-facing camera #301′ may not have an overlap (depending on the model).

Usage of the cameras beyond detecting danger approaching a user of the earwear, Huawei presents a second usage case scenario that may enhance listening experience. In FIG. 8 the user is able to use hand gestures in front of the cameras to control music playback.

For those interested in delving into the details of this invention, review Huawei's U.S. Patent 20230209020.

Although it's not a "health" application, this is certainly a feature that all premium headphones, earbuds OEMs should consider implementing as a public safety feature.

If you have an opinion on this, send your comments to our Twitter feed.