An Apple patent reveals an AR Interface feature for Headsets & Glasses that will allow users to control external devices with in-air gestures
Today the US Patent & Trademark Office published a patent application from Apple that relates to reality interfaces, and more specifically to techniques for controlling external devices like a TV using a next generation augmented reality interface included with mixed reality headsets and glasses that work with in-air gestures as noted in our report's cover graphic.
Apple notes early on that a computer-generated reality environment (e.g., virtual reality or mixed reality environment) can have varying degrees of virtual content and/or physical content.
In some embodiments, a reality environment provides an intuitive interface for a user to interact with their physical environment. For example, using a reality interface that displays a representation of the user's physical environment, a user can control one or more external devices in the physical environment.
Specifically, using the reality interface, the user is able to obtain information (e.g., operating status) regarding the one or more external devices or control a function of the one or more external devices.
In some embodiments, the user can provide hand gestures to interact with the reality interface and cause an external device of the one or more external devices to perform a function.
Apple's patent FIGS. 3A-3D below provides exemplary techniques for controlling an external device using a reality interface. FIG. 3A depicts physical environment #302 that includes external devices #304 (television), #306 (speakers), and #308 (a floor lamp). Physical environment #302 is, for example, the physical environment of the user.
For instance, the user can be sitting in their living room and the physical environment (#302) is at least a portion of the user's living room that is directly in front of the user.
The user may wish to control one of external devices (television, speakers and /or lamp). The user can utilize a reality interface provided by the user's device #312 which could be an iPhone, iPad or MR Head Mounted Device to control one of the noted external devices.
The user will be able to control functionality of their TV or speakers via a reality interface found on their iDevice or Headset and turn the volume up or down on the speaker or turn the TV on or off, change the channel, or control the brightness of a living room lamp and much, much more. In some examples, the user will control the interface with hand gestures.
Apple's patent FIG. 3A below depicts a physical environment that includes external devices including a stand-up lamp; FIG. 3B depicts a user device displaying a reality interface with a representation of a physical environment and is able to control the lamp using gestures; FIG. 3C depicts a user device concurrently displaying, in a reality interface, a representation of a physical environment and an affordance corresponding to a function of an external device in the physical environment; FIG. 3D depicts a user interacting with a reality interface of a user device by providing an in-air gesture to turn on or off a television.
Apple's patent FIGS. 4A and 4B below depict a flow chart of an exemplary process for controlling an external device using a reality interface.
The patent also covers how the new AR Interface could work with devices like that iPhone or iPad.
Apple's patent application 20210365228 is a continuation patent that also now incorporates several patents on record that Apple lists under "Cross-Reference to Related Applications." The patent also includes an additional 20 new patent claims that you could review here.
With the new analytical report posted by Ming-Chi Kuo this morning, it makes every mixed reality patent on record all the more exciting, knowing that many of them will be coming to life over time.
Two Listed inventors
Justin Stoyles: Engineering Program Management at Apple (Augmented Reality)
Michael Kuhn: Software Development Manager. Kuhn came to Apple via the acquisition of Metaio GmbH in 2015 where his title was VP Product Development.