Apple Invents a new 'Reality Interface' Feature for Future iDevices and MR Glasses that will be able to control all Living Room Devices
Today the US Patent & Trademark Office published a patent application from Apple that generally relates to reality interfaces, and more specifically to techniques for controlling external devices using a next generation 'reality interface' on future iDevices and HMD.
Apple notes early on that a computer-generated reality environment (e.g., virtual reality or mixed reality environment) can have varying degrees of virtual content and/or physical content.
In some embodiments, a reality environment provides an intuitive interface for a user to interact with their physical environment. For example, using a reality interface that displays a representation of the user's physical environment, a user can control one or more external devices in the physical environment.
Specifically, using the reality interface, the user is able to obtain information (e.g., operating status) regarding the one or more external devices or control a function of the one or more external devices.
In some embodiments, the user can provide hand gestures to interact with the reality interface and cause an external device of the one or more external devices to perform a function.
Apple's patent FIGS. 3A-3D below provides exemplary techniques for controlling an external device using a reality interface. FIG. 3A depicts physical environment #302 that includes external devices #304 (television), #306 (speakers), and #308 (a floor lamp). Physical environment #302 is, for example, the physical environment of the user.
For instance, the user can be sitting in their living room and the physical environment (#302) is at least a portion of the user's living room that is directly in front of the user.
The user may wish to control one of external devices (television, speakers and /or lamp). The user can utilize a reality interface provided by the user's device #312 which could be an iPhone, iPad or MR Head Mounted Device to control one of the noted external devices.
The user will be able to control functionality of their TV or speakers via a reality interface found on their iDevice or Headset and turn the volume up or down on the speaker or turn the TV on or off, change the channel, or control the brightness of a living room lamp and much, much more. In some examples, the user will control the interface with hand gestures.
Apple's patent FIG. 3A below depicts a physical environment that includes external devices; FIG. 3B depicts a user device displaying a reality interface with a representation of a physical environment; FIG. 3C depicts a user device concurrently displaying, in a reality interface, a representation of a physical environment and an affordance corresponding to a function of an external device in the physical environment; FIG. 3D depicts a user interacting with a reality interface of a user device by providing a gesture.
In one way, the invention appears to be describing a next generation remote. Instead of it being limited to your TV or Apple TV box, the new remote will be one that will be able to control any external device in your living room. Technically, it could apply to any room the user is in. So, it could technically apply to a user's kitchen.
Apple's continuation patent application 20200201444 that was published today by the U.S. Patent Office was filed back in Q1 2020 with various aspects of the patent deriving from previous Apple patents from 2017 and 2018. Considering that this is a patent application, the timing of such a product to market is unknown at this time.
Apple's continuation patent is a mixed application that covers several previous patent applications and some never before made public. While the heart of this most recent patent and invention is about this futuristic reality interface app or feature for controlling external devices using a future iDevices and/or an HMD, this patent application also covers a secondary simpler invention about manipulating imagery while using a Headset as noted in the patent FIGS. 1i and 1E below.
As you can see, the user will use Augmented Reality to add a hat to a user in a photo or live-shot and or manipulate it so that the user could put a person in one photo near a tree and place them in a sunset beach photo that originally had no people in the shot.