Apple Invents an AR Windshield that could take FaceTime Calls as well as translate road signs to different languages
Last month Patently Apple posted a patent report titled "Apple's Third Patent Surfaces Relating to a Future Mixed Reality Windshield for Turn-by-Turn Directions and more." Yesterday the US Patent & Trademark Office published Apple's fourth patent application to date regarding an AR windshield. This latest patent filing adds a few more features that Apple's engineers are working on for the future.
Apple notes in their patent filing that in many situations, a graphical overlay can be provided on an environment that is perceived through a transparent surface, including a window – which translates to a windshield doubling as an AR display.
A graphical overlay can provide information to an observer, including information which is relevant to one or more portions of the perceived environment. In some cases, a graphical overlay is used in a vehicle, where a graphical overlay can be perceived by an occupant/driver of the vehicle and provides them with relevant information including vehicle speed and turn-by-turn guidance.
Two of the notable features being added to this project is the ability to participate in a FaceTime call with the driver only seeing a faint view of the caller so as not to interfere with the drivers view of the road. This is shown in patent FIG, 10 below.
Further below Apple's patent FIG. 3B is describing how road signs could be set so that they are translated into another language. This way if you're traveling through Europe with various foreign language road signs popping up, the AR windshield will be able to translate them into English, Spanish or German as preset by the driver. That's an interesting feature that will be appreciated in the future.
More specifically, Apple notes that FIG. 10 illustrates a vehicle comprising a transparent surface which provides an augmented reality display which enables user selection of a portion of the environment perceived via the transparent surface and which enables video communication with a remotely-located user via the transparent surface.
In another scenario it appears that in some cases that the driver will only see a small avatar (Memoji) of the caller that will reside on the windshield. Although not detailed, the caller will likely not appear directly in front of the driver's view of the road as a distraction which would be illegal (and foolish).
The patent also hints that some elements will only be available to a driver at a particular speed. It's possible on that basis that a full FaceTime call for the driver may only be allowed when the vehicle is parked whereas the right-side passenger could view a full FaceTime representation.
Although confusingly presented, it appears that the AR display system may also be applicable to other occupants of the vehicle, such as the back seat. In that case, the augmented reality display system will be able to generate various display elements as part of an interactive game, based at least in part upon occupant actions, gestures, interactions with one or more user interfaces, etc.
In the passenger seat beside the driver, they'll be able to view streaming video on the windshield.
To learn and explore Apple's patent application 20200258480, click here. Apple originally filed the patent in Q1 2018. Considering that this is a patent application, the timing of such a product to market is unknown at this time.