Apple's SVP of Software Engineering Demonstrates the next-gen iPad Pro Keyboard Trackpad with unique to iPadOS Features
Apple Invents Optical Systems for Mixed Reality HMDs & Smartglasses that Include Microprojectors

Apple Invents an HMD that functions like an iPhone, Plays Games & Projects images like a Virtual Keyboard on a table

1 x2 Cover HMD System interacting with 3D images


Today the US Patent & Trademark Office published a patent application from Apple that relates to a future Head Mounted Device (HMD) that allows a user to interact with 3D images on the display with their finger and interact with a virtual keyboard sitting on a table top. In this invention Apple discusses a projector being integrated into the HMD and/or the option of being able to attach a projector unit to the HMD that could project a game or virtual keyboard onto a table top surface.  


Wearable electronic devices such as head-mounted devices have displays for presenting images to users while the devices are being worn.


In some situations, it can be challenging for a user to interact with an electronic device such as a head-mounted electronic device. When a head-mounted electronic device is being worn by a user, the user can view content on the display of the head-mounted device. In this mode of operation, selectable on-screen options may be presented on the display with which the user may interact to supply input to the device. When the head-mounted electronic device is not being worn, however, the user cannot view content on the display, which hinders the ability of the user to interact with the device and associated equipment.


Apple's invention covers a Head Mounted Device that has an inner display mounted in a housing. The housing may have head-mounted support structures configured to support the inner display and to support associated lenses adjacent to a user's head. The inner display may output light that passes through lenses to eye boxes. When the eyes of the user are located in the eye boxes, the user may view images being presented on the inner display.


The electronic device may have input-output devices that are accessible on external surfaces of the HMD. The input-output devices may include, for example, buttons, keys in a keyboard, touch sensors such as touch sensors overlapping displays, and other input-output devices.


The input-output devices of the HMD may be used to gather user input from the user when the HMD is not being worn on the user's head. For example, the input-output devices may gather user input when the HMD is being held in a user's hand or is resting on a table top.


The head-mounted device may communicate with associated external electronic equipment such as a computer, gaming system, cellular telephone, or other host.


In some configurations, the head-mounted device may serve as a stand-alone device.


In some configurations, an input-output component may be formed on a movable member that can be moved between a stored position and a deployed position.


A projector may project images onto nearby surfaces. The images may include a virtual keyboard with which a user may interact.


If desired, a removable electronic device may be coupled to head-mounted support structures or other housing structures in the head-mounted device.


2 x  HMD patent figs 1  3 & 5


In Apple's patent FIG. 3 above we're able to see a perspective view of the HMD in an illustrative configuration in which the outer display #44 is formed on exterior portions of the HMD when it's not being worn on a user's head. The HMD is sitting on a table top or other support structure and/or may be held in the hand.


In this type of scenario, the HMD and the sensors and other input-output devices #12 of the HMD can be used as an accessory controller for external electrical equipment #32.


For example, if a user desires to use an online service, the user may interact with keyboard keys and/or on-screen keys on display #44 to provide the online service with text and other information, to supply the online service with commands (e.g., by selecting on-screen options), and/or by otherwise providing the service with input using the input circuitry of the HMD.


Corresponding output from the service may be provided to the user using the output circuitry of the HMD (e.g., audio output, visual output, haptic output, etc.).


Because the HMD doesn't need to be placed over the eyes of the user while the user supplies input and receives output from HMD, the user may stay engaged with people in the vicinity of the user and may remain aware of the user's surroundings.


If desired, display #44 may mirror some or all of the content being displayed on display #14 (e.g., while the HMD is being worn by the user) and/or may be used by control circuitry to display content that is associated with the content being displayed on the display.


For example, if the HMD display is being used to present a game to a user, the outer display #44 may display some or all of the same game images to people or friends of the user nearby.  


Game statistics such as game scores, text that informs others that the user is playing the game (e.g., "game in progress"), and/or other suitable information related to the content on display #14 may also be displayed on outer display #44.


The HMD can Project a Virtual Keyboard


Apple's patent FIG. 5 above is a diagram of the HMD in an illustrative scenario in which the HMD is resting on a table top or other external surface such as surface #60.


The projector #46 in the HMD may display images such as image #62 on surface #60. The content of image #62 may be the same content that would have been displayed on the HMD display if worn on the user's head.


As an example, a user may use a web browser application that is running on the HMD to obtain information from the internet while this information is being presented to the user as part of image #62 on the table top.


If desired, a sensor (e.g. a three-dimensional image sensor or other sensor in device 10) may sense finger input at various portions of image #62 from a user's finger #26. This allows the user to interact with the content being displayed on the table top surface.


For example, the HMD may display a selectable option that is associated with an online service or an application running on the HMD in image #62. The user may place their finger on that selectable option (e.g., using a single tap, double tap, or other suitable user input). In response to detecting the user finger input corresponding to the selectable option (e.g., by analyzing data from the three-dimensional sensor), device 10 can conclude that the user has selected the selectable option and can initiate an operation in the HMD that is associated with the selectable option.


If desired, the projector #46 may project a virtual keyboard onto the surface and the three-dimensional sensor or other suitable sensor in the HMD may gather virtual keyboard input from the user as the user types on alphanumeric labels on virtual keys in the virtual keyboard.


Apple's patent application that was published today by the U.S. Patent Office was filed back in Q2 2019 Considering that this is a patent application, the timing of such a product to market is unknown at this time.


10.51FX - Patent Application Bar


The comments to this entry are closed.