Last week the U.S. Patent and Trademark Office officially published a patent application from Apple that relates to future smartglasses and, more specifically, to methods and techniques for managing smartglasses having additive displays for use in computer-generated reality environments.

To conserve battery life, Apple has designed a pair of smartglasses that provides a primary and secondary form of display that relays different kinds of information. In fact, there's a there's a third display, a tertiary display, in the form LEDs that surround the inner frame of the glasses that act as an indicator system that will become more evident in some of the patent figures presented further below in our report.

Apple notes that the HMD device will include both a primary display that extends across a field-of-view with a first resolution, and a secondary display that is physically and electronically coupled to the primary display that has a second resolution that is lower than the first resolution. Displaying a first portion of a virtual object is done via the secondary display and then displayed in the primary display when an object (e.g., app) is called up to interact with.

Apple's patent 2A-2B below depict an exemplary system that includes a device that has additive displays for use in various computer-generated reality environments.

In the next series of patent figures Apple illustrates how the tertiary display in the form LEDs surrounding the inner frame of the glasses act as an indicator system. In the example below, the user is looking for his keys. The LED display will light up portions of the lights indicating the direction of where his keys could be found. This could only work if the keys are attached to an AirTag device. While the patent doesn't spell that out, there's no other way for the smartglasses to know where the item being sought could be found.

In patent figure 17A below we see how apps could be called up when needed in the lower resolution display area. When the user decides to check new messages or emails, a first message will appear in the higher resolution display for easy reading. Users will have to touch a sensor to call place a message in the primary display or could work with Siri. In the case of a health app is present, the LED display could light up if the app determines the user is undergoing a heart problem that needs attention. For instance, the LEDs would light up red.

In patent FIG. 19B above we could see the glasses illustrate a vehicle with an outline around it. The outline would point out to the user which car happens to be their Uber ride. The visual aid could be for one car to pulsate or enlarge to indicate which is your ride.  

In patent FIGS. 23A and 23B we see that the secondary (outer) display being used as a stopwatch or timer which is handy when working out or cooking.

In patent FIG. 22B above you'll see a FaceTime call. Then the patent describes how the LED display could be used to indicate the mood of the caller (happy, sad, depressed) with different colored lights. It's difficult to know why that could be of assistance. Apple describes algorithms are used to determine a mood. I think most adults can tell someone's one mood by their demeaner – though it's an option for those who are completely daft. 

In Apple's updated patent, they've added 14 new Patent Claims as follows:

1. A head-mounted display (HMD) device, comprising: one or more image sensors; a primary display that extends across a field-of-view and has a first resolution; a secondary display that is physically and electronically coupled to the primary display and has a second resolution that is lower than the first resolution; one or more processors; and memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for: detecting an object in a computer-generated reality (CGR) environment at a first location; and in response to detecting the object in the CGR environment at the first location: in accordance with a determination that the first location is within a first predetermined distance outside of the field-of-view of the primary display of the HMD device, displaying, via the secondary display of the HMD device, a first modified representation of the object that is visually distinguished from a first actual representation of the object.
2. The HMD device of claim 1, wherein first modified representation has visual content that is different from visual content of the first actual representation.

3. The HMD device of claim 1, wherein the first modified representation has a size that is based on a distance between the first location and the field-of-view of the primary display of the HMD device.

4. The HMD device of claim 1, wherein the first modified representation has a brightness that is based on a size of the object.

5. The HMD device of claim 1, wherein the first modified representation has a blur that is higher than a blur of the first actual representation.

6. The HMD device of claim 1, wherein displaying the first modified representation includes displaying an animation.

7. The HMD device of claim 1, the one or more programs further including instructions for: in response to detecting the object in the CGR environment at the first location: in accordance with a determination that the first location is within the field-of-view of the primary display and a field-of-view of the secondary display: displaying, via the primary display, a second actual representation of a first portion of the object; and while displaying, via the primary display, the second actual representation of the first portion of the object, displaying, via the secondary display, a second modified representation of a second portion of the object.

8. The HMD device of claim 7, wherein the second actual representation is displayed at a higher resolution than the second modified representation.

9. The HMD device of claim 7, wherein the second actual representation is displayed adjacent to an edge of the primary display.

10. The HMD device of claim 7, wherein: displaying, via the primary display, the second actual representation includes applying a visual effect to the second actual representation; and the second actual representation of the object after applying the visual effect has a blur that is greater than a blur of the second actual representation of the object before the visual effect was applied.

11. The HMD device of claim 1, the one or more programs further including instructions for: in response to detecting the object in the CGR environment at the first location: in accordance with a determination that the first location is within a second predetermined distance outside of the field-of-view of the primary display of the HMD device, wherein the second predetermined distance is different from the first predetermined distance: displaying, via the primary display, a representation that corresponds to directional information to locate the object, wherein the representation that corresponds to the directional information to locate the object is different from the first actual representation.

12. The HMD device of claim 1, the one or more programs further including instructions for: in response to detecting the object in the CGR environment at the first location: in accordance with a determination that the first location is within the field-of-view of the primary display and not within the field-of-view of the secondary display: displaying, via the primary display, the first actual representation; and forgoing to display, via the secondary display, the first modified representation.

13. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a head-mounted display (HMD) device having one or more sensors, a primary display that extends across a field-of-view and has a first resolution, and a secondary display that is physically and electronically coupled to the primary display and has a second resolution that is lower than the first resolution, the one or more programs including instructions for: detecting an object in a computer-generated reality (CGR) environment at a first location; and in response to detecting the object in the CGR environment at the first location: in accordance with a determination that the first location is within a first predetermined distance outside of the field-of-view of the primary display of the HMD device, displaying, via the secondary display of the HMD device, a first modified representation of the object that is visually distinguished from a first actual representation of the object.

14. A method, comprising: at a system having one or more processors, memory, one or more image sensors, and a head-mounted display (HMD) device, the HMD device including a primary display that extends across a field-of-view and has a first resolution, and a secondary display that is physically and electronically coupled to the primary display and has a second resolution that is lower than the first resolution: detecting an object in a computer-generated reality (CGR) environment at a first location; and in response to detecting the object in the CGR environment at the first location: in accordance with a determination that the first location is within a first predetermined distance outside of the field-of-view of the primary display of the HMD device, displaying, via the secondary display of the HMD device, a first modified representation of the object that is visually distinguished from a first actual representation of the object.

Always keep in mind that this is a patent application and not a design patent. The patent figures only represent an overview of the concept and not the actual finished product that we know will be modern and likely not round.  

Inventors

• Jonathan Moisant-Thompson: Senior Display Exploration Engineer
• William Lindmeier: Manager, Interaction Designer
• Rahul Nair: Prototyping Engineer
• Devin Williams: Former Apple Product Design Engineer (now at Zipline)