When Apple Vision Pro was introduced at WWDC23 event, we saw several applications where the user was able to control the size of a virtual display for reviewing photos, watching movies and Apple TV+ content, playing games and pulling up Safari during a FaceTime call and more. Last Thursday, the US Patent & Trademark Office published a patent application from Apple that relates to extended reality, and more specifically, to techniques for displaying a virtual display in an extended reality setting as noted in our cover graphic from the WWDC23 event.

During the event, Alan Dye, VP, Human Interface stated: "The entire visionOS interface looks and feels truly present in your room. Every element has been crafted to have a sense of physicality. The have dimension, respond dynamically to light, and even cast shadows to help you understand scale and distance. Experiences in Vision Pro aren't constrained to the boundaries of a display – they can freely fill the space around you."  

Displaying a Virtual Display

Apple's patent application covers techniques for displaying a virtual display in an extended reality setting. Various conventional systems may be equipped to display media to a user in an extended reality setting, such as via a virtual display within the extended reality setting. However, given the increasing complexity of extended reality systems, conventional techniques for displaying media to a user are not compatible with various new extended reality modes.

For example, conventional systems are unable to adequately display media to a user when the user is viewing an environment using augmented reality. Given the difficulties of simultaneously displaying both representations of actual content and virtual content, a method and system for displaying a virtual display in an extended reality setting is desired.

Apple's patent FIGS. 3A-3C below depict an exemplary process for displaying a virtual display in an XR setting.

In patent FIG. 3B above, virtual display #312 being placed in a fixed location may result in the appearance of a "floating" display within XR setting #302. In some examples, the predefined distance in front of the user may be obtained directly (or derived) from one or more standards including specifications for a desirable display placement. The predefined distance may further be created, modified, or otherwise customized by a user.

After the virtual display is initially displayed in the XR setting, any user head movements or other movements within the XR setting will result in the virtual display maintaining the initial position.

For FIG. 3, In accordance with a determination that the intent to adjust the virtual display includes an intent to move the virtual display, such as a finger pinch and a controller extension, virtual display is displayed as moving to an alternative location within XR setting. In FIG. 3C the user has taken the large virtual display of FIG. 3B and reduced the size of the virtual display to fit on a portion of the wall with a finger pinch. Siri may also be called up to move the display to where the user wants with simple commands such as "Ok System, move the display to center of the wall."

Although Apple's WWDC23 event video laid much of this out, those who want to review the technical aspects behind the invention can do so be reading patent application 20230298281.