The U.S. Patent and Trademark Office officially published a series of 63 newly granted patents for Apple Inc. today. In this particular report we cover Apple work on a future AR Experience Sharing App that includes games and much, much more.

GUI's for Depth-Based AR Annotation - AR Experience Sharing App  

Apple notes that the development of computer systems for augmented media has increased significantly in recent years. Examples of augmented media include augmented reality environments that include at least some virtual elements that replace or augment the physical world and augmented stored media that include at least some virtual elements that replace or augment stored media, such as image and video content.

Conventional methods of augmenting media often require multiple separate inputs (e.g., individual annotation of multiple frames and/or placement of augmentations relative to objects in media) to achieve an intended outcome (e.g., annotating a portion of a stored video or a live video feed and/or displaying virtual objects at positions that correspond to surfaces of physical objects in stored media).

Some of the patent figures provide an overview of intuitive ways for a user to augment media such as stored content, still images, and/or live video captured by one or more cameras of a device (e.g., by using depth data stored and/or captured in conjunction with image data to place augmentations and to maintain a fixed spatial relationship between augmentations and portions of a physical environment in a field of view of a camera).

Apple's granted patent covers systems, methods, and GUIs that will improve user interface interactions for augmenting media in multiple ways. For example, they make it easier to: relocalize an annotation, annotate a video, add virtual objects to previously captured media, and initiate a shared annotation session.

Apple's patent FIGS. 8C, 8R and 8U below illustrate example user interfaces for initiating a shared annotation session.

FIG. 8C: In response to receiving the request for initiating the shared annotation session, device #100-2 displays a prompt #8016 that includes instructions for accepting the request for the shared annotation session.

In FIG. 8R: Illustrates a physical environment #8068 in which a first user #8064a operates a first device #100-1 and a second user #8066a operates a second device #100-2. The figure illustrates a game application user interface #8063-1. Game application user interfaces #8063-1 and #8063-2 display basketball hoops #8074 and #8076, respectively. By providing input in the respective game application user interfaces, the users launch virtual basketball objects into the displayed representation of the field of view of the respective device cameras in an effort to make baskets in the respective basketball hoops.

There are dozens more patent figures and examples that Apple provides us with that are just too lengthy for our report. You could review Apple's full granted patent 10,785,413 here.