On Tuesday Patently Apple posted a granted patent report covering one of Apple's acquired patents from Metaio about augmented reality in respect to 3D reconstruction. Today, Patently Apple was first to discover that the US Patent and Trademark Office published a patent application from Apple that relates to another acquired patent from Metaio about another aspect of augmented reality titled "Scene Classification. The main inventor of this patent is Peter Georg Meier, former owner of Metaio and now with Apple as Director – Algorithm Lead, Technology Development Group.

The invention covers techniques for providing content using a computer-generated reality interface depicting virtual objects in combination with a representation of a physical environment.

In one exemplary technique, image data corresponding to the physical environment are obtained using one or more cameras. At least one portion of an entity in the physical environment is identified based on the image data. Based on the identified at least one portion of the entity, whether the entity is an entity of a first type is determined. One or more virtual objects and a representation of the entity are displayed.

An augmented reality (AR) environment refers to a simulated environment in which one or more virtual objects are superimposed over a physical environment, or a representation thereof. For example, an electronic system for presenting an AR environment may have a transparent or translucent display through which a person may directly view the physical environment.

The system may be configured to present virtual objects on the transparent or translucent display, so that a person, using the system, perceives the virtual objects superimposed over the physical environment.

Alternatively, a system may have an opaque display and one or more imaging sensors that capture images or video of the physical environment, which are representations of the physical environment.

The system composites the images or video with virtual objects, and presents the composition on the opaque display. A person, using the system, indirectly views the physical environment by way of the images or video of the physical environment, and perceives the virtual objects superimposed over the physical environment.

Apple's patent FIG. 2A below depicts a user device displaying a representation of an indoor physical environment; FIG. 2B depicts a block diagram of a user device including classifiers configured to identify one or more entities of an indoor physical environment.

Apple's patent FIGS. 2C and 2D above depict various flows for classifying identified entities and determining the type of the classified entity. Here we're able to see that everything in that photo scene taken by an iPhone with its TrueDepth camera can add AR information about the sink, the cat, the appliances and so forth.

Apple's patent FIG. 2H below depicts a user device displaying a computer-generated reality interface including virtual objects and representations of identified entities.

Each room of the house can be set to an augmented reality world. When in the kitchen, the user will be able to ask Siri to call up a recipe to follow and it will pop-up in the users view inside a head-mounted device (HMD).

In the living room, the HMD could correspond to virtual objects, such as controls for operating a TV, a user-interaction mechanism for ordering movies, or a user-interaction mechanism for subscribing magazines.

Further, the headset could work with an augmented reality TV guide service or TV subscription service can be provided regardless of whether the TV entity is located in a bedroom or a living room.

Accordingly, the user device is configured to present one or more virtual objects based on the identified TV entity (e.g., a virtual object enabling the user to receive an on-line movie streaming service) without having to determine the type of physical environment (e.g., whether the physical environment is a living room or bedroom).

Other scenarios are discussed in the patent filing, such as one relating to outdoor environments. Apple's patent FIG. 3E below depicts a user device displaying a computer-generated reality interface including virtual objects and representations of identified entities.

A user may be walking in a park when a message could pop up to notify the user that a concert is coming to the park based on GPS sensors and enable the user to order tickets for a particular concert in the park.

Apple's patent application 20190318168 that was published today by the U.S. Patent Office was filed back in Q2 2019 with work dating back to Q2 2018. Considering that this is a patent application, the timing of such a product to market is unknown at this time.

About Making Comments on our Site: Patently Apple reserves the right to post, dismiss or edit any comments.