Today the U.S. Patent and Trademark Office officially granted Apple a patent that relates to the future use of Virtual Paper on devices that will be able to crumple, flip, fold, bend while keeping the graphics intact in varying positions as the virtual paper changes angles and perspectives. Virtual paper could include mixed reality content in 2D, 3D and animation where the image could be constantly shifting depending on a particular state that it's in. This is pretty cool stuff that's now in-the-works.

The patent is not really meant for consumers, though we'll eventually be able to enjoy virtual paper on our future head-mounted devices, iPhones and iPads.

While the patent appears to be geared towards developers, consumers will one day be interacting with virtual paper on iPhone, iPads and Mixed Reality Headsets.

In the big picture, Apple's patent covers devices, apparatuses, systems, and methods for intermingling stereoscopic and conforming virtual content to a virtual paper.

Apple notes that in contrast to previously available computer-generated reality (CGR) systems, various embodiments disclosed in their patent filing provide a multi-dimensional CGR environment. In some embodiments, the multi-dimensional CGR environment includes a display of a virtual paper that is represented as a bounded surface or a bounded region.

For example, while the 3D virtual content is displayed such that the 3D virtual content is bounded within the perimeter of the virtual paper, the 3D virtual content is able to protrude out from a first side of the virtual paper, and the display of the 3D virtual content is based on the second set of world coordinates.

The 2D virtual content, on the other hand, is displayed as conforming to the surface of the virtual paper based on the first set of world coordinates. In some embodiments, a second side of the virtual paper, which is the opposite of the first side of the virtual paper (e.g., the backside), is displayed with a rasterized conforming representation of the 3D virtual content, e.g., a blur effect or shadow of the 3D virtual content, along with a rasterized conforming representation of the 2D virtual content.

In some embodiments, a native user CGR environment is a CGR environment in which a user or an avatar representing the user is located. For example, the CGR environment (#400) includes a user (#401) or an avatar representing the user.  As such, the scene depicted in FIG. 4A below can be a view of the CGR environment from the perspective of the user or a bystander.

Additionally, we see in Patent FIG. 4A below the 2D text "Matrix" (#410) appears to be floating on the surface of the front side of the virtual paper (#405), and the 3D chicken (#420) appears to be inside the virtual paper.

As such, the 2D text "Matrix" appears to be in front of (e.g., overlaid on) the 3D chicken from the perspective of the user. Accordingly, the 2D text "Matrix" occludes part of the crest of the 3D chicken inside the virtual paper.

Apple's patent FIG. 4B illustrates the backside of the virtual paper with the chicken, the bounded surface (#405) appears to be translucent or semitransparent, such that we can see the outline of a blurred chicken image with just the tip of the beak present on the far right of the image to show you that graphics are following the movement of the virtual paper.

The patent then goes 10 miles deep into the lighting, rasterized images, spheres, pixel depths that could lose the average reader.

As shown in patent FIGS. 4C-4E above, during the transformation of the display of the virtual paper, the display of the virtual contents associated with the virtual paper also transforms in response to transforming the contour of the virtual paper. For example, parts of the 3D-chicken protrude from one side of the virtual paper as the virtual paper or virtual page is turning. 

in Apple's patent FIGS. 4L and 4M below, Apple notes that in addition to the stereoscopic virtual object of the 3D chicken being displayed in a web page, the 3D chicken's head could be animated by turning, blinking and so forth.

Apple's patent illustrates Virtual Paper being used in future applications such as Apple Maps, Apple Music and other apps relating to travel and obviously books, magazines and so forth.  In patent FIG. 5i below, Apple tells us that a consumer viewing an image on an mall ad or magazine will be able to turn the boat around by gesture in order to view different parts of the ship and so forth.

In patent FIG. 5i above, Apple tells us that a consumer viewing an image on an mall ad or magazine will be able to turn the boat around by gesture in order to view different parts of the ship and so forth.

In patent FIG. 5i below, Apple tells us that a consumer viewing an image on an mall ad or magazine will be able to turn the boat around by gesture in order to view different parts of the ship and so forth.

For more details, review Apple's granted patent 11,328,497.