Today the U.S. Patent and Trademark Office officially granted Apple a patent that generally relates to monoscopic (1 view) or stereoscopic (2 view) 180-degree (hemisphere) or 360-degree (full sphere) photography and post-production for distribution to dome, mobile or head-mounted display virtual reality applications.

In Apple's patent background they note that conventional 180 degree or 360-degree video and/or images are stored in flat storage formats using equirectangular or cubic projections to represent spherical space. If these videos and/or images are edited in conventional editing or graphics applications, it is difficult for the user to interpret the experience of the final result when the video or images are distributed and presented in a dome projection, cubic, or mapped spherically within a virtual reality head-mounted display. Editing and manipulating such images in these flat projections requires special skill and much trial and error.

Further, it is not an uncommon experience to realize after manipulation of images or videos composited or edited with spherical that subsequent shots are misaligned, or stereoscopic parallax points do not match in a natural way. Apple's granted patent remedies these shortfalls.

Apple's granted patent is generally directed to methods and systems for transmitting monoscopic or stereoscopic 180 degree or 360-degree still or video images from a host editing or visual effects software program as equirectangular projection, or other spherical projection, to the input of a simultaneously running software program on the same device that can continuously acquire the orientation and position data from a wired or wirelessly connected head-mounted display's orientation sensors, and simultaneously render a representative monoscopic or stereoscopic view of that orientation to the head mounted display, in real time.

Apple's method may include receiving image data from a GPU buffer, wherein the GPU buffer is associated with a media manipulation application, obtaining an orientation data for a display device, rendering a preview image using the image data and the orientation data, and displaying the preview image using the display device, wherein the preview image is displayed while the image data is utilized by the media manipulation application, and wherein the preview image is dynamically modified when the image data is modified by the media manipulation application.

Apple's patent FIG. 5 below shows an example for diagram for presenting a preview image by a 3D display device.

More specifically, Apple's patent FIG. 5 illustrates the user interface of a video or image editing or graphics manipulation software program (#501) with an equirectangularly projected spherical image displayed in the canvas #502 and a compositing or editing timeline #503.

The image output of the video or image editing or graphics manipulation software program can be output via a video output processing software plugin module #504 and passed to a GPU image buffer shared memory and then passed efficiently to the image receiver #507 of the head-mounted display processing program #506.

The 3D image processing routine #508 of the head-mounted display processing program will texture the inside of a virtual sphere or cube with a 3D viewpoint at the center of said sphere or cube. The virtual view for each of the left and right eyes will be accordingly cropped, duplicated (if necessary), distorted and oriented based on the lens/display specifications and received orientation data #509 of the wired or wirelessly connected head-mounted display's #510 orientation sensor data. Once the prepared image is rendered by the 3D image processing routine, the image can then be passed to the connected head-mounted display #511 for immediate presentation to the wearer within the head-mounted display.

For greater more details, review Apple's granted patent 10,986,330