Overview of Apple's Movie Glasses

Apple has been granted their second patent relating to a head-mounted display which is more commonly known as video glasses. Their first granted patent was issued in September 2009 and we originally covered the patent application back in 2008.

Apple's second granted patent was never explored by Patently Apple as a patent application. Therefore our coverage of Apple's second granted patent will be treated as though it's a patent application where we'll explore the patent in greater detail.

According to Apple, a Head Mounted Display (HMD) is also technically known within the industry as a near-to-eye display. A HMD has either one or two small LCD or OLED displays with magnifying lenses and other associated optical elements. The display(s) and optics are typically embedded in a helmet, glasses, or a visor, which a user can wear. Lenses and other optical components are used to give the user the perception that the images are coming from a greater distance, to prevent eyestrain.

In HMDs that use a single display, the image is typically projected through optics that split the image into two identical images, and redirects each image to the respective eye. With two displays, the HMD can show stereoscopic images. The stereoscopic images attempt to create depth to the images by simulating the angular difference between the images viewed by each eye when looking at an object, due to the different positions of the eyes. This angular difference is one of the key parameters the human brain uses in processing images to create depth perception or distance in human vision.

Video Glasses with Augmented Reality

Some HMDs can be used to view a see-through image imposed upon a real world view, thereby creating what is typically referred to as an augmented reality. This is accomplished by reflecting the video images through partially reflective mirrors, such that the real world is seen through the mirrors' reflective surfaces. The augmented reality can be combined with the stereoscopic images in various types of applications.

Markets for Video Glasses

Some examples include applications in surgery, where radiographic data, such as CAT scans or MRI imaging can be combined with the surgeon's vision. Military, police and firefighters use HMDs to display relevant tactical information, such as maps or thermal imaging data. Engineers and scientists use HMDs to provide stereoscopic views of CAD schematics, simulations or remote sensing applications. Consumer devices are also available for use in gaming and entertainment applications.

The Problem with Video Glasses of the Past

A problem with HMDs, primarily HMDs that are not of the see-through kind, is that the image on the display in front of each eye fills the central but not the peripheral field of view of the user. Consequently, the visual experience is similar to looking into a box or tunnel having a small screen at a distance. Peripheral vision is good at detecting motion and as a result, occluded peripheral vision in HMDs can cause a user to experience motion sickness symptoms after some time. Thus, whereas existing HMDs may work well for their intended purposes for short periods of time, there is a continuing need for improved HMDs that provide enhanced long-time visual experience for the user.

Apple's Solution

Apple's invention generally relates to methods and apparatus for treating the peripheral area of a user's field of view in a head mounted display, and thereby creating improved comfort and usability for head mounted displays.

The peripheral area adjacent to the displayed image is treated, such that the peripheral area is coordinated with the image on the display. The coordination can be in the form of color projections, achieved, for example, by light emitting diodes (LEDs) or other displays, such that the colors surrounding the display dynamically matches what is shown on the display. As a result, the peripheral area "converges" with the display area, which reduces the "tunnel effect" or "box effect" experienced by the user. Various embodiments of the invention allow users to customize different viewing parameters of the head mounted displays to accommodate for variation in the individual users' eyes.

In general, in one aspect, the invention provides methods and apparatus, including computer program products, implementing and using techniques for projecting a source image in a head-mounted display apparatus for a user. A first display projects an image viewable by a first eye of the user. A first peripheral light element is positioned to emit light of one or more colors in close proximity to the periphery of the first display. A receives data representing a source image, processes the data representing the source image to generate a first image for the first display and to generate a first set of peripheral conditioning signals for the first peripheral light element, directs the first image to the first display, and directs the first set of peripheral conditioning signals to the first peripheral light element. As a result, an enhanced viewing experience is created for the user.

Video Glasses Could Work with Apple's iDevices

Advantageous implementations can include one or more of the following features. The processor can receive data representing the source image from an external source. The external source (think iDevice) can be a portable source and include a memory in which the source image is stored. A memory operatively coupled to the processor can be provided in which the data representing the source image is stored. The first display can have an aspect ratio of 16:9. The data representing the source image and the first image can be image frames of a movie, home movie or music video. Viewing movies is one of the main marketing focuses of this particular patent.

The various embodiments of the invention can be implemented to include one or more of the following advantages. One advantage is that the treatment of the peripheral area of the field of view leads to increased viewing comfort compared to conventional HMDs, and may also lead to a smaller likelihood of the user experiencing "motion sickness" phenomena during extended viewing. Another advantage is that users can make individual adjustments of their HMDs to fit the distance between their eyes. Further advantages include a greater immersive experience, larger virtual field of view, and increased overall image brightness.

Apple's Head Mounted Display & Image Generation System

Apple's patent FIG. 2 shown below illustrates a schematic top view of their Head Mounted Display. As can be seen, the HMD has two displays (202; 204) that are situated in front of the user's eyes (212a; 212b). Optical components (210a; 210b) focus the images from the respective displays onto the user's eyes at a comfortable viewing distance. The periphery of each of the displays is surrounded by one or more peripheral light elements (206; 208). The peripheral light elements illuminate the peripheral area around the displays in such a way that the peripheral area is color coordinated with a current image that is displayed on the display. That is, when a movie is displayed, the peripheral colors also change dynamically to match the colors in the image. Apple's patent FIG. 3 shows a schematic view of a HMD image generation system.

Some embodiments have peripheral light elements that are electro luminescent displays (ELDs).

Optics Part One: Aspherical Optics & More

Another important component of the HMD is the optics which actually transforms the image on the displays into an image that can be viewed by the human eye. A brief overview of various optical arrangements are shown and explained below.

Aspherical optical arrangements include various types of prism and/or lens arrangements. Apple's patent FIGS. 5A to 5C illustrate some examples of this type of optical arrangement.

Apple's patent FIG. 5A shows a coaxial arrangement of a set of lenses arranged along a common axis, and FIG. 5B shows a concave mirror arrangement. The basic functions of these arrangements are to magnify the micro-display image and present them at a focal point that is comfortable for the typical eye vision.

Human vision typically can't focus closer than 10 inches. In HMD applications, the displays are placed in close proximity to the eyes and thus require these optical systems. Coaxial lens systems basically bend incoming light (image). A concave mirror arrangement does so similarly, but also folds the optical path to create virtual optical viewing distance (i.e., the viewing chart in an optometry office is projected through several mirrors to simulate 20 m length even though the office is much smaller).

Apple's patent FIG. 5C shows a free shaped prism arrangement, in which the image from the display is redirected and magnified. This arrangement provides a large field of view and is compatible with a wide variety of display devices, although it may be somewhat more bulky than other optical arrangements described herein.

Optics Part Two: Diffractive Optics & More

Another type of optical arrangement that Apple discusses is that of "diffractive optical arrangements. These kinds of arrangements bend and spread light. Apple's patent FIGS. 6A to 6C show some examples of diffractive optical arrangements that can be used in HMDs.

Apple's patent FIG. 6A shows a light-guided optical element (LOE), which can be made of planar transparent glass or plastic. The LOE configuration increases the eye motion box for a given image and can be made as thin as about 2 mm. The LOE is see-through capable and provides a large field of view (up to about 40 degrees), and can also be encapsulated into a larger lens, if need be. The LOE can be used together with LCD and LCoS displays (Liquid Crystal on Silicon displays).

Apple's patent FIG. 6B shows a binocular light-guided optical element (BLOE), which can be made of planar transparent glass or plastic. In the BLOE configuration, a centered input image is directed to binocular images. Just like the LOE in FIG. 6A, the BLOE can be made thin, typically about 3 mm thickness, and is see-through capable. It can be used together with LCD and OLED displays. The advantage of BLOE over LOE is the alignment between the two images is inherent in the BLOE design. Left and right should be aligned with the eyes or it may cause physical distress to the user.

In Apple's patent FIG. 6C shows a wedge display. The image enters the edge of the wedge optics, and travels through an expansion region before it is displayed to the user's eye. Its thickness can be as small as about 2 mm, and its size can be very large--up to 50-inch prototypes have been manufactured to date. It is also possible to use in a folded design, which may save space compared to other optical arrangements, and it accepts a wide variety of display types. This technology has been primarily focused on screen displays such as computer monitors and TVs, but can be used display screens in HMDs as well.

Telephonic Capabilities

Finally, Apple states that the processor could optionally be coupled to a computer or telecommunications network, for example, an Internet network, or an intranet network, using a network connection, through which the processor can receive information from the network, or might output information to the network in the course of performing the above-described method steps. Considering that Apple's original patent application predated the original iPhone, it was quite a revelation to make for its time.

Patent Credits

Apple's patent application was originally filed in Q4 2006 by inventors John Tang and Apple's former VP of the iPod Division, Anthony Fadell. Originally, Fadell was hired by Apple to assemble and run the iPod & Special Projects group. Apple's video glasses/Head Mounted Display would definitely have fallen under the "special projects" category. Apple was granted this patent on July 03, 2012.

Note to Referring Sites: We ask that referring sites limit the use of our graphics to a maximum of two per report. Thank you for your cooperation.

Notice

Patently Apple presents a detailed summary of patent applications with associated graphics for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for full and accurate details. Revelations found in patent applications shouldn't be interpreted as rumor or fast-tracked according to rumor timetables. About Comments: Patently Apple reserves the right to post, dismiss or edit comments.