Apple has been granted a patent for a Unique Augmented Reality HMD Lens System with Camera Optics called 'Lenslets'
The U.S. Patent and Trademark Office published a granted patent for Apple that reveals a unique lens system design well-suited to augmented reality head-mounted devices that includes a new lens system with 'Lenslets.'
Apple notes in their patent background that Augmented Reality (AR) or Mixed Reality (MR) systems merge virtual imagery onto a view of the real world. In AR head-mounted devices (HMDs), virtual content is displayed with an image projection system located near the eye. Real-world content can either be viewed directly using an "optical see-through" lens design, or it can be rendered digitally using scene cameras for "video see-through."
Current video see-through HMDs suffer from a shifted perspective of the real-world because the scene cameras are mounted anterior (in front of), superior (above), and/or lateral (to the side) of the user's eyes.
Scene cameras placed lateral to the eyes create a mismatch in Inter-pupillary distance (IPD) compared to the user's eyes. As a result, the user can suffer from double vision, blurred vision, dizziness, headache, nausea, and fatigue. This mismatch can also result in the incorrect perceived distance, and scale, of objects. Scene cameras placed superior or anterior to the eyes can create a perspective that translates with exaggeration to head movement, or incorrect motion parallax. These effects can also cause the user discomfort and this is what Apple's granted patent is out to solve.
Apple's granted patent describes a novel lens system having an entrance pupil at a location matching, or nearly matching, the eye's optical system entrance pupil.
The described lens system is significantly thinner than a conventional lens system making it well-suited to head-mounted device (HMD) applications. The system's thinness is achieved by using an array of camera elements.
Each camera element containing camera optics called "lenslets" and a sensor (e.g., the sensor could be an image sensor or a depth sensor), where a lenslet can include one or more lenses (e.g., a compound lens).
Individual camera elements can be arranged such that their optical axes intersect at, or near, the eye's entrance pupil location. In addition, each camera element's field-of-view (FOV) corresponds to a small sector of a large FOV. The FOVs of adjacent camera elements can be non- or slightly-overlapping so that a wide-angle image can be formed by concatenation of the individual camera element images.
Apple's patent FIG. 2 below illustrates a camera system having its individual elements arranged along a spherical surface.
While camera systems in accordance with this disclosure are not limited to use in head-mounted devices (HMD), they may be particularly useful in same because they can be fabricated to be slimmer and less bulky than conventional HMD systems. Apple's patent FIG. 6A above illustrates an augmented reality (AR) HMD system #600 in accordance with one or more embodiments includes multiple camera elements (represented by shaped elements) for each eye (e.g., 605L and 605R) arranged along a spherical surface, one or more display elements (#610L and #610R), one or more modules (#615L and #615R) and structure #620 to which the other components may be affixed and which can be mounted onto a user's head. While only five camera elements are illustrated for each eye, more or fewer may be used in any given implementation depending upon the device's intended use and targeted FOV.
Apple's patent FIG. 6B above is another embodiment AR-HMD system #625 includes optic and electronic systems (6051, 6101, and modules 615) affixed to structure (#630) for only a single eye. In the illustrated system, electronic modules #6151 and #615R are shown although both may not be necessary.
There a lot of detail that you could review in Apple's granted patent 11,143,865.