Today the U.S. Patent Office published a patent application of Apple's that relates to an HMD that includes a method that uses front-facing cameras to acquire passthrough content, using a graphics renderer to generate virtual content, mitigating double vision or improving vergence comfort by modifying at least the passthrough content, merging the passthrough content with the virtual content, and displaying the merged content.

Apple notes in their patent background that it could be challenging to design a head-mounted device for user's who wear prescription glasses. Most standard eyeglass prescriptions, including those that require prism correction, can be accommodated for using additional clip-on lenses in front of the displays. Prism correction is a specific type of eyeglass prescription that helps people with double vision by aligning the images seen by both eyes.

Double vision occurs when the eyes do not work together properly, causing them to see two separate images of a single object. Prism correction can also be used to provide improved vergence comfort, even for users without double vision.

The use of clip-on lenses, however, can be overly bulky and can degrade the accuracy and limit the range of eye tracking sensors in the head-mounted devices.

 A Future version of Vision Pro may provide Double Vision Compensation and Vergence Comfort Improvement

 Apple’s patent application covers a future version of Vision Pro that may provide Double Vision Compensation and Vergence Comfort Improvement. An aspect of the invention provides a method of operating an HMD to acquire passthrough content with one or more image sensors to generate virtual content with a graphics rendering subsystem, mitigating double vision or improving vergence comfort by modifying the passthrough content to produce corresponding modified passthrough content, merging the modified passthrough content with the virtual content to produce corresponding merged content, and presenting the merged content with one or more displays.

The passthrough content can be modified by cropping or transforming either left eye passthrough content or right eye passthrough content at an image signal processor. The passthrough content can be modified by warping either the left eye passthrough content or the right eye passthrough content to apply a lateral shift in the field of view or to apply a transformation in the point of view.

The graphics rendering subsystem can be configured to generate the virtual content based on a first virtual camera that imposes constraints defining a first point of view from which left eye virtual content is being rendered at the graphics rendering subsystem and a second virtual camera that imposes constraints defining a second point of view from which the right eye virtual content is being rendered at the graphics rendering subsystem. The first virtual camera can have a first view frustum, whereas the second virtual camera can have a second view frustum.

The graphics rendering subsystem can be configured to shift either the left eye virtual content or the right eye virtual content on the one or more displays by adjusting an angle limit (boundary) for a corresponding one of the first and second view frustums. The graphics rendering subsystem can be configured to laterally shift a selected one of the first and second view frustums along a common projection plane. The graphics rendering subsystem can be configured to transform a selected one of the first virtual camera and the second virtual camera. The virtual content can be modified by selectively warping the left eye virtual content or the right eye virtual content to apply a lateral shift in the field of view or to apply a transformation in the point of view.

Another aspect of the patent provides a method of operating a device that includes acquiring passthrough content with one or more image sensors, generating virtual content with a graphics rendering subsystem, obtaining prism prescription values for correcting double vision or improving vergence comfort, computing compensation parameters based on the prism prescription values, modifying the passthrough content based on the compensation parameters, and displaying a blended version of the modified passthrough content and the virtual content.

The method can further include modifying the passthrough content by shifting or transforming left eye passthrough content or right eye passthrough content. The method can further include modifying the virtual content based on the compensation parameters. The method can further include providing a user with an opportunity to adjust the compensation parameters to minimize double vision or improve vergence comfort. The method can further include reading a scannable code and obtaining the prism prescription values from the scannable code.

An aspect of the disclosure provides a method of operating HMD that includes acquiring passthrough content, generating virtual content, displaying a stimulus point, monitoring a point of gaze to estimate a misalignment between the point of gaze and an expected point of gaze while the stimulus point is being displayed, computing compensation parameters based on the estimated misalignment, modifying the passthrough content based on the compensation parameters, and displaying a blended version of the modified passthrough content and the virtual content. The method can further include estimating the misalignment by obtaining a plurality of gaze data points, removing one or more outliers from the plurality of gaze data points, and averaging the plurality of gaze data points.

• FIG. 1 is a top view of an illustrative head-mounted device
• FIG. 3 is a diagram showing illustrative hardware and/or software subsystems within an electronic device configured to perform double vision compensation
• FIG. 4 is a diagram illustrating shifting of the projected content to be displayed to achieve double vision correction
• FIG. 5 is a diagram illustrating camera perspective transform to achieve double vision correction
• FIG. 8 is a flow chart of illustrative steps for generating double vision compensation parameters based on measured gaze data

• Sabine Webel: Senior Software Engineering Manager visionOS
• Christian Gosch: Software Engineer, VisionOS
• Ritesh Sholapur: Software Engineer

A Second HMD Patent: Dynamic Portals

A secondary HMD patent from Apple covers various implementations, systems, and methods for displaying a portal. In various implementations, the method is performed by a device including a display, one or more processors, and non-transitory memory. The method includes obtaining a user location in a first three-dimensional coordinate system of a first environment.

The method includes obtaining an interest location in a second three-dimensional second coordinate system of a second environment. The method includes transforming the interest location to a transformed interest location in the first three-dimensional coordinate system.

The method further includes determining a portal location in the first three-dimensional coordinate system based on the user location and the transformed interest location. The method includes displaying, on the display at the portal location, a portal corresponding to at least a partial view of the second environment.

For full details, review Apple’s patent application 20250124670.