Today the US Patent & Trademark Office published a patent application from Apple that relates to a head-mounted system, such as smartglasses, that comprises a holographic element and an optical scanning device which records the light incident on an eye by way of the holographic element. The invention is highly technical with a glimpse of the smartglasses being used as an AR device that uses GPS and other sensors to provide visual cues that are accurate. The invention covers being able to use various light systems to ensure users could use the glasses in low light and on rainy days. Lastly, Apple notes that their smartglasses could be used as part of a "driver assistance system."

Apple acquired Metaio back in 2015. Metaio was the worldwide leader in Augmented Reality software, research and technology. With over 10 years of experience in Augmented Reality, Holographic technologies and Computer Vision, Metaio served over 100,000 developers with over 1,000 published apps. In total, Metaio's AR software reaches over 30 Million consumers around the world.

Metaio CTO Peter Meier once stated that "Everyone is talking about wearable computing eyewear like Google Glass, but no one is talking about the best way to actually use those devices. We need natural, convenient interface to navigate the technology of tomorrow, and that's why we developed 'Thermal Touch'."

Today's "continuation patent" titled "Information System and Method for Providing Information Using a Holographic Element," covers a head-mounted system comprising of an optical device which detects light incident on an eye, wherein the optical device is configured to project an infrared light onto the eye, detect light reflected off the eye in response to the projected light, and track an eye movement based on the detected reflected light; and a holographic element by which light is projected into the eye to present a virtual object, wherein the light is projected based on the tracked movement."

The invention dates back to Metaio's original 2010 granted patent dating back to 2002 as illustrated below.

Although the entire invention/patent is new to us that haven't seen it before, the fact is that Apple has changed the nature of the invention under this "continuation patent." Continuation patents focus entirely on the patent claims be expanded, altered or changed in some material way.

In this case Apple has refocused the patent claims so that the "holographic element" is in context with a "head-mounted display." Although "Spectacles" is listed in the original Metaio patent claims three times, out of the new 20 patent claims, 14 now focus on a "Head-mounted system."

Apple's patent Claim #13, which is now the first claim (with previous patent claims #1-12 being cancelled) states the following:

"A head-mounted system, comprising: an optical device which detects light incident on an eye, wherein the optical device is configured to: project an infrared light onto the eye, detect light reflected off the eye in response to the projected light, and track an eye movement based on the detected reflected light; and a holographic element by which light is projected into the eye to present a virtual object, wherein the light is projected based on the tracked movement."

The other patent claims mentioning a Head-Mounted system are presented below:

#14: The head-mounted system of claim 13, further comprising a position sensor system to determine a position and orientation of the head-mounted system.

#15: The head-mounted system of claim 13, wherein the light is projected into the eye is characterized differently than a detected ambient light.

#16: The head-mounted system of claim 13, wherein the system further comprises a waveguide by which the light is projected into the eye.

#17: The head-mounted system of claim 13, wherein characteristics of the holographic element are electronically changeable.

#18: The head-mounted system of claim 13, further comprising: a second optical device which detects light incident on a second eye; and a second holographic element by which light is projected into the second eye to present the virtual object.

#19:  The head-mounted system of claim 13, further comprising a sensor system configured to obtain information from an environment based on transit time of light reflected back from the environment.

#20: The head-mounted system of claim 13, further comprising computer readable code to: obtain information from an environment by a sensor system based on transit time of light reflected back from the environment.

#21: A non-transitory computer readable medium comprising computer readable code executable by one or more processors to: project, by an optical device of a head-mounted device which detects light incident on an eye, an infrared light onto the eye, detect light reflected off the eye in response to the projected light, and track an eye movement based on the detected reflected light; and present a virtual object by a holographic element by which light is projected into the eye, wherein the light is projected based on the tracked movement.

#22: The non-transitory computer readable medium of claim 21, further comprising computer readable code to determine a position and orientation of the head-mounted system based on output of a position sensor of the head-mounted device.

#28: The method of claim 27, further comprising determining a position and orientation of the head-mounted system based on output of a position sensor of the head-mounted device.

#30: The method of claim 27, wherein the head-mounted device further comprises a waveguide by which the light is projected into the eye.

A Projecting Information System 

"Continuation patents" are always about changes and/or extensions to an invention's patent claims. In this case, this is the first time this invention via Metaio has come to light. Therefore I wanted to add a little about the heart of the invention in this report so as to provide you with an overview of the invention. Considering that smartglasses could be a very possible new device from Apple in the future, this invention is key to understanding one dimension of the invention worth noting.  

Apple's patent FIG. 2 above is a schematic representation of a projecting information system #200 which is designed in the form of spectacles having two holographic elements #220 constructed as spectacle lenses #220 each arranged in front of an eye #210. On the left bow #201-L of the spectacles a projection device #242 is fastened which is capable of projecting light beams #233 onto or into the eye #210.

The projection device preferably comprises a light guiding device (not shown), for example, a scanner device which determines the momentary projecting direction of the projection device and changes it with respect to time according to a predetermined projection pattern. Naturally, an additional projection device, which is not shown, could also be fastened on the right bow 201-R.

In Apple's patent FIG. 6 above we're able to see a schematic view of a detail of an information system which has a projection device #642, a scanning device #641, a light guiding device #645 and a holographic element #620 arranged in front of an eye #610.

Apple explains that the projection device projects a light beam #633, which partially passes through a splitter mirror #644 and is directed by the light guiding device #645 by way of the holographic element onto the retina #611 of the eye. There, the projected light beam is scattered back into different directions as reflex light rays #631.

Some of the reflex light rays are focused by the lens #613 of the eye such that the beams almost parallel but opposite to the projection beam through the pupil #614. Subsequently, these reflex light rays are refracted by the holographic element, directed by the light guiding device, partially reflected at the splitter mirror and detected by the scanning device. Other rays 631b of the reflex light rays are prevented from emerging from the eye, for example, by the iris #612 of the eye.

Apple's patent focuses more on the mechanics of the invention rather than how it would be used or marketed. Though there are pockets of interesting little tidbits of information about features of the smartglasses.

For AR glasses to work accurately Apple points to a "position sensor system integrated in the information system. For example, an IR or RF triangulation device, a GPS receiver, and/or gyrosensors, which cooperates, if required, with remote components, such as fixedly positioned transmitters, satellites or the like, the position and/or the orientation of the information system can be determined.

In another point, Apple notes that "a detection of infrared light from the field of vision is of interest, for example, for detecting images from the field of vision in darkness, half-light, rain, etc., for example, in an embodiment of the information system according to the invention as low-light-level spectacles or as a driver assistance system."

In a smartglasses device, the information system is to provide users with holographic virtual objects or information that augments what the user is viewing.

Apple's patent application 20190150731 that was published today by the U.S. Patent Office was originally filed back in Q2 2002 and granted in January 2010 to Metaio. Apple acquired Metaio in 2015 along with its Intellectual Property.  Considering that this is a continuation patent application, the timing of such a product to market is unknown at this time.

Our cover graphic is an original Metaio smartglasses image. Apple was granted a smartglasses patent based on Metaio's original IP on March 12, 2019.

About Making Comments on our Site: Patently Apple reserves the right to post, dismiss or edit any comments. Those using abusive language or negative behavior will result in being blacklisted on Disqus.