Apple Envisions the Workplace & Auditoriums adopting a next-gen Communications System delivering AR/VR Content to HMDs through Walls
The U.S. Patent and Trademark Office published a newly granted patent for Apple in the last few weeks that relates generally to wireless communication systems and techniques for directing directional wave signals between components of a wireless communication system.
More specifically, Apple envisions the future workplace and other facilities (like schools and beyond) adopting their newly advanced wireless communications system that supports millimeter wave signals capable of delivering rich AR and VR content to future headsets at 1Gbps. More importantly, delivering such content to a workforce dedicated to wearing some form of Head Mounted Display Device (HMDs) – be it AR Glasses or a full-blown Virtual Headset.
While Apple doesn't provide us with specific marketing examples of what such a communications system will deliver, it's clear that professionals along with everyday workers and students will be able to communicate at a much higher level with Apple's new system using next-gen AR/VR applications.
For teachers, they'll be able to pull up and transmit 3D maps or imagery directly to student's wearing HMDs in the class to make the subject more realistic and meaningful; to allow the teacher to manipulate a 3D object while explaining various properties of the object to students and thereby delivering an all-new richer teacher-student experience.
A Surgeon at a hospital could be waiting for the latest image of a brain scan of a patient who was in a bad car accident. Time is everything to know if the patient could survive surgery or not. Receiving richly detailed imagery of a patient's brain scan could be sent to the doctor's headset where they could manipulate the image in real time and make a life-saving decision on the spot.
While some of this is being done today using iPad's, a future Headset on a next-gen communications system will be capable of offering surgeons' much greater detail of scans at much faster speeds.
Apple's newly granted patent presented in-part below provides you with a detailed look at their new communications system designed for the next generation of communications where including AV and AR content will be commonplace in the not-too-distant future. Though realistically, we're talking years here, not months.
Apple's patent filing they explain that in virtual or augmented reality applications, a user device may wirelessly communicate significant amounts of data with a base station using a millimeter wave.
In fact, millimeter waves having wavelengths between 1 millimeter and 10 millimeters (e.g. 30-300 GHz waves) may communicate data at rates at or greater than 1 gigabyte per second.
However, as the user moves within a space, such as a room, objects may block a current link path between the user device and the base station. In such situations, the user device or base station may perform a scan or sweep to identify a new link path (if available) or may lose communications between the user device and base station.
Yet even if a new link path is available, communication between the user device and the base station may stop while a scan or sweep is being performed to identify the new link path. This may cause a virtual reality or augmented reality scene being displayed on the user device to freeze, or otherwise become non-responsive.
Such interruptions may produce a negative user experience. Also, in other applications, such as streaming video or audio, such interruptions may cause a negative user experience.
Collecting Positional Data
In some embodiments, position information about a user device may be received from one or more position sensors already included in the user device.
For example, many user devices may include GPS systems, and/or inertial measurement units, such as gyroscopes and/or accelerometers, that collect position information about a current location or orientation of the user device.
In some embodiments, such position information may be used by applications such as a virtual reality application that adjusts a display scene based on a position or orientation of the user device.
In some embodiments, such position information collected by position sensors of a user device may also be provided to a beam forming decision engine to determine a link path between a base station or relay dock and the user device.
Beam Forming Decision Engine + a Relay Dock or Base Station
In some embodiments, a communication station, such as a relay dock or base station, may further utilize position information about a user device to hand-off communications with the user device between a base station and a relay dock or between relay docks.
For example, in some embodiments, a beam forming decision engine may determine a direction of travel of a user device or a predicted change in orientation of a user device and may predict a future location or future orientation of the user device based on the determined direction of travel or the predicted orientation change.
In some embodiments, a beam forming decision engine, may also identify a relay dock or a base station that has a link path to the predicted future location or future orientation of the user device that is not obstructed.
For example, a beam forming decision engine may determine a next communication link device (e.g. base station or relay dock) and a next communication link path from the next communication link device prior to losing a current communication link or prior to the current communication link being degraded more than a threshold amount.
In such embodiments, the beam forming decision engine may switch communications between the user device and a base station from a current link path to a new link path that utilizes a next relay dock or next base station.
In some embodiments, such communication link changes may be performed without performing a sweep or scan operation to identify the next relay dock or the next base station.
Also, in some embodiments, a beam forming decision engine may predict a future location of the user device relative to the next relay dock or the next base station, such that scanning or sweeping to identify a beam direction for a directional wave signal to be transmitted from the next relay dock or base station to the user device may be omitted or an amount of sweeping or scanning may be reduced.
For example, a beam forming decision engine may use position information to determine how to form a beam directed at a predicted location or orientation of a user device without needing to sweep or scan for a link path to the user device at the predicted location or orientation. In this way, interruption time when changing link paths may be reduced or eliminated, which may improve application performance and user experience.
In Apple's patent FIG. 1A below we're able to see an illustration of a wireless communication system comprising a base station, a relay dock, and user devices; FIG. 1B is a block diagram illustrating the components of a relay dock.
Apple's patent FIG. 3A below is a block diagram illustrating components of a communication station, such as a base station or a relay dock; FIG. 3B illustrates a wireless communication system that includes relay docks with cameras or infrared sensors.
Apple's patent FIG. 5A below is a block diagram illustrating components of a user device; FIG. 5B is a more detailed view of position/motion sensors of a user device.
Apple's patent FIGS 8-10 illustrate alternative layouts or configurations depending on the room and whether there are obstructions. Specifically, FIG. 8 below illustrates a wireless communication system comprising multiple base stations and multiple cameras or infrared sensors; FIG. 9 illustrates a wireless communication system comprising wall-mounted relay docks; FIG. 10 illustrates a wireless communication system comprising floor-mounted relay docks.
Apple's patent FIGS. 4A-4C below illustrate a user device that emits a light signal and relay docks that include cameras or infrared sensors.
Lastly, Apple further noted that in some embodiments, relay docks may be mounted in a combination of locations. For example, in a very large room or venue such as an auditorium, relay docks may be mounted on side walls, a ceiling of the auditorium, and on the floor of the auditorium. In some embodiments, various other combinations of mounting locations may be used for relay docks and/or base stations.
The best way to show off their larger communications system for auditoriums would be for Apple to demonstrate it live with attendees at an Apple event at the Steve Jobs theater with attendees wearing their next-gen Head Mounted Display Device in action. Only time will tell if we'll see such a demonstration in the future.
Apple's granted patent was issued weeks ago by the US Patent and Trademark Office. It was originally filed in H2 2018. Some of the work incorporated into this patent dates back to H2 2017.
All five inventors work on Apple's advanced Antenna Design Team. One engineer, Mr. Wu, has worked on novel imaging radar systems, radar antenna design, miniaturized antenna design and more.