Today the US Patent & Trademark Office published a patent application from Apple that relates to Apple's future mixed reality headset or HMD device. More specifically, the invention relates to a secondary audio system. The HMD device will come with built-in speakers but it appears that Apple may have invented an accessory that will provide an advanced audio system in the form of over-the-ear cups, that don't have a headband. Instead, Apple has invented a magnetic system that securely holds the ear cups to the sides of the HMD device while allowing the cups to pivot so as to adjust to a user's comfort level. When the advanced audio ear cups are on the HMD device,  it provides the user with a menu and indicators that allow the user to adjust the audio as desired but also communicate with the user when, for instance, a game scene is about to show an explosion that may be loud so that the user could adjust the sound accordingly. It's a pretty cool invention to be sure.

In Apple's patent background they note that virtual reality systems provide content to a user, which may include visual and aural content. The visual content may be provided by a head-mounted display unit (HMD), while the aural content is often provided by headphones that are physically separate from the head-mounted display unit. Apple's new invention has a different idea of how advanced audio could be implemented for a mixed reality headset. 

Head-Mounted Displays with Detachable Speaker Units

Apple's patent covers implementations of display systems that include a head-mounted display unit and an audio output device, such as an audio headset. The audio output device is detachable from or is otherwise movable relative to the head-mounted display unit in various manners.

The main headset is designed as a mixed reality to view VR and a means of view the exterior environment to provide augmented reality. The invention could also apply to Apple's cheaper model of a headset that uses an iPhone as its primary display.

Aural content provided from the audio output device and/or a built-in audio output device may be output according to the position of the movable audio output device in various manners. Visual content may also be provided according to the position of the movable audio output device in various manners.

Audio signal processing of the aural content output by the detachable speaker unit may change as the detachable speaker unit is moved toward a coupling location on the head-mounted display unit.

The audio signal processing includes one or more of changing a volume, equalization, or dynamic range of the aural content output by the detachable speaker unit.

When the detachable speaker unit is in a field of view of the head-mounted display unit, the visual content may include a visual indicator in spatial proximity to the detachable speaker unit. The visual indicator may indicate a sound output capability of the detachable speaker unit to the user.

In another aspect, a display system includes a head-mounted display unit and an audio headset. The head-mounted display unit includes first and second built-in speakers. The audio headset includes first and second detachable speaker units that are detachably coupleable to the head-mounted display unit.

In another aspect, a method is provided for providing content with a display system having a head-mounted display unit, a movable audio output devices that is movable relative to the head-mounted display unit, a controller, and a sensor.

The method includes determining, by the controller with the sensor, a position of the movable audio output device, and providing content with the head-mounted display unit according to the position of the movable audio output device.

Apple's patent FIG. 1B below is a top view of the display system. You'll note that there isn't a headphone band going across the user's head because the cups are independent units; FIG. 1C is a schematic view of the display system; and FIG. 1D is a schematic view of a detachable speaker unit of the display system.

The patent figures presented below illustrate the magnetic of the head in various views including how the headphone cup could pivot for comfort while its supporting system is firmly attached to the Head-Mounted Display device. The illustrations 4A and 4B show how magnets in the detachable speakers will allow for the two cups to mate for storage where the inner cups are protected. 

In the patent figures below, Apple illustrates what a user will see when the headset is on their heads and how the audio could have various modes. Just like expensive headphones that offer users the choice of mono, stereo or surround sound-like effects, Apple's detachable speakers will provide users with audio that is based on distance from the user's eyes. Motion and other sensors could determine what setting is best and provide an indicator for the user as shown in figures 10B and 10C.

The visual indicators could appear on the screen in the form of an animation, an icon or other indicator which may be in spatial proximity to the audio headset. Apple notes that "The visual indicator (#1060 or #1070) may suggest or otherwise indicate the sound output capability of the audio headset and/or provide instructions for use thereof."

In one example Apple notes that "the visual indicator is an audio-related icon, such as sound waves (as shown), a depiction of traditional headphones, or other icon suggestive of the audio capabilities or use of the audio headset."

In another example, Apple notes that "The visual indicator may visually pulsate according to a beat of the sound or music (e.g., faster or slower according to a faster or slower beat, respectively, of the sound) or intensity of a sonic event (e.g., increasing in size and/or pulsating faster coincident with a real or virtual event expected to produce a loud and/or intense sound, such as an explosion).

Lastly, the visual indicator may be positioned in spatial proximity (e.g., in close proximity, overlaying, and/or surrounding) to the audio headset and/or the detachable speaker units individually. The location of the audio headset may be determined, for example, by using the sensors of the head-mounted display unit (e.g., the camera for video recognition of the audio headset, or the motion sensors) and/or the sensing electronics of the detachable speaker units (e.g., camera, motion sensors, etc.).

Apple's patent application 20190387299 that was published today by the U.S. Patent Office was filed back in Q3 2018. Considering that this is a patent application, the timing of such a product to market is unknown at this time.

Some of Apple's Inventors

Neal Evans: Acoustic Design Engineer

James Vandyke: Product Design Engineer

Rob Silfvast: Engineering, Technology, Product Development Leadership

Chris Eubank: Engineering Manager, Audio