Last Thursday an Apple patent was published by the World IP Organization (WIPO) that relates to Vision Pro in that it could guide and direct a user with attention, memory, and cognition. Basically, it's a user interface that will allow a user to record their memories that are filmed with Spatial Video and be able to add notes and commentary so that years from now, the user will be able to recount with accuracy, who was in the video, where it was taken and commentary to better detail the memory.  

During WWDC23, En Kelly, Senior Engineering Program Technology Development Group stated: "Because of the powerful capabilities of Vision Pro, we were able to go even further (than making iPhone photos leap unto a gigantic panorama virtual display), adding a new dimension to photos and videos (spatial video and photos). Vision Pro is Apple's first 3D camera. It let's you capture and relive your memories in 3D with Spatial Audio" … and more as captured in the video clip below.

Side Note: Apple, for whatever reason, usually puts a request into YouTube to remove any portion of their keynote being posted in reports, even if it's promoting their products. So in the days after this report is made public, you can go to the WWDC23 video on YouTube and roughly go to the 1 hour and 34 minute mark and watch it play out covering spatial video and more.

Of course every meaningful and unnoticeable thing that Apple has invented has one or several patents supporting it. On October 12, 2023, one of Apple's patents covering Vision Pro providing users with the capability of reliving their precious memories was published by the World Intellectual Property Organization (WIPO).

Patent: Head-Mountable Device for User Guidance

Apple notes in their patent that head-mountable devices, such as head-mountable displays, headsets, visors, smartglasses, head-up display, etc., can perform a range of functions that are managed by the components (e.g., sensors, circuitry, and other hardware) included with the wearable device. The head-mountable device can provide a user experience that is immersive or otherwise natural so the user can easily focus on enjoying the experience without being distracted by the mechanisms of the head-mountable device.

Various electronic devices can be used to capture, create, and edit media, for example, by capturing images, recording audio, storing text, and other such media formats.

Electronic devices have become increasingly portable in recent years, offering more opportunities for a user to create media while experiencing everyday life. In this regard, media can be any content that is generated by a user as records, inputs, and/or other data capable of being recorded by an electronic device, such as a head-mountable device.

 In many cases, media from a user can be based on and/or inspired by interactions with and/or observations of the external environment and/or outputs from an electronic device. As such, the user may create content under circumstances that are detectable, recordable, and reproducible.

In some instances, a user may be interrupted and/or otherwise desire to resume a thought process from an earlier time. However, the circumstances may have changed since that time. Inasmuch as the user’s thoughts were mentally connected to the circumstances of that earlier thought process, it can be difficult for the user to resume such thoughts. It can be desirable to provide outputs to a user that will refresh the user’s recollection and aid the user further develop thoughts and ideas from a prior mode. It can be further desirable to provide outputs that will respond to a user’s attention state and help the user focus and/or perform activities that regain focus.

A head-mountable device can facilitate user thought processes by recording user-perceivable experiences during a first mode while the head-mountable device is operated in a capture mode. While in the capture mode, the head-mountable device can record inputs from the user. During a second mode, the head-mountable device can reproduce the previously recorded experiences as well as the user inputs so that the user can resume development of the thoughts and ideas associated with the first mode. The head-mountable device can also track the user’s conditions to monitor attention levels of the user and provide indicators to prompt a user to perform activities that will help the user refocus.

Apple even went one step further. The user will be able go back to a spatial video and add some thoughts about that recorded scene. It could be the user wants to have on record who is important in the video, where it was taken and other thoughts about the video to help a user remember finite details that they may one day simply forget. Yes, as one gets older, it's easy to forget a memory that 10, 20+ years ago. This Apple feature will definitely come in handy. How many times has an older user looked back a photo in an album and simply forgot exactly where that photo was taken or the circumstance they were there or who was in the photo. This is an added dimension that is likely to come with Spatial Photos and Videos.  

Apple's patent FIG. 1 illustrates an HMD (Vision Pro) as the device that the invention relates to; FIG. 2 illustrates a top view of a head-mountable device in use by a user; FIG. 3 illustrates a view of the head-mountable device of FIG. 2 providing a user interface in a capture mode.

Further, in FIG. 3 above, the display #140 can provide a user interface #142 that outputs the view captured by a camera. As the input devices of the head-mountable device #100 receive input from a user, such inputs can be output by the head-mountable device. For example, an output #144 can include a representation of the input received and/or recorded from the user. Such an output can allow the user to review, track, and/or monitor the user’s own input as received by the head-mountable device.  As such, the head-mountable device can facilitate receiving and recording the user’s inputs provided thereto and allow the user to make a record of thoughts and ideas generated during a first session (e.g., capture session) while the head-mountable device is in a first mode (e.g., capture mode).

To review the full details of Apple's invention, review Apple's patent application WO2023196257 published by WIPO.