A 4th Patent this Quarter focuses on a possible future Handheld Controller system with added Tracking Devices for HMDs, Macs+
The U.S. Patent and Trademark Office has published a series of patent applications from Apple (01, 02 and 03) in Q1 relating to a possible future hand controller for HMDs, Macs, a TV (or TV box) and gaming devices with each patent offering new insights. Last Thursday, a fourth patent in this family was made public and added yet more insight into this possible handheld controller and tracking device.
Coincidentally, MacRumors has posted a rumor report this morning stating that "Apple has internally tested a new Apple Pencil with visionOS support, according to a source familiar with the matter."
Tracking Devices For Handheld Controllers
Apple notes that electronic devices that are configured to be held in the hand of a user may be used to gather user input and to provide a user with output. For example, electronic devices that are configured to control one or more other electronic devices, which are sometimes referred to as controllers, handheld controllers, input devices, or handheld input devices, may be used to gather user input and to supply output or with games. If the user wishes to switch to an application that requires more precise position tracking, such as a game or creative application, the user may expand the housing of the controller.
A handheld input device may, as an example, include an inertial measurement unit with an accelerometer for gathering information on input device motions such as swiping motions, waving motions, writing movements, drawing movements, shaking motions, rotations, etc., and may include wireless communications circuitry for communicating with external equipment such as a head-mounted device, may include tracking features such as active or passive visual markers that can be tracked with an optical sensor in an external electronic device, may include input devices such as touch sensors, force sensors, buttons, knobs, wheels, etc., and may include sensors for gathering information on the interactions between the handheld input device, the user's hands interacting with the input device, and the surrounding environment. The handheld input device may include a haptic output device to provide the user's hands with haptic output and may include other output components such as one or more speakers, visual indicators, etc.
One or more handheld input devices may gather user input from a user. The user may use the handheld input devices to control a virtual reality or mixed reality device (e.g., head-mounted equipment such as glasses, goggles, a helmet, or other device with a display). During operation, the handheld input device may gather user input such as information on interactions between the handheld input device(s) and the surrounding environment, interactions between a user's fingers or hands and the surrounding environment, and interactions associated with virtual content displayed for a user.
The user input may be used in controlling visual output on a display (e.g., a head-mounted display, a computer display, etc.). Corresponding haptic output may be provided to the user's fingers using the handheld input device. Haptic output may be used, for example, to provide the fingers of a user with a desired sensation (e.g., texture, weight, torque, pushing, pulling, etc.) as the user interacts with real or virtual objects using the handheld input device.
Haptic output can also be used to create detents, to provide localized or global haptic feedback in response to user input that is supplied to the handheld input device, and/or to provide other haptic effects.
Handheld input devices can be held in one or both of a user's hands. Users can use the handheld input devices to interact with any suitable electronic equipment. For example, a user may use one or more handheld input devices to interact with a virtual reality or mixed reality system (e.g., a head-mounted device with a display), to supply input to a desktop computer, tablet computer, cellular telephone, watch, ear buds, or other accessory, to control household items such as lighting, televisions, thermostats, appliances, etc., or to interact with other electronic equipment.
A tracking device may be removably attached to a handheld input device to enhance the precision with which an external electronic device tracks the location of the handheld input device.
The tracking device may serve as a secondary input device, may have larger dimensions than the handheld input device, may have different tracking features than the handheld input device, and/or may have an expandable housing so that visual markers on the tracking device can be tracked with enhanced accuracy.
If desired, the tracking device may include input-output devices to enhance the input-output capabilities of the handheld controller or other object. The tracking device may include input-output components, haptics, sensors, a battery, and/or other circuitry and may have a recess or other engagement structure configured to attach to the tip of a handheld controller. When the tracking device is coupled to the handheld controller, the tracking device and handheld controller may form a combined handheld input device (e.g., for a head-mounted device or other electronic device), with greater tracking precision and additional input-output capabilities provided by the tracking device. When the tracking device is removed from the handheld controller, the tracking device and handheld controller may be operated independently of one another and/or may be used as two separate handheld controllers for a head-mounted device or other device.
For full details, review Apple's patent application 20240094831.
A Few other Vision Pro / HMD Patents
- 20240094548: Electronic Devices With Movable Optical Assemblies
- 20240094549: Electronic Devices With Drop Protection
- 20240094534: Display Systems Having Imaging Capabilities
- 20240095879: Image Generation With Resolution Constraints
Comments