Apple’s engineering teams appear to have been assigned the task of inventing a multipurpose handheld device for a future XR Headset users could use to write, draw and control video games with. We’ve covered previous Apple patents on this theme in 2024 (01, 02, and 03) In some patents the pencil-like device is rather slender while others, like this current patent, presents a rather thick device that could be grasped by the users hand. Apple’s latest patent application expands on this pencil-like tool/controller concept.

In Apple’s patent background they note that spatial tracking of input devices (e.g., handheld input devices such as a remote game controller, stylus, or another type of input device) and of user interaction with handheld input devices are optionally used for inputs (e.g., to another computing system). Yet spatial tracking techniques can be resource intensive. For example, one spatial tracking technique for a handheld input device, referred to as outside-in optical tracking, uses a constellation of light emitters (e.g., light emitting diodes (LEDs)) in the handheld input device that is driven. A computer system can capture (e.g., via one or more cameras) an image and determine the spatial location (e.g., up to six degrees of freedom posing) of the handheld input device using the captured light information (e.g., using the geometry of the constellation of the handheld input and/or the emitter wavelengths of the constellation of light emitters of the handheld input device).

Such a method for spatial tracking can be resource intensive and/or wasteful of power, especially when the constellation of light emitters is driven independent of whether one or more light emitters of the constellation of light emitters are optically occluded and/or outside of the field of view of the one or more cameras of the computer system.

Systems, Methods, and Devices including Optical Occlusion Detection

Apple’s patent provides improved systems and methods for optical tracking detect and/or act upon detection of optical occlusion at a handheld input device.

For example, the systems and methods described in Apple’s full patent application can increase synchronization of electronic devices in communication with the handheld input device, reduce power consumption (e.g., lowering a driving current demand and generation of heat) at the handheld input device. In particular, power consumption can be reduced by selectively driving one or more light emitter of a constellation of light emitters of the handheld input device in accordance with an understanding of whether one or more light emitters are optically occluded.

Present examples are directed to systems and methods for detecting local optical occlusion on a handheld input device and/or for smartly driving one or more light emitters of a constellation of light emitters of the handheld input device.

In some examples, a method includes driving the one or more light emitters, detecting an amount of light via the one or more light detectors, and determining that a first light emitter of the one or more light emitters is optically occluded by an object outside a housing of the handheld electronic device based on the amount of light detected via the first light detector.

In some examples, when the first light emitter is determined to be optically occluded, the method includes forgoes driving the light emitter at that location for outside-in tracking (or reduces the frequency or power of driving), while continuing driving the other light emitters that are not determined to be optically occluded, thus saving power for the handheld input device.

The input device #230 of FIG. 2 below can include a housing #290 (better seen in FIG. 3 below) that is partially or fully transparent to one or more light emitter wavelengths (e.g., optionally a range of wavelengths). In some examples, the housing includes a cylindrical body or another body shape optionally with a tip portion.

In some examples, the tip portion can be part of a unibody housing and in some examples, the tip portion can be removable from the body.

In some examples, the housing of the input device can include a cylindrical body or another body shape optionally without a tip portion (e.g., opposite ends of the input device housing have similar appearances or dimensions).

The housing can include an ergonomic depression (or multiple ergonomic depressions), optionally as a guide for placement of one of a user's fingers (e.g., thumb or index finger).

The input device can be held by a user in different ways (e.g., a range of hand poses). The ergonomic depression can optionally assist the user achieving a variety of poses and/or enable the user to provide additional inputs (e.g., to find a touch sensitive region with a particular finger).

Overall, the sensors in the input device can include optical sensors (#240 of FIG. 2) among other possible sensors. In some examples, the optical sensors are distributed in the input device. In some examples, THE input device can include an outward facing camera (#250 of FIG. 2), a beacon transmitter (#246, e.g., using any electromagnetic signals), an ultrasonic sensor (and/or transmitter) (#258), a force sensor (#244, e.g., such as a strain gauge, capacitive-gap force sensor, a piezoelectric sensor #242), an inertial measurement unit (IMU) (#260) (and/or other motion or orientation device such as an accelerometer or gyroscope), a capacitive electrode or other capacitive sensor (#248), a radiofrequency sensor (and/or transmitter) (#252), an infrared sensor (and/or transmitter) (#254), a magnetic sensor (and/or generator) (#256), among other suitable sensors.

For full details, check out Apple’s patent application 20250110579.

Apple Inventors

• Tong Chen: Optical Sensing Engineer
• Xiaoyang Zhang: Manager, Display Quality Engineering
• Christopher Ewy: Product Design Engineer
• Ian Colahan: Product Design Manager