Apple Invents a Next-Gen Apple Pencil that could work on an iPhone, on non-touch surfaces and with AR/VR/MR Applications
Today the US Patent & Trademark Office published a patent application from Apple that reveals a possible new Apple Pencil that could be used with an iPhone, perhaps to challenge stylus smartphones from Samsung. Further, this next-gen Apple Pencil could be used with AR/VR/MR applications.
Apple Pencil with Optical Sensors
Apple's invention covers a next-gen input device (Apple Pencil) that includes optical sensors that can be used for content generation on an iPhone or surface that isn't touch-sensitive.
In some examples, optical sensors can include optical flow sensors and/or laser speckle flow sensors that can detect spatial-temporal image brightness variation (e.g., frame-to-frame brightness variation) to estimate motion of the input device. In some examples, an optical flow sensor can include an emitter (e.g., a light-emitting diode (LED)) to emit light to a target (e.g., surface), a detector (e.g., an image sensor) that captures an image of an environment to detect light reflected from or incident on the target (e.g., surface) in response to light emitted from the input device, and a lens or lens array configured to collimate or condition emitted light beams.
Laser speckle flow sensors can detect speckle patterns resulting from coherent illumination of an imaging configuration to estimate motion of the input device. In some examples, a laser speckle flow sensor can include an emitter (e.g., vertical-cavity surface-emitting laser (VCSEL)) to emit light to a surface and a detector (e.g., an image sensor) that captures an image of an environment to detect light reflected from or incident on the surface in response to light emitted from the input device.
In some examples, in addition to emitter and detector, the laser speckle flow sensor includes a lens or a lens array configured to collimate or condition emitted light beams. In some examples, the laser speckle flow sensor does not include a lens or a lens array. It should be appreciated that the laser speckle flow sensors can detect a speckle pattern (e.g., interference pattern) in an imaging configuration to estimate motion of the input device without a lens or lens array.
In some examples, the optical sensors can be used to detect characteristics of the input device including position, orientation, and/or motion of the input device.
In some examples, some or all of the characteristics of the input device can be used in processing to generate content, including textual character input and three-dimensional objects. In some examples, the generation of content can use information from one or more additional sensors of the input device and/or from additional devices in combination with the characteristics of the input device based on the optical sensors.
In some examples, the information acquired from the one or more additional sensors can include force applied by the input device, force applied by a stylus tip, inertial measurements, magnetic measurement, and/or computer vision measurements.
Apple's patent FIG. 1A below illustrates an exemplary mobile telephone (iPhone) that includes a touch screen #124 that can accept input from an input device, such as an active stylus, via a touch-sensitive surface (e.g., touch screen 124) and/or via a non-touch-sensitive surface; FIG. 2 illustrates a block diagram of an example system including an example computing system and an example input device.
Apple's patent FIG. 8 above illustrates a block diagram of an example input device including one optical sensor; FIG. 11 illustrates an example stylus input device configuration including example trackball is disposed at a tip of a stylus input device.
In some examples, the trackball 1104 can be disposed partially within the stylus body 1100. The optical sensor(s) 1102 are configured to track the trackball 1104 to estimate displacement of the stylus as the stylus is moved across a non-touch-sensitive surface 1106. Although disposing the trackball 1104 at the stylus tip prevents the optical sensor(s) 1102 from accurately measuring displacement with respect to the z-axis, tilt, orientation, and/or axial rotation of the stylus, disposing the trackball 1104 at the stylus tip causes the surface 1106 to be less prone to scratch and contamination, to provide improved signal-to-noise ratios when detected by the optical sensor(s) 1102, and to be agnostic to surface optical properties.
A system using an input device with optical sensors can in some cases be used to provide input to an augmented, virtual or mixed reality application.
Lastly, the stylus, comprising of at least two optical sensors, can be configured to track displacement of the stylus in three, four, five, or six dimensions.
For full details, review patent application 20240004483.
Comments