It appears that certain Apple engineering teams are working on new forms of Apple Pencil that could work with the current or future versions of Apple Vision Pro. On January 11^th, Patently Apple posted a report titled "Apple invents a new kind of Hand Controller that is designed to work with HMDs …" then on January 30, we posted another IP report titled "Apple wins a patent for an Apple Pencil with motion tracking to work with Apple Vision Pro to assist users peer around 3D objects in games +." Today the U.S. Patent and Trademark Office officially published a patent application from Apple that once again relates to a future version of Apple Pencil working with Vision Pro that includes a specialized sleeve integrated with sensors that Vision Pro could recognize.   

Apple Pencil with New Specialized Sleeve

According to Apple's patent filing, a system may include an electronic device such as a head-mounted device and a handheld input device for controlling the electronic device. The head-mounted device or other device may have a display configured to display virtual content that is overlaid onto real-world content.

The handheld input device may include a form of Apple Pencil that  includes a removable sleeve. The input-output capabilities of the handheld input device may be shared between the stylus (Apple Pencil) and the removable sleeve.

Apple Pencil may include touch sensor circuitry, a force-sensitive tip, and a motion sensor. The sleeve may include conductors for translating touch input on the sleeve to the touch sensor circuitry on Apple Pencil, a deformable member for translating forces on the sleeve to the force-sensitive tip of Apple Pencil, and visual markers that can be detected by an external camera (on Vision Pro or other HMD) and used with motion sensor data from Apple Pencil to track the handheld input device. The removable sleeve may include haptic output devices and a battery and may be attached to an item without electronics.

A handheld input device may, as an example, include an inertial measurement unit with an accelerometer for gathering information on input device motions such as swiping motions, waving motions, writing movements, drawing movements, shaking motions, rotations, etc., and may include wireless communications circuitry for communicating with external equipment such as a head-mounted device, may include tracking features such as active or passive visual markers that can be tracked with an optical sensor in an external electronic device, may include input devices such as touch sensors, force sensors, buttons, knobs, wheels, etc., may include sensors for gathering information on the interactions between the handheld input device, the user's hands interacting with the input device, and the surrounding environment.

The handheld input device may include a haptic output device to provide the user's hands with haptic output and may include other output components such as one or more speakers.

One or more handheld input devices may gather user input from a user. The user may use the handheld input devices to control a virtual reality or mixed reality device (e.g., head-mounted equipment such as glasses, goggles, a helmet, or other device with a display). During operation, the handheld input device may gather user input such as information on interactions between the handheld input device(s) and the surrounding environment, interactions between a user's fingers or hands and the surrounding environment, and interactions associated with virtual content displayed for a user.

The user input may be used in controlling visual output on a display (e.g., a head-mounted display, a computer display, etc.). Corresponding haptic output may be provided to the user's fingers using the handheld input device. Haptic output may be used, for example, to provide the fingers of a user with a desired sensation (e.g., texture, weight, torque, pushing, pulling, etc.) as the user interacts with real or virtual objects using the handheld input device.

Haptic output can also be used to create detents, to provide localized or global haptic feedback in response to user input that is supplied to the handheld input device, and/or to provide other haptic effects.

Handheld input devices can be held in one or both of a user's hands. Users can use the handheld input devices to interact with any suitable electronic equipment. For example, a user may use one or more handheld input devices to interact with a virtual reality or mixed reality system (e.g., a head-mounted device with a display), to supply input to a desktop computer, tablet computer, cellular telephone, watch, ear buds, or other accessory, to control household items such as lighting, televisions, thermostats, appliances, etc., or to interact with other electronic equipment.

A sleeve may be used to convert items into handheld input devices and/or to enhance the input-output capabilities of other input devices. The sleeve may include input-output components, sensors, and/or other circuitry and may be configured to wrap around an item that may or may not contain any electronics or circuitry.

In some arrangements, the sleeve may wrap around an item without electronics such as a pen, a pencil, a paint brush, an eating utensil, or other handheld item.

When the sleeve is placed on the item, the user can use the item normally (e.g., by writing with the pen or pencil, eating with the eating utensil, and/or performing other tasks with the item), while the sleeve provides input-output capabilities by tracking the motion of the item, sensing information about the environment, providing haptic feedback, etc.

In some arrangements, the sleeve may wrap around an Apple Pencil or other input device. With this type of arrangement, the sleeve may enhance the existing input-output capabilities of Apple Pencil and may enhance the input-output capabilities of the sleeve.

When the sleeve is located on Apple Pencil, the sleeve and Apple Pencil may form a combined handheld input device (e.g., for a head-mounted device or other electronic device) with both the input-output capabilities of the sleeve as well as the input-output capabilities of Apple Pencil. When the sleeve is removed from Apple Pencil, it may be used normally (e.g., by providing input to a touch screen). This allows the user to easily switch between electronic devices using a single input device.

Apple's patent FIG. 2 below is a perspective view of an illustrative handheld input such as an advanced version of Apple Pencil; FIG. 3 is a diagram of an illustrative system including Apple Pencil with a sleeve and an electronic device such as a head-mounted device (Vision Pro).

For full details, review Apple's patent application 20240061520.