Today the U.S. Patent and Trademark Office officially granted Apple a patent that relates to an Apple Pencil used with Vision Pro (or other HMD) to manipulate Virtual Objects as presented in our cover graphic.

Apple's granted patent covers selecting and manipulating virtual objects in a content generation environment using a pointing device (e.g., a stylus/Apple Pencil).

In some embodiments, the orientation of the pointing device in physical space determines the virtual object that is selected for input. In some embodiments, a selection input is received via the pointing device while the pointing device is pointed at a respective virtual object and depending on the type of selection input, either a first type of manipulation or a second type of manipulation is performed on the respective virtual object.

In some embodiments, the respective virtual object changes visual characteristics based on whether the pointing device is pointed at the virtual object or whether the virtual object has been selected for input.

Apple's patent FIGS. 4A-4B below illustrate an exemplary method of moving a virtual object according to embodiments of the disclosure. In FIG. 4A, content generation environment #400 can be displayed by an electronic device such as an HMD. In some embodiments, the content generation environment is displayed by a display generation component (e.g., a display device) that is in communication with the device (e.g., integrated with or external to the device), such as a monitor, a touch screen, a projector, a television, head-mounted display, etc.

In FIG. 4A specifically, while stylus #408 is pointed at cube #404, the device detects that hand #406 is performing “Gesture B” (e.g., with “Gesture B” indicated by the letter “B” in the torus representing the hand).

In some embodiments, “Gesture B” corresponds to a second type of selection gesture (e.g., different from Gesture A), such as a single tap-and-hold on the stylus (Apple Pencil) by a finger of a hand (e.g., a forefinger, a thumb, a middle finger, etc.).

In some embodiments, a single tap-and-hold input includes a contact with the stylus by a finger, followed by a continued contact with stylus by the finger (e.g., a continued contact of the initial touch-down, without lift-off between the initial touch-down and the continued contact).

In some embodiments, the second type of selection gesture corresponds to a request to move the cube (e.g., a request to perform a movement operation on the cube). It is understood that Gesture B need not be a single tap-and-hold gesture, but can be any gesture predetermined to correspond to a movement operation, optionally including taps and/or slides with one or more fingers, movements, and/or rotations of the stylus, etc.

In Patent FIG. 4B below, the virtual object is moved from the middle of the table in FIG. 4a to the right side of the table.

Apple's patent FIGS. 5A-5B above illustrate an exemplary method of modifying the orientation of a virtual object; FIG. 6 is a flow diagram illustrating a method of manipulating objects in a three-dimensional environment.

To review the full details of this invention, check out granted patent 12236546.