On Tuesday, Patently Apple posted a granted patent report titled "Apple wins a patent for a possible Future Controller Device for Apple TV, Gaming & more." Today the US Patent & Trademark Office published a patent application from Apple for a controller device that's more focused on video games.  

Apple's invention covers a controller device that can be used with networked devices, such as computers, tablet computing devices, video streaming media player devices, head-mountable devices, virtual reality devices, augmented reality devices, and/or gaming devices.

The controller device includes at least one controller segment. Each controller segment can include an interface portion for a user to operate and provide inputs. The inputs can include buttons, joysticks, and/or touch input.

In some applications, the controller segments can also detect positioning or motion of the user's hands for navigating augmented reality or virtual reality programming.

For other uses, the controller segments can be attached together for other uses, such as navigating interfaces or gaming activities. The controller segment can releasably engage with a mating controller segment via an engagement mechanism. The controller segments can engage with each other directly without any intermediate devices. When the controller segments are attached, the controller device can provide additional inputs and/or provide a familiar user experience.

Controller devices can provide a user with multiple modes of operation. For example, the user can hold and operate the controller device in a first configuration for control of a first device and/or first mode of a device (e.g. augmented reality or virtual reality programming), and the user can hold and operate controller segments of the controller device in a second configuration for control of a second device and/or second mode of a device (e.g. navigating an interface). The controller segments can be selectively engaged and disengaged to facilitate user operation in the different modes.

Apple's patent FIG. 1 illustrates an elevation view of a controller device; FIG. 2 illustrates an elevation view of the controller device of FIG. 1 with the controller segments separated; FIG. 3 illustrates a perspective view of a controller device with the controller segments separated; FIG. 4 illustrates a perspective view of the controller device of FIG. 3 with the controller segments joined.

Apple's patent FIGS. 7-9 cover Engagement Mechanisms for separating the controller into two segments.

As illustrated in FIG. 1, the body segments #112a and #112b of each controller segment #110a and #110b can have a form factor or shape configured to be held in a user's hand independent of the other controller segment. As can be appreciated, each controller segment can be used independently while in either the first position or the second position. For example, in the first position or “aligned” position, the controller segments can have a form factor that allows the user to grasp the handle portions #140a and 140b between the user's fingers and palm, leaving the user's thumb to interact with the interface portions #116a and 116b.

Further, in the second position or “pistol” position, the controller segments #110a and 110b can have a form factor that allows the user to grasp the handle portions #140a and 140b between the user's fingers and palm while the movable portions #113a and 113b extend away from the handle portions   leaving the user's thumb to interact with the interface portion.

Later Apple points to several sensors in the controller such as motion, proximity, orientation, accelerometer, chemical, optical, ozone, touch, force IR, magnetometer, temperature and other sensors.

The sensors can include one or more sensors for tracking position, movement, orientation, and/or one or more other characteristics of the controller device during user. For example, the controller device can include a camera for capturing a view of an environment external to the controller device. The camera can include an optical sensor, such as a photodiode or a photodiode array. Additionally or alternatively, the camera can include one or more of various types of optical sensors that are arranged in various configurations for detecting user inputs. The camera may be configured to capture an image of a scene or subject located within a field of view of the camera.

For more details, review Apple's patent application number 20230233929.

Some of the Team Members on this Apple Project

• Fletcher Rothkopf: Senior Director, Technology Development Group
• Erin Bosch: Product Design Engineer
• Stephen Dey: Director / Head of Product Design Engineering (left Apple in late 2022)