In a WSJ interview  last month, Apple's Craig Federighi explained that Apple teams work on various parts of an invention over a very long period of time before an invention comes to market, if it comes to market. One particular project that Apple's engineers have been working on is delivering a keyless keyboard so that an input region could morph into several kinds of input interfaces for typing playing music, playing games, drawing and so forth. One of Apple's first patent filings on this project was covered way back in a 2012 with other Patently Apple reports posted over the years (01, 02 and 03).  

Today the U.S. Patent and Trademark Office officially granted Apple a patent that greatly advances their keyless keyboard project in greater detail with patent figures illustrating multiple morphing interface options that are on the drawing board.

One such interface that is illustrated is one that will be able to control iTunes / Apple Music. While the illustration is clear, it's limited. The second display could of course provide greater details such as colorful equalizer controls and much more. Apple also quickly mentions an interface with virtual "gaming inputs" as another possible option supported by this keyless keyboard.

The other major advancement in this project focuses on delivering realistic touch on this virtual keyboard. The second display is designed to be flexible and in-tune with force sensors and a unique haptic board beneath the second display.

Lenovo who brought a virtual keyboard notebook to market, as shown in our cover graphic, simply used a regular LCD display with no give for typing, making it more like a smartphone's virtual keyboard. Apple is going out of their way to provide a whole new experience closer to the feel of a physical keyboard.

Apple notes that traditional computing input devices, such as mice, keyboards, and trackpads, tend to operate using dedicated keys or buttons. The operation of each key or button may be tied to a particular function or command. However, traditional input devices lack the flexibility to accommodate expansive features offered by newer devices, operating systems, and software. As a further drawback, the dedicated keys or buttons of traditional input devices are unable to adapt to different user needs and preferences.

Alternative input devices, such as touch-input devices, appear to offer some greater flexibility for input scenarios and customization than mechanical keyboards, mice, and similar input devices. However, touch sensitive input devices often have a flat, inflexible input surface that gives little or no tactile feedback to a user and may therefore be less desirable for many scenarios than traditional input devices.

Apple's invention covers a keyless keyboard with an input surface having multiple differentiated input regions. The keyless keyboard will provide multiple force touch areas. 

The multiple differentiated input regions may correspond to keys of a keyboard. The multiple differentiated input regions may be visually differentiated on the top member. The input device may be configured to detect a key press of a particular input region by detecting, within a given group of the differentiated input regions, both a touch location and a force value satisfying a force threshold. The input device may further comprise a haptic output system configured to produce a tactile output in response to detecting the key press.

The keyboard may be incorporated into an electronic device that includes a display coupled to the enclosure, wherein the display is distinct from the keyboard. The input surface may include input regions representing character input keys, and the first actuator and the second actuator may be configured to provide haptic feedback to a user to induce a sensation representative of a mechanical key.

One example of such an input device is a keyboard that does not have mechanical or movable keys. Instead, the keyboard may have a flat, keyless input surface, such as a glass or metal layer, and may include touch and/or force sensing systems to determine when a user touches and/or presses on the surface. Haptic actuators may provide physical feedback to indicate that a user has pressed the keyless surface with sufficient force to register an input. The haptic actuators may induce a physical sensation that is similar to or representative of a mechanical key. For example, when a user presses the surface of the keyboard with sufficient force, the surface may vibrate or otherwise move to indicate to the user that the intended input has been registered.

For local force sensing, the keyboard may determine an amount or magnitude of force applied to a particular location or locations on the surface. One example local force sensing system uses a pixelated capacitive sense layer below the surface of the keyboard. When pressed, the user's finger may form a depression in the keyboard surface beneath the finger. The pixelated capacitive sense layer may detect the depth and/or location of the depression to determine both an amount and a location of a force. Keyboards may use either global or local force sensing alone, or they may use a combination of these techniques.

Because the keyboard doesn't have mechanical keys, the keyboard may provide numerous other features and functions beyond mere keyboard input. For example, the keyboard may include an adaptive display to render visual information, such as an outline of an input region (e.g., representing a key) and an indication of its function (e.g., a glyph). In this way, the location, size, spacing and/or arrangement of the keys may vary. As another example, the input surface of the keyboard may act as a touch pad to detect touch inputs (e.g., moving a cursor, manipulating user interface elements) as well as typing inputs.

Apple's patent FIG. 14 below depicts an example process of detecting a key press; FIGS. 15A and 15B depict generation of input regions according to user interaction. Users will be able to choose a straight keyboard setup or one that is ergonomically laid out as shown in FIG. 15C.

An input device that includes force sensing, haptic outputs, and an adaptive display may be used to define user interfaces other than traditional keyboards. Apple's patent FIG. 16 below depicts an example input device #1600 incorporated within a notebook or laptop computer in which an alternative user input is produced on an adaptive input surface #1604. The interface supports iTunes / Apple Music; FIG. 17 illustrates virtual keys and a virtual trackpad. While a browser is open on the MacBook, the virtual trackpad is moved higher than normal so as to make it easier to navigate.

Apple's patent FIGS. 18-19 above depict example embodiments of input devices such as an iMac and iPad Pro.

Apple notes that FIG 6B below depicts an example haptic actuator #601 that may produce localized deformations or deflections of a cover display. Apple further notes that the haptic actuator may be configured to retract and/or extend (FIG. 6C) vertically (as illustrated above) to impart tactile outputs.

Apple's patent FIG. 6E above shows us another example arrangement of haptic actuators #601 relative to the cover. In this example, the haptic actuators may be formed from a single sheet or otherwise interconnected via one or more connecting elements #606.

The connecting elements may be the same material that formed the compliant layers #602 of the actuators themselves. In some cases, the haptic actuators and the connecting elements may be an integrated structure.

Apple's patent FIGS. 11A-B depict simplified cross-sectional views of a force sensing system, showing example strain gauges.

For more details, review Apple's granted patent US 11500538 B2.