A patent that was filed in Q3 2011 (which was published in Q3 2012) illustrating a future iPad cover with a built-in keyboard. In Q4 2012 Microsoft introduces their first Surface tablet with a cover with a built-in keyboard. In Q3 2011 Apple filed a patent (which was published in Q2 2013) illustrating a notebook with a removable display as a tablet. The system used magnetics to hold the display until it was removed. Microsoft introduces Surface Book in Q4 2015 which is a notebook with a removable display. It too uses magnets to hold the display in place until it's removed. Apple owns half a dozen patents or more covering in-Air gesturing for PC's dating back to at least Q3 2012. So what is Microsoft aiming for now? I'll give you one guess.
3D Depth Device Camera to Support In-Air Gesturing
Microsoft's patent FIG. 1 shows an example environment in which a user interacts with one or more computing devices via hand gestures.
More specifically, Microsoft's patent FIG. 1 shows a user interacting with a future Surface tablet or Surface Book (#104). To facilitate detection of hand gestures performed by the user, Surface Book will include an infrared (IR) camera #106 which together constitute a machine-vision system #107 that may be used to interpret the hand gestures performed by the user.
While the IR camera is not a conventional 3D camera, the IR camera may be used estimate depth--defined as a coordinate (e.g., Cartesian z) parallel to the optical axis of the IR camera--and thus estimate the three-dimensional shapes of objects such as the hands of the user. Microsoft is likely to use Intel's RealSense camera or they may invent their own.
When three-dimensional gestures are performed by one or both of the hands of a user and sensed by the IR camera, it may may be interpreted to affect various aspects of surface Book (or other Surface PC).
As shown in FIG. 1, gestural input supplied by the hand of the user is interpreted to affect aspects of an operating system such as causing to open a file running on the Surface device which will flash on the display "Filed Opened."
It will be appreciated that any future Surface PC with such a camera may receive other types of natural user input (NUI) in addition to hand gestures. For example, the Surface PC may interpret speech via Cortana or eye movements expressed by the user which is commonly known as gaze controls.
Moreover, the future Surface PC may interpret gestures performed by body parts of the user other than their hands. In some examples, the system may be configured to receive input via so-called conventional input devices (e.g., keyboard, touch-screen, mouse, game-system controller, wireless remote). Further, the IR camera may be used to image non-human subjects, including inanimate objects--e.g., those held by the user.
Microsoft notes that the IR camera may three-dimensionally image a surface in an environment as noted in FIG. 1 above, by projecting IR or near-IR (NIR) light onto the surface and measuring the intensity of the projected light reflected by the surface. Such projection of light into the environment may enable the generation of two-dimensional (2D) IR images that comprise a plurality of IR pixels each specifying an intensity of IR light reflected by a surface imaged by that pixel. As such, the IR camera may be referred to as a 2D IR camera. The IR intensity of that pixel may be combined with predetermined knowledge of IR reflection phenomena to infer the depth of the surface imaged by that pixel. As described in further detail below, such predetermined knowledge may include a predetermined inverse square relationship between depth and IR intensity.
Microsoft's patent FIG. 3 noted above schematically shows an example IR image; FIG. 4 schematically shows a visual representation of an example depth map; FIG. 5 schematically shows a visual representation of an example skeletal hand model; and FIG. 6 shows an example IR camera operable to capture IR images of a scene.
Although it's not a sure thing that this feature will debut with Microsoft's Surface all-in-one PC desktop next week, we're clearly able to see and acknowledge by this patent filing and Microsoft's current trend of trying to get ahead of Apple with their own ideas, that they're bound to at least beat Apple to market with in-air gesturing in the not-too-distant future.
While I'd love to think that Apple will race to get their solution out to market so as to be ahead of Microsoft and break this current trend, Apple is on their own time table as always. If they so choose to proceed with this feature, then it will roll out when it's right for Apple and not a moment sooner. Well, that's according to the Apple Bible, and you don't mess with that.