In April 2019 Apple was granted a patent for a scanning depth engine. Apple's patent covered their invention related to methods and devices for projection and capture of optical radiation, and particularly to optical 3D mapping. The mapping apparatus includes a transmitter, which is configured to emit a beam including pulses of light, and a scanner, which is configured to scan the beam, within a predefined scan range, over a scene. An image from that patent is presented below. The application shown was related to a depth scanner associated with a gaming console. Apple filed a continuation patent for this invention in late June 2019.

Today the US Patent & Trademark Office published a patent application from Apple titled "Calibration of a depth sensing array using color image data," that's a part of Apple's depth scanning device project that stems from Apple's engineers in Israel. The engineers were originally from PrimeSense, a company Apple acquired. Some of their technology helped to invent Face ID.  

Apple's latest invention relates to improved systems and methods for depth mapping, and particularly to sensor arrays used in time-of-flight sensing.

Existing and emerging consumer applications have created an increasing need for real-time three-dimensional (3D) imagers. These imaging devices, also known as depth sensors or depth mappers, enable the remote measurement of distance (and often intensity) to each point in a target scene--referred to as target scene depth--by illuminating the target scene with an optical beam and analyzing the reflected optical signal. Some systems capture a color image of the target scene, as well, and register the depth map with the color image.

A commonly-used technique to determine the distance to each point on the target scene involves transmitting one or more pulsed optical beams towards the target scene, followed by the measurement of the round-trip time, i.e. time-of-flight (ToF), taken by the optical beams as they travel from the source to the target scene and back to a detector array adjacent to the source.

Apple's invention covers imaging apparatus, including a radiation source, which is configured to emit a first plurality of pulsed beams of optical radiation toward a target scene. An array of a second plurality of sensing elements is configured to output signals indicative of respective times of incidence of photons on the sensing elements, wherein the second plurality exceeds the first plurality.

Objective optics are configured to form a first image of the target scene on the array of sensing elements.

An image sensor is configured to capture a second image of the target scene in registration with the first image.

Processing and control circuitry is coupled to receive the signals from the array and is configured to identify, responsively to the signals, areas of the array on which the pulses of optical radiation reflected from corresponding regions of the target scene are incident, and to process the signals from the sensing elements in the identified areas in order measure depth coordinates of the corresponding regions of the target scene based on the times of incidence, while identifying, responsively to the second image, one or more of the regions of the target scene as no-depth regions.

In some embodiments, the second image is a color image. In a disclosed embodiment, the processing and control circuitry is configured to identify the no-depth regions responsively to respective colors of corresponding parts of the color image. For example, the processing and control circuitry can be configured to identify a blue part of the color image as sky, and to mark the sky as a no-depth region.

Apple's patent FIG. 1 below is a schematic, pictorial illustration of a depth mapping system; FIG. 2 is a schematic side view of the depth mapping system of FIG. 1

This is a highly technical invention that engineers and developers will appreciate. To review the deeper details, see patent application 20210208262 here.

Considering that this is a patent application, the timing of such a product to market is unknown at this time.