Today the US Patent & Trademark Office published a patent application from Apple that relates to object detection and more particularly to long wavelength infrared foveated vision for resolving objects with diminished visibility in a wide field of view for a vehicle.

Apple states that objects along a travel path of a vehicle, particularly moving objects, such as animals, that intersect the travel path of the vehicle, are challenging to avoid.

Autonomous or semi-autonomous vehicles may include various sensor systems for object detection for driver assistance in avoiding such objects. However, conventional sensor systems often fail in adverse light conditions, including nighttime, low visibility weather (e.g., fog, snow, rain, etc.), glare, and/or the like that obscure or diminish the visibility of such objects.

For example, monochromatic sensors generally require active illumination to detect objects in low light conditions and are prone to saturation during glare. As such, objects remain hidden from detection by monochromatic sensors in low light conditions and in the presence of glare, for example, due to external light sources, such as the headlights of other vehicles.

Other conventional sensor systems eliminate the need for active illumination by using passive sensors, such as long wavelength infrared sensors. However, such sensor systems typically fail to identify objects in adverse light conditions due to low resolution. Many other conventional sensor systems are cost, weight, and/or size prohibitive for deployment into a vehicle for object detection.

Accordingly, objects remain hidden from conventional sensor systems in adverse light conditions, thereby exacerbating the challenge of avoiding such objects. It is with these observations in mind, among others, that various aspects of the present invention were conceived and developed.

Detecting Hidden Objects in a Field of View

Apple's invention provides autonomy for a vehicle in adverse light conditions, such as nighttime, low visibility weather (e.g., fog, snow, rain, etc.), low light conditions, glare, and/or the like that obscure or diminish the visibility of objects. For example, disparate nighttime environments have differing degrees of ambient light, which impacts a sensitivity of a sensor suite of the vehicle used to detect objects.

A city environment typically has abundant ambient light from street lamps, adjacent buildings, city congestions, and the like. Meanwhile, a rural environment has limited ambient light that originates primarily from starlight, moonlight, and airglow. In between these environments, a suburban environment has ambient light from street lamps, housing, and vehicular traffic.

Objects may be hidden from detection in the field of view for a vehicle during such adverse light conditions. For example, a mammal, such as a deer, may be not be visible at a side of the street in the dark and dart across the street as the vehicle approaches. Due to the thermal signature of such objects, long wavelength infrared (LWIR) vision permits objects to be detected at various distances from the vehicle in adverse light conditions.

However, LWIR typically suffers from a narrow field of view and poor resolution, such that objects may remain hidden from detection depending on where they are located relative to the vehicle. Thus, the presently disclosed technology concentrates resolution of LWIR vision at designated regions in the field of view to detect and identify objects that are otherwise hidden from detection.

By using such LWIR foveated vision, thermal energy for objects may be detected at higher resolution in a designated region of a wide field of view in which hidden objects may be located.

Additionally, an extended depth of field may be created to obtain additional detail about the hidden objects in the designated region using multiple LWIR images through stereo vision.

The distance to the object is determined by extending a range of distance over which the object remains in focus. Finally, the LWIR foveated vision may be used in combination with other imaging and/or detection systems, including monochromatic sensors, red/green/blue (RGB) sensors, light detection and ranging (LIDAR) sensors, and/or the like for enhanced object detection.

Apple's patent FIG. 8 below depicts an example front longitudinal far field of view for long wavelength infrared foveated vision; the mid-figure is patent FIG. 14 which illustrates example operations for object detection; and FIG. 5 shows us an example sensor suite maximizing a field of view while maintaining spatial resolution.

The image in patent FIG. 5  above relates to a sensor suit as seen in a modern vehicle system illustrated below that gives a better sense of what it could appear like. 

Apple's patent application number 20200342623 that was published today by the U.S. Patent Office was filed back Q2 2020 with some work on this project done one year earlier under a provisional patent. Considering that this is a patent application, the timing of such a product to market is unknown at this time.