From September 2022 until late December 2023, Patently Apple has posted eight patent reports covering a future iPhone with Face ID cameras under the display. More specifically, we covered four granted patents (01, 02, 03 and 04) and four patent applications (01, 02, 03 and 04) on this single topic.

Last week Apple was granted their fifth granted patent since Sept. 2022 for a camera under a display. The focus of this particular patent relates to the use of a black masking layer and more. The goal is to hide the MacBook and iPhone notches and present a full display as Samsung offers on their Galaxy Z Fold display as presented below.

In Apple's granted patent background they note that there's a trend towards borderless electronic devices with a full-face display. These devices, however, may still need to include sensors such as cameras, ambient light sensors, and proximity sensors to provide other device capabilities. Since the display now covers the entire front face of the electronic device, the sensors (cameras) will have to be placed under the display stack. In practice, however, the amount of light transmission through the display stack is very low (i.e., the transmission might be less than 20% in the visible spectrum), which severely limits the sensing performance under the display. It is within this context that the embodiments herein arise.

Black Masking Layer in Display having Transparent Openings

Apple's newly granted patent describes an electronic device, such as an iPhone, that may include a display and an optical sensor formed underneath the display.

The display may have both a full pixel density region and a low pixel density region or pixel removal region. The pixel removal region includes a plurality of high-transmittance areas that overlap the optical sensor. Each high-transmittance area may be devoid of thin-film transistors and other display components. The plurality of high-transmittance areas regions is configured to increase the transmittance of light through the display to the sensor. The high-transmittance areas may therefore be referred to as transparent windows in the display.

The display may include a black masking layer with a plurality of horizontal portions and a plurality of vertical portions that form a grid. The horizontal and vertical portions may be regularly spaced or irregularly spaced. The horizontal and vertical portions may be linear or zig-zag. The grid defines a plurality of apertures.

The black masking layer may additionally include patches in the apertures. The patches may reduce transmission through the pixel removal region. However, the patches may be optimally positioned to mitigate diffraction artifacts in the sensor that operates through the pixel removal region. The patches may have unique sizes and shapes.

The display may include first and second pixel removal regions that overlap respective first and second optical sensors. The first pixel removal region may have a black masking layer with a first layout whereas the second pixel removal region may have a black masking layer with a second layout that is different than the first layout. The layouts may cause complementary diffraction artifacts such that a single artifact-free image may be constructed using the first and second optical sensors.

Apple's patent FIG. 5 below is a top view of an illustrative display showing how high-transmittance areas may be incorporated into a pixel removal region #332 of the display. As shown, the display may include a plurality of pixels. In FIG. 5, there are a plurality of red pixels (R), a plurality of blue pixels (B), and a plurality of green pixels (G). The red, blue, and green pixels may be arranged in any desired pattern. Different nomenclature may be used to refer to the red, green, and blue pixels in the display. As one option, the red, blue, and green pixels may be referred to simply as pixels. As another option, the red, blue, and green pixels may instead be referred to as red, blue, and green sub-pixels. In this example, a group of sub-pixels of different colors may be referred to as pixel. In high-transmittance areas #324, no sub-pixels are included in the display (even though sub-pixels would normally be present if the normal sub-pixel pattern was followed).

To provide a uniform distribution of sub-pixels across the display surface, an intelligent pixel removal process may be implemented that systematically eliminates the closest sub-pixel of the same color (e.g., the nearest neighbor of the same color may be removed).

The pixel removal process may involve, for each color, selecting a given sub-pixel, identifying the closest or nearest neighboring sub-pixels of the same color (in terms of distance from the selected sub-pixel), and then eliminating/omitting those identified sub-pixels in the final pixel removal region.

With this type of arrangement, there may be high-transmittance areas in the pixel removal region, allowing a sensor or light-emitting component to operate through the display in the pixel removal region.

Additionally, because some of the pixels remain present in the pixel removal region (e.g., 50% of the pixels in the layout of FIG. 5), the pixel removal region may not have a perceptibly different appearance from the rest of the display for a viewer. 

Apple's patent FIG. 8B below is a top view of an illustrative black masking layer #54 that includes patches for diffraction mitigation;  FIG. 10 shows how different pixel removal regions may have different designs.

Apple's patent FIGS. 12A and 12B above are top views of black masking layers that may be used in pixel removal regions 332-1 and 332-2 of FIG. 10, respectively;  Black masking layer patches may be added to pixel removal regions that have different initial layouts. For example, FIGS. 13A and 13B are top views of black masking layers that may be used in pixel removal regions 332-1 and 332-2 of FIG. 10, respectively.

This is a deep and technical patent. To review its full details, review Apple's granted patent 11864452.