Today the US Patent & Trademark Office published a patent application from Apple that relates to eliminating the iPhone's notch and repositioning the TrueDepth camera system behind the iPhone's display. This is a future iPhone feature that most Apple fans can't wait to see unfold. A recent fan on Twitter wrote to me saying: "I've always dreamed of an iPhone with an edge - to - edge screen without the notch! I really hope that by 2027 this will happen!" Well, today an Apple patent goes to demonstrate that Apple's engineers are continuing to work on this feature that we all want to see come to the iPhone as soon as possible and hopefully well before 2027. 

Today, Apple's TrueDepth camera notch reduces the potential size of the display, as the camera occupies a portion of the front surface of an iPhone preventing the portion from being occupied by the display.

Apple's patent describes the iPhone including a display having a front surface and a back surface. The display includes a plurality of pixel regions that emit light from the front surface to display a displayed image and a plurality of apertures that transmit light from the front surface to the back surface. The iPhone includes a camera disposed on a side of the back surface of the display. The camera is configured to capture images.

The iPhone includes a processor coupled to the display and the camera. The processor is configured to receive the captured image and apply a first digital filter to a first portion of the captured image and a second digital filter, different than the first digital filter, to a second portion of the captured image to reduce image distortion caused by the display.

Apple's patent FIG. 2 below illustrates the current iPhone Notch design and FIG. 3 illustrates a future iPhone design with the TrueDepth camera system hidden behind the back of the display imagery.

Apple's patent FIG. 4 above illustrates a future operating environment #400 wherein the camera system is behind the display imagery. Apple notes that "although the camera #420 does not necessarily contact the display #410, the camera is closer to the back surface (#418) of the display than the front surface #417.

Apple's patent FIG. 12 below illustrates a functional block diagram of a device (#1200) including restoration optics #1220 configured to reduce image distortion caused by a display #1210.

Apple's patent FIG. 11 above is a flowchart representation of a method of restoring a captured image with a digital filter selected from a plurality of digital filters

While Apple has been granted this patent, today's patent extends their patent claims that either adds features and/or adds legal protections for this invention. While this doesn't guarantee the invention will come to market, it does illustrate that it's an invention that has potential of coming to market. Considering that this is a feature in high demand, Apple's new work on this is promising.

Below are the 20 new technical patent claims added to their original invention:

Claim #1: An apparatus comprising: a display having a front surface and a back surface, the display including a plurality of pixel regions that emit light from the front surface to display a displayed image and a plurality of apertures that transmit light from the front surface to the back surface; a camera disposed on a side of the back surface of the display, the camera configured to capture a captured image; and a processor coupled to the display and the camera, the processor configured to: receive the captured image; determine an imaging condition of the captured image; select one of a plurality of digital filters based on the imaging configuration; and apply the selected digital filter to the captured image to reduce image distortion caused by the display.

2. The apparatus of claim 1, wherein the processor is configured to determine the imaging condition by determining a noise level of captured image.

3. The apparatus of claim 2, wherein the processor is configured to select the one of the plurality of digital filters by: selecting a first digital filter with a first inversion strength based on determining a first noise level; and selecting a second digital filter with a second inversion strength greater than the first inversion strength based on determining a second noise level less than the first noise level.

4. The apparatus of claim 1, wherein the processor is configured to determine the imaging condition by determining a subject of the captured image.

5. The apparatus of claim 4, wherein the processor is configured to select the one of the plurality of digital filters by: selecting a first digital filter based on determining the subject of the captured image is a portrait; and selecting a second digital filter, different than the first filter, based on determining the subject of the captured image is not a portrait.

6. The apparatus of claim 1, wherein the processor is configured to determine the imaging condition by determining a purpose of the captured image.

7. The apparatus of claim 6, wherein the processor is configured to select the one of the plurality of digital filters by: selecting a first digital filter based on determining the purpose of the captured image is facial recognition; and selecting a second digital filter, different than the first filter, based on determining the purpose of the captured image is not facial recognition.

8. The apparatus of claim 1, wherein the processor is configured to determine the imaging condition based on at least one of an exposure time of the captured image, a light coherence of the captured image, or a camera identity of a camera of a plurality of cameras used to generate the captured image.
9. The apparatus of claim 1, wherein the processor is configured to select the one of the plurality of digital filters by selecting one of a plurality of digital filters stored in a non-transitory memory before determining the imaging condition.

10. The apparatus of claim 1, wherein the processor is configured to select the one of the plurality of digital filters by generating the digital filter based on a point spread function of the display and the imaging condition.

11. The apparatus of claim 1, wherein the processor is configured to determine the imaging condition by determining an illumination condition when the image was captured.

12. The apparatus of claim 11, wherein the processor is configured to select the one of the plurality of digital filters by: selecting a first digital filter based on determining the illumination condition is a near-infrared dot projection; and selecting a second digital filter, different than the first filter, based on determining the illumination condition is a near-infrared flood illumination.

13. A method comprising: receiving, from a camera disposed on a side of a back surface of a display that emits light from a front surface of the display, a captured image; determining an imaging condition of the captured image; selecting one of a plurality of digital filters based on the imaging condition; and applying the selected one of the plurality of digital filters to at least a portion of the captured image to reduce image distortion caused by the display.

14. The method of claim 12, wherein determining the imaging condition includes determining a noise level of captured image.

15. The method of claim 12, wherein determining the imaging condition includes determining a subject of the captured image.

16. The method of claim 12, wherein determining the imaging condition includes determining a purpose of the captured image.

17. The method of claim 12, wherein determining the imaging condition is based on at least one of an exposure time of the captured image, a light coherence of the captured image, or a camera identity of a camera of a plurality of cameras used to generate the captured image.

18. The method of claim 12, wherein selecting the one of the plurality of digital filters includes generating the digital filter based on a point spread function of the display and the imaging condition.

19. The method of claim 12, wherein determining the imaging condition includes determining an illumination condition when the image was captured.

20. A non-transitory computer-readable medium having instructions encoded thereof which, when executed by a processor of a device, causes the device to: receive, from a camera disposed on a side of a back surface of a display that emits light from a front surface of the display, a captured image; determine an imaging condition of the captured image; select one of a plurality of digital filters based on the imaging condition; and apply the selected one of the plurality of digital filters to at least a portion of the captured image to reduce image distortion caused by the display.

For more details, review Apple's patent application number 20230168711.