An Apple Home Security camera system patent reveals methods of detecting packages delivered to and/or stolen from outside your Home
Last Saturday, we posted a report titled "Apple has invented a sophisticated Home Surveillance System with face & body recognition streamed to an in-home HomePod+." Yesterday, Patently Apple discovered an Apple patent application filed in Europe that was about yet another sophisticated security camera system that is designed to monitor package deliveries and or theft of packages delivered to a home or business.
Although the patents are a little odd for Apple, we did cover a patent about bringing a camera system to HomePod in April 2020 titled "Apple Invents HomePod with Built-in Camera that Provides Siri with Eyes to deliver Next-Gen Commands & Longer-Range Face ID. The image below is from that patent that clearly illustrates a HomePod.
In Apple's latest patent background, they note that as digital camera technology has become cheaper and more prevalent, it has become commonplace for cameras to be positioned outside of structures (such as residential buildings, commercial buildings, manufacturing buildings, homes etc.) to monitor activities occurring outside the structures, or in an unsecured portion of the structures to monitor activities occurring within the unsecured portion of the structures. For example, cameras have been implemented in doorbells and security systems to monitor for activities occurring outside the structure. Additionally, the increase in online sales of goods has resulted in more packages being delivered to structures. The delivery of packages to the structures has led to challenges, including theft of the packages.
Apple's patent application covers the use of cameras to determine when packages have been delivered to or picked up from structures (homes, businesses).
More specifically, the patent covers systems, computer-readable media, methods, and approaches that may identify delivery and/or pickup of packages. Further, a notification may be provided indicating that the package has been delivered and/or picked up. The notification may allow for a user to be aware of the delivery and/or pickup of the package, which may cause the user to retrieve the package from the area.
In some instances, the notification may prevent issues presented by delivery of packages, such as theft of the packages, accidental incorrect deliveries, and/or lack of delivery of the packages.
Further, a system may have one or more cameras, or may be coupled to one or more cameras, that are directed to capture video and/or images of areas where packages may be delivered or placed for pickup, such as outside of a structure, within an unsecured portion of a structure, and/or within another portion of a structure where packages may be intended to be placed for delivery and/or pickup. The cameras may be directed to areas where packages are expected to be delivered by a delivery person or placed for pickup by a delivery person.
The system may identify delivery and/or pickup of packages within the video or images captured by the cameras. Based on the system identifying delivery and/or pickup of the packages, the system may determine notification settings associated with the cameras and provide notification of the delivery and/or pickup of the packages.
In some embodiments, a method may include detecting motion within video received from a camera, capturing, in response to detecting the motion, a representative image from the video, and retrieving a canonical image that represents an average of one or more frames received from the camera before detecting the motion.
Apple's patent FIG.1 below illustrates an example system arrangement #100. In particular, the system arrangement illustrates one example of a system #102 that may utilize one or more cameras to identify delivery and/or pickup of packages. The approaches implemented by the system can accurately and efficiently identify the delivery and/or pickup of packages. Further, the system may indicate to a user when a package has been delivered and/or picked up, which may address some of the issues of packages being placed outside of structures and/or in unsecured portions of structures, such as theft of packages.
Apple's patent FIG.3 above illustrates an example of a canonical image #300 which may be an image that was captured prior to the detection of the motion. The image does capture a delivery truck nearby. In FIG. 4 we see that the camera has captured a package being left near the home or business and that the delivery truck is no longer in the picture. This confirms delivery of a package.
Apple's patent FIG.16 illustrates an example procedure for identifying objects and/or package(s).
For more details, review Apple's patent application number WO2022245751 titled "Techniques for Detection/Notification of Package Delivery and Pickup." Apple purposely filed one patent in the U.S. while filing a related patent in Europe to make it difficult to discover different parts of this project.
Some of the inventors in the U.S. Patent that we referred to at the top of this report were also present in the European filing. Those common inventors are presented below in bold black text for your convenience.
Mey Khalili: A Machine Learning Engineer at Apple. Mey previously worked at "Lighthouse AI" prior to Apple acquiring the company and its 3D Depth Home Camera System in 2019. Mey's LinkedIn profile shows that she actually joined Apple in December 2018 that may confirm when the acquisition actually took place. In August 2020, Apple won a patent for an Indoor Security Camera System that they acquired from Lighthouse AI. This goes a long way in explaining the origins of Apple's latest patent applications about security camera systems. Though make no mistakes about it, the other 3 out of four engineers on the patent are from Apple. So, these patents indicate that they're part of a larger camera project Apple is working on.
Hendrik Dahlkamp: Machine Learning Manager, HomeKit Secure Video
Jonghoon J.: Deep learning for computer vision
Michael Bebenita: Building Things [strange entry. Came to Apple from 11 years as the Principal Research Engineer at Mozilla Corporation.]