An Apple Patent Discovered in Europe today covers 'Range of View' and Data Compression Systems for Project Titan
Back in November 2017 Patently Apple posted a report titled "Apple Research Paper Touts VoxelNet as being Superior to LiDAR Regarding Autonomous Vehicle 3D Detection Methods." The report noted that Apple's Senior AI Researcher Yin Zhou and Machine Learning Research Scientist Oncel Tuzel had published their work on how self-driving cars could better detect pedestrians and cyclists while using fewer sensors. A few of the many images in provided in their paper included those presented below.
Earlier today Patently Apple discovered a patent application from Apple in Europe relating to the work Apple's scientists revealed last year about methods of detection by discussing new "Range field of view sensors," and diving deeper into image compression based on distance information.
The detailed computational systems that are required to have a vehicle operate on a semi or total autonomous basis is mind boggling.
Autonomous vehicles require advanced sensory system working intimately with video systems to understand what objects are coming in front of the vehicle in realtime in order to decide to ignore, apply the brakes cautiously or come to a full stop at a light or to stop for a pedestrian at a crosswalk or maneuver to avoid a cyclist that is dangerously coming toward the vehicle.
Today's patent is about just that and how the vehicle could store the hours and hours of data coming into the system that has to be stored.
Some human drivers can't always get it right but autonomous vehicles have to get it right 99% of the time in order for this to become a viable business for autonomous cabs, shuttle crafts, personal vehicles or even other craft-like boats and aircraft that people could trust. A number of major accidents involving Tesla vehicles have come to light, especially one that killed an Apple engineer.
In fact it was reported yesterday that Tesla is pulling back and not offering the full autonomous driving option until further notice. The Bloomberg report noted that "The electric-car maker has dropped from its online design studios the option to pay thousands of dollars more for what it called full self-driving, a higher-level feature for its Autopilot system." The report further noted that CEO Elon Musk had said Thursday that the feature '"was causing too much confusion.'"
Apple's Project Titan covers perhaps dozens if not hundreds of patents that we'll see over time. To date Patently Apple has been able to report on 19 such patents with today's raising it to 20.
At the moment we're unable to see the big Project Titan picture. All we're able to see are tiny glimpses of the many systems that Apple is working. They're likely filed in random order so as to keep their competition guessing.
The patent application presented today was published in Europe on Thursday October 18th and originally filed back in Q2 2018.
Apple begins their European patent filing by noting that images, such as still images or video images, can be utilized as inputs to control automated systems. Machine vision refers broadly to techniques and processes for identifying features in images
In some machine vision applications, image information is used to extract geometric features. Image information may be obtained and annotated to train machine learning models to recognize features in still images or video images.
In some applications, large amounts of image information are utilized for training purposes. If a mobile apparatus, such as a vehicle, is utilized to obtain image information, the amount of storage space available for image storage may be limited. Image compression techniques can be utilized to reduce the amount of storage space consumed by a still image or by a segment of a video.
Apple further notes that one aspect of the invention covers a method that includes determining compression parameters for image portions based on a distance value for each of the image portions such that compression rates applied by the compression parameters depend on the distance values, and encoding the image portions using the compression parameters.
A second aspect of Apple's invention covers a method that includes obtaining an image, obtaining range information, defining image portions from the image, correlating the range information to the image portions to define correlated range values that each describe a distance that is associated with a respective one of the image portions, and encoding the image to generate an encoded image.
Encoding the image is performed using compression parameters for at least some of the image portions to apply a higher compression rate to at least some of the image portions having correlated range values that correspond to shorter distances and to apply a lower compression rate for at least some of the some of the image portions having correlated range values that correspond to longer distances.
The third and last aspect of Apple's invention covers a method that includes obtaining an image, obtaining range information, defining image portions from the image, identifying a first group of image portions from the image portions and a second group of image portions from the image portions, correlating the range information to the image portions in the first group of image portions to define correlated range values that each describe a distance that is associated with a respective one of the image portions, determining range-based compression parameters for the image portions in the first group of image portions based on the correlated range values, and encoding the image portions using the range-based compression parameters for the first group of image portions to generate an encoded image.
Apple's patent FIG. 1 below is a block diagram showing a system for image compression based on distance information.
Apple's patent FIG. 3 presented below is an illustration that shows an example of a system #300 for image compression based on distance information that is implemented in a vehicle #301.
The system implements image compression based on distance information. The system includes an image sensor #302 and a range sensor #306.
The image sensor and the range sensor are supported by the vehicle and are oriented such that an image sensor field of view #303 of the image sensor and a range sensor field of view #307 of the range sensor at least partially overlap and are oriented toward an area of interest, such as a roadway #324 that is located ahead of the vehicle relative to an intended direction of travel of the vehicle.
Information from the image sensor and the range sensor are provided to a computing device #326 that is operable to process and store the information including the functions relate to a correlation module, a compression parameter module and an encoder.
Apple's patent FIGS. 2A and 2B are illustrations that show a first and second example of correlating image portions with range information.
Considering that this is a patent application, the timing of such a product to market is unknown at this time.
One of the inventors listed on Apple's patent is Mr. Basile, an advanced material scientist. His profile notes that he has investigated the use of novel solvents known as ionic liquids as electrolytes for both Lithium and Sodium battery technologies over 10 years. Ionic liquids show promise as electrolytes for their inherent properties that benefit energy storage applications.
Patently Apple presents a detailed summary of patent applications and/or granted patents with associated graphics for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for full and accurate details.