On Thursday the U.S. Patent and Trademark Office published a patent application from Apple that revealed manufacturing systems using a plenoptic camera. More particularly, Apple's invention relates to enhancing manufacturing operations by using machine vision incorporating one or more plenoptic cameras.
Apple's Patent Background
Advances in manufacturing have provided a variety of techniques for reproducing high quality consumer goods. Many of these techniques incorporate machine vision allowing certain robotic devices to perform advanced operations based on images captured by cameras. However, despite the potential utility of such operations, many camera and vision techniques are subject to various discrepancies. Often times multiple cameras are required in order for a robot to receive both an accurate image of a part and be able to perform a quality operation. Moreover, manufacturing lines can often be forced to stop during image capture, thus slowing down the manufacturing operation. During a pause in manufacturing, multiple cameras may need to capture multiple views of a part, as well as perform post-processing on each image, further delaying the robotic operations to be performed on a part.
Apple's invention generally relates to the use of plenoptic cameras for improving various manufacturing processes. The new system includes a method for identifying a proximity of a manufactured part using a plenoptic camera during a manufacturing process. The method includes a step of generating a plurality of image slices derived from a light field array captured by a plenoptic camera. The light field array is based on a collimated array of light reflected from the manufactured part. The method can further include a step of comparing one or more image slices of the plurality of image slices to a focal distance between the manufactured part and a light source in order to determine the proximity of the manufactured part for use during the manufacturing process.
Additionally, the embodiments include a manufacturing system for generating a composite image of a moving part using a light field array. The manufacturing system can include a plenoptic camera configured to capture a light field array of a moving part. The manufacturing system can further include an image processing unit communicatively coupled to the plenoptic camera and configured to receive data corresponding to the light field array in order to derive the composite image of the moving part based on a plurality of image slices of the light field array.
In yet other embodiments, a method for operating a robotic device of a manufacturing system according to image data captured by a plenoptic camera is disclosed. The method can include extracting, from a light field array captured by the plenoptic camera, geometric data corresponding to dimensions of a manufactured part. Additionally, the method can include converting the geometric data into instructions for the robotic device that are capable of causing the robotic device to perform an operation based on the dimensions of the manufactured part.
Apple's patent FIG. 1 illustrates a manufacturing system for generating a two-dimensional and/or a three dimensional image of a manufactured part during a manufacturing process; FIG. 2 illustrates a snapshot view of the light source emitting, and plenoptic camera receiving, collimated light incident upon the manufactured part and the conveyor belt; FIG. 3 illustrates a perspective representation of a light field array of a plurality of image slices that can be captured by the plenoptic camera.
Apple's patent FIG. 5 illustrates examples of the robotic operations that can be based on measurements derived from a light field array captured by a plenoptic camera in the manufacturing system.
Apple's patent FIG. 9 noted below illustrates a manufacturing system configured to use a plenoptic camera to concurrently or simultaneously detect the distance of a manufactured part and perform a surface analysis of a manufactured part.
Apple notes that the plenoptic camera is used to capture a multi-dimensional image of a manufactured part at a point in time in the manufacturing process. The multi-dimensional image or light field array can include numerous image slices that can be analyzed to determine dimensions of the manufactured part. A collimated laser can be used to provide an array of laser points on the manufactured part when the light field array is captured. The various laser points can be incident upon the manufactured part at different regions having different heights relative to the plenoptic camera. Additionally, if the manufactured part has apertures, bends, blemishes, or non-uniform features, the reflected laser points will be modified according to the non-uniform features. Using data regarding the focal point of the laser and depth of field for the plenoptic camera, data can be generated regarding the orientation, surface geometry, and quality of the manufactured part, among other properties of the manufactured part. For example, when a pick and place operation requires the dimensions and orientation of a manufactured part before a robot can grasp the part, a plenoptic camera and a collimated laser can be used to capture a light field array based on the manufactured part. The light field array of the manufactured part can thereafter be processed to determine the orientation and dimensions of a manufactured part relative to a conveyor belt or other surface on which the manufactured part is moving or placed in the manufacturing process. The orientation and area can be converted into robotic instructions for guiding the robot to reach the appropriate area of the conveyor belt where the manufactured part resides, and grasp the manufactured part according to the dimensions calculated from the light field array.
To learn more about this system, see Apple's patent application 20160063691 which was originally filed in September 2014. Whether this system is now being used in the manufacturing of any Apple product is unknown at this time.
Patently Apple presents a detailed summary of patent applications with associated graphics for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for full and accurate details. About Making Comments on our Site: Patently Apple reserves the right to post, dismiss or edit any comments. Comments are reviewed daily from 5am to 6pm MST and sporadically over the weekend.