Apple wins 3D camera patent for games, apps and facial recognition

The U.S. Patent and Trademark Office today officially announced Apple’s 62 newly granted patents. In this article, Xiao Bian will introduce you to Apple's acquisition of a patent from PrimeSense, a technology related to the 3D depth camera. The technology can control applications and games through gestures and may immediately be used for facial recognition features in Apple devices, such as iMac or other theoretical products.

Patent name: "Depth Mapping based on Pattern Matching and Stereoscopic Information"

Apple Patent No. 9,582,889 relates to the invention of computer vision, particularly three-dimensional mapping and imaging. The #20 in FIG. 1 is a system for three-dimensional mapping and imaging, wherein the imaging component #24 is configured to capture and process a three-dimensional perspective image of user interaction with a game and/or application in the computer.

Although Apple has described the camera's functionality, it has been explained and reported before.

For the new information on this authorized patent, I noticed that the entry in the reference section (usually used by patent examiners or patent applicants like Apple) cites "Automatic Face and Gesture Recognition". The company has previously used gesture recognition in Apple/PrimesSense’s patent applications several times, but “automatic facial recognition” does not. So this may be a clue.

When Apple acquired the Israeli company RealFace, people began to speculate that Apple will integrate facial recognition capabilities. KGI analyst Guo Mingchi previously claimed that the iPhone 8 will carry facial recognition capabilities, but he is based on Apple's patent and makes judgments, not manufacturing rumors.

In the end, the 3D camera described in this patent can be used for face recognition and integrated into the iMac to combat Wintel's "Windows Hello" instant face recognition login feature.

FIG. 1 is a schematic diagram of a three-dimensional mapping system; FIG. 2 is an imaging element/camera system; and FIG. 3 is a workflow diagram of three-dimensional mapping.

Technically speaking, stereo depth mapping or three-dimensional mapping cameras usually require two or more cameras spaced at a certain distance to capture images of the same scene. The two images will show the same image features, but due to the distance between the camera and the distance between the image features and the camera plane, the image features in different pictures will appear in the relative offset position. Thus, the measurement offset for a given image feature, and the known spacing between the cameras, can be used to calculate the depth coordinate of the image feature.

In general, multiple cameras used by a stereoscopic system have the same type and features. In the embodiment shown in FIG. 2, the two different types of cameras used include an IR camera embodied in subassembly #36 and an RGB color camera embodied in subassembly #42.

Of course, different cameras have different spectral responses and can also have different pixels. However, these cameras are always present in system #20 so that the computer can reconstruct the 3D color image of the object. Therefore, the two heterogeneous cameras used for stereoscopic depth mapping only generate little or no additional hardware cost, and the only requirement is to process the images output by the camera.

Apple's PrimeSense team of engineers used the same patent as FIG. 1 and covered a wide range of 3D mapping inventions, including head tracking and eye tracking, floating gestures, and gaze point control. The patent was originally filed in the third quarter of 2010 and is today issued by the U.S. Patent and Trademark Office.