“Even people who are no longer interested in driving, with the upgrade of automobile electronics in recent years, can clearly feel the huge upgrade of the driving experience. Drivers who may have needed family members to help command when they back up and enter the garage, cooperate with the outside of the car. The advanced cameras, sensors, and automatic parking technology can also easily enter the seat, and the sensing device outside the car greatly improves the driving experience and safety.
✦ This article is reproduced from: Feixiang.com
✦ Author: Wei Deling
Even people who are no longer interested in driving, with the upgrade of automobile electronics in recent years, can clearly feel the huge upgrade of the driving experience. Drivers who may have needed family members to help command when they back up and enter the garage, cooperate with the outside of the car. The advanced cameras, sensors, and automatic parking technology can also easily enter the seat, and the sensing device outside the car greatly improves the driving experience and safety.
However, there is still a huge room for improvement in the driving experience of the in-car world. Drivers unconsciously reply to messages with their mobile phones, or operate on the touch panel during high-speed driving, causing back pain after long-term driving. Pain and blurred eyes, and children in the back row who can be naughty and free from the safety seat, the potential safety hazards may be more sudden and risky than small scratches outside the car.
The in-car sensing market is about to grow rapidly
According to Yole’s report, the global vehicle camera sales volume is expected to be 172 million in 2021, and the sales volume will reach 364 million by 2026. Among these 5 times growth, the fastest growth rate will be the internal camera, CAGR reached 22.4%.
In the future, a car will not only have cameras and sensors outside the car, but also greatly increase the number of cameras inside the car. The vehicle evaluation standard system Euro NCAP specifically states that an effective driver monitoring system is also a prerequisite for autonomous driving to ensure the necessary Return control to a driver fit and able to drive.
This also means that the vehicle needs to have the ability to determine whether the driver can currently drive. The Driver Monitoring System (DMS) is a system that focuses on observing the driver. It can mainly monitor whether the driver is paying attention, whether he is distracted or dozing off, because these phenomena can seriously endanger driving safety.
Cabin/Occupancy Monitoring System (CMS/OMS) is mainly used to observe passenger conditions. For example, it can remind users to avoid leaving children in the car after getting out of the car. Died from being trapped in a car. In addition, the system also has the functions of detecting passenger identification, airbag adaptation, and intruder detection.
For various applications of in-vehicle perception, it is undoubtedly closely related to the popular computer vision.
Global Shutter Powers Computer Vision
“What is computer vision? In fact, most people use this technology every day.” Lin Guozhi, technical marketing manager of STMicroelectronics Asia Pacific Imaging Division, vividly introduced the current wide range of applications of this technology at a recent media conference. The current situation, whether it is the facial recognition of smartphones or laptops, or the code scanning guns used in supermarket convenience stores, all use computer vision.
At present, the global shutter image sensor is an important technical element for realizing computer vision.
As Lin Guozhi said, this technology is not unfamiliar to many people. The face recognition unlocking function on many laptops is realized through the global shutter technology, and it can also realize functions such as gesture control and privacy protection. , and then cooperate with ToF and ALS for presence detection, battery life and automatic adjustment of screen brightness.
In addition, the global shutter image sensor also plays an important role in a large number of application scenarios such as eye tracking and gesture control on AR/VR/MR, object detection and scene analysis in industrial control.
The reason why the global shutter is more in line with the requirements of computer vision than the rolling shutter is that the rolling shutter technology’s progressive image capture principle is limited by the fact that the entire image has only one frame. For fast-moving objects, the image is likely to be distorted. , the subject cannot be displayed as it is. The global shutter captures the entire image at one time, with a very short shooting time and very accurate imaging results.
For example, STMicroelectronics’ VD55G0 and VD56G3 global shutter sensors provide excellent technical guarantees for engineers to develop related computer vision functions in the field of industrial automation and consumer products. They have the highest quantum efficiency at 940nm and bring extremely high NIR. sensitivity. The resolution of the square sensor is optimally matched with the lens, which greatly shortens the sensor shooting time, enables accurate imaging and reduces system power consumption.
Realize the upgrade of in-vehicle experience
For the automotive field, RGB color images are required for cabin occupancy monitoring, and NIR images are required for driver monitoring. Currently, only a single camera can support both RGB and NIR.
Two generations of ST’s global shutter sensors are now available for in-vehicle surveillance needs.
The first generation of products has achieved the coverage of all customer needs, including four types of products: VD5661A with 1.6 million resolution, mainly used for DMS. The VD5761A has a resolution of 2.3 million and can also be used for DMS because it belongs to Mono (monochrome). And VD6763A with color RGB and VD1762A with RGB-NIR integration.
According to Lin Guozhi, technical marketing manager of STMicroelectronics Asia Pacific Imaging Division, the first-generation ST global shutter sensor has three characteristics. The first is high contrast, which can output extremely high-contrast images; the second is data management of RGB and NIR; and the third is high dynamic, even in too bright or too dark ambient conditions, HDR sensors can obtain clear images.
The second-generation product VB56G4A has a resolution of 1.5 million and is mainly used for driver monitoring. It has three characteristics of high sensitivity, small sensor area and built-in processing. It is convenient for manufacturers to install inside the mechanical device by using a smaller camera module. In addition, the intelligent automatic exposure algorithm can automate the processing and operation process without manual operation.
In other words, with the help of ST’s global shutter sensor, the functions of the driver monitoring system (DMS) and the cabin/occupancy monitoring system (CMS/OMS) can be realized for the interior of the car.
In addition, OEMs can also choose finished camera modules based on ST’s global shutter sensors, which can be directly integrated into automotive systems or sensor performance evaluation.
In addition to enabling the interior of the car to realize monitoring functions, ST’s global shutter sensor also has the possibility to realize gesture control. Manufacturers can also develop innovations based on this sensor, allowing drivers to directly use gestures. Quick control of some vehicle functions to avoid distracted operation of the touch screen, further improving safety during driving.
At the same time, STMicroelectronics also provides products such as ToF and ambient light sensors to meet the innovative needs of manufacturers to integrate into various scenarios through different products.
In the past, reversing radar was something that many old drivers looked down on. Now, all-round radar has become the standard configuration of a large number of cars, and all drivers also enjoy it. What is more worth looking forward to is that the new experience in the car is also beckoning to people.