Open Advanced Search
Get 20M+ Full-Text Papers For Less Than $1.50/day.
Start a 14-Day Trial for You or Your Team.
Learn More →
An Improved VM Obstacle Identification Method for Reflection Road
An Improved VM Obstacle Identification Method for Reflection Road
Jiang, Guoxin;Xu, Yi;Sang, Xiaoqing;Gong, Xiaotong;Gao, Shanshang;Zhu, Ruoyu;Wang, Liming;Wang, Yuqiong
Hindawi Journal of Robotics Volume 2022, Article ID 3641930, 14 pages https://doi.org/10.1155/2022/3641930 Research Article An Improved VM Obstacle Identification Method for Reflection Road Guoxin Jiang , Yi Xu , Xiaoqing Sang , Xiaotong Gong , Shanshang Gao , Ruoyu Zhu , Liming Wang , and Yuqiong Wang School of Transportation and Vehicle Engineering, Shandong University of Technology, Zibo 255000, China Correspondence should be addressed to Yi Xu; email@example.com Received 23 January 2022; Accepted 18 February 2022; Published 14 March 2022 Academic Editor: L. Fortuna Copyright © 2022 Guoxin Jiang et al. *is is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. An obstacle detection method based on VM (VIDAR and machine learning joint detection model) is proposed to improve the monocular vision system’s identiﬁcation accuracy. When VIDAR (Vision-IMU-based detection and range method) detects unknown obstacles in a reﬂective environment, the reﬂections of the obstacles are identiﬁed as obstacles, reducing the accuracy of obstacle identiﬁcation. We proposed an obstacle detection method called improved VM to avoid this situation. *e experimental results demonstrated that the improved VM could identify and eliminate unknown obstacles. Compared with more advanced detection methods, the improved VM obstacle detection method is more accurate. It can detect unknown obstacles in reﬂection, reﬂective road environments. machine learning has an extremely high recognition rate for 1. Introduction speciﬁc images. While machine learning is capable of ac- Obstacle detection has become a major concern in the ﬁeld curate classiﬁcation, it can only be used to identify known of driver assistance systems due to the complexity of the obstacles. While the vehicle is in motion, using machine outdoor environment. Cameras (monocular, binocular, learning to identify unknown obstacles may result in mis- infrared, etc.), lidar, and millimeter-wave radar are all ex- identiﬁcation, posing a serious risk to the vehicle’s safety amples of obstacle identiﬁcation equipment. While lidar and (Figure 1). As a result, a method for detection and ranging millimeter-wave radar are highly accurate at detecting ob- using vision and an IMU (inertial measurement unit) has stacles, their high cost limits their use in low-end vehicles been proposed . Given that VIDAR requires more time to [1–3]. Due to the low cost, high detection accuracy, and run than machine learning, a method combining VIDAR speed of vision-based obstacle identiﬁcation equipment, it and machine learning to detect obstacles has been proposed has become more suitable for various vehicles [4, 5]. *e (called the VM method). Machine learning is used to identify known obstacles in the proposed method, while vision-based sensor used in this study is a camera. Camera, GPS, and IMU constitute an innovative sensor combination. VIDAR is used to detect unknown obstacles. Compared with single sensor, the application of multisensor To avoid the situation in which VIDAR detects the re- information fusion technology can improve the reliability of ﬂection as an obstacle when used in a reﬂective environment the whole system, enhance the reliability of data, improve (Figure 2) and improve detection accuracy, a VIDAR-based the accuracy, and increase the information utilization rate of pseudo-obstacle detection method (called improved the system in solving the problems of detection, tracking, VIDAR) has been proposed. *is method’s identiﬁcation and target recognition. procedure is as follows. *e rectangle of the obstacle was Machine learning is the process of training and iden- determined. *e width of the obstacle rectangle was cal- tifying images using deep convolutional neural networks. culated using the transformation relationship between pixel Compared with other image recognition technologies, coordinates and world coordinates, and then the height of 2 Journal of Robotics machine learning concepts such as SegNet, YOLO v5, faster the obstacle rectangle was calculated using the transfor- mation relationship between pixel coordinates and world RCNN, BigGAN, and mask RCNN are developed [13–20]. While machine learning is capable of accurate classiﬁcation, coordinates. *e true obstacle is determined by the fact that the actual height of the obstacle rectangle remains constant it can only be used to identify known obstacles. Unknown throughout the ego-vehicle movement. If the obstacle is a obstacles may be missed while the vehicle is moving, which real one, tracking is continued. will cause a serious impact on the vehicle’s safety. To accelerate the detection speed of improved VIDAR, Generally, obstacles are detected using signiﬁcant in- we combined it with machine learning (this article uses the formation such as color and prior shape. Zhu et al. proposed faster RCNN algorithm) to identify known obstacles, which a method for detecting vehicles based on their edges and we refer to as improved VM. *e improved VM obstacle symmetry characteristics . *ey hypothesized the ve- detection method can quickly and accurately detect obstacles hicle’s location based on the image’s detected symmetric regions. *e vehicle’s bounding box is determined using the on reﬂection roads. *e enhanced VM obstacle detection procedure is as follows: ﬁrst, machine learning is used to projected image of the enhanced vertical and horizontal edges. Zhang et al. [21–23] used color information for identify known obstacles; second, the identiﬁed obstacles are removed from the background area; and ﬁnally, pseudo- background removal and shadow detection to improve obstacles are eliminated through the use of enhanced object segmentation and background updating. *is method VIDAR. is capable of rapidly and precisely detecting moving objects. Zhang et al.  introduced Deep Local Shapes (DeepLS), which are high-quality 3D shapes that can be encoded and 2. Related Work reconstructed without requiring an excessive amount of As the core section of automobile-assisted driving, obstacle storage. *is local shape of the scene decomposition sim- detection has emerged as a critical area of research in recent pliﬁes the prior distribution that the network must learn and accelerates and accurately detects obstacles. However, in an years. Due to its simple ranging principle, the monocular vision sensor has become the primary obstacle identiﬁcation environment with reﬂections, the reﬂections contain sig- niﬁcant information about the obstacles they use, reducing equipment in obstacle identiﬁcation. Many scholars have conducted related research on obstacle identiﬁcation to the accuracy of obstacle detection. *e vehicle’s position is generally determined by high- accelerate the process. Traditional image object classiﬁcation and detection algorithms and strategies are diﬃcult to meet light information, such as the highlighted area and contour the requirements of image and video big data in terms of features. Park and Song  proposed a front vehicle processing eﬃciency, performance, and intelligence. Deep identiﬁcation algorithm based on contrast enhancement and learning establishes the mapping from low-level signals to vehicle lamp pairing. Lin et al.  discovered that the high-level semantics by simulating the hierarchical structure characteristics of headlights were more distinctive than the similar to the human brain, so as to realize the hierarchical contours of vehicles and had a greater identiﬁcation eﬀect and thus proposed the use of lamps as a sign for vehicle feature expression of data, and has powerful visual infor- mation processing capabilities. *erefore, in the ﬁeld of identiﬁcation at night. *e Hough transform was proposed by Dai et al.  as a method for intelligent vehicle iden- machine vision, the representative of deep learning-con- volutional neural network (CNN) is widely used [7, 8]. tiﬁcation at night. *is method divides the extracted lamps Convolutional neural networks are also called cellular into connected domains, extracts the lamps’ edges, and then nonlinear networks. Arena et al. have stressed the universal identiﬁes the circle using the Hough transform. Finally, by role that cellular nonlinear networks (CNNs) are assuming pairing the lamps, the vehicle’s location is determined. today. It is shown that the dynamical behavior of 3D CNN- Kavya et al.  proposed a method for detecting vehicles based models allows us to approach new emerging problems, based on the color of the brake lamp during braking in the to open new research frontiers . Shustanov and Yakimov captured color image. *e feature information required for proposed an implementation of the traﬃc signs recognition the above identiﬁcation method in a reﬂective environment will detect lamp reﬂections. *e vehicle’s lamps will also be algorithm using a convolution neural network; training of the neural network is implemented using the TensorFlow paired, which will reduce the vehicle’s accuracy of identi- ﬁcation. We used a modiﬁed VM to detect obstacles, library and massively parallel architecture for multithreaded programming CUDA; and the experiment proves the high allowing for eliminating obstacles in the reﬂection, thereby eﬃciency of this method . Zhu et al. have proposed a increasing obstacle detection accuracy. novel image classiﬁcation framework that combines CNN and KELM (kernel extreme learning machines). *ey 3. Methodology of Improved VIDAR’s Pseudo- extracted feature categories using DenseNet as a feature Obstacle Detection extractor and radial basis function kernel ELM as a classiﬁer to improve image classiﬁcation performance . Wang *e monocular visual identiﬁcation method, based on et al. proposed the occlusion-free road segmentation net- machine learning, is limited to identifying previously work, a fully convolutional neural network. *rough fore- identiﬁed obstacles. A vehicle collision accident may occur ground objects and visible road layouts, this method can when unknown obstacles are present on the road. When predict roads in the semantic domain . *e accuracy of VIDAR is used to detect obstacles, pseudo-obstacles in the obstacle identiﬁcation is also continuously improved as new reﬂection environment are mistaken for real obstacles. *us, Journal of Robotics 3 Figure 1: *e results of obstacle identiﬁcation based on faster RCNN. Car Car Car Car Car Car Car Car Car Car Figure 2: *e results of car detection based on VIDAR on reﬂection roads. to increase the speed and accuracy of obstacle detection, we homogeneous coordinates. Combine the rotation matrix R use an improved VM. and the oﬀset matrix T to obtain the external parameter matrix K, where c and c are the oﬀsets. *e coordinate x y transformation is shown in Figure 3. 3.1. Transformation from World Coordinates to Pixel *e internal parameters of the camera are obtained by Coordinates. *e camera can project objects in the three- Zhang Zhengyou demarcate to determine the transforma- dimensional world into a two-dimensional image by cap- tion relationship between world coordinates and pixel turing an image. In reality, the imaging model establishes a coordinates. projection mapping relationship between three-dimensional q � MKQ. (1) and two-dimensional space. *e coordinate transformation x f 0c is required to convert the world coordinate system’s coor- x x ⎢ ⎥ ⎢ ⎥ ⎡ ⎢ ⎤ ⎥ ⎡ ⎢ ⎤ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ Among them: q � y , M � 0f c , ⎣ ⎦ ⎣ ⎦ dinates to the camera coordinate system’s coordinates. A y y w 001 rigid body transformation is used to convert the world X x f 0c coordinate system to the camera coordinate system. It is x x R t R t ⎡ ⎢ ⎤ ⎥ ⎡ ⎢ ⎤ ⎥ ⎡ ⎢ ⎤ ⎥ 3×3 3×1 ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ 3×3 3×1 ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ K � , and Q � ⎣ Y ⎦, ⎣ y ⎦ � ⎣ 0f c ⎦ determined by the camera’s external parameters. *e camera y y 0 1 01 W w 001 coordinate system to pixel coordinate system transformation converts three-dimensional coordinates to two-dimensional ⎡ ⎢ ⎤ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ Y ⎥ plane coordinates, as determined by the camera’s internal ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥. ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎣ ⎦ parameters. Although both the pixel and image coordinate Z systems are located on the imaging plane, their origins and units of measurement are distinct. *e origin of the image 3.2. Obstacle Ranging Model. *e range of obstacles is as coordinate system is the point at which the camera’s optical follows (Figure 4). Let f be the focal length of the camera; h axis intersects the imaging plane, which is typically the be the installation height of the camera; μ be the pixel size; z imaging plane’s midpoint. be the camera pitch angle; (x , y ) be the intersection of the 0 0 Suppose the internal parameters matrix is M. Project image plane and the optical axis of the camera; set to (0, 0); Q(X, Y, Z) in the physical world to the image plane q(x, y). (x, y) be the coordinates of intersection points of obstacles By adding a dimension w to q(x, y), which is the expansion and pavement plane set to P; and the horizontal distance d of q(x, y, w), obtain w � Z. Point q is in the form of between the object point and the camera is 4 Journal of Robotics Imaging coordinate have height if d ≠ d + Δd, and static obstacles can be 1 2 system identiﬁed by Δl if Δd is known. Additionally, if the obstacle is moving (as illustrated in Figure 6), the Δl can also be used as an obstacle judgment. (x, y) *e veriﬁcation process has been shown in the paper . Pixel Coordinate system 3.3. Static Obstacle Identiﬁcation Model. *ere are two types Yw of static obstacles: real static obstacles and static pseudo- (u,v) Zw obstacles. Static real obstacles refer to actual road obstacles. Camera (X,Y,Z) *e reﬂections identiﬁed as real obstacles during the obstacle coordinate system Xw identiﬁcation process are called static pseudo-obstacles. It is World coordinate Yc a type of pseudo-obstacle that reﬂects some road obstacles system but does not aﬀect the vehicle’s driving safety. To improve (c ,c ) x y the accuracy of obstacle identiﬁcation, we must identify and Xc remove static pseudo-obstacles. Figure 3: Schematic diagram of coordinate transformation. 3.3.1. Static Real Obstacle Identiﬁcation. First, we used the VIDAR to detect stereo obstacles and determine which d � . (2) object point on the obstacle is the furthest away in the tan z + arctan