Open Advanced Search
Get 20M+ Full-Text Papers For Less Than $1.50/day.
Start a 14-Day Trial for You or Your Team.
Learn More →
An Obstacle Detection and Distance Measurement Method for Sloped Roads Based on VIDAR
An Obstacle Detection and Distance Measurement Method for Sloped Roads Based on VIDAR
Jiang, Guoxin;Xu, Yi;Gong, Xiaotong;Gao, Shanshang;Sang, Xiaoqing;Zhu, Ruoyu;Wang, Liming;Wang, Yuqiong
Hindawi Journal of Robotics Volume 2022, Article ID 5264347, 18 pages https://doi.org/10.1155/2022/5264347 Research Article An Obstacle Detection and Distance Measurement Method for Sloped Roads Based on VIDAR Guoxin Jiang , Yi Xu , Xiaotong Gong , Shanshang Gao , Xiaoqing Sang , Ruoyu Zhu , Liming Wang , and Yuqiong Wang School of Transportation and Vehicle Engineering, Shandong University of Technology, Zibo 255000, China Correspondence should be addressed to Yi Xu; firstname.lastname@example.org Received 6 January 2022; Accepted 18 March 2022; Published 15 April 2022 Academic Editor: Arturo Buscarino Copyright © 2022 Guoxin Jiang et al. *is is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Environmental perception systems can provide information on the environment around a vehicle, which is key to active vehicle safety systems. However, these systems underperform in cases of sloped roads. Real-time obstacle detection using monocular vision is a challenging problem in this situation. In this study, an obstacle detection and distance measurement method for sloped roads based on Vision-IMU based detection and range method (VIDAR) is proposed. First, the road images are collected and processed. *en, the road distance and slope information provided by a digital map is input into the VIDAR to detect and eliminate false obstacles (i.e., those for which no height can be calculated). *e movement state of the obstacle is determined by tracking its lowest point. Finally, experimental analysis is carried out through simulation and real-vehicle experiments. *e results show that the proposed method has higher detection accuracy than YOLO v5s in a sloped road environment and is not susceptible to interference from false obstacles. *e most prominent contribution of this research work is to describe a sloped road obstacle detection method, which is capable of detecting all types of obstacles without prior knowledge to meet the needs of real-time and accurate detection of slope road obstacles. segmentation, and distance estimation, have become a key 1. Introduction component of autonomous vehicles. *ese systems can not With increasing public attention to the ﬁeld of traﬃc safety, only provide important traﬃc parameters for autonomous the automobile industry is developing in the direction of driving but also perceive surrounding obstacles, such as intelligence, with many studies on autonomous driving by stationary or moving objects, including roadblocks, pedes- engineers and scientiﬁc researchers. Autonomous driving trians, and other elements . During the vehicle’s move- does not refer to a single technological ﬁeld, but it is a ment, radar (laser, millimeter wave), infrared and vision product of the development and integration of automotive sensors are used to collect environmental information to determine whether a target is in a safe area [9–11]. However, electronics, intelligent control, and breakthroughs related to the Internet of *ings [1, 2]. *e principle is that autono- the price of infrared sensors and radars is relatively high, and mous driving systems obtain information on the vehicle and most of them are limited to advanced vehicles . Com- the surrounding environment through an environmental pared with other sensor systems, monocular vision requires perception system. *en, the information is analyzed and only one camera to capture images and analyze scenes, processed by the processor, and the obstacle information in thereby reducing the cost of detection solutions. Moreover, front of the vehicle is detected. Combining with the vehicle the camera can work at a high frame rate and provide rich dynamics model, the obstacle avoidance path planning and information from long distances under good lighting and lateral control of the vehicle are realized [3–7]. favorable weather conditions ; therefore, detection Environmental perception systems, which need to per- methods based on machine vision are being more and more form functions such as object classiﬁcation, detection, widely adopted. 2 Journal of Robotics Machine learning can be used to achieve object classi- proposed a novel image classiﬁcation framework that in- ﬁcation for vision-based obstacle detection [14, 15]. How- tegrates a convolutional neural network (CNN) and a kernel ever, traditional machine learning methods can only detect extreme learning machine to distinguish the categories of known types of obstacles (see Figure 1). If the vehicle cannot extracted features, thus improving the performance of image detect an unknown type of obstacles accurately, it is very classiﬁcation . Nguyen proposed an improved frame- likely that a traﬃc accident will occur. *is situation is not work based on fast response neural network (Fast R-CNN). conducive to the safe driving of the vehicle; therefore, in this *e basic convolution layer of Fast R-CNN was formed study, we propose an unsupervised learning-based obstacle using the MobileNet architecture, and the classiﬁer was detection method, which allows the detection of both formed using the deep separable convolution structure of the known- and unknown-type obstacles in complex MobileNet architecture, which improved the accuracy of environments. vehicle detection . Yi proposed the improved YOLO v3 Traditional obstacle detection methods, such as motion neural network model, which introduced the concept of compensation [16–18] and optical ﬂow methods [19–22], Faster R-CNN’s anchor box, and used a multiscale strategy, allow the detection of obstacles of diﬀerent shapes and at thus greatly improving the robustness of the network in various speeds. However, these methods require the ex- small object detection . Wang K.W. proposed an eﬃcient traction and matching of a large number of object points, fully convolutional neural network, which could predict the which increases the computational load. *erefore, in this occluded part of the road by analyzing foreground objects study, we adopt a Vision-IMU (inertial measurement unit)- and the existing road layout, thereby improving the per- based detection and ranging method, abbreviated as VIDAR, formance of the neural network . Although the above which can realize fast matching and feature point processing methods improved the accuracy of obstacle detection, they of the detection area and improve the obstacle detection require a large number of sample data for network training speed and detection eﬀectiveness. and the range of samples must cover all obstacle types; VIDAR is an obstacle detection method developed for otherwise, the obstacles cannot be detected. horizontal roads. When obstacles and test vehicles are lo- Monocular ranging pertains to the use of a single camera to capture images and perform distance calculations. Zhang cated on diﬀerent slopes, there will be imaging parallax, which will lead to the detection of false obstacles as real ones, et al. used a stereo camera system to compute a disparity resulting in a large measurement error, thereby aﬀecting the map and use it for obstacle detection. *ey applied diﬀerent detection accuracy. To cope with the impact of slope computer vision methods to ﬁlter the disparity map and changes, in this study, we take the slope of road into account remove noise in detected obstacles, and a monocular camera during the model establishment, and analyze the speciﬁc in combination with the histogram of oriented gradients and situation according to the position relationship between the support vector machine algorithms to detect pedestrians and detected vehicle and the obstacle. We thus propose an vehicles . Tkocz studied the ranging and positioning of a obstacle detection and distance measurement method for robot in motion, considering the scale ambiguity of mon- sloped roads based on VIDAR. In the proposed method, ocular cameras. However, only experimental research has slope and distance information are provided by digital maps been done on the speed and accuracy of measurement . [23–26]. Meng C designed a distance measurement system based on a *e rest of this study is structured as follows: in Section ﬁtting method, where a linear relationship between the pixel 2, we review the research on obstacle detection and visual value and the real distance is established according to the ranging. In Section 3, the conversion process from world pixel position of the vehicle in the imaging plane coordinate, coordinates to camera coordinates and the ranging principle thus realizing adaptive vehicle distance measurement under of VIDAR are introduced. In Section 4 the detection process monocular vision . Zhe proposed a method for detecting of real obstacles on sloped roads is outlined and the ranging vehicles ahead, which combined machine learning and prior and speed measurement models are established. Simulated knowledge to detect vehicles based on the horizontal edge of and real experiments are presented in Section 5 and the the candidate area . *ese methods were only used for experimental results are compared with the detection results the measurement of distance to other vehicles and are not of YOLO v5s to demonstrate the detection accuracy of the applicable to other types of obstacles. proposed method. In Section 6, the proposed method and Rosero proposed a method for sensor calibration and our ﬁndings are summarized, and the study is concluded. obstacle detection in an urban environment. *e data from a radar, 3D LIDAR, and stereo camera sensors were fused to- gether to detect obstacles and determine their shape . 2. Related Work Garnett used a radar to determine the approximate location of Obstacle detection still forms one of the most signiﬁcant obstacles, and then used bounding box regression to achieve research foci in the development of intelligent vehicles. With accurate positioning and identiﬁcation . Caltagirone pro- the improvement and optimization of monocular vision, posed a novel LIDAR-camera fusion fully convolutional net- obstacle detection based on monocular vision has attracted work and achieved the most advanced performance on the the attention of researchers. Most of the research on the KITTI road benchmark . Although sensor fusion methods detection of obstacles using monocular vision is based on the reduce the processing load and achieve improved detection accuracy, these methods are based on ﬂat roads and are not optimization of machine vision and digital image processing to improve the accuracy and speed of detection. S. Wang suitable for complex slope road environments. Journal of Robotics 3 Figure 1: Fast R-CNN. Normal cars are detected, but the overturned car and the box are not detected. To solve the above problems, we propose an obstacle equation from the world to the pixel coordinate system is detection and distance measurement method for sloped shown in roads based on VIDAR. *is method does not require a μ 1/d 0 μ f 0 0 X x 0 w priori knowledge of the scene and uses the road slope in- ⎡ ⎢ ⎤ ⎥ ⎡ ⎢ ⎤ ⎥⎡ ⎢ ⎤ ⎥⎡ ⎢ ⎡ ⎢ ⎤ ⎥ ⎤ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎢ ⎥ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎢ ⎥ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎢ ⎥ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎢ ⎥ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎢ ⎥ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎢ ⎥ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎢ ⎥ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎢ ⎥ ⎥ Z � ⎢ v ⎥ � ⎢ 0 1/d v ⎥⎢ 0 f 0 ⎥⎢R⎢ Y ⎥ + T⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎢ ⎥ ⎥ formation provided by a digital map and the vehicle driving C ⎢ ⎥ ⎢ y 0 ⎥⎢ ⎥⎢ ⎢ w ⎥ ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎢ ⎥ ⎥ ⎣ ⎦ ⎣ ⎦⎣ ⎦⎣ ⎣ ⎦ ⎦ state provided by an IMU to construct distance measure- 1 0 0 1 0 0 1 Z ment and speed measurement models, which allow the (1) a 0 μ X x 0 w detection of obstacles in real time, as well as the distance and ⎢ ⎥⎢ ⎥ ⎡ ⎢ ⎤ ⎥⎡ ⎢ ⎤ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥ movement state of the obstacles. ⎢ ⎥⎢ ⎥ � ⎢ 0 a v ⎥⎢ R Y ⎥ + T, ⎢ ⎥⎢ ⎥ ⎢ y 0 ⎥⎢ w ⎥ ⎢ ⎥⎢ ⎥ ⎣ ⎦⎣ ⎦ 0 0 1 Z 3. Methodology where R and T are the external parameters. *e internal and *e obstacle detection model of VIDAR is based on pinhole external parameters can be obtained through camera camera model, which can accurately calculate the distance calibration. between vehicles and obstacles. 3.2. Obstacle Ranging Method. *e obstacle ranging prin- 3.1. Coordinate Transformation. *e camera can map the ciple is also based on the pinhole model principle. For the coordinate points of the three-dimensional world to the two- convenience of expression, we installed the camera on a test dimensional imaging plane. *is imaging principle is con- vehicle and a vehicle on a sloped road was regarded as the obstacle. *e feature points of the obstacle were detected, sistent with the pinhole model principle, so camera imaging can be described by pinhole model. and the lowest point was taken as the intersection point If we want to determine the correspondence between the between the obstacle and the road surface (see Figure 3). In object point and the image point, we must establish the the case of normal detection by the system, the camera coordinate system needed by vision system, including world collects image information, and by processing the image coordinate system, camera coordinate system, imaging plane information, feature points in the image can be extracted. By coordinate system, and pixel coordinate system. *e measuring the distance of the feature point, it can be de- transformation process from the world coordinate system to termined whether the obstacle where the feature point is the pixel coordinate system is shown in Figure 2. located has a height. For real obstacles, tracking the feature point at the lowest position can calculate the moving speed Pixel coordinate (u, v) and image plane coordinate (x, y) are on the same plane, and the X and Y axes are of the obstacle, judge the motion state of the obstacle, and parallel. *e corresponding position of the original point provide data support for the safe driving of the vehicle. As in the image plane coordinate system is (u , v ). Both the long as the camera can capture images normally, all obstacles 0 0 world and the camera coordinate systems are 3D coor- in the captured scene can be detected. *e number of de- dinates, which are associated through the camera. tected obstacles is related to the number of extracted feature According to the principle of keyhole imaging, the camera points. coordinate system can be obtained through a transfor- Let f be the eﬀective focal length of the camera, z be the mation of the coordinate axes of the world coordinate pitch angle, μ be the pixel size, h be the mounting height of system, so the conversion relation between the two co- the camera and the camera center be the optical center of the ordinate systems must be deduced. *e conversion lens. Let (x , y ) be the coordinate origin of imaging plane 0 0 4 Journal of Robotics World coordinate Camera coordinate Imaging plane Pixel coordinate (Xw, Yw, Zw) (Xc, Yc, Zc) coordinate (x, y) (u, v) Figure 2: Transformation between coordinate systems. Focal Length (f) (x, y) Len’ s Center (x , y ) x 0 0 Horizontal Line Image Plane Optic Axis Bottom point (P) Feature point detection Road Plane Figure 3: Schematic diagram of obstacle ranging model (in order to visualize the detection principle, the nonreal proportional relationship is shown in the ﬁgure). coordinate system, and (x, y) be the intersection coordinate distance between the camera and C , and d be the hori- ii of the obstacle and the road plane in the image plane co- zontal distance between the camera and C . ordinate system. *e horizontal distance between the Using triangle similarity, equation (3) can be obtained camera and the obstacle can be obtained using through the geometric relationships shown in Figure 4: ⎧ 1 � tan α, d � . (2) ′ ⎪ d − S ⎪ ii tan ϑ + arctan