Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

An Improved VM Obstacle Identification Method for Reflection Road

An Improved VM Obstacle Identification Method for Reflection Road Hindawi Journal of Robotics Volume 2022, Article ID 3641930, 14 pages https://doi.org/10.1155/2022/3641930 Research Article An Improved VM Obstacle Identification Method for Reflection Road Guoxin Jiang , Yi Xu , Xiaoqing Sang , Xiaotong Gong , Shanshang Gao , Ruoyu Zhu , Liming Wang , and Yuqiong Wang School of Transportation and Vehicle Engineering, Shandong University of Technology, Zibo 255000, China Correspondence should be addressed to Yi Xu; xuyisdut@163.com Received 23 January 2022; Accepted 18 February 2022; Published 14 March 2022 Academic Editor: L. Fortuna Copyright © 2022 Guoxin Jiang et al. *is is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. An obstacle detection method based on VM (VIDAR and machine learning joint detection model) is proposed to improve the monocular vision system’s identification accuracy. When VIDAR (Vision-IMU-based detection and range method) detects unknown obstacles in a reflective environment, the reflections of the obstacles are identified as obstacles, reducing the accuracy of obstacle identification. We proposed an obstacle detection method called improved VM to avoid this situation. *e experimental results demonstrated that the improved VM could identify and eliminate unknown obstacles. Compared with more advanced detection methods, the improved VM obstacle detection method is more accurate. It can detect unknown obstacles in reflection, reflective road environments. machine learning has an extremely high recognition rate for 1. Introduction specific images. While machine learning is capable of ac- Obstacle detection has become a major concern in the field curate classification, it can only be used to identify known of driver assistance systems due to the complexity of the obstacles. While the vehicle is in motion, using machine outdoor environment. Cameras (monocular, binocular, learning to identify unknown obstacles may result in mis- infrared, etc.), lidar, and millimeter-wave radar are all ex- identification, posing a serious risk to the vehicle’s safety amples of obstacle identification equipment. While lidar and (Figure 1). As a result, a method for detection and ranging millimeter-wave radar are highly accurate at detecting ob- using vision and an IMU (inertial measurement unit) has stacles, their high cost limits their use in low-end vehicles been proposed [6]. Given that VIDAR requires more time to [1–3]. Due to the low cost, high detection accuracy, and run than machine learning, a method combining VIDAR speed of vision-based obstacle identification equipment, it and machine learning to detect obstacles has been proposed has become more suitable for various vehicles [4, 5]. *e (called the VM method). Machine learning is used to identify known obstacles in the proposed method, while vision-based sensor used in this study is a camera. Camera, GPS, and IMU constitute an innovative sensor combination. VIDAR is used to detect unknown obstacles. Compared with single sensor, the application of multisensor To avoid the situation in which VIDAR detects the re- information fusion technology can improve the reliability of flection as an obstacle when used in a reflective environment the whole system, enhance the reliability of data, improve (Figure 2) and improve detection accuracy, a VIDAR-based the accuracy, and increase the information utilization rate of pseudo-obstacle detection method (called improved the system in solving the problems of detection, tracking, VIDAR) has been proposed. *is method’s identification and target recognition. procedure is as follows. *e rectangle of the obstacle was Machine learning is the process of training and iden- determined. *e width of the obstacle rectangle was cal- tifying images using deep convolutional neural networks. culated using the transformation relationship between pixel Compared with other image recognition technologies, coordinates and world coordinates, and then the height of 2 Journal of Robotics machine learning concepts such as SegNet, YOLO v5, faster the obstacle rectangle was calculated using the transfor- mation relationship between pixel coordinates and world RCNN, BigGAN, and mask RCNN are developed [13–20]. While machine learning is capable of accurate classification, coordinates. *e true obstacle is determined by the fact that the actual height of the obstacle rectangle remains constant it can only be used to identify known obstacles. Unknown throughout the ego-vehicle movement. If the obstacle is a obstacles may be missed while the vehicle is moving, which real one, tracking is continued. will cause a serious impact on the vehicle’s safety. To accelerate the detection speed of improved VIDAR, Generally, obstacles are detected using significant in- we combined it with machine learning (this article uses the formation such as color and prior shape. Zhu et al. proposed faster RCNN algorithm) to identify known obstacles, which a method for detecting vehicles based on their edges and we refer to as improved VM. *e improved VM obstacle symmetry characteristics [21]. *ey hypothesized the ve- detection method can quickly and accurately detect obstacles hicle’s location based on the image’s detected symmetric regions. *e vehicle’s bounding box is determined using the on reflection roads. *e enhanced VM obstacle detection procedure is as follows: first, machine learning is used to projected image of the enhanced vertical and horizontal edges. Zhang et al. [21–23] used color information for identify known obstacles; second, the identified obstacles are removed from the background area; and finally, pseudo- background removal and shadow detection to improve obstacles are eliminated through the use of enhanced object segmentation and background updating. *is method VIDAR. is capable of rapidly and precisely detecting moving objects. Zhang et al. [24] introduced Deep Local Shapes (DeepLS), which are high-quality 3D shapes that can be encoded and 2. Related Work reconstructed without requiring an excessive amount of As the core section of automobile-assisted driving, obstacle storage. *is local shape of the scene decomposition sim- detection has emerged as a critical area of research in recent plifies the prior distribution that the network must learn and accelerates and accurately detects obstacles. However, in an years. Due to its simple ranging principle, the monocular vision sensor has become the primary obstacle identification environment with reflections, the reflections contain sig- nificant information about the obstacles they use, reducing equipment in obstacle identification. Many scholars have conducted related research on obstacle identification to the accuracy of obstacle detection. *e vehicle’s position is generally determined by high- accelerate the process. Traditional image object classification and detection algorithms and strategies are difficult to meet light information, such as the highlighted area and contour the requirements of image and video big data in terms of features. Park and Song [25] proposed a front vehicle processing efficiency, performance, and intelligence. Deep identification algorithm based on contrast enhancement and learning establishes the mapping from low-level signals to vehicle lamp pairing. Lin et al. [26] discovered that the high-level semantics by simulating the hierarchical structure characteristics of headlights were more distinctive than the similar to the human brain, so as to realize the hierarchical contours of vehicles and had a greater identification effect and thus proposed the use of lamps as a sign for vehicle feature expression of data, and has powerful visual infor- mation processing capabilities. *erefore, in the field of identification at night. *e Hough transform was proposed by Dai et al. [27] as a method for intelligent vehicle iden- machine vision, the representative of deep learning-con- volutional neural network (CNN) is widely used [7, 8]. tification at night. *is method divides the extracted lamps Convolutional neural networks are also called cellular into connected domains, extracts the lamps’ edges, and then nonlinear networks. Arena et al. have stressed the universal identifies the circle using the Hough transform. Finally, by role that cellular nonlinear networks (CNNs) are assuming pairing the lamps, the vehicle’s location is determined. today. It is shown that the dynamical behavior of 3D CNN- Kavya et al. [28] proposed a method for detecting vehicles based models allows us to approach new emerging problems, based on the color of the brake lamp during braking in the to open new research frontiers [9]. Shustanov and Yakimov captured color image. *e feature information required for proposed an implementation of the traffic signs recognition the above identification method in a reflective environment will detect lamp reflections. *e vehicle’s lamps will also be algorithm using a convolution neural network; training of the neural network is implemented using the TensorFlow paired, which will reduce the vehicle’s accuracy of identi- fication. We used a modified VM to detect obstacles, library and massively parallel architecture for multithreaded programming CUDA; and the experiment proves the high allowing for eliminating obstacles in the reflection, thereby efficiency of this method [10]. Zhu et al. have proposed a increasing obstacle detection accuracy. novel image classification framework that combines CNN and KELM (kernel extreme learning machines). *ey 3. Methodology of Improved VIDAR’s Pseudo- extracted feature categories using DenseNet as a feature Obstacle Detection extractor and radial basis function kernel ELM as a classifier to improve image classification performance [11]. Wang *e monocular visual identification method, based on et al. proposed the occlusion-free road segmentation net- machine learning, is limited to identifying previously work, a fully convolutional neural network. *rough fore- identified obstacles. A vehicle collision accident may occur ground objects and visible road layouts, this method can when unknown obstacles are present on the road. When predict roads in the semantic domain [12]. *e accuracy of VIDAR is used to detect obstacles, pseudo-obstacles in the obstacle identification is also continuously improved as new reflection environment are mistaken for real obstacles. *us, Journal of Robotics 3 Figure 1: *e results of obstacle identification based on faster RCNN. Car Car Car Car Car Car Car Car Car Car Figure 2: *e results of car detection based on VIDAR on reflection roads. to increase the speed and accuracy of obstacle detection, we homogeneous coordinates. Combine the rotation matrix R use an improved VM. and the offset matrix T to obtain the external parameter matrix K, where c and c are the offsets. *e coordinate x y transformation is shown in Figure 3. 3.1. Transformation from World Coordinates to Pixel *e internal parameters of the camera are obtained by Coordinates. *e camera can project objects in the three- Zhang Zhengyou demarcate to determine the transforma- dimensional world into a two-dimensional image by cap- tion relationship between world coordinates and pixel turing an image. In reality, the imaging model establishes a coordinates. projection mapping relationship between three-dimensional q � MKQ. (1) and two-dimensional space. *e coordinate transformation x f 0c is required to convert the world coordinate system’s coor- x x ⎢ ⎥ ⎢ ⎥ ⎡ ⎢ ⎤ ⎥ ⎡ ⎢ ⎤ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ Among them: q � y , M � 0f c , ⎣ ⎦ ⎣ ⎦ dinates to the camera coordinate system’s coordinates. A y y w 001 rigid body transformation is used to convert the world X x f 0c coordinate system to the camera coordinate system. It is x x R t R t ⎡ ⎢ ⎤ ⎥ ⎡ ⎢ ⎤ ⎥ ⎡ ⎢ ⎤ ⎥ 3×3 3×1 ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ 3×3 3×1 ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ K � 􏼢 􏼣, and Q � ⎣ Y ⎦, ⎣ y ⎦ � ⎣ 0f c ⎦􏼢 􏼣 determined by the camera’s external parameters. *e camera y y 0 1 01 W w 001 coordinate system to pixel coordinate system transformation converts three-dimensional coordinates to two-dimensional ⎡ ⎢ ⎤ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ Y ⎥ plane coordinates, as determined by the camera’s internal ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥. ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎣ ⎦ parameters. Although both the pixel and image coordinate Z systems are located on the imaging plane, their origins and units of measurement are distinct. *e origin of the image 3.2. Obstacle Ranging Model. *e range of obstacles is as coordinate system is the point at which the camera’s optical follows (Figure 4). Let f be the focal length of the camera; h axis intersects the imaging plane, which is typically the be the installation height of the camera; μ be the pixel size; z imaging plane’s midpoint. be the camera pitch angle; (x , y ) be the intersection of the 0 0 Suppose the internal parameters matrix is M. Project image plane and the optical axis of the camera; set to (0, 0); Q(X, Y, Z) in the physical world to the image plane q(x, y). (x, y) be the coordinates of intersection points of obstacles By adding a dimension w to q(x, y), which is the expansion and pavement plane set to P; and the horizontal distance d of q(x, y, w), obtain w � Z. Point q is in the form of between the object point and the camera is 4 Journal of Robotics Imaging coordinate have height if d ≠ d + Δd, and static obstacles can be 1 2 system identified by Δl if Δd is known. Additionally, if the obstacle is moving (as illustrated in Figure 6), the Δl can also be used as an obstacle judgment. (x, y) *e verification process has been shown in the paper [6]. Pixel Coordinate system 3.3. Static Obstacle Identification Model. *ere are two types Yw of static obstacles: real static obstacles and static pseudo- (u,v) Zw obstacles. Static real obstacles refer to actual road obstacles. Camera (X,Y,Z) *e reflections identified as real obstacles during the obstacle coordinate system Xw identification process are called static pseudo-obstacles. It is World coordinate Yc a type of pseudo-obstacle that reflects some road obstacles system but does not affect the vehicle’s driving safety. To improve (c ,c ) x y the accuracy of obstacle identification, we must identify and Xc remove static pseudo-obstacles. Figure 3: Schematic diagram of coordinate transformation. 3.3.1. Static Real Obstacle Identification. First, we used the VIDAR to detect stereo obstacles and determine which d � . (2) object point on the obstacle is the furthest away in the tan z + arctan􏼂 y − y􏼁 μ/f􏼃􏼁 horizontal and vertical directions to construct a rectangle (the obstacle rectangle, Figure 7). Let A (x , y ) be the first 1 1 Assume that Y is the Y-axis in the previous image, Y is 1 2 imaging point for the identification of the width of the the Y-axis in the previous image. When the camera moves rectangular road surface of the obstacle, B (x , y ) be the 2 1 from Y to Y on the axis of the imaging plane (Figure 5), let 1 2 other imaging point for the identification of the width of the A be an imaging point for obstacle’s top in the subsequent rectangular road surface of the obstacle, and A be the object image, B be the same imaging point for obstacle’s top in the point of A; similarly, B is the object point of B. *e hor- ′ ′ latter image, A is the object point of A, and B is the object izontal distances d between A and the camera can be ′ 1 point of B. d is the horizontal distance between A and the calculated by Equation (2). Similarly, the horizontal dis- camera; similarly, d is the horizontal distance between B tances d between B and the camera can also be calculated. and the camera. d and d can be obtained from Equation 1 2 *e width of the obstacle rectangle can be calculated using (3). *e camera moved a certain distance Δd during the time the pinhole imaging principle and the geometrical rela- between the previous and subsequent images; d � d + Δd, 1 2 tionship between cameras. ′ ′ but d � d + Δd + Δl actually. As a result, the A and B 1 2 􏽶��������������������������������������������������������� � 􏽱������􏽱������ 2 2 2 2 2 2 2 2 2 􏼐2f + 2y + x + x − μ x − x 􏼁 􏼑 h + d h + d 1 1 2 2 1 1 2 2 2 2 􏽱���������� �􏽱���������� � (3) w � 2h + d + d − . 1 2 2 2 2 2 2 2 f + x + y f + x + y 1 1 2 1 When the camera moves from Y to Y on the axis of the the width of the opposite side of the obstacle, C be the object 1 2 imaging plane (Figure 8), let C (x , y ) be an imaging point point of C, and D be the object point of D. (4) and (5) have 3 2 for identifying the width of the opposite side of the obstacle, the same width. *e height of the pseudo-obstacle rectangle D(x is calculated. , y ) be another imaging point for the identification of 4 2 2 2 2 2 2 2 2 2 2 2 2f + 2y + x + x − μ x − x 2 h − h + d + d − w 􏼁 􏼁 2 3 4 4 3 v 3 4 􏽱���������� �􏽱���������� � 􏽱������������􏽱������������ � . (4) 2 2 2 2 2 2 2 2 2 2 f + x + y f + x + y h − h 􏼁 + d h − h 􏼁 + d 3 2 4 2 v 3 v 4 3.3.2. Static Pseudo-Obstacle Identification. *e procedure pseudo-obstacles using VIDAR and construct a rectangle for identifying pseudo-obstacles is similar to the procedure (pseudo-obstacle rectangle) from the object points on the for identifying real obstacles. However, when obstacles are pseudo-obstacle (the object points are the farthest in the detected using VIDAR, the object points of the obstacles are horizontal and vertical directions). Let A be the first imaging different from their actual positions (Figure 9). We detect point for pseudo-obstacle width identification with (x , y ), 1 1 Focal Length (f) Journal of Robotics 5 (x, y) Len′s Center (x , y ) 0 0 Horizontal Line Image Plane Optic Axis Road Plane Figure 4: Schematic diagram of the horizontal distance between the object point and the camera. Y Y 1 2 Obstacle′s Imaging Point First Imaging Point Δl Δd B′ A′ Figure 5: Static obstacle imaging. Y Y 1 2 Obstacle′s Imaging Point First Imaging Point Δl Δd s A′ B′ Figure 6: Moving obstacle imaging. BA B′ A′ Figure 7: Schematic diagram of static obstacle. hv 6 Journal of Robotics B be the other imaging point for pseudo-obstacle width camera can also be calculated using (2), and the width of the identification with (x , y ), A be the object point of point A, pseudo-obstacle rectangle can be calculated by the pinhole 2 1 and B be the object point of point B. *e horizontal dis- imaging principle and the geometrical relationship between tances d between A and the camera can be calculated using cameras. (2). Similarly, the horizontal distances d between B and the 􏽶��������������������������������������������������������� � 􏽱������􏽱������ 2 2 2 2 2 2 2 2 2 2 􏼐2f + 2y + x + x − μ x − x 􏼁 􏼑 h + d h + d 1 1 2 2 1 1 2 2 2 2 􏽱���������� �􏽱���������� � (5) w � 2h + d + d − . 1 2 2 2 2 2 2 2 f + x + y f + x + y 1 1 2 1 When the camera moves from Y to Y on the axis of the point of D. At this point, the width of the object point of the 1 2 imaging plane (Figure 10), after the pseudo-obstacle moved, pseudo-obstacle changes from W to W . Similarly, W can 1 2 2 let C (x , y ) be an imaging point of the rectangular width of be solved using the pinhole imaging principle and the 3 2 the identified pseudo-obstacle and D (x , y ) be another geometrical relationship between cameras. *e rectangular 4 2 imaging point of the rectangular width of the pseudo-ob- height of the pseudo-obstacle can be obtained by (5) and (6) stacle, and C be the object point of C and D be the object and the triangle similarity principle. ′ ′ 2 2 2 2 2 2 2 2 2 2 2 2f + 2y + x + x − μ x − x 􏼁 2 h − h 􏼁 + d + d − h/h + h 􏼁 w 2 3 4 4 3 v 3 4 v 􏽱���������� �􏽱���������� � 􏽱������������􏽱������������ � . (6) 2 2 2 2 2 2 2 2 2 2 f + x + y f + x + y h − h 􏼁 + d h − h 􏼁 + d v v 3 2 4 2 3 4 3.4. Static Obstacle Identification Model. Moving obstacles are 3.4.1. Static Pseudo-Obstacle Identification. *e steps for classified as either moving real obstacles or moving pseudo- identifying real moving obstacles are identical to those for obstacles. Moving real obstacles refers to obstacles on the road. real static obstacles (Figure 11). VIDAR is used to detect *e reflections identified as real obstacles during the obstacle stereo obstacles, construct obstacle rectangles, and calculate identification process are referred to as moving pseudo-ob- their width. After the ego-vehicle and obstacle have been stacles. It is a type of pseudo-obstacle that replicates some road moved, the width of the obstacle rectangle is recalculated obstacles but does not destroy the vehicle’s driving safety. We and then the obstacle height is solved using the triangle must identify and remove moving pseudo-obstacles to improve similarity principle. accuracy when identifying obstacles. 2 2 2 2 2 2 2 2 2 2 2f + 2y + x + x − μ x − x 􏼁 2 h − h 􏼁 + d + d − w 4 3 v 2 3 4 3 4 􏽱���������� �􏽱���������� � 􏽱������������􏽱������������ � . (7) 2 2 2 2 2 2 2 2 2 2 f + x + y f + x + y h − h 􏼁 + d h − h 􏼁 + d 3 2 4 2 v 3 v 4 3.4.2. Moving Pseudo-Obstacle Identification. *e steps for rectangle, and calculating the pseudo-obstacle rectangle’s identifying moving pseudo-obstacles are identical to those width. Following the ego-vehicle and pseudo-obstacle for static pseudo-obstacles (Figure 12): detecting stereo movement, the height of the pseudo-obstacle is calculated obstacles with VIDAR, determining the pseudo-obstacle using the width of the pseudo-obstacle’s imaging point. 2 2 2 2 2 2 2 2 2 2 2 2f + 2y + x + x − μ x − x 􏼁 2 h − h 􏼁 + d + d − h/h + h 􏼁 w 4 3 v v 2 3 4 3 4 􏽱���������� �􏽱���������� � 􏽱������������􏽱������������ � . (8) 2 2 2 2 2 2 2 2 2 2 f + x + y f + x + y h − h 􏼁 + d h − h 􏼁 + d v v 3 2 4 2 3 4 3.5. Removal Model of Pseudo-Obstacles. *e ego-vehicle vehicle resumes motion, the width and height of the obstacle movement assesses the obstacle’s authenticity. Using (3) and and pseudo-obstacle are also calculated using (4) and (6), (5), the widths of obstacles and pseudo-obstacles when the and the heights of the obstacle and pseudo-obstacle are vehicle moves for the first time are calculated. When the determined by their widths. Compared with the calculated Journal of Robotics 7 Y Y Y Y Y Y 1 1 2 2 D C D C A BA BA X l A 5 6 C B l 6 l l D y D 1X f 2 m 1 X m α 2 D′ 4 h d 3 D′ d Δd d m C′ h 4 3 2 m v h 1 B′ C′ 2 Δd 3 h v W d S A′ Figure 11: Schematic diagram of moving obstacle. Figure 8: Schematic diagram of static pseudo-obstacles after the ego-vehicle moved. D C B A 3 A B l l l 3 5 l 4 BA 1X α 2 f m B′ X Δd 1 h 3 D′ l d 2 4 l d 1 d 3 C′ A′ m Figure 12: Schematic diagram of moving pseudo-obstacle after the ego-vehicle moved. B′ d A′ are the furthest apart in the horizontal and vertical directions (Figure 13 and Figure 14). (3) Calculate the horizontal distance. Determine the Figure 9: Schematic diagram of static pseudo-obstacle. horizontal distance from the object point to the camera according to VIDAR. Y (4) Identify obstacles. First, calculate the width W of the D C obstacle rectangle. Second, determine the relation- BA ship between the height h and width W through the triangle similarity principle. 6 A l y B (5) Calculate the rectangular height of real and pseudo- X β 1 X obstacles by using the same width when vehicles and h m 4 d D′ 4 obstacles move. Calculate the height value twice, Δd C′ compare the two height values, and determine the W identified obstacle. *e overall flow of obstacle identification is shown in Figure 15. Figure 10: Schematic diagram of static pseudo-obstacle after the ego-vehicle moved. 4. Obstacle Identification Experiment and results for obstacles, detected obstacles are those with a Effect Analysis similar height. We analyze the identification effect of the VM and improved *e process of obstacle identification is as follows: VM in two environments. On the movable platform, the (1) Confirm stereo unknown obstacles. Machine experimental equipment, including the camera unit and learning is used to identify known obstacles, obtain IMU, is installed (Figure 16(a)). A scale model of the vehicle images after removing the known obstacles, and then is used to simulate a specific obstacle. To simulate the un- screen out stereo unknown obstacles using VIDAR’s known obstacle, a beverage bottle cap is used (Figure 16(b)). obstacle detection principle. *e polished paper is used to create a reflection of the road (2) Construct an obstacle rectangle. To construct a (Figure 16(c)). *e camera’s video captured at a frame rate of rectangle, locate the object points on the obstacle that 20 fps is utilized to generate an image sequence, and then 8 Journal of Robotics Obstacle Point Rectangle (a) (b) Figure 13: *e first image. (a) *e feature points. (b) Construction of the obstacle rectangle by the feature points. Point Obstacle Rectangle (a) (b) Figure 14: *e second image. (a) *e feature points. (b) Construction of the obstacle rectangle by the feature points. obstacle detection on the generated image sequence is height unchanged is confirmed, and the real obstacles are performed. marked. *e previous image and latter image are used to judge whether the height of the obstacle rectangle has changed 4.1. Improved VIDAR and Improved VM Simulation (Figure 17). Experiments. A beverage bottle cap is used as an unknown In the VM and improved VM comparison experiments, obstacle, and the angular acceleration and acceleration of the the faster RCNN is used to identify known obstacles and to ego-vehicle are obtained from the IMU installed in the ego- identify known obstacles as background, while VIDAR and vehicle. *e quaternion method is used to solve the camera improved VIDAR are used to perform secondary detection attitude and update the camera pitch angle. *e image is on the background-removed image to identify unknown processed by a fast image region matching method based on obstacles (Figure 18, Figure 19, and Figure 20). MSER. Acceleration is used to calculate the horizontal *e detection of unknown obstacles in this paper is distance between the vehicle and the obstacles. *e height of shown in Figure 19 and Figure 20. While VIDAR in the VM the obstacle rectangle by keeping the actual width constant is capable of identifying the bottle cap, the cap’s reflection is during the vehicles and obstacles movement is calculated, also detected as an obstacle, resulting in low obstacle the authenticity of the identified obstacles by keeping the identification accuracy. When the improved VIDAR is used Journal of Robotics 9 Machine learning at Background Feature Points Non-road Construct Obstacle t to detect known ROI Extraction extraction Extraction Obstacle Screening Rectangle obstacles Inertial Data Camera Data t=0 Acquisition Update W at t Camera Data at t Δd Calculation t=t+1/f h and h at Feature Points v1 v2 t=t+2Δt Camera Moving t=t+Δt Camera Moving t+Δt Tracking Feature Points h and h at N Y v3 v4 h = h ? h = h ? Pseudo Obstacle v1 v3 v2 v4 Tracking t+2Δt Real Obstacle Figure 15: Improved VM pseudo-obstacle identification flowchart. IMU Movable Platform Camera Unit (a) (b) (c) Figure 16: Experimental equipment. (a) Mobile platform, camera unit, IMU. (b) Vehicle scaled model (known obstacle) and unknown obstacle model. (c) Polished paper simulates reflection road. car car Obstacle Figure 17: Detection result of bottle cap. Figure 18: Faster RCNN identification effect diagram. to detect unknown obstacles, the obstacles in the reflection without height can be eliminated, compensating for the MV-VDF300SC industrial digital camera is used as a unknown obstacles being misdetected in the reflective en- vironment. As a result, the improved VM detects obstacles monocular vision sensor. *is model camera adopts the USB 2.0 standard interface and has a high resolution, precision, more precisely than the baseline VM. and clarity. *e camera’s performance parameters are listed in Table 1. *e camera is installed at the height of 1.60 m and 4.2. Analysis of the Identification Result of Improved VM and collects real-time environmental data (we only used the left Improved VIDAR. In the experimental test, a pure electric camera). *e HEC295 IMU is mounted on the bottom of the vehicle is used as the test vehicle (Figure 21). A test vehicle and is used to locate and read the vehicle’s е 10 Journal of Robotics car car Obstacle Obstacle Figure 19: VM identification effect diagram. car car Obstacle Figure 20: Improved VM identification effect diagram. Camera Computing processing unit and digital map GPS+IMU Figure 21: Schematic diagram of the test vehicle. Table 1: Some performance parameters of the MV-VDF300SC camera. MV-VDF300SC Highest resolution 2048 ∗1536 Power requirements (V) 5 Output color Color Power consumption (W) Rated <5 Frame rate (fps) 12 Operating temperature ( C) 0–60 Output method USB 2.0 Dimensions (mm) 43.3 ∗ 29 ∗ 29 motion status in real time. GPS is used to determine a precise to perform real-time data processing. In the process of location. Digital maps are utilized to obtain precise road calculation, multisensor data processing is the combination data, such as distance and slope. *e computing unit is used and processing of multisource information, which is rather Journal of Robotics 11 0.35 Drag to select outliers 0.3 -400 0.25 -200 0.2 0.15 0.1 0.05 1000 0 Z (millimeters) 0 -500 0 5 10 15 20 X (millimeters) Images Overall Mean Error: 0.25 points Figure 22: *e camera calibration result. *e calibration of the camera’s external parameters can Table 2: Calibration results of camera external parameters. be calculated by taking the edge object points of lane lines. External parameter type Parameter size *e calibration results are shown in Table 2. Pitch angle 1.25 Due to a lack of reflection road images in the public data Yaw angle 3.65 set and the fact that different camera parameters would affect Rotation angle 2.45 range accuracy, we created a VIDAR-Reflection Road da- tabase (Figure 23) with a total of 2000 images. *e MV- VDF300SC camera unit was used to record the experiment complicated. Fuzzy logic can deal with complex systems in its natural environment. *e test roads were Xuezhai [29]. It can coordinate and combine the acquired infor- Road and Jiefang East Road in Jinan, Shandong Province, mation to improve the efficiency of the system and effectively and traffic environment images were collected during rainy deal with the knowledge acquired in the scene. days from 10 : 00 to 11 : 00 and 19 : 00 to 20 : 00. Accurate calibration of camera parameters was a pre- Figure 24 depicts the identification result for the two requisite for the whole experiment and is a very important images. *e VM and improved VM accuracy are compared task for obstacle detection methods. In this paper, Zhang by counting the number of TP, FP, TN, and FN obstacles in Zhengyou’s camera calibration method was adopted to each image frame. Let a be an obstacle that is correctly calibrate the DaYing camera. First, the camera was fixed to identified as a positive example; b be an obstacle that is capture images of a checkerboard at different positions and incorrectly identified as a positive example; c be an obstacle angles. *en, key points of the checkerboard were selected that is correctly identified as a negative example; and d be an and used to establish a relationship equation. Finally, the obstacle that is incorrectly identified as a negative example. internal parameter calibration was realized. *e camera n n n *en, TP � 􏽐 a , FP � 􏽐 b , TN � 􏽐 c , and i i i i�1 i�1 i�1 calibration process and result are shown in Figure 22. FN � d . *e comparison of the detection effects of VM i�1 i Camera distortion includes radial distortion, thin lens and improved VM in the reflection environment is shown in distortion, and centrifugal distortion. *e superposition of Table 3. the three kinds of distortion results in a nonlinear distortion, In the results’ analysis, accuracy (A), recall (R), and the model of which can be expressed in the image coordinate precision (P) were used as evaluation indices for the two system as follows: obstacle detection methods, calculated through 2 2 3 2 2 ⎧ ⎨δ (x, y) � s x􏼐x + y 􏼑 + 2p xy + p y + k x􏼐x + y 􏼑 x 1 1 2 1 TP + TN ⎩ 2 2 3 2 2 (11) A � , δ (x, y) � s y x + y + 2p xy + p y + k x x + y 􏼐 􏼑 􏼐 􏼑 y 2 2 1 1 TP + TN + FP + FN (9) TP R � , (12) where s and s are the centrifugal distortion coefficients; k TP + FN 1 2 1 and k are the radial distortion coefficients, and p and p 2 1 2 TP are the distortion coefficients of thin lenses. (13) P � . Because the centrifugal distortion of the camera is not TP + FP considered in this paper, the internal reference matrix of the *e accuracy, recall, and precision of the method pro- camera can be expressed as shown in posed in this paper are shown in Table 4. As demonstrated by the experimental results in Table 4, 5.9774e + 03 0 949.8843 ⎢ ⎥ ⎡ ⎢ ⎤ ⎥ ⎢ ⎥ ⎢ ⎥ the accuracy of obstacle identification is increased when ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ M � ⎢ 0 5.9880e + 03 357.0539 ⎥. (10) ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎣ ⎦ using the improved VM for obstacle identification in a 0 0 1 reflective environment. Due to the weather and other fac- tors, there are times when misidentification and missed Mean Error in Points Y (millimeters) 12 Journal of Robotics Figure 23: A partial sample of the VIDAR-Reflection Road data set. Figure 24: Comparison of detection results of VM and improved VM in an environment with reflection. identification occur during the experiment. However, the *e term “real time” refers to the processing of each improved method proposed in this paper improves obstacle image frame collected over time. In terms of detection speed, identification accuracy. 2000 images were processed using improved VIDAR, VM, Additionally, we compared our method’s detection ac- VIDAR, VM, and YOLO v5. Table 6 summarizes the average curacy to other commonly used target detection methods. detection times for the five identification methods. Table 5 summarizes the detection results. It is obvious that As shown in Table 6, an improved VM takes longer to the proposed obstacle detection method outperforms state- determine the authenticity of obstacles than a VM. Similarly, of-the-art methods in terms of accuracy. improved VIDAR requires more time to determine the Journal of Robotics 13 Table 3: Comparison of the detection effect of VM and improved VM in an environment with reflection. Input TP FP TN FN VM 7356 6591 365 279 392 Improved VM 7356 6832 224 252 243 Table 4: Evaluation indices of the proposed method. Detection method A (%) R (%) P (%) VM 90.05 94.38 94.63 Improved VM 93.81 96.56 96.82 Table 5: Evaluation indices of advanced target detection methods and the proposed method. Detection method A (%) Fast RCNN 76.53 SSD 71.28 Fast YOLO 78.95 SSD 300 73.96 YOLO v5s 81.73 Proposed method 90.36 Table 6: Evaluation indices of detections of YOLO v5s and proposed method. Improved VIDAR VIDAR Improved VM VM YOLO v5 Identification time (s) 0.627 0.538 0.372 0.296 0.273 needs a lot of calculations, improving the efficiency of the authenticity of obstacles when compared with VIDAR. Due to the fewer feature points, improved VM detects faster than proposed method will be the next research direction. In improved VIDAR, and less time is required. As a result, addition, obstacle detection is a prerequisite for obstacle using an improved VM for obstacle detection takes not only avoidance, and an improved obstacle avoidance method is advantage of machine learning’s speed but also improves also a future research direction. identification accuracy. Data Availability 5. Conclusion Data are available on request to the corresponding author. *is paper first proposes an improved VIDAR method based Conflicts of Interest on VIDAR and then combines machine learning to propose an improved method for VM obstacle identification. On the *e authors declare that they have no conflicts of interest. basis of machine learning to detect known obstacles, VIDAR is used to determine whether there is an obstacle with height Acknowledgments by calculating the position of road imaging points, the obstacle rectangle is determined for nonroad obstacles, and *is work was supported in part by the National Natural then the obstacle height (including real obstacles and Science Foundation of China under Grant 51905320, the pseudo-obstacles) is calculated by using the obstacle imaging China Postdoctoral Science Foundation under Grants points of two frames before and after the vehicle moves. By 2018M632696 and 2018M642684, the Shandong Key R and calculating the height after moving again (including real D Plan Project under Grant 2019GGX104066, and SDUT obstacles and pseudo-obstacles), the two heights are com- and Zibo City Integration Development Project under Grant pared to determine the authenticity of the obstacle, so as to 2017ZBXC133. realize the obstacle detection. *is paper aims to show the effect of obstacle detection using improved VM in the en- References vironment with reflection. *e experimental results indicate that when compared with VM, the improved VM method [1] D. Gusland, B. Torvik, E. Finden, F. Gulbrandsen, and for obstacle detection is more accurate in a reflective en- R. Smestad, “Imaging radar for navigation and surveillance on vironment. Because the method proposed in this paper an autonomous unmanned ground vehicle capable of 14 Journal of Robotics identifying obstacles obscured by vegetation,” in Proceedings [17] K. H. Lin, H. M. Zhao, J. J. Lv et al., “Face identification and of the 2019 IEEE Radar Conference (RadarConf), pp. 1–6, segmentation based on improved mask R-CNN,” Discrete Movings Boston, MA, USA, 2019. in Nature and Society, vol. 2020, Article ID 9242917, 2020. [18] Y. Tian, G. Yang, Z. Wang, E. Li, and Z. Liang, “Instance seg- [2] M. P. Muresan, S. Nedevschi, and I. Giosan, “Real-time object identification using a sparse 4-layer LIDAR,” in Proceedings of mentation of apple flowers using the improved mask R-CNN the 2017 13th IEEE International Conference on Intelligent model,” Biosystems Engineering, vol. 193, pp. 264–278, 2020. Computer Communication and Processing (ICCP), pp. 317–322, [19] X. X. Zhang and X. Zhu, “Moving vehicle identification in Cluj-Napoca, 2017. aerial infrared image sequences via fast image registration and [3] M. Cho, “A study on the obstacle identification for autono- improved YOLOv3 network,” International Journal of Remote mous driving RC car using LiDAR and thermal infrared Sensing, vol. 11, no. 41, pp. 4312–4335, 2020. camera,” in Proceedings of the 2019 Eleventh International [20] T. Haas, C. Schubert, M. Eickhoff, and H. Pfeifer, “BubCNN: Conference on Ubiquitous and Future Networks (ICUFN), bubble identification using Faster RCNN and shape regres- pp. 544–546, Zagreb, Croatia, 2019. sion network,” Chemical Engineering Science, vol. 216, Article [4] S. Wang and X. Li, “A real-time monocular vision-based ID 115467, 2020. obstacle identification,” in Proceedings of the 2020 6th In- [21] M. Zhang, W. Zhao, and X. Li, “Shadow detection of moving ternational Conference on Control, Automation and Robotics objects in traffic monitoring video,”vol. 9, pp. 1983–1987, in Proceedings of the 2020 IEEE 9th Joint International Infor- (ICCAR), pp. 695–699, Singapore, 2020. [5] W. Song, Y. Yang, M. Fu, Y. Li, and M. Wang, “Lane iden- mation Technology and Artificial Intelligence Conference tification and classification for forward collision warning (ITAIC), vol. 9, IEEE, Chongqing, China, December 2020. system based on stereo vision,” IEEE Sensors Journal, vol. 12, [22] D. J. R. Del Carmen and R. D. Cajote, “Assessment of vision- pp. 5151–5163, 2018. based vehicle tracking for traffic monitoring applications,” in [6] X. Yi, G. Song, and T. Derong, “Fast road obstacle identifi- Proceedings of the 2018 Asia-Pacific Signal and Information cation method based on maximally stable extremal regions,” Processing Association Annual Summit and Conference International Journal of Advanced Robotic Systems, vol. 15, (APSIPA ASC), pp. 2014–2021, IEEE, Honolulu, Hawai, no. 1, Article ID 1729881418759118, 2018. November 2018. [7] A. Sharif Razavian, H. Azizpour, and J. Sullivan, “CNN [23] S. H. Shaikh, K. Saeed, and N. Chaki, “Moving object de- features off-the-shelf: an astounding baseline for recognition,” tection using background subtraction,” Moving Object De- Proceedings of the IEEE conference on computer vision and tection Using Background Subtraction, Springer, Cham, pattern recognition workshops, pp. 806–813, 2014. pp. 15–23, 2014. [24] R. Chabra, J. E. Lenssen, E. Ilg et al., “Deep local shapes: learning [8] S. Y. Wang, O. Wang, and R. Zhang, “Cnn-generated images are surprisingly easy to spot for now,” in Proceedings of the local sdf priors for detailed 3d reconstruction,” in Computer IEEE/CVF conference on computer vision and pattern recog- Vision - ECCV 2020, pp. 608–625, Springer, Cham, 2020. nition, pp. 8695–8704, Seattle, WA, USA, June 2020. [25] J. Park and B. C. Song, “Night-time vehicle identification [9] P. Arena, M. Bucolo, S. Fazzino, L. Fortuna, and M. Frasca, using low exposure video enhancement and lamp identifi- “*e CNN paradigm: shapes and complexity,” International cation,” in Proceedings of the 2016 International Conference on Journal of Bifurcation and Chaos, vol. 15, no. 07, pp. 2063– Electronics, Information, and Communications (ICEIC), 2090, 2005. pp. 1-2, Da Nang, Danang, Vietnam, January 2016. [10] A. Shustanov and P. Yakimov, “CNN design for real-time [26] H. P. Lin, P. H. Liao, and Y. L. Chang, “Long-distance vehicle traffic sign recognition,” Procedia Engineering, vol. 201, identification algorithm at night for driving assistance,” in pp. 718–725, 2017. Proceedings of the 3rd IEEE International Conference on In- telligent Transportation Engineering, pp. 296–300, Singapore, [11] X. Zhu, Z. Li, X.-Y. Zhang, P. Li, Z. Xue, and L. Wang, “Deep convolutional representations and kernel extreme learning September 2018. machines for image classification,” Multimedia Tools and [27] X. Dai, D. Liu, L. Yang, and Y. Liu, “Research on headlight Applications, vol. 78, no. 20, pp. 29271–29290, 2019. technology of night vehicle intelligent identification based on [12] K. Wang, F. Yan, B. Zou, L. Tang, Q. Yuan, and C. Lv, Hough transform,” in Proceedings of the 2019 International “Occlusion-free road segmentation leveraging semantics for Conference on Intelligent Transportation, Big Data & Smart autonomous vehicles,” Sensors, vol. 19, no. 21, p. 4711, 2019. City (ICITBS), pp. 49–52, Changsha, China, 2019. [13] G. D. Guo and N. Zhang, “A survey on deep learning based [28] T. S. Kavya, E. Tsogtbaatar, Y. Jang, and S. Cho, “Night-time face identification,” Computer Vision and Image Under- vehicle identification based on brake/tail light color,” in standing, vol. 189, Article ID 102805, 2019. Proceedings of the 2018 International SoC Design Conference [14] W. K. Jia, Y. Y. Tian, R. Luo, Z. H. Zhang, J. Lian, and (ISOCC),, pp. 206-207, Daegu, Korea (South), 2018. Y. J. Zheng, “Identification and segmentation of overlapped [29] M. Bucolo, L. Fortuna, and M. LaRosa, “Complex dynamics fruits based on optimized mask R-CNN application in apple through fuzzy chains,” IEEE Transactions on Fuzzy Systems, vol. 12, no. 3, pp. 289–295, 2004. harvesting robot,” Computers and Electronics in Agriculture, vol. 172, Article ID 105380, 2020. [15] S. H. Wang, J. D. Sun, I. Mehmood, C. C. Pan, Y. Chen, and Y. D. Zhang, “Cerebral micro-bleeding identification based on a nine-layer convolutional neural network with stochastic pooling,” Concurrency and Computation: Practice and Ex- perience, vol. 1, no. 2, Article ID e5130, 2020. [16] B. Sariturk, B. Bayram, Z. Duran, and D. Z. Seker, “Feature extraction from satellite images using segnet and fully con- volutional networks (FCN),” International Journal of Elec- tronic Governance, vol. 3, no. 5, pp. 138–143, 2020. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Journal of Robotics Hindawi Publishing Corporation

An Improved VM Obstacle Identification Method for Reflection Road

Download PDF
14 pages

Loading next page...
 
/lp/hindawi-publishing-corporation/an-improved-vm-obstacle-identification-method-for-reflection-road-bwfhIPKobZ

References (30)

Publisher
Hindawi Publishing Corporation
Copyright
Copyright © 2022 Guoxin Jiang et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
ISSN
1687-9600
eISSN
1687-9619
DOI
10.1155/2022/3641930
Publisher site
See Article on Publisher Site

Abstract

Hindawi Journal of Robotics Volume 2022, Article ID 3641930, 14 pages https://doi.org/10.1155/2022/3641930 Research Article An Improved VM Obstacle Identification Method for Reflection Road Guoxin Jiang , Yi Xu , Xiaoqing Sang , Xiaotong Gong , Shanshang Gao , Ruoyu Zhu , Liming Wang , and Yuqiong Wang School of Transportation and Vehicle Engineering, Shandong University of Technology, Zibo 255000, China Correspondence should be addressed to Yi Xu; xuyisdut@163.com Received 23 January 2022; Accepted 18 February 2022; Published 14 March 2022 Academic Editor: L. Fortuna Copyright © 2022 Guoxin Jiang et al. *is is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. An obstacle detection method based on VM (VIDAR and machine learning joint detection model) is proposed to improve the monocular vision system’s identification accuracy. When VIDAR (Vision-IMU-based detection and range method) detects unknown obstacles in a reflective environment, the reflections of the obstacles are identified as obstacles, reducing the accuracy of obstacle identification. We proposed an obstacle detection method called improved VM to avoid this situation. *e experimental results demonstrated that the improved VM could identify and eliminate unknown obstacles. Compared with more advanced detection methods, the improved VM obstacle detection method is more accurate. It can detect unknown obstacles in reflection, reflective road environments. machine learning has an extremely high recognition rate for 1. Introduction specific images. While machine learning is capable of ac- Obstacle detection has become a major concern in the field curate classification, it can only be used to identify known of driver assistance systems due to the complexity of the obstacles. While the vehicle is in motion, using machine outdoor environment. Cameras (monocular, binocular, learning to identify unknown obstacles may result in mis- infrared, etc.), lidar, and millimeter-wave radar are all ex- identification, posing a serious risk to the vehicle’s safety amples of obstacle identification equipment. While lidar and (Figure 1). As a result, a method for detection and ranging millimeter-wave radar are highly accurate at detecting ob- using vision and an IMU (inertial measurement unit) has stacles, their high cost limits their use in low-end vehicles been proposed [6]. Given that VIDAR requires more time to [1–3]. Due to the low cost, high detection accuracy, and run than machine learning, a method combining VIDAR speed of vision-based obstacle identification equipment, it and machine learning to detect obstacles has been proposed has become more suitable for various vehicles [4, 5]. *e (called the VM method). Machine learning is used to identify known obstacles in the proposed method, while vision-based sensor used in this study is a camera. Camera, GPS, and IMU constitute an innovative sensor combination. VIDAR is used to detect unknown obstacles. Compared with single sensor, the application of multisensor To avoid the situation in which VIDAR detects the re- information fusion technology can improve the reliability of flection as an obstacle when used in a reflective environment the whole system, enhance the reliability of data, improve (Figure 2) and improve detection accuracy, a VIDAR-based the accuracy, and increase the information utilization rate of pseudo-obstacle detection method (called improved the system in solving the problems of detection, tracking, VIDAR) has been proposed. *is method’s identification and target recognition. procedure is as follows. *e rectangle of the obstacle was Machine learning is the process of training and iden- determined. *e width of the obstacle rectangle was cal- tifying images using deep convolutional neural networks. culated using the transformation relationship between pixel Compared with other image recognition technologies, coordinates and world coordinates, and then the height of 2 Journal of Robotics machine learning concepts such as SegNet, YOLO v5, faster the obstacle rectangle was calculated using the transfor- mation relationship between pixel coordinates and world RCNN, BigGAN, and mask RCNN are developed [13–20]. While machine learning is capable of accurate classification, coordinates. *e true obstacle is determined by the fact that the actual height of the obstacle rectangle remains constant it can only be used to identify known obstacles. Unknown throughout the ego-vehicle movement. If the obstacle is a obstacles may be missed while the vehicle is moving, which real one, tracking is continued. will cause a serious impact on the vehicle’s safety. To accelerate the detection speed of improved VIDAR, Generally, obstacles are detected using significant in- we combined it with machine learning (this article uses the formation such as color and prior shape. Zhu et al. proposed faster RCNN algorithm) to identify known obstacles, which a method for detecting vehicles based on their edges and we refer to as improved VM. *e improved VM obstacle symmetry characteristics [21]. *ey hypothesized the ve- detection method can quickly and accurately detect obstacles hicle’s location based on the image’s detected symmetric regions. *e vehicle’s bounding box is determined using the on reflection roads. *e enhanced VM obstacle detection procedure is as follows: first, machine learning is used to projected image of the enhanced vertical and horizontal edges. Zhang et al. [21–23] used color information for identify known obstacles; second, the identified obstacles are removed from the background area; and finally, pseudo- background removal and shadow detection to improve obstacles are eliminated through the use of enhanced object segmentation and background updating. *is method VIDAR. is capable of rapidly and precisely detecting moving objects. Zhang et al. [24] introduced Deep Local Shapes (DeepLS), which are high-quality 3D shapes that can be encoded and 2. Related Work reconstructed without requiring an excessive amount of As the core section of automobile-assisted driving, obstacle storage. *is local shape of the scene decomposition sim- detection has emerged as a critical area of research in recent plifies the prior distribution that the network must learn and accelerates and accurately detects obstacles. However, in an years. Due to its simple ranging principle, the monocular vision sensor has become the primary obstacle identification environment with reflections, the reflections contain sig- nificant information about the obstacles they use, reducing equipment in obstacle identification. Many scholars have conducted related research on obstacle identification to the accuracy of obstacle detection. *e vehicle’s position is generally determined by high- accelerate the process. Traditional image object classification and detection algorithms and strategies are difficult to meet light information, such as the highlighted area and contour the requirements of image and video big data in terms of features. Park and Song [25] proposed a front vehicle processing efficiency, performance, and intelligence. Deep identification algorithm based on contrast enhancement and learning establishes the mapping from low-level signals to vehicle lamp pairing. Lin et al. [26] discovered that the high-level semantics by simulating the hierarchical structure characteristics of headlights were more distinctive than the similar to the human brain, so as to realize the hierarchical contours of vehicles and had a greater identification effect and thus proposed the use of lamps as a sign for vehicle feature expression of data, and has powerful visual infor- mation processing capabilities. *erefore, in the field of identification at night. *e Hough transform was proposed by Dai et al. [27] as a method for intelligent vehicle iden- machine vision, the representative of deep learning-con- volutional neural network (CNN) is widely used [7, 8]. tification at night. *is method divides the extracted lamps Convolutional neural networks are also called cellular into connected domains, extracts the lamps’ edges, and then nonlinear networks. Arena et al. have stressed the universal identifies the circle using the Hough transform. Finally, by role that cellular nonlinear networks (CNNs) are assuming pairing the lamps, the vehicle’s location is determined. today. It is shown that the dynamical behavior of 3D CNN- Kavya et al. [28] proposed a method for detecting vehicles based models allows us to approach new emerging problems, based on the color of the brake lamp during braking in the to open new research frontiers [9]. Shustanov and Yakimov captured color image. *e feature information required for proposed an implementation of the traffic signs recognition the above identification method in a reflective environment will detect lamp reflections. *e vehicle’s lamps will also be algorithm using a convolution neural network; training of the neural network is implemented using the TensorFlow paired, which will reduce the vehicle’s accuracy of identi- fication. We used a modified VM to detect obstacles, library and massively parallel architecture for multithreaded programming CUDA; and the experiment proves the high allowing for eliminating obstacles in the reflection, thereby efficiency of this method [10]. Zhu et al. have proposed a increasing obstacle detection accuracy. novel image classification framework that combines CNN and KELM (kernel extreme learning machines). *ey 3. Methodology of Improved VIDAR’s Pseudo- extracted feature categories using DenseNet as a feature Obstacle Detection extractor and radial basis function kernel ELM as a classifier to improve image classification performance [11]. Wang *e monocular visual identification method, based on et al. proposed the occlusion-free road segmentation net- machine learning, is limited to identifying previously work, a fully convolutional neural network. *rough fore- identified obstacles. A vehicle collision accident may occur ground objects and visible road layouts, this method can when unknown obstacles are present on the road. When predict roads in the semantic domain [12]. *e accuracy of VIDAR is used to detect obstacles, pseudo-obstacles in the obstacle identification is also continuously improved as new reflection environment are mistaken for real obstacles. *us, Journal of Robotics 3 Figure 1: *e results of obstacle identification based on faster RCNN. Car Car Car Car Car Car Car Car Car Car Figure 2: *e results of car detection based on VIDAR on reflection roads. to increase the speed and accuracy of obstacle detection, we homogeneous coordinates. Combine the rotation matrix R use an improved VM. and the offset matrix T to obtain the external parameter matrix K, where c and c are the offsets. *e coordinate x y transformation is shown in Figure 3. 3.1. Transformation from World Coordinates to Pixel *e internal parameters of the camera are obtained by Coordinates. *e camera can project objects in the three- Zhang Zhengyou demarcate to determine the transforma- dimensional world into a two-dimensional image by cap- tion relationship between world coordinates and pixel turing an image. In reality, the imaging model establishes a coordinates. projection mapping relationship between three-dimensional q � MKQ. (1) and two-dimensional space. *e coordinate transformation x f 0c is required to convert the world coordinate system’s coor- x x ⎢ ⎥ ⎢ ⎥ ⎡ ⎢ ⎤ ⎥ ⎡ ⎢ ⎤ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ Among them: q � y , M � 0f c , ⎣ ⎦ ⎣ ⎦ dinates to the camera coordinate system’s coordinates. A y y w 001 rigid body transformation is used to convert the world X x f 0c coordinate system to the camera coordinate system. It is x x R t R t ⎡ ⎢ ⎤ ⎥ ⎡ ⎢ ⎤ ⎥ ⎡ ⎢ ⎤ ⎥ 3×3 3×1 ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ 3×3 3×1 ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ K � 􏼢 􏼣, and Q � ⎣ Y ⎦, ⎣ y ⎦ � ⎣ 0f c ⎦􏼢 􏼣 determined by the camera’s external parameters. *e camera y y 0 1 01 W w 001 coordinate system to pixel coordinate system transformation converts three-dimensional coordinates to two-dimensional ⎡ ⎢ ⎤ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ Y ⎥ plane coordinates, as determined by the camera’s internal ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥. ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎣ ⎦ parameters. Although both the pixel and image coordinate Z systems are located on the imaging plane, their origins and units of measurement are distinct. *e origin of the image 3.2. Obstacle Ranging Model. *e range of obstacles is as coordinate system is the point at which the camera’s optical follows (Figure 4). Let f be the focal length of the camera; h axis intersects the imaging plane, which is typically the be the installation height of the camera; μ be the pixel size; z imaging plane’s midpoint. be the camera pitch angle; (x , y ) be the intersection of the 0 0 Suppose the internal parameters matrix is M. Project image plane and the optical axis of the camera; set to (0, 0); Q(X, Y, Z) in the physical world to the image plane q(x, y). (x, y) be the coordinates of intersection points of obstacles By adding a dimension w to q(x, y), which is the expansion and pavement plane set to P; and the horizontal distance d of q(x, y, w), obtain w � Z. Point q is in the form of between the object point and the camera is 4 Journal of Robotics Imaging coordinate have height if d ≠ d + Δd, and static obstacles can be 1 2 system identified by Δl if Δd is known. Additionally, if the obstacle is moving (as illustrated in Figure 6), the Δl can also be used as an obstacle judgment. (x, y) *e verification process has been shown in the paper [6]. Pixel Coordinate system 3.3. Static Obstacle Identification Model. *ere are two types Yw of static obstacles: real static obstacles and static pseudo- (u,v) Zw obstacles. Static real obstacles refer to actual road obstacles. Camera (X,Y,Z) *e reflections identified as real obstacles during the obstacle coordinate system Xw identification process are called static pseudo-obstacles. It is World coordinate Yc a type of pseudo-obstacle that reflects some road obstacles system but does not affect the vehicle’s driving safety. To improve (c ,c ) x y the accuracy of obstacle identification, we must identify and Xc remove static pseudo-obstacles. Figure 3: Schematic diagram of coordinate transformation. 3.3.1. Static Real Obstacle Identification. First, we used the VIDAR to detect stereo obstacles and determine which d � . (2) object point on the obstacle is the furthest away in the tan z + arctan􏼂 y − y􏼁 μ/f􏼃􏼁 horizontal and vertical directions to construct a rectangle (the obstacle rectangle, Figure 7). Let A (x , y ) be the first 1 1 Assume that Y is the Y-axis in the previous image, Y is 1 2 imaging point for the identification of the width of the the Y-axis in the previous image. When the camera moves rectangular road surface of the obstacle, B (x , y ) be the 2 1 from Y to Y on the axis of the imaging plane (Figure 5), let 1 2 other imaging point for the identification of the width of the A be an imaging point for obstacle’s top in the subsequent rectangular road surface of the obstacle, and A be the object image, B be the same imaging point for obstacle’s top in the point of A; similarly, B is the object point of B. *e hor- ′ ′ latter image, A is the object point of A, and B is the object izontal distances d between A and the camera can be ′ 1 point of B. d is the horizontal distance between A and the calculated by Equation (2). Similarly, the horizontal dis- camera; similarly, d is the horizontal distance between B tances d between B and the camera can also be calculated. and the camera. d and d can be obtained from Equation 1 2 *e width of the obstacle rectangle can be calculated using (3). *e camera moved a certain distance Δd during the time the pinhole imaging principle and the geometrical rela- between the previous and subsequent images; d � d + Δd, 1 2 tionship between cameras. ′ ′ but d � d + Δd + Δl actually. As a result, the A and B 1 2 􏽶��������������������������������������������������������� � 􏽱������􏽱������ 2 2 2 2 2 2 2 2 2 􏼐2f + 2y + x + x − μ x − x 􏼁 􏼑 h + d h + d 1 1 2 2 1 1 2 2 2 2 􏽱���������� �􏽱���������� � (3) w � 2h + d + d − . 1 2 2 2 2 2 2 2 f + x + y f + x + y 1 1 2 1 When the camera moves from Y to Y on the axis of the the width of the opposite side of the obstacle, C be the object 1 2 imaging plane (Figure 8), let C (x , y ) be an imaging point point of C, and D be the object point of D. (4) and (5) have 3 2 for identifying the width of the opposite side of the obstacle, the same width. *e height of the pseudo-obstacle rectangle D(x is calculated. , y ) be another imaging point for the identification of 4 2 2 2 2 2 2 2 2 2 2 2 2f + 2y + x + x − μ x − x 2 h − h + d + d − w 􏼁 􏼁 2 3 4 4 3 v 3 4 􏽱���������� �􏽱���������� � 􏽱������������􏽱������������ � . (4) 2 2 2 2 2 2 2 2 2 2 f + x + y f + x + y h − h 􏼁 + d h − h 􏼁 + d 3 2 4 2 v 3 v 4 3.3.2. Static Pseudo-Obstacle Identification. *e procedure pseudo-obstacles using VIDAR and construct a rectangle for identifying pseudo-obstacles is similar to the procedure (pseudo-obstacle rectangle) from the object points on the for identifying real obstacles. However, when obstacles are pseudo-obstacle (the object points are the farthest in the detected using VIDAR, the object points of the obstacles are horizontal and vertical directions). Let A be the first imaging different from their actual positions (Figure 9). We detect point for pseudo-obstacle width identification with (x , y ), 1 1 Focal Length (f) Journal of Robotics 5 (x, y) Len′s Center (x , y ) 0 0 Horizontal Line Image Plane Optic Axis Road Plane Figure 4: Schematic diagram of the horizontal distance between the object point and the camera. Y Y 1 2 Obstacle′s Imaging Point First Imaging Point Δl Δd B′ A′ Figure 5: Static obstacle imaging. Y Y 1 2 Obstacle′s Imaging Point First Imaging Point Δl Δd s A′ B′ Figure 6: Moving obstacle imaging. BA B′ A′ Figure 7: Schematic diagram of static obstacle. hv 6 Journal of Robotics B be the other imaging point for pseudo-obstacle width camera can also be calculated using (2), and the width of the identification with (x , y ), A be the object point of point A, pseudo-obstacle rectangle can be calculated by the pinhole 2 1 and B be the object point of point B. *e horizontal dis- imaging principle and the geometrical relationship between tances d between A and the camera can be calculated using cameras. (2). Similarly, the horizontal distances d between B and the 􏽶��������������������������������������������������������� � 􏽱������􏽱������ 2 2 2 2 2 2 2 2 2 2 􏼐2f + 2y + x + x − μ x − x 􏼁 􏼑 h + d h + d 1 1 2 2 1 1 2 2 2 2 􏽱���������� �􏽱���������� � (5) w � 2h + d + d − . 1 2 2 2 2 2 2 2 f + x + y f + x + y 1 1 2 1 When the camera moves from Y to Y on the axis of the point of D. At this point, the width of the object point of the 1 2 imaging plane (Figure 10), after the pseudo-obstacle moved, pseudo-obstacle changes from W to W . Similarly, W can 1 2 2 let C (x , y ) be an imaging point of the rectangular width of be solved using the pinhole imaging principle and the 3 2 the identified pseudo-obstacle and D (x , y ) be another geometrical relationship between cameras. *e rectangular 4 2 imaging point of the rectangular width of the pseudo-ob- height of the pseudo-obstacle can be obtained by (5) and (6) stacle, and C be the object point of C and D be the object and the triangle similarity principle. ′ ′ 2 2 2 2 2 2 2 2 2 2 2 2f + 2y + x + x − μ x − x 􏼁 2 h − h 􏼁 + d + d − h/h + h 􏼁 w 2 3 4 4 3 v 3 4 v 􏽱���������� �􏽱���������� � 􏽱������������􏽱������������ � . (6) 2 2 2 2 2 2 2 2 2 2 f + x + y f + x + y h − h 􏼁 + d h − h 􏼁 + d v v 3 2 4 2 3 4 3.4. Static Obstacle Identification Model. Moving obstacles are 3.4.1. Static Pseudo-Obstacle Identification. *e steps for classified as either moving real obstacles or moving pseudo- identifying real moving obstacles are identical to those for obstacles. Moving real obstacles refers to obstacles on the road. real static obstacles (Figure 11). VIDAR is used to detect *e reflections identified as real obstacles during the obstacle stereo obstacles, construct obstacle rectangles, and calculate identification process are referred to as moving pseudo-ob- their width. After the ego-vehicle and obstacle have been stacles. It is a type of pseudo-obstacle that replicates some road moved, the width of the obstacle rectangle is recalculated obstacles but does not destroy the vehicle’s driving safety. We and then the obstacle height is solved using the triangle must identify and remove moving pseudo-obstacles to improve similarity principle. accuracy when identifying obstacles. 2 2 2 2 2 2 2 2 2 2 2f + 2y + x + x − μ x − x 􏼁 2 h − h 􏼁 + d + d − w 4 3 v 2 3 4 3 4 􏽱���������� �􏽱���������� � 􏽱������������􏽱������������ � . (7) 2 2 2 2 2 2 2 2 2 2 f + x + y f + x + y h − h 􏼁 + d h − h 􏼁 + d 3 2 4 2 v 3 v 4 3.4.2. Moving Pseudo-Obstacle Identification. *e steps for rectangle, and calculating the pseudo-obstacle rectangle’s identifying moving pseudo-obstacles are identical to those width. Following the ego-vehicle and pseudo-obstacle for static pseudo-obstacles (Figure 12): detecting stereo movement, the height of the pseudo-obstacle is calculated obstacles with VIDAR, determining the pseudo-obstacle using the width of the pseudo-obstacle’s imaging point. 2 2 2 2 2 2 2 2 2 2 2 2f + 2y + x + x − μ x − x 􏼁 2 h − h 􏼁 + d + d − h/h + h 􏼁 w 4 3 v v 2 3 4 3 4 􏽱���������� �􏽱���������� � 􏽱������������􏽱������������ � . (8) 2 2 2 2 2 2 2 2 2 2 f + x + y f + x + y h − h 􏼁 + d h − h 􏼁 + d v v 3 2 4 2 3 4 3.5. Removal Model of Pseudo-Obstacles. *e ego-vehicle vehicle resumes motion, the width and height of the obstacle movement assesses the obstacle’s authenticity. Using (3) and and pseudo-obstacle are also calculated using (4) and (6), (5), the widths of obstacles and pseudo-obstacles when the and the heights of the obstacle and pseudo-obstacle are vehicle moves for the first time are calculated. When the determined by their widths. Compared with the calculated Journal of Robotics 7 Y Y Y Y Y Y 1 1 2 2 D C D C A BA BA X l A 5 6 C B l 6 l l D y D 1X f 2 m 1 X m α 2 D′ 4 h d 3 D′ d Δd d m C′ h 4 3 2 m v h 1 B′ C′ 2 Δd 3 h v W d S A′ Figure 11: Schematic diagram of moving obstacle. Figure 8: Schematic diagram of static pseudo-obstacles after the ego-vehicle moved. D C B A 3 A B l l l 3 5 l 4 BA 1X α 2 f m B′ X Δd 1 h 3 D′ l d 2 4 l d 1 d 3 C′ A′ m Figure 12: Schematic diagram of moving pseudo-obstacle after the ego-vehicle moved. B′ d A′ are the furthest apart in the horizontal and vertical directions (Figure 13 and Figure 14). (3) Calculate the horizontal distance. Determine the Figure 9: Schematic diagram of static pseudo-obstacle. horizontal distance from the object point to the camera according to VIDAR. Y (4) Identify obstacles. First, calculate the width W of the D C obstacle rectangle. Second, determine the relation- BA ship between the height h and width W through the triangle similarity principle. 6 A l y B (5) Calculate the rectangular height of real and pseudo- X β 1 X obstacles by using the same width when vehicles and h m 4 d D′ 4 obstacles move. Calculate the height value twice, Δd C′ compare the two height values, and determine the W identified obstacle. *e overall flow of obstacle identification is shown in Figure 15. Figure 10: Schematic diagram of static pseudo-obstacle after the ego-vehicle moved. 4. Obstacle Identification Experiment and results for obstacles, detected obstacles are those with a Effect Analysis similar height. We analyze the identification effect of the VM and improved *e process of obstacle identification is as follows: VM in two environments. On the movable platform, the (1) Confirm stereo unknown obstacles. Machine experimental equipment, including the camera unit and learning is used to identify known obstacles, obtain IMU, is installed (Figure 16(a)). A scale model of the vehicle images after removing the known obstacles, and then is used to simulate a specific obstacle. To simulate the un- screen out stereo unknown obstacles using VIDAR’s known obstacle, a beverage bottle cap is used (Figure 16(b)). obstacle detection principle. *e polished paper is used to create a reflection of the road (2) Construct an obstacle rectangle. To construct a (Figure 16(c)). *e camera’s video captured at a frame rate of rectangle, locate the object points on the obstacle that 20 fps is utilized to generate an image sequence, and then 8 Journal of Robotics Obstacle Point Rectangle (a) (b) Figure 13: *e first image. (a) *e feature points. (b) Construction of the obstacle rectangle by the feature points. Point Obstacle Rectangle (a) (b) Figure 14: *e second image. (a) *e feature points. (b) Construction of the obstacle rectangle by the feature points. obstacle detection on the generated image sequence is height unchanged is confirmed, and the real obstacles are performed. marked. *e previous image and latter image are used to judge whether the height of the obstacle rectangle has changed 4.1. Improved VIDAR and Improved VM Simulation (Figure 17). Experiments. A beverage bottle cap is used as an unknown In the VM and improved VM comparison experiments, obstacle, and the angular acceleration and acceleration of the the faster RCNN is used to identify known obstacles and to ego-vehicle are obtained from the IMU installed in the ego- identify known obstacles as background, while VIDAR and vehicle. *e quaternion method is used to solve the camera improved VIDAR are used to perform secondary detection attitude and update the camera pitch angle. *e image is on the background-removed image to identify unknown processed by a fast image region matching method based on obstacles (Figure 18, Figure 19, and Figure 20). MSER. Acceleration is used to calculate the horizontal *e detection of unknown obstacles in this paper is distance between the vehicle and the obstacles. *e height of shown in Figure 19 and Figure 20. While VIDAR in the VM the obstacle rectangle by keeping the actual width constant is capable of identifying the bottle cap, the cap’s reflection is during the vehicles and obstacles movement is calculated, also detected as an obstacle, resulting in low obstacle the authenticity of the identified obstacles by keeping the identification accuracy. When the improved VIDAR is used Journal of Robotics 9 Machine learning at Background Feature Points Non-road Construct Obstacle t to detect known ROI Extraction extraction Extraction Obstacle Screening Rectangle obstacles Inertial Data Camera Data t=0 Acquisition Update W at t Camera Data at t Δd Calculation t=t+1/f h and h at Feature Points v1 v2 t=t+2Δt Camera Moving t=t+Δt Camera Moving t+Δt Tracking Feature Points h and h at N Y v3 v4 h = h ? h = h ? Pseudo Obstacle v1 v3 v2 v4 Tracking t+2Δt Real Obstacle Figure 15: Improved VM pseudo-obstacle identification flowchart. IMU Movable Platform Camera Unit (a) (b) (c) Figure 16: Experimental equipment. (a) Mobile platform, camera unit, IMU. (b) Vehicle scaled model (known obstacle) and unknown obstacle model. (c) Polished paper simulates reflection road. car car Obstacle Figure 17: Detection result of bottle cap. Figure 18: Faster RCNN identification effect diagram. to detect unknown obstacles, the obstacles in the reflection without height can be eliminated, compensating for the MV-VDF300SC industrial digital camera is used as a unknown obstacles being misdetected in the reflective en- vironment. As a result, the improved VM detects obstacles monocular vision sensor. *is model camera adopts the USB 2.0 standard interface and has a high resolution, precision, more precisely than the baseline VM. and clarity. *e camera’s performance parameters are listed in Table 1. *e camera is installed at the height of 1.60 m and 4.2. Analysis of the Identification Result of Improved VM and collects real-time environmental data (we only used the left Improved VIDAR. In the experimental test, a pure electric camera). *e HEC295 IMU is mounted on the bottom of the vehicle is used as the test vehicle (Figure 21). A test vehicle and is used to locate and read the vehicle’s е 10 Journal of Robotics car car Obstacle Obstacle Figure 19: VM identification effect diagram. car car Obstacle Figure 20: Improved VM identification effect diagram. Camera Computing processing unit and digital map GPS+IMU Figure 21: Schematic diagram of the test vehicle. Table 1: Some performance parameters of the MV-VDF300SC camera. MV-VDF300SC Highest resolution 2048 ∗1536 Power requirements (V) 5 Output color Color Power consumption (W) Rated <5 Frame rate (fps) 12 Operating temperature ( C) 0–60 Output method USB 2.0 Dimensions (mm) 43.3 ∗ 29 ∗ 29 motion status in real time. GPS is used to determine a precise to perform real-time data processing. In the process of location. Digital maps are utilized to obtain precise road calculation, multisensor data processing is the combination data, such as distance and slope. *e computing unit is used and processing of multisource information, which is rather Journal of Robotics 11 0.35 Drag to select outliers 0.3 -400 0.25 -200 0.2 0.15 0.1 0.05 1000 0 Z (millimeters) 0 -500 0 5 10 15 20 X (millimeters) Images Overall Mean Error: 0.25 points Figure 22: *e camera calibration result. *e calibration of the camera’s external parameters can Table 2: Calibration results of camera external parameters. be calculated by taking the edge object points of lane lines. External parameter type Parameter size *e calibration results are shown in Table 2. Pitch angle 1.25 Due to a lack of reflection road images in the public data Yaw angle 3.65 set and the fact that different camera parameters would affect Rotation angle 2.45 range accuracy, we created a VIDAR-Reflection Road da- tabase (Figure 23) with a total of 2000 images. *e MV- VDF300SC camera unit was used to record the experiment complicated. Fuzzy logic can deal with complex systems in its natural environment. *e test roads were Xuezhai [29]. It can coordinate and combine the acquired infor- Road and Jiefang East Road in Jinan, Shandong Province, mation to improve the efficiency of the system and effectively and traffic environment images were collected during rainy deal with the knowledge acquired in the scene. days from 10 : 00 to 11 : 00 and 19 : 00 to 20 : 00. Accurate calibration of camera parameters was a pre- Figure 24 depicts the identification result for the two requisite for the whole experiment and is a very important images. *e VM and improved VM accuracy are compared task for obstacle detection methods. In this paper, Zhang by counting the number of TP, FP, TN, and FN obstacles in Zhengyou’s camera calibration method was adopted to each image frame. Let a be an obstacle that is correctly calibrate the DaYing camera. First, the camera was fixed to identified as a positive example; b be an obstacle that is capture images of a checkerboard at different positions and incorrectly identified as a positive example; c be an obstacle angles. *en, key points of the checkerboard were selected that is correctly identified as a negative example; and d be an and used to establish a relationship equation. Finally, the obstacle that is incorrectly identified as a negative example. internal parameter calibration was realized. *e camera n n n *en, TP � 􏽐 a , FP � 􏽐 b , TN � 􏽐 c , and i i i i�1 i�1 i�1 calibration process and result are shown in Figure 22. FN � d . *e comparison of the detection effects of VM i�1 i Camera distortion includes radial distortion, thin lens and improved VM in the reflection environment is shown in distortion, and centrifugal distortion. *e superposition of Table 3. the three kinds of distortion results in a nonlinear distortion, In the results’ analysis, accuracy (A), recall (R), and the model of which can be expressed in the image coordinate precision (P) were used as evaluation indices for the two system as follows: obstacle detection methods, calculated through 2 2 3 2 2 ⎧ ⎨δ (x, y) � s x􏼐x + y 􏼑 + 2p xy + p y + k x􏼐x + y 􏼑 x 1 1 2 1 TP + TN ⎩ 2 2 3 2 2 (11) A � , δ (x, y) � s y x + y + 2p xy + p y + k x x + y 􏼐 􏼑 􏼐 􏼑 y 2 2 1 1 TP + TN + FP + FN (9) TP R � , (12) where s and s are the centrifugal distortion coefficients; k TP + FN 1 2 1 and k are the radial distortion coefficients, and p and p 2 1 2 TP are the distortion coefficients of thin lenses. (13) P � . Because the centrifugal distortion of the camera is not TP + FP considered in this paper, the internal reference matrix of the *e accuracy, recall, and precision of the method pro- camera can be expressed as shown in posed in this paper are shown in Table 4. As demonstrated by the experimental results in Table 4, 5.9774e + 03 0 949.8843 ⎢ ⎥ ⎡ ⎢ ⎤ ⎥ ⎢ ⎥ ⎢ ⎥ the accuracy of obstacle identification is increased when ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ M � ⎢ 0 5.9880e + 03 357.0539 ⎥. (10) ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎣ ⎦ using the improved VM for obstacle identification in a 0 0 1 reflective environment. Due to the weather and other fac- tors, there are times when misidentification and missed Mean Error in Points Y (millimeters) 12 Journal of Robotics Figure 23: A partial sample of the VIDAR-Reflection Road data set. Figure 24: Comparison of detection results of VM and improved VM in an environment with reflection. identification occur during the experiment. However, the *e term “real time” refers to the processing of each improved method proposed in this paper improves obstacle image frame collected over time. In terms of detection speed, identification accuracy. 2000 images were processed using improved VIDAR, VM, Additionally, we compared our method’s detection ac- VIDAR, VM, and YOLO v5. Table 6 summarizes the average curacy to other commonly used target detection methods. detection times for the five identification methods. Table 5 summarizes the detection results. It is obvious that As shown in Table 6, an improved VM takes longer to the proposed obstacle detection method outperforms state- determine the authenticity of obstacles than a VM. Similarly, of-the-art methods in terms of accuracy. improved VIDAR requires more time to determine the Journal of Robotics 13 Table 3: Comparison of the detection effect of VM and improved VM in an environment with reflection. Input TP FP TN FN VM 7356 6591 365 279 392 Improved VM 7356 6832 224 252 243 Table 4: Evaluation indices of the proposed method. Detection method A (%) R (%) P (%) VM 90.05 94.38 94.63 Improved VM 93.81 96.56 96.82 Table 5: Evaluation indices of advanced target detection methods and the proposed method. Detection method A (%) Fast RCNN 76.53 SSD 71.28 Fast YOLO 78.95 SSD 300 73.96 YOLO v5s 81.73 Proposed method 90.36 Table 6: Evaluation indices of detections of YOLO v5s and proposed method. Improved VIDAR VIDAR Improved VM VM YOLO v5 Identification time (s) 0.627 0.538 0.372 0.296 0.273 needs a lot of calculations, improving the efficiency of the authenticity of obstacles when compared with VIDAR. Due to the fewer feature points, improved VM detects faster than proposed method will be the next research direction. In improved VIDAR, and less time is required. As a result, addition, obstacle detection is a prerequisite for obstacle using an improved VM for obstacle detection takes not only avoidance, and an improved obstacle avoidance method is advantage of machine learning’s speed but also improves also a future research direction. identification accuracy. Data Availability 5. Conclusion Data are available on request to the corresponding author. *is paper first proposes an improved VIDAR method based Conflicts of Interest on VIDAR and then combines machine learning to propose an improved method for VM obstacle identification. On the *e authors declare that they have no conflicts of interest. basis of machine learning to detect known obstacles, VIDAR is used to determine whether there is an obstacle with height Acknowledgments by calculating the position of road imaging points, the obstacle rectangle is determined for nonroad obstacles, and *is work was supported in part by the National Natural then the obstacle height (including real obstacles and Science Foundation of China under Grant 51905320, the pseudo-obstacles) is calculated by using the obstacle imaging China Postdoctoral Science Foundation under Grants points of two frames before and after the vehicle moves. By 2018M632696 and 2018M642684, the Shandong Key R and calculating the height after moving again (including real D Plan Project under Grant 2019GGX104066, and SDUT obstacles and pseudo-obstacles), the two heights are com- and Zibo City Integration Development Project under Grant pared to determine the authenticity of the obstacle, so as to 2017ZBXC133. realize the obstacle detection. *is paper aims to show the effect of obstacle detection using improved VM in the en- References vironment with reflection. *e experimental results indicate that when compared with VM, the improved VM method [1] D. Gusland, B. Torvik, E. Finden, F. Gulbrandsen, and for obstacle detection is more accurate in a reflective en- R. Smestad, “Imaging radar for navigation and surveillance on vironment. Because the method proposed in this paper an autonomous unmanned ground vehicle capable of 14 Journal of Robotics identifying obstacles obscured by vegetation,” in Proceedings [17] K. H. Lin, H. M. Zhao, J. J. Lv et al., “Face identification and of the 2019 IEEE Radar Conference (RadarConf), pp. 1–6, segmentation based on improved mask R-CNN,” Discrete Movings Boston, MA, USA, 2019. in Nature and Society, vol. 2020, Article ID 9242917, 2020. [18] Y. Tian, G. Yang, Z. Wang, E. Li, and Z. Liang, “Instance seg- [2] M. P. Muresan, S. Nedevschi, and I. Giosan, “Real-time object identification using a sparse 4-layer LIDAR,” in Proceedings of mentation of apple flowers using the improved mask R-CNN the 2017 13th IEEE International Conference on Intelligent model,” Biosystems Engineering, vol. 193, pp. 264–278, 2020. Computer Communication and Processing (ICCP), pp. 317–322, [19] X. X. Zhang and X. Zhu, “Moving vehicle identification in Cluj-Napoca, 2017. aerial infrared image sequences via fast image registration and [3] M. Cho, “A study on the obstacle identification for autono- improved YOLOv3 network,” International Journal of Remote mous driving RC car using LiDAR and thermal infrared Sensing, vol. 11, no. 41, pp. 4312–4335, 2020. camera,” in Proceedings of the 2019 Eleventh International [20] T. Haas, C. Schubert, M. Eickhoff, and H. Pfeifer, “BubCNN: Conference on Ubiquitous and Future Networks (ICUFN), bubble identification using Faster RCNN and shape regres- pp. 544–546, Zagreb, Croatia, 2019. sion network,” Chemical Engineering Science, vol. 216, Article [4] S. Wang and X. Li, “A real-time monocular vision-based ID 115467, 2020. obstacle identification,” in Proceedings of the 2020 6th In- [21] M. Zhang, W. Zhao, and X. Li, “Shadow detection of moving ternational Conference on Control, Automation and Robotics objects in traffic monitoring video,”vol. 9, pp. 1983–1987, in Proceedings of the 2020 IEEE 9th Joint International Infor- (ICCAR), pp. 695–699, Singapore, 2020. [5] W. Song, Y. Yang, M. Fu, Y. Li, and M. Wang, “Lane iden- mation Technology and Artificial Intelligence Conference tification and classification for forward collision warning (ITAIC), vol. 9, IEEE, Chongqing, China, December 2020. system based on stereo vision,” IEEE Sensors Journal, vol. 12, [22] D. J. R. Del Carmen and R. D. Cajote, “Assessment of vision- pp. 5151–5163, 2018. based vehicle tracking for traffic monitoring applications,” in [6] X. Yi, G. Song, and T. Derong, “Fast road obstacle identifi- Proceedings of the 2018 Asia-Pacific Signal and Information cation method based on maximally stable extremal regions,” Processing Association Annual Summit and Conference International Journal of Advanced Robotic Systems, vol. 15, (APSIPA ASC), pp. 2014–2021, IEEE, Honolulu, Hawai, no. 1, Article ID 1729881418759118, 2018. November 2018. [7] A. Sharif Razavian, H. Azizpour, and J. Sullivan, “CNN [23] S. H. Shaikh, K. Saeed, and N. Chaki, “Moving object de- features off-the-shelf: an astounding baseline for recognition,” tection using background subtraction,” Moving Object De- Proceedings of the IEEE conference on computer vision and tection Using Background Subtraction, Springer, Cham, pattern recognition workshops, pp. 806–813, 2014. pp. 15–23, 2014. [24] R. Chabra, J. E. Lenssen, E. Ilg et al., “Deep local shapes: learning [8] S. Y. Wang, O. Wang, and R. Zhang, “Cnn-generated images are surprisingly easy to spot for now,” in Proceedings of the local sdf priors for detailed 3d reconstruction,” in Computer IEEE/CVF conference on computer vision and pattern recog- Vision - ECCV 2020, pp. 608–625, Springer, Cham, 2020. nition, pp. 8695–8704, Seattle, WA, USA, June 2020. [25] J. Park and B. C. Song, “Night-time vehicle identification [9] P. Arena, M. Bucolo, S. Fazzino, L. Fortuna, and M. Frasca, using low exposure video enhancement and lamp identifi- “*e CNN paradigm: shapes and complexity,” International cation,” in Proceedings of the 2016 International Conference on Journal of Bifurcation and Chaos, vol. 15, no. 07, pp. 2063– Electronics, Information, and Communications (ICEIC), 2090, 2005. pp. 1-2, Da Nang, Danang, Vietnam, January 2016. [10] A. Shustanov and P. Yakimov, “CNN design for real-time [26] H. P. Lin, P. H. Liao, and Y. L. Chang, “Long-distance vehicle traffic sign recognition,” Procedia Engineering, vol. 201, identification algorithm at night for driving assistance,” in pp. 718–725, 2017. Proceedings of the 3rd IEEE International Conference on In- telligent Transportation Engineering, pp. 296–300, Singapore, [11] X. Zhu, Z. Li, X.-Y. Zhang, P. Li, Z. Xue, and L. Wang, “Deep convolutional representations and kernel extreme learning September 2018. machines for image classification,” Multimedia Tools and [27] X. Dai, D. Liu, L. Yang, and Y. Liu, “Research on headlight Applications, vol. 78, no. 20, pp. 29271–29290, 2019. technology of night vehicle intelligent identification based on [12] K. Wang, F. Yan, B. Zou, L. Tang, Q. Yuan, and C. Lv, Hough transform,” in Proceedings of the 2019 International “Occlusion-free road segmentation leveraging semantics for Conference on Intelligent Transportation, Big Data & Smart autonomous vehicles,” Sensors, vol. 19, no. 21, p. 4711, 2019. City (ICITBS), pp. 49–52, Changsha, China, 2019. [13] G. D. Guo and N. Zhang, “A survey on deep learning based [28] T. S. Kavya, E. Tsogtbaatar, Y. Jang, and S. Cho, “Night-time face identification,” Computer Vision and Image Under- vehicle identification based on brake/tail light color,” in standing, vol. 189, Article ID 102805, 2019. Proceedings of the 2018 International SoC Design Conference [14] W. K. Jia, Y. Y. Tian, R. Luo, Z. H. Zhang, J. Lian, and (ISOCC),, pp. 206-207, Daegu, Korea (South), 2018. Y. J. Zheng, “Identification and segmentation of overlapped [29] M. Bucolo, L. Fortuna, and M. LaRosa, “Complex dynamics fruits based on optimized mask R-CNN application in apple through fuzzy chains,” IEEE Transactions on Fuzzy Systems, vol. 12, no. 3, pp. 289–295, 2004. harvesting robot,” Computers and Electronics in Agriculture, vol. 172, Article ID 105380, 2020. [15] S. H. Wang, J. D. Sun, I. Mehmood, C. C. Pan, Y. Chen, and Y. D. Zhang, “Cerebral micro-bleeding identification based on a nine-layer convolutional neural network with stochastic pooling,” Concurrency and Computation: Practice and Ex- perience, vol. 1, no. 2, Article ID e5130, 2020. [16] B. Sariturk, B. Bayram, Z. Duran, and D. Z. Seker, “Feature extraction from satellite images using segnet and fully con- volutional networks (FCN),” International Journal of Elec- tronic Governance, vol. 3, no. 5, pp. 138–143, 2020.

Journal

Journal of Robotics – Hindawi Publishing Corporation

Published: Mar 14, 2022

There are no references for this article.