Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Monocular Vision-Based Robot Localization and Target Tracking

Monocular Vision-Based Robot Localization and Target Tracking Hindawi Publishing Corporation Journal of Robotics Volume 2011, Article ID 548042, 12 pages doi:10.1155/2011/548042 Research Article Bing-Fei Wu, Wang-Chuan Lu, and Cheng-Lung Jen Department of Electrical Engineering, National Chiao Tung University, 1001 University Road, Hsinchu 300, Taiwan Correspondence should be addressed to Cheng-Lung Jen, cljen0130@gmail.com Received 14 June 2011; Revised 15 October 2011; Accepted 16 October 2011 Academic Editor: Huosheng Hu Copyright © 2011 Bing-Fei Wu et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. This paper presents a vision-based technology for localizing targets in 3D environment. It is achieved by the combination of different types of sensors including optical wheel encoders, an electrical compass, and visual observations with a single camera. Based on the robot motion model and image sequences, extended Kalman filter is applied to estimate target locations and the robot pose simultaneously. The proposed localization system is applicable in practice because it is not necessary to have the initializing setting regarding starting the system from artificial landmarks of known size. The technique is especially suitable for navigation and target tracing for an indoor robot and has a high potential extension to surveillance and monitoring for Unmanned Aerial Vehicles with aerial odometry sensors. The experimental results present “cm” level accuracy of the localization of the targets in indoor environment under a high-speed robot movement. 1. Introduction vehicles use the Global Position System (GPS) and Inertial Measurement Units (IMUs) for land vehicle applications Knowledge about the environment is a critical issue for [5]. However, most of inertial navigation system sensors are autonomous vehicle operations. The capability of localizing expensive for some applications in the indoor environment. targets with a robot for environment is highly demanded. Optical wheel encoders and an electrical compass provide In [1], it investigates the vision-based object recognition linear and angular velocities respectively. Both of two sensors technique for detecting targets of the environment which are basic odometry sensors and widely used owing to their have to be reached by a robot. After the robot finishes the low cost, simplicity, and easy maintenance. Encoders provide target detection, it is an essential task for robots to know the a way of measuring the velocity to estimate the position positions of targets. The tasks include navigation and object of the robot and compasses are often used to detect the tracking. As a result, the approach to localize targets for orientations of the robot. Based on the sensor information, specific tasks is an essential issue in some applications. For motion control is done and then the localization of the robot UAVs, there are some similar demands [2, 3] that UAVs have is estimated [6, 7]. Despite some limitations of an encoder, to track the positions of ground objects for reconnaissance most researchers agree that an encoder is a vital part of the or rescue assistance with monocular vision. Besides, UAVs robot’s navigation system and the tasks will be simplified need to observe the changing surroundings to understand if the encoder accuracy is improved. Besides, the additional the movements of the aircraft better, or the localizing system has to direct the aircraft to a region of interest after taking sensor, a camera, allows a robot to perform a variety of tasks ground observations. autonomously. The use of computer vision for localization In recent years, odometry sensors have been widely used has been investigated for several decades. The camera has for estimating the motion of vehicles moving in a 3D space not been at the center of robot localization while most of environment such as UAVs and Unmanned Ground Vehicles researchers have more attention to other sensors such as laser (UGVs). For instance, Inertial Navigation Systems (INSs) range-finders and sonar. However, it is surprising that vision are applied to measure linear acceleration and rotational is still an attractive choice for sensors because cameras are velocity, and capable of tracking the position, velocity and compact, cheaper, well understood, and ubiquitous. In this attitude of a vehicle by integrating these signals [4]ormobile paper, the algorithm is achieved by combining different types 2 Journal of Robotics of sensors including optical wheel encoders, an electrical compass, and a single camera. Mobile vehicles such as robots, UGVs, and UAVs are becoming fully intelligent robotic systems which can handle more complicated tasks and environments. It is necessary to provide them with higher autonomy and better function- w w w 1/ρ m(θ , ϕ ) i i i alities. For instance, robot localization and target tracking are mandatory. For these tasks, some of sensors such as GPSs, IMUs, laser range-finders, encoders, and compasses are possibly combined to describe the time evolution of w θ G the motion model. Additionally, visual sensors, such as electro-optic or infrared cameras, have been included in the on-boarded sensor suit of many robots to increase their Parallax value. Therefore, it is a popular research topic regarding w w how to combine hybrid sensors to achieve a special task. r − r Recent researches have shown that the way to improve sequential estimation and achieve repeatable motion with multisensors is to adopt the probabilistic method. The Kalman filter has been widely used for data fusion such as w w Figure 1: The robot moves from r to r .Based on thetwo navigation systems [8], virtual environment tracking systems i k views, the parallax is produced. The initial depth can be refined [9], and 3D scene modeling [10]. The Kalmna filter is a to approach the real distance and the position of the ith target is mathematical method to provide an iterated procedure for estimated. a least-square estimation of a linear system. The procedure includes predicting state by using the motion model and then correcting the state by the measurement innovation. the initial estimate for the unknown information while The EKF is a varied type of the Kalman filter for a non- simultaneously updating the localization of targets and the linear system such as the target tracking system in this paper. robot pose. Finally, Chelesky decomposition and forward- More details about the Kalman filter are introduced in the and-back substitution are presented to calculate the inverse material of [11–14]. In this paper, the algorithm represents covariance matrix efficiently. a new approach of EKF-based multisensor data fusion. The information from different types of sensors is combined to 2.1. The Origins of the Proposed Algorithm. A single camera improve the estimations in the system state. In order to is mounted on our system as one of the sensors. The make oursystemabletodealwiththe varietyoftasks such monocular vision infers that the depth of the target is not as navigation for a mobile robot or ground target tracking able to be measured by only one image but estimated by for UAVs, we are determined to choose monocular vision the sequential images. Therefore, the single camera has to rather than binocular vision as one type of our sensors. Based estimate the depth by observing the target repeatedly to get on the monocular vision, simultaneous robot localization parallax between different captured rays from the target to and target tracking become more complex with a higher the robot. The orientations of target are estimated in the computational loading due to unknown depths of targets in world coordinate system only by one image. Y is a six- only one image observation. Thus, it is an important issue i dimension state vector and used to describe the position of regarding how to make use of computational tricks or adopt the ith target in 3D space. Its equation is addressed as more efficient algorithms to reduce the running time. In this paper, Chelesky decomposition [15] and forward-and- w w w w w back substitution are used to invert the covariance matrix for Y = r θ ϕ ρ . (1) i i i i i improving the computational efficiency because the property of covariance matrix is semipositive definite. r is the location of the robot when the robot detects the The rest of this paper is structured as follows. Section 2 ith target in the first time. G is defined as the position of presents the proposed algorithm and explains the details. the target with respect to the camera coordinate system and Section 3 explains two key points about the proposed algo- denoted by rithm. The experimental results are provided in Section 4, and finally, Section 5 concludes the paper and suggests future 1 c cw w w w w G = R r − r + m θ , ϕ . (2) i i k w i i work. w w θ and ϕ are the orientations of the ith target and calculated i i 2. Explanation of the Algorithm by the pixel location of the target in only one image. The The algorithm is mainly divided into five parts including relationship between the position of the target G and the motion modeling, new target adding, measurement mod- current robot localization r is presented in Figure 1. eling, image matching, and EKF updating. By sequential G is the position of the target with respect to the world w w measurement from sensors, the EKF is capable of improving frame system. 1/ρ is defined as the distance from r to i i Journal of Robotics 3 w w the target and m(θ , ϕ ) is its unit vector. Consequently, where i i w w w w 1/ρ m(θ , ϕ ) is seen as the vector from r to the target ⎡ ⎤ i i i i Δθ w w w w w w and then G is calculated as ((r − r )+1/ρ m(θ , ϕ )). i i i i i cos k ⎡ ⎤ ⎢ ⎥ c w 2 q ⎢ ⎥ Finally, G is estimated if the depth 1/ρ is known in advance. 1 i i ⎢ ⎥ ⎢ ⎥ Δθ ⎢ ⎥ ⎢ ⎥ However, the depth of the target is not able to be measured by u sin ⎢ ⎥ ⎢ ⎥ w w ⎢ ⎥ c c 2 ⎢ ⎥ a single image. The EKF is applied to estimate 1/ρ , θ ,and q ω + Ω = = ⎢  ⎥ . (6) i i k k ⎢ ⎥ ⎢ ⎥ Δθ w k ⎢ q ⎥ 3 ⎢ ⎥ ϕ and they converge toward more correct values under the u sin i ⎣ ⎦ y ⎢ ⎥ ⎢ ⎥ recursive iteration by using sequential image measurement 4 ⎣ ⎦ Δθ information. To sum up, (2) is the basic concept and origin u sin of our proposed algorithm. c c u u u x y z u = [ ] is equal to ω Δt/ω Δt. k+1 k+1 2.2. Motion Modeling. We first derive the continuous-time The covariance is predicted and modified if considering c w system model that describes the time evolution in the state the control covariance Cov(v , ω , a ). We assume that the k k k estimate. This model allows us to employ the sampled process noise of the control vector is not correlated with w c w measurements of the wheel encoder and the compass for the process state vector Xm so that Cov(Xm , v , ω , a ) k k k k k state propagation. The process state is described by the vector is set to be a zero matrix. By using (A.4)in Appendix A, the predicted covariance of the system model is expressed as T T T T w w w c Xm = r q v ω ,(3) k k k k − T Cov Xm = F Cov(Xm )F + Q,(7) Xm k Xm k+1 where r is the position of the robot with respect to the world where frame. q is the quaternion that represents the orientation of w T c T w T the world frame for the robot. The linear velocity in the world Q = F Cov v F + F Cov ω F + F Cov a F . v v ω ω a a k k k frame and the angular velocity with respect to the camera (8) w c frame are denoted by v and ω , respectively. In this system, k k the control inputs are measured by wheel encoders and the T The first item F Cov(Xm )F represents the noise from Xm Xm compass which provide the linear velocity and the angular the process state vector and Q describes the noise from the velocity, respectively. Besides, the linear velocity is used to measurement of the wheel encoders and the compass. The simulate the acceleration data of the robot for describing how Jacobian matrix of F is shown as Xm components of the system state vector change completely. w w The definition of acceleration is defined as a = (v − ∂f ∂f ∂f ∂f k k ∂f F = = . (9) Xm w w w c v )/Δt. In order to have similar on-line simulation, we k−1 ∂r ∂q ∂v ∂ω ∂Xm k k k k w 13 × 13 w w w w define a as (v − v )/Δt rather than (v − v )/Δt. k k k−1 k+1 k Although the former definition is not better than the later The Jacobian matrixes of F and F are as the same as ∂f/∂v v ω one which is closer to the real acceleration, an iterated EKF and ∂f/∂ω ,respectively. is still capable of compensating for errors when adopting w w (v − v )/Δt as the robot’s acceleration. The system model k k−1 2.3. Adding New Target. It is remarkable about our algorithm describing the time evolution of the wheel encoders and of that new targets are initialized by using only one image. the compass is given by The initialization includes the state initial values and the covariance assignment. ⎡ ⎤ ⎡ ⎤ w w w 2 r + v Δt + a Δt k+1 k k k ⎢ ⎥ ⎢ ⎥ ⎢   ⎥ ⎢ ⎥ 2.3.1. Target Initialization. Equation (1) is used to define the ⎢ ⎥ w c c ⎢ q ⎥ k+1 ⎢ q × q ω + Ω ⎥ ⎢ ⎥ k k Xm = = ⎢ ⎥ . (4) location of the new target and estimate the six parameters of k+1 ⎢ ⎥ ⎢ ⎥ w w ⎢ v ⎥ w ⎢ ⎥ k+1 v + a Δt Y with the EKF prediction-update loop. If the robot senses ⎣ ⎦ k k i ⎣ ⎦ the 1st target at the state k for the first time, the new target ω c c k+1 ω + Ω k k information is added and then the process state vector is modified. The expanded process state vector is represented The compass is used to measure Δθ which is defined as by the following equation: c c (θ − θ ). ω and ω aredefinedas(θ − θ )/Δt and k k−1 k k−1 k+1 k c c ⎡ ⎤ (θ − θ )/Δt,respectively. If ω is assumed to be (ω + − k−2 k−1 k+1 k Xm c c w k ⎣ ⎦ Ω ), Ω c is derived as (θ − 2θ +θ )/Δt. q is denoted k−2 k k k k−1 X = , (10) k+1 by 1 ⎡ ⎤ ⎡ ⎤ where q −q −q −q q 1 2 3 4 1 ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ w wT w w w q q −q q q ⎢ 2 1 3 ⎥ ⎢ ⎥ 2 Y = r θ ϕ ρ . (11) 1 1 1 1 1 ⎢ ⎥ ⎢ ⎥ 6 × 1 q = ,(5) k+1 ⎢ ⎥ ⎢ ⎥ ⎢ q q q −q ⎥ ⎢ q ⎥ 3 4 1 3 ⎣ ⎦ ⎣ ⎦ Thefirsttimeobservation of the 1sttargetisdoneatthe w w w q −q q q q 4 2 1 3 4 current camera location r . m(θ , ϕ ) is defined as an unit 1 1 1 4 Journal of Robotics w w The unit vector m(θ , ϕ )isderived as afunctionofthe 1 1 orientations of the target and described as m(θ , ϕ ) ⎡ ⎤ 1 1 w w cos ϕ sin θ 1 1 ⎢ ⎥ ⎢ w ⎥ w w − sin ϕ m θ , ϕ = ⎢ ⎥ . (16) 1 1 1 ϕ ⎣ ⎦ w w cos ϕ cos θ 1 1 y(1) X Thefinalparameter 1/ρ is not able to be measured by only one image but estimated by the sequential images with EKF prediction-update loop. As a result, it is assumed to be an initial value 1/ρ . W 0 Figure 2: The orientations of the target with respect to the world 2.3.2. New Covariance Assignment. The system covariance frame can be estimated as a function of its undistorted pixel location is modified after adding a new target. By using (B.1)in h . Appendix B, the new covariance Cov(X ) and the function of Y are denoted by ⎡ ⎤ vector from the location r to the 1sttargetwithrespect to Cov(Xm ) B ⎣ ⎦ the world frame. Figure 2 illustrates the relationship about Cov X = , T B A w w w w 1 1 uv (17) m(θ , ϕ ), θ ,and ϕ . h = [ ] is defined as the pixel 1 1 1 1 location of the 1st target in an undistorted image. g is the w w w w Y = k r , q , h (u, v), ρ , 1 k k 1 location of the 1st target with respect to the camera frame. The location of the 1st target with respect to the world frame where g is addressed as ⎡ ⎤ ⎡ ⎤ r ⎡ ⎤ (u − u)D 0 x ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ x r ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎡ ⎤ w w wc c wc c ⎢ ⎥ (v − v)D w g = ⎢ g ⎥ = R g = R ⎢ 0 y ⎥ g , (12) z r ⎢ r ⎥ ⎢ ⎥ 1 ⎣ ⎦ ⎢ ⎥ ⎢ ⎥ ⎣ ⎦ w ⎢ ⎥ ⎢ ⎥ g w w w z ⎢ ⎥ θ g g ⎢ ⎥ 1 x z 1 ⎢ −1 ⎥ ⎢ ⎥ tan , k = = ⎢ ⎥ . (18) c c ⎢ ⎥ ⎢ g g ⎥ z z ⎢ ϕ ⎥ 1 ⎢ ⎥ ⎣ ⎦ ⎢  ⎥ where ⎛ ⎞ ⎢   ⎥ w 2 2 1 ⎢ w w ⎥ −g g g wc ⎢ x z ⎥ −1 ⎝ ⎠ ⎢ tan , + ⎥ c c c ⎢ ⎥ g g g z z z ⎣ ⎦ ⎡     ⎤ 2 2 2 q +q − q − q 2 q q − q q 2 q q + q q 2 3 1 4 2 4 1 3 1 3 4 2 ρ ⎢ ⎥ ⎢ ⎥ 2 2 2 2 2 q q + q q q −q + q − q 2 q q − q q = ⎢ ⎥ . 2 3 1 4 1 3 4 3 4 1 2 ⎣ ⎦ B is not a zero matrix because the new target’s location is 2 2 2 2 q q − q q 2 q q + q q q −q − q + q 2 4 1 3 3 4 1 2 1 3 4 correlated with the estimates of the robot. According to (B.4) (13) and (B.5)in Appendix B, B and A are derived as 1 1 Then we get B = k Cov(Xm ) = k k 0 0 , (19) r q 6 × 3 6 × 3 1 x k ⎡ ⎤ (u − u)D 0 x g ⎢ ⎥ T T ⎢ ⎥ A = k Cov(r)k + k Cov q k + a, (20) g 1 r r q q z ⎢ ⎥ wc (v − v)D 0 y = R ⎢ ⎥ . (14) ⎢ ⎥ ⎣ ⎦ where T T a = k Cov(h )k + k Cov ρ k . (21) h d h ρ ρ d d w c w c w w Only g /g and g /g rather than g and g are computed x z y z x y by using h .Itisimpossibletoknow g by only one image. 2.4. Measurement Modeling. The robot moves continuously w c w c However, it is possible to make use of g /g and g /g to x z y z and records the sequential images. This is a process of w w calculate θ and ϕ which are shown as 1 1 detecting and identifying the targets. The parameters of ⎡  ⎤ targets have been set into the process state vector and the w w g g x z −1 tan , covariance has been estimated with the recursive loop. ⎢ ⎥ ⎡ ⎤ c c ⎢ ⎥ g g w z z ⎢ ⎥ ⎢ ⎥ ⎣ ⎦ = ⎢ ⎞ ⎥ . (15) w 2 2 2.4.1. Sensor Model. The predicted pixel locations of targets ⎢ w ⎥ w w ϕ −g 1 g g ⎢ y  ⎥ x z −1 ⎣  ⎦ are estimated as a function of the prior process state vector tan , + c c c g g g z z z which is described in (4). According to (2), the location of Journal of Robotics 5 the ith target in the camera frame can be defined in another The actual image target the robot takes is the distorted way: pixel location rather than the undistorted one. Therefore, we have to calculate Zd to get the measurement innovation k(i) ⎡ ⎤ for EKF updating. x(i) ⎢ ⎥ ⎢ ⎥ c cw w w w w w T = ⎢ T ⎥ = R r − r ∗ ρ + m θ , ϕ . y(i) i i k i i i ⎣ ⎦ 2.4.2. Measurement Covariance Assignment. The measure- z(i) ment covariance is expected to describe what the likely (22) variation of measurement is by the sensor under the current condition. The variation is infected by the variables in the Our measurement sensor of the camera is monocular vision. process state vector and the noise which corrupts the sensor. There is a camera sensor model to describe how the sensor The measurement Jacobian matrix H is maps the variables in the process state vector into the sensor variables. By using the pinhole model [16], T is derived as a function of the undistorted pixel location of the ith target ∂h ∂h d(i) d(i) H = 0 ··· 0 ··· 0 , i 2 × 6 2 × 6 2 × 6 (u , v ) and denoted by i i ∂Xm ∂Y (26) ⎡ ⎤ (u − u )D 0 i x ⎢ ⎥ where ⎢ ⎥ ⎢ ⎥ c c w (v − v )D T = ⎢ 0 i y ⎥ g ρ . (23) i i z(i) ⎢ ⎥ ∂h ∂h ∂h ∂h ∂h ⎣ ⎦ d(i) d(i) d(i) d(i) d(i) = , w w w c 1 ∂Xm ∂r ∂q ∂v ∂ω k k k k k 2 × 13 (27) ∂h ∂h ∂h ∂h ∂h Equation (22)and (23) are combined to derive the predicted d(i) d(i) d(i) d(i) d(i) = . w w w w w pixel location without distortion for the ith target Z = k(i) ∂Y ∂r ∂θ ∂ϕ ∂ρ i i i i i 2 × 6 (u , v ). Its equation is addressed as i i ⎡ ⎤ c When there are N targets observed concurrently, they are x(i) ⎡ ⎤ u − stacked in one measurement vector Zd to form a single 0 k ⎢ c ⎥ u D T i   ⎢ x ⎥ z(i) − batch-form update equation which is shown as ⎣ ⎦ ⎢ ⎥ Z = = h Xm = . (24) k(i) c k ⎢ ⎥ v ⎣ f ⎦ i y(i) v − D T T T z(i) Zd = Zd ··· Zd . (28) k(1) k(N) By using image correct, the predicted pixel location of the ith target within distortion Zd = (u , v ) is addressed as k(i) d(i) d(i) Similarly, the batch measurement Jacobian matrix is derived as ⎡ ⎤ (u − u ) i 0 ⎡ ⎤ 2 4 ⎢ ⎥ 1+3k r +5k r 0 1 2 T ⎢ d d ⎥ ⎣ ⎦ Zd = h Xm = ⎢ ⎥ + , (25) ∂h ∂h ∂h k(i) d T T ⎣ ⎦ (v − v ) H = ··· = H ··· H , i 0 v k w w 1 N ∂Xm ∂Y ∂Y k 1 N 2 4 1+3k r +5k r 1 2 d d (29) where k and k are coefficients for the radial distortion of 1 2 the image. where ⎡ ⎤ ∂h ∂h 1 1 0 0 ... 0 2 × 6 2 × 6 2 × 6 ⎢ w ⎥ ∂Xm ∂Y ⎢ k ⎥ 2 × 13 1 2 × 6 ⎢ ⎥ ∂h ∂h ⎢ 2 2 ⎥ ⎢ 0 0 ... 0 ⎥ 2 × 6 2 × 6 2 × 6 ⎢ ⎥ ∂Xm ∂Y 2 × 13 2 2 × 6 ⎢ ⎥ ⎢ ⎥ ∂h ∂h 3 3 ⎢ ⎥ 0 0 ... 0 H = 2 × 6 2 × 6 2 × 6 . (30) ⎢ ⎥ ∂Xm ∂Y k 2 × 13 3 ⎢ 2 × 6 ⎥ ⎢ ⎥ . . . . . ⎢ ⎥ . . . . . . ⎢ . . . . . ⎥ ⎢ ⎥ ⎢ ⎥ ∂h ∂h ⎣ ⎦ N N 0 0 0 ... 2 × 6 2 × 6 2 × 6 ∂Xm ∂Y k 2 × 13 N 2 × 6 2.5. Image Matching by Using Cross-Correlation. The image when the robot senses them in the first time. The predicted patch information of the targets whose variables are set in pixel locations of the targets within distortion are estimated the process state vector has been stored in the data-base by using (25). Next important issue is how to search for 6 Journal of Robotics the actual pixel locations of the targets in a 320 × 240 Zd − Zd and H , respectively. The measurement k k k(2N×(13+6N)) image. Image matching is a fundamental and vital aspect covariance and Kalman gain can be performed as of many problems in computer vision including motion − T Cov(Zd ) = H Cov(X ) H , tracking and object recognition. The image features are k k 2N×2N k k (13+6N)×(13+6N) invariant to rotation, image scaling, change in illumination, (33) T −1 and 3D camera viewpoint. It is still a popular research K = Cov(X )H [Cov(Zd )] . k(13+6N)×2N k k topic regarding how to allow a single candidate image to Finally, the process state vector and its covariance are be correctly matched with high probability. Based on the updated at each iteration state and presented as estimation of measurement covariance, the small area where the target lies with high probability is able to be predicted. X = X + K Zd − Zd , The measurement covariance of the ith target is defined as k((13+6N)×1) k k(2N×1) k(2N×1) ⎡ ⎤ 2 2 − − σ σ Cov(X ) = Cov X − K H Cov X , k k k uu(i) uv(i) (13+6N)×(13+6N) k k ⎣ ⎦ Cov Zd = . (31) k(i) 2 × 2 2 2 (34) σ σ vu(i) vv(i) where Zd is the actual measurement and detected by using The template match technique is used to search the actual image matching search. Zd is the predicted measurement image pixel location of the ith target in the small search area k and computed by the sensor model. The error in the estima- whose width and length are kσ and kσ ,respectively. uu(i) vv(i) tion is reduced by the iteration and the unknown depths of The coefficient k is a constant and defines how large the the targets converge toward the real values gradually. search area is. The larger k is, the more is the search time. There are two advantages regarding the search area which is estimated by the measurement covariance. One advantage is 2.7. Fast Inverse Transformation for Covariance Matrices. The that a lot of time is saved because the image pixel location inverse matrix of the measurement covariance has to be com- of the ith target is detected in a small search area rather than puted at each iteration step. It needs plenty of running time in a 320 × 240 image. The other one is that the successful by using the transpose of the matrix of cofactors to invert search rate increases dramatically because the search area the measurement covariance. In order to deal with the large allows the image pixel location to be correctly matched size of the inverse matrix within the variations of N targets with high probability. As a result, it is not necessary to use efficiently, Cholesky decomposition is applied to invert the a complicated object recognition algorithm such as Scale measurement covariance and reduce the running time at Invariant Feature Transform (SIFT) [17] for image matching each iteration step. The measurement covariance is factored in our system. Cross-correlation search is applied to be our in the form Cov(Zd ) = LL for a lower-left corner of the template match algorithm and the computing loading is matrix L because the measurement covariance is positive lower due to its simplification. This approach uses cross- semidefinite. The matrix L is not unique, and so multiple correlation of image to search a suitable image patch. I is factorizations of a given matrix Cov(Zd ) are possible. A defined as the template image patch for the ith target and standard method for factoring positive semidefinite matrices stored in the database. I is defined as a candidate image patch is the Cholesky factorization. The element of L is computed in the search area whose width is kσ and length is kσ . uu(i) vv(i) by using Cholesky decomposition and addressed as follows: The cross-correlation value of I with I is given by t D j−1 I (u, v) − I I (u, v) − I D D t t L = A − L , jj jj jk . (32) k=1 M − 1 σ σ I I D t uv (35) ⎛ ⎞ j−1 M is the number of pixels in an image patch and σ 1 ⎝ ⎠ L = A − L L , ij ij ik jk and σ are the stand deviations of I and I ,respectively. I I D D D L t jj k=1 and I are the average values of I and I ,respectively. The t D t maximum value of cross-correlation of I with I is the best t D where A is the element of A = C(Zd ). ij k template matching and it is seen as the matching target pixel A matrix equation in the form LX = b or UX = b can be patch for the ith target. solved by forward substitution for lower triangular matrix L and back substitution for upper triangular matrix U,respec- tively. In our proposed method, Cholesky decomposition 2.6. Iterated EKF Updating. The EKF is one of the most and forward-and-back substitution are combined to invert widely used nonlinear estimators due to its similarity to the optimal linear filter, its simplicity of implementation, the measurement covariance for reducing the computational loading. and its ability to provide accurate estimates in practice. We employ the iterated EKF to update the state. At each iteration step k, the prior process state vector is computed 3. The Analysis of the Algorithm by using (4) and then Zd is calculated as a function of Xm by using (28). Next, the measurement innovation In this section, there are two key points about the proposed and the measurement Jocobian matrix are computed as algorithm presented as follows. The first key point is that Journal of Robotics 7 the details of algorithm will be analyzed to prove that it is not necessary to get the depth information in our algorithm even though it is designed to track the 3D localization of the targets. The other key point is introduced to explain why the simple template match technique, cross-correlation search, is applied. Frame = 50 Frame = 55 Frame = 60 Frame = 65 Frame = 70 3.1. Depth Information Analysis in the Proposed Algorithm. Figure 3: Detect the actual target pixel location by using cross- The depth information should be known while the robot correlation search. localizes the targets in 3D space by using monocular vision. However, the proposed algorithm does not need the depth information to localize targets. As presented in (14), it is not w where necessary to know the depth information to calculate θ and w w w c ∂θ ∂θ ∂g ∂g ∂h ϕ . Similarly, it seems that the depth information should be u i i = . (43) w c known to compute k in (19). k is derived as q q ∂h ∂g ∂g ∂h ∂h d u d w c wc c ∂g /∂g = R and ∂h /∂h can be computed without g but T T T T u d w w w w ∂ϕ ∂ρ ∂r ∂θ w w i i i i ∂θ /∂g cannot be calculated without g according to (39). k = , (36) q i z w w w w ∂q ∂q ∂q ∂q k k k k 6 × 4 However, ∂θ /∂h still can be estimated without knowing g i z w w c and g if ∂θ /∂g and ∂g /∂h are combined. The details are z i where proved by the following equations: w w w ∂θ ∂θ ∂g i i w c w c w c g /g −g /g = ∗ . (37) ∂θ ∂g w z z x z w w i ∂q ∂g ∂q k k w 2 2 2 2 w c w c w c w c ∂g ∂h g /g + g /g g /g + g /g x z z z x z z z The depth information g has to be known for calculating w w 1 ∂g ∂θ /∂g according to the following equation: × ∗ , g ∂h w w g −g ∂θ z x (44) = . (38) w w 2 w 2 w 2 w 2 ∂g (g ) +(g ) (g ) +(g ) x z x z where ⎡ ⎤ −D However, ∂θ /∂q still can be computed without knowing x i k w ⎢ ⎥ g because of ∂g /∂q . The details are proved as follows: c f z ⎢ ⎥ ∂g ⎢ ⎥ −D = ⎢ ⎥ g . (45) w c w c ⎢ 0 ⎥ g /g −g /g ∂h ∂θ u z z x z i ⎣ f ⎦ 2 2 2 2 w c w c w c w c ∂q g /g + g /g g /g + g /g k x z z z x z z z c w Therefore, ∂θ /∂h is computed without g and g by using 1 ∂g d i z z × , w c w c w c c (44) and applying (14)tocalculate g /g , g /g and g /g . g ∂q x z y z z z w w w Based on the results of calculating ∂θ /∂q , ∂θ /∂h and (39) d i k i (14), it infers that we have solved a problem that the depth c w where information, g and g , is not able to be measured only z z w wc wc wc wc by one image. This is a very important characteristic of the c c c c g g g g 1 ∂g ∂R ∂R ∂R ∂R = . proposed algorithm. c c c c c ∂q g ∂q g ∂q g ∂q g g ∂q z k 1 z 2 z 3 z 4 z (40) 3.2. Analysis Regarding Using Cross-Correlation Search. Gen- w c w c w c erally speaking, cross-correlation search is though as a simple g /g , g /g ,and g /g are computed by (14). Equation (39) x z y z z z template match algorithm and it is not as robust as SIFT. can be calculated without knowing the depth information g However, I is still detected correctly in the search area due to the following equation: D by using cross-correlation search because the small area is ⎡ ⎤ (u − u)D 0 x estimated by an iterated EKF and it includes the actual ⎢ ⎥ c ⎢ ⎥ pixel location of the target with high probability. As shown ⎢ ⎥ (v − v)D = ⎢ 0 y ⎥ . (41) in Figure 3, the predicted pixel locations of the targets are ⎢ ⎥ ⎣ ⎦ f estimated by using the sensor model and denoted by the green crosses. The pixel locations of the red crosses are the corresponding target pixel locations and detected by In the same way, the depth information should be known applying cross-correlation search. Based on the testing result to compute k in equation (19). k is derived as h h d d that the red crosses are closer to the actual target pixel T locations, it has proved that it is feasible for the proposed T T T T w w w w ∂r ∂θ ∂ϕ ∂ρ i i i i algorithm to apply cross-correlation search as its template k = , (42) ∂h ∂h ∂h ∂h d d d d 6 × 2 match algorithm. 8 Journal of Robotics Door Wall (a) (b) (c) Figure 4: The encoder shown in (a) is used to measure the linear velocity and simulate the acceleration for the robot. The electrical compass shown in (b) is applied to measure the angular velocity of Robot the robot. As shown in (c), the camera with wide angle lens is the Moving paths additional measurement sensor. The three sensors are combined to estimate the localization of targets. A single camera Figure 5: The experiments are designed to localize the natural landmarks as targets with varied linear and angular velocity patterns 4. Experimental Results in different moving paths including the straight and curved paths. In order to validate the proposed algorithm for localizing targets when the ground truth is available we have performed a number of experiments. The algorithm is implemented by C++ and performed by PC with 2.0 GHz microprocessor. The monocular sensor is a single camera with wide angle lens because we hope that more targets can be observed in one image and tracking rate can be higher. The camera’s field of view is 170 with a focal length of 1.7 mm. The image measurements received at a rate of 15 Hz are distorted with noise σ = 20 pixel. The addressed experimental results are tested under the high speed robot motion because the average velocity of each case is higher than 20 cm/sec and the maximum velocity of all cases is 69.11 cm/sec. For the duration of experiments, the initial distance between the camera and the targets ranges from 1.68 m to 5.76 m. The unknown depth of the target is estimated by sequential images with EKF and six cases (3.0 m, 3.5 m, 4.0 m, 4.5 m, Figure 6: The predicted pixel locations of the targets are denoted 5.0 m, and 5.5 m) are assumed to be default depths for each by the green crosses and estimated by using the sensor model. The experiment case. All of the sensors mounted on our system pixel locations of the red crosses are the corresponding target pixel are shown in Figure 4. locations and detected by applying cross-correlation search. There are some measurement errors caused by the cam- era distortion when using the camera with wide angle lens. Before validating the proposed algorithm, we performed an 4.1. Results of Target Localization. For demonstrating the experiment to estimate the distorted noise by making use of performance and practical applications to the proposed the artificial landmarks. We chose the corners of the artificial MVLT, the experiments for a mobile robot go through the landmarks as targets. The undistorted pixel locations of the doors by tracking the feature targets located upon the door corners are estimated by the pinhole model and then the and localize robot location simultaneously. image correction is applied to compute their predicted pixel locations with distortion. Owing to using a camera with 4.1.1. Target Localization Experiments. Once a robot has to wide angle lens, there is an error between the predicted pass through doors, one should recognize the accurate doors pixel location with distortion and its actual pixel location position. The experiment shown in Figure 5 is designed to which is detected by the cross-correlation search. In the verify accuracy of MVLT for the door as tracked targets. proposed algorithm, the predicted pixel location without Figure 6 shows the real environmental features as targets distortion is estimated in terms of the prior process state whose positions require to be estimated for a task that the vector. The distorted error is produced if transforming the robot passing through the door. Since the developed system undistorted pixel locations of targets to the distorted pixel does not require an initial setting at the first iteration, it locations. Therefore, the distorted error should be taken provides a practical usage under a dynamical motion if target into consideration very carefully. Based on the experimental is detected instead of training processes of the environment result, the distorted noise is assumed to be 20 pixels. features for robot localization. The two different moving Journal of Robotics 9 Table 1: Error comparison of target tracking. Estimated target position (m) Algorithm Item Track OK Target distance error (m) X YZ Target 1 in Path A 0.7600 0.1423 0.6528 OK 0.1369 Target 2 in Path A 0.7800 −0.0374 0.6909 OK 0.1303 MVTL Target 1 in Path B 0.8100 0.0975 0.6853 OK 0.0839 Target 2 in Path B 0.7600 −0.1135 0.5819 OK 0.1523 MVTL Average Error 0.1049 Target 1 in Path A 0.7800 0.0460 0.8234 OK 0.2042 Target 2 in Path A 0.8370 −0.0218 1.0201 OK 0.3723 MonoSLAM [18] Target 1 in Path B 1.1158 0.0896 1.2330 Failed 0.6160 Target 2 in Path B 1.2281 −0.1161 1.4326 Failed 0.8435 MonoSLAM [18]Average Error 0.5090 The 2nd experiment moving paths 4 3 3 2.5 1.5 By MVTL algorithm By MonoSLAM Figure 9: The comparison of the target 2 depth errors (meter) 0.5 between MVTL and MonoSLAM [18]inPathB. 0.5 0.4 0.3 0.2 0.1 0 −0.1 precise measurement information to make MonoSLAM [18] Path A correcting the prior process state variables under a high- Path B speed movement when the control input is not provided. It also shows that the image-distorted noise σ = 20 pixel is Figure 7: Ground truth of experiments. too large for MonoSLAM [18] model. Owing to combining hybrid sensor information, the proposed MVTL algorithm is capable of localizing targets with image distorted noise σ = 20 pixel and its average error is lower than 0.11 m. It has proved that MVTL algorithm is robust to track targets under a higher-speed movement with larger measurement noise. 4.1.2. Acceleration Approximation-Based Error Comparison. We also performed an experiment about the comparison By MVTL algorithm between encoders and IMU. We choose an encoder as By MonoSLAM our sensor instead of IMU in indoor environment. The Figure 8: The comparison of the target 1 depth errors (meter) w w w acceleration a is approximated as (v − v )/Δt by (k) (k) (k−1) between MVTL and MonoSLAM [18]inPathB. encoders in order to have similar on-line simulation data. It will increase additional acceleration noise if we use w w prevelocity v rather than v to simulate the acceleration (k) (k+1) paths with varied linear velocities and orientations are of the robot at state k. However, an iterated EKF is robust examined as shown in Figure 7. The initial true depth for enough to compensate for errors from this low-cost sensor paths A and B between robot and the doors is about 3.5 and the definition of acceleration. The error of acceleration meter, path A is a straight forward path, and path B is a left from a simulated IMU is lower than 0.001 m/s .Based way of robot to move forward to the doors. The experimental on the result shown in Table 2, the errors of localization result is summarized in Table 1, and the averaged error of by using encoders are still accepted even though we use proposed approach is about 10 cm. prevelocity v to simulate acceleration at iterated state k.It (k) According to the experimental result shown in Figures has proved that errors are able to be reduced gradually with 8 and 9, it infers that the camera is not able to provide the EKF recursive loop. 134 10 Journal of Robotics Table 2: Comparison of localization errors between the encoder Table 3: Comparison of execution time. and IMU. Targetnumber 123 4 Maximum Average IMU Encoder Cholesky decomposition (sec) 0.877 1.182 1.548 1.92 Robot motion Velocity Velocity Error Error Matrix of cofactors (sec) 0.871 1.216 7.07 516.742 (m/sec) (m/sec) (m) (m) Low Velocity 0.363 0.254 0.052 0.089 High Velocity 0.582 0.446 0.142 0.144 able to localize targets with “cm” level accuracy under a Localization error with different default depth values higher-speed movement. Besides, we have validated that 2.5 it is practical to use odometry sensors to track targets as 1.5 well. Not only does the system start the recursive procedure without the initial setting but also the robot can move 0.5 rapidly to localize targets. The EKF-based algorithm is really −0.5 −1 practical and robust to reduce errors from sensors even −1.5 Frame though the low-cost sensors are used in our system. The Default = 3 Default = 4.5 efficiency of the proposed algorithm is impressive by using Default = 3.5 Default = 5 Cholesky decomposition and some computational tricks. Default = 4 Default = 5.5 In terms of the experimental result shown in Figure 10, we could conclude that the ability of localizing targets Figure 10: The unknown depth of the target is estimated by depends on the parallax from different views rather than sequential images with EKF and six cases (3.0 m, 3.5 m, 4.0 m, the distance between the robot and the target. Consequently, 4.5 m, 5.0 m, and 5.5 m) are assumed to be default depths for each the proposed algorithm has a high potential extension to experiment case. surveillance and monitoring for UAVs with aerial odometry sensors. In the same way, it is able to be used widely for robot tasks as well. 4.1.3. Performance of Depth Initial Conditions. In terms of the experimental result shown in Figure 10,wecould conclude that the ability of localizing targets depends on 5.2. Future Work. The targets are assumed to be stationary the parallax from different views rather than the distance landmarks in the proposed algorithm. It will be an interesting between the robot and the target. It is a very crucial and challenging research if the algorithm is modified to track viewpoint to analyze the stability of the system in terms a moving target. This type of the technique is necessary and of means and deviations. Not only do we analyze one the going to be widely used in many applications. For instance, target localization but also we understand the details about UGV has to know the intended motions of other moving multiple target localization. Refering to the above experi- objects in order to plan its navigating path for the next ments, MVTL algorithm localizes targets with a higher-speed state. Therefore, it will be a good future work to add a motion and six kinds of default depth values converge to the new approach into the proposed algorithm to track moving similar value with little errors. targets. Besides, there is another important future work for our system. This future work is to find some ways or 4.2. Experiments of Fast Inverse Matrix Transformation. Fi- algorithms to improve accuracy of the measurement data by nally, we performed an experiment to verify Cholesky the low-cost sensor because it will improve the localization decomposition and forward-and-back substitution. In this errors in terms of more accurate measurement data. experiment, the totally testing time is 4.2 sec with 15 Hz image rate. At first, the transpose of the matrix of cofactors is used to invert the measurement covariance matrix whose size Appendices ranges from 2 × 2to8 × 8. The experimental results shown in Table 3 present that the running time is long if the system The proof of the system covariance is presented in appendix. inverts a 6 × 6 or a more dimension matrix. However, the It derives the modified covariance in motion model at each running time is reduced dramatically if the system inverts an iteration step and the new covariance after adding new 8 × 8 matrix by using Cholesky decomposition and forward- targets. and-back substitution. The test result shows that the system is capable of being real-time even though localizing four targets concurrently. A. The Modified System Covariance in Motion Model 5. Conclusion and Future Work The process state vector is predicted by the system process 5.1. Conclusion. Based on the different experiments we model. It is determined by the old state and the control performed, it has proved that the proposed algorithm is inputs applied in the process. Y is a control vector and Error (m) 93 Journal of Robotics 11 X Cov(X ) X Cov(X ) u is corrupted by mean-zero process. Both of equations are new new new new described as X Cov(X ) old old Y = u + w, (A.1) Cov(Y) = Cov(w). x G Cov(x) Cov(x ) y x y 1 1 The prior estimate error covariance Cov(X )isderived as a function of the prior state estimate X and represented as Figure 11: The size of the posterior estimate error covariance Cov(X ) increases with the number of the targets. The size of the Cov X ≈ F CF ,(A.2) (x,y) (x,y) k system covariance is modified from a 13 × 13 matrix to a 19 × 19 matrix while the robot senses one target in the process state vector. where ⎡ ⎤ Cov(X ) Cov(X , Y ) k−1 k−1 k−1 ⎣ ⎦ The function g and the covariance matrix of the new target C = , Cov(Y , X ) Cov(Y ) k−1 k−1 k−1 are shown as (A.3) ∂f(X, Y) x = g(x, z),(B.3) F = F F  X , Y . X Y (X,Y ) k−1 k−1 ∂(X, Y) T T If the control vector is not correlated with the state X , k−1 A = G Cov(x)G + G Cov(z)G,(B.4) x z x z Cov(X , Y )and Cov(Y , X )are equaltozero k−1 k−1 k−1 k−1 matrices. The prior estimate error covariance Cov(X ) B = G Cov(x). (B.5) simplifies to i x − T T Cov X ≈ F Cov(X )F + F Cov(Y )F . (A.4) X k−1 Y k−1 k X Y Acknowledgment This work is supported by the National Science Council B. The New Covariance after Adding under Grant no. NSC 99-2221-E-009-159. New Targets When the robot tracks a target, the location information of References new target is added into the process state vector. The form of [1] G. Cicirelli, T. D’Orazio, and A. Distante, “Target recognition the new information is determined by sensor measurement by components for mobile robot navigation,” Journal of model. x is composed of the random variables of the new Experimental and Theoretical Artificial Intelligence, vol. 15, no. target information. The new covariance is addressed as 3, pp. 281–297, 2003. ⎡ ⎤ [2] S. Sohn, B. Lee, J. Kim, and C. Kee, “Vision-based real-time Cov(X ) B old target localization for single-antenna GPS-guided UAV,” IEEE ⎣ ⎦ Cov(X ) = . (B.1) new Transactions on Aerospace and Electronic Systems, vol. 44, no. 4, BA pp. 1391–1401, 2008. [3] P. Per ´ ez, J. Vermaak, and A. Blake, “Data fusion for visual The size of the posterior estimate error covari- tracking with particles,” Proceedings of the IEEE, vol. 92, no. ance Cov(X ) increases with the number of the targets 3, pp. 495–513, 2004. (Figure 11). The EFK is a multihypothesis and a proper way [4] F. M. Mirzaei and S. I. Roumeliotis, “A Kalman filter-based to include all the measurement information. There are two algorithm for IMU-camera calibration: observability analysis cases regarding the relationship between the new target and and performance evaluation,” IEEE Transactions on Robotics, other variables. One case is that the estimate of the new vol. 24, no. 5, pp. 1143–1156, 2008. target’s location is independent of the estimates of other [5] S. B. Kim, S. Y. Lee, J. H. Choi, K. H. Choi, and B. T. Jang, targets and the variables of the robot. In this case, the “A bimodal approach for GPS and IMU integration for land covariance matrix of the new target is given by vehicle applications,” in Proceedings of the 58th IEEE Vehicular Technology Conference (VTC ’03), pp. 2750–2753, October A = Cov x , y1 [6] K. Park, H. Chung, J. Choi, and J. G. Lee, “Dead reckoning (B.2) navigation for an autonomous mobile robot using a differ- B = Cov x , x = 0. i y i ential encoder and a gyroscope,” in Proceedings of the 8th International Conference on Advanced Robotics (ICAR ’97),pp. A is a covariance matrix and B is a cross-covariance matrix. 441–446, July 1997. B is a zero matrix because the new target is independent of [7] S. Suksakulchai, S. Thongchai, D. M. Wilkes, and K. Kawa- other targets and the robot by definition. The other case is mura, “Mobile robot localization using an electronic compass that the new target is determined as a function g of its spatial for corridor environment,” in Proceedings of the IEEE Interna- relation z to other target locations. The new target’s location tional Conference on Systems, Man and Cybernetics, pp. 3354– is correlated with the estimates of other targets and the robot. 3359, October 2000. 12 Journal of Robotics [8] J. Schroeder, S. Galler, K. Kyamakya, and K. Jobmann, “Practical considerations of optimal three-dimensional indoor localization,” in Proceedings of the IEEE International Con- ference on Multisensor Fusion and Integration for Intelligent Systems (MFI ’06), pp. 439–443, September 2006. [9] S. Emura and S. Tachi, “Sensor fusion based measurement of human head motion,” in Proceedings of the 3rd IEEE International Workshop on Robot and Human Communication, pp. 124–129, July 1994. [10] P. Grandjean and A. Robert De Saint Vincent, “3-D modeling of indoor scenes by fusion of noisy range and stereo data,” in Proceedings of the IEEE International Conference on Robotics and Automation, vol. 2, pp. 681–687, May 1989. [11] P. S. Maybeck, Stochastic Models Estimation and Control, vol. 1, Academic Press, 1979. [12] S. Thrun, W. Burgard, and D. Fox, Probabilistic Robotics,The MIT Press, 2005. [13] R. Siegwart andI.R.Nourbakhsh, Introduction to Autonomous Mobile Robots, The MIT Press, 2004. [14] R. Smith, M. Self, and P. Cheeseman, “Estimating uncertain spatial relationships in robotics,” in Proceedings of IEEE International Conference on Robotics and Automation, vol. 4, p. 850, 1987. [15] D. Dereniowski and M. Kubale, “Cholesky factorization of matrices in parallel and ranking of graphs,” in Proceedings of the 5th International Conference on Parallel Processing and Applied Mathematics, vol. 3019 of Lecture Notes on Computer Science, pp. 985–992, Springer, 2004. [16] J. Heikkila and O. Silven, “Four-step camera calibration procedure with implicit image correction,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1106–1112, June 1997. [17] S. Se, D. Lowe, and J. Little, “Vision-based mobile robot localization and mapping using scale-invariant features,” in Proceedings of the IEEE International Conference on Robotics and Automation (ICRA ’01), vol. 2, pp. 2051–2058, May 2001. [18] A. J. Davison, I. D. Reid, N. D. Molton, and O. Stasse, “MonoSLAM: real-time single camera SLAM,” IEEE Transac- tions on Pattern Analysis and Machine Intelligence, vol. 29, no. 6, pp. 1052–1067, 2007. International Journal of Rotating Machinery International Journal of Journal of The Scientific Journal of Distributed Engineering World Journal Sensors Sensor Networks Hindawi Publishing Corporation Hindawi Publishing Corporation Hindawi Publishing Corporation Hindawi Publishing Corporation Hindawi Publishing Corporation http://www.hindawi.com http://www.hindawi.com Volume 2014 http://www.hindawi.com Volume 2014 http://www.hindawi.com Volume 2014 http://www.hindawi.com Volume 2014 Volume 2014 Journal of Control Science and Engineering Advances in Civil Engineering Hindawi Publishing Corporation Hindawi Publishing Corporation http://www.hindawi.com Volume 2014 http://www.hindawi.com Volume 2014 Submit your manuscripts at http://www.hindawi.com Journal of Journal of Electrical and Computer Robotics Engineering Hindawi Publishing Corporation Hindawi Publishing Corporation http://www.hindawi.com Volume 2014 http://www.hindawi.com Volume 2014 VLSI Design Advances in OptoElectronics International Journal of Modelling & Aerospace International Journal of Simulation Navigation and in Engineering Engineering Observation Hindawi Publishing Corporation Hindawi Publishing Corporation Hindawi Publishing Corporation Hindawi Publishing Corporation Volume 2014 http://www.hindawi.com Volume 2014 http://www.hindawi.com Volume 2010 Hindawi Publishing Corporation http://www.hindawi.com Volume 2014 http://www.hindawi.com http://www.hindawi.com Volume 2014 International Journal of Active and Passive International Journal of Antennas and Advances in Chemical Engineering Propagation Electronic Components Shock and Vibration Acoustics and Vibration Hindawi Publishing Corporation Hindawi Publishing Corporation Hindawi Publishing Corporation Hindawi Publishing Corporation Hindawi Publishing Corporation http://www.hindawi.com Volume 2014 http://www.hindawi.com Volume 2014 http://www.hindawi.com Volume 2014 http://www.hindawi.com Volume 2014 http://www.hindawi.com Volume 2014 http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Journal of Robotics Hindawi Publishing Corporation

Monocular Vision-Based Robot Localization and Target Tracking

Download PDF
13 pages

Loading next page...
 
/lp/hindawi-publishing-corporation/monocular-vision-based-robot-localization-and-target-tracking-lBeb60mj7a

References

References for this paper are not available at this time. We will be adding them shortly, thank you for your patience.

Publisher
Hindawi Publishing Corporation
Copyright
Copyright © 2011 Bing-Fei Wu et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
ISSN
1687-9600
eISSN
1687-9619
DOI
10.1155/2011/548042
Publisher site
See Article on Publisher Site

Abstract

Hindawi Publishing Corporation Journal of Robotics Volume 2011, Article ID 548042, 12 pages doi:10.1155/2011/548042 Research Article Bing-Fei Wu, Wang-Chuan Lu, and Cheng-Lung Jen Department of Electrical Engineering, National Chiao Tung University, 1001 University Road, Hsinchu 300, Taiwan Correspondence should be addressed to Cheng-Lung Jen, cljen0130@gmail.com Received 14 June 2011; Revised 15 October 2011; Accepted 16 October 2011 Academic Editor: Huosheng Hu Copyright © 2011 Bing-Fei Wu et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. This paper presents a vision-based technology for localizing targets in 3D environment. It is achieved by the combination of different types of sensors including optical wheel encoders, an electrical compass, and visual observations with a single camera. Based on the robot motion model and image sequences, extended Kalman filter is applied to estimate target locations and the robot pose simultaneously. The proposed localization system is applicable in practice because it is not necessary to have the initializing setting regarding starting the system from artificial landmarks of known size. The technique is especially suitable for navigation and target tracing for an indoor robot and has a high potential extension to surveillance and monitoring for Unmanned Aerial Vehicles with aerial odometry sensors. The experimental results present “cm” level accuracy of the localization of the targets in indoor environment under a high-speed robot movement. 1. Introduction vehicles use the Global Position System (GPS) and Inertial Measurement Units (IMUs) for land vehicle applications Knowledge about the environment is a critical issue for [5]. However, most of inertial navigation system sensors are autonomous vehicle operations. The capability of localizing expensive for some applications in the indoor environment. targets with a robot for environment is highly demanded. Optical wheel encoders and an electrical compass provide In [1], it investigates the vision-based object recognition linear and angular velocities respectively. Both of two sensors technique for detecting targets of the environment which are basic odometry sensors and widely used owing to their have to be reached by a robot. After the robot finishes the low cost, simplicity, and easy maintenance. Encoders provide target detection, it is an essential task for robots to know the a way of measuring the velocity to estimate the position positions of targets. The tasks include navigation and object of the robot and compasses are often used to detect the tracking. As a result, the approach to localize targets for orientations of the robot. Based on the sensor information, specific tasks is an essential issue in some applications. For motion control is done and then the localization of the robot UAVs, there are some similar demands [2, 3] that UAVs have is estimated [6, 7]. Despite some limitations of an encoder, to track the positions of ground objects for reconnaissance most researchers agree that an encoder is a vital part of the or rescue assistance with monocular vision. Besides, UAVs robot’s navigation system and the tasks will be simplified need to observe the changing surroundings to understand if the encoder accuracy is improved. Besides, the additional the movements of the aircraft better, or the localizing system has to direct the aircraft to a region of interest after taking sensor, a camera, allows a robot to perform a variety of tasks ground observations. autonomously. The use of computer vision for localization In recent years, odometry sensors have been widely used has been investigated for several decades. The camera has for estimating the motion of vehicles moving in a 3D space not been at the center of robot localization while most of environment such as UAVs and Unmanned Ground Vehicles researchers have more attention to other sensors such as laser (UGVs). For instance, Inertial Navigation Systems (INSs) range-finders and sonar. However, it is surprising that vision are applied to measure linear acceleration and rotational is still an attractive choice for sensors because cameras are velocity, and capable of tracking the position, velocity and compact, cheaper, well understood, and ubiquitous. In this attitude of a vehicle by integrating these signals [4]ormobile paper, the algorithm is achieved by combining different types 2 Journal of Robotics of sensors including optical wheel encoders, an electrical compass, and a single camera. Mobile vehicles such as robots, UGVs, and UAVs are becoming fully intelligent robotic systems which can handle more complicated tasks and environments. It is necessary to provide them with higher autonomy and better function- w w w 1/ρ m(θ , ϕ ) i i i alities. For instance, robot localization and target tracking are mandatory. For these tasks, some of sensors such as GPSs, IMUs, laser range-finders, encoders, and compasses are possibly combined to describe the time evolution of w θ G the motion model. Additionally, visual sensors, such as electro-optic or infrared cameras, have been included in the on-boarded sensor suit of many robots to increase their Parallax value. Therefore, it is a popular research topic regarding w w how to combine hybrid sensors to achieve a special task. r − r Recent researches have shown that the way to improve sequential estimation and achieve repeatable motion with multisensors is to adopt the probabilistic method. The Kalman filter has been widely used for data fusion such as w w Figure 1: The robot moves from r to r .Based on thetwo navigation systems [8], virtual environment tracking systems i k views, the parallax is produced. The initial depth can be refined [9], and 3D scene modeling [10]. The Kalmna filter is a to approach the real distance and the position of the ith target is mathematical method to provide an iterated procedure for estimated. a least-square estimation of a linear system. The procedure includes predicting state by using the motion model and then correcting the state by the measurement innovation. the initial estimate for the unknown information while The EKF is a varied type of the Kalman filter for a non- simultaneously updating the localization of targets and the linear system such as the target tracking system in this paper. robot pose. Finally, Chelesky decomposition and forward- More details about the Kalman filter are introduced in the and-back substitution are presented to calculate the inverse material of [11–14]. In this paper, the algorithm represents covariance matrix efficiently. a new approach of EKF-based multisensor data fusion. The information from different types of sensors is combined to 2.1. The Origins of the Proposed Algorithm. A single camera improve the estimations in the system state. In order to is mounted on our system as one of the sensors. The make oursystemabletodealwiththe varietyoftasks such monocular vision infers that the depth of the target is not as navigation for a mobile robot or ground target tracking able to be measured by only one image but estimated by for UAVs, we are determined to choose monocular vision the sequential images. Therefore, the single camera has to rather than binocular vision as one type of our sensors. Based estimate the depth by observing the target repeatedly to get on the monocular vision, simultaneous robot localization parallax between different captured rays from the target to and target tracking become more complex with a higher the robot. The orientations of target are estimated in the computational loading due to unknown depths of targets in world coordinate system only by one image. Y is a six- only one image observation. Thus, it is an important issue i dimension state vector and used to describe the position of regarding how to make use of computational tricks or adopt the ith target in 3D space. Its equation is addressed as more efficient algorithms to reduce the running time. In this paper, Chelesky decomposition [15] and forward-and- w w w w w back substitution are used to invert the covariance matrix for Y = r θ ϕ ρ . (1) i i i i i improving the computational efficiency because the property of covariance matrix is semipositive definite. r is the location of the robot when the robot detects the The rest of this paper is structured as follows. Section 2 ith target in the first time. G is defined as the position of presents the proposed algorithm and explains the details. the target with respect to the camera coordinate system and Section 3 explains two key points about the proposed algo- denoted by rithm. The experimental results are provided in Section 4, and finally, Section 5 concludes the paper and suggests future 1 c cw w w w w G = R r − r + m θ , ϕ . (2) i i k w i i work. w w θ and ϕ are the orientations of the ith target and calculated i i 2. Explanation of the Algorithm by the pixel location of the target in only one image. The The algorithm is mainly divided into five parts including relationship between the position of the target G and the motion modeling, new target adding, measurement mod- current robot localization r is presented in Figure 1. eling, image matching, and EKF updating. By sequential G is the position of the target with respect to the world w w measurement from sensors, the EKF is capable of improving frame system. 1/ρ is defined as the distance from r to i i Journal of Robotics 3 w w the target and m(θ , ϕ ) is its unit vector. Consequently, where i i w w w w 1/ρ m(θ , ϕ ) is seen as the vector from r to the target ⎡ ⎤ i i i i Δθ w w w w w w and then G is calculated as ((r − r )+1/ρ m(θ , ϕ )). i i i i i cos k ⎡ ⎤ ⎢ ⎥ c w 2 q ⎢ ⎥ Finally, G is estimated if the depth 1/ρ is known in advance. 1 i i ⎢ ⎥ ⎢ ⎥ Δθ ⎢ ⎥ ⎢ ⎥ However, the depth of the target is not able to be measured by u sin ⎢ ⎥ ⎢ ⎥ w w ⎢ ⎥ c c 2 ⎢ ⎥ a single image. The EKF is applied to estimate 1/ρ , θ ,and q ω + Ω = = ⎢  ⎥ . (6) i i k k ⎢ ⎥ ⎢ ⎥ Δθ w k ⎢ q ⎥ 3 ⎢ ⎥ ϕ and they converge toward more correct values under the u sin i ⎣ ⎦ y ⎢ ⎥ ⎢ ⎥ recursive iteration by using sequential image measurement 4 ⎣ ⎦ Δθ information. To sum up, (2) is the basic concept and origin u sin of our proposed algorithm. c c u u u x y z u = [ ] is equal to ω Δt/ω Δt. k+1 k+1 2.2. Motion Modeling. We first derive the continuous-time The covariance is predicted and modified if considering c w system model that describes the time evolution in the state the control covariance Cov(v , ω , a ). We assume that the k k k estimate. This model allows us to employ the sampled process noise of the control vector is not correlated with w c w measurements of the wheel encoder and the compass for the process state vector Xm so that Cov(Xm , v , ω , a ) k k k k k state propagation. The process state is described by the vector is set to be a zero matrix. By using (A.4)in Appendix A, the predicted covariance of the system model is expressed as T T T T w w w c Xm = r q v ω ,(3) k k k k − T Cov Xm = F Cov(Xm )F + Q,(7) Xm k Xm k+1 where r is the position of the robot with respect to the world where frame. q is the quaternion that represents the orientation of w T c T w T the world frame for the robot. The linear velocity in the world Q = F Cov v F + F Cov ω F + F Cov a F . v v ω ω a a k k k frame and the angular velocity with respect to the camera (8) w c frame are denoted by v and ω , respectively. In this system, k k the control inputs are measured by wheel encoders and the T The first item F Cov(Xm )F represents the noise from Xm Xm compass which provide the linear velocity and the angular the process state vector and Q describes the noise from the velocity, respectively. Besides, the linear velocity is used to measurement of the wheel encoders and the compass. The simulate the acceleration data of the robot for describing how Jacobian matrix of F is shown as Xm components of the system state vector change completely. w w The definition of acceleration is defined as a = (v − ∂f ∂f ∂f ∂f k k ∂f F = = . (9) Xm w w w c v )/Δt. In order to have similar on-line simulation, we k−1 ∂r ∂q ∂v ∂ω ∂Xm k k k k w 13 × 13 w w w w define a as (v − v )/Δt rather than (v − v )/Δt. k k k−1 k+1 k Although the former definition is not better than the later The Jacobian matrixes of F and F are as the same as ∂f/∂v v ω one which is closer to the real acceleration, an iterated EKF and ∂f/∂ω ,respectively. is still capable of compensating for errors when adopting w w (v − v )/Δt as the robot’s acceleration. The system model k k−1 2.3. Adding New Target. It is remarkable about our algorithm describing the time evolution of the wheel encoders and of that new targets are initialized by using only one image. the compass is given by The initialization includes the state initial values and the covariance assignment. ⎡ ⎤ ⎡ ⎤ w w w 2 r + v Δt + a Δt k+1 k k k ⎢ ⎥ ⎢ ⎥ ⎢   ⎥ ⎢ ⎥ 2.3.1. Target Initialization. Equation (1) is used to define the ⎢ ⎥ w c c ⎢ q ⎥ k+1 ⎢ q × q ω + Ω ⎥ ⎢ ⎥ k k Xm = = ⎢ ⎥ . (4) location of the new target and estimate the six parameters of k+1 ⎢ ⎥ ⎢ ⎥ w w ⎢ v ⎥ w ⎢ ⎥ k+1 v + a Δt Y with the EKF prediction-update loop. If the robot senses ⎣ ⎦ k k i ⎣ ⎦ the 1st target at the state k for the first time, the new target ω c c k+1 ω + Ω k k information is added and then the process state vector is modified. The expanded process state vector is represented The compass is used to measure Δθ which is defined as by the following equation: c c (θ − θ ). ω and ω aredefinedas(θ − θ )/Δt and k k−1 k k−1 k+1 k c c ⎡ ⎤ (θ − θ )/Δt,respectively. If ω is assumed to be (ω + − k−2 k−1 k+1 k Xm c c w k ⎣ ⎦ Ω ), Ω c is derived as (θ − 2θ +θ )/Δt. q is denoted k−2 k k k k−1 X = , (10) k+1 by 1 ⎡ ⎤ ⎡ ⎤ where q −q −q −q q 1 2 3 4 1 ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ w wT w w w q q −q q q ⎢ 2 1 3 ⎥ ⎢ ⎥ 2 Y = r θ ϕ ρ . (11) 1 1 1 1 1 ⎢ ⎥ ⎢ ⎥ 6 × 1 q = ,(5) k+1 ⎢ ⎥ ⎢ ⎥ ⎢ q q q −q ⎥ ⎢ q ⎥ 3 4 1 3 ⎣ ⎦ ⎣ ⎦ Thefirsttimeobservation of the 1sttargetisdoneatthe w w w q −q q q q 4 2 1 3 4 current camera location r . m(θ , ϕ ) is defined as an unit 1 1 1 4 Journal of Robotics w w The unit vector m(θ , ϕ )isderived as afunctionofthe 1 1 orientations of the target and described as m(θ , ϕ ) ⎡ ⎤ 1 1 w w cos ϕ sin θ 1 1 ⎢ ⎥ ⎢ w ⎥ w w − sin ϕ m θ , ϕ = ⎢ ⎥ . (16) 1 1 1 ϕ ⎣ ⎦ w w cos ϕ cos θ 1 1 y(1) X Thefinalparameter 1/ρ is not able to be measured by only one image but estimated by the sequential images with EKF prediction-update loop. As a result, it is assumed to be an initial value 1/ρ . W 0 Figure 2: The orientations of the target with respect to the world 2.3.2. New Covariance Assignment. The system covariance frame can be estimated as a function of its undistorted pixel location is modified after adding a new target. By using (B.1)in h . Appendix B, the new covariance Cov(X ) and the function of Y are denoted by ⎡ ⎤ vector from the location r to the 1sttargetwithrespect to Cov(Xm ) B ⎣ ⎦ the world frame. Figure 2 illustrates the relationship about Cov X = , T B A w w w w 1 1 uv (17) m(θ , ϕ ), θ ,and ϕ . h = [ ] is defined as the pixel 1 1 1 1 location of the 1st target in an undistorted image. g is the w w w w Y = k r , q , h (u, v), ρ , 1 k k 1 location of the 1st target with respect to the camera frame. The location of the 1st target with respect to the world frame where g is addressed as ⎡ ⎤ ⎡ ⎤ r ⎡ ⎤ (u − u)D 0 x ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ x r ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎡ ⎤ w w wc c wc c ⎢ ⎥ (v − v)D w g = ⎢ g ⎥ = R g = R ⎢ 0 y ⎥ g , (12) z r ⎢ r ⎥ ⎢ ⎥ 1 ⎣ ⎦ ⎢ ⎥ ⎢ ⎥ ⎣ ⎦ w ⎢ ⎥ ⎢ ⎥ g w w w z ⎢ ⎥ θ g g ⎢ ⎥ 1 x z 1 ⎢ −1 ⎥ ⎢ ⎥ tan , k = = ⎢ ⎥ . (18) c c ⎢ ⎥ ⎢ g g ⎥ z z ⎢ ϕ ⎥ 1 ⎢ ⎥ ⎣ ⎦ ⎢  ⎥ where ⎛ ⎞ ⎢   ⎥ w 2 2 1 ⎢ w w ⎥ −g g g wc ⎢ x z ⎥ −1 ⎝ ⎠ ⎢ tan , + ⎥ c c c ⎢ ⎥ g g g z z z ⎣ ⎦ ⎡     ⎤ 2 2 2 q +q − q − q 2 q q − q q 2 q q + q q 2 3 1 4 2 4 1 3 1 3 4 2 ρ ⎢ ⎥ ⎢ ⎥ 2 2 2 2 2 q q + q q q −q + q − q 2 q q − q q = ⎢ ⎥ . 2 3 1 4 1 3 4 3 4 1 2 ⎣ ⎦ B is not a zero matrix because the new target’s location is 2 2 2 2 q q − q q 2 q q + q q q −q − q + q 2 4 1 3 3 4 1 2 1 3 4 correlated with the estimates of the robot. According to (B.4) (13) and (B.5)in Appendix B, B and A are derived as 1 1 Then we get B = k Cov(Xm ) = k k 0 0 , (19) r q 6 × 3 6 × 3 1 x k ⎡ ⎤ (u − u)D 0 x g ⎢ ⎥ T T ⎢ ⎥ A = k Cov(r)k + k Cov q k + a, (20) g 1 r r q q z ⎢ ⎥ wc (v − v)D 0 y = R ⎢ ⎥ . (14) ⎢ ⎥ ⎣ ⎦ where T T a = k Cov(h )k + k Cov ρ k . (21) h d h ρ ρ d d w c w c w w Only g /g and g /g rather than g and g are computed x z y z x y by using h .Itisimpossibletoknow g by only one image. 2.4. Measurement Modeling. The robot moves continuously w c w c However, it is possible to make use of g /g and g /g to x z y z and records the sequential images. This is a process of w w calculate θ and ϕ which are shown as 1 1 detecting and identifying the targets. The parameters of ⎡  ⎤ targets have been set into the process state vector and the w w g g x z −1 tan , covariance has been estimated with the recursive loop. ⎢ ⎥ ⎡ ⎤ c c ⎢ ⎥ g g w z z ⎢ ⎥ ⎢ ⎥ ⎣ ⎦ = ⎢ ⎞ ⎥ . (15) w 2 2 2.4.1. Sensor Model. The predicted pixel locations of targets ⎢ w ⎥ w w ϕ −g 1 g g ⎢ y  ⎥ x z −1 ⎣  ⎦ are estimated as a function of the prior process state vector tan , + c c c g g g z z z which is described in (4). According to (2), the location of Journal of Robotics 5 the ith target in the camera frame can be defined in another The actual image target the robot takes is the distorted way: pixel location rather than the undistorted one. Therefore, we have to calculate Zd to get the measurement innovation k(i) ⎡ ⎤ for EKF updating. x(i) ⎢ ⎥ ⎢ ⎥ c cw w w w w w T = ⎢ T ⎥ = R r − r ∗ ρ + m θ , ϕ . y(i) i i k i i i ⎣ ⎦ 2.4.2. Measurement Covariance Assignment. The measure- z(i) ment covariance is expected to describe what the likely (22) variation of measurement is by the sensor under the current condition. The variation is infected by the variables in the Our measurement sensor of the camera is monocular vision. process state vector and the noise which corrupts the sensor. There is a camera sensor model to describe how the sensor The measurement Jacobian matrix H is maps the variables in the process state vector into the sensor variables. By using the pinhole model [16], T is derived as a function of the undistorted pixel location of the ith target ∂h ∂h d(i) d(i) H = 0 ··· 0 ··· 0 , i 2 × 6 2 × 6 2 × 6 (u , v ) and denoted by i i ∂Xm ∂Y (26) ⎡ ⎤ (u − u )D 0 i x ⎢ ⎥ where ⎢ ⎥ ⎢ ⎥ c c w (v − v )D T = ⎢ 0 i y ⎥ g ρ . (23) i i z(i) ⎢ ⎥ ∂h ∂h ∂h ∂h ∂h ⎣ ⎦ d(i) d(i) d(i) d(i) d(i) = , w w w c 1 ∂Xm ∂r ∂q ∂v ∂ω k k k k k 2 × 13 (27) ∂h ∂h ∂h ∂h ∂h Equation (22)and (23) are combined to derive the predicted d(i) d(i) d(i) d(i) d(i) = . w w w w w pixel location without distortion for the ith target Z = k(i) ∂Y ∂r ∂θ ∂ϕ ∂ρ i i i i i 2 × 6 (u , v ). Its equation is addressed as i i ⎡ ⎤ c When there are N targets observed concurrently, they are x(i) ⎡ ⎤ u − stacked in one measurement vector Zd to form a single 0 k ⎢ c ⎥ u D T i   ⎢ x ⎥ z(i) − batch-form update equation which is shown as ⎣ ⎦ ⎢ ⎥ Z = = h Xm = . (24) k(i) c k ⎢ ⎥ v ⎣ f ⎦ i y(i) v − D T T T z(i) Zd = Zd ··· Zd . (28) k(1) k(N) By using image correct, the predicted pixel location of the ith target within distortion Zd = (u , v ) is addressed as k(i) d(i) d(i) Similarly, the batch measurement Jacobian matrix is derived as ⎡ ⎤ (u − u ) i 0 ⎡ ⎤ 2 4 ⎢ ⎥ 1+3k r +5k r 0 1 2 T ⎢ d d ⎥ ⎣ ⎦ Zd = h Xm = ⎢ ⎥ + , (25) ∂h ∂h ∂h k(i) d T T ⎣ ⎦ (v − v ) H = ··· = H ··· H , i 0 v k w w 1 N ∂Xm ∂Y ∂Y k 1 N 2 4 1+3k r +5k r 1 2 d d (29) where k and k are coefficients for the radial distortion of 1 2 the image. where ⎡ ⎤ ∂h ∂h 1 1 0 0 ... 0 2 × 6 2 × 6 2 × 6 ⎢ w ⎥ ∂Xm ∂Y ⎢ k ⎥ 2 × 13 1 2 × 6 ⎢ ⎥ ∂h ∂h ⎢ 2 2 ⎥ ⎢ 0 0 ... 0 ⎥ 2 × 6 2 × 6 2 × 6 ⎢ ⎥ ∂Xm ∂Y 2 × 13 2 2 × 6 ⎢ ⎥ ⎢ ⎥ ∂h ∂h 3 3 ⎢ ⎥ 0 0 ... 0 H = 2 × 6 2 × 6 2 × 6 . (30) ⎢ ⎥ ∂Xm ∂Y k 2 × 13 3 ⎢ 2 × 6 ⎥ ⎢ ⎥ . . . . . ⎢ ⎥ . . . . . . ⎢ . . . . . ⎥ ⎢ ⎥ ⎢ ⎥ ∂h ∂h ⎣ ⎦ N N 0 0 0 ... 2 × 6 2 × 6 2 × 6 ∂Xm ∂Y k 2 × 13 N 2 × 6 2.5. Image Matching by Using Cross-Correlation. The image when the robot senses them in the first time. The predicted patch information of the targets whose variables are set in pixel locations of the targets within distortion are estimated the process state vector has been stored in the data-base by using (25). Next important issue is how to search for 6 Journal of Robotics the actual pixel locations of the targets in a 320 × 240 Zd − Zd and H , respectively. The measurement k k k(2N×(13+6N)) image. Image matching is a fundamental and vital aspect covariance and Kalman gain can be performed as of many problems in computer vision including motion − T Cov(Zd ) = H Cov(X ) H , tracking and object recognition. The image features are k k 2N×2N k k (13+6N)×(13+6N) invariant to rotation, image scaling, change in illumination, (33) T −1 and 3D camera viewpoint. It is still a popular research K = Cov(X )H [Cov(Zd )] . k(13+6N)×2N k k topic regarding how to allow a single candidate image to Finally, the process state vector and its covariance are be correctly matched with high probability. Based on the updated at each iteration state and presented as estimation of measurement covariance, the small area where the target lies with high probability is able to be predicted. X = X + K Zd − Zd , The measurement covariance of the ith target is defined as k((13+6N)×1) k k(2N×1) k(2N×1) ⎡ ⎤ 2 2 − − σ σ Cov(X ) = Cov X − K H Cov X , k k k uu(i) uv(i) (13+6N)×(13+6N) k k ⎣ ⎦ Cov Zd = . (31) k(i) 2 × 2 2 2 (34) σ σ vu(i) vv(i) where Zd is the actual measurement and detected by using The template match technique is used to search the actual image matching search. Zd is the predicted measurement image pixel location of the ith target in the small search area k and computed by the sensor model. The error in the estima- whose width and length are kσ and kσ ,respectively. uu(i) vv(i) tion is reduced by the iteration and the unknown depths of The coefficient k is a constant and defines how large the the targets converge toward the real values gradually. search area is. The larger k is, the more is the search time. There are two advantages regarding the search area which is estimated by the measurement covariance. One advantage is 2.7. Fast Inverse Transformation for Covariance Matrices. The that a lot of time is saved because the image pixel location inverse matrix of the measurement covariance has to be com- of the ith target is detected in a small search area rather than puted at each iteration step. It needs plenty of running time in a 320 × 240 image. The other one is that the successful by using the transpose of the matrix of cofactors to invert search rate increases dramatically because the search area the measurement covariance. In order to deal with the large allows the image pixel location to be correctly matched size of the inverse matrix within the variations of N targets with high probability. As a result, it is not necessary to use efficiently, Cholesky decomposition is applied to invert the a complicated object recognition algorithm such as Scale measurement covariance and reduce the running time at Invariant Feature Transform (SIFT) [17] for image matching each iteration step. The measurement covariance is factored in our system. Cross-correlation search is applied to be our in the form Cov(Zd ) = LL for a lower-left corner of the template match algorithm and the computing loading is matrix L because the measurement covariance is positive lower due to its simplification. This approach uses cross- semidefinite. The matrix L is not unique, and so multiple correlation of image to search a suitable image patch. I is factorizations of a given matrix Cov(Zd ) are possible. A defined as the template image patch for the ith target and standard method for factoring positive semidefinite matrices stored in the database. I is defined as a candidate image patch is the Cholesky factorization. The element of L is computed in the search area whose width is kσ and length is kσ . uu(i) vv(i) by using Cholesky decomposition and addressed as follows: The cross-correlation value of I with I is given by t D j−1 I (u, v) − I I (u, v) − I D D t t L = A − L , jj jj jk . (32) k=1 M − 1 σ σ I I D t uv (35) ⎛ ⎞ j−1 M is the number of pixels in an image patch and σ 1 ⎝ ⎠ L = A − L L , ij ij ik jk and σ are the stand deviations of I and I ,respectively. I I D D D L t jj k=1 and I are the average values of I and I ,respectively. The t D t maximum value of cross-correlation of I with I is the best t D where A is the element of A = C(Zd ). ij k template matching and it is seen as the matching target pixel A matrix equation in the form LX = b or UX = b can be patch for the ith target. solved by forward substitution for lower triangular matrix L and back substitution for upper triangular matrix U,respec- tively. In our proposed method, Cholesky decomposition 2.6. Iterated EKF Updating. The EKF is one of the most and forward-and-back substitution are combined to invert widely used nonlinear estimators due to its similarity to the optimal linear filter, its simplicity of implementation, the measurement covariance for reducing the computational loading. and its ability to provide accurate estimates in practice. We employ the iterated EKF to update the state. At each iteration step k, the prior process state vector is computed 3. The Analysis of the Algorithm by using (4) and then Zd is calculated as a function of Xm by using (28). Next, the measurement innovation In this section, there are two key points about the proposed and the measurement Jocobian matrix are computed as algorithm presented as follows. The first key point is that Journal of Robotics 7 the details of algorithm will be analyzed to prove that it is not necessary to get the depth information in our algorithm even though it is designed to track the 3D localization of the targets. The other key point is introduced to explain why the simple template match technique, cross-correlation search, is applied. Frame = 50 Frame = 55 Frame = 60 Frame = 65 Frame = 70 3.1. Depth Information Analysis in the Proposed Algorithm. Figure 3: Detect the actual target pixel location by using cross- The depth information should be known while the robot correlation search. localizes the targets in 3D space by using monocular vision. However, the proposed algorithm does not need the depth information to localize targets. As presented in (14), it is not w where necessary to know the depth information to calculate θ and w w w c ∂θ ∂θ ∂g ∂g ∂h ϕ . Similarly, it seems that the depth information should be u i i = . (43) w c known to compute k in (19). k is derived as q q ∂h ∂g ∂g ∂h ∂h d u d w c wc c ∂g /∂g = R and ∂h /∂h can be computed without g but T T T T u d w w w w ∂ϕ ∂ρ ∂r ∂θ w w i i i i ∂θ /∂g cannot be calculated without g according to (39). k = , (36) q i z w w w w ∂q ∂q ∂q ∂q k k k k 6 × 4 However, ∂θ /∂h still can be estimated without knowing g i z w w c and g if ∂θ /∂g and ∂g /∂h are combined. The details are z i where proved by the following equations: w w w ∂θ ∂θ ∂g i i w c w c w c g /g −g /g = ∗ . (37) ∂θ ∂g w z z x z w w i ∂q ∂g ∂q k k w 2 2 2 2 w c w c w c w c ∂g ∂h g /g + g /g g /g + g /g x z z z x z z z The depth information g has to be known for calculating w w 1 ∂g ∂θ /∂g according to the following equation: × ∗ , g ∂h w w g −g ∂θ z x (44) = . (38) w w 2 w 2 w 2 w 2 ∂g (g ) +(g ) (g ) +(g ) x z x z where ⎡ ⎤ −D However, ∂θ /∂q still can be computed without knowing x i k w ⎢ ⎥ g because of ∂g /∂q . The details are proved as follows: c f z ⎢ ⎥ ∂g ⎢ ⎥ −D = ⎢ ⎥ g . (45) w c w c ⎢ 0 ⎥ g /g −g /g ∂h ∂θ u z z x z i ⎣ f ⎦ 2 2 2 2 w c w c w c w c ∂q g /g + g /g g /g + g /g k x z z z x z z z c w Therefore, ∂θ /∂h is computed without g and g by using 1 ∂g d i z z × , w c w c w c c (44) and applying (14)tocalculate g /g , g /g and g /g . g ∂q x z y z z z w w w Based on the results of calculating ∂θ /∂q , ∂θ /∂h and (39) d i k i (14), it infers that we have solved a problem that the depth c w where information, g and g , is not able to be measured only z z w wc wc wc wc by one image. This is a very important characteristic of the c c c c g g g g 1 ∂g ∂R ∂R ∂R ∂R = . proposed algorithm. c c c c c ∂q g ∂q g ∂q g ∂q g g ∂q z k 1 z 2 z 3 z 4 z (40) 3.2. Analysis Regarding Using Cross-Correlation Search. Gen- w c w c w c erally speaking, cross-correlation search is though as a simple g /g , g /g ,and g /g are computed by (14). Equation (39) x z y z z z template match algorithm and it is not as robust as SIFT. can be calculated without knowing the depth information g However, I is still detected correctly in the search area due to the following equation: D by using cross-correlation search because the small area is ⎡ ⎤ (u − u)D 0 x estimated by an iterated EKF and it includes the actual ⎢ ⎥ c ⎢ ⎥ pixel location of the target with high probability. As shown ⎢ ⎥ (v − v)D = ⎢ 0 y ⎥ . (41) in Figure 3, the predicted pixel locations of the targets are ⎢ ⎥ ⎣ ⎦ f estimated by using the sensor model and denoted by the green crosses. The pixel locations of the red crosses are the corresponding target pixel locations and detected by In the same way, the depth information should be known applying cross-correlation search. Based on the testing result to compute k in equation (19). k is derived as h h d d that the red crosses are closer to the actual target pixel T locations, it has proved that it is feasible for the proposed T T T T w w w w ∂r ∂θ ∂ϕ ∂ρ i i i i algorithm to apply cross-correlation search as its template k = , (42) ∂h ∂h ∂h ∂h d d d d 6 × 2 match algorithm. 8 Journal of Robotics Door Wall (a) (b) (c) Figure 4: The encoder shown in (a) is used to measure the linear velocity and simulate the acceleration for the robot. The electrical compass shown in (b) is applied to measure the angular velocity of Robot the robot. As shown in (c), the camera with wide angle lens is the Moving paths additional measurement sensor. The three sensors are combined to estimate the localization of targets. A single camera Figure 5: The experiments are designed to localize the natural landmarks as targets with varied linear and angular velocity patterns 4. Experimental Results in different moving paths including the straight and curved paths. In order to validate the proposed algorithm for localizing targets when the ground truth is available we have performed a number of experiments. The algorithm is implemented by C++ and performed by PC with 2.0 GHz microprocessor. The monocular sensor is a single camera with wide angle lens because we hope that more targets can be observed in one image and tracking rate can be higher. The camera’s field of view is 170 with a focal length of 1.7 mm. The image measurements received at a rate of 15 Hz are distorted with noise σ = 20 pixel. The addressed experimental results are tested under the high speed robot motion because the average velocity of each case is higher than 20 cm/sec and the maximum velocity of all cases is 69.11 cm/sec. For the duration of experiments, the initial distance between the camera and the targets ranges from 1.68 m to 5.76 m. The unknown depth of the target is estimated by sequential images with EKF and six cases (3.0 m, 3.5 m, 4.0 m, 4.5 m, Figure 6: The predicted pixel locations of the targets are denoted 5.0 m, and 5.5 m) are assumed to be default depths for each by the green crosses and estimated by using the sensor model. The experiment case. All of the sensors mounted on our system pixel locations of the red crosses are the corresponding target pixel are shown in Figure 4. locations and detected by applying cross-correlation search. There are some measurement errors caused by the cam- era distortion when using the camera with wide angle lens. Before validating the proposed algorithm, we performed an 4.1. Results of Target Localization. For demonstrating the experiment to estimate the distorted noise by making use of performance and practical applications to the proposed the artificial landmarks. We chose the corners of the artificial MVLT, the experiments for a mobile robot go through the landmarks as targets. The undistorted pixel locations of the doors by tracking the feature targets located upon the door corners are estimated by the pinhole model and then the and localize robot location simultaneously. image correction is applied to compute their predicted pixel locations with distortion. Owing to using a camera with 4.1.1. Target Localization Experiments. Once a robot has to wide angle lens, there is an error between the predicted pass through doors, one should recognize the accurate doors pixel location with distortion and its actual pixel location position. The experiment shown in Figure 5 is designed to which is detected by the cross-correlation search. In the verify accuracy of MVLT for the door as tracked targets. proposed algorithm, the predicted pixel location without Figure 6 shows the real environmental features as targets distortion is estimated in terms of the prior process state whose positions require to be estimated for a task that the vector. The distorted error is produced if transforming the robot passing through the door. Since the developed system undistorted pixel locations of targets to the distorted pixel does not require an initial setting at the first iteration, it locations. Therefore, the distorted error should be taken provides a practical usage under a dynamical motion if target into consideration very carefully. Based on the experimental is detected instead of training processes of the environment result, the distorted noise is assumed to be 20 pixels. features for robot localization. The two different moving Journal of Robotics 9 Table 1: Error comparison of target tracking. Estimated target position (m) Algorithm Item Track OK Target distance error (m) X YZ Target 1 in Path A 0.7600 0.1423 0.6528 OK 0.1369 Target 2 in Path A 0.7800 −0.0374 0.6909 OK 0.1303 MVTL Target 1 in Path B 0.8100 0.0975 0.6853 OK 0.0839 Target 2 in Path B 0.7600 −0.1135 0.5819 OK 0.1523 MVTL Average Error 0.1049 Target 1 in Path A 0.7800 0.0460 0.8234 OK 0.2042 Target 2 in Path A 0.8370 −0.0218 1.0201 OK 0.3723 MonoSLAM [18] Target 1 in Path B 1.1158 0.0896 1.2330 Failed 0.6160 Target 2 in Path B 1.2281 −0.1161 1.4326 Failed 0.8435 MonoSLAM [18]Average Error 0.5090 The 2nd experiment moving paths 4 3 3 2.5 1.5 By MVTL algorithm By MonoSLAM Figure 9: The comparison of the target 2 depth errors (meter) 0.5 between MVTL and MonoSLAM [18]inPathB. 0.5 0.4 0.3 0.2 0.1 0 −0.1 precise measurement information to make MonoSLAM [18] Path A correcting the prior process state variables under a high- Path B speed movement when the control input is not provided. It also shows that the image-distorted noise σ = 20 pixel is Figure 7: Ground truth of experiments. too large for MonoSLAM [18] model. Owing to combining hybrid sensor information, the proposed MVTL algorithm is capable of localizing targets with image distorted noise σ = 20 pixel and its average error is lower than 0.11 m. It has proved that MVTL algorithm is robust to track targets under a higher-speed movement with larger measurement noise. 4.1.2. Acceleration Approximation-Based Error Comparison. We also performed an experiment about the comparison By MVTL algorithm between encoders and IMU. We choose an encoder as By MonoSLAM our sensor instead of IMU in indoor environment. The Figure 8: The comparison of the target 1 depth errors (meter) w w w acceleration a is approximated as (v − v )/Δt by (k) (k) (k−1) between MVTL and MonoSLAM [18]inPathB. encoders in order to have similar on-line simulation data. It will increase additional acceleration noise if we use w w prevelocity v rather than v to simulate the acceleration (k) (k+1) paths with varied linear velocities and orientations are of the robot at state k. However, an iterated EKF is robust examined as shown in Figure 7. The initial true depth for enough to compensate for errors from this low-cost sensor paths A and B between robot and the doors is about 3.5 and the definition of acceleration. The error of acceleration meter, path A is a straight forward path, and path B is a left from a simulated IMU is lower than 0.001 m/s .Based way of robot to move forward to the doors. The experimental on the result shown in Table 2, the errors of localization result is summarized in Table 1, and the averaged error of by using encoders are still accepted even though we use proposed approach is about 10 cm. prevelocity v to simulate acceleration at iterated state k.It (k) According to the experimental result shown in Figures has proved that errors are able to be reduced gradually with 8 and 9, it infers that the camera is not able to provide the EKF recursive loop. 134 10 Journal of Robotics Table 2: Comparison of localization errors between the encoder Table 3: Comparison of execution time. and IMU. Targetnumber 123 4 Maximum Average IMU Encoder Cholesky decomposition (sec) 0.877 1.182 1.548 1.92 Robot motion Velocity Velocity Error Error Matrix of cofactors (sec) 0.871 1.216 7.07 516.742 (m/sec) (m/sec) (m) (m) Low Velocity 0.363 0.254 0.052 0.089 High Velocity 0.582 0.446 0.142 0.144 able to localize targets with “cm” level accuracy under a Localization error with different default depth values higher-speed movement. Besides, we have validated that 2.5 it is practical to use odometry sensors to track targets as 1.5 well. Not only does the system start the recursive procedure without the initial setting but also the robot can move 0.5 rapidly to localize targets. The EKF-based algorithm is really −0.5 −1 practical and robust to reduce errors from sensors even −1.5 Frame though the low-cost sensors are used in our system. The Default = 3 Default = 4.5 efficiency of the proposed algorithm is impressive by using Default = 3.5 Default = 5 Cholesky decomposition and some computational tricks. Default = 4 Default = 5.5 In terms of the experimental result shown in Figure 10, we could conclude that the ability of localizing targets Figure 10: The unknown depth of the target is estimated by depends on the parallax from different views rather than sequential images with EKF and six cases (3.0 m, 3.5 m, 4.0 m, the distance between the robot and the target. Consequently, 4.5 m, 5.0 m, and 5.5 m) are assumed to be default depths for each the proposed algorithm has a high potential extension to experiment case. surveillance and monitoring for UAVs with aerial odometry sensors. In the same way, it is able to be used widely for robot tasks as well. 4.1.3. Performance of Depth Initial Conditions. In terms of the experimental result shown in Figure 10,wecould conclude that the ability of localizing targets depends on 5.2. Future Work. The targets are assumed to be stationary the parallax from different views rather than the distance landmarks in the proposed algorithm. It will be an interesting between the robot and the target. It is a very crucial and challenging research if the algorithm is modified to track viewpoint to analyze the stability of the system in terms a moving target. This type of the technique is necessary and of means and deviations. Not only do we analyze one the going to be widely used in many applications. For instance, target localization but also we understand the details about UGV has to know the intended motions of other moving multiple target localization. Refering to the above experi- objects in order to plan its navigating path for the next ments, MVTL algorithm localizes targets with a higher-speed state. Therefore, it will be a good future work to add a motion and six kinds of default depth values converge to the new approach into the proposed algorithm to track moving similar value with little errors. targets. Besides, there is another important future work for our system. This future work is to find some ways or 4.2. Experiments of Fast Inverse Matrix Transformation. Fi- algorithms to improve accuracy of the measurement data by nally, we performed an experiment to verify Cholesky the low-cost sensor because it will improve the localization decomposition and forward-and-back substitution. In this errors in terms of more accurate measurement data. experiment, the totally testing time is 4.2 sec with 15 Hz image rate. At first, the transpose of the matrix of cofactors is used to invert the measurement covariance matrix whose size Appendices ranges from 2 × 2to8 × 8. The experimental results shown in Table 3 present that the running time is long if the system The proof of the system covariance is presented in appendix. inverts a 6 × 6 or a more dimension matrix. However, the It derives the modified covariance in motion model at each running time is reduced dramatically if the system inverts an iteration step and the new covariance after adding new 8 × 8 matrix by using Cholesky decomposition and forward- targets. and-back substitution. The test result shows that the system is capable of being real-time even though localizing four targets concurrently. A. The Modified System Covariance in Motion Model 5. Conclusion and Future Work The process state vector is predicted by the system process 5.1. Conclusion. Based on the different experiments we model. It is determined by the old state and the control performed, it has proved that the proposed algorithm is inputs applied in the process. Y is a control vector and Error (m) 93 Journal of Robotics 11 X Cov(X ) X Cov(X ) u is corrupted by mean-zero process. Both of equations are new new new new described as X Cov(X ) old old Y = u + w, (A.1) Cov(Y) = Cov(w). x G Cov(x) Cov(x ) y x y 1 1 The prior estimate error covariance Cov(X )isderived as a function of the prior state estimate X and represented as Figure 11: The size of the posterior estimate error covariance Cov(X ) increases with the number of the targets. The size of the Cov X ≈ F CF ,(A.2) (x,y) (x,y) k system covariance is modified from a 13 × 13 matrix to a 19 × 19 matrix while the robot senses one target in the process state vector. where ⎡ ⎤ Cov(X ) Cov(X , Y ) k−1 k−1 k−1 ⎣ ⎦ The function g and the covariance matrix of the new target C = , Cov(Y , X ) Cov(Y ) k−1 k−1 k−1 are shown as (A.3) ∂f(X, Y) x = g(x, z),(B.3) F = F F  X , Y . X Y (X,Y ) k−1 k−1 ∂(X, Y) T T If the control vector is not correlated with the state X , k−1 A = G Cov(x)G + G Cov(z)G,(B.4) x z x z Cov(X , Y )and Cov(Y , X )are equaltozero k−1 k−1 k−1 k−1 matrices. The prior estimate error covariance Cov(X ) B = G Cov(x). (B.5) simplifies to i x − T T Cov X ≈ F Cov(X )F + F Cov(Y )F . (A.4) X k−1 Y k−1 k X Y Acknowledgment This work is supported by the National Science Council B. The New Covariance after Adding under Grant no. NSC 99-2221-E-009-159. New Targets When the robot tracks a target, the location information of References new target is added into the process state vector. The form of [1] G. Cicirelli, T. D’Orazio, and A. Distante, “Target recognition the new information is determined by sensor measurement by components for mobile robot navigation,” Journal of model. x is composed of the random variables of the new Experimental and Theoretical Artificial Intelligence, vol. 15, no. target information. The new covariance is addressed as 3, pp. 281–297, 2003. ⎡ ⎤ [2] S. Sohn, B. Lee, J. Kim, and C. Kee, “Vision-based real-time Cov(X ) B old target localization for single-antenna GPS-guided UAV,” IEEE ⎣ ⎦ Cov(X ) = . (B.1) new Transactions on Aerospace and Electronic Systems, vol. 44, no. 4, BA pp. 1391–1401, 2008. [3] P. Per ´ ez, J. Vermaak, and A. Blake, “Data fusion for visual The size of the posterior estimate error covari- tracking with particles,” Proceedings of the IEEE, vol. 92, no. ance Cov(X ) increases with the number of the targets 3, pp. 495–513, 2004. (Figure 11). The EFK is a multihypothesis and a proper way [4] F. M. Mirzaei and S. I. Roumeliotis, “A Kalman filter-based to include all the measurement information. There are two algorithm for IMU-camera calibration: observability analysis cases regarding the relationship between the new target and and performance evaluation,” IEEE Transactions on Robotics, other variables. One case is that the estimate of the new vol. 24, no. 5, pp. 1143–1156, 2008. target’s location is independent of the estimates of other [5] S. B. Kim, S. Y. Lee, J. H. Choi, K. H. Choi, and B. T. Jang, targets and the variables of the robot. In this case, the “A bimodal approach for GPS and IMU integration for land covariance matrix of the new target is given by vehicle applications,” in Proceedings of the 58th IEEE Vehicular Technology Conference (VTC ’03), pp. 2750–2753, October A = Cov x , y1 [6] K. Park, H. Chung, J. Choi, and J. G. Lee, “Dead reckoning (B.2) navigation for an autonomous mobile robot using a differ- B = Cov x , x = 0. i y i ential encoder and a gyroscope,” in Proceedings of the 8th International Conference on Advanced Robotics (ICAR ’97),pp. A is a covariance matrix and B is a cross-covariance matrix. 441–446, July 1997. B is a zero matrix because the new target is independent of [7] S. Suksakulchai, S. Thongchai, D. M. Wilkes, and K. Kawa- other targets and the robot by definition. The other case is mura, “Mobile robot localization using an electronic compass that the new target is determined as a function g of its spatial for corridor environment,” in Proceedings of the IEEE Interna- relation z to other target locations. The new target’s location tional Conference on Systems, Man and Cybernetics, pp. 3354– is correlated with the estimates of other targets and the robot. 3359, October 2000. 12 Journal of Robotics [8] J. Schroeder, S. Galler, K. Kyamakya, and K. Jobmann, “Practical considerations of optimal three-dimensional indoor localization,” in Proceedings of the IEEE International Con- ference on Multisensor Fusion and Integration for Intelligent Systems (MFI ’06), pp. 439–443, September 2006. [9] S. Emura and S. Tachi, “Sensor fusion based measurement of human head motion,” in Proceedings of the 3rd IEEE International Workshop on Robot and Human Communication, pp. 124–129, July 1994. [10] P. Grandjean and A. Robert De Saint Vincent, “3-D modeling of indoor scenes by fusion of noisy range and stereo data,” in Proceedings of the IEEE International Conference on Robotics and Automation, vol. 2, pp. 681–687, May 1989. [11] P. S. Maybeck, Stochastic Models Estimation and Control, vol. 1, Academic Press, 1979. [12] S. Thrun, W. Burgard, and D. Fox, Probabilistic Robotics,The MIT Press, 2005. [13] R. Siegwart andI.R.Nourbakhsh, Introduction to Autonomous Mobile Robots, The MIT Press, 2004. [14] R. Smith, M. Self, and P. Cheeseman, “Estimating uncertain spatial relationships in robotics,” in Proceedings of IEEE International Conference on Robotics and Automation, vol. 4, p. 850, 1987. [15] D. Dereniowski and M. Kubale, “Cholesky factorization of matrices in parallel and ranking of graphs,” in Proceedings of the 5th International Conference on Parallel Processing and Applied Mathematics, vol. 3019 of Lecture Notes on Computer Science, pp. 985–992, Springer, 2004. [16] J. Heikkila and O. Silven, “Four-step camera calibration procedure with implicit image correction,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1106–1112, June 1997. [17] S. Se, D. Lowe, and J. Little, “Vision-based mobile robot localization and mapping using scale-invariant features,” in Proceedings of the IEEE International Conference on Robotics and Automation (ICRA ’01), vol. 2, pp. 2051–2058, May 2001. [18] A. J. Davison, I. D. Reid, N. D. Molton, and O. Stasse, “MonoSLAM: real-time single camera SLAM,” IEEE Transac- tions on Pattern Analysis and Machine Intelligence, vol. 29, no. 6, pp. 1052–1067, 2007. International Journal of Rotating Machinery International Journal of Journal of The Scientific Journal of Distributed Engineering World Journal Sensors Sensor Networks Hindawi Publishing Corporation Hindawi Publishing Corporation Hindawi Publishing Corporation Hindawi Publishing Corporation Hindawi Publishing Corporation http://www.hindawi.com http://www.hindawi.com Volume 2014 http://www.hindawi.com Volume 2014 http://www.hindawi.com Volume 2014 http://www.hindawi.com Volume 2014 Volume 2014 Journal of Control Science and Engineering Advances in Civil Engineering Hindawi Publishing Corporation Hindawi Publishing Corporation http://www.hindawi.com Volume 2014 http://www.hindawi.com Volume 2014 Submit your manuscripts at http://www.hindawi.com Journal of Journal of Electrical and Computer Robotics Engineering Hindawi Publishing Corporation Hindawi Publishing Corporation http://www.hindawi.com Volume 2014 http://www.hindawi.com Volume 2014 VLSI Design Advances in OptoElectronics International Journal of Modelling & Aerospace International Journal of Simulation Navigation and in Engineering Engineering Observation Hindawi Publishing Corporation Hindawi Publishing Corporation Hindawi Publishing Corporation Hindawi Publishing Corporation Volume 2014 http://www.hindawi.com Volume 2014 http://www.hindawi.com Volume 2010 Hindawi Publishing Corporation http://www.hindawi.com Volume 2014 http://www.hindawi.com http://www.hindawi.com Volume 2014 International Journal of Active and Passive International Journal of Antennas and Advances in Chemical Engineering Propagation Electronic Components Shock and Vibration Acoustics and Vibration Hindawi Publishing Corporation Hindawi Publishing Corporation Hindawi Publishing Corporation Hindawi Publishing Corporation Hindawi Publishing Corporation http://www.hindawi.com Volume 2014 http://www.hindawi.com Volume 2014 http://www.hindawi.com Volume 2014 http://www.hindawi.com Volume 2014 http://www.hindawi.com Volume 2014

Journal

Journal of Robotics – Hindawi Publishing Corporation

Published: Jan 11, 2012

References

  • Target recognition by components for mobile robot navigation,
    G. Cicirelli, T. D'Orazio, and A. Distante
  • Vision-based real-time target localization for single-antenna GPS-guided UAV,
    S. Sohn, B. Lee, J. Kim, and C. Kee
  • Data fusion for visual tracking with particles,
    P. Pérez, J. Vermaak, and A. Blake
  • A Kalman filter-based algorithm for IMU-camera calibration: observability analysis and performance evaluation,
    F. M. Mirzaei and S. I. Roumeliotis
  • A bimodal approach for GPS and IMU integration for land vehicle applications,
    S. B. Kim, S. Y. Lee, J. H. Choi, K. H. Choi, and B. T. Jang
  • Dead reckoning navigation for an autonomous mobile robot using a differential encoder and a gyroscope,
    K. Park, H. Chung, J. Choi, and J. G. Lee
  • Mobile robot localization using an electronic compass for corridor environment,
    S. Suksakulchai, S. Thongchai, D. M. Wilkes, and K. Kawamura
  • Practical considerations of optimal three-dimensional indoor localization,
    J. Schroeder, S. Galler, K. Kyamakya, and K. Jobmann
  • Sensor fusion based measurement of human head motion,
    S. Emura and S. Tachi
  • 3-D modeling of indoor scenes by fusion of noisy range and stereo data,
    P. Grandjean and A. Robert De Saint Vincent
  • Estimating uncertain spatial relationships in robotics,
    R. Smith, M. Self, and P. Cheeseman
  • Cholesky factorization of matrices in parallel and ranking of graphs,
    D. Dereniowski and M. Kubale
  • Four-step camera calibration procedure with implicit image correction,
    J. Heikkila and O. Silven
  • Vision-based mobile robot localization and mapping using scale-invariant features,
    S. Se, D. Lowe, and J. Little
  • MonoSLAM: real-time single camera SLAM,
    A. J. Davison, I. D. Reid, N. D. Molton, and O. Stasse