Download

You can download the elo (aka Ego Localization) module here elo-1.5.zip

1. Introduction

Baidu ego localization system is an accurate ego localization solution for self-driving. By combining sensor information, Global navigation satellite systems (GNSS), and Baidu HD Map into one system, Baidu ego localization system is able to offer a localization solution with high accuracy.

GNSS positioning observes range measurements from orbiting GNSS satellites, while GNSS in general has some restrictions that it may not be available for high accuracy requirements of self-driving. Baidu ego localization system combines sensor information with HD Map and GNSS solution to provide high-accuracy localization solution. System extracts features from sensors and Baidu HD Map for feature matching. After feature matching, system is able to locate current vehicle in HD Map. Motion compensation is proposed to compensate current localization solution.

The main components of Baidu ego localization system is introduced as follow:

• Sensors: include vision sensors (in-vehicle cameras, industrial camera CCDs and other image collecting sensors) and position sensors (include GPS, GLONASS, Beidou and other GNSS).

• HD Map: HD Map provides road information within specific area.

• Real-time location feature: Baidu ego localization system applies deep neural network architecture ENet (see: ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation) to extract lane lines and mark features by sensor for feature matching.

• Pre-stored location feature: obtains lane lines and mark features provided by HD Map.

• Feature matching: Baidu ego localization system locates current ego vehicle in HD Map by feature matching.

• Motion compensation: compensates current frame and adjusts the matching result via historical information.

2. Requirements: software

a. External dependencies

• Requirements for cmake 2.8.7 or higher (see: cmake installation instructions)

• Requirements for OpenCV 3.X (see: OpenCV installation instructions)

• Recommended requirements for Protobuf (see: github)

• Recommended requirements for CUDA (see: CUDA installation instructions)

• Recommended requirements for Eigen 3 (see: Eigen installation instructions)

• Recommended requirements for jsoncpp (see: github)

b. Internal dependencies

• Requirements for modified Caffe (lib/libcaffe.so)

• Requirements for HD Map database (config/hadmap/hadmap.db)

3. Requirements: hardware

Deployed on NVIDIA Drive PX2 (PDK 4.1.4.0).

4. Data format

• Input file:

  Note that optimum solution from SPAN-CPT (ground_truth_longitude and ground_truth_latitude) are provided to make comparison with solution from Baidu ego localization system.

  image_name longitude latitude ground_truth_longitude ground_truth_latitude
  

• Config file:

  # effective image rectangle: top-left corner point, width and height of resulting cropped image in BGR format, pixel
  top-left_corner_point_x top-left_corner_point_y width height
  
  # width and height of ENet input image, pixel
  segmented_image_width segmented_image_height
  
  # camera parameters: principal point, focal length and rotation angle of camera related to vehicle coordinate system, pixel and radian
  principal_point_x principal_point_y focal_length_x focal_length_y pitch yaw roll
  
  # center point of image, pixel
  center_point_x center_point_y
  
  # ENet parameters file
  enet_para_path
  
  # ENet structure file
  enet_structure_path
  
  # camera parameters: translation of camera related to vehicle coordinate system, pixel
  translation_x translation_y
  
  # database of HD Map
  database_path
  

  Sample config parameters

  510 808 1410 1068
  448 224
  948.617 628.628 1918.64 1924.48 -3.9 0.6 0
  960 604
  ../data/model/model_binary.caffemodel
  ../data/model/net.prototxt
  0 150
  ../data/hadmap/hadmap.xml
  

• Output format:

  image_name
  [+] Begin Localization
  [-] End Localization, misc time: runtime ####
  [INFO] GROUNDTRUTH (ground_truth_longitude, ground_truth_latitude)  LOCALIZATION (localization_longitude, localization_latitude)
  

  Sample output:

  20170628_000039131.jpg
  [+] Begin Localization
  [-] End Localization, misc time: 91.95475700ms ####
  [INFO] GROUNDTRUTH (133.02665542, 25.40116628)   LOCALIZATION (133.02666082, 25.40117062)
  

5. Installation

• Install and build dependencies in 2. Recommended requirements: software, then do the following steps.

• a. System software dependencies can be installed with:

  sudo apt-get install cmake libhdf5-dev liblmdb-dev libleveldb-dev libatlas-dev libatlas-base-dev libgflags-dev libgoogle-glog-dev libopencv-dev libmatio-dev libcurl4-openssl-dev
  

• b. Find the elo in modules:

  We’ll call the directory elo into P_ROOT

• c. Compile and run localization:

  cd $P_ROOT
  mkdir build
  cd build
  cmake ..
  make
  ./localization_test config_file input_file image_path
  

6. Demo

• Hardware description:

  image: SEKONIX SF3323, 30 Hz;

  GNSS: U-Blox M8, 8 Hz;

  SPAN-CPT: 10 Hz;

• Test data path:

  testdata.zip

  unzip testdata in /data folder.

• To run the localization demo in testdata1:

  ./localization_test ../config/apollo_release.config ../data/testdata1/test_list.txt ../data/testdata1/image/