Skip to main content

LiDAR Object Detection Algorithm

Function Introduction​

The LiDAR object detection algorithm is a CenterPoint algorithm model trained on the nuscenes dataset using OpenExplorer.

The algorithm takes 32-channel LiDAR point cloud data as input and outputs information including 3D detection bounding boxes, confidence scores and categories of targets. It supports six target detection categories: car, truck, bus, barrier, motorcycle and pedestrian.

This example uses local LiDAR point cloud files as input, leverages the BPU for algorithm inference, and publishes rendered image messages containing point cloud data, target detection bounding boxes and orientations. The algorithm results are rendered and displayed in a browser on the PC side.

Code Repository: (https://github.com/D-Robotics/hobot_centerpoint)

Supported Platforms​

PlatformRunning MethodExample Function
RDK UltraUbuntu 20.04 (Foxy)Local playback, with inference rendering results displayed via web
RDK S100, RDK S100PUbuntu 22.04 (Humble)Local playback, with inference rendering results displayed via web

Preparation​

RDK Platform​

  1. The RDK has been flashed with the Ubuntu 20.04/Ubuntu 22.04 system image.
  2. TogetheROS.Bot has been successfully installed on the RDK.
  3. Ensure the PC can access the RDK via the network.

Usage Guide​

RDK Platform​

Local Point Cloud File Playback​

The LiDAR object detection algorithm example uses LiDAR point cloud file playback for inference, then publishes image messages with rendered algorithm results via the websocket package, enabling the display of published images and corresponding algorithm results in a browser on the PC side.

Prepare LiDAR point cloud files:

# Download playback point cloud files on the board
wget http://archive.d-robotics.cc/TogetheROS/data/hobot_centerpoint_data.tar.gz

# Decompress the file
mkdir config
tar -zxvf hobot_centerpoint_data.tar.gz -C config
# Decompressed data is in the config/hobot_centerpoint_data directory

Launch the algorithm example:

# Configure the tros.b environment
source /opt/tros/setup.bash

# Launch the websocket service
ros2 launch websocket websocket_service.launch.py

# Launch the launch file
ros2 launch hobot_centerpoint hobot_centerpoint_websocket.launch.py lidar_pre_path:=config/hobot_centerpoint_data

Result Analysis​

After launching the algorithm example, the following information is output in the running terminal:

[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [hobot_centerpoint-1]: process started with pid [22470]
[INFO] [websocket-2]: process started with pid [22472]
[hobot_centerpoint-1] [WARN] [0948485758.916907430] [centerpoint_node]:
[hobot_centerpoint-1]  preprocess_config: config/centerpoint_preprocess_5dim.json
[hobot_centerpoint-1]  model_file: config/model/model.hbm
[hobot_centerpoint-1]  lidar_list_file: ./config/nuscenes_lidar_val.lst
[hobot_centerpoint-1]  is_show: 1
[hobot_centerpoint-1]  is_loop: 1
[hobot_centerpoint-1]  pub_topic_name: /hobot_centerpoint
[hobot_centerpoint-1]  lidar_pre_path: ./config/hobot_centerpoint_data
[hobot_centerpoint-1] [BPU_PLAT]BPU Platform Version(1.3.3)!
[hobot_centerpoint-1] [HBRT] set log level as 0. version = 3.14.25.0
[hobot_centerpoint-1] [DNN] Runtime version = 1.12.3_(3.14.25 HBRT)
[hobot_centerpoint-1] [WARN] [0948485759.205674972] [dnn]: Run default SetOutputParser.
[hobot_centerpoint-1] [WARN] [0948485759.205820889] [dnn]: Set output parser with default dnn node parser, you will get all output tensors and should parse output_tensors in PostProcess.
[hobot_centerpoint-1] [WARN] [0948485759.208895472] [hobot_centerpoint]: A total of 81 files were fetched!
[hobot_centerpoint-1] [WARN] [0948485759.400904472] [CenterPoint_Node]: input fps: -1.00, out fps: -1.00, infer time ms: 61, post process time ms: 57
[hobot_centerpoint-1] [WARN] [0948485759.839328014] [CenterPoint_Node]: input fps: -1.00, out fps: -1.00, infer time ms: 27, post process time ms: 53
[hobot_centerpoint-1] [WARN] [0948485760.281992264] [CenterPoint_Node]: input fps: -1.00, out fps: -1.00, infer time ms: 28, post process time ms: 53
[hobot_centerpoint-1] [WARN] [0948485760.731948223] [CenterPoint_Node]: input fps: 2.93, out fps: 3.01, infer time ms: 27, post process time ms: 56
[hobot_centerpoint-1] [WARN] [0948485761.155906223] [CenterPoint_Node]: input fps: 2.93, out fps: 3.01, infer time ms: 28, post process time ms: 56
[hobot_centerpoint-1] [WARN] [0948485761.572980640] [CenterPoint_Node]: input fps: 2.93, out fps: 3.01, infer time ms: 27, post process time ms: 53
[hobot_centerpoint-1] [WARN] [0948485761.983718973] [CenterPoint_Node]: input fps: 2.40, out fps: 2.40, infer time ms: 28, post process time ms: 55
[hobot_centerpoint-1] [WARN] [0948485762.396930973] [CenterPoint_Node]: input fps: 2.40, out fps: 2.40, infer time ms: 28, post process time ms: 55
[hobot_centerpoint-1] [WARN] [0948485762.816782057] [CenterPoint_Node]: input fps: 2.40, out fps: 2.40, infer time ms: 27, post process time ms: 56
[hobot_centerpoint-1] [WARN] [0948485763.239294099] [CenterPoint_Node]: input fps: 2.39, out fps: 2.39, infer time ms: 27, post process time ms: 57
[hobot_centerpoint-1] [WARN] [0948485763.661555807] [CenterPoint_Node]: input fps: 2.39, out fps: 2.39, infer time ms: 27, post process time ms: 57
[hobot_centerpoint-1] [WARN] [0948485764.084410183] [CenterPoint_Node]: input fps: 2.39, out fps: 2.39, infer time ms: 27, post process time ms: 57
[hobot_centerpoint-1] [WARN] [0948485764.502788849] [CenterPoint_Node]: input fps: 2.37, out fps: 2.37, infer time ms: 27, post process time ms: 55

The output log shows that the topic for publishing algorithm inference results is /hobot_centerpoint, and a total of 81 playback point cloud files are loaded. After inference and post-processing (including rendering and publishing of inference results), the algorithm runs at a frame rate of approximately 2.4fps.

Enter http://IP:8000 in the browser on the PC side to view the images and algorithm rendering effects (IP is the IP address of the RDK):

Last updated on