Skip to main content

BEV Perception Algorithm

Function Introduction​

The BEV Perception Algorithm is a BEV multi-task model trained on the nuscenes dataset using OpenExplorer.

The algorithm takes six sets of image data as input, namely the front, front-left, front-right, rear, rear-left and rear-right views. The model outputs 10 categories of targets with their corresponding 3D detection bounding boxes (including obstacles, various types of vehicles, traffic signs, etc.), as well as semantic segmentation of lane lines, sidewalks and road edges.

This example uses local image data as input, leverages the BPU for algorithm inference, and publishes image messages with rendered algorithm perception results, which are displayed in a browser on the PC side.

Code Repository: (https://github.com/D-Robotics/hobot_bev.git)

Supported Platforms​

PlatformRunning MethodExample Function
RDK UltraUbuntu 20.04 (Foxy)Local playback with inference rendering results displayed via web
RDK S100, RDK S100PUbuntu 22.04 (Humble)Local playback with inference rendering results displayed via web

Preparation​

  1. The RDK has been flashed with the Ubuntu 20.04/Ubuntu 22.04 system image.
  2. TogetheROS.Bot has been successfully installed on the RDK.
  3. Ensure the PC can access the RDK via the network.

Usage Guide​

Local Dataset Playback​

This example uses local dataset playback for inference, then publishes image messages with rendered algorithm results. The published images and corresponding algorithm results are displayed in a browser on the PC side via the websocket package.

Prepare Playback Dataset

# Download the dataset on the board
wget http://archive.d-robotics.cc/TogetheROS/data/hobot_bev_data.tar.gz

# Decompress the file
mkdir -p hobot_bev_data
tar -zxvf hobot_bev_data.tar.gz -C hobot_bev_data

# The decompressed dataset is in the hobot_bev_data/data directory

Run Dataset Playback

# Configure the TogetheROS.Bot environment
source /opt/tros/setup.bash

# Launch the websocket service
ros2 launch websocket websocket_service.launch.py

# Launch the run script and specify the dataset path
ros2 launch hobot_bev hobot_bev.launch.py image_pre_path:=hobot_bev_data/data

Result Analysis​

The following information is output in the running terminal:

[INFO] [launch]: All log files can be found below /root/.ros/log/2023-07-05-17-47-07-232907-hobot-2627970
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [hobot_bev-1]: process started with pid [2627972]
[INFO] [websocket-2]: process started with pid [2627974]
[hobot_bev-1] [WARN] [1688579227.907268364] [bev_node]:
[hobot_bev-1]  image_pre_path: hobot_bev_data/data
[hobot_bev-1] [BPU_PLAT]BPU Platform Version(1.3.3)!
[hobot_bev-1] [HBRT] set log level as 0. version = 3.14.25.0
[hobot_bev-1] [DNN] Runtime version = 1.12.3_(3.14.25 HBRT)
[hobot_bev-1] [WARN] [1688579228.714778531] [dnn]: Run default SetOutputParser.
[hobot_bev-1] [WARN] [1688579228.714925489] [dnn]: Set output parser with default dnn node parser, you will get all output tensors and should parse output_tensors in PostProcess.
[hobot_bev-1] [WARN] [1688579228.886846489] [bev_node]: loop 0/1002
[hobot_bev-1] [WARN] [1688579229.474568573] [bev_node]: loop 1/1002
[hobot_bev-1] [WARN] [1688579230.058551781] [bev_node]: loop 2/1002
[hobot_bev-1] [WARN] [1688579230.691667198] [bev_node]: loop 3/1002
[hobot_bev-1] [WARN] [1688579231.324658782] [bev_node]: loop 4/1002
[hobot_bev-1] [WARN] [1688579231.365145532] [bev_node]: input fps: 2.47, out fps: 2.52, infer time ms: 12, post process time ms: 659
[hobot_bev-1] [WARN] [1688579231.915645741] [bev_node]: loop 5/1002
[hobot_bev-1] [WARN] [1688579231.996993824] [bev_node]: input fps: 2.47, out fps: 2.52, infer time ms: 12, post process time ms: 658

Enter http://IP:8000 in the browser on the PC side to view the images and algorithm rendering effects (IP refers to the IP address of the RDK):

Last updated on