Face 106 Landmarks Detection
Feature Overview
The Face 106 Landmarks Detection example subscribes to image data and smart messages containing face bounding box information, performs algorithm inference using the BPU, and publishes algorithm messages containing face 106 landmarks information.
Code repository: (https://github.com/D-Robotics/face_landmarks_detection)
Supported Platforms
| Platform | Runtime Environment | Example Functionality |
|---|---|---|
| RDK X3, RDK X3 Module | Ubuntu 22.04 (Humble) | Launch MIPI/USB camera and display inference results via Web |
| RDK X5, RDK X5 Module | Ubuntu 22.04 (Humble) | Launch MIPI/USB camera and display inference results via Web |
Algorithm Details
| Model | Platform | Input Size | Inference FPS |
|---|---|---|---|
| faceLandmark106pts | X3 | 1×3×128×128 | 741.93 |
| faceLandmark106pts | X5 | 1×3×128×128 | 985.75 |
Prerequisites
RDK Platform
- RDK has been flashed with the Ubuntu 22.04 system image.
- TogetheROS.Bot has been successfully installed on the RDK.
- An MIPI or USB camera has been installed on the RDK.
- Ensure your PC can access the RDK over the network.
Usage Instructions
The Face 106 Landmarks Detection (face_landmarks_detection) package subscribes to images published by the sensor package and face bounding box detection results published by the human detection and tracking package. After performing inference, it publishes algorithm messages, which are rendered and displayed in a web browser on the PC via the websocket package.
Publishing Images Using an MIPI Camera
# Configure the tros.b environment
source /opt/tros/humble/setup.bash
# Copy required configuration files for running the example from the tros.b installation path.
cp -r /opt/tros/${TROS_DISTRO}/lib/mono2d_body_detection/config/ .
# Configure MIPI camera
export CAM_TYPE=mipi
# Launch the launch file
ros2 launch face_landmarks_detection body_det_face_landmarks_det.launch.py
Publishing Images Using a USB Camera
# Configure the tros.b environment
source /opt/tros/humble/setup.bash
# Copy required configuration files for running the example from the tros.b installation path.
cp -r /opt/tros/${TROS_DISTRO}/lib/mono2d_body_detection/config/ .
# Configure USB camera
export CAM_TYPE=usb
# Launch the launch file
ros2 launch face_landmarks_detection body_det_face_landmarks_det.launch.py
Result Analysis
The following output appears in the terminal when running:
[mono2d_body_detection-3] [WARN] [1731988336.541394391] [example]: This is mono2d body det example!
[face_landmarks_detection-5] [WARN] [1731988336.637554206] [face_landmarks_det_node]: => face_landmarks_det_node params:
[face_landmarks_detection-5] => feed_type: 0
[face_landmarks_detection-5] => is_sync_mode: 0
[face_landmarks_detection-5] => model_file_name: /root/zhikang.zeng/work_humble_ws_x5/tros_ws/install/share/face_landmarks_detection/config/faceLandmark106pts.hbm
[face_landmarks_detection-5] => is_shared_mem_sub: 1
[face_landmarks_detection-5] => dump_render_img: 0
[face_landmarks_detection-5] => ai_msg_pub_topic_name: /hobot_face_landmarks_detection
[face_landmarks_detection-5] [INFO] [1731988336.638429674] [dnn]: Node init.
[face_landmarks_detection-5] [INFO] [1731988336.638482188] [face_landmarks_det_node]: => Set node para.
[face_landmarks_detection-5] [INFO] [1731988336.638589050] [dnn]: Model init.
[mono2d_body_detection-3] [WARN] [1731988336.641041791] [mono2d_body_det]: Parameter:
[mono2d_body_detection-3] is_sync_mode_: 0
[mono2d_body_detection-3] model_file_name_: config/multitask_body_head_face_hand_kps_960x544.hbm
[mono2d_body_detection-3] is_shared_mem_sub: 1
[mono2d_body_detection-3] ai_msg_pub_topic_name: /hobot_mono2d_body_detection
[mono2d_body_detection-3] ros_img_topic_name: /image_raw
[mono2d_body_detection-3] image_gap: 1
[face_landmarks_detection-5] [BPU_PLAT]BPU Platform Version(1.3.6)!
[face_landmarks_detection-5] [HBRT] set log level as 0. version = 3.15.54.0
[face_landmarks_detection-5] [DNN] Runtime version = 1.23.10_(3.15.54 HBRT)
[mono2d_body_detection-3] [BPU_PLAT]BPU Platform Version(1.3.6)!
[mono2d_body_detection-3] [HBRT] set log level as 0. version = 3.15.54.0
[mono2d_body_detection-3] [DNN] Runtime version = 1.23.10_(3.15.54 HBRT)
The logs indicate that the program is running successfully. During inference, both input and output frame rates are 30 fps, and the FPS statistics refresh once per second.
Enter http://IP:8000 in your PC’s web browser to view the rendered images and algorithm results (replace IP with the RDK’s IP address):
