Skip to main content

5.4.6 Voice-Controlled Car Movement

Feature Introduction​

The voice-controlled car movement feature enables controlling the robot to move forward, backward, left, or right through voice commands. This functionality requires integration with the D-Robotics RDK robotic operating system's intelligent voice module. The workflow is illustrated in the figure below:

This application uses a simulated virtual car in the Gazebo environment on a PC as an example. The published control commands can also be directly applied to physical robots.

Code repository: (https://github.com/D-Robotics/audio_control.git)

Supported Platforms​

PlatformRuntime EnvironmentExample Functionality
RDK X3Ubuntu 20.04 (Foxy), Ubuntu 22.04 (Humble)Launch the intelligent voice module to parse voice commands and perform voice control; demonstrate control effects via Gazebo
RDK X5, RDK X5 ModuleUbuntu 22.04 (Humble)Launch the intelligent voice module to parse voice commands and perform voice control; demonstrate control effects via Gazebo

Note: Only RDK X3 is supported; RDK X3 Module is currently not supported.

Prerequisites​

RDK Platform​

  1. RDK has been flashed with Ubuntu 20.04 or Ubuntu 22.04 image.

  2. TogetheROS.Bot has been successfully installed on RDK.

  3. The intelligent voice algorithm package has been successfully installed on RDK. Installation commands:

    sudo apt update
    sudo apt install tros-hobot-audio
Note

If the sudo apt update command fails or returns an error, refer to the FAQ section Common Issues, specifically Q10: How to resolve issues when apt update fails or reports errors?

  1. A compatible audio board has been properly connected to the RDK (refer to the Intelligent Voice chapter).

  2. A PC on the same network segment as the RDK (either wired or connected to the same Wi-Fi network; the first three segments of the IP addresses must match). The following environments must be installed on the PC:

  • Ubuntu 20.04 system and ROS2 Foxy Desktop
  • Gazebo and Turtlebot3-related packages. Installation commands:
sudo apt-get install ros-foxy-gazebo-*
sudo apt install ros-foxy-turtlebot3
sudo apt install ros-foxy-turtlebot3-simulations

Usage Instructions​

Start the simulation environment on the PC:

source /opt/ros/foxy/setup.bash
export TURTLEBOT3_MODEL=burger
ros2 launch turtlebot3_gazebo empty_world.launch.py

After successful launch, the simulated car appears as follows in the Gazebo environment:

Launch the program on the RDK platform:

  1. Copy the audio configuration files
# Configure the tros.b environment
source /opt/tros/setup.bash
# Copy required configuration files for the example from the tros.b installation path.
cp -r /opt/tros/${TROS_DISTRO}/lib/hobot_audio/config/ .

  1. Verify the microphone device

    The microphone device ID is configured via the micphone_name field in the configuration file config/audio_config.json. The default value is "hw:0,0", indicating audio device Card 0, Device 0. You can check available devices using the command ls /dev/snd, e.g., "pcmC0D1c": the trailing "c" denotes a capture device, "C0" indicates Card 0, and "D1" indicates Device 1. In this case, set the parameter to "hw:0,1".

  2. Launch the program

    # Configure the tros.b environment
    source /opt/tros/setup.bash
    # Launch the launch file
    ros2 launch audio_control audio_control.launch.py

    After successful startup, you can control the car’s movement using voice commands such as “Move forward,” “Move backward,” “Turn left,” “Turn right,” and “Stop moving.”

Result Analysis​

The RDK terminal outputs the following information upon execution:

        This is audio control package.

============================================
        audio control usage

Wake up device is "D-Robotics Hello".
Audio control command word definitions are:
        "Move forward": move front.
        "Move backward": move back.
"Turn right": rotate robot to right.  
        "Turn left": rotate robot to left.  
============================================

The above log captures a segment of output after launching the audio control package. The log shows that the wake-up word configured for this voice control module is "D-Robotics hello," and the command words for controlling the robot car's movement include: "Move forward," "Move backward," "Turn left," and "Turn right."

On the PC side, you can use the terminal command ros2 topic list to query the RDK's topic information:

$ ros2 topic list
/audio_smart
/cmd_vel

Here, /audio_smart is the algorithm message published by X3 containing smart voice recognition results, and /cmd_vel is the motion control command published by the RDK.

On the PC side, you can use the terminal command ros2 topic echo /cmd_vel to view the motion control commands published by the RDK:

linear:
  x: 0.30000001192092896
  y: 0.0
  z: 0.0
angular:
  x: 0.0
  y: 0.0
  z: 0.0
---
linear:
  x: 0.0
  y: -0.30000001192092896
  z: 0.0
angular:
  x: 0.0
  y: 0.0
  z: -0.5
---
linear:
  x: 0.0
  y: 0.30000001192092896
  z: 0.0
angular:
  x: 0.0
  y: 0.0
  z: 0.5
---

In the PC-side simulation environment, the robot car moves according to the voice control commands. The simulated robot car's motion is shown below:

Last updated on