Luxonis Oak-D
This tutorial will guide you through the process of connecting an Oak-D camera to your Leo Rover.
OAK-D is the ultimate camera for robotic vision that perceives the world like a human by combining stereo depth camera and high-resolution color camera with an on-device Neural Network inferencing and Computer Vision capabilities. It uses USB-C for both power and USB3 connectivity. In robotics, stereo cameras are commonly used for mapping the environment, object recognition, 3D scanning and in many other.
What to expect?
After completing this tutorial, you will have successfully connected an Oak-D stereo camera to the Leo Rover in ROS, and you will possess basic knowledge about the use of stereo cameras.
Prerequisites
List of components
- Oak-D Series Stereo Camera
- Any mounting solution (eg. Universal Camera Mast)
- USB-A to USB-C wire
- (optional) USB-C to DC power supply
Mechanical integration
There are multiple use cases for stereo cameras. Since we cannot predict them all, we cannot design a perfect mount for all of them. That's why we provide a Universal Camera Mast that can fit a wide range of projects.
![](/img/robots/leo/integrations/oak-d/oak-d-ucm.webp)
Wiring and electronics connection
Whether you are using our Universal Camera Mast or your own custom solution, you'll need to connect the camera to the rover somehow.
The USB port placed on the mounting plate of the rover does not support USB 3.0 that's necessary to run the camera. That's why you'll need to connect the USB-C wire straight to the USB ports of the rovers built-in Raspberry Pi.
Kacper from our team likes to remove the micro-usb port entirely and run the wire straight through the newly created hole. This solution, combined with the use of a cable gland, keeps the MEB box waterproof.
Second common solution involves printing a new cover for the MEB box. Dev cover found here, provides a wide opening for running wires straight into the MEB. This solution can be extremely useful in projects that don't need to be waterproof.
![](/img/robots/leo/addons/meb-covers/00188-dev-meb-cover.webp)
If the camera isn't the only thing connected to the rover you might need to provide an external power connection. Powerbox might come in handy in such situations.
Software integration
Installing the packages
Add USB rules to your system:
echo 'SUBSYSTEM=="usb", ATTRS{idVendor}=="03e7", MODE="0666"' | sudo tee /etc/udev/rules.d/80-movidius.rules
sudo udevadm control --reload-rules && sudo udevadm trigger
Install depthai-ros
packages(only available in ROS noetic):
sudo apt install ros-noetic-depthai-ros
Adding camera model
As we want to have the camera model visible with the rover model, we need to
change robot.urdf.xacro
file in /etc/ros/urdf
directory. Package
depthai_descriptions
is installed alongside depthai-ros
and it contains Oak
camera models and xacro macros that we can easily add.
To include the Oak camera macro add following lines to
/etc/ros/urdf/robot.urdf.xacro
file:
<xacro:include filename="$(find depthai_descriptions)/urdf/include/depthai_macro.urdf.xacro"/>
<!-- TODO add correct cam position-->
<xacro:depthai_camera camera_name="oak"
camera_model="OAK-D-PRO"
base_frame="oak-d_frame"
parent="base_link"
cam_pos_x="0.0"
cam_pos_y="0.0"
cam_pos_z="0.0"
cam_roll="0.0"
cam_pitch="0.0"
cam_yaw="0.0"/>
This macro contains a lot of parameters that are set to a default Oak-D Pro camera setup on a rover. You can adjust those parameters to your needs (e.g. if you have the camera mounted in a different position or you are using a different camera model).
In the end, your robot.urdf.xacro
file should look like this:
<?xml version="1.0" encoding="utf-8"?>
<robot xmlns:xacro="http://www.ros.org/wiki/xacro"
name="leo">
<xacro:include filename="$(find leo_description)/urdf/macros.xacro"/>
<xacro:leo default_antenna="true"
rockers_fixed="true"
footprint_link="true"
link_prefix=""
joint_prefix=""/>
<xacro:include filename="$(find depthai_descriptions)/urdf/include/depthai_macro.urdf.xacro"/>
<!-- TODO add correct cam position-->
<xacro:depthai_camera camera_name="oak"
camera_model="OAK-D-PRO"
base_frame="oak-d_frame"
parent="base_link"
cam_pos_x="0.0"
cam_pos_y="0.0"
cam_pos_z="0.0"
cam_roll="0.0"
cam_pitch="0.0"
cam_yaw="0.0"/>
</robot>
Creating launch and configuration files
To launch the camera you can, create your own launch file, use one of the
example packages from depthai-ros
package or you can use the provided launch
file which provides basic functionalities such as camera image and point cloud
created from depth data.
In /etc/ros
create a .yaml
configuration file. This example contains
configuration for Oak-D Pro point cloud mapping.
/oak:
camera_i_nn_type: none
camera_i_laser_dot_brightness: 1000
camera_i_enable_imu: true
camera_i_enable_ir: true
imu_i_acc_freq: 50
imu_i_rot_freq: 50
imu_i_gyro_freq: 50
rgb_i_fps: 10.0
rgb_i_resolution: '720P'
left_i_publish_topic: false
left_i_fps: 10.0
right_i_publish_topic: false
right_i_fps: 10.0
stereo_i_align_depth: true
stereo_i_subpixel: true
stereo_i_stereo_conf_threshold: 110
stereo_i_enable_brightness_filter: false
stereo_i_brightness_filter_min_brightness: 1
stereo_i_brightness_filter_max_brightness: 225
stereo_i_enable_decimation_filter: true
stereo_i_publish_right_rect: true
Next, also in /etc/ros
create a launch file that will use the previously added
configuration and will create a nodelet
manager and appropriate nodelets.
Nodelets are used for communication efficiency.
<?xml version="1.0"?>
<launch>
<arg name="name" default="oak" />
<arg name="params_file" default="/etc/ros/leo_oak_pcl.yaml"/>
<rosparam file="$(arg params_file)" />
<node pkg="rosservice" if="$(optenv DEPTHAI_DEBUG 0)" type="rosservice" name="set_log_level" args="call --wait /oak_nodelet_manager/set_logger_level 'ros.depthai_ros_driver' 'debug'" />
<node pkg="nodelet" type="nodelet" name="$(arg name)_nodelet_manager" args="manager" output="screen">
<remap from="/nodelet_manager/load_nodelet" to="$(arg name)/nodelet_manager/load_nodelet"/>
<remap from="/nodelet_manager/unload_nodelet" to="$(arg name)/nodelet_manager/unload_nodelet"/>
<remap from="/nodelet_manager/list" to="$(arg name)/nodelet_manager/list"/>
</node>
<node name="$(arg name)" pkg="nodelet" type="nodelet" output="screen" required="true" args="load depthai_ros_driver/Camera $(arg name)_nodelet_manager"/>
<node pkg="nodelet" type="nodelet" name="depth_image_to_rgb_pointcloud"
args="load depth_image_proc/point_cloud_xyzrgb $(arg name)_nodelet_manager">
<param name="queue_size" value="1"/>
<remap from="rgb/camera_info" to="$(arg name)/rgb/camera_info"/>
<remap from="rgb/image_rect_color" to="$(arg name)/rgb/image_raw"/>
<remap from="depth_registered/image_rect" to="$(arg name)/stereo/image_raw"/>
<remap from="depth_registered/points" to="$(arg name)/points"/>
</node>
</launch>
For more information about oak configuration and usage examples you can check out the depthai-ros github repository.
Launching camera nodes on system startup
Include your launch file in the robot.launch
file, so that your node will
start at boot.
In /etc/ros/robot.launch
add this line between <launch>
tags:
<include file="/etc/ros/leo_oak_pcl.launch"/>
Now your Oak-D camera ROS nodes will start at launch.
Examples
Reading and visualizing the data
The topics from camera node should be now visible. You can check them with
rostopic
tool - topics from your node will have prefix specified by you in the
camera
ROS argument.
If you have ROS installed on your computer, you can get the camera image with rqt, and see the robot model with camera attached using rviz.
Before starting any of the programs, make sure you completed the Connecting another computer to ROS network of ROS development tutorial:
Now you can start rqt
, and from plugins
-> visualization
choose
image view
and then choose your topic.
Don't worry if the quality is low, and the image is lagging. It is due to transfer of the data through your rover and computer.
You can also start RViz, and add robot model in the display view. Chose
also base_link
as fixed frame, and you should see the rover model with
camera attached.
You can also display tf tree
in rqt, to see if everything is configured
correctly (plugins -> visualization -> TF Tree)
![](/img/robots/leo/integrations/oak-d/tf-tree-oak-d.webp)
What next?
Stereo cameras are commonly used in projects involving autonomous navigation, you might be interested in a tutorial about it.
They are however, not the only way of teaching a Leo Rover how to move on it's own. Check out our line follower tutorial if you want to learn more. You can also check our Integrations page for more instructions.