Stereolabs ZED
In this tutorial, we will show you how to connect and use ZED camera, on your rover.
ZED stereo camera is a type of camera capable of capturing image from two points at the same time. Using stereo disparity - the difference in object location as seen by the left and right lenses - cameras like ZED are capable of distinguishing the distance of objects from the camera. This data can then be used to in a wide array of cases. In robotics, stereo cameras are commonly used for mapping the environment, object recognition, 3D scanning and in many other.
Prerequisites
List of components To complete this tutorial, you need to have additional computer on your rover, which supports CUDA operations (for example NVIDIA Jetson), has internet connection, and ROS installed (on your and the rover computers).
Mechanical integration
As every ZED camera model has many mounting options and flat bottom, each of the camera models can be easily integrated with the rover. You can 3D print a special plate, which you will attach to the rover using mounting holes, and attach the camera to this plate. Anything that works for you is okay.
Wiring and electronics connection
To connect ZED camera to your additional computer, you need to have one free USB slot on the computer, and connect the camera with a USB cable.
Software integration
ZED SDK
First, you need to install the ZED SDK on your additional computer. Go to the stereolabs page and download installer version, that corresponds your additional computer, and its configuration.
Then, open a terminal in a directory where you have downloaded the installer script and add execution permission to it:
chmod +x ZED_SDK_<installer version>.run
And then, start the installation:
./ZED_SDK_<installer version>.run
At the beginning of the installation, you will be shown the Software License,
just hit Q to close it after reading. Next you will be asked for
administrator password, and might have to answer some questions on dependencies,
tools and samples installation. Type y
for yes and n
for no and hit
Enter (hitting Enter without typing anything chooses default option).
Remember that you need to have CUDA on your board computer, to use ZED SDK. If it's not installed, when you run ZED SDK installer, it will ask you, whether should it be installed during SDK installation. On Jetson computers CUDA is automatically installed with JetPack, but if you have any problems, try:
sudo apt install nvidia-cuda
ROS packages
The ZED ROS wrapper package
lets you use the ZED stereo cameras with ROS. It provides you data like left and
right rectified
/unrectified
images, depth map
, 3D point cloud
and more.
To install it, go to src
directory in your ROS workspace on board computer,
and clone the github repository
git clone --recursive https://github.com/stereolabs/zed-ros-wrapper.git
Now you have to install all required dependencies. Assuming you are still in
src
directory, go to the parent directory (cd ..
) and type
source devel/setup.bash
rosdep install --from-paths src -i
Now you have to build the packages
source devel/setup.bash
catkin build
If build somehow fails with error including CUDA_TOOLKIT_ROOT_DIR
, it means
you don't have CUDA installed. Try
sudo apt install nvidia-cuda
Running the package
Running the nodes is actually simple, but we want to have the data visible on other computers connected to the rovers as well. To ensure that, we need to say where (on what computer) is the ROS master. To do so, first source your ROS workspace directory
source devel/setup.bash
Path given in this command is relative assuming you are in your ROS workspace directory.
Now we need to export environment variables before running the node. This way we will communicate with the correct ROS master. To export the variables, type on your board computer:
export ROS_MASTER_URI=http://master.lan:11311
export ROS_IP=<IP address of your board computer in the rover network>
Now you are ready to launch the node
roslaunch zed_wrapper <your ZED camera model>.launch
Make sure to provide correct launch file, based by your ZED camera model. Possible options:
- zed.launch
- zedm.launch
- zed2.launch
- zed2i.launch
Now if everything is fine, you should see on every computer connected to the rover topics from the ZED node. You can list them (after sourcing ROS workspace) with
rostopic list
Examples
Visualizing the data
With the zed_wrapper
node running, you can visualize the data from ZED camera
in RViz. To do so, you need to add camera display in displays section,
and choose the correct image topic
.
![](/img/robots/leo/integrations/zed/rviz-camera-display.webp)
![](/img/robots/leo/integrations/zed/rviz-camera-topic.webp)
For the image topic, you can choose anything you want from the options. Different topics give different images, for example
rgb/image_rect_color
gives color rectified imagergb_raw/image_raw_color
gives color unrectified image
You can also choose whether you want the left or right camera sensor (it's included in the image topic name)
With this setup you can see video image from the ZED camera connected to your board computer.
You may occur low quality video, if you try to visualize the camera output like this (without processing) on other computer connected to the rover (not the one that is connected to ZED camera with USB cable).
The reason is large size of the data that you are sending. Surely changing
Transport Hint (option below camera image) to compressed
will help a bit,
but won't solve the problem completely
What next?
Stereo cameras are commonly used in projects involving autonomous navigation, you might be interested in a tutorial about it.
They are however, not the only way of teaching a Leo Rover how to move on it's own. Check out our line follower tutorial if you want to learn more. You can also check our Integrations page for more instructions.