Skip to main content

Stereolabs ZED

In this tutorial, we will show you how to connect and use ZED camera, on your rover.

ZED stereo camera is a type of camera capable of capturing image from two points at the same time. Using stereo disparity - the difference in object location as seen by the left and right lenses - cameras like ZED are capable of distinguishing the distance of objects from the camera. This data can then be used to in a wide array of cases. In robotics, stereo cameras are commonly used for mapping the environment, object recognition, 3D scanning and in many other.

Prerequisites​

📄Connect to Leo Rover AP
Learn how to connect to your Leo Rover via WiFi.
📄Connect via SSH
Learn how to establish an SSH connection with your Leo Rover and access its terminal using Putty or OpenSSH.
📄Install ROS on your computer
Learn how to install the Robot Operating System (ROS) on your computer. Step-by-step guide for beginners.

List of components To complete this tutorial, you need to have additional computer on your rover, which supports CUDA operations (for example NVIDIA Jetson), has internet connection, and ROS installed (on your and the rover computers).

Mechanical integration​

As every ZED camera model has many mounting options and flat bottom, each of the camera models can be easily integrated with the rover. You can 3D print a special plate, which you will attach to the rover using mounting holes, and attach the camera to this plate. Anything that works for you is okay.

Wiring and electronics connection​

To connect ZED camera to your additional computer, you need to have one free USB slot on the computer, and connect the camera with a USB cable.

Software integration​

ZED SDK​

First, you need to install the ZED SDK on your additional computer. Go to the stereolabs page and download installer version, that corresponds your additional computer, and its configuration.

Then, open a terminal in a directory where you have downloaded the installer script and add execution permission to it:

chmod +x ZED_SDK_<installer version>.run

And then, start the installation:

./ZED_SDK_<installer version>.run

At the beginning of the installation, you will be shown the Software License, just hit Q to close it after reading. Next you will be asked for administrator password, and might have to answer some questions on dependencies, tools and samples installation. Type y for yes and n for no and hit Enter (hitting Enter without typing anything chooses default option).

info

Remember that you need to have CUDA on your board computer, to use ZED SDK. If it's not installed, when you run ZED SDK installer, it will ask you, whether should it be installed during SDK installation. On Jetson computers CUDA is automatically installed with JetPack, but if you have any problems, try:

sudo apt install nvidia-cuda

ROS packages​

The ZED ROS wrapper package lets you use the ZED stereo cameras with ROS. It provides you data like left and right rectified/unrectified images, depth map, 3D point cloud and more. To install it, go to src directory in your ROS workspace on board computer, and clone the github repository

git clone --recursive https://github.com/stereolabs/zed-ros-wrapper.git

Now you have to install all required dependencies. Assuming you are still in src directory, go to the parent directory (cd ..) and type

source devel/setup.bash
rosdep install --from-paths src -i

Now you have to build the packages

source devel/setup.bash
catkin build
note

If build somehow fails with error including CUDA_TOOLKIT_ROOT_DIR, it means you don't have CUDA installed. Try

sudo apt install nvidia-cuda

Running the package​

Running the nodes is actually simple, but we want to have the data visible on other computers connected to the rovers as well. To ensure that, we need to say where (on what computer) is the ROS master. To do so, first source your ROS workspace directory

source devel/setup.bash
note

Path given in this command is relative assuming you are in your ROS workspace directory.

Now we need to export environment variables before running the node. This way we will communicate with the correct ROS master. To export the variables, type on your board computer:

export ROS_MASTER_URI=http://master.lan:11311
export ROS_IP=<IP address of your board computer in the rover network>

Now you are ready to launch the node

roslaunch zed_wrapper <your ZED camera model>.launch
warning

Make sure to provide correct launch file, based by your ZED camera model. Possible options:

  • zed.launch
  • zedm.launch
  • zed2.launch
  • zed2i.launch

Now if everything is fine, you should see on every computer connected to the rover topics from the ZED node. You can list them (after sourcing ROS workspace) with

rostopic list

Examples​

Visualizing the data​

With the zed_wrapper node running, you can visualize the data from ZED camera in RViz. To do so, you need to add camera display in displays section, and choose the correct image topic.

For the image topic, you can choose anything you want from the options. Different topics give different images, for example

  • rgb/image_rect_color gives color rectified image
  • rgb_raw/image_raw_color gives color unrectified image

You can also choose whether you want the left or right camera sensor (it's included in the image topic name)

With this setup you can see video image from the ZED camera connected to your board computer.

info

You may occur low quality video, if you try to visualize the camera output like this (without processing) on other computer connected to the rover (not the one that is connected to ZED camera with USB cable).

The reason is large size of the data that you are sending. Surely changing Transport Hint (option below camera image) to compressed will help a bit, but won't solve the problem completely

What next?​

Stereo cameras are commonly used in projects involving autonomous navigation, you might be interested in a tutorial about it.

They are however, not the only way of teaching a Leo Rover how to move on it's own. Check out our line follower tutorial if you want to learn more. You can also check our Integrations page for more instructions.