Setup Kinect V2 in ROS for Jetson Nano

1. Hardware Hacking
2. libfreenect2
3. iai_Kinect2
4. Others

Hardware Hacking

Kinect V2 is originally designed for Microsoft XBOX One, and it’s capable to connect to console in a single wire with a special port (which is similar to USB 3.0 type B, but with additional pin ). However, KinectV2 developer kit add an additional power adapter accepting 110V-220V AC and a HUB to transfer the special port to generic USB 3.0 type B, which makes it impossible to be mounted on a tiny robotic vehicle. I have done a fews tests to prove the special is just an USB 3.0 type B with additional 12V DC power in, and the HUB is just a USB 3.0 Hub, of which the only function is to trigger a LED to indicate whether USB is plugged in. (Microsoft, great works) So, I tear down the shield of KinectV2 and weld a power cable with XT-30 connector and buy a USB 3.0 type B to USB 3.0 type C cable. The total costs comes to $10, saving official adapter and HUB. It works perfectly!

libfreenect2

Libfreenect2 is an open source Kinect V2 driver that supports multi-platforms.
The most recent update driver to 5 years ago, the dependence is quite outdated.

• Make sure that cmake .. can find CUDA dependence.
• Disable OpenCL in CMakeLists.txt, which because JetPack doesn’t have OpenCL dependence, and ./Protonect cl lead errors. It’s fine to clone the source code of OpenCL and compile locally, but it’s not necessary.
• Build with cmake .. -DENABLE_CXX11=ON -DCUDA_PROPAGATE_HOST_FLAGS=off -DCMAKE_INSTALL_PREFIX=$HOME/freenect2
• Don’t forget make install, and library should goes to $HOME/freenect2
• Install USB driver for Kinect V2 sudo cp ../platform/linux/udev/90-kinect2.rules /etc/udev/rules.d/
• Note that Kinect V2 only supports USB 3, and some USB-C to USB-A is sensitive for flipping.
• If tegraxusb reports error when launching Pronect, try to disable it in CMakeLists.txt

iai_Kinect2

This is a bridge connecting libfreenect2 driver to ROS. The original iai_kinect2 is not compatible with OpenCV 4, so the modified fork is a better choice.
When compiling, OpenCV is still reporting warning on not finding OpenCV 3.2 but OpenCV 4, it’s just fine.
Make sure libfreenect2 is tested before.
My current environment contains a self compiled OpenCL with this guide. If errors occurs when launching kinect2_bridge, try to compile and install OpenCL.

Others

• When streaming Kinect2 images, only depth images’s process is on CUDA, which makes it a very high CPU usage, and when streaming 2 videos in parallel, serval packet loss occurs.
• Compressed videos comes with significant lower latency.