In this project I’ve developed a 3D object detection and tracking pipeline for autonomous driving with Python and ROS without any labeled 3D data. Our hardware were only a Livox mid-100 lidar sensor and a RealSense camera. After calibrating the sensors, the pipeline is as follows:
- Get 2D instance segmentation of the current camera image using YOLACT.
- Update the id numbers of the detected entitites using SORT
- Project the masks into 3D point cloud using LDLS
- Compute the 3D bounding boxes for the detected areas.
Source code can be found here page. For the implementation with ROS, please get in touch.