3D Object Detection using YOLO4 | LiDAR Dataset

This is a tutorial on how to perform 3D object detection on LiDAR Dataset.
I have used Kitti dataset in the Implementation.

Topics covered:
1- what is 3D object detection
2- Difference between 2D vs 3d Object detection
3- What is LiDAR
4- Dataset (discuss about the dataset which we are going to use in our class and how to download it)
5- Practical Implementation step by step .

2D prediction only provides 2D bounding boxes but with 3D Object detection, we can know various details of that object like size of an object, position of that object and orientation of that object. And we can use them in variety of applications in robotics, self-driving vehicles, augmented reality.
There are various 3D datasets available which are related to street scenes due to the popularity of research into self-driving cars. Those datasets are created by 3D capture sensors like LIDAR.

LiDAR use laser to measure the distance from the objects.
Lidar use Ultraviolet Visible Infrared to sense the object.

It basically checks how much time the omitted light takes to return back to the sensor and on the basis of that it gets the distance from the objects.

To run the model, you need to download and unzip the following data:

Velodyne point clouds (29 GB): Information about the surrounding for a single frame gathered by Velodyne HDL64 laser scanner. This is the primary data we use. It is 100 milliseconds snapshot of the 3d world around the car 

Left color images of object data set (12 GB): The cameras were one color camera stereo pairs. We use left Images corresponding to the velodyne point clouds for each frame.

Camera calibration matrices of object data set (16 MB): Used for calibrating and rectifying the data captured by the camera and sensor.

Training labels of object data set (5 MB).

Join this channel to get access to perks:

[bushratahir]

can you please mention which python and pip version worked with these dependencies torch==1.5.0
torchvision==0.6.0
easydict==1.9
opencv-python==4.2.0.34
numpy==1.22.0
torchsummary==1.5.1
tensorboard==2.2.1
scikit-learn==0.22.2. Also mention the mayavi, Pyqt5 and vtk versions.

[kratos]

Hello ma'am .. can i have the pre trained model please. Need it for a project and i don't have the compute resource to train the model

[eck]

I'm confused. This trained model can work with 2d images and give 3d bounding boxes with their coordinates or that model can work only with Lidar data?

[sri-d]

Hi Ma'am, How can I annotate 3D images, and are there any other models available for predicting defects in 3D images?

[paulosantosbim]

Is there any way to use my Yolov8 fine tuned model for point cloud segmentation ? This model was trained with custom images dataset

[manmj]

Thank you for your content here. What I like about this channel is that it is more about implementation and practice. Thanks for your effort and time.

[GRANATE]

your camera needs to have lidar? A normal smartphone camera stream (no lidar) is not enough for detection?

[soheilmajidi]

thanks
will be training video for custom data ?

[jamilahmarramduarte]

hi mam, how can i use this to self navigate our trash collector robot maam? because were using lidar sensor maam

[gizemce]

Hello, l would like to make object detection directly point cloud data, it means l will use camera, is it possible .

[shprai]

Mam I have downloaded torch for CPU and when I'm running the train.py in command prompt it's showing: torch not compiled with cuda enabled
What do I do mam

[JRFVRSEC]

thanku maam for this useful video, maam i want to classify this object like person, car etc how can i do??

[parthasarathyk]

Thank you so much mam. I was the first one asked to make a video on 3D object detection.

[seratmostaqueem]

hi, great work but can I try it with just a part of data not all data, I mean i use about 1000 image for image _2 and velodyne but i have error is that the reason ? the error is BrokenPipeError: [Errno 32] Broken pipe

[NERELLAAKHIL]

When I try to unzip the above datasets in Google Colab, The RAM is getting finished and runtime disconnection.Can you please tell an alternative

[shirshakpattnaik]

Is it possible to train dataset using yolov7?

[teetanrobotics]

thank you so much Aarohi for the content. please keep updating the playlists as well

[aloktiwari]

Ma'am after running test.py. my results folder is not created. Please help me with that, and can you please make a video on how to analyze these results

[NERELLAAKHIL]

@CodeWithAarohi I have a query, What do we have inside .bin files in velodyne folder in dataset. How to find the lidar 3D coordinates of detected objects.

[chittaranjanjena]

Hello, please can you provide any sensor fusion model on Lidar poaints and 2d-images? any model for object detection using lidar pcd in YOLOv5 or YOLOV7 also help full..