UAV Object Detection & Tracking with YOLO and Faster R-CNN

Project

This project uses the YOLO (YOLO11 or YOLOv8) and Faster R-CNN RESNET50 models to perform real-time object detection in drone (UAV) videos, while the CSRT Tracker tracking algorithm, developed by OpenCV, is used for object tracking. Once the video is successfully processed, the object is first detected and tracking is initiated with the tracker algorithm. If tracking is lost for 20 frames, the detection algorithm re-detects the object and resumes tracking. Furthermore, the object's location, distance, and performance metrics (FPS, distance, etc.) are calculated and visualized in the video.

Sample Outputs

Basic Video Output(YOLO)	Model Video Output(Faster R-CNN)

---

YOLO Basic

 --- YOLO --- <
Average FPS: 14.44

Faster R-CNN Basic

 --- FASTER R-CNN --- <
Average FPS: 20.09

YOLO Model

--- YOLO ---

First detection: airplane - Confidence Thershold: 0.73
Video playback time: 110.25 seconds
Highest FPS: 30.80
Average FPS: 20.99
Average distance: 155.05 px

Faster R-CNN Model

--- Faster R-CNN RESNET50 ---

First detection: airplane - Confidence Thershold: 0.99
Video playback time: 73.67 seconds
Highest FPS: 54.23
Average FPS: 31.93
Average distance: 154.20 px

Observations

In its basic and model versions, Faster R-CNN delivers higher speed and accuracy compared to YOLO. Because Faster R-CNN is more demanding than YOLO, it can be preferred for short-term, lightweight projects. However, for projects requiring higher hardware requirements and long-term high speed and good average accuracy, the YOLO versions (especially the YOLOv8 and YOLO11 models) are preferred due to their lightweight and user-friendly nature.

Contents

• basic_yolo_detection.py : Simple YOLO-based object detection and tracking with CSRT.
• basic_frcnn_detection.py : Simple Faster R-CNN-based object detection and tracking with CSRT.
• frcnn_uav_model.py : Drone detection with Faster R-CNN, ROI and distance calculation.
• yolo_uav_model.py : Drone detection with YOLO, ROI and distance calculation.
• videos/ : Test videos.
• outputs/ : Result videos of test videos.
• assets/ : GIFs of result videos.

Installation and Requirements

To clone the project:

git clone https://github.com/ZehraKucuker/uav_detection_and_tracking
cd uav_detection_and_tracking

Create and activate virtual environment:

For Windows:

# Create virtual environment
python -m venv .venv

# Activate virtual environment
.\.venv\Scripts\Activate.ps1

For Linux/Mac:

# Create virtual environment
python -m venv .venv

# Activate virtual environment
source .venv/bin/activate

Install required packages:

pip install --upgrade pip
pip install -r requirements.txt

To run scripts:

python basic_yolo_detection.py
python basic_frcnn_detection.py
python frcnn_uav_model.py
python yolo_uav_model.py

Required packages:

• torch>=2.0.0 - PyTorch deep learning framework
• torchvision>=0.15.0 - Computer vision package for PyTorch
• ultralytics>=8.0.0 - YOLO models implementation
• opencv-python>=4.8.0 - Computer vision and image processing
• numpy>=1.25.0 - Numerical computing
• matplotlib>=3.7.0 - Plotting and visualization
• Pillow>=10.0.0 - Image processing library
• scipy>=1.11.0 - Scientific computing