Sky Scout
Overview
Sky Scout democratizes drone operations by allowing anyone to control drones using plain English commands. Instead of complex joysticks or mission planning software, users can simply say "inspect the north wall for cracks" or "find all red vehicles in the parking lot," and Sky Scout translates these natural language requests into safe, autonomous drone missions. This was my first foray into ROS2 (Robot Operating System) and building a complete robotics system from scratch.
The Challenge
The drone industry has a usability problem. Current drone control systems require extensive training on complex GCS interfaces, understanding of waypoint navigation, manual piloting skills, and technical knowledge to interpret sensor data. I wanted to create a system where domain experts could leverage drones without becoming drone experts.
My Approach
I built a reliable bridge between human intent expressed in natural language and the precise control commands drones require. This involved integrating LLMs for command understanding, implementing a ROS2 architecture for real-time control, adding computer vision for object detection, and creating a web interface for mission monitoring—all while ensuring safety and mission success. As my first hardware build, I assembled a Holybro X500 V2 drone from scratch, complete with Pixhawk 6C flight controller, carbon fiber frame, and ExpressLRS radio system.
The Solution
Sky Scout features a modular ROS2 architecture with custom nodes for mission planning and execution, real-time computer vision pipeline with YOLO object detection, natural language processing using OpenAI and Gemini APIs, and a Next.js web interface with live telemetry visualization. The system supports various mission types including search, inspect, count, map, delivery, patrol, survey, and emergency response.
Technologies Used
Python 3.10+
Next.js 15
OpenAI/Gemini APIs
TypeScriptOutcomes
Successfully demonstrated natural language control of autonomous drones, reducing mission planning time from minutes to seconds. The system makes drone operations accessible to non-technical users like construction managers, farmers, and emergency responders. The modular architecture can adapt to different drone platforms and has been tested in simulation, ready for real-world deployment on Pixhawk-based drones.
Project Gallery
