Project information
- Category AI · Computer Vision
- Stack Unity 3D + Qwen2.5 7B (local LLM)
- Platform Local - MacBook Pro M2
- Video YouTube link
About the project
An Embodied AI drone simulation built in Unity 3D, with Qwen2.5 7B running locally as the drone's brain. Tasks can be assigned through a command prompt, and the drone acts accordingly.
So far, I've implemented one hazard scenario: fire detection and response. When the fire button is clicked, a fire spawns at a random location in the environment. The drone is then given a command: "Do patrol. If you find anything unusual, take action."
The drone observes its surroundings and reasons about what to do next. The LLM is provided with information about the drone's available capabilities and tools:
- Moving in different directions.
- Splashing water.
- Sounding an alarm.
Based on the situation, the LLM generates an action sequence and sends it back to the Unity simulation for execution. Everything runs locally on a MacBook Pro M2, so the reasoning and execution take some time due to limited computing resources - that's why the video is fast-forwarded.