Real-Time Tracking of Animal Behaviour with Simulation and Machine Learning

Project description

This is a set of interconnected projects aimed at using machine learning and other computational techniques to study the neuroscience of vision and behaviour in insects (esp. hoverflies and butterflies). Students will be working with members of the Neuroscience of Insect Vision (NIVi) laboratory (College of Medicine and Public Health).

1. The NIVi lab makes use of an experimental rig that allows tethered insects to “fly” around in a virtual reality space. Navigation through the space is facilitated by real-time camera tracking of wing movements. The VR rig has since been enhanced with an open-source toolkit based on Deep Learning (DeepLabCut), to more accurately track fine movements of the insects. There is an ongoing need for computationally-minded students at Honours or Masters by Research level, especially with computer vision, AI or VR/games development skills, to further enhance these research tools in collaboration with members of the NIVi lab. This could be on the VR side, or on the motion-tracking-with-video-analysis-and-AI side.
2. Another project in the same lab aims to create a real-time visual simulation of the view a butterfly sees when flying through a vegetated environment. This could be achieved by mounting a camera on a robot arm in the Robotics Lab at Tonsley, and programming the arm to move according to known butterfly flight paths. The resulting video would then be able to be played back inside the "virtual reality" rig. This project could be suitable for a Masters Project, but could also be adapted for a suitable Research Masters or Honours candidate.

Assumed knowledge

Essential

• Strong computer programming skills.
• 1: experience in using machine learning and/or deep learning

Desirable

• 1: Experience with 3D graphics, virtual reality, games development esp. Unity
• 2: Experience with programmable robots
• Both: interest in and knowledge of animal behaviour and/or neuroscience would of course be desirable (but unexpected)!

Note: You need to register interest in projects from different supervisors (not a number of projects with the one supervisor).
You must also contact each supervisor directly to discuss both the project details and your suitability to undertake the project.