To track moving objects, such as fruit flies, with a high resolution camera, I combined a 3D realtime tracking system developed by my colleague Andrew Straw with a pan/tilt head that I built and instrumented. I used ROS (Robot Operating System) to develop a controller and DLT calibration routine for the system. This made it possible to follow, and focus on, moving objects over a large spatial volume.
Movie: Wide field view shows a rod with a bright red tip at the end. The 3D tracking system tracked the red tip, and a PID controller directed the pan/tilt/focus unit to follow the tracked object (video shown in the inset).
Fly movie: Demonstration of tracking a freely moving fruit fly using the same system.
I wrote a library of plotting functions that serve as helpful examples for scientific plots of complicated datasets, with an emphasis on trajectory type data.
Building on the python-sympy library, I wrote a python package to do symbolic lie algebra, including algebraic simplification: https://github.com/florisvb/Lie_Algebra
To better understand how flies visually experience the world in my experiments I built a python package that interfaces with Blender to output movies of how a fly sees the world, based on code originally written by my colleague Andrew Straw.
I wrote a ROS (Robot Operating System) package for 2-dimensional tracking in realtime an arbitrary number of moving objects using computer vision (opencv), kalman filtering, and data association. Data are saved as hdf5 files and analyzed with the python-pandas package. I have successfully used the system for tracking and analyzing trajectories of up to 10 flies for over 24 hours.