Automated High Resolution Pan/Tilt/Focus Tracking

To track moving objects, such as fruit flies, with a high resolution camera, I combined a 3D realtime tracking system developed by my colleague Andrew Straw with a pan/tilt head that I built and instrumented. I used ROS (Robot Operating System) to develop a controller and DLT calibration routine for the system. This made it possible  to follow, and focus on, moving objects over a large spatial volume.

Movie: Wide field view shows a rod with a bright red tip at the end. The 3D tracking system tracked the red tip, and a PID controller directed the pan/tilt/focus unit to follow the tracked object (video shown in the inset).

Fly movie: Demonstration of tracking a freely moving fruit fly using the same system.

Multi Object Camera Tracking

I wrote a ROS (Robot Operating System) package for 2-dimensional tracking in realtime an arbitrary number of moving objects using computer vision (opencv), kalman filtering, and data association. Data are saved as hdf5 files and analyzed with the python-pandas package. I have successfully used the system for tracking and analyzing trajectories of up to 10 flies for over 24 hours.

Codebase:  https://github.com/florisvb/multi_tracker