ROS Webcam demo on Macbook running Ubuntu

  1. Instructions to install Ubuntu on an external SSD (takes about an hour).
  2. Run this shell script to (takes about 10-15 min):
    • Install ROS (and a few other things)
    • Set up your Macbook’s built in webcam (more detail)
  3. Demo rostopic pub / echo:
    $ roscore # starts roscore
    $ rostopic pub test std_msgs/String 'this is a test' # publishes a message to a topic
    $ rostopic list # lists all active topics
    $ rostopic echo /test # live feed of data on topic
  4. Demo multiple machines: note both machines must have full connectivity (ssh)
    • On master:
      $ ifconfig # take note of your master’s ip address
      $ export ROS_IP=master.ip.address # can put in ~/.bashrc to make persistent
      $ echo $ROS_IP # check to see what that environmental variable is set to
      $ roscore # start roscore
    • On client:
      $ ifconfig # take note of the client’s ip address
      $ export ROS_IP=client.ip.address
      $ export ROS_MASTER_URI=http://master.ip.address:11311
    • Accessing a remote machine:
      $ ssh username@ip.address # must have openssh-server installed on remote machine
  5. Follow these instructions to set up a catkin_workspace
  6. The catkin instructions above do not mention to do this, but for most simple purposes, I suggest editing your .bashrc file:
    • The .bashrc file is a shell script that is run every time you open a new terminal window. Use a text editor (like gedit, nano, vim) to edit the ~/.bashrc file (gedit ~/.bashrc).
    • Add this line to the bottom (replace USERNAME):
      source /home/USERNAME/catkin_ws/devel/setup.bash
    • Reload your .bashrc by opening a new terminal window, or typing
      $ . ~/.bashrc
  7. Get ROS access to the webcam with usb_cam:
    • Check that camera is working
      $ guvcviewer
    • Install usb_cam
      $ cd ~/catkin_ws/src
      $ git clone https://github.com/ros-drivers/usb_cam.git
      $ cd ~/catkin
      $ catkin_make
    • Run usb_cam:
      $ cd ~/catkin_ws/src/usb_cam/launch
      $ roslaunch usb_cam-test.launch
    • Explore ROS environment:
      $ rosnode list
      $ rqt_graph
      $ rostopic list
      $ rostopic echo imagetopic
      $ roshz imagetopic
  8. Live image processing
    • Download optic flow example
      $ cd ~/catkin_ws/src
      $ git clone https://github.com/florisvb/optic_flow_example
      $ cd ~/catkin
      $ catkin_make
    • Explore directory structure (more detail)
    • Explore mono conversion node
    • Run mono conversion node
      $ rosrun optic_flow_example color_to_mono.py
      $ rostopic list
      $ rosrun image_view image_view image:=/imagetopic
    • Explore optic flow node
    • Run optic flow node
    • Edit optic flow node to also publish a single average value
  9. Remotely control a motor with live image data
  10. Real-time multi-target blob tracking
  11. Rosbag record, rosbag play, load data as hdf5
  12. Highlight modularity

Accessing Mac webcam in ROS (running Ubuntu on the Mac)

Here I show how to get access to the built in webcam on a MacBook in ROS (Robot Operating System).

This post assumes you have followed these two previous posts:
1. Install Ubuntu on an external USB drive so that you can run it from your MacBook
2. Installing ROS

Update

In the terminal, run the following two commands:

$ sudo apt-get update
$ sudo apt-get upgrade

Give Ubuntu access to your Mac’s webcam

From the terminal, run these commands:

$ cd /etc/local/src
$ git clone https://github.com/patjak/bcwc_pcie.git
$ cd bcwc_pcie/firmware
$ sudo make
$ sudo make install
$ cd ..
$ sudo make
$ sudo install
$ sudo depmod
$ sudo modprobe -r bdc_pci
$ sudo modprobe facetimehd

These instructions come from medium and askubuntu. Note: I received an error on the second install command, but it still works fine.

Test that the camera is working

Install guvcview:

$ sudo apt-get install guvcview

Then run it (by typing gucview into the terminal). If all goes well, a live view feed will pop up!

Install the usb_cam ROS driver:

To get access we will use the usb_cam driver. These steps assume that you have already built a catkin workspace, called catkin_ws.

$ cd ~/catkin_ws/src
$ git clone https://github.com/ros-drivers/usb_cam.git
$ cd ~/catkin
$ catkin_make

Launch the camera driver / viewer

$ cd ~/catkin_ws/src/usb_cam/launch
$ roslaunch usb_cam-test.launch

If all is good, a live video feed will now pop up. Note that the launch file spins up two nodes: usb_cam and image_view.

Explore the ROS topics

Type “rostopic list” into the terminal window to see a list of the ros topics that are available to you in the ROS ecosystem.

Install ROS on Ubuntu (on an external drive on your MacBook)

This post picks up from: how to install Ubuntu on an external USB drive so that you can run it from your MacBook.

  1. Follow these instructions to install ROS Kinetic on Ubuntu.
  2. Follow these instructions to set up a catkin_workspace
  3. The catkin instructions above do not mention to do this, but for most simple purposes, I suggest editing your .bashrc file:
    • The .bashrc file is a shell script that is run every time you open a new terminal window. Use a text editor (like gedit, nano, vim) to edit the ~/.bashrc file (gedit ~/.bashrc).
    • Add this line to the bottom (replace USERNAME): source /home/USERNAME/catkin_ws/devel/setup.bash
    • Reload your .bashrc by opening a new terminal window, or typing “. ~/.bashrc”

Automated High Resolution Pan/Tilt/Focus Tracking

To track moving objects, such as fruit flies, with a high resolution camera, I combined a 3D realtime tracking system developed by my colleague Andrew Straw with a pan/tilt head that I built and instrumented. I used ROS (Robot Operating System) to develop a controller and DLT calibration routine for the system. This made it possible  to follow, and focus on, moving objects over a large spatial volume.

Movie: Wide field view shows a rod with a bright red tip at the end. The 3D tracking system tracked the red tip, and a PID controller directed the pan/tilt/focus unit to follow the tracked object (video shown in the inset).

Fly movie: Demonstration of tracking a freely moving fruit fly using the same system.