ROS Webcam demo on Macbook running Ubuntu

  1. Instructions to install Ubuntu on an external SSD (takes about an hour).
  2. Run this shell script to (takes about 10-15 min):
    • Install ROS (and a few other things)
    • Set up your Macbook’s built in webcam (more detail)
  3. Demo rostopic pub / echo:
    $ roscore # starts roscore
    $ rostopic pub test std_msgs/String 'this is a test' # publishes a message to a topic
    $ rostopic list # lists all active topics
    $ rostopic echo /test # live feed of data on topic
  4. Demo multiple machines: note both machines must have full connectivity (ssh)
    • On master:
      $ ifconfig # take note of your master’s ip address
      $ export ROS_IP=master.ip.address # can put in ~/.bashrc to make persistent
      $ echo $ROS_IP # check to see what that environmental variable is set to
      $ roscore # start roscore
    • On client:
      $ ifconfig # take note of the client’s ip address
      $ export ROS_IP=client.ip.address
      $ export ROS_MASTER_URI=http://master.ip.address:11311
    • Accessing a remote machine:
      $ ssh username@ip.address # must have openssh-server installed on remote machine
  5. Follow these instructions to set up a catkin_workspace
  6. The catkin instructions above do not mention to do this, but for most simple purposes, I suggest editing your .bashrc file:
    • The .bashrc file is a shell script that is run every time you open a new terminal window. Use a text editor (like gedit, nano, vim) to edit the ~/.bashrc file (gedit ~/.bashrc).
    • Add this line to the bottom (replace USERNAME):
      source /home/USERNAME/catkin_ws/devel/setup.bash
    • Reload your .bashrc by opening a new terminal window, or typing
      $ . ~/.bashrc
  7. Get ROS access to the webcam with usb_cam:
    • Check that camera is working
      $ guvcviewer
    • Install usb_cam
      $ cd ~/catkin_ws/src
      $ git clone https://github.com/ros-drivers/usb_cam.git
      $ cd ~/catkin
      $ catkin_make
    • Run usb_cam:
      $ cd ~/catkin_ws/src/usb_cam/launch
      $ roslaunch usb_cam-test.launch
    • Explore ROS environment:
      $ rosnode list
      $ rqt_graph
      $ rostopic list
      $ rostopic echo imagetopic
      $ roshz imagetopic
  8. Live image processing
    • Download optic flow example
      $ cd ~/catkin_ws/src
      $ git clone https://github.com/florisvb/optic_flow_example
      $ cd ~/catkin
      $ catkin_make
    • Explore directory structure (more detail)
    • Explore mono conversion node
    • Run mono conversion node
      $ rosrun optic_flow_example color_to_mono.py
      $ rostopic list
      $ rosrun image_view image_view image:=/imagetopic
    • Explore optic flow node
    • Run optic flow node
    • Edit optic flow node to also publish a single average value
  9. Remotely control a motor with live image data
  10. Real-time multi-target blob tracking
  11. Rosbag record, rosbag play, load data as hdf5
    • $ python ./bag2hdf5.py ~/2018-05-01-16-43-29.bag --out=~/2018-05-01-16-43-29.hdf5 --topic=/optic_flow_mean_vx
    • Analyze example
  12. Highlight modularity

Accessing Mac webcam in ROS (running Ubuntu on the Mac)

Here I show how to get access to the built in webcam on a MacBook in ROS (Robot Operating System).

This post assumes you have followed these two previous posts:
1. Install Ubuntu on an external USB drive so that you can run it from your MacBook
2. Installing ROS

Update

In the terminal, run the following two commands:

$ sudo apt-get update
$ sudo apt-get upgrade

Give Ubuntu access to your Mac’s webcam

From the terminal, run these commands:

$ cd /etc/local/src
$ git clone https://github.com/patjak/bcwc_pcie.git
$ cd bcwc_pcie/firmware
$ sudo make
$ sudo make install
$ cd ..
$ sudo make
$ sudo install
$ sudo depmod
$ sudo modprobe -r bdc_pci
$ sudo modprobe facetimehd

These instructions come from medium and askubuntu. Note: I received an error on the second install command, but it still works fine.

Test that the camera is working

Install guvcview:

$ sudo apt-get install guvcview

Then run it (by typing gucview into the terminal). If all goes well, a live view feed will pop up!

Install the usb_cam ROS driver:

To get access we will use the usb_cam driver. These steps assume that you have already built a catkin workspace, called catkin_ws.

$ cd ~/catkin_ws/src
$ git clone https://github.com/ros-drivers/usb_cam.git
$ cd ~/catkin
$ catkin_make

Launch the camera driver / viewer

$ cd ~/catkin_ws/src/usb_cam/launch
$ roslaunch usb_cam-test.launch

If all is good, a live video feed will now pop up. Note that the launch file spins up two nodes: usb_cam and image_view.

Explore the ROS topics

Type “rostopic list” into the terminal window to see a list of the ros topics that are available to you in the ROS ecosystem.

Install ROS on Ubuntu (on an external drive on your MacBook)

This post picks up from: how to install Ubuntu on an external USB drive so that you can run it from your MacBook.

  1. Follow these instructions to install ROS Kinetic on Ubuntu.
  2. Follow these instructions to set up a catkin_workspace
  3. The catkin instructions above do not mention to do this, but for most simple purposes, I suggest editing your .bashrc file:
    • The .bashrc file is a shell script that is run every time you open a new terminal window. Use a text editor (like gedit, nano, vim) to edit the ~/.bashrc file (gedit ~/.bashrc).
    • Add this line to the bottom (replace USERNAME): source /home/USERNAME/catkin_ws/devel/setup.bash
    • Reload your .bashrc by opening a new terminal window, or typing “. ~/.bashrc”

Installing Ubuntu on an external SSD drive on a Macbook

Warning: This is a complicated process, and you might screw things up. I am not an expert, and likely cannot solve any issues you encounter. Proceed at your own risk!

This guide will show you, step by step, how to install a full version of bootable Ubuntu on a SSD (solid state drive, or any other external drive), using only your Macbook. We will generally be following these instructions, with a few key modifications.

Why would you want to do this?

  • You want to have a portable full install of an Ubuntu system for prototyping
  • You do NOT want a virtual machine because they are slow
  • You do NOT want to make any permanent changes to your Macbook

Alternative options:

  • Use a persistent Ubuntu live session… certain features do not work
  • Use rEFInd. This requires permanent changes to your Mac

What you will need:

  1. An Intel powered Macbook (may have to be newer than 2013) with 2 USB ports
  2. A bootable USB flash drive for installing Ubuntu: instructions.
  3. A blank SSD drive, like this SSD USB drive ($70-130 on Amazon for the 128 GB and 256 GB versions). Why this drive? According to the specs it is FAST, and it is a SSD, not a standard USB Flash drive. So it should be more reliable (and faster) than a generic USB Flash drive. It is more appropriate for a full install of an operating system than a flash drive is.

How long will it take?

Maybe an hour. If you mess up, a good part of a day.

Let’s get started!

Part 1: Formatting your external SSD drive

  1. Insert your external SSD drive into your Mac
  2. Open your Utilities folder
  3. Double-click Disk Utility
  4. Select your external SSD drive (NOT your Mac’s hard drive!)
  5. Click Erase with these options (if it fails, just try again):
    • Name: EmptyDrive
    • Format: Mac OS Extended (Journaled)
    • Scheme: GUID Partition Map
  6. With the external SSD drive selected, click Partition. Reduce the size of the EmptyDrive partition by at least 128 MB. In this example I changed the original ~128 GB to ~127 GB. That is overkill, but it makes things simple.
  7. Apply the changes. In the window that pops up informing you that a new Untitled partition will be created, accept by clicking Partition.
  8. Select the Untitled partition, click erase, and rename it to Ubuntu Boot Loader

Part 2: Install Ubuntu on the external SSD

Start up an Ubuntu live session

  1. Shutdown your Mac.
  2. Unplug the external SSD drive.
  3. Insert your USB flash drive that has the Ubuntu installer on it.
  4. With the option key pressed on your keyboard, boot up your Mac.
  5. Select the Orange EFI Boot option (right arrow, enter).
  6. Choose “Try Ubuntu” (default option)

Prepare your SSD drive

  1. Plug in SSD drive.
  2. Open GParted (click the upper left corner icon, type in “GParted” and click the GParted Partition Editor application icon).
  3. In GParted, find your external SSD drive in the upper right box (e.g. select /dev/sdd, or whatever the appropriate letter is for you). Make sure the size of the drive makes sense.
  4. Delete all the partitions EXCEPT the hfs+ Ubuntu Boot Loader partition. Mine had two partitions to delete: a fat32 system with label EFI, and a hfs+ system with label EmptyDrive. For each partition to delete:
    • Select it
    • Click the red circle/slash symbol
  5. The delete operations are now queued, but have not been applied. Click the green check box to apply the operations.
  6. Add a new partition (click Partition > new).
  7. Use these options:
    • New Size: 8192 MiB (this should match your RAM, e.g. 8 GB = 8192 MiB)
    • File system: linux-swap
  8. Add another new partition with the following options:
    • New size: default (unless you want yet another partition)
    • File system: ext4
    • Label: Whatever you like
  9. Apply the new partitions by clicking the green check
  10. Make a note of the ext4 partition name, e.g. /dev/sdd2
  11. Close GParted

Start installing Ubuntu

  1. Connect to WiFi (this saves time later, but is optional)
  2. Double click “Install Ubuntu” in upper left corner on the desktop.
  3. Choose your language.
  4. Check the boxes:
    • Download updates
    • Install third party software
  5. Select: something else.
  6. Find the ext4 formatted partition you just created in GParted. Your best confirmation is to look at the size of the disk. Don’t screw this up.
  7. Click change… and apply these options (there should be a better way to do this, but this works):
    • default size
    • Ext4 journaling file system
    • check Format the partition
    • Mount point: /
  8. Choose this device for boot loader installation: select the device that corresponds to the ext4 partition. (Do not select the SSD itself)
  9. Go through the rest of the installation.. will take about 10 minutes.
  10. When it is finished, choose “continue testing”, as we have a few more things to do.

Making Ubuntu bootable part 1

Ubuntu has been installed on your SSD, but you won’t be able to boot into it since there’s no EFI boot loader, needed for Mac firmware to recognize the OS as bootable. So, we need to go into the Ubuntu installation and create a boot.efi file that we’ll use to boot it. While still in the live Ubuntu session…

  1. Open up a Terminal window (similar to launching GParted):
  2. Run this command in the terminal window (replace ‘/dev/sdaN’ with the device that corresponds to your /ext4 partition that you made a note of earlier – mine was ‘/dev/sdd2’)) (it mounts your /ext4 Ubuntu installation to /mnt):
    sudo mount /dev/sdaN /mnt
  3. Tip: you can select something on this page, copy it with ctrl-c, and paste it into the terminal with shift-ctrl-v. But make sure to double check the formatting, especially for underscores, and dashes.
  4. You’ll also need to mount several of the top-level directories, which is easiest to do with the following (long one line) command:
    for i in /dev /dev/pts /proc /sys /run; do sudo mount -B $i /mnt$i; done
  5. Now, we can use the chroot command to change our root directory to /mnt, so that the bash session in the Terminal is effectively running inside the Ubuntu installation:
    sudo chroot /mnt
  6. This allows us to configure GNU GRUB, the “GRand Unified Bootloader,” with:
    grub-mkconfig -o boot/grub/grub.cfg
  7. Now save this to a standalone file with (this is one long line):
    grub-mkstandalone -o boot.efi -d usr/lib/grub/x86_64-efi -O x86_64-efi --compress=xz boot/grub/grub.cfg
  8. Exit the chrooted partition:
    exit
  9. Here are all the terminal commands together:
  10. The boot.efi file is still in the new installation, however, we want to copy it to the live session:
    cp /mnt/boot.efi /home/ubuntu
  11. Now you will need to access the boot.efi file from your Mac OS system. The easiest is to email the file to yourself, or save it to a google drive. You can find the boot.efi file in your home directory (click the file cabinet icon in the upper left, which launches a folder view of your home folder).
  12. Now you can shutdown Ubuntu. The computer might hang.. give it 30 seconds and then manually shutdown.

Making Ubuntu bootable part 2

If you are running El Capitan or later, your Mac has extra security precautions called SIP (System Integrity Protection). This makes the last few steps a little harder. Following these instructions (copied from here), you can get around it:

  1. Reboot your Mac into Recovery Mode by restarting your computer and holding down Command+R until the Apple logo appears on your screen.
  2. Click Utilities > Terminal.
  3. In the Terminal window, type in:
    csrutil disable
  4. Press Enter.
  5. Restart your Mac.

Note: if you ever get a black screen with the words “grub”, just type exit.

Making Ubuntu bootable part 3

Final stretch!

  1. Launch your terminal (Go > Utilities > Terminal)
  2. Plug in your Ubuntu SSD and type in:
    cd /Volumes/Ubuntu\ Boot\ Loader/
  3. Tip: on your mac (unlike Ubuntu), copy/paste to the Terminal does not require shift.
  4. Type in the following lines (one at a time) (the password is your Mac’s password):
    sudo mkdir System mach kernel
    cd System
    sudo mkdir -p Library/CoreServices
    cd Library/CoreServices
  5. Download your boot.efi file from your email or google drive.
  6. Assuming your boot.efi file is now in your Downloads folder, type this into the terminal:
    sudo cp ~/Downloads/boot.efi ./
  7. Tip: In both Mac OS and Ubuntu, you can find a file through the user interface / finder, and drag the file into a terminal window, and it will insert the full path to the file.
  8. Now we need to make a SystemVersion.plist file. You can see what needs to be in it, and download the file directly via this link.
  9. Copy the SystemVersion.plist file to the same directory on the external SSD where your boot.efi is (/Volumes/Ubuntu\ Boot\ Loader/System/Library/CoreServices). Go back to the terminal window you had open from before and type:
    sudo cp ~/Downloads/SystemVersion.plist ./
  10. Open Disk Utility (Go > Utilities > Disk Utility) to check what disk number your bootloader is. In this case mine is disk2s3.
  11. Now we “bless” the partition for booting, and set the boot flag, like so (This is the step that required us to disable the SIP earlier):
    sudo bless --device /dev/diskNsK --setBoot
    Where you replace N and K with the letters from step 12.

Turn SIP back on (for security).

  1. Shut down your Mac
  2. Remove your USB devices

  3. Reboot your Mac into Recovery Mode by restarting your computer and holding down Command+R until the Apple logo appears on your screen.

  4. Click Utilities > Terminal.
  5. In the Terminal window, type in:
    csrutil enable
  6. Press Enter.
  7. Shutdown the Mac.

Boot into Ubuntu!!

  1. Plug in your external SSD drive onto which we have installed Ubuntu
  2. Press the option key while booting up the Mac
  3. Select the orange Boot EFI device
  4. Now you have a functional Ubuntu install!

A few tips:

  1. Whenever you have that SSD drive plugged into the computer, boot up while holding the option key. If it is not plugged in, just use your Mac as normal.

  2. Sometimes you may encounter this screen when booting into your Mac after using the Ubuntu drive:

    Don’t panic. Just type exit and press enter.
    To fix this permanently, go to System Preferences > Startup Disk, unlock, then select your Mac harddrive and restart. This will take longer than usual, so be patient. After that you won’t see the grub screen anymore on restarting your Mac.

  3. Get a right click working (better yet, use an external mouse):

    1. Click the “Activities” icon and type in Universal Access
    2. Select “Pointing and Clicking”, and turn on Simulated Secondary Click. Now if you click and hold for a brief moment, you will get a right click.
  4. If, when you shutdown Ubuntu, it seems to take forever (e.g. 1 minute and 30 seconds), press the “Esc” key when you see the Ubuntu… logo. This will tell you what the computer is doing. If you see this:
    A stop job is running for remote CUPS printers available locally
    Then implement unhammer’s (the second) reply explain here.

  5. Get familiar with the command line editor nano

Basler GigE cameras, aravis, ROS

These are some basic instructions for setting up the Basler Ace GigE cameras (A640) to work with ROS Kinetic on Ubuntu 16.04. We use the driver Aravis. Aravis changed the API at some points, and the available ROS nodes have not been well maintained. Thus, we will use older code sets to get things working.

1. Install some necessary packages:
sudo apt-get install intltool gobject-introspection gtk-doc-tools

2. Download an older version of aravis (0.3.7) here:
ftp://ftp.acc.umu.se/pub/GNOME/sources/aravis/0.3

3. From inside the aravis_0.3.7 directory, run the following commands:
./configure
make
sudo make install

4. Now you will need to set up your ethernet interface.
4a. Install a dhcp server: sudo apt-get install isc-dhcp-server
4b. Check your interfaces by running ifconfig. Note the interfaces (e.g. eth1, enp3s0f1, etc.) that correspond to the port where you intend to plug in your camera.
4c. Edit the following files, using the linked github files as templates. The …93… in the ip addresses can be whatever you like, so long as it does not collide with other ip’s on the local network.
i. /etc/network/interfaces
ii. /etc/default/isc-dhcp-server
iii. /etc/dhcp/dhcpd.conf  (note, all punctation including ‘;’ is important!)
4d. Restart the computer. If you have trouble connecting to the internet, try removing any mention of the interface you use to connect to the internet from the /etc/network/interfaces file.

5. Download the ROS node (git clone into your catkin workspace):
https://github.com/florisvb/camera_aravis
Note: this is forked from a no longer existing github repo, which was forked from ssafarik. I keep a copy just so I can make sure I know where to find a compatible version.

6. In ROS kinetic and newer, the driver_base stack is no longer supported, but it is necessary for the camera_aravis node. You can resolve this by manually installing the following (git clone into your catkin workspace):
https://github.com/florisvb/driver_common
Note: this is forked from ros-drivers, just so I can make sure I know where to find a compatible version.

7. From inside the catkin workspace base directory run:
catkin_make

8. If all goes well, you can now run:
roscore
rosrun camera_aravis camnode
rosrun image_view image_view image:=/camera/image_raw

Tips for debugging

1. Check that the camera works.

On another computer (e.g. a laptop), plug the camera into your ethernet port and run pylon view app: https://www.baslerweb.com/en/support/downloads/software-downloads/

2. Double check that your dhcp config is correct.

sudo systemctl status isc-dhcp-server

Will tell you if there is an error in the dhcpd.config.

More tips: https://killtacknine.com/building-an-ubuntu-16-04-router-part-2-dhcp/

3. If the encoding is strange.

Launch the pylon viewer app and switch to mono 8: https://www.baslerweb.com/en/support/downloads/software-downloads/

Jupyter and virtual environments

To use all your system site packages in addition to some packages that are only installed in your virtual environment you need a few steps:

 

virtualenv –system-site-packages ENV # creates the virtual env

# install a package

pip install –ignore-installed jupyter # installs jupyter into this ENV

Running Ubuntu off a USB 3.1 SSD drive (on a MacbookPro)

Get one of these drives:  https://www.sandisk.com/home/usb-flash/extremepro-usb ($70-130 on Amazon for the 128 GB and 256 GB versions)

Why this drive? According to the specs it is FAST, and it is a SSD, not a standard USB Flash drive. So it should be more reliable (and faster) than a generic USB Flash drive.

In order to maximize the usefulness of the drive, I wanted two partitions, one for Ubuntu, and one that could serve as a data partition that is accessible from both Ubuntu and Mac OS. To the best of my knowledge, the best format for this second partition is HFS+ (not journaled). Curiously, new versions of Mac don’t allow you to format the drive in this way. Neither does Ubuntu (by default). On your Ubuntu machine, install hfsprogs (sudo apt-get install hfsprogs). Then install gparted if it is not already installed (sudo apt-get install gparted). Plug in the drive, open gparted, and erase the drive and create a gpt partition table. Now create an HFS+ partition, as well as an ext4 partition (you will install Ubuntu on the ext4 partition). After that, you’ll need to add permissions for the HFS+ partition (“sudo chown -r username:username drivelocation” and replace username and drivelocation with the appropriate values).

Now plug a ubuntu boot drive and the USB SSD into your Mac. While booting up, press the option key. Select the Boot EFI option (orange drive), and install ubuntu on the ext4 partition.

All done! This USB SSD comes with a boot.efi partition that causes the Mac to boot into Ubuntu if the drive is plugged in. If you get a black grub screen when booting into Mac without the Ubuntu SSD plugged in, just type “exit” and it’ll boot into Mac OS.

There are a few annoying things with the Mac keyboard.. solve them with these tricks:

Get the tilde key working:

The solution that worked for Timofey did not work for me for some reason. However, a solution which did work was to put the line
echo 0 > /sys/module/hid_apple/parameters/iso_layout
into /etc/rc.local.
Get the right click work around:

 

 

 

 

Remote Jupyter Notebook Access

Setting up remote access to a Jupyter Notebook Server is easy, but there are a few tricks. Generally, follow these great instructions: Jupyter public server.

Then, depending on how your network is setup, you will need to take these steps:

  1. If your computer is behind a router, set up port forwarding to your computer’s IP address for the jupyter port (and whichever other ports you want).
  2. For protection, set up your firewall. Install gufw (a gui based firewall), default to deny. Then, for each ip address range you want to allow access, enter:
    sudo ufw allow from xxx.xxx.xxx.0/24

    to allow all traffic in the range xxx.xxx.xxx.0 to xxx.xxx.xxx.254. This way you can limit access to your server to computers on, say, your local network. Then, with VPN, you can log in to your local network from anywhere, and get access to your server.