Pixy (CMUcam5) the camera with Image Processing on Board.
One of the drawbacks of the Arduino as a robot controller is the lack of processing power capable of a robot vision system. The CMUcam5 Pixy, by Charmed Labs, is aimed at fixing that. Pixy started on Kickstarter but is now available at places like Amazon. By including a dedicated image processor on the same PCB as the camera, the Pixy only sends the information you want to your Arduino. And it does this at a frame rate of 50 Hz. This frees up both your personal and processor time for other things.
The Pixy and the Pan/Tilt mechanism both come in eye catching boxes. The cartoon pictures are clear and colorful and right on the box it leads you to Charmed Lab’s website where they have a Quick Start Guide. I subscribed to their mailing list for Pixy Updates and selected the option for “Pixy for Arduino”. Next was downloading and installing the computer interface PixyMon. This software allows you to see what Pixy sees and teach objects to Pixy. Keep in mind the Mini USB cable you need to connect Pixy to your computer is not included. This is not required if you are using Pixy with Arduino and want to teach Pixy using the on board push button. I would recommend using PixyMon as it helps you understand what image processing is taking place but it is nice to know that having a computer isn’t required. The instructions tell you to install PixyMon but not launch it or plug in your Pixy until you have updated it’s software. You do this by pressing the white button on the top of Pixy while plugging it in and then launching PixyMon. This will allow you to push the firmware update you downloaded. The drivers will install automatically and you will see video from Pixy on PixyMon. Teaching Pixy an object is as easy as drawing a box inside the color you want to track. Here I have taught it a pink Post-it. Then I added another pink and taught it green, blue, and yellow. You can see it assigns the same number to the matching color pairs of pink and green.
To get the optional Pan/Tilt Unit working go back to the Pixy website and follow the instructions for assembly. The instructions were good and the pictures were clear but it was more tedious than I expected. Smaller hands would have helped for sure.
When you are finished, provided you took your time, you are rewarded with a very compact and functional unit. The black acrylic looks good and the zip ties keep everything pinned up nicely. The unit can pan just over 180 degrees and tilts about 100 degrees. Combine this with the fairly wide angle of the camera itself and the Pan/Tilt Pixy has a very good fixed mounted range.
Once assembled the Pan/Tilt Demo can be run directly from PixyMon. It is nice that Pixy handles the servo control directly and does not waste additional Arduino I/O pins. I taught Pixy the pink Post-it and ran the Pan/Tilt Demo. The Pixy took about 4 sweeps and then did a great job of keeping the Post-it in the center of the frame. I tested this by changing the angle and orientation of the base and moving the Post-it note around the room.
Next I wanted to get the libraries set up to use Pixy with my Arduino. You can find them on the wiki and will need to place the .zip in your Arduino folder or use the import .zip library from the IDE. Once you have the library the example sketch is enough to get you information. I uploaded this sketch to my Arduino. Hooking the Pixy up to an Arduino is as easy as plugging in the installed ribbon connector from Pixy to the Arduino.
This will also power the Arduino from Pixy so you could leave the Pixy plugged in and disconnect the Arduino from USB. In a mobile build I would power Pixy and the Arduino from the same rail but check to make sure I had enough available current. For now I wanted to see what the Arduino was reading so I plugged the Arduino back in, opened the serial monitor, and got:
Detected 1: block 0: sig: 1 x: 105 y: 161 width: 12 height: 13 Detected 1: block 0: sig: 1 x: 106 y: 161 width: 12 height: 13 Detected 1: block 0: sig: 1 x: 105 y: 162 width: 12 height: 12 Detected 1: block 0: sig: 1 x: 105 y: 161 width: 11 height: 12 Detected 1: block 0: sig: 1 x: 105 y: 162 width: 11 height: 12
Pretty impressive that without any work I get size and location of the object in data that is easy to use. And because Pixy operates at 50 frames per second, that information comes quickly. There were also instructions for how to set up Pixy with other mediums like Lego Mindstorms, Raspberry Pi, or BeagleBone Black. Both the instructions and Charmed Lab’s support are top notch. I emailed them several times while using Pixy and each request was handled within an hour. I may have the advantage of living in the correct timezone and emailing during business hours but I was impressed with the speed nonetheless. They really have made this as accessible as possible, as students of any level could use Pixy. I didn’t look at image processing until graduate school for electrical engineering. It took me 500 and 600 level classes in pattern recognition and machine learning to be able to do in Matlab what the Pixy does on the fly. This would be a great tool for anyone wanting to learn about vision systems, play with robotics, perform automated monitoring, or just wants something fun on their desk.