Android + Raspberry Pi + Microcontroller = Intelligent gadgets with everything possible

1. What is Rapsberry Pi?

I have always wanted to talk about my favorite thing of the year: the cheap tiny linux PC - Yes, the Raspberry Pi!

It will be avaiblable soon by the end of the year 2011, let's hope.

Size: same as a credit card (for model A, even smaller, like a pen)

Price: 25 dollars

It is a full linux PC on which we can install Ubuntu, or even play Quake smoothly.

2. Why is Raspebrry Pi so interesting for DIY Phone Gadgets?


Android = Gorgeous user Interface, portable telecommunications services and the Internet of things everywhere with you

Raspberry Pi = Cheap, smart, tiny linux PC that can be powerful brains of any intelligent gadget and easily uses the USB port to add Microcontrollers like Arduino to control any motor or light or whatever sensors

Conclusion: Every project that combines Android and Raspberry Pi can make a powerful gadget.

3. How to make DIY Phone Gadgets combining Rasperry Pi and Android?


Mode 1. Android as a standalone remote control and Raspebrry Pi as a part of the gadget.

There are two possible types of communications:

a. Short range low delay communication


We can plug USB WiFi or Bluetooth dongles on Raspberry Pi, connecting it to the Arduino or whatever microcontroller, and use Android to interact with it.

Typical examples can be home automation gadgets such as an NFC lock or an intelligent airconditionner, showing user interface on the Android phone. WiFi toys with augmented reality feature can be interesting too. In UAV, you can connect Raspberry Pi to ArduPilot, and do some near real-time telemetry and video feedback via WiFi to be displayed on the Android phone, seamlessly working with Google Maps.

b. Cloud communication

We can connect the Raspberry Pi powered gadget to the Internet, and use Android to "chat" with it from everywhere in the world.

An example that I can imagine would be a surveillance robot with USB camera at home (of course its brain is the tiny Raspberry Pi). While the robot is wandering at home, we can chat with it on Google Talk using our Android phone from another end of the world. The QoS on some 3G or 4G networks can be good enough to support video chat and voip. Android will use XMPP, Web RTC or whatever promising or classical Internet protocols to interract with the gadget.

Mode 2. Android and Raspberry Pi together as a part of the gadget.

This mode takes advantage of the Android's built-in software and hardware (GPS, sensors, WiFi, bluetooth, etc) and the advantage of Raspberry Pi's good hardware extensitivity (because it is a linux PC with USB 2.0 OTG ports).

In this mode, the Android is directly connected to Raspberry Pi using a USB cable, just as it is connected to any PC. So ADB and USB tethering can be great protocols to help realizing fast bidirectional communication.

An example can be a sophisticated RC controller with many physical buttons and joysticks, and of course Android's large touch screen. You can download many different Android applications for this gadget, or add different 2.4Ghz radio modules. So there is a both a greater software and hardware flexibility compared to the classical radio controls from different manufacturers.

Mode 3. Installing Android on Raspberry Pi...And make cheap gadgets.


This seems a bit difficult for now because higher versions of Android are demanding in terms of hardware capabilties.

3. What can be done to help developers of DIY Phone Gadgets?


The idea is to buid up some good libraries in Raspberry Pi that can work with a special microcontroller such as Arduino, while supporting different types of connections with Android. The ultimate purpose is to provide a simple and single way to develop, instead of always developing on different platforms and trying to figure out how to bridge.

I really like the way how projects like IOIO and Amarino integrate the electronics programming in Android development. If Raspberry Pi can provide such a platform to handle PWM, UART etc while easily supporting WiFi dongles, and it is so cheap, why not?

4. Limitations of using Raspebrry Pi in DIY Phone Gadgets


Unfortunately, as Raspeberry Pi is so tiny and squeezes all cool things into a small board, there are not enough I/O pins for hardware developers. This is what is better considered on the more expensive BeagleBone. However, we can always connect a cheap Arduino board to it and it's all done easy and clean. Remember it is a PC and it is USB-friendly. For those of you who are curious about the I/O on Raspberry Pi, please read the official WiKi:

http://elinux.org/RaspberryPiBoard#General_Purpose_Input.2FOutput_.28GPIO.29.2C_I2C.2C_I2S.2C_SPI

There are approximately 16 spare GPIOs, which on the Alpha board are brought out to 1.27mm pin-strip. Voltage levels are 3v3. The connector choice is deliberately annoying to connect to directly; there is no over-voltage protection on the board so the intention is that people interested in serious interfacing will use an external board with buffers, level conversion and analog I/O rather than soldering directly onto the main board. It brings 2x I2C (3v3), I2S and an SPI (3v3) interface out to the same connector. It supports one slave interface for I2C and one for SPI. The UART has four PINs: 3.3V, GND, TX and RX. Kernel boot messages go to this UART at 115200bps.

Good ideas are happily discussed on Raspberry Pi Forum here:
http://www.raspberrypi.org/forum?mingleforumaction=viewtopic&t=1158
or in IOIO User Group here:
http://groups.google.com/group/ioio-users/browse_thread/thread/d23be2c908a9d5ae/be8101778a202a52

Endless ways to use Android for creating hardware-enriched intelligent systems

It might sound exaggerating that I use the word "endless" because most applications in Android Market don't give us such an impression. In our daily life we don't use our smartphones to interact with other machines pretty often. 

I'd say, it is just a start.

AR.Drone players may say: 

"Isn't the phone a cool remote-control, a console that is fully programmable and you can take everywhere?"

Arduino lovers or robotics/electronics system designers may say: 

"Well, more than a console. We love reading beautiful data graphs on Android's large screen. That's a perfect portable oscilloscope. Of course, we can take advantage of Android's great processor to perform complex computing. We can also use an USB cable to integrate it as an on-board "brain".

RC model makers say, "Sure we love the brain mode to have Android integrated on-board", as the DIY Drone's PhoneDrone board suggests:

"With Android you can switch between RC control and Android control or mix the two. You steer your vehicle via RC, but an Android phone does the actual control sending M2M commands to the on-board IMU. On a car, that would allow every turn to be a high-speed controlled drift, for instance.

Or, with a UAV, you might have the Android phone doing high-level image processing and object tracking, sending mission commands to an autopilot board. You might also want to use the phone's long-distance wireless instead of an Xbee for two-way telemetry.  This can either replace the original control board if you've got equivalent code running on Android, or compliment it with the Android device doing image processing or long-distance wireless communications."

OK, that's a lot of messy information when I put all the remarks here. It took me a while (actually two years) to observe, to test and to conclude what people really need Android to do in such hardware-enriched intelligent systems.

I've finally made a list of all possible outputs from Android and what we can input to Android:

1. Possible outputs from Android:

1.1 Sending high-level human-readable commands via user-interface by using the phone's built-in hardware

There is a big screen to display images and text, a speaker to perform text-to-speech or play music, a vibrator to give some notifications, etc.

This is what most applications in Android Market do. 

1.2 Sending high-level commands to tell one or multiple external devices what tasks to perform and controlling the actions of each external device

Examples:

1.2.1 All the remote control applications using WiFi or bluetooth. 

High-tech toys are pretty popular now, like this Sphero Toy Ball controlled by iPhone and Android:

Of course we have some more serious use-cases: e.g.: using Android to replace your PC's keyboard and mouse.  I use RemoteDroid personally in bed: 

1.2.2 Autonomous control system where Android is integrated on-board and outputs the commands automatically like this:

And this open-source project called Droid-Copter:

http://code.google.com/p/droid-copter/

             

1.3 Sending built-in sensor data  (GPS, gyroscope, accelerometer, digital compass , proximity sensor, etc.) from the phone to one or multiple external devices

Examples: 

GPS tethering: search "GPS tether" in Android Market and you will get many applications.

In DIY quadcopter, we can use Android to replace completely the MCU control board like ArduPilot. In that way we will use Android's gyroscope and compass. I am confident enough that on DIY Drones some intelligent guys can finally make a UAV with their on-board Android phone in the near future, like this unfinished project:

But let's see a prototype from the Cellbots open source project that I really like. The Arduino on the robot reads the phone's digital compass output to adjust orientations in an autonomous way, you get the idea:

2. Possible inputs to Android:

2.1 Receiving human commands via user-interface input by using the phone's built-in hardware

We have:

• a touch screen for getting user's command,
• a camera for "recording" the optical environment and some image processing to recognize a face,
• a microphone to get what people say and may perform voice recognition like "speech-to-text", etc.

This is what most applications in Android Market do. 

2.2 Receiving high-level commands from external devices that can be processed and figuring out what to do next

Examples:

• M2M commands between Android and Raspberry Pi
• WiFi Direct between two Androids since Android 4.0 Ice Cream Sandwich
• WiFi Hotspot Adhoc P2P Connection between two Androids
• Bluetooth Chat between two Androids
• Autonomous XMPP Chat between Android and PC or another Android:

• Autonomous SMS chat between two phones
• Various Internet protocols between Android and cloud. In Google IO 2011 there is more about Cloud Robotics, ROS Java and Android:

3. Receiving low-level sensor data from one or multiple external devices to get processed by the phone

This usually requires Bluetooth, WiFi, UART or serial connector to realize the sensor reading.  

Examples can be using an external GPS to replace the built-in GPS to provide accuracy, or using temperature sensors to be displayed on Android, etc.  

And today we just received this cool project from Mr. Al Linke, who proudly presents his cute funny artistic masterpiece, the Android Breathalyser:

Mr. Al Linke said that as a casual programmer, he spent just 3 nights to realize this, which shows the ease of using the IOIO board for Android. 

Whoever wants to replicate this project within 2 nights (if you want a challenge:)), here is the main parts list:

1. LiPo battery

2. LiPo rider: the LiPo rider takes the 3.7V from the LiPo battery and converts to +5V that the IOIO needs on its Vin. Plus the LiPo rider is also a charger so the whole thing becomes a self contained rechargeable unit.

3. The grove alcohol sensor

4. And of course the magical IOIO board.

All in all:

There are endless ways to use Android for creating hardware-enriched intelligent systems. We hope that before designing your new system, you take a serious glance at what Android can provide and which inputs/outputs are capable to meet your needs, in order to keep the project simple and powerful. A good selection and combination of those Android inputs and outputs can be the key to success.

Benefits and different ways for Android devices to control digital cameras

You might wonder why bother using an Android device to control your digital camera. Well, there sure are a lot of benefits:

1. Combination of Android software/hardware with your camera's hardware capabilities to create a super machine:

The built-in camera is lousy and the digital camera doesn't have GPS nor Internet connection. By mixing the two in one system, you can use Open CV to edit the camera view, do some advanced image processing, face recognition, instantly share your pictures using social networks like Facebook and blogs...etc.

2. One set of hardware with multiple/custom user interfaces to flexibly satisfy different end-users:

If your grandma is uncomfortable with those complicated buttons and configurations on the camera itself, why not download an application called "Canon DSLR for Dummies". Your grandma will just take a tablet or a phone along with the camera, touch 3 huge buttons to take pictures and videos. And cool, she will be using the most complicated camera in her daily life! You can also make the user interface super scientific in the Android application, to fine-tune, save, share, download and upload different configurations for your camera.

3. It turns your camera into dual-screen camera. 

The best thing about having the dual screen on your digital camera is that you can take really good self portrait pictures and so no-one has to be missing from the pictures any more. You can position the camera and see as you stand how the picture will look and so this makes taking these types of pictures much easier. The dual screen also offers other advantages though and can be used to display a range of information. One of the coolest features is the way they can display a nice smiling image just before the camera takes the photo and so the subjects can tell when to smile and know when to smile for the photo. This is brilliant if you are taking photos of children as they can display something that they will like such as a clown and so they know to smile when they see this image.

4. Remote control capability. 

Your phone is perfect for working with those IP surveillance cameras. You can easily get live video streaming anywhere, and change the camera angle by touching your phone screen. 

How to control using an Android device?

1. Direct control solutions from Android to Camera:

Usually this approach is wired. Sometimes you need to have custom ROM on your Android device. For example, the DSLR Controller is an EOS utility app for use with Canon DSLR cameras:

Or this on-going project with nice features:

http://www.camera-control.com/p/feature-list.html

2. Indirect control solutions using a microcontroller (e.g.: from Android to Arduino to Camera):

As far as I know, there are many non-standard hacks that already used the Arduino board to control or trigger camera, for example:

http://www.circuitsathome.com/camera-control/digital-camera-control-using-arduino-usb-host-shield-part-1-basics

To add Android control, you only need to add connection between Android and Arduino (using bluetooth or USB cable), which is easily done here (I made a list of solutions):

http://www.diyphonegadgets.com/2011/10/diyers-lets-make-complete-list-of.html

3. Using WiFi to control Cameras with WiFi or Ethernet features (e.g.: an IP surveillance camera or a camera with a smartphone OS):

This is an example using Android to control the Samsung SH100 using the "Remote Viewfinder": 

Here are some connectivity modes between Android and another WiFi device:

http://www.diyphonegadgets.com/2011/10/3-possible-ways-for-android-to-control.html

I am sure there are more cool solutions to connect Android to your camera. DIYers and manufacturers, let us know what your choice is!