Endless ways to use Android for creating hardware-enriched intelligent systems

It might sound exaggerating that I use the word "endless" because most applications in Android Market don't give us such an impression. In our daily life we don't use our smartphones to interact with other machines pretty often. 

I'd say, it is just a start.

AR.Drone players may say: 

"Isn't the phone a cool remote-control, a console that is fully programmable and you can take everywhere?"

Arduino lovers or robotics/electronics system designers may say: 

"Well, more than a console. We love reading beautiful data graphs on Android's large screen. That's a perfect portable oscilloscope. Of course, we can take advantage of Android's great processor to perform complex computing. We can also use an USB cable to integrate it as an on-board "brain".

RC model makers say, "Sure we love the brain mode to have Android integrated on-board", as the DIY Drone's PhoneDrone board suggests:

"With Android you can switch between RC control and Android control or mix the two. You steer your vehicle via RC, but an Android phone does the actual control sending M2M commands to the on-board IMU. On a car, that would allow every turn to be a high-speed controlled drift, for instance.

Or, with a UAV, you might have the Android phone doing high-level image processing and object tracking, sending mission commands to an autopilot board. You might also want to use the phone's long-distance wireless instead of an Xbee for two-way telemetry.  This can either replace the original control board if you've got equivalent code running on Android, or compliment it with the Android device doing image processing or long-distance wireless communications."

OK, that's a lot of messy information when I put all the remarks here. It took me a while (actually two years) to observe, to test and to conclude what people really need Android to do in such hardware-enriched intelligent systems.

I've finally made a list of all possible outputs from Android and what we can input to Android:

1. Possible outputs from Android:

1.1 Sending high-level human-readable commands via user-interface by using the phone's built-in hardware

There is a big screen to display images and text, a speaker to perform text-to-speech or play music, a vibrator to give some notifications, etc.

This is what most applications in Android Market do. 

1.2 Sending high-level commands to tell one or multiple external devices what tasks to perform and controlling the actions of each external device

Examples:

1.2.1 All the remote control applications using WiFi or bluetooth. 

High-tech toys are pretty popular now, like this Sphero Toy Ball controlled by iPhone and Android:

Of course we have some more serious use-cases: e.g.: using Android to replace your PC's keyboard and mouse.  I use RemoteDroid personally in bed: 

1.2.2 Autonomous control system where Android is integrated on-board and outputs the commands automatically like this:

And this open-source project called Droid-Copter:

http://code.google.com/p/droid-copter/

             

1.3 Sending built-in sensor data  (GPS, gyroscope, accelerometer, digital compass , proximity sensor, etc.) from the phone to one or multiple external devices

Examples: 

GPS tethering: search "GPS tether" in Android Market and you will get many applications.

In DIY quadcopter, we can use Android to replace completely the MCU control board like ArduPilot. In that way we will use Android's gyroscope and compass. I am confident enough that on DIY Drones some intelligent guys can finally make a UAV with their on-board Android phone in the near future, like this unfinished project:

But let's see a prototype from the Cellbots open source project that I really like. The Arduino on the robot reads the phone's digital compass output to adjust orientations in an autonomous way, you get the idea:

2. Possible inputs to Android:

2.1 Receiving human commands via user-interface input by using the phone's built-in hardware

We have:

• a touch screen for getting user's command,
• a camera for "recording" the optical environment and some image processing to recognize a face,
• a microphone to get what people say and may perform voice recognition like "speech-to-text", etc.

This is what most applications in Android Market do. 

2.2 Receiving high-level commands from external devices that can be processed and figuring out what to do next

Examples:

• M2M commands between Android and Raspberry Pi
• WiFi Direct between two Androids since Android 4.0 Ice Cream Sandwich
• WiFi Hotspot Adhoc P2P Connection between two Androids
• Bluetooth Chat between two Androids
• Autonomous XMPP Chat between Android and PC or another Android:

• Autonomous SMS chat between two phones
• Various Internet protocols between Android and cloud. In Google IO 2011 there is more about Cloud Robotics, ROS Java and Android:

3. Receiving low-level sensor data from one or multiple external devices to get processed by the phone

This usually requires Bluetooth, WiFi, UART or serial connector to realize the sensor reading.  

Examples can be using an external GPS to replace the built-in GPS to provide accuracy, or using temperature sensors to be displayed on Android, etc.  

And today we just received this cool project from Mr. Al Linke, who proudly presents his cute funny artistic masterpiece, the Android Breathalyser:

Mr. Al Linke said that as a casual programmer, he spent just 3 nights to realize this, which shows the ease of using the IOIO board for Android. 

Whoever wants to replicate this project within 2 nights (if you want a challenge:)), here is the main parts list:

1. LiPo battery

2. LiPo rider: the LiPo rider takes the 3.7V from the LiPo battery and converts to +5V that the IOIO needs on its Vin. Plus the LiPo rider is also a charger so the whole thing becomes a self contained rechargeable unit.

3. The grove alcohol sensor

4. And of course the magical IOIO board.

All in all:

There are endless ways to use Android for creating hardware-enriched intelligent systems. We hope that before designing your new system, you take a serious glance at what Android can provide and which inputs/outputs are capable to meet your needs, in order to keep the project simple and powerful. A good selection and combination of those Android inputs and outputs can be the key to success.