Mobile augmented reality for the Blind or Visually Impaired
The vOICe for Android application adds a sonic augmented reality overlay to the live camera view in real-time, thereby giving even completely blind people live and detailed information about their visual environment which they would otherwise not perceive. It may also serve as an interactive mobile learning app for teaching visual concepts to blind children. The vOICe for Android is a universal translator for mapping images to sounds.
Once started, The vOICe for Android continuously grabs and sounds snapshots from the camera. Each camera snapshot is sounded via a polyphonic left-to-right scan through the snapshot while associating height with pitch and brightness with loudness. For example, a bright rising line on a dark background sounds as a rising pitch sweep, and a small bright spot sounds as a short beep. This approach allows for sounding arbitrary images while largely preserving the image content as needed in sensory substitution for the blind. The visual sounds encode a visual resolution of up to 176 × 64 pixels (comparable to an implant with 10,000 electrodes). Note that The vOICe for Android always runs in landscape mode, i.e., rotated 90 degrees left with respect to portrait mode. Try it on simple visual patterns first or you will be utterly confused!
The vOICe for Android is available as a free application on Google's Android Market. To find it from the Android Market home screen, search for "seeingwithsound" (written as one word), or use the above QR code.
Alternatively, you can download The vOICe for Android as the signed Google Android APK file "vOICeAndroid.apk" directly from the URL www.seeingwithsound.com/phone/vOICeAndroid.apk (version 1.22, for Android 1.6 and later, now including Android 3.0!)
The vOICe for Android can also be obtained from the Amazon Appstore, AppBrain, AndAppStore and Phoload. Installation directly from this or other websites using the Android browser may initially fail with a message like ``Install blocked. For security, your phone is set to block installation of applications not sourced in Android Market''. If this happens, you need to first enable "Unknown Sources" (from the main screen selecting Menu | Settings | Applications | Unknown Sources) to allow installation from the web, and try again. Do not download The vOICe for Android from other websites than those listed here, to minimize any risks of malware.
The vOICe for Android runs on Android-based smartphones and tablets, such as the Motorola Droid (Milestone), Droid 2 and Droid X, Nexus One, HTC Desire, Samsung Galaxy, Galaxy Tablet, Motorola XOOM, and many many others. For installation instructions consult your Android device manual. Android 1.6 or later is required. You can find your current firmware version in Settings | About phone | firmware version.
The vOICe automatically uses the Text-To-Speech library of Android 1.6 or later if installed, thereby supplementing its built-in speech with nearly complete screen-reader-like accessibility for menu and dialogs (tip: use the trackball or trackpad to only change focus). T-Mobile G1 or Dream users need to install Google's SpeechSynthesis Data Installer from Android Market to get the needed voices.
Take care that The vOICe for Android, as a CPU-intensive application, may drain your smartphone's battery relatively quickly. For maximum battery life set screen brightness to a low level. Please read the disclaimer, and see the update history if you are considering an upgrade from an earlier version. On devices with an ARM CPU, The vOICe for Android uses native code to speed up execution and reduce CPU load independent of the presence of the JIT compiler that is included with Android 2.2 and later.