Although augmented reality is most commonly associated with the overlaying of digital images and information about the real world, researchers have continued to explore potential audio applications of AR, generally using headphones or audio glasses to convey data into a wearer’s exploration of spaces to involve. On October 20, Foursquare will jump into the audio AR field with Marsbot for AirPods, a “lightweight virtual assistant” that whispers location-specific instructions in your ear as you walk through unfamiliar places.
Digital assistants have caught on in the past decade, as the Google Assistant used cloud servers to proactively organize users’ emails and calendars, Amazon’s Alexa made it easy for voice-controlled shopping, and Apple’s Siri did the processing promoted on the device to reduce user privacy concerns. But for better and worse, these assistants were passive – only upon request – and not stubborn in the sense that they would otherwise remain silent. Inspired by Scarlett Johansson’s human-like AI assistant in the film Her, Foursquare explores how a more active and location-based virtual assistant could function if it could always communicate with you using audio cues as flags instead of on-screen notifications. The AirPods project builds on the company’s previous Marsbot, which uses text messaging to provide location-based tips.
As users walk through New York City with Marsbot for AirPods, tips for specific places, attractions, and people nearby, ranging from rating a restaurant to being with a Foursquare contact, are automatically displayed. Some of the tips are provided straight from Foursquare, including useful facts that business users might consider, but there is another level. Users can record their own audio clips to share with the public, as well as Foursquare’s examples – “Get the burger!” and “Don’t get the burger!” – Do not scratch the surface of recordings made by individuals or companies. Anticipating some concerns, the company says it will “proactively monitor uploaded clips for abusive or offensive content” and let users mark content for human review.
Foursquare is charging Marsbot for AirPods as an “experiment” to “show what can be done with location-based audio,” and the company plans to update the app with additional functionality based on the results of user interaction. This could pave the way for more useful versions of Google Assistant and Siri – or usher in an era of even more annoying advertising as McDonald’s tries to steer you away from a local restaurant, or five different stores vie for your attention as you walk through a neighborhood. Currently, the app should be light and deliver “no more than 10 spoken syllables” per tip and generally only a few tips per day.
In its version 1, Marsbot will work “anywhere in the world” when users walk, run, ride a bike or drive. However, in major cities like Chicago, Los Angeles, New York, San Francisco, and Seattle, the content is best preinstalled. The tip timing is optimized for walking outdoors. Unlike audio AR experiences designed for art galleries or museums, Marsbot is not intended to offer tours of specific rooms, but rather to offer tips and facts as you walk around and automatically turn down the volume of music and podcasts to provide guidance. Foursquare detects that the app is not interrupting voice or video calls.
Marsbot for AirPods will be available as a free download from the iPhone’s App Store on October 20 at 8:00 a.m. Pacific Time and will actually work with all headphones despite the name. Foursquare has also set up a waiting list for an Android version of the app to be named, which should be available in the near future.