Hasta La Ideas

We knew it was a matter of time before AI-powered talking robots would start doing human jobs. But I thought the jobs would be manual labor, and that the robots would be humanoid. Instead Boston Dynamics just unveiled this robot tour guide, equipped with at least two surprising voices, particularly as they’re coming out of their dog-shaped Spot ‘bot:

I love how the pause after the “accent” comment reads as hilariously rude.

So why a tour guide? The company explains:

“In particular, we were interested in a demo of Spot using Foundation Models as autonomy tools—that is, making decisions in real-time based on the output of FMs. Large Language Models (LLMs) like ChatGPT are basically very big, very capable autocomplete algorithms; they take in a stream of text and predict the next bit of text. We were inspired by the apparent ability of LLMs to roleplay, replicate culture and nuance, form plans, and maintain coherence over time, as well as by recently released Visual Question Answering (VQA) models that can caption images and answer simple questions about them.

“A robot tour guide offered us a simple demo to test these concepts—the robot could walk around, look at objects in the environment, use a VQA or captioning model to describe them, and then elaborate on those descriptions using an LLM. Additionally, the LLM could answer questions from the tour audience, and plan what actions the robot should take next. In this way, the LLM can be thought of as an improv actor—we provide a broad strokes script and the LLM fills in the blanks on the fly.

“This sort of demo plays to the strengths of the LLM—infamously, LLMs hallucinate and add plausible-sounding details without fact checking; but in this case, we didn’t need the tour to be factually accurate, just entertaining, interactive, and nuanced. The bar for success is also quite low—the robot only needs to walk around and talk about things it sees.”