Spot Takes the Stage

For 37 seconds, everything goes according to plan.

The lights come up on five Spot robots, all lined up in a row, and their arms shoot out in kaleidoscopic patterns, in perfect time with Queen’s “Don’t Stop Me Now.” The crowd at an America’s Got Talent filming gasps to see Spot lip-syncing, and when the robots break apart and begin to strut their stuff individually.

Then, just as Freddie Mercury sings the word “gravity,” one of the Spots suddenly crumples to the stage, like a marionette whose strings have been cut.

“It was like the robot was struck by lightning,” says Rosa Kurtz, a Boston Dynamics senior software engineer who spent weeks preparing the robots for the performance. “It was such a rare hardware failure. Leading up to the performance, we ran the robot more than 100 times. We looked for things like, what if the floor is slippery? We looked for software issues. But this one hardware failure was incredibly random, and very difficult to catch.”

Standing backstage with the operators, Hannah Rossi, marketing project engineer, has only seconds to make a decision: Stop the performance, or let the remaining Spots dance.

“You want to keep the show going. We had one take, and it was in front of a live audience,” says Rossi. “As long as everything was safe, we were going to continue.”

The dance was designed to minimize the risk of collisions and failures, and Rossi’s decision to go on with the show pays off. Although the one Spot remains down near the back of the stage, the other four robots bring the America’s Got Talent crowd to its feet.

“It was weirdly better that one of them died,” says judge Simon Cowell. “Because it showed how difficult this was.”

Almost as soon as the words are out of Cowell’s mouth, the sleeping Spot springs back to life, and the crowd goes wild. “That’s my favorite one!” exclaims judge Howie Mandel, pointing to the stage.

Ultimately, all four judges vote to advance the team of Spots to the next round of the competition. Spot will appear on August 26, with an even “bigger and better” performance, Rossi says.

Why Send Robots to Hollywood

The hardware failure at the America’s Got Talent filming was unintentional, but it illustrates several of the goals of these public performances: Spot is a fun—even endearing—and highly sophisticated piece of technology, it’s not something out of science fiction. Whether working autonomously, performing choreography, or being driven manually, Spot is a tool to extend human intelligence and ingenuity.

“I think people’s attitudes change once they get to see the robot in person, or even get a chance to drive it,” says Rossi. “That’s important. It’s an opportunity to change the narrative from the idea that robots are scary, which is what a lot of people think from watching movies.”

Spot has a long history of starring in viral performances. In late 2018, before the robot was even commercially available, Boston Dynamics unveiled Uptown Spot, a video showing a single Spot dancing to “Uptown Funk.” In 2020, Do You Love Me? featured a menagerie of robots doing the twist and the mashed potato to the classic oldie song. The next year, Spot’s On It showed seven different Spot robots dancing together in highly coordinated choreography to the music of K-pop group BTS, and Spot Me Up showed a band of Spots recreating the iconic video from the Rolling Stones.

These early dance routines were the brainchild of Boston Dynamics founder Marc Raibert (now executive director of the Robotics and AI Institute), who saw them as a fun creative challenge, but also as a way to accelerate software and hardware development. 

Today, Spot is deployed in hundreds of commercial and industrial settings around the world, performing inspections at manufacturing sites, helping human teams at nuclear decommissioning work, and assisting bomb squads as they remotely inspect suspicious packages. These real-world uses lend an urgency to Boston Dynamics’ efforts to make sure people are comfortable around the robot. And America’s Got Talent—the most-watched summer broadcast series in the U.S. for 20 years—provides an ideal megaphone. 

“We want robots to be out in the world and used for work,” Kurtz adds. “But it’s also part of our background that we’ve always wanted to push the limits, and we’ve wanted to bring people in and just show them robots. So, when we do these big performances, it’s still part of that original goal of showing people: Here’s how fun and wonderful robots can be.”

How Spot Gets Its Groove On

Seeing Spot’s fluid dance, some viewers may think the robot is capable of freestyling in response to a good beat. But in reality, each performance takes weeks or months of painstaking work by a team of animators and engineers.

For this performance, the team had to first choose a song. They went with “Don’t Stop Me Now” in part because it’s a crowd pleaser, but also because the song’s range presented more opportunities to show Spot’s range. The team then spent a couple of weeks brainstorming potential moves and sequences, watching dance reference materials for inspiration.

Choreographer, Boston Dynamics’ custom drag-and-drop dance software for Spot, makes it easy for teams to experiment with different sequences of existing dance moves. Boston Dynamics animators also build new animation moves by defining high frequency sequences of position data for Spot. For example, custom movements with 120 keyframes per second mean a brief 10-second dance sequence will contain well over 1000 individual positioning commands telling Spot exactly where each part of its body should be at any given moment.

Dance provides engineers with a way to test the limits of the hardware. In a software environment, animators can depict Spot performing essentially any movement. But in the real world, these behaviors run up against the constraints of friction, hardware tolerances, and gravity. “There are definitely more dynamic animation behaviors that we’d like to be possible, that we can’t do yet,” Kurtz says. “With these dances, we’re always trying to do more, and to do it more stably, with a higher success rate. Sometimes we hit those limits, and we have to pull back.”

A live performance with no second takes adds to the challenge. “On video, some of the moves are maybe only 70 percent reliable,” Rossi says. “For real life, we need to get close to at least 90 percent.”

Boston Dynamics engineers have also used Choreographer to test more practical movements than pirouettes or the Texas two-step. For example, when the company was first looking into using Spot for bomb disposal, engineers used the software to test out different arm positions. “It was the best way to smoothly send the arm to pre-defined poses,” Kurtz says.

Ongoing Innovation

Dance and entertainment aren’t just  training grounds for Spot. A handful of sports teams and theme parks have used Spot and Choreographer in their entertainment programming. Spot has also started showing up in college curricula, with students at Brown University learning to program Spot in both dance and comp sci classes. Even middle- and high-school students can use Choreographer’s drag-and-drop interface to get Spot moving and get hands-on experience to spark their interest in STEM. At Central High School in Louisville, students have been using Spot since 2021, using commands written in Python to get the robot to wave pom poms and clean up garbage. 

Arun Kumar, a software engineer who works on Spot’s behaviors and controls, says he sees the robot as inspiring the next generation of technicians, roboticists, and animators. “I see Spot as having two personalities,” he says. “There’s the worker Spot doing industrial inspections in factories. That’s inspirational in the sense that our robots are actually being useful and providing valuable data to our customers. Then, through behaviors like dance, we’re repeatedly redefining humans’ conceptions of what robots can physically do. Work like this inspires people to think bigger and push robots further.”

The Boston Dynamics team continues to reimagine ways to get Spot moving in new ways. Engineers are experimenting with AI models to brainstorm and validate new dance moves. Additionally, a new Choreographer feature supports synchronizing Spot dances based on timecode input—a method already used in stage programming to sync effects like music and lights. 

“In the past, we’ve pressed a button and music plays, and the robot dances,” says Kurtz. “But as we’ve done more and more complicated shows, we’ll have all these fancy stage effects going on. We’re just responsible for the robots dancing, but when we design these dances, we want everything to be really well synced up. It’s important that our robots are in time with each other and in time with the music.”

The majority of Spots out in the world operate far away from the spotlight, with the robots working alongside forklifts, drones, and oil rigs. And yet, many people are mostly familiar with the robot from viral videos—and now, from prime-time television. 

This seeming incongruity sometimes leads to humorous interactions. “We get this very question all the time at conferences,” Rossi says. “People will say: ‘You’re the dancing robot company, why are you talking about industrial inspections?’ It’s a great opener. I think it helps us, if only to start a conversation: ‘Now let me tell you more about how this could help you onsite.’”

This article was written by Calvin Hennick. Hennick has written about business, technology, travel, and other topics for dozens of publications, including The Boston Globe, Runner’s World, and Esquire. His debut memoir, Once More to the Rodeo, was named one of the Best 100 Books of 2019 by Amazon.