Google’s new robot manages to surprise everyone with its abilities

Last week, Google research scientist Fei Xia sat in the middle of a bright, open-plan kitchen and typed a command into a laptop computer connected to a one-armed, wheeled robot that resembled a large floor lamp: “I’m hungry.” The robot immediately proceeded to a nearby counter, carefully picked up a bag of grain chips with a large plastic gripper, and turned to offer Xia a snack.

The most impressive thing about this demonstration, held at Google’s robotics lab in Mountain View, California, was that no coder had programmed the robot to understand what to do in response to Xia’s command. The control software had learned how to translate a verbal expression into a series of physical actions using millions of pages of text collected from the web.

This system means that a person doesn’t have to use certain pre-approved phrases to give commands, as might be required by virtual assistants like Alexa or Siri. When you say “I’m thirsty” to the robot, it may try to find you something to drink; When you tell him “shit, I spilled my drink,” he may come to you with a sponge.

“To cope with the diversity of the real world, robots need to be able to adapt and learn from experience,” said Karol Hausman, a senior research scientist at Google, during the demonstration, which included bringing the robot a sponge to clean up a spill. To interact with humans, machines need to learn to grasp how words can be put together in various ways to produce different meanings. “It’s up to the robot to understand all the subtleties of language,” says Hausman.

Google’s demonstration was an impressive step towards its longstanding goal of developing robots capable of interacting with humans in complex environments. Researchers have discovered over the past few years that feeding large amounts of text from books or the web into large machine learning models can provide programs with impressive language capabilities, including OpenAI’s text generator GPT-3. By learning many forms of online writing, the software can acquire the ability to summarize or answer text-related questions, compose coherent articles on a particular topic, and even deliver persuasive speeches.

The robot Hausman showcased was powered by the most powerful language model Google has ever announced, known as PaLM. He has many different abilities, including explaining in natural language how he came to a certain conclusion when answering a question. The same approach is used to create a series of steps that the robot will execute to perform a particular task.