Google debuts tool for programming robots with natural language commands

AI

Google LLC today debuted an internally developed software tool that can reduce the amount of effort involved in training a robot to perform new tasks.

The tool is known as Code as Policies, or CaP. It’s available on GitHub under an open-source license.

Before a manufacturer can deploy robotic arms in a factory, it has to customize the systems according to its requirements. The process usually involves writing custom code. For example, a manufacturer’s developers might write code that instructs a robot arm to pick up products from a production line and place them in a crate.

A company must produce custom code every time it wishes to train its robots to perform a new task. Moreover, adapting software written for one robot to run on other machines often requires manual modifications. As a result, the development process can involve a significant amount of time and effort.

Google says its newly debuted CaP tool can save time for developers by automatically generating robot configuration code. CaP generates code using artificial intelligence models. In recent years, Google and other companies have developed advanced AI systems capable of writing software based on user prompts. Using such AI systems, CaP can generate code that enables a robot to perform tasks specified by the user.

Google researchers tested CaP’s capabilities in a series of internal evaluations. During one test, the researchers evaluated if CaP can teach a robot how to change the location of toy blocks on a table. After receiving the instruction “arrange the blocks in a square around the middle,” CaP successfully generated code that allowed the robot to rearrange the blocks.

According to Google, the tool can also be used to train robots for a variety of other tasks. “CaP allows a single system to perform a variety of complex and varied robotic tasks without task-specific training,” Google research intern Jacky Liang and research scientist Andy Zeng explained in a blog post.

The AI systems that power CaP originally weren’t designed to generate robot configuration code. According to Google, its researchers trained the systems to do so using a method known as few-shot learning.

Teaching an AI system to perform a new task usually involves supplying it with a large number of examples that demonstrate how the task should be performed. With few-shot learning, researchers can train an AI system using only a few examples, which speeds up development. Google’s researchers trained CaP by supplying it with examples of how natural-language instructions can be translated into robot configuration code.

CaP writes software in the Python programming language. In addition to producing new code, the tool can also draw on software libraries, pre-packaged collections of code that automate common tasks. Google says that its approach has proven more effective than existing approaches to configuring robots for new tasks.

“Our experiments demonstrate that outputting code led to improved generalization and task performance over directly learning robot tasks and outputting natural language actions,” Liang and Zeng detailed.

Alongside the code for CaP, Google has released a benchmark testing tool to support further research. The benchmark tool will enable researchers to more easily compare how well different AI systems perform robotics-related tasks. 

Image: Google