The virtual worlds where robots are trained

**The virtual worlds where robots are trained**

Freddo the robot walks across the office and accepts a plastic bottle handed to him by a staff member.

Given that a robot recently beat Usain Bolt's 100m sprint record, that is hardly the most astonishing feat.

What is impressive is how quickly Freddo was trained to walk, identify the bottle and pick it up. Those abilities were developed and uploaded to Freddo in just a few minutes. His creators say competing systems could take days to learn the same skills.

I’m at Vsim, a British start-up in Cambridge. Founders Michelle Lu and Kier Storey hope that one day their software will direct robots capable of navigating and carrying out useful tasks in homes and workplaces.

But there is still a long road ahead.

“It’s a weird situation with robotics because actually the stuff that we find as humans to be incredibly difficult, like gymnastics, you can get robots to do reasonably well. The stuff that humans are really good at, like fine dexterity, is really hard in robots,” Storey says.

Freddo’s abilities were developed in a virtual setting, where a task can be run in computer simulation millions of times. Once the best solution, known as a policy, is identified, it can be uploaded and used by the physical robot — in this case, Freddo.

Virtual simulations are a widely used method for training robots. Tech giant Nvidia has a system called Isaac Sim that works in this way — Lu and Storey both worked on an early version of it.

In 2022, they decided to launch Vsim to build their own training environment and other tools.

Starting from scratch allowed Lu and Storey to optimise the software so it could take advantage of the powerful computer chips used in AI, known as graphics processing units, or GPUs.

“The underlying algorithms that we were using for most of these robotic simulations they hark back to the 1970s and 1980s, but those algorithms are not really brilliant fits for GPUs,” Storey says.

Within months, they realised their system could operate much faster than anything they had seen before.

“Eighteen months in and we actually have a completely functional, super high-performance simulator,” says Lu.

The software is so efficient that it can run on the hardware carried by Freddo. That means the robot can carry out tens of thousands of simulations while it is moving.

“It can look about a second, or so, ahead into the future for 20,000 different kind of combinations of things that might happen,” Storey explains.

That would be crucial for a robot operating in an unstructured environment such as the average home.

“Things outside of the robot's control, like humans, animals or even other robots, could do things that require a change of strategy. These unexpected events could happen very quickly and the robot needs to be able to quickly adapt to ensure its actions remain safe and on-mission,” Lu says.

Vsim is a start-up with 10 engineers working on its technology. Nvidia sits at the other end of the industry. It dominates the market for AI computer chips and has a leading robotics software division with hundreds of engineers.

It does not build robots; instead, it offers a suite of software designed to help organisations train and control them.

That includes virtual simulation training systems and a so-called world model, called Cosmos, which gives a robot an understanding of the physics of the real world and how its surroundings may change as it moves.

But even with the powerful computing resources available to Nvidia, the software only provides a basic understanding of the real world.

“Manipulation, - where I just grab a bottle, that's not too hard. The problem is when you start doing long-horizon tasks, where I say: 'I want you to take the bottle and I want you to fill it up and I want you to go pour',” says Spencer Huang, director of product for robotics at Nvidia.

Still, he is confident that strong progress is being made. This year Nvidia has begun using AI agents to help create virtual environments for training robots and to check whether the results from training actually work.

“When we talk about creating the [virtual] world and actually scanning it in - a lot of that is actually manual labour.

“We're just throwing agents at it... it's basically given us a huge workforce,” Huang says.

Simulation is not the only way to train robots. They can also learn by observing humans or video demonstrations.

Rika Antonova has worked in robotics since 2015 and is now an associate professor in the Department of Computer Science and Technology at the University of Cambridge.

Antonova works with a training system called MuJoCo, owned by Google’s DeepMind since 2021. It is open-source software, meaning researchers can use it for free and are permitted to modify the code.

“It is very, very user-friendly. So for research groups or for small start-ups, that's useful,” she says.

She says Vsim’s approach — extremely fast simulation — is promising.

“If you have a very, very fast simulator, then you can simulate hundreds of millions of samples in that few seconds that your robot is thinking about how to adjust its motion, and then you can change the motion almost in real time,” she says.

But simulated environments remain rough approximations of the real world, which limits what can be trained.

“There are certain things that are hard to model in simulation, like highly deformable objects and cutting,” she says.

That is a challenge Nvidia and Lu and Storey at Vsim are trying to solve.

Lu says their system has “reduced approximation, using accurate simulations to train models that genuinely work in reality as well as they do in simulations.”

Soon, a second robot called Nacho will help develop that technology.

Lu says that should accelerate their development process and make sure their software can run on different machines.

And, of course, give Freddo some company.