Computers learn by playing with blocks

by Dan Falk News Learning in Machines & Brains 06.04.2016
AI
The program learns by looking at images of stacked blocks – some of which will topple.

When an infant plays with wooden blocks, it’s not just playing – it’s also learning about the physical world by interacting with it, pushing and pulling and poking at its various parts. Now a team of artificial intelligence researchers at Facebook are trying to get a computer program to understand its environment in the same way.

The program, called a convolutional neural network, learns by looking at hundreds of thousands of computer-generated images of stacked blocks – some of them stable, some unstable. If the stack is unstable, the network sees where the blocks end up after the tower topples over. When it’s then exposed to new images – either computer-generated images or pictures of real-life wooden blocks – the network does a remarkably good job of predicting whether the stacks are stable, and if not where the various blocks will end up.

On the computer-generated data, the network performs better than humans do, says Rob Fergus (New York University, Facebook AI Research), a senior fellow in CIFAR’s Learning in Machines & Brains program. “And on real-world data, it’s almost as good as humans,” he says.

Importantly, the network is never taught the rules of mechanics, or of gravity – it simply watches and learns, similar to the way a child learns.

“I don’t think we have Newton’s laws baked into our genes,” says Fergus. “It’s much more likely that we learn by trying things out. When you’re an infant, you play with these blocks, and you get an instinctive feeling. ‘The tower’s about to tip over. The top block is going to fall over, to the left.’ That kind of thing.”

For an infant, playing with blocks may seem like a game, but the child is actually learning how to take in a visual scene and predict what’s going to happen next. And that, in turn, depends on “a kind of physical common sense,” Fergus says. “At the moment, machines don’t have that.”

Networks like the kind Fergus works on still need to be told what to pay attention to. But some day a machine capable of completely unsupervised learning may be possible, he says, although that is probably still far off.

“That is still an unsolved problem,” Fergus says.

 

 

Leave a Comment

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

Related Ideas

Recommended | Learning in Machines & Brains

The New York Times: The Man Who Helped Turn Toronto Into a High-Tech Hotbed

“Canada beckoned with a research position at the Canadian Institute For Advanced Research. He moved to Toronto and eventually set...

News | Child & Brain Development | Learning in Machines & Brains

Forgetting can make you smarter

For most people having a good memory means being able to remember more information clearly for long periods of time....

News | Learning in Machines & Brains

A biologically plausible technique for training a neural net

Artificial intelligence researchers have made advances in recent years by designing neural networks that are based on the structure of...

Video | Bio-inspired Solar Energy | Institutions, Organizations & Growth | Learning in Machines & Brains | CIFAR Azrieli Global Scholars

Introducing the Future of Research

Three members of the inaugural group of CIFAR Azrieli Global Scholars spoke to an invited audience about their work and...

News | Learning in Machines & Brains

Government renews, increases support for CIFAR, invests in AI Strategy

In the federal budget last month, there were two pieces of good news for CIFAR. First, Ottawa renewed and increased...