Computers learn by playing with blocks

by Dan Falk Learning in Machines & Brains News 06.04.2016
The program learns by looking at images of stacked blocks – some of which will topple.

When an infant plays with wooden blocks, it’s not just playing – it’s also learning about the physical world by interacting with it, pushing and pulling and poking at its various parts. Now a team of artificial intelligence researchers at Facebook are trying to get a computer program to understand its environment in the same way.

The program, called a convolutional neural network, learns by looking at hundreds of thousands of computer-generated images of stacked blocks – some of them stable, some unstable. If the stack is unstable, the network sees where the blocks end up after the tower topples over. When it’s then exposed to new images – either computer-generated images or pictures of real-life wooden blocks – the network does a remarkably good job of predicting whether the stacks are stable, and if not where the various blocks will end up.

On the computer-generated data, the network performs better than humans do, says Rob Fergus (New York University, Facebook AI Research), a senior fellow in CIFAR’s Learning in Machines & Brains program. “And on real-world data, it’s almost as good as humans,” he says.

Importantly, the network is never taught the rules of mechanics, or of gravity – it simply watches and learns, similar to the way a child learns.

“I don’t think we have Newton’s laws baked into our genes,” says Fergus. “It’s much more likely that we learn by trying things out. When you’re an infant, you play with these blocks, and you get an instinctive feeling. ‘The tower’s about to tip over. The top block is going to fall over, to the left.’ That kind of thing.”

For an infant, playing with blocks may seem like a game, but the child is actually learning how to take in a visual scene and predict what’s going to happen next. And that, in turn, depends on “a kind of physical common sense,” Fergus says. “At the moment, machines don’t have that.”

Networks like the kind Fergus works on still need to be told what to pay attention to. But some day a machine capable of completely unsupervised learning may be possible, he says, although that is probably still far off.

“That is still an unsolved problem,” Fergus says.



Leave a Comment

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

Related Ideas

Learning in Machines & Brains | News

A ‘surprisingly popular’ way to extract group wisdom

Is Philadelphia the capital of Pennsylvania? The answer may surprise you. Even more important, just how many people are surprised...

Learning in Machines & Brains | News

Learning algorithms find a new music teacher

Computers have a new music teacher to help them master the likes of Johann Sebastian Bach and Wolfgang Amadeus Mozart...

Learning in Machines & Brains | Case Study

Making artificial intelligence an everyday reality

One of today’s most exciting areas of artificial intelligence research focuses on “deep learning”. This case study outlines the critical...

Learning in Machines & Brains | Recommended

Scientific American | Springtime for AI: The Rise of Deep Learning

Computers generated a great deal of excitement in the 1950s when they began to beat humans at checkers and to...

Learning in Machines & Brains | News

Computers recognize memorable images

Why do some images stay fixed in our memories, while others quickly fade away? Researchers have developed a deep learning...