2 things limiting advances in machine learning… and what it would take to overcome them

In its current form, the vast majority of artificial intelligence (A.I.) relies on some version of machine learning. Almost all of the serious A.I. efforts we’ve seen thus far are in actuality exercises in adaptive machine learning (which we discussed before here). We feed neural networks made up of dozens, hundreds or thousands of computer chips (whether CPUs or GPUs or both) a mountain of data, it uses algorithms to sort and make sense of all that data, recognizes patterns from that data, and then draws conclusions based on those patterns.

To quote MIT Professor Patrick Winston, these advancements are more an achievement in “computational statistics” than ‘real’ artificial intelligence. The machines aren’t making sentient conclusions — they’re making the most informed choices they can given the data presented and the algorithms programming their behavior.

Now, that’s not to discredit or downplay the impact, importance or potential of this type of brute-force machine learning. In many ways, it presages what all is to come; and, until we can get this right, the higher forms (read: more powerful/useful) of A.I. will still elude us.

So what’s the foremost limitation of this brand of A.I. as currently constructed?

Two things:

1. It requires a boatload of data to work
2. To receive timely answers requires a mountain of computing power

The MIT Technology Review puts it in perspective:

“Unlike humans, who can recognize coffee cups from seeing one or two examples, AI networks based on simulated neurons need to see tens of thousands of examples in order to identify an object. Imagine trying to learn to recognize every item in your environment that way, and you begin to understand why AI software requires so much computing power.”

When looking at like this, the primary advancement needed is pretty simple to suss out — we need to develop neural networks/machine learning protocols that require less data and less computing power to perform the same tasks A.I. is tackling today.

No small feat, I realize, but it’s the holy grail of adaptive machine learning as we understand it today. And that long-awaited advancement might be upon on.

“If researchers could design neural networks that could be trained to do certain tasks using only a handful of examples, it would ‘upend the whole paradigm,’ Charles Bergan, vice president of engineering at Qualcomm, told the crowd at MIT Technology Review’s EmTech China conference earlier this month,” according to to MITTR. “If neural networks were to become capable of ‘one-shot learning,’ Bergan said, the cumbersome process of feeding reams of data into algorithms to train them would be rendered obsolete.”

Nvidia, one of the cornerstones of the A.I. movement given their preeminence in the GPU space (which most neural networks rely on far more so than CPUs), are efforting this kind of advancement as we speak. By employing a process called “network pruning”, the company is training its neural networks to become smaller and more efficient by eliminating the ‘neurons’ that don’t directly contribute to output.

“There are ways of training that can reduce the complexity of training by huge amounts,” Bill Dally, chief scientist at Nvidia, said at the same EmTech China conference.

This is but one approach to the elusive goal of ‘one-shot’ machine learning. Google’s DeepMind and a host of startups are efforting the same goal right now. And if one of them gets it right, it will end up changing computing forever.