Artificial Intelligence and thought vectors

Since I sold my latest start-up, I have been looking into ways to make sense of the deluge of data created by new technologies; data that Google, Facebook, Twitter and the like only partially organize.

I have had the privilege to take a journey without a PHD supervisor or having had to write proposals, something that would have been required in an academic environment. I have explored different areas: computation, descriptions, understanding and epistemology. I have looked into neuroscience, bio-inspired and traditional AI models and lately deep learning.

Deep learning is so successful in some real world applications that it is inspiring computational neuroscientists, even though deep learning has its roots in mathematics and computer science, not bio-inspiration. From the artificial intelligence perspective, we are likely living a watershed moment, one in which we realize that traditional symbolic AI models are being superseded by new data-driven approaches. The thinking was that AI was about rule based systems, painstakingly programmed rule by rule; it is becoming apparent that there are more powerful methods that extract structure from data in a semi-supervised way.

If you are interested in AI (and what a thought vector might be) the video above is a talk by Professor Geoffrey Hinton at the Royal Academy in London. I was there and asked a question about the limitation of the model for which I am looking for a solution; my question is at 39:20.

My critique about thought vectors in particular and Deep Learning in general is that, at the core, they are statistical techniques; as such they need a lot of data to settle on to averages. That is why there are tools, say Google search, which are able to find averages well, but do not find specific information, for which there is not enough statistics. The very foundation on which these models are built has taken us down to a path where the average wins; but in the real world I look for deviations from the average, that is what I am interested in.