Currently, neural networks like DeepMind and Watson need to perform billions of tasks in parallel, requiring numerous CPU memory calls. Placing large amounts of resistive random access memory directly onto a CPU would solve this, as such chips can fetch data as quickly as these can process it, thereby reducing neural network training times and power required.
The research paper claims that, “This massively-parallel RPU architecture can achieve acceleration factors of 30,000 compared to state-of-the-art microprocessors—problems that currently require days of training on a data centre size cluster with thousands of machines can be addressed within hours on a single RPU accelerator.” Although these chips are still in the research phase, scientists say that these can be built using regular complementary metal-oxide semiconductor technology.
Locally-done deep learning.
Speaking of reducing processing power and learning times, can you imagine deep learning being performed locally on a mobile phone, without depending on the Cloud? Well, Apple revealed at 2016’s Worldwide Developer’s Conference (WWDC) that it can do precisely that. The company announced that it is applying advanced, deep learning techniques to bring facial recognition to iPhone, and it is all done locally on the device. Some of this success can be attributed to Perceptio, a company that Apple acquired last year. Perceptio is developing deep learning tech that allows smartphones to identify images without relying on external data libraries.
Facebook intends to understand your intent
Facebook Artificial Intelligence Research (FAIR) group has come out with innumerable innovations, which are so deeply woven with their products that we do not even realise that we are using deep learning every time we use Facebook. FbLearner Flow is Facebook’s internal platform for machine learning. It combines several machine learning models to process data points drawn from the activity of the site’s users, and makes predictions such as which user is in a photograph, which post is spam and so on. Algorithms that come out of FbLearner Flow help Facebook to identify faces in photos, select content for your news feed and more.
One of their recent innovations is DeepText, a deep learning based text understanding engine that can understand the textual content posted on Facebook in 20-plus languages. Understanding text might be easy for humans, but for a machine it includes multiple tasks such as classification of a post, recognition of entities, understanding of slang, disambiguation of confusing words and so on.
All this is not possible using traditional NLP methods, and makes deep learning imperative. DeepText uses many DNN architectures, including convolutional and recurrent neural nets to perform word-level and character-level learning. FbLearner Flow and Torch are used for model training.
But, why would Facebook want to understand the text posted by users? Understanding conversations helps to understand intent. For example, if a user says on Messenger that “the food was good at XYZ place,” Facebook understands that he or she is done with the meal, but when someone says, “I am hungry and wondering where to eat,” the system knows the user is looking for a nearby restaurant. Likewise, the system can understand other requirements like the need to buy or sell something, hail a cab, etc. This helps Facebook to present the user with the right tools that solve their problems. Facebook is also trying to build deep learning architectures that learn intent jointly from textual and visual inputs.
Facebook is constantly trying to develop and apply new deep learning technologies. According to a recent blog post, bi-directional recurrent neural nets (BRNNs) show a lot of promise, “as these aim to capture both contextual dependencies between words through recurrence and position-invariant semantics through convolution.” The teams have observed that BRNNs achieve lower error rates (sometimes as low as 20 per cent) compared to regular convolutional or recurrent neural nets for classification.
If Google open-sourced its deep learning software engine, Facebook open-sourced its AI hardware last year. Known as Big Sur, this machine was designed in association with Quanta and NVIDIA. It has eight GPU boards, each containing dozens of chips. It has been found that deep learning using GPUs is much more efficient compared to the use of traditional processors. GPUs are power-efficient and help neural nets to analyse more data, faster.
In a media report, Yann LeCun of Facebook said that, open-sourcing Big Sur had many benefits. “If more companies start using the designs, manufacturers can build the machines at a lower cost. And in a larger sense, if more companies use the designs to do more AI work, it helps accelerate the evolution of deep learning as a whole—including software and hardware. So, yes, Facebook is giving away its secrets so that it can better compete with Google—and everyone else.”