December 14, 2020
...AND CHEAPER...:
Tiny four-bit computers are now all you need to train AI: Powerful neural networks could soon train on smartphones with dramatically faster speeds and less energy. (Karen Hao, December 11, 2020, MIT Technology Review)
Deep learning is an inefficient energy hog. It requires massive amounts of data and abundant computational resources, which explodes its electricity consumption. In the last few years, the overall research trend has made the problem worse. Models of gargantuan proportions--trained on billions of data points for several days--are in vogue, and likely won't be going away any time soon.Some researchers have rushed to find new directions, like algorithms that can train on less data, or hardware that can run those algorithms faster. Now IBM researchers are proposing a different one. Their idea would reduce the number of bits, or 1s and 0s, needed to represent the data--from 16 bits, the current industry standard, to only four.The work, which is being presented this week at NeurIPS, the largest annual AI research conference, could increase the speed and cut the energy costs needed to train deep learning by more than sevenfold.
Posted by Orrin Judd at December 14, 2020 12:00 AM