Sparse Neural Networks

Modern neural networks have a strange efficiency problem. They can contain millions or billions of parameters, yet many of those connections may contribute very little to the final prediction. Dense models activate nearly everything by…