A few years ago, many neural networks processed information in a fairly rigid way. Recurrent models moved through sequences step by step, while convolutional networks concentrated heavily on local patterns. Both approaches worked well, but…
Attention Mechanisms