Attention Is All* You Need * (except for fully connected layers, softmax, positional encoders, layer norm, skip connections…)…

2019-02-06 IFTTT, Twitter, jeremyphoward

Attention Is All* You Need

* (except for fully connected layers, softmax, positional encoders, layer norm, skip connections…) pic.twitter.com/BnSQD9oEPA
— Jeremy Howard (@jeremyphoward) February 6, 2019

(via http://twitter.com/jeremyphoward/status/1093273889449205760)