Search

Home > This Week in Machine Learning & Artificial Intelligence (AI) Podcast > Single Headed Attention RNN: Stop Thinking With Your Head with Stephen Merity - #325
Podcast: This Week in Machine Learning & Artificial Intelligence (AI) Podcast
Episode:

Single Headed Attention RNN: Stop Thinking With Your Head with Stephen Merity - #325
Category: Technology
Duration: 00:59:04
Publish Date: 2019-12-12 13:04:00
Description:

Today we’re joined by Stephen Merity, startup founder and independent researcher, with  a focus on NLP and Deep Learning. In our conversation, we discuss:

  • Stephen’s newest paper, Single Headed Attention RNN: Stop Thinking With Your Head.
  • His motivations behind writing the paper; the fact that NLP research has been recently dominated by the use of transformer models, and the fact that these models are not the most accessible/trainable for broad use.
  • The architecture of transformers models.
  • How Stephen decided to use SHA-RNNs for this research.
  • How Stephen built and trained the model, for which the code is available on Github.
  • His approach to benchmarking this project.
  • Stephen’s goals for this research in the broader NLP research community. 

The complete show notes for this episode can be found at twimlai.com/talk/325. There you’ll find links to both the paper referenced in this interview, and the code. Enjoy!
Total Play: 0