Search

Home > Programming Throwdown > 172: Transformers and Large Language Models
Podcast: Programming Throwdown
Episode:

172: Transformers and Large Language Models

Category: Education
Duration: 01:26:08
Publish Date: 2024-03-11 15:00:00
Description:

172: Transformers and Large Language Models


Intro topic: Is WFH actually WFC?

News/Links:


Book of the Show


Patreon Plug https://www.patreon.com/programmingthrowdown?ty=h


Tool of the Show


Topic: Transformers and Large Language Models

  • How neural networks store information
    • Latent variables
  • Transformers
    • Encoders & Decoders
  • Attention Layers
    • History
      • RNN
        • Vanishing Gradient Problem
      • LSTM
        • Short term (gradient explodes), Long term (gradient vanishes)
    • Differentiable algebra
    • Key-Query-Value
    • Self Attention
  • Self-Supervised Learning & Forward Models
  • Human Feedback
    • Reinforcement Learning from Human Feedback
    • Direct Policy Optimization (Pairwise Ranking)



★ Support this podcast on Patreon ★
Total Play: 0