Attention, Transformers, and LLMs: a hands-on introduction in Pytorch#

Instructor#

  • Instructor: D. Hudson Smith

  • Office: 2105 Barre Hall, Clemson University

  • Email: dane2 AT clemson DOT edu

Workshop Description#

This workshop focuses on developing an understanding of the fundamentals of attention and the transformer architecture so that you can understand how LLMs work and use them in your own projects.

Prerequisites#

All students should have a Palmetto Cluster account. If you do not already have an account, you can visit our getting started page. Students should be familiar with the Python programming language. This requirement could be fulfilled by personal projects, coursework, or completion of the Introduction to Python Programming workshop series. The workshop assumes experience in numeric computing with the numpy, pytorch, tensorflow, or similar. Students are encouraged to work through our Introduction to Deep Learning with Pytorch series if they have no prior Pytorch experience.