Instructor: Asa Ben-Hur
Email: asa＠colostate.edu

Course description

In this course you will learn the theory and practice of deep learning, from linear and logistic regression to the transformer architecture behind modern language models. Nothing stays hidden: every component is implemented in full, in notebooks you can read, run and take apart. You will train a small language model from scratch and watch fluent English emerge from it, adapt a pretrained model to a new task, and develop a sense of which architecture a problem actually calls for.

Course learning objectives

In this course you will learn to:

• Apply deep neural networks to complex prediction tasks in computer vision and natural language processing
• Interpret what a deep neural network has learned
• Investigate the advantages and limitations of complex deep networks compared to simpler, traditional approaches
• Judge the validity of machine learning claims
• Design, conduct, and report results of thoughtful machine learning experiments

Along the way, you will also learn to:

• Read data files of various formats and visualize characteristics of the data
• Use deep learning libraries such as PyTorch and the standard Python machine learning library, scikit-learn

Setting up your system

For implementation, we will be using Python. Previous experience with Python and the NumPy package is helpful. On Canvas, you will find detailed instructions on how to install the necessary packages for this course.

CS 545 compared to CS 445

Note there is strong overlap between CS 545 and CS 445 in terms of material; CS 545 will go into more depth and assumes greater mathematical maturity.

Please ask questions!

Class meetings will be a combination of lectures by the instructor and discussions of student questions. All questions are welcome, no matter how simple you think they are; it is always true that someone else has a similar question. You can ask questions in class or via the class Teams group.

Textbook

There is no textbook. All course materials will be provided as Jupyter notebooks available through the course’s GitHub repository. The course materials are based on the textbook Dive into Deep Learning. but are being significantly updated to reflect current practices in deep learning.

Topics

During this course we will cover the following topics:

Grading

Your grade in the course will be based on assignments, coding exercises, a final project, and a final exam:

Each assignment (and the course project) will require the submission of a jupyter notebook. Your notebook will be graded for correct implementation and results, thorough discussion of your code and observations. Your notebooks also need to be well-organized, concise with good grammar and spelling.

Late assignments will not be accepted unless you make arrangements with the instructor at least two days before the due date.

Four to five regular assignments are planned during the semester. The final assignment is a project designed by you, and will allow you to explore your choice of datasets with machine learning methodologies.

Grading scale

Final Exam

The final exam will be held at the regular scheduled time according to the registrar’s office.