PROG74040 — Advanced Topics in Artificial Intelligence and Machine Learning

I teach advanced topics in artificial intelligence and machine learning through examples students can follow from observation to mathematics and implementation. Two I return to are reinforcement learning and Transformer architectures.

What I want students to learn

I want students to explain how an agent can learn from the consequences of its actions and connect that explanation to a value update in code. For a Transformer, I want them to follow an example through the architecture and describe the role of its parts.

Representative classroom activities

From FrozenLake to Q-learning

I introduce an agent moving across a frozen lake. We follow an action, the state it leads to and the reward it encounters. Once the class can describe a move, we ask how an action can be valuable before a reward arrives. That question gives the Bellman equation a purpose. We then connect the update to a Q-learning implementation, with the lake as a reference for what each part of the code represents.

Following a translation through a Transformer

I reveal an animated Transformer diagram in stages while following a translation example. At each stage, students can connect a component to a change in the example. We return to the complete architecture so that the parts they have just seen make sense together.