Advances in large language models recently popularized by ChatGPT represent a remarkable leap forward in language processing by machines. We invite you to join the conversation shaping the future of communication technology. What does this mean for us, how can we make the most of these advancements, and what are the risks? What research opportunities have opened up? What kinds of evaluation are called for? We will bring together a group of practitioners and experts for guided discussions, hands-on experimentation, and project critiques. If you want to join the class, please fill out this interest form and come to the first class on Wednesday, 2/8. Bring a laptop and be prepared to start experimenting!
This course will be formatted as a combination workshop and seminar. Students will engage through readings, class participation, and project work. Students may choose to either complete a project or produce a research project proposal. For the active project track, students will form teams, pitch projects, and get feedback along the way. For the project proposal track, they will present a literature review mid-semester, and submit a written research project proposal. Project should be focused on one of the main areas identified by the course. We will come together to share and critique projects through the semester, culminating in final project presentations. Students will also be expected to present to the class on readings and hands-on workshop output.
The current class schedule is below (subject to change)
Date | Description | Course Materials |
---|---|---|
Feb 8 | Part 1: Background on LLMs
[Slides]
Create a prompting task in groups. |
Required Readings: |
Feb 15 | Part 1: Evaluating models
[Slides]
How can we best evaluate these models for accuracy, fairness, bias, robustness, and other factors? Speaker: Rishi Bommasani (Stanford) Title: Holistically Evaluating Language Models on the Path to Evaluating Foundation Models Part 2: LLMs in Applications [Slides] People are increasingly interacting with human-facing tools that incorporate LLMs, like ChatGPT, writing assistants, and character generators. How might we go about evaluating these systems and their impacts on people? In this session we will consider 10 recent commercial and research applications of LLMs. Students will be asked to come prepared to critique the designs of one of these applications along different dimensions that we will describe in week 1. |
Required Readings:
Recommended Readings:
|
Feb 22 | Part 1: Using LLMs for Consensus Across Preferences
[Slides]
Speaker: Michiel Bakker (DeepMind) Title: Fine-tuning Language Models to Find Agreement among Humans with Diverse Preferences Part 2: Project Pitch [Slides] Students present their project idea and form teams. |
Required Readings: |
Mar 1 |
Part 1: Emergent Abilities of LLMs
[Slides] This talk will cover broad intuitions about how large language models work. First, we will begin by examining some examples of what language models can learn by reading the internet. Second, we will consider why language models have gained traction recently and what new abilities they have that were not present in the past. Third, we will cover how language models can perform complex reasoning tasks. Finally, the talk will discuss how language models can have an improved user interface via instruction following. Speaker: Jason Wei (OpenAI) Title: Emergence in Large Language Models Part 2: NLP Evaluation Methods and Red Teaming [Slides] |
Required Readings: Recommended Readings: |
Mar 8 | Part 1: Evaluating Human-model Interactions
[Slides] Speaker: Mina Lee (Stanford) Title: Designing and Evaluating Language Models for Human Interaction Part 2: Human Experiments and Evaluation Methods [Slides] |
Required Readings: Recommended Readings: |
Mar 15 |
Media Lab Research Panel Speaker: Ziv Epstein (PhD Student at MIT Media Lab, Human Dynamics) Title: Social Science Methods for Understanding Generative AI Speaker: Matt Groh (PhD Student at MIT Media Lab, Affective Computing) Title: Deepfake Detection Speaker: Trudy Painter (UROP/MEng at MIT Media Lab, Viral Communications) Title: Latent Lab: Generative ML as an Exploration Partner Speaker: Belén Saldias Fuentes (PhD Student at MIT Media Lab, MIT Center for Constructive Communication) Title: Community-aligned Content Moderation with Rationale Generation Speaker: Hang Jiang (PhD Student at MIT Media Lab, MIT Center for Constructive Communication) Title: CommunityLM: Probing Partisan Worldviews from Language Models |
Related Readings: Ziv Epstein:
|
Mar 22 | Part 1: AI-Mediated Communication
[Slides] This talk will discuss the phenomenon of AI-Mediated Communication (AI-MC) and its potential impact on human communication outcomes, language use, and interpersonal trust. The author outlines early experimental findings showing that AI involvement can shift written content and opinions, change message ownership, impact blame assignment, and affect trust evaluations, highlighting the need for new approaches to the development and deployment of these technologies. Speaker: Mor Naaman (Cornell Tech) Title: "My AI must have been broken": Understanding our Future of AI-Mediated Communication Part 2: Discussion on the public policies on AI-generated content. [Slides] |
Required Readings: Recommended Readings: |
Mar 29 | Break | |
Apr 5 | Part 1: LLMs as Simulated Agents
[Slides]
Speaker: John Horton (MIT) Title: Large Language Models as Simulated Economic Agents: What Can We Learn from Homo Silicus? Part 2: Discussion on the call for 6-month AI morotorium: "Pause Giant AI Experiments: An Open Letter". [Slides] |
Required Readings:
Recommended Readings:
|
Apr 12 | Societal Impacts of LLMs [Slides] |
Required Readings:
|
Apr 19 | Risks and Tools for Transparency
[Slides] |
Required Readings:
|
April 26 | Final project presentations I | |
May 3 | Final project presentations II |
|
May 10 | No Class (work on final papers) |
|
May 17 | Project Submission Deadline |
|