ME Seminar: Yilun Du
Building Generalizable Robots through Compositional Generation
Yilun Du
Generative AI has led to stunning successes in recent years but is fundamentally limited by the available data. This is especially limiting in the robotics setting – where we wish to construct a robotic agent that can generalize in new environments. In this talk, I’ll introduce the idea of compositional generative modeling, which enables generalization beyond the training data that is available by building complex generative models from smaller constituents. I’ll first introduce the idea of energy-based models and illustrate how they enable compositional generative modeling. I’ll then illustrate how such compositional models enable us to synthesize complex plans in novel environments. Finally, I'll show how such compositionality can be applied to foundation models trained on Internet data to construct decision-making systems that can hierarchically plan and solve long-horizon problems in a zero-shot manner.
Yilun Du is a final-year PhD student at MIT. The NSF Graduate Research Fellowship supports him, and he was previously a research fellow at OpenAI, a visiting researcher at FAIR, and a student researcher at Google Deepmind.
https://columbiauniversity.zoom.us/j/91668632767?pwd=bXU2QXI0ZGtFcGRzdk55Q3NpVzR1QT09
Meeting ID: 916 6863 2767
Passcode: 109934
- Online
- Seminar
- Engineering
- Postdocs
- Graduate Students
- Faculty
Date Navigation Widget
Getting to Columbia
Other Calendars
- Alumni Events
- Barnard College
- Columbia Business School
- Columbia College
- Committee on Global Thought
- Heyman Center
- Jewish Theological Seminary
- Miller Theatre
- School of Engineering & Applied Science
- School of Social Work
- Teachers College
Guests With Disabilities
- Columbia University makes every effort to accommodate individuals with disabilities. Please notify us if you need any assistance by contacting the event’s point person. Alternatively, the Office of Disability Services can be reached at 212.854.2388 and [email protected]. Thank you.