PhD Position F/M Fine-Tuning Foundation Models for Physics Based Rendering and Editing of Mixed 3D Scenes

Contexte et atouts du poste

The Ph.D. thesis will take place at the GRAPHDECO group (

Mission confiée

This research project will explore the exciting prospects of fine-tuning modern foundational models for creating, editing, and rendering realistic 3D digital environments for the case of mixed scenes, i.e., scenes containing 3D content which is artist-created, AI generated, or reconstructed from images/video.

While captured representations of real-world scenes offer compelling realism, they currently lack the flexibility of traditional artist-created assets, particularly when it comes to modifying lighting, materials, and geometry.

In addition, AI generated content is very flexible, but hard to control, especially with respect to physics-based rules. This project will investigate several different avenues of model fine-tuning, e.g., using synthetic data generated with our rendering algorithms, but also using the pow...