Revealing and Leveraging the Visual Information in Diffusion Models

Generating high-quality photo-realistic and creative visual content using diffusion models is a thriving area of research. In this talk, I will focus not on the generation process, but on understanding and leveraging the rich visual semantic information represented within diffusion models. Specifically, I will present our work that uses mechanistic interpretability tools such as k-sparse autoencoders (k-SAE) to probe various layers and denoising timesteps of different diffusion architectures. Next, I will present how to uncover monosemantic interpretable concepts pertaining to safety and photographic styles and steer the generation process thereby offering more controllability to users.