"Solve the stable training objective first When generative AI was dominated by GANs, the founder of Inception faced a training process he described as 'very very unstable, very very difficult to get to work' because two neural networks had to out-compete each other in a game-theoretic loop. His team's breakthrough was reframing the problem as a simpler, stable task: 'train a neural network to remove noise,' which is 'a fairly standard, relatively easy kind of like optimization problem.' Anyone building or evaluating a new technical approach should ask whether the core training objective is inherently stable before investing in scale. Source: Arash Vahdat, Latent Space Podcast, Diffusion LLMs with Inception AI"