"Understand why RLHF flattened AI creative voice Researcher Jasmine Sun traced the decline in AI writing quality to a specific training decision. Earlier models like GPT-2 were 'weird,' 'variable,' and 'surprising,' but post-training layers — including human feedback rating systems — optimized them toward helpfulness. She explained: 'These post-trained models have been trapped in a way or trained or guided towards a very particular character or persona that is a very helpful assistant but might be very bad at writing in creative and surprising ways.' Anyone trying to use AI for creative work should understand that the blandness is a designed outcome of optimization for corporate utility, not a fundamental ceiling. Source: Jasmine Sun, Hard Fork, AI Layoffs and the Human Skill That Eludes AI"