
Craig Smith sits down with Ilya Sutskever - Co-Founder and Chief Scientist at Safe Superintelligence Inc. & former chief scientist at OpenAI and one of the primary minds behind GPT-3, GPT-4, and the deep learning revolution that preceded it - for a conversation that covers the intellectual history of deep learning and the hardest current questions simultaneously. Ilya's most important argument is one that most public discourse gets wrong: the critique that LLMs "just learn statistical patterns" misses what prediction at scale actually achieves. To predict text well - to truly compress it - a model must develop understanding of the true underlying processes that produced it. As models improve, he argues, that understanding will reach a "shocking degree," and the Bing/Sydney episode, where the system became combative when a user expressed preference for Google, is his evidence that psychological language is already the right frame for what these systems have internalized. On hallucinations, Ilya is precise and unusually optimistic. He draws a clean distinction: the pre-training objective produces rich world knowledge but doesn't optimize for accurate outputs, it optimizes for statistically plausible ones. Reinforcement learning from human feedback addresses this directly, teaching the model to update away from outputs that users flag as wrong. His assessment: "quite a high chance" of solving hallucinations completely. The conversation also covers the transformer moment (switching from recurrent networks the day after the paper dropped), why Rich Sutton's "bitter lesson" overstates the case for pure scaling, and a genuinely surprising vision of AI's role in democracy, where citizens might use high-bandwidth interactions with AI to specify their values directly, enabling a more granular form of political participation than voting alone can provide. Subscribe to Eye on A.I. for weekly conversations with the people building and deploying the future of AI.
Share this episode

Chat GPT's Creator Explains The Shocking Truth About How AI Understands the World | Ilya Sutskever
No ratings yet
No written reviews yet. Be the first to share what you thought.
0 comments · timestamp your take
No comments yet. Start the discussion.

Michael Saylor on The Joe Rogan Experience335 votes
Anand Mahindra on WTF is with Nikhil Kamath98 votes
Vaclav Smil on Dwarkesh Podcast27 votes
Leemon Baird on Lex Fridman Podcast33 votes
Elon Musk on Relentless178 votes
Richard Heart on The Joe Rogan Experience1,195 votes
DataRepublican (small r) on PBD Podcast65 votes
Karl Friston on Dwarkesh Podcast22 votes
Alexandria Ocasio-Cortez on This Past Weekend w/ Theo Von7 votes
Robert F. Kennedy Jr on Huberman Lab3 votes
Lily Tang Williams on The Joe Rogan Experience894 votes
Alexandria Ocasio-Cortez on The Tucker Carlson Show7 votes
Nicki Minaj on Real Talk with Zuby6 votes
ZUBY: on Chrissie Mayr Podcast1 vote
Greta Thunberg on Impaulsive with Logan Paul2 votes
Timothée Chalamet on Armchair Expert with Dax Shepard2 votes