Jensen Huang and Ilya Sutskever on GPT-4, March 2023
44:12 “Yeah, that’s incredible. You know, one of the things that you said earlier about an AI generating, generating tests to train another AI, you know, there was a paper that was written about and I don’t, I don’t completely know whether it’s factual or not, but, but that there’s a total amount of somewhere between 4 trillion to something like…” Share
Jensen Huang speaks with Ilya Sutskever just after GPT-4’s release about the path from neural networks and AlexNet to OpenAI’s GPT models. Sutskever explains unsupervised learning as compression, scaling, reinforcement learning and human feedback, then discusses GPT-4’s reasoning, reliability, retrieval limits and visual capabilities. They close on synthetic data, future reliability and the unexpected effectiveness of large neural networks.
Passages
20:29 “You know, there’s a misunderstanding that ChatGPT is in itself just one giant large language model. There’s a system around it that’s fairly complicated.” Share
25:22 “ChatGPT came out just a few months ago. Fastest growing application in the history of humanity.” Share
35:48 “Multimodality GPT4 has the ability to learn from text and images and respond to input from text and images. First of all, the foundation of multimodality learning, of course, transformers has made it possible for us to learn from multimodality tokenized text and images, but at the foundational level, help us understand how multimodality enhances the understanding of the world…” Share
51:49 “The models that you’ve trained and the amount of data you’ve trained from the, what you did on Alexnet to now is about a million times and no one in the world of computer science would have believed that the amount of computation that was done in that 10 years time would be a million times larger. And that you…” Share
Search what Huang said from Claude, ChatGPT or your own code. See plans