OpenAI CEO 山姆·阿尔特曼 (Sam Altman) 采访实录之三

本系列为Lex Fridman 采访 OpenAI 的CEO，Sam Altman，的文字实录, 共分为8个部分。本篇为第三部分。

其他章节见豆列 -- Lex Fridman Podcast #367

3. AGI (Artificial General Intelligence)

3-1. Will AI replace human?

3-2. “AI takeoff”

3-3. Is GPT-4 an AGI?

3-4. Is GPT-4 conscious?

Lex: So I've spoken with Noam Chomsky who's been kind of one of the many people that are critical of large language models being able to achieve general intelligence. Right? And so it's an interesting question that they've been able to achieve so much incredible stuff. Do you think it's possible that large language models really are the way we build AGI?

Sam: I think it's part of the way. I think we need other super important things.

Lex: This is philosophizing a little bit. Like what kind of components do you think in a technical sense or a poetic sense… does it need to have a body that can experience the world directly?

Sam: I don't think it needs that, but I wouldn't, I wouldn't say any of this stuff with certainty. Like we're deep into the unknown here. For me, a system that cannot go significantly add to the sum total of scientific knowledge we have access to, kind of discover, invent, whatever you wanna call it, new fundamental science is not a superintelligence. And to do that really well, I think we will need to expand on the GPT paradigm in pretty important ways that we're still missing ideas for. But I don't know what those ideas are. We're trying to find them.

Lex: I could argue sort of the opposite point, that you could have deep, big scientific breakthroughs with just the data that GPTs trained on. If you prompted correctly…

Sam: Look, if an Oracle told me far from the future that GPT-10 turned out to be a true AGI, somehow with maybe just some very small new ideas, I would be like, okay, I can believe that. Not what I would've expected sitting here and would've said a new big idea. But I can believe that.

Lex: This prompting chain, if you extended very far and then increase at scale the number of those interactions, like what kind of these things start getting integrated into human society and starts building on top of each other. I mean, like, I don't think we understand what that looks like. It's like you said, it's been six days.

Sam: The thing that I am so excited about with this is not that it's a system that kind of goes off and does its own thing, but that it's this tool that humans are using in this feedback loop helpful for us for a bunch of reasons. We get to, you know, learn more about trajectories through multiple iterations. But I am excited about a world where AI is an extension of human will and a amplifier of our abilities. And this like, you know, most useful tool yet created. And that is certainly how people are using it. And I mean, just like look at Twitter, like the results are amazing. People's like self-reported happiness with getting to work with us are great. So yeah, like maybe we never build AGI but we just make humans super great. Still a huge win.

Lex: Yeah. I said I'm part of those people. Like the amount, I derive a lot of happiness from programming together with GPT. Part of it is a little bit of terror…

Sam: Can you say more about that?

Lex: There's a meme I saw today that everybody's freaking out about sort of GPT taking programmer jobs. No, the reality is just… it's going to be taking… like if it's going to take your job, it means you were a shitty programmer. There's some truth to that. Maybe there's some human element that's really fundamental to the creative act, to the act of genius that is in great design that is involved the programming. And maybe I'm just really impressed by the all the boiler plate, but that I don't see as boiler plate, but is actually pretty boiler plate.

Sam: Yeah. And maybe that you create, like, you know, in a day of programming you have one really important idea. And that's the contribution. And there may be like… I think we're gonna find… so I suspect that is happening with great programmers and that GPT like models are far away from that one thing, even though they're gonna automate a lot of other programming. But again, most programmers have some sense of, you know, anxiety about what the future's going to look like. But mostly they're like, this is amazing. I am 10 times more productive. Yeah. Don't ever take this away from me. There's not a lot of people that use it and say like, turn this off. You know?

Lex: Yeah. So I think so to speak to the psychology of terror is more like, this is awesome, this is too awesome — I’m scared. There is a little bit of — this coffee tastes too good.

Sam: You know, when Kasparov lost to Deep Blue, somebody said, and maybe it was him that like, chess is over now. If an AI can beat a human at chess, then no one's gonna bother to keep playing. Right. Because like, what's the purpose of us? Or whatever… that was 30 years ago, 25 years ago, something like that. I believe that chess has never been more popular than it is right now. And people keep wanting to play and wanting to watch. And by the way, we don't watch two AI play each other. Yeah. Which would be a far better game in some sense than whatever else. But that's, that's not what we choose to do. Like we are somehow much more interested in what humans do in this sense and whether or not Magnus loses to that kid than what happens when two much, much better AI play each other.

Lex: Well actually, when two AI play each other, it's not a better game by our definition of better…

Sam: Because we just can't understand it.

Lex: No, I think, I think they just draw each other. I think the human flaws — and this might apply across the spectrum here — with AI will make life way better, but we'll still want drama.

Sam: We will, that's for sure.

Lex: We'll still want imperfection and flaws and AI will not have as much of that.

Sam: Look, I mean I hate to sound like utopic tech bro here, but if you'll excuse me for three seconds, like the level of the increase in quality of life that AI can deliver is extraordinary. We can make the world amazing and we can make people's lives amazing. We can cure diseases, we can increase material wealth, we can like help people be happier, more fulfilled, all of these sorts of things. And then people are like, oh, well no one is gonna work. But people want status. People want drama. People want new things. People want to create. People want to like feel useful. People want to do all these things and we're just gonna find new and different ways to do them, even in a vastly better, like unimaginably good standard of living world.

Lex: But that world, the positive trajectories with AI, that world is with an AI that's aligned with humans and doesn't hurt, doesn't limit, doesn't, doesn't try to get rid of humans. And there are some folks who consider all the different problems with the super-intelligent AI system. So one of them is Eliezer Yudkowsky. He warns that AI will likely kill all humans. And there's a bunch of different cases. But I think one way to summarize it is that it's almost impossible to keep AI aligned as it becomes super intelligent. Can you steel man the case for that? And to what degree do you disagree with that trajectory?

Sam: So first of all, I'll say I think that there's some chance of that and it's really important to acknowledge it because if we don't talk about it, if we don't treat it as potentially real, we won't put enough effort into solving it. And I think we do have to discover new techniques to be able to solve it. I think a lot of the predictions — this is true for any new field — but a lot of the predictions about AI in terms of capabilities, in terms of what the safety challenges and the easy parts are going to be have turned out to be wrong. The only way I know how to solve a problem like this is iterating our way through it, learning early and limiting the number of one shot to get it right scenarios that we have. To steel man… well, there's… I can't just pick like one AI safety case or AI alignment case, but I think Eliezer wrote a really great blog post. I think some of his work has been sort of somewhat difficult to follow or had what I view as like quite significant logical flaws. But he wrote this one blog post outlining why he believed that alignment was such a hard problem that I thought was, again, don't agree with a lot of it, but well reasoned and thoughtful and and very worth reading. So I think I'd point people to that as the steel man.

Lex: Yeah, and I'll also have a conversation with him. There is some aspect and I, and I'm torn here because it's difficult to reason about the exponential improvement of technology. But also I've seen time and time again how transparent and iterative trying out as you improve the technology, trying it out, releasing it, testing it, how that can improve your understanding of the technology in such that the philosophy of how to do, for example, the safety of any technology. But AI safety gets adjusted over time rapidly.

Sam: A lot of the formative AI safety work was done before people even believed in deep learning and certainly before people believed in large language models. And I don't think it's like updated enough given everything we've learned now and everything we will learn going forward. So I think it's gotta be this very tight feedback loop. I think the theory does play a real role of course, but continuing to learn what we learn from how the technology trajectory goes is quite important. I think now is a very good time and we're trying to figure out how to do this to significantly ramp up technical alignment work. I think we have new tools, we have new understanding and there's a lot of work that's important to do that we can do now.

Lex: So one of the main concerns here is something called “AI takeoff” or “fast takeoff” that the exponential improvement would be really fast to where…

Sam: Like in Days.

Lex: In days. Yeah, I mean there's, this is an, this is a pretty serious, at least to me it's become more of a serious concern. Just how amazing ChatGPT turned out to be and then the improvement of GPT-4. Yeah, almost like to where it surprised everyone seemingly, you can correct me, including you.

Sam: So GPT-4 is not surprising me at all in terms of reception there. ChatGPT surprised us a little bit but I still was like advocating that we'd do it cuz I thought it was gonna do really great. So like you know, maybe I thought it would've been like the 10th fastest growing product in history and not the number one fastest and like okay, you know, I think it's like hard, you should never kind of assume something's gonna be like the most successful product launch ever. But we thought it was… at least many of us thought it was gonna be really good. GPT-4 has weirdly not been that much of an update for most people. You know, they're like oh it's better than 3.5 but I thought it was gonna be better than 3.5 and it's cool. But you know, this is like someone said to me over the weekend, you shipped an AGI and I somehow like I’m just going about my daily life and I'm not that impressed. And I obviously don't think we shipped an AGI, but I get the point and the world is continuing on.

Lex: When you build or somebody builds an artificial journal intelligence, would that be fast or slow? Would we know it's happening or not? Would we go about our day on the weekend or not?

Sam: So I'll come back to the, would we go about our day or not thing. I think there's like a bunch of interesting lessons from COVID and the UFO videos and a whole bunch of other stuff that we can talk to there. But on the “takeoff” question, if we imagine a two by two matrix of short timelines ’til AGI starts, long timelines ’til AGI starts, slow takeoff, fast takeoff, do you have an instinct on what do you think the safest quadrant would be?

Lex: So the different options are like next year.

Sam: Yeah. Say the takeoff, the we start the takeoff period. Yep. Next year or in 20 Years? 20 Years and, and then it takes one year or 10 years. Well you can even say one year or five years, whatever you want for the takeoff. I feel like now is safer.

Sam: So do I. So I'm in the…

Lex: Longer and now.

Sam: I'm in the slow takeoff short timelines. It's the most likely good world and we optimize the company to have maximum impact in that world to try to push for that kind of a world. And the decisions that we make are, you know, there's like probability masses but weighted towards that. And I think I'm very afraid of the fast takeoffs. I think in the longer timelines it's harder to have a slow takeoff. There's a bunch of other problems too, but that's what we're trying to do. Do you think GPT-4 is an AGI?

Lex: I think if it is just like with the UFO videos, we wouldn't know immediately. I think it's actually hard to know that. When I was playing with GPT-4 and thinking how would I know if it's an AGI or not? Because I think in terms of… to put it in a different way, how much of AGI is the interface I have with the thing and how much of it is the actual wisdom inside of it? Like part of me thinks that you can have a model that's capable of superintelligence and it just hasn't been quite unlocked. What I saw with ChatGPT just doing that little bit of with human feedback makes it think some how much more impressive, much more usable. So maybe if you have a few more tricks, like you said there's like hundreds of tricks inside open ai, a few more tricks and all of a sudden, holy shit, this thing…

Sam: So I think that GPT-4 although quite impressive is definitely not an AGI. But isn't it remarkable we're having this debate?

Lex: Yeah. So what's your intuition why it's not?

Sam: I think we're getting into the phase where specific definitions of AGI really matter or we just say, you know, I know it when I see it and I'm not even gonna bother with the definition. But under the, I know it when I see it, it doesn't feel that close to me. Like if, if I were reading a sci-fi book and there was a character that was an AGI and that character was GPT-4, I'd be like, well this is a shitty book. Like you know, that's not very cool. Like I was, I would've hoped we had done better.

Lex: To me, some of the human factors are important here. Do you think GPT-4 is conscious?

Sam: I think no, but I asked GPT-4 and of course it says no. Do you think GPT-4 is conscious?

Lex: I think it knows how to fake consciousness.

Sam: How to fake consciousness?

Lex: Yeah. If you provide the right interface and the right prompts.

Sam: It definitely can answer as if it were.

Lex: Yeah. And then it starts getting weird. It's like what is the difference between pretending to be conscious and conscious if you trick me?

Sam: I mean you don't know, obviously we can go to like the freshman year dorm late at Saturday night kind of thing. You don't know that you're not a GPT-4 rollout in some advanced simulation.

Lex: Yeah. Yes.

Sam: So if we're willing to go to that level, sure…

Lex: I live in that. Well but that's an important level. That's important, that's a really important level because one of the things that makes it not conscious is declaring that it's a computer program, therefore it can't be conscious. So I'm not going to, I'm not even going to acknowledge it. But that just puts in the category of other. I believe AI can be conscious. So the question is what would it look like when it's conscious? What would it behave like? And it would probably say things like, first of all I'm conscious, second of all, display capability of suffering, an understanding of self, of having some memory of itself and maybe interactions with you. Maybe there's a personalization aspect to it. And I think all of those capabilities are interface capabilities, not fundamental aspects of the actual knowledge inside you on that.

Sam: Maybe I can just share a few like disconnected thoughts here. Sure. But I'll tell you something that Ilya said to me once a long time ago that has like stuck in my head.

Lex: Ilya Sutskever?

Sam: Yes. My co-founder and the chief scientist of OpenAI and sort of legend in the field. We were talking about how you would know if a model were conscious or not. And I've heard many ideas thrown around, but he said one that that I think is interesting — if you trained a model on a data set that you were extremely careful to have no mentions of consciousness or anything close to it in the training process, like not only was the word never there, but nothing about the sort of subjective experience of it or related concepts. And then you started talking to that model about here are some things that you weren’t trained about. And for most of them the model was like, I have no idea what you're talking about. But then you asked it… you sort of described the experience, the subjective experience of consciousness, and the model immediately responded unlike the other questions. Yes, I know exactly what you're talking about. That would update me somewhat.

Lex: I don't know because that's more in the space of facts versus like emotions.

Sam: I don't think consciousness is an emotion.

Lex: I think consciousness is ability to sort of experience this world really deeply. There's a movie called Ex Machina.

Sam: I've heard of it but I haven't Seen it.

Lex: You haven't seen it?

Sam: No.

Lex: The director Alex Garland had a conversation. So it's where AGI system is built, embodied in the body of a woman, and something he doesn't make explicit but he said… he put it in the movie without describing why, but at the end of the movie, spoiler alert, when the AI escapes, the woman escapes, she smiles for nobody, for no audience. She smiles like at the freedom she's experiencing, experiencing, I don't know anthropomorphizing. But he said the smile to me was passing the touring test for consciousness that you smile for no audience, you smile for yourself. It’s an interesting thought. It's like you taking an experience for the experience's sake. I don't know, that seemed more like consciousness versus the ability to convince somebody else that you're conscious and that feels more like a realm of emotion versus facts. But yes, if it knows…

Sam: Yes. So I think there's many other tasks, tests like that that we could look at too. But you know, my personal beliefs… consciousness is of something very strange is going on. I’ll say that.

Lex: Do you think it's attached to the particular medium of our of the human brain? Do you think an AI can be conscious?

Sam: I'm certainly willing to believe that consciousness is somehow the fundamental substrate and we're all just in the dream or the simulation or whatever. Yeah, I think it's interesting how much sort of the Silicon Valley religion of the simulation has gotten close to like Grumman and how little space there is between them but from these very different directions. So like maybe that's what's going on but if it is like physical reality as we understand it and all of the rules of the game and what we think they are, then there's something…I still think it's something very strange.