Dario Amodei — “We are near the end of the exponential”

…yeah i would say actually the underlying technology like the exponential of the technology has gone broadly speaking i would say about as i expected it to…filler · filler

…technology has gone broadly speaking i would say about as i expected it to go i mean there's like plus or minus you know a couple there's plus or minus a year or two here there's plus…hedge · filler

…at the exponential it is roughly what i expected in terms of the march of the models from like you know smart high school student to smart college student to like you know beginning to do ph…filler · filler

…the march of the models from like you know smart high school student to smart college student to like you know beginning to do ph d and professional stuff and in the case of code reaching beyond…filler · filler

…you have people talking about these you know just the same tired old hot button political issues and like you know around us we're like near the end of the exponential i want yeah so i have…filler · filler

…you know just the same tired old hot button political issues and like you know around us we're like near the end of the exponential i want yeah so i have actually the same hypothesis that i…filler · filler

…all the cleverness all the techniques all the kind of we need a new method to do something like that doesn't matter very much there are only a few things that matter and i think i listed…proper use

…there are only a few things that matter and i think i listed seven of them one is like how much raw compute you have the other is the quantity of data that you have then the…filler · filler

…such objective function right another objective function is you know the kind of rl objective function that says like you have a goal you're going to go out and reach the goal within that of course there's…quotative · filler

…a goal you're going to go out and reach the goal within that of course there's objective rewards like you know like you see in math and coding and there's more subjective rewards like you see in…filler · filler

…going to go out and reach the goal within that of course there's objective rewards like you know like you see in math and coding and there's more subjective rewards like you see in rl from human…proper use

…there's objective rewards like you know like you see in math and coding and there's more subjective rewards like you see in rl from human feedback or kind of higher order versions of that and then the…proper use

…kind of higher order versions of that and then the sixth and seventh were things around kind of like normalization or conditioning like you know just getting the numerical stability so that kind of the big blob…filler · filler

…versions of that and then the sixth and seventh were things around kind of like normalization or conditioning like you know just getting the numerical stability so that kind of the big blob of compute flows in…filler · filler

…scaling laws were one example of kind of what we see there and indeed those have continued going like you know you know i think now it's been widely reported like you know we feel good about…filler · filler

…and indeed those have continued going like you know you know i think now it's been widely reported like you know we feel good about pre training like pre training is continuing to give us gains what…filler · filler

…you know i think now it's been widely reported like you know we feel good about pre training like pre training is continuing to give us gains what has changed is that now we're also seeing the…filler · filler

…seeing the same thing for rl right so we're seeing a pre training phase and then we're seeing like an rl phase on top of that and with rl it's actually just the same like you know…hedge · filler

…we're seeing like an rl phase on top of that and with rl it's actually just the same like you know even other companies have published like you know in some of their releases have published things…filler · filler

…of that and with rl it's actually just the same like you know even other companies have published like you know in some of their releases have published things that say look you know we train the…filler · filler

…trained on these data sets that didn't represent a wide you know distribution of text right you had like you know these very standard you know kind of language modeling benchmarks and gpt 1 itself was trained…filler · filler

…was trained on a bunch of i think it was fan fiction actually but you know it was like literary you know it was like literary text which is a very small fraction of the text that…filler · filler

…i think it was fan fiction actually but you know it was like literary you know it was like literary text which is a very small fraction of the text that you get and what we found…filler · filler

…text that you get and what we found with that you know and in those days it was like a billion words or something so small data sets and represented a pretty narrow distribution right like a…hedge · filler

…was like a billion words or something so small data sets and represented a pretty narrow distribution right like a narrow distribution of kind of what you can see in the world and it didn't generalize well…proper use

…wouldn't generalize that well to kind of the other tab you know we had all these measures of like you know how well does the model do at predicting all of these other kinds of texts you…filler · filler

…on the you know the internet when you kind of did a general internet scrape right from something like you know common crawl or scraping links on reddit which is what we did for gpt 2 it's…proper use

…i think we're seeing the same thing on rl that we're starting with first very simple rl tasks like training on math competitions then we're kind of moving to you know kind of broader training that involves…proper use

…on math competitions then we're kind of moving to you know kind of broader training that involves things like code as a task and now we're moving to do kind of many many other tasks and then…proper use

…which is that on pre training when we train the model on pre training you know we use like trillions of tokens right and humans don't see trillions of words so there is an actual sample efficiency…hedge · filler

…trained if we give them a long context length the only thing blocking a long context length is like inference but if we give them like a context length of a million they're very good at learning…filler · filler

…context length the only thing blocking a long context length is like inference but if we give them like a context length of a million they're very good at learning and adapting within that context length and…hedge · filler

…know the full answer to this but i think there's something going on that pre training it's not like the process of humans learning it's somewhere between the process of humans learning and the process of human…proper use

…of humans learning it's somewhere between the process of humans learning and the process of human evolution it's like it's somewhere between like we get many of our priors from evolution our brain isn't just a blank…filler · filler

…somewhere between the process of humans learning and the process of human evolution it's like it's somewhere between like we get many of our priors from evolution our brain isn't just a blank slate right whole books…filler · filler

…have been written about i think the language models they're much more blank slates they literally start as like random weights whereas the human brain starts with all these regions it's connected to all these inputs and…proper use

…something between long term human learning and short term human learning so you know there's this hierarchy of like there's evolution there's long term learning there's short term learning and there's just human reaction and the lom…filler · filler

…right that was the transition from gpt 1 to gpt 2 that i saw up close which is like you know the model reaches a point you know i like had these moments where i was like…filler · filler

…that i saw up close which is like you know the model reaches a point you know i like had these moments where i was like oh yeah you just give the model like you just give…filler · filler

…like you know the model reaches a point you know i like had these moments where i was like oh yeah you just give the model like you just give the model a list of numbers that's…quotative · filler

…you know i like had these moments where i was like oh yeah you just give the model like you just give the model a list of numbers that's like you know you know this is the…filler · filler

…oh yeah you just give the model like you just give the model a list of numbers that's like you know you know this is the cost of the house this is the square feet of the…filler · filler

…this is the square feet of the house and the model completes the pattern and does linear regression like not great but it does it but it's never seen that exact thing before and so you know…filler · filler

…a specific document or a specific skill but because we want to generalize i think of it as like two there's kind of two cases to be made here all right two claims you could make one…filler · filler

…of two cases to be made here all right two claims you could make one of which is like stronger and the other of which is weaker so i think starting with the weaker claim you know…filler · filler

…so i think starting with the weaker claim you know when i first saw the scaling back in like you know 2019 you know i wasn't sure you know this was the whole this was kind of…filler · filler

…that was you know and my claim was this is much more likely than anyone thinks it is like this is wild no one else would even consider this maybe there's a 50 chance this happens on…filler · filler

…get to you know what i call kind of country of geniuses in a data center i'm at like 90 on that and it's hard to go much higher than 90 because the world is so unpredictable…hedge · filler

…unpredictable yeah maybe the irreducible uncertainty would be if we were at 95 where you get to things like i don't know maybe you know multiple companies have you know kind of internal turmoil and nothing happens…proper use

…multiple companies have you know kind of internal turmoil and nothing happens and then taiwan gets invaded and like all the fabs get blown up by missiles and you know and then now you would drink to…filler · filler

…one little bit of fundamental uncertainty even on long timescales is this thing about tasks that aren't verifiable like planning a mission to mars like you know doing some fundamental scientific discovery like crispr like you know…proper use

…even on long timescales is this thing about tasks that aren't verifiable like planning a mission to mars like you know doing some fundamental scientific discovery like crispr like you know writing a novel hard to verify…filler · filler

…tasks that aren't verifiable like planning a mission to mars like you know doing some fundamental scientific discovery like crispr like you know writing a novel hard to verify those tasks i am almost certain that we…proper use

…aren't verifiable like planning a mission to mars like you know doing some fundamental scientific discovery like crispr like you know writing a novel hard to verify those tasks i am almost certain that we have a…filler · filler

…to verify those tasks i am almost certain that we have a reliable path to get there but like if there was a little bit uncertainty it's there so on the 10 years i'm like you know…filler · filler

…there but like if there was a little bit uncertainty it's there so on the 10 years i'm like you know 90 which is about as certain as you can be like i think it's crazy to…filler · filler

…on the 10 years i'm like you know 90 which is about as certain as you can be like i think it's crazy to say that this won't happen by 2035 like in some sane world it…quotative · filler

…certain as you can be like i think it's crazy to say that this won't happen by 2035 like in some sane world it would be outside the mainstream no no no this is why i'm almost…filler · filler

…is the world in which we do we do all the things that are verifiable and then they like you know many of them generalize but we kind of don't get fully there we don't fully you…filler · filler

…the job of swe that's part of the job of the company but i do think swe involves like design documents and other things like that which by the way the models are not bad they're already…proper use