Dario Amodei — “We are near the end of the exponential”

…now yeah i would say actually the underlying technology like the exponential of the technology has has gone broadly speaking i would say about about as i expected…filler · filler

…has gone broadly speaking i would say about about as i expected it to go i mean there's like plus or minus you know a couple there's plus or minus a year or two here there's plus…filler · filler

…the exponential it it is roughly what i expected in terms of the march of the models from like you know smart high school student to smart college student to like you know beginning to do phd…filler · filler

…the march of the models from like you know smart high school student to smart college student to like you know beginning to do phd and professional stuff and in the case of code reaching beyond that…filler · filler

…have people talking about these these you know just the same tired old hot button political issues and like you know are around us for like near the end of the exponential i i want to understand…filler · filler

…know just the same tired old hot button political issues and like you know are around us for like near the end of the exponential i i want to understand actually the same hypothesis that i had…hedge · filler

…cleverness all the techniques all all the kind of we need a new method to to do something like that doesn't matter very much there are only a few things that matter and i think i listed…proper use

…there are only a few things that matter and i think i listed seven of them one is like how much raw compute you have the other is the quantity of data that you have then the…filler · filler

…function right an another objective function is you know the the kind of rl objective function that says like you have a goal you're gonna go out and reach the goal within that of course there's objective…filler · filler

…have a goal you're gonna go out and reach the goal within that of course there's objective rewards like you know like you see in math and coding and there's more subjective rewards like you see in…proper use

…you're gonna go out and reach the goal within that of course there's objective rewards like you know like you see in math and coding and there's more subjective rewards like you see in rl from human…filler · filler

…there's objective rewards like you know like you see in math and coding and there's more subjective rewards like you see in rl from human feedback or kind of higher order higher order versions of that and…proper use

…higher order higher order versions of that and then the sixth and seventh were things around kind of like normalization or conditioning like you know just getting the numerical stability so that kind of the big blob…filler · filler

…versions of that and then the sixth and seventh were things around kind of like normalization or conditioning like you know just getting the numerical stability so that kind of the big blob of compute flows in…filler · filler

…were one example of what of kind of what we see there and indeed those have continued going like you know uh you know i think i think now it's been it's been widely reported like you…filler · filler

…going like you know uh you know i think i think now it's been it's been widely reported like you know we feel good about pretraining like pretraining is continuing to give us gains what has changed…filler · filler

…think i think now it's been it's been widely reported like you know we feel good about pretraining like pretraining is continuing to give us gains what has changed is that now we're also seeing the same…filler · filler

…also seeing the same thing for rl right so we're seeing a pretraining phase and then we're seeing like an rl phase on top of that um with rl it's it's actually just the same like you…filler · filler

…seeing like an rl phase on top of that um with rl it's it's actually just the same like you know even even other companies have have published um uh um like um you know in some…filler · filler

…it's actually just the same like you know even even other companies have have published um uh um like um you know in some of their in some of their releases have published things that say look…filler · filler

…trained on these data sets that didn't represent a wide you know distribution of text right you had like you know these very standard you know kind of language modeling benchmarks and gbt1 itself was trained on…filler · filler

…a bunch of i think it was fan fiction actually um but you know it was it was like literary you know it's like literary text which is a very small fraction of the text that you…filler · filler

…it was fan fiction actually um but you know it was it was like literary you know it's like literary text which is a very small fraction of the text that you get and what we found…filler · filler

…text that you get and what we found with that you know and in those days it was like a billion words or something so small data sets and represented a pretty narrow distribution right like a…hedge · filler

…was like a billion words or something so small data sets and represented a pretty narrow distribution right like a narrow distribution of kind of what what you can see what you can see in the world…filler · filler

…wouldn't generalize that well to kind of the other tat you know we had all these measures of like you know how well does the how well does a model do at predicting all of these other…filler · filler

…you know the internet when you when you kind of did a general internet scrape right from something like you know common crawl or scraping links on reddit which is what we did for gpt2 it's only…proper use

…i think we're seeing the same thing on rl that we're starting with first very simple rl tasks like training on math competitions then we're kind of moving to you know kind of broader broader training that…proper use

…math competitions then we're kind of moving to you know kind of broader broader training that involves things like code as a task and now we're moving to do kind of many many other tasks and then…proper use

…way which is that on pretraining when we train the model on pretraining you know we we use like trillions of tokens right and and humans don't see trillions of words so there is an actual sample…hedge · filler

…trained if we give them a long context length the only thing blocking a long context length is like inference but if we give them like a context length of a million they're very good at learning…filler · filler

…context length the only thing blocking a long context length is like inference but if we give them like a context length of a million they're very good at learning and adapting within that context length and…filler · filler

…the full answer to this but but i think there's something going on that pretraining it's it's not like the process of humans learning it's somewhere between the process of humans learning and the process of human…proper use

…of humans learning it's somewhere between the process of humans learning and the process of human evolution it's like it's somewhere between like we get many of our priors from evolution our brain isn't just a blank…filler · filler

…somewhere between the process of humans learning and the process of human evolution it's like it's somewhere between like we get many of our priors from evolution our brain isn't just a blank slate right whole books…filler · filler

…have been written about i think the language models they're much more blank slates they literally start as like random weights whereas the human brain starts with all these regions it's connected to all these inputs and…filler · filler

…do as something between longterm human learning and shortterm human learning so you know there's this hierarchy of like there's evolution there's longterm learning there's shortterm learning and there's just human reaction and the lom phases exist…filler · filler

…was that was the transition from gpt one to gpt two that i saw up close which is like you know the model reaches a point you know i i i i like had these moments where…filler · filler

…up close which is like you know the model reaches a point you know i i i i like had these moments where i was like oh yeah you just give the model like you just give…filler · filler

…the model reaches a point you know i i i i like had these moments where i was like oh yeah you just give the model like you just give the model a list of numbers that's…quotative · filler

…i i i like had these moments where i was like oh yeah you just give the model like you just give the model a list of numbers that's like you know um you know this is…filler · filler

…oh yeah you just give the model like you just give the model a list of numbers that's like you know um you know this is the cost of the house this is the square feet of…filler · filler

…this is the square feet of the house and the model completes the pattern and does linear regression like not great but it does it but it's never seen that exact thing before and and so to…filler · filler

…skill but because we want to generalize two there's kind of two cases to be made here or like two claims you could make one of which is like stronger and the other of which is weaker…filler · filler

…of two cases to be made here or like two claims you could make one of which is like stronger and the other of which is weaker so i think starting starting with the weaker claim you…filler · filler

…think starting starting with the weaker claim you know when when i first saw the scaling back in like you know 2019 um you know i wasn't sure you know this was the the whole this was…hedge · filler

…was you know and and my claim was this is much more likely than anyone thinks it is like this is wild no one else would even consider this maybe there's a 50 chance this happens um…filler · filler

…you know you know what i call kind of country of geniuses in a data center i'm at like 90 on that um and it's hard to go much higher than 90 because the world is so…hedge · filler

…unpredictable um maybe the irreducible uncertainty would be if we were at 95 where you get to things like i don't know may maybe multiple you know multiple companies have you know kind of internal turmoil and…proper use

…multiple companies have you know kind of internal turmoil and nothing happens and then taiwan gets invaded and like all the all the fabs get blown up by missiles and and you know and then now you've…filler · filler

…now you've a changed a scenario yeah but you know just you could construct a scenario where there's like a 5 chance that it it or you know you you can construct a 5 world where like…hedge · filler

…like a 5 chance that it it or you know you you can construct a 5 world where like things things get delayed for for 10 years that's maybe 5 there's another 5 which is that i'm…filler · filler

…little bit of of fundamental uncertainty even on long timescales is this thing about tasks that aren't verifiable like planning a mission to mars like uh you know doing some fundamental scientific discovery like like crispr like…proper use

…even on long timescales is this thing about tasks that aren't verifiable like planning a mission to mars like uh you know doing some fundamental scientific discovery like like crispr like you know writing uh writing a…proper use

…that aren't verifiable like planning a mission to mars like uh you know doing some fundamental scientific discovery like like crispr like you know writing uh writing a novel hard to hard to verify those tasks i…proper use

…aren't verifiable like planning a mission to mars like uh you know doing some fundamental scientific discovery like like crispr like you know writing uh writing a novel hard to hard to verify those tasks i am…proper use

…like planning a mission to mars like uh you know doing some fundamental scientific discovery like like crispr like you know writing uh writing a novel hard to hard to verify those tasks i am almost certain…proper use

…to verify those tasks i am almost certain that we have a reliable path to get there but like if there was a little bit of uncertainty it's there so so so so on the 10 years…filler · filler

…there was a little bit of uncertainty it's there so so so so on the 10 years i'm like you know 90 which is about as certain as you can be like i think it's i think…quotative · filler

…on the 10 years i'm like you know 90 which is about as certain as you can be like i think it's i think it's crazy to say that this won't happen by by by 2035 like…filler · filler