Anthropic Transcript: Technical Leads Predict AGI in Next Couple Years and Exponential Returns to Intelligence
August 2026, American Optimist Napa Discussion with Nick Marwell and Schult Douglas
Episode Intro
Host: Models which are as or more capable than all humans are very likely to occur in the next couple of years. Each marginal unit of intelligence is worth exponentially more than the unit that came before. You can keep increasing, potentially doubling the productive intellectual and physical capacity of humanity in such a way that allows you to truly get to a post-scarcity world.
Host: You guys are at the edge of the AI frontier. What are the most exciting things you're working on?
Nick Marwell: AI is going to be the most important technology in the life sciences, certainly of our lifetime and maybe ever. But it also comes with massive risks.
Schult Douglas: We shouldn't be releasing these technologies into the world before we can keep the bad actors from doing a lot of effort.
Host: Tech leaders I know who are more skeptical are saying, "Oh, well, Dario is trying to scare everyone to create more rules to slow things down."
Schult Douglas: So far we've slowed ourselves down much more than anyone else.
Host: Is there any scenario you'd slow America and let China go ahead?
Schult Douglas: CEO and co-founder... now I'm here.
Host: Nick Marwell and Schult Douglas are key technical leaders at Anthropic, the fastest-growing AI company in the world. They're both very optimistic about the future despite some really serious threats we're facing as a civilization. What are these threats? How do we think about offensive versus defensive dynamics in bio, in cyber, and other key areas? Some people are pretty angry at Anthropic for regulatory capture. What is their view on this? What is Anthropic trying to do? What do they think of the battles with distillation in China? If everyone's able to copy their models, how is Anthropic still going to be worth trillions of dollars? And why, if you're a young person in the workforce, if you're trying to build your career but you're not in the center of AI, what should you be doing for the next 10 years? An optimistic insider's view on one of the fastest-moving companies in the world at a fascinating time. Excited for you to hear from them.
Self-Taught ML and Unlikely Paths to Anthropic
Host: Welcome to American Optimist. Really excited to have today two amazing talents from Anthropic. Nick, you're the key member of the technical staff, help lead RL science, which is reinforcement learning science, and Schult Douglas, also a tech lead in reinforcement learning. Is that right?
Schult Douglas: Yep.
Host: And how long have you guys been in Anthropic?
Nick Marwell: Just over 3 years for me.
Host: The company's only how old, though?
Schult Douglas: It's 5 years or so.
Host: Yeah, it's amazing. It's like one of the most important, biggest companies, fastest-growing companies now in the world, and it's only been around for 5 years. So this is a heady time here in California. We're filming here in Napa today. Thanks for being here, guys.
Nick Marwell: Well, thank you for inviting us. It's going to be fun.
Schult Douglas: Thanks for having us.
Host: Let's start with your backgrounds. So where'd you grow up, Schult? Like, where are you from?
Schult Douglas: I grew up in Sydney. I got very convinced that scaling and in general we were on track to get AGI in the 2020s in about 2020, reading a mixture of blog posts and papers. And I worked very hard on nights and weekends to prove that I could do the work of the quality bar that was demanded by a DeepMind and Anthropic and OpenAI and these kind of companies.
Host: Had you studied computer science?
Schult Douglas: I studied robotics in undergrad. So I graduated in 2019, and then I was doing my own research basically. Ended up working for DeepMind for 2 years and at Anthropic for the last 18 months.
Host: Awesome. And Nick, where are you from?
Nick Marwell: Yeah, I'm actually from San Francisco originally, which is rare nowadays, I suppose, in folks in AI. I grew up in a household with a father in particular who had sort of spent the first half of his career doing sort of very traditional business things, and then had found this interesting path in sort of the latter half of his life and sort of my more formative years where he had gone and spent his time trying to figure out how to give back. And in particular, figuring out how to leverage technology to give back.
Nick Marwell: And I think a lot of sort of as I was thinking about what I wanted to do, I was looking for things that sort of let me get this experience of sort of both at the same time building the skills to sort of become an effective capitalist or whatever you want to call it, and also to use this to be able to sort of make social good. I think Anthropic was sort of this very perfect coincidence where I was getting very concerned about AI safety. I was actually a resident working on a very different company at Thrive Capital at the time, and it was just one of these things where it sort of came out of nowhere, and I was like, I think this is the thing for me to go do.
What Are the Most Exciting Things You're Working On?
Host: Amazing. Well, since you guys have joined, Anthropic, I think, has become the biggest of the AI companies. You guys don't release numbers officially, but lots of stuff leaks. The stuff that leaks, just for the audience, it says it's here in August of 2026...
Nick Marwell: I keep my face very still.
Schult Douglas: Very still.
Nick Marwell: Very still, no reactions.
Host: It's supposedly doing over $80 billion of run rate, growing really fast, ahead of OpenAI, ahead of everyone. You guys are at the edge of the AI frontier. What are the most exciting things you're working on internally right now that you can tell us about? What's it like to be at that cutting edge?
Schult Douglas: Well, I think what I can talk about a lot is how the experience of programming has changed, because I think this is the most visceral way that we felt progress. 18 months ago, I was typing all of my lines of code by hand, basically. Then, a few months into Anthropic, I was guiding a model. I was intervening every few minutes. I'd say what I wanted it to do, write the next function or this kind of thing, but I'd have to check in every few minutes because it would make mistakes, it would go off course, and this kind of thing.
Schult Douglas: And where we're at now is I can ask models to do a day or 2 days of work independently and drive progress basically like a junior team member. And that's just over the last 18 months. We've gone from what is really very much a tool where you have to stay in the loop to something that feels very much like a junior team member.
Host: So, as we're talking right now, they're the junior team members, the AI is working hard for you?
Schult Douglas: Yeah, they are. There's a bunch of them working hard.
Host: What kind of stuff are they working on, if you could say? Is there any high level you could tell us?
Schult Douglas: Just a bunch of RL science stuff with some of these team members.
Nick Marwell: Awesome. I'll give a slightly different answer here. I think a lot of sort of model development up to this point has really been about getting models to be able to do tasks that humans are actually already quite good and capable at. And really what we're trying to do is capabilities that bring lots of additional leverage to things humans were sort of already cognitively capable of achieving, right? Building a piece of software that like a team of human developers could also have built in the same period of time.
Nick Marwell: I think what you're starting to see now, and I think some of the most recent sort of results in math, for example, are a good example of this, is people trying to figure out how to use models to push past sort of the frontier of human intelligence and human achievement.
Host: This is like some of the recent math things?
Nick Marwell: Yeah, there have been a lot of new math proofs that have come out that are sort of things humans have been interested in for a long time and sort of never been able to solve.
Host: Some of those, it just seemed like a really smart mathematician realized that the computer is good enough to brute force something that a person couldn't have brute forced. You see what I'm saying? Some of them still seem like they're good at specific things that humans are using, or is it really like a whole new theory that they're getting at? How do you think about that?
Nick Marwell: I think it is not sort of, at least yet, some idea where they're coming up with ideas that humans would have been incapable of coming up with. But I actually think very little progress or intellectual achievement ever looks like this in many ways. I think, there are of course exceptions, but I think most of the things that society has achieved are really just about sort of connecting the dots and building upon the prior intellectual achievements of things that came before.
Nick Marwell: And I think that what you're describing, something like putting 2 ideas that different people had together to find something new, that is the foundation of like what humans do. And the important thing that's different now is it's really putting together sort of new ideas, or forming new ideas from combining other human ideas that weren't happening before. Whereas I don't think writing another web app was actually some new intellectual achievement for humanity.
Have We Reached AGI — And What Comes Next?
Host: We've never seen companies scale this fast in Silicon Valley. I think again, not to 10x, 10x, 10x, growing really fast obviously. We had another company here in Napa last night that I'm on the board of staying over, and it's like a new company every 60 to 90 days. There's so much happening, right? And I think this is like the experience of a lot of us. You're right in the core of the fastest-growing company. What's it like versus your life before? Where do we go from here? Is it going to keep accelerating? How do we think about this?
Schult Douglas: I think one thing that's been really nice about Anthropic has been that our ambitions have been able to scale now with the acceleration of the company. It really felt like 18 months ago we had to be so insanely focused on code, and really proving out that we could be the best in the world at something. And now the mission of the company is so much grander than make models that are good at code. And as the company has scaled, as the revenue has scaled, it's let us really start putting together the pieces to start pushing on more of that.
Host: Let's talk more about that mission. I was with another friend who's senior in AI earlier this week, and I understand he was saying how first you got to be the very best at code and math, but then there's going to be like new things it's going to be the very best at. There's going to be all these other human skills you can get to be A++ at. Is that the idea, bringing in a bunch of new skills? What's the mission from here?
Schult Douglas: Well, the mission from here is always we think that AGI really is achievable in the next couple of years.
Host: Explain to our audience what you mean by that.
Schult Douglas: So we think that models which are as or more capable than all humans are very likely to occur in the next couple of years. So it's something that can do all of the things that a human could do on a computer, or once we get sufficiently advanced robotics, all of the things that a human could do in the physical world.
Host: So it's going to be like much better at the podcast. I could just let the robot jump in?
Schult Douglas: Well, the podcast, you know...
Host: Let's go to the vineyard. It's more fun.
Schult Douglas: Exactly. And we think this both has huge upsides, right? This would allow you to multiply the intellectual labor and physical labor done in the world today. It would allow you to get centuries of progress, potentially much, much sooner.
How to Measure AI Progress
Schult Douglas: But it also comes with massive risks, and so really the goal of the company is shepherding the world through those risks so that we can get the upsides on the other side.
Host: Let's step back on the AI wave in general, where we are on this AI wave. Obviously, you were saying in 2020, I think you realized that AGI was going to become possible. I think that's right around GPT-3 coming out from OpenAI. I was looking back at some emails because we're doing like a 10-year ADC book right now, and I was making jokes even in 2020 about how we're going to replace all the associates. And so it was kind of in the air that, okay, this is starting to become likely.
Host: Talking to Dario earlier this year, by some metrics some capabilities were doubling every few months. How do we think about quantifying progress? You talked about the fact you could just use the AI as opposed to a junior employee. Is there any other way to think about how much better it gets in the next couple of years, metrics-wise?
Schult Douglas: Yeah, I think there's a lot of evals, and these are basically exams where people test the models. And on these, progress is extremely rapid. A great one is, I don't want to lean too much on math as an example, but math is maybe the best in some ways, where we went from 0% on a set of problems assembled by professors across different fields to well over 40%, 50%, 60% just over the course of last year. And that's an eval called FrontierMath.
Host: You need something that's falsifiable, right? You need something to iterate on. So could you have it talk to a girl on Instagram and get a certain percent chance that she wants to go on a date with you?
Schult Douglas: Right. I mean, it's innocent enough to ask, like train on that. Right?
Host: I'm married, by the way, not for me.
Schult Douglas: We haven't trained on that. But that's an example.
Host: Other sorts of things like that would be probably very useful for, especially some of our friends in San Francisco.
Schult Douglas: There's 2 ways the models improve, right? One of them is via training on vast corpuses of text from the internet. And then the second form, reinforcement learning, what we both work on, is doing these worked problems and checking whether or not you got the right answer. And so this is what you're talking about, which is very easy for math and for computer science. It's pretty hard for romance or poems or other things, right?
Host: Yeah.
Schult Douglas: And that's why you've seen math and computer science advance so much faster than everything else. But we actually think it's not that hard to put other problems in this frame.
Nick Marwell: Yeah. I actually think there's, in some ways, a technical explanation for why math and computer science came first. I also think to some degree there's like a social explanation for why this happened, which is that you took a group of people who primarily came from backgrounds in math and computer science, and you asked them to make intelligence, and of course they decided that the first things they wanted to do were to make intelligence in the domains they understood, because they both were most interested in them, and because it's much easier.
Nick Marwell: It's amazing how much time, when you're trying to train good models, you actually spend reading what the model is doing, and trying to understand as a result how capable it's becoming, what kinds of things you might need to focus on, sort of areas it needs to get better. And it's much easier to do this for domains where you're an expert, you understand what's going on.
Host: Do you anthropomorphize it when you think about it?
Nick Marwell: I don't think I personally do. I think there are some people who probably do, and I'm sure some of them work at Anthropic.
Host: Our brains are kind of programmed, like there's whole parts of our brains that are set up to think of the person we're looking at and mirror what we think their emotions are in order to talk to them. So we have all these things built in from evolution to treat an intelligence as another intelligence. It seems like it'd be hard to avoid doing that when you're working on this day-to-day.
Schult Douglas: This is maybe one of the bigger debates in terms of how people try and frame these things as tools or as entities. And I really think that entity is a useful frame, because with a tool, you are using it the entire time. And with these models, what you're really doing is you're taking this thing that is very intelligent, and you're releasing it actually out into the world in some respects, where it's going to take actions on your behalf. And so it's often more useful to think about these as an entity for that reason.
Is Biology the Next Great Frontier?
Host: Are there categories or areas of things we should be having this intelligence do that we're not yet? There are things you guys are excited about or working on where we're going to use this a lot more in these other areas. What are those areas? I think the big one's biology, right?
Nick Marwell: Yeah. I think it depends if you're asking a question that's about what are models already great at that we think are just underutilized skills, versus where are we most excited about where models are going. I think on the latter question, we can get more into why biology is the thing that I am most excited about over the next, say, 6 to 24 months in terms of where I think models will really begin to make an impact in the way that they have in software engineering, but not just in the sense that they'll be widely adopted, but that the sort of societal positives that come out of this will be really diffuse and really, really powerful.
Host: Curing diseases and making all of us healthier and longevity. It seems like there's all sorts of weird things about epigenetics that are tied to longevity that we're kind of just discovering through the Nobel Prizes starting 20 years ago, and all of our friends have companies now in the Bay Area. There's billions of dollars trying to figure out how to trigger these things and how to make parts of your body younger and stuff. So you think AI is going to be critical to advance that?
Nick Marwell: I think AI is going to be the most important technology in the life sciences, certainly of our lifetime and maybe ever. In fact, I am confident to the point that I am almost more concerned if we have the physical and lab infrastructure required to take advantage of this than I am about whether AI will be able to be a useful intellectual tool.
Host: 3 years ago we should have been probably building a lot more compute, for example. We talk about this too much. Should someone be creating a lot more lab space for you to use or something right now?
Nick Marwell: I think so. And I think that there are a couple of axes on which this is important. I think one axis on which this is important is literally how much lab space do you have. But I think that there are also a number of sort of metrics around quality of the labs that we have in the US that are clearly lagging behind places like China, really. And I think it is one of the great sort of societal infrastructure projects that people should be thinking about how to go solve, is how to get the supply chain and lab complex in America back up to being sort of a gold standard best in the world.
Schult Douglas: This will be the rate limiter on biology progress.
Nick Marwell: Yeah.
Host: All right. Well, let's go solve that after the podcast. That sounds like a good one. Get some money from the Department of Defense for supply chains and build some labs.
Career Advice & the Decade of the Generalists
Host: Stepping back a little bit on where we are in the wave, what would you do as a new college grad or someone who's in the workforce? You're not in Anthropic, you're not in the middle of Silicon Valley. You might be a smart person who's ambitious. How should you think about your career given what's happening with you guys in the next few years?
Nick Marwell: I think that there are a couple of different scenarios we could end up in. Anthropic sort of has a responsibility to speak about the risks that it's bringing into the world, and one of them is we're actually quite concerned about unemployment. But I do think it's possible that this technology will take long enough to diffuse into the world that there are a bunch of scenarios in which you have a 10- or 20-year period where there are actually a ton of opportunities to be part of the diffusion of this technology, and sort of have that be what defines your career.
Nick Marwell: And a thing I've said to some folks—I have a brother who's in college right now and actually thinking about this exact question—is I think that the next decade, if we are in such a scenario, is really going to be one that sort of belongs to the generalists in a lot of ways. I think a lot of the way you sort of had a great career over the prior, I don't know, 20 or 50 years is you had a sort of set of skills that were very valuable and very marketable. And it was very easy to have a set of skills and depend on other people.
Nick Marwell: Now everyone has the power of a 1,000-person company at their back. And what it's really going to mean is that the people who are capable of best understanding what are the problems to go out and be solved, what are the things that would make my community better, whether it's from a commercial standpoint, from a social good standpoint, etc., they'll have all of the tools they need to go affect that change. And so problem selection and things like that will be the defining skill.
Host: You said, "What are the problems to be solved? How do I make my community better?" One of the fundamental backdrops to my view of the world, being Jewish, is you're supposed to repair the world 6 days a week, and then you pretend you're done on the Sabbath, the day of rest, which I think a lot of our major religions have something similar. And my optimism is that there's always going to be things for each of us to repair and help and fix with the people around us, with the communities around us. Unless you believe we're going to be in utopia, there's going to be things to do for people, right? I think it's an optimistic framework.
Nick Marwell: Absolutely, yeah.
Schult Douglas: Yeah, there's a lot of work between us and utopia.
Host: Exactly. The whole fundamental implication of the religion is it's not ever utopia here until the Messiah comes or something, and so therefore you have to just keep fixing things, right? And that's a positive thing.
What Could Go Wrong? What Are the Biggest AI Risks?
Host: I know you guys are optimists and there are lots of positive things. I want to go into some concerns first of all, because this is where a lot of people are very curious. Obviously on the coast, I think we're seeing all these really positive things around us in our communities. We all have friends building companies faster than ever before. It's this really positive energy, at least for me when I come out to San Francisco, when I go to New York, even things I'm doing in Austin, Texas.
Host: But I think a lot of the rest of the country feels like there are threats to their business, threats to their livelihood. It's scary. They don't know that you guys aren't just going to be some kind of wacky people who are going to conquer with this or whatever. What are people missing? Is there a plausible bad path that you worry about? Is the bad path they worry about wrong? How do you think about this problem?
Schult Douglas: I think the concerns are extremely reasonable. There's a couple of big categories that we're worried about, but all of them are ones that we think if we take the right actions over the next couple of years, we can end up with a radically better world in the 2030s.
Schult Douglas: Those categories are: a big one we've mentioned already is unemployment, right? And we should have a long discussion around that and why we think that's a risk and what we think the right things to do about that are. There's also the more immediate near-term concerns of bio and cyber risk that are literally happening like this year right now.
Host: There's all sorts of annoying things there where people would make fun of Anthropic because you couldn't ask it about the mitochondria of a cell was a classic case. And it's like, well, actually probably you should be able to do that and just maybe don't help make a bioweapon or something.
Schult Douglas: You should definitely be allowed to do that. But we're sort of putting in place the infrastructure so that we can feel confident that no one can make a bioweapon out of them.
Nick Marwell: Yeah. What's incredibly hard with things like bio and cyber is that these are inherently what's known as dual-use capabilities. I think cyber is the easiest one to understand. If I prompt Claude and I say, "Here's a codebase. Please find all of the vulnerabilities," it is just as likely that I am the owner of that codebase trying to harden my defenses as it is that I am the attacker trying to find my way in.
Host: Yep.
Nick Marwell: And this is what fundamentally makes things like cyber, or bio which has many of the same sort of dual-use characteristics, so hard to police.
Host: You're trying to target something in a cell, and it's a really good chance, probably a 99.9% chance, you're trying to kill it to get rid of cancer or something, but there's some tiny chance you're trying to target something terrible.
Nick Marwell: Right. And so I think we're working on a number of things that are sort of meant to enable us to make sure that good actors get to use these technologies and bad actors are kept out. But I think that our general feeling is we shouldn't be releasing these technologies into the world before we can keep the bad actors from doing harmful things.
Host: There are definitely people, and it's not even Anthropic, I think they're using open models from China with terrorist groups, for example in Africa, to figure out how to more easily make certain bombs that they didn't know how to make before, and it's just smart enough to help them because they weren't very smart at that. There are scary uses. But at the same time, my bias is overall with more intelligence, more people can understand the scary uses and use that to stop them as well, right? There's a tradeoff.
Schult Douglas: And so a big dynamic here that we should talk about is whether an area is offense- or defense-dominant.
Host: Yes.
Schult Douglas: Right now both cyber and bio are offense-dominant, but I think that over the next 2 years cyber becomes defense-dominant.
Host: Especially if everyone has access to intelligence to hack themselves and build defenses.
Schult Douglas: Exactly. Because you can preemptively try and hack yourselves, build up the defenses, patch all the vulnerabilities, and then eventually we'll end up in a much more cyber-secure world.
Host: Where everyone already has access to intelligence, they figure out how they're going to break in. That's what we're doing ahead of time. I think one of my friends helped break into something at the Pentagon and the banks when they asked them and they showed them, but they're using new AI because AI is so good at it, and so now everyone has to do it quickly.
Offense vs Defense: How Technology Evolves
Host: Let's step back for a second. The offense-defense thing is super interesting, right? This has been a concept for thousands of years in human history. It changes a lot. I'm a big fan of the very free city-states that used to compete in Europe, and this was a really good thing. If one city-state was doing something badly, you can go to the other one.
Host: And then unfortunately, these jerks came along with cannons. They could just knock down the walls really easily, and then they built these big empires and you couldn't just leave your city-state anymore because now you're under them regardless. And so there's different times when things have moved between offense and defense.
Host: And I agree, I think cyber right now is super offense-dominated to the point where people should be scared. Anyone listening, if you're not on the cutting edge using AI to challenge your systems, they probably will be broken into, and it might be a very bad person. So right now it's scary. I agree eventually cyber could become more defense-dominated used correctly.
Host: There's a question with AI. One of the fears for me is, are there certain hidden things in the world that are just super offense-dominated? For example, and I don't believe this is the case, but if you can make a self-replicating nanobot with enough intelligence that just eats the whole world, and one genius makes it and then eats the whole world and then we're all dead. Again, I don't think that's going to happen, but we have to make sure we're using the intelligence to find out whatever is super offense-dominated sooner.
Schult Douglas: Right. And actually bio is super offense-dominant until we put a huge amount of work into making the world defense-dominant. There are ways you can make the world defense-dominant against bio, but it's going to require hundreds of billions of dollars of infrastructure. It's not unreasonable to do until you get dramatically advanced capabilities in robotics in the 2030s to help you build this out.
Host: This is a very scary period in a sense where it's currently offense-dominant. We want to make it defense-dominant. I guess the solution is to make sure these things are not too secretive, and to maybe show them off. My view is you want lots of people using intelligence to challenge everything, not just having a single group doing it in secret.
Schult Douglas: We very much agree on that note.
Answering the Regulatory Capture Critique
Host: Anthropic is generally very admired. Also, like any company, I compare it to Microsoft in the 90s: when you become the champion, people get jealous and attack you. Sometimes those attacks are because they're jealous and you have lots of aura and you're dominating, and sometimes certain things are legitimate as well. A lot of my very smart friends have the impression that there's some kind of regulatory capture strategy Anthropic is trying to do that's not a good thing. How do you react to that?
Nick Marwell: I think there's probably 2 things that are worth talking about here. I think one is the rise of the regulatory capture narrative tied to the open-weights discussion that's been going on. And then there's also probably just, if you were trying to do regulatory capture in a very self-serving way, would you have done it the way Anthropic has?
Nick Marwell: I think on the first one, it's not unreasonable, if you're not in the details of what's going on, to interpret something to the effect of open weights is an important public-good technology that we want to keep around. It is in some ways a natural opponent to parts of Anthropic's business, and Anthropic wants to use any regulation it can to push around the people who are developing open weights, because historically open weights or open source have been communities that are smaller, less well-funded, and have a hard time navigating the regulatory system.
Nick Marwell: I think that's not what's going on with open weights here. If you look at who the people who are pushing and developing open-weights models and serving those models are, it is largely the most gigantic corporations in the world. In the US, it's people like Nvidia and Amazon and Microsoft who are pushing the frontier of open weights forward. And then even with, for example, the Chinese labs, these are extremely well-funded, more and more often now public labs that are pushing this. And so these are companies who, even though these are open-weights models, are very capable of navigating a regulatory environment.
Nick Marwell: And so I'm not particularly concerned about having some regulation of models, both open and closed, being something that slows down the developers of open weights in a way that's disproportionate to the developers of closed weights.
Nick Marwell: I think the other part of this is just more directly: if you were doing regulatory capture in a way that was meant purely to advantage you, it's very unclear to me that Anthropic would have done the set of things that it's chosen to do. One very good example of this—and there are probably others, maybe Schult has some other things in mind—is what happened with Fable. If you think about Fable, or Mythos before that, from a pure economic motivation standpoint, Anthropic had this model that was far and away the best model in the world. Nobody had a model even close to it at that period of time.
Nick Marwell: And Anthropic basically decided to go work with the government to roll it out in a way the government was comfortable with, when it could have chosen to take a multi-month-long model lead and probably put to bed any questions of could other labs be competitive with the company. And this was just a decidedly bad economic decision for the company, but one that was aligned with our values about how we think models should be deployed and regulated.
Nick Marwell: And so it's not a strategy of regulatory capture for economic interests. It's a strategy of: we think the government has a really important role to play in how critical and possibly dangerous technologies are developed and deployed. And we think it's our responsibility, as someone at the frontier, to make sure that they're informed of that and that they participate in that process, because otherwise who's going to represent everyone?
Host: It's interesting too because obviously there are quite a few smart people in this government, and there's a lot of people who don't know what they're doing at all as well. You end up with a silly circumstance where, for example, maybe you can't say it but I can say it, which is that Anthropic, from my understanding, would have had more cyber capabilities that people could have used for defense, but then they were nerfed in order for the government to let it go out, which I thought was a mistake.
Schult Douglas: I think that they are leaning into the classic American public-private partnership that has been such an important part of our history. We are not built in a way that the government has to be the most knowledgeable, competent body about every subject in the world. What we want to be able to do is build a great public-private partnership where the government knows how to engage with Anthropic, with OpenAI, with Meta, whoever else it may be, to understand and then decide how to regulate.
Host: And to the tech leaders I know who are more skeptical, who are saying, "Oh, well, Dario is trying to scare everyone to create more rules to slow things down because he's going to be one of the dominant people who owns that process," what's the answer to them?
Schult Douglas: So far we've slowed ourselves down much more than anyone else. In fact, if you take a look at the discussion around Demis' proposal for the closed-model providers—OpenAI, DeepMind, Anthropic—to coordinate in a way that they can actually have safety thresholds that we have to meet before we can release our models, this is us discussing slowing ourselves down because of our principles. Now, I think also by the way we should clarify what our overall stance is on open source.
Anthropic's Stance on Open Source
Host: What actually is it?
Schult Douglas: Personally, we're very supportive of open source. We genuinely believe that it's a very good thing for this to exist in the world. I learned how to do ML on open-source models, right? We also believe that there are a lot of risks, and there will be increasing risks in the future. So what we believe is that basically there should be thresholds for what society at large feels they're comfortable releasing. If a model can do cyber attacks, what threshold of capability are you comfortable with? And we believe that closed and open-source models should have to meet the same bar. Other than that, people should be able to do what they want.
Host: Is it worth even trying to do that if China's just going to be releasing random stuff that everyone could use anyway? And similarly, what's the probability we over-regulate out of fear, and then China doesn't, and there's a problem anyway and they're ahead? How do you think about that?
Schult Douglas: The ideal world is one where we can actually work with China such that they—I mean, they also don't want cyber attacks on their country.
Host: I think they want cyber attacks on us, though, is my impression.
Would You Slow Down and Let China Go Ahead?
Schult Douglas: Maybe they don't want cyber attacks on themselves. That's fair. The ideal world is one where we can work together with them on that. But that's tricky, right? That is going to be one of the most difficult parts of the next couple of years as AI capabilities keep racing ahead. This is going to make a lot of people very uncomfortable for many different reasons—for bio, for cyber, for employment. And as a result, you're seeing already pushback, like stopping the data sets.
Schult Douglas: This is one of those debates where it becomes really tricky to pause unless you can coordinate with the other parties and trust the other parties to hold to that.
Host: China's not going to pause, though. So is there any scenario you just slow America and let China go ahead?
Schult Douglas: I don't think any scenario that we're comfortable with would involve slowing America and letting China go ahead.
Host: Which seems to be the only option under our control right now.
Schult Douglas: Yeah. For any slowing scenario, it would have to be fully coordinated in a way that we fully trust every counterparty. Basically, there's no way we'd slow America.
Anthropic's View on Distillation
Host: You're surprisingly, relative to what other people think, more for open source. It's a good thing, it's affected your lives positively. Some of this open-source stuff seems to be coming from people stealing it from you, which I would be annoyed by if I were you. It's called distillation. A lot of people say, "Well, why don't they just stop it by being really smart with their AI and watching everything?" Why aren't you able to stop distillation? How do you feel about it? What's happening with that?
Schult Douglas: Well, on the one hand, distillation—there's a couple of things to talk about here. One of those is: why are we against distillation? Is distillation good or bad? And the other is: why can't you stop it if you think it's bad?
Schult Douglas: Let's close off the second, which is there's a bit of a cat-and-mouse game, right? People want a lot of access to their models. They want to be able to inspect the outputs, they want to run on their local computer, they want to look at the thinking traces. But every additional bit of access you give makes it easier to distill. And so there is this tension between how open your tool and product can be and how easy it is to distill, in effect.
Nick Marwell: And then on the first bit, which is why should we be unhappy about distillation period, the way I often think about it is this: you want AI to make progress forwards because we think getting smarter and smarter models is going to do all of these great things for the world. And distillation is not a method that's capable of bringing you past the frontier of current capability.
Host: It's just copying you.
Nick Marwell: It's a copy. And the problem with this is that getting to the next frontier requires enormous upfront capital investments. We're going to be in regimes, forget about today, eventually where there are $10 billion, $100 billion, $1 trillion training runs. And you can only support that kind of R&D investment if you're able to then make money selling the thing that you invested in.
Nick Marwell: The problem is if everyone else is able, for a fraction of the cost, to take your thing, copy it, and put out a copy in a week or two, they will actually be an economically advantaged competitor against you because they don't have this big R&D load. And so if you want progress on the frontier, then you want to protect the sort of IP and rights of people training these models.
Nick Marwell: People often have compared this to sort of like drug development. I actually think AI is very interesting because protecting against distillation gets you a lot of the good things about how we protect IP in drug development and avoids some of the bad things. What are the good things? We get more drug development in the US as a result of protecting IP.
Host: Obviously I can't invest in drug development if I can't make money on it.
Nick Marwell: Exactly. Some of the bad things are that costs are extremely high as a result. What's interesting about AI is, unlike in drug development where you get this protection for a decade or more—it's 20 or 30 years, I think—in AI, intelligence is incredibly deflationary because the shelf life of the frontier is measured in months. And so every 6 months, yes, the frontier is going to be expensive, but everything behind the frontier, which felt frontier 6 months ago, is going to be made abundantly cheap by the fact that you push the frontier forward more. This is sort of an artifact of how training goes.
Nick Marwell: So it gives you a lot of the good and actually avoids most of the bad in the regimes that we've been in. You can very concretely see this in the cost of a given level of intelligence. You can look at heaps of eval dashboards; there is one on Artificial Analysis that shows this exact trend line: it gets roughly 10x cheaper every year. It gets cheaper year after year. There is this incredible deflationary extent, and by the way, this is happening without distillation, just because of the natural progression of the technology.
Host: Of course. And it's just intuitively unfair for someone to be able to copy, and it would break the whole system if everyone did it.
Nick Marwell: Yeah.
Host: Maybe a follow-up question: isn't that a crazy business model where your thing gets obsoleted? In some respects, after definitely a year, if not 6 months, the frontier seems like it has to keep moving forward for you to win.
Schult Douglas: The frontier feels like it has to keep moving forward for you to win. And I think this is a very important part of our general worldview, which is that actually AI has gone a very tiny amount into the global economy. The sum total of AI revenues is somewhere north of $100 billion into a tens and tens of trillions of dollars economy. And that's assuming the economy won't grow dramatically as a consequence of this technology, which we expect it will.
Schult Douglas: And so there is this frontier. In many ways, you can think of the level just behind the frontier, the lagging edge, getting commoditized in many ways, but then the leading edge has even more value.
Nick Marwell: There's a huge amount more value at the leading edge. We often talk about this as exponential economic returns to intelligence, where each marginal unit of intelligence that you're capable of is worth exponentially more than the unit that came before it. We've already seen this occur. If you look at something like coding, the first unit of intelligence in coding that was useful at all to people was tab auto-completion. And you moved from this into the next marginal unit of intelligence where you made agentic coding a reality in your terminal with Claude Code. And this was a 100,000x more valuable capability.
Nick Marwell: And so even though tab auto-completion got commoditized behind it, you can't even really have a company based on tab auto-completion at this point. These next incremental units of intelligence are just extremely valuable.
Nick Marwell: This was very unintuitive to me when I was thinking about Anthropic a couple of years ago, because my intuition from everything else I experience as a human being living in this world is that the exponentials don't go for that long. So you think, okay, well, this is really valuable for a couple of years, but at some point there's not an exponential going. In some ways it requires you to be extremely optimistic, right? Because you have to believe that you're going to go from tab completion in coding to agentic coding, and what could be more valuable than agentic coding? Maybe we can cure cancer.
Nick Marwell: And even between those, it's like you have a full-on drop-in software engineer that's better than any software engineer in the world, and that feels extraordinarily cheap. And you have an executive that helps you with all sorts of things. And so by the time people talk about, "Oh, won't this just lead to commoditization?"—by the time you're even in a hope of complete commoditization, the entire economy is radically transformed. Literally the entire economy by that point is being done by AIs, both intellectual and physical.
Host: And probably it is a much, much larger economy. And if Anthropic is driving that forward, that's obviously worth trillions of dollars.
Nick Marwell: Exactly.
Is a Post-Scarcity World Achievable?
Host: All right, so we're all rich. Just kidding. There's obviously a lot of concerns, but you're optimists. So why are we pushing this ahead so fast? Remind us again of that. What is going on here? What are you fighting for?
Schult Douglas: To come back to why we're concerned: because we're going to get something that's a drop-in for a human, right? It's going to be as good as a human at everything they can do on a computer. Once we have robots, it's going to be as good as a human at everything in the physical world. But at the same time, that unlocks massive, radical improvements. It means you can literally take technology that would have required the next couple of centuries to develop with our population, and compress that into the next decade, a couple of decades.
Host: I'm already seeing this with aerospace, where airplane design stuff is going so much faster that there are probably really advanced planes we would have gotten in the 2040s that we're probably going to start seeing in the next few years. It's really cool.
Schult Douglas: And so you can keep increasing and potentially doubling the productive intellectual and physical capacity of humanity in such a way that allows you to truly get to a post-scarcity world. This is a world where literally everything is reduced to the cost of energy, where building homes is reduced to the cost of energy.
Host: The middle class will all have giant homes if they want.
Schult Douglas: You could literally have whatever you want in that world.
Host: You can go to the mountain and do a 500,000-square-foot thing, and it's a middle-class thing.
Schult Douglas: Exactly. That would be a middle-class thing in the same way that things we have today are unimaginable to even the kings of centuries ago, right? Where we've cured every disease, where we've probably cured aging and solved longevity. And so you're in this world where everyone on Earth has a level of abundance available only—not even to the richest people in the world today. And that's the world we're fighting for.
Schult Douglas: But it's a very tricky world, because it means that you need to chart this course through to it, and two, you have to figure out how do we share the proceeds of that world in such a way that you don't end up with massive inequality, right? That the returns don't just flow to people who happen to have capital in the pre-existing world, and instead everyone, if they so want to, can have that kind of abundance. And so there's a big sociopolitical question of how do you share in the benefits.
Host: There are a lot of mistakes you can make by assuming we're already there though, too, right? Because the dignity of work right now is real.
Schult Douglas: We're not there. We're not even close to there. But if we invest in the technology in the right ways, then we've got a shot at getting there. And so we think that basically, let's draw the trend lines out a little bit from today. Over the course of the last 4 or 5 years, we've been 2xing or 3xing the amount of compute capacity devoted to AI every year. Together, the hyperscalers are spending about $1 trillion this year on capex related to AI.
Schult Douglas: A very interesting question will be: will that trend line hold? Will it go to $2 trillion next year and then $4 trillion in 2028? And if that trend line broadly holds, and you can sort of expand that to encompass the broader robotics industry and this kind of stuff, then that means in the early 2030s or something like this, you start to get to the point where you're effectively doubling the GDP of humanity in the early 2030s. Which again, is a little bit of a ridiculous concept and requires a lot of things to go right, but the productivity growth could just start accelerating rapidly.
Host: It feels like it should even be in the next couple of years. Kevin Warsh is the new chairman of the Fed. I don't know if he's AGI-pilled, but he's very bullish on AI productivity right now. A very smart friend of ours. It feels like we're going to start seeing that a lot more, but it feels insane to say it could just accelerate even much more.
Nick Marwell: It could accelerate even more. One of the interesting charts I saw recently was this chart of employment in the Philippines, which went up.
Host: I was surprised by this. I thought BPO was going to get really hit there. BPO stands for business process outsourcing. They're doing low-end call centers, and yet they're still growing.
Nick Marwell: And yet they're still growing, and actually software engineering employment is up as well. And I think a lot of this is because all of a sudden people are dramatically more productive than they were before. A very interesting trend line would just be: how long does this bump continue where people are pairing?
Host: There's so much more to do. You're using this stuff, which is a positive sign.
Nick Marwell: At least in the short term, it seems that way. But I do think it's important that we don't convince ourselves too much that it's just all going to be fine. Right now, as Schult put it, you're in this possibly good moment where all your employees are still very important parts of taking advantage of AI. And as a result, what happened is you got an enabling technology that made every one of them more valuable, and so you wanted to hire more of them.
Nick Marwell: But at some point, our belief is that we're going to go from a world where you need a human paired with an AI model to sort of get all the value out of it, to a world where you don't. And I think that's the world where we start to get very nervous about the consequences.
Host: This is like with chess: for 30 years the best chess player was a human plus the machine, and now the machines are just crushing, and we think that's just a much shorter time period.
Nick Marwell: Yeah. One thing that was a big update for me over the last 3 years, maybe over the last 5 years when I first started thinking about the rate of AI progress: I believed eventually we would sort of get to a place that looked like where we are today or where I think we'll be in a year, but that it would take longer. In large part, I thought it would take longer because I thought that scaling the data required to do this was just going to be an extremely time- and dollar-expensive pursuit.
Nick Marwell: And the thing that happened that, even by my lofty expectations at the time of AI progress, was unexpected was that the ramp of the revenue and the scale of these businesses—of Anthropic and OpenAI—was so fast that all of a sudden it became completely plausible for the people developing this technology to throw money at the problem of blasting through what we thought a lot of these bottlenecks might be much faster. The compute ramp is another great example of this. It would have been inconceivable to most people 5 years ago that we would spend $1 trillion in a year.
Host: It was inconceivable to people a year ago.
Nick Marwell: Yeah, it was a year ago. It does seem like we are in a timeline where a lot of stuff that has a very large interaction term with each other is happening at once, some good and some bad. We talk about the move into space for compute; it's kind of interesting that AI is coming around at roughly the same time that SpaceX is coming into its own. The timing of that was really interesting. You could imagine if SpaceX had taken 10 years longer to get here, that AI timelines might have looked different.
Host: It's all thanks to Elon, too. He was crazy enough to start it way before anyone else.
Nick Marwell: On the other hand, you have some much more concerning things that are happening. I don't think it's great for the world, for all sorts of reasons including race dynamics, including that AI fundamentally is a technology that can benefit authoritarianism more than most other forms of government, that AI is arriving at a time when we're entering certainly the most bipolar world in my lifetime.
Host: Isn't that linked, though? Isn't there something about social media and how it uses machine learning and improving AI that does polarize us? Is it something about the attention economy?
Nick Marwell: When I'm saying bipolar, I'm really referring to the China-US global picture.
Host: You don't mean the left-right polarization in the US.
Nick Marwell: No, that's probably another thing that's worth thinking of. But I really do think I would have updated us at least slightly more positively on the prospects for AI going very well if it was coming into the world, say, in the early 2000s when you have this more unipolar world. It's safer.
Schult Douglas: It's like a tragedy worthy of a sci-fi novel that the most valuable resource in the universe is produced in this little island between the 2 world superpowers.
Nick Marwell: It is quite poetic in some respects.
How Do We Preserve Values, Tradition & Community?
Host: Overall there's a lot of optimism in terms of where this goes for humanity if we get through this period. It seems to be the high-level thing. One other question I think a lot of our listeners are wondering about: it's one thing to have extraordinary wealth, but there are also things we care about in our civilization. There are virtues, there's family, there are traditional values which a lot of people base their lives on. There's the dignity of work, but there's all these other things around that, which is our traditions that make us who we are.
Host: And I think there's some concern that because this is happening in San Francisco, where those traditions maybe are not as valued on average, is there some kind of agency of the people in charge where now they're just going to remake the world and get rid of my traditional sense of, whether it's Christianity or whether it's monogamy or whatever else? Are there institutions we can be building to help preserve these things? How do we think about that from the perspective of being in the center of San Francisco and changing everything?
Schult Douglas: Interestingly, in San Francisco we actually expect a resurgence in the importance of these institutions, because as work decreases in its primacy to people's lives, and as pure capitalism decreases in its primacy to people's lives because we progressively have more and more abundance, we think that the important parts of people's lives are going to be tradition and ritual and providing values to other people, your local community, and those bonds.
Nick Marwell: Counterintuitively, many intellectual pursuits and the pursuit of figuring out how to become someone who knows how to think about the world is actually something that perhaps grows in importance, just not in an economic sense. In a world where you want for nothing, it's sort of the last thing that's left to entertain yourself, to feel part of a community, and all of these things.
Host: You guys are both young and very successful. Let's just say Anthropic keeps working out. Everyone there is going to have resources, and the whole world's got more resources. Have you thought about building institutions to help with these things? Is there something you're passionate about there to create?
Schult Douglas: Yes. Nick, you've got a great example with the early childhood stuff that your dad's working on, but this is a very big topic of discussion at Anthropic right now. Philanthropy is a very important part of Anthropic's culture. There's a great blog post on this called something like "The New Wave of American Philanthropy," where people talk about how the philanthropic outpourings of OpenAI's foundation and the employee base of Anthropic would be like hundreds of Arc Institutes or tens of Gates Foundations.
Schult Douglas: There's all kinds of things that people are thinking of. One foundation that I've donated to is a program that intends to try and end all viral disease. A lot of people are focusing on things which are health-related. One of my colleagues has donated to a nonprofit that's focusing on trying to figure out the right way to handle labor-force impacts: really study them, figure out what's going on, what should we be actually concerned about, what's the real picture and data on the ground, and what policy should we be advocating for so that everyone makes it through this transition supported and better off on the other side.
Nick Marwell: I am certainly heavily influenced by my father in this regard, who spent the last 15 or 20 years focused on education. I just talked about how education—while the things we may want to learn I think will change a lot, it's unclear to me that we want to be going out and getting PhDs in highly technical fields 20 years from now—I do think the pursuit of knowledge, the joy of learning, the ability to engage intellectually with one another, and frankly to understand the world that we're living in (which is going to be a very confusing world for most of us if things play out in the way we've been discussing) is only growing in importance.
Nick Marwell: One thing that many people in AI believe, that I actually think they get wrong, is that AI is just going to fix education. If you care about the education of everybody, and not just how do I give a really great education to a child with 2 parents who are in the 1% and who are highly educated themselves, I think that there's 2 really important things to understand.
Nick Marwell: The first one is your education is a deeply, deeply compounding thing. A good example of this is the number one predictor of how well a student will do in math is whether they can read at grade level at the end of third grade, because when you get to the end of third grade, this is where you transition from learning to read to reading to learn. And there are many things in education that are like this, especially because children and many humans have this psychological thing where when they perceive themselves to not be good at something, they lose motivation to keep doing it.
Nick Marwell: And the problem, if you believe this, is that there's one thing that at least so far AI seems to be extremely bad at: computers cannot hold the attention of young kids. In fact, they seem to be an incredibly distracting force.
Host: That could be a whole new thing you train them on if you want. You've got to figure out the game, although I'm not sure.
Nick Marwell: And so while I think that there are parts of the process of educating children that AI will help in extreme ways, in particular making sure we're teaching kids the right thing for them at the right time, I really don't think it's going to be this panacea where we just get really smart AI and all of the problems of education go away.
Predictions for 2028
Host: There are a lot of smart people working on that. I agree it is really critical. Flash forward to August 2028. You're looking back at this podcast. What has to have happened for you to say, "Wow, that went much faster than I expected"?
Schult Douglas: Much faster than we expected? We expect things to go pretty fast.
Host: Much faster.
Schult Douglas: Timelines to get data centers in space is a big one. I think there's wide error bars on this depending on who you ask.
Host: And that's a really big part of it, because there's so much more you can do up there. The compute ramp, once you have that, will be even faster, and that's the main thing we need.
Schult Douglas: I broadly expect something like a terawatt to be online by then, but TBD on the actual ramp. I would be interested to see how many humanoid robots we have in homes in 2028. I think this is very likely to be a whole bunch of early deployments—maybe tens of thousands of robots in 2028 in homes doing laundry, some basic cleaning, and this kind of stuff. But it could also be a fair bit faster than that.
Host: We just put out an episode with Generalist. It's pretty amazing how good the world models are getting, and it's faster than I expected. It seems like that could be the thing that makes everything go a few years faster.
Schult Douglas: Exactly. And then once you get robots, it really compounds; progress goes fast.
Nick Marwell: I think a lot of the things here are data-related in some ways. One of the biggest regulators of progress in AI capabilities is how fast we can scale the data in the domains that we care about. And I think that this has already gone faster than I expected, for a reason I discussed earlier: there's been a lot more money to throw at the problem than I thought there would be because of what lab revenues have become.
Nick Marwell: But my expectation is we will be slowed by some rate at which humanity can produce this data. And if you told me, for example, that we went on a crusade of building tons of labs to produce biological data for training models, and that started today and was in full swing a year from now, I probably would update my timeline sooner.
Host: Now we're focused obviously on all these types of data for very specific tasks like coding and math and everything else. Do you think one day we'll be working on data for shaping virtuous people, or good character, or mental health, or something like that? Obviously it's much more complicated, but could there be these other really complex problems the AI solves in the future?
Schult Douglas: Those problems I kind of want humans to solve, to be honest.
Host: Wouldn't we use a tool to solve the problem, though?
Schult Douglas: Definitely AI will help us in those discussions, right? But I think that falls into the broad category of data which I expect to be both quite tricky to scale and extremely dependent on the person. That comes later than when we solve robotics.
Host: You're still somewhat human. It sounds like there's still room for us.
Schult Douglas: Would I expect an AI to win a Pulitzer Prize in 2028? Probably not.
Host: Well, what if we didn't know we were giving it to the AI?
Schult Douglas: That's true. Would I expect it to win a Fields Medal in 2028? Quite possibly, right? Extremely likely. Nobel Prize also not crazy. It feels very likely before the end of the decade.
Host: Well, it's exciting to chat with you guys. We definitely have some challenges to work through, but I'm feeling very optimistic about the future of humanity if we get this stuff right.
Schult Douglas: Me too. And thank you for joining us.
Nick Marwell: Thanks for having us.