Google DeepMind Transcript: Yao Shunyu on Training Claude 3.7 and Gemini 3, the Death of Individual Heroism, and Why AI is Fundamentally Simple
Q1 2026 Interview with Google DeepMind Researcher Yao Shunyu on the Evolving Frontier of Large Language Models
Introduction: The Two Yao Shunyus of Silicon Valley
Xiaojun: Hello everyone, I am Xiaojun. Today, our guest is Yao Shunyu, a researcher at Google DeepMind. There are actually two famous individuals named Yao Shunyu in Silicon Valley. One previously worked at OpenAI and later moved to Tencent to become their Chief AI Scientist, and he has been on our show before. Today, I have invited the other Yao Shunyu, who was previously at Anthropic and is now at Google DeepMind. We will start by talking about the recent series of massive changes in the model landscape. Let us dive into my interview with Shunyu.
Yao Shunyu: My name is Yao Shunyu. Obviously, there is another researcher with an almost identical name who is currently the Chief AI Scientist at Tencent and a former OpenAI researcher. Our career paths have some overlap, so it might look very difficult to tell us apart. I originally studied physics, completing my undergraduate degree at Tsinghua University where I worked on condensed matter theory, and then went to Stanford University to study theoretical high-energy physics, quantum information, and black hole physics. After leaving Stanford, I briefly stayed at Berkeley for two weeks as a postdoctoral researcher before quitting to join Anthropic, where I stayed for a year. Around late September to early October last year, I joined the Gemini team at Google DeepMind. If people want to tell us apart, the biggest difference is that the other Shunyu has done computer science from the very beginning, whereas I came into this field halfway after doing theoretical physics.
Xiaojun: Are you two good friends? You seem to have known each other since college and were in the same year, correct? How would you evaluate him and yourself?
Yao Shunyu: Yes, we have known each other since our undergraduate days at Tsinghua. He studied computer science from the start in the Yao Class, which is the computer science experimental class, while I studied physics in the Ji Class. Later, he went to Princeton and I went to Stanford. This might be another confusing point, because people typically think Stanford is where computer science students go and Princeton is where physics students go, but we did the opposite. We really are quite different. I think he is a much more interesting person than I am, and I have learned a lot from him. He spends a lot of time thinking about human-AI interaction and product-related concepts. When we were in Silicon Valley, we met quite frequently, perhaps every few weeks, mostly just to hang out, go for walks, chat about random things, have meals, and play cards. Even after he returned to China, we still call each other every few months to catch up and share updates.
Xiaojun: Has he tried to recruit you to join his team at Tencent?
Yao Shunyu: He probably has, but I do not think it matters. Personally, I have not figured out my next long-term steps yet. I did not join any Chinese companies because when I left Anthropic around September of last year, my biggest motivation was simply to learn something different. I was not focused on leading a massive project; I wanted to prioritize learning new technical domains, which is why I chose to join the Gemini team.
The Shift in AI Competition: Leveling the Playing Field
Xiaojun: The other Yao Shunyu famously stated last year that AI has entered the second half of its development cycle. What stage do you think AI is at today?
Yao Shunyu: For me, the definition of a first and second half has never been particularly clear. However, AI has certainly entered a stage where industry players worry less about whether a model can achieve a certain capability and more about whether the target problem itself is well-defined. A year ago, when I was at Anthropic, everyone was anxious about whether we had a chance to catch up with or surpass OpenAI's reasoning capabilities. Today, among the top labs like Google Gemini, OpenAI, and Anthropic, no one is seriously worried about falling behind. The harder part now is figuring out what we actually want the models to do. This requires significant human insight and involves making major strategic bets.
Xiaojun: Does that mean model capabilities have leveled out and become homogenized, leaving little room for differentiation on paper?
Yao Shunyu: On paper, meaning on standardized public benchmarks, everyone is extremely close. Standardized measurement frameworks like SWE-bench, AIME, or IMO used to clearly distinguish models, showing which was stronger at reasoning or coding. Now, the top models are all crowded around the same high tier, often around eighty percent on complex benchmarks, where a one or two percentage point difference is mostly noise rather than a signal of true superiority. Despite this, users can still feel the differences in actual day-to-day experience. Claude remains the most capable general-purpose model for tool-using agents. In pure coding, other competitors have closed the gap, and Gemini excels at pure reasoning and everyday conversational scenarios.
Xiaojun: Are these capabilities a result of deliberate prioritization by the companies, or is it a matter of engineering limitations?
Yao Shunyu: Prioritization is a massive factor. In the past, when model differences were visible on paper, willingness to prioritize a specific direction allowed teams to dedicate the necessary effort to build the right data pipelines and infrastructure. Data curation takes an immense amount of time and effort. Currently, both prioritization and fundamental definition issues are at play. When a problem is not clearly defined, the actual model performance differences often stem from factors we would not even initially imagine. For example, years ago, researchers noticed that models trained on web data were surprisingly good at writing code. It turned out that because GitHub data is naturally structured and formatted to a higher standard than typical web pages, the overall quality of the pre-training data was inadvertently elevated.
Product Moats and the Evolution of AI Assistants
Xiaojun: As a frontline researcher, what do you think of the recent excitement surrounding open-source AI computer-use agents like OpenClaw? What discussions are happening within the research community?
Yao Shunyu: Interestingly, the discussion outside the industry seems more intense than inside. For industry insiders, this product form is not particularly surprising. Inside the top companies, researchers had already built similar internal demos and experiments; they just had not packaged, polished, and marketed them as public consumer products. The early version of the code for OpenClaw on GitHub was not exceptionally clean, but it was important because it proved the viability of the concept to the public. Shortly after showcasing this possibility, the creator of OpenClaw joined OpenAI. The major AI labs will quickly catch up, polishing these capabilities into robust, consumer-ready products. Even before OpenClaw was released, Google was actively working on similar agents, but large corporations naturally have longer development and release processes. Technically, these agentic capabilities were already viable when Claude 3.5 Opus was released, as it possessed superior tool-use capabilities compared to its contemporaries.
Xiaojun: What is the qualitative difference between earlier agent frameworks like Manus and newer ones like OpenClaw? Why did both choose to sell their technology to larger players like Meta and OpenAI?
Yao Shunyu: To be honest, I am still not entirely sure what the qualitative difference is or why older frameworks could not achieve the same viral success; perhaps it came down to execution and timing. As for why they chose to sell, it illustrates that for an AI startup to survive long-term, it needs a moat, and currently, the primary moat lies on the model side. We have not yet seen any consumer-facing AI-native application successfully establish a true data flywheel. Aside from developer-focused agentic coding tools, there are very few purely AI-native applications that have achieved massive success. Chatbots themselves are largely an interactive extension of traditional search engines. They satisfy a similar informational need but offer a highly interactive, iterative experience that summarizes and distills answers.
Xiaojun: Does the acquisition of these application wrappers by model companies prove that startups cannot easily escape the gravitational pull of the foundational model providers?
Yao Shunyu: Currently, there are two distinct paths for wrapper startups to survive. The first is to achieve escape velocity through hyper-growth. If you grow fast enough, by the time the model companies catch up to your product form, you have already captured user mindshare and can begin training your own specialized models. Cursor is a prime example of this strategy. They began as a seamless partner of Anthropic, but as Anthropic developed its own developer tools like Claude Code, Cursor began training its own Composer model, pushing them into a highly competitive relationship. In productivity and developer tools, the market is often a winner-takes-all dynamic, which creates a strong sense of urgency for both startups and model providers. The second path is to operate in a market that is small enough that the major model companies cannot be bothered to compete directly. Midjourney is a great example of this. While a company like Google could easily allocate resources to replicate what Midjourney does, the market size is not large enough to justify diverting focus away from core general-purpose models.
Xiaojun: If it were you, which path would you choose?
Yao Shunyu: Personally, I would love to swing for the fences and chase the massive, one-in-ten-thousand opportunity. However, realistically, you cannot get there overnight, so securing a smaller, high-upside niche first is often the more practical strategy. Regarding acquisitions, when large companies buy external product teams, a significant benefit is acquiring top-tier product and engineering talent, particularly in regions like Asia. While the US still leads in cutting-edge foundational model research, China has a deep pool of exceptional product and engineering talent. Startups can move fast, write messy code, and launch experimental features. Large enterprises face massive legal, brand, and resource constraints, preventing them from shipping highly experimental products that request deep system permissions on a user's computer.
Scaling Laws and the Myth of Pre-training Limitations
Xiaojun: Standing in Q1 2026, what are your personal expectations for the model landscape over the coming year?
Yao Shunyu: There are so many exciting possibilities. On the technical side, I am a strong advocate for the philosophy of training with finite context but using it as if it were infinite. This means training models on limited context lengths but enabling them to handle near-infinite context windows during inference. If we achieve this, it will unlock entirely new applications, such as a continuous personal assistant that constantly interacts with you, dynamically evaluates the conversation, discards unimportant information, and retains crucial context over long periods. I believe we will definitely achieve this technically this year. The industry has not yet reached a consensus on the exact technical path, but we have multiple promising ideas that we are actively testing under real-world user scenarios to find the most efficient approach.
Xiaojun: Do you feel that the pace of model improvement is slowing down?
Yao Shunyu: Not at all. It is hard to quantify the velocity curve because if you only look at standard benchmarks, progress will naturally appear to slow down as scores approach one hundred percent. However, the qualitative difference felt by the user does not scale linearly with benchmark gains. Going from seventy percent to seventy-five percent on a complex task can feel like a much larger leap in usability than going from fifty to sixty percent. As a researcher, my personal observation is that the models' fundamental capacity to learn is getting stronger. Tasks that used to require massive engineering effort to teach a model can now be achieved with far less friction. The bottleneck is no longer the model's capacity to learn, but our ability to clearly define the problem and construct high-quality data environments.
Xiaojun: Why is the learning capacity getting stronger? Some people have argued that the Scaling Law for pre-training is hitting a wall.
Yao Shunyu: My experience is that pre-training scaling has not hit a wall, and I do not see any signs of it plateauing. When people claim the Scaling Law is reaching its limit, it is usually due to one of three things. Either they believe the physical limits of the paradigm have been reached, or they believe we have run out of usable data to scale further, or they simply have an unnoticed bug in their own implementation. In the vast majority of cases where teams hit a wall, it is due to the third reason. In this industry, fixing a single subtle bug in your scaling science or data pipeline can yield progress that far outpaces any fancy algorithmic trick. When you run into a bug, if you believe the paradigm is limited, you declare that you have hit a wall. If you believe it can be fixed, you keep digging. This is why systematic problem-solving, which both Gemini and Anthropic do exceptionally well, is so critical. You must design clean ablation experiments to isolate and test every variable.
Xiaojun: What is the primary driver of this continued progress: compute, data, or algorithms?
Yao Shunyu: They all contribute, but data and compute are heavily correlated; as compute scales, you naturally require and attract more data, and vice versa. Algorithmic breakthroughs tend to occur in phases. When you are stuck at a conceptual bottleneck, an algorithmic breakthrough is absolutely critical because it suddenly makes the impossible possible, just as the invention of the Transformer architecture did. Once the framework is established, algorithmic progress becomes more gradual, focused on improving computational and data efficiency. Within our current, clear frameworks of pre-training and post-training, compute and data remain the primary drivers. However, in unsolved scientific domains like multimodal generation, the paradigm is not yet fixed, making algorithmic innovation the dominant factor there. For natural language, the scientific framework is relatively clear, and the challenge lies in the massive engineering execution.
The Explosion of AI Coding Tools
Xiaojun: Why has coding been the fastest-developing application area for AI models over the past several months?
Yao Shunyu: Coding has two massive advantages that make it perfect for AI. First, it has a highly objective and well-defined reward signal. If you ask a model to write code to implement a specific feature, you can easily run tests to see if the input matches the expected output. The feedback loop is immediate and unambiguous. Second, coding has an incredible, decades-long data foundation in GitHub, which provides a massive repository of high-quality code written by excellent programmers. This allows researchers to build highly robust simulation environments. From a product perspective, the user requirements for coding are relatively singular and standardized. Unlike social media or gaming, where user tastes are highly subjective and diverse, excellent programmers generally agree on what constitutes clean, concise, structurally clear, and well-abstracted code. This consensus makes developing coding products much more straightforward.
Xiaojun: In your daily work as a researcher, what percentage of your code is generated by AI?
Yao Shunyu: A conservative estimate is that over ninety percent of the raw code I use is model-generated. If I am being less conservative, it is closer to ninety-nine or one hundred percent. However, I still spend a significant amount of time reviewing, editing, and verifying the logic to ensure it aligns with my exact design. The core of software engineering has shifted from writing syntax to designing system architecture, defining logic, and providing the model with the right context and reference files. The portion of code that the model cannot write is shrinking rapidly. Compared to eighteen months ago, the efficiency of implementing ideas and running research experiments has increased by twenty to fifty times. You can run multiple experimental ideas in parallel, and let the model monitor the runs and analyze the results. This has actually made my working hours longer and more intense; because the iteration cycle is so fast, there are constantly more ideas I want to test immediately. In the generative AI field, no one is coasting. If you are self-driven and passionate about the technology, the pace is incredibly demanding.
Xiaojun: Will programmers eventually be completely replaced by AI?
Yao Shunyu: I believe that day will come, but it will be a gradual, phased process rather than an overnight layoff of all developers. We can already see this starting to happen. AI is a highly centralized technology; it empowers a small group of highly capable individuals to become incredibly productive, but it may diminish the unique value of average, entry-level developers. The final result for traditional software engineering might be that one-thousandth of the people do the work of the entire past workforce, earning significantly higher compensation. My advice to programmers is to embrace these new tools and focus on learning how to effectively collaborate with AI. Focus on high-level system design, strategic planning, and understanding how your technical work integrates into a large organization. AI can write specific code blocks incredibly well, but breaking down a massive, ambiguous project into actionable, modular steps is still a human-dominated skill—at least for now.
Evaluating the Gap Between Chinese and Western AI Models
Xiaojun: What are your thoughts on ByteDance's recent successes in the multimodal generation space, such as their Seed model? Does this pressure the team at Google DeepMind?
Yao Shunyu: It certainly brings some pressure, especially for the teams directly responsible for multimodal generation. However, I do not think it reflects a fundamental paradigm shift. ByteDance has historically possessed strong advantages in user data and product engineering, and they execute these details incredibly well. Multimodal generation remains a highly active scientific problem where the training paradigms are not yet completely standardized across the industry. If I had to guess, ByteDance's strong performance comes from superior data curation and meticulous attention to engineering details. Regarding Wu Yonghui leading ByteDance’s large model team, I have not worked with him directly, but looking at his past contributions and code commits at Google, he is one of the rare, highly senior leaders who still maintains incredibly sharp hands-on technical skills.
Xiaojun: If we take a snapshot of the industry in Q1 2026, is the capability gap between Chinese and Western AI models widening or narrowing?
Yao Shunyu: The absolute capability gap is narrowing, and Chinese model developers have gotten incredibly good at building competitive models despite facing severe hardware and compute constraints. However, whether they can completely close the gap or surpass Western models remains an open question. The compute disadvantage has forced Chinese labs to become highly innovative in other areas, such as model distillation and multi-agent training systems.
The Dynamic Between Hard and Soft Distillation
Xiaojun: Anthropic's CEO recently criticized several competitors for distilling data from their models. How do you view the practice of model distillation?
Yao Shunyu: Model distillation is an open secret in the industry, but there is a major difference between brute-force distillation and smart, soft distillation. Brute-force, or hard distillation, involves simply taking a massive batch of outputs generated by a model like Claude and training your own model directly on those tokens. Commercially, this is questionable, and intellectually, it is somewhat foolish because it shows that a team does not know what they want to build; they are merely copying someone else's distribution to score higher on standard benchmarks. On the other hand, soft distillation involves using other models as assistants or evaluators within your own proprietary data generation pipeline. This is a fascinating scientific approach. It has effectively turned some Chinese labs into pioneers of multi-agent training systems, as they are integrating diverse model distributions into a single cohesive training ecosystem. ByteDance's Doubao is quite distinctive; while it may not match the raw reasoning intelligence of Gemini or Claude, its real-time voice generation is arguably the best in the world. They have optimized heavily for their specific consumer user base, prioritizing low latency and engaging user experience over raw benchmark performance.
Xiaojun: Why have US companies not prioritized those real-time consumer features to the same extent?
Yao Shunyu: The primary focus for US model developers has been maximizing work efficiency and raw cognitive intelligence. In China, there is a much stronger focus on the consumer side, creating products that are engaging and fit into daily life. For instance, the Doubao phone is an excellent product concept that executes tasks in real time, though the underlying technical overhead must be carefully optimized to ensure the cost of running the model does not exceed the value of the service it provides. Apple is also trying to address this consumer assistant space through its partnership with Gemini, recognizing that while they care deeply about Siri and their AI strategy, their in-house model development has lagged behind the frontier.
The Future of Robotics and World Models
Xiaojun: There is a lot of excitement surrounding humanoid robots. Do you have any observations on the robotics space, and how does it compare to the development timeline of large language models?
Yao Shunyu: I have looked at some of the humanoid robots and their pricing; the hardware supply chain in China is incredibly mature, making these physical platforms surprisingly cheap. However, on the software side, robotics is still very much in its feature engineering era. Robots can be optimized to perform incredibly well in highly specific, deterministic environments, but they lack generalizability. If you train a robot to fold clothes or pour water, that capability does not naturally abstract and generalize to other tasks. Language models have crossed that critical threshold where scaling up the base model lifts all downstream capabilities simultaneously. Robotics has not yet reached its GPT-1 moment; we have not figured out how to truly scale the training of physical action models. Many robotics labs are now experimenting with using multimodal language models as a base, training vision-language-action models to bridge this gap. Visiting these labs is incredibly fun—they are far more dynamic than language model offices, with researchers constantly collecting physical data and controlling machines.
Xiaojun: What is your definition of a world model? There seems to be significant disagreement in the industry on this term.
Yao Shunyu: Indeed, ask ten thousand people and you will get ten thousand different definitions. The world models being built by academic labs are very different from the end-to-end models we focus on. For Gemini, a world model is trained end-to-end so that it does not just generate a static video from a description, but actually models a physical scenario. The model generates a state, takes an action input, and predicts the subsequent state as a direct function of the previous state and action. It is still in a highly exploratory phase, and we do not yet have a singular industry-wide consensus on the ultimate form of a world model.
Yao Shunyu’s Journey From Theoretical Physics to Artificial Intelligence
Xiaojun: Let us talk about your personal background. You studied theoretical physics at Tsinghua and Stanford before transitioning into AI. How did you make that transition?
Yao Shunyu: I was born in Dawukou, a small coal-mining city in Ningxia, but moved to Shanghai with my parents during elementary school. I attended a relatively ordinary middle school and high school, and I was quite mediocre when I was young. I did not enter Tsinghua through the elite olympiad competition path. In high school, I had the choice between attending a regular class at one of Shanghai's top four high schools or joining the competitive physics class at Gezhi High School, which was considered an underdog. I chose the underdog class because I love challenging myself with things I am not naturally good at. That decision ultimately led me to Tsinghua's physics department. During my senior year of high school, I boldly messaged the Tsinghua admissions office asking why they were only opening independent enrollment testing to Beijing students. They agreed to let us take the exam, and I got in. That taught me a massive life lesson: be bold and fight for opportunities, because if you do not ask, the answer is always no.
Xiaojun: At Tsinghua and Stanford, you worked on condensed matter theory and non-Hermitian systems. Can you explain that research in simple terms?
Yao Shunyu: I will try to speak in human terms. In basic quantum mechanics, we assume isolated systems undergo unitary evolution described by a Hermitian matrix, meaning the energy states are mathematically self-consistent. However, real-world systems are never perfectly isolated; they constantly exchange information and matter with their environment. These are open quantum systems, described by non-Hermitian Hamiltonians. During my undergraduate research, we noticed that our analytical calculations for these systems did not match our numerical simulations. We eventually discovered that in non-Hermitian systems, the energy eigenstates can physically accumulate at the boundaries of the material, a phenomenon we systematically modeled and established a mathematical framework to describe. That research taught me a crucial lesson that applies directly to AI: do not trust pure theory too much, and use numerical simulations to constantly verify and refine your understanding of complex systems.
Xiaojun: Why did you choose to leave high-energy physics during your PhD at Stanford?
Yao Shunyu: High-energy physics is incredibly beautiful and intellectually challenging, but it has developed to a point where theories cannot be experimentally verified in our lifetime. Without empirical data, the field relies heavily on mathematical self-consistency and the subjective opinions of senior academics. I realized I did not want to spend my life serving the academic establishment of a field that lacked objective, real-world feedback. I wanted to work on things that could actively impact the world and had clear, objective criteria for success. During the final years of my PhD, I explored quantum computing and AI. I realized that the primary bottleneck in quantum computing is experimental hardware engineering, which is not my strength. AI, however, is a digital sandbox where you can conceive an idea and immediately run numerical experiments to verify it. It felt very much like the golden age of nineteenth-century physics, where theoretical speculation and active experimentation went hand-in-hand.
Working on Claude 3.7 and 4.5 at Anthropic
Xiaojun: How did your journey at Anthropic begin, and what was the culture like during the development of Claude 3.7 and 4.5?
Yao Shunyu: I had former academic colleagues at Anthropic, which has historically hired many researchers from theoretical physics backgrounds. When I interviewed in late 2024, the reinforcement learning team was very small—about ten or eleven people on the core team. At the time, RL for large language models was highly immature; OpenAI's o1 had not been publicly released, and most labs were still trying to figure out how to make large-scale RL stable. The team manager, who also had a physics background, invited me to work on these scaling challenges. I hand-rolled my own basic GPT systems using open-source projects like Andrej Karpathy's nanoGPT to prepare for the technical interviews, and joined the team shortly after.
Xiaojun: What made Anthropic unique as a research organization during that period?
Yao Shunyu: Anthropic has a highly effective, top-down decision-making structure. Because the core technical leaders like Jared Kaplan and Sam McCandlish are also co-founders of the company, they possess immense technical credibility and corporate authority. When they receive a clear technical signal, they can instantly align the entire company to go all-in on a specific bet, such as prioritizing coding capabilities. This is very difficult for other companies to replicate. At OpenAI, internal political struggles disrupted this alignment, and at Google, the organization is historically much more bottom-up and distributed. Anthropic's co-founding team has a deep level of mutual trust built from fighting battles together since their days at OpenAI, which keeps the organization incredibly reactive and focused.
Xiaojun: How was Claude 3.7 developed, and what role did you play in that project?
Yao Shunyu: Before Claude 3.7, post-training was largely a process of patching up pre-trained models rather than a scalable paradigm of its own. The breakthrough came when the industry figured out how to scale reinforcement learning by utilizing clean, objective environments with clear feedback signals—specifically software engineering environments. The core development of Claude 3.7 took about four to five months. My work was focused on the algorithmic stability of large-scale RL and building the modular training environments. However, AI development at this scale is a massive, highly collaborative systems engineering effort. The era of individual heroism in AI is over; the technology is an unstoppable wave, and researchers are simply surfers trying to catch the crest. If one individual does not make a specific discovery, another team in a well-funded lab inevitably will a few weeks later.
Contrasting the Cultures of OpenAI, Anthropic, and Google DeepMind
Xiaojun: Why did you decide to leave Anthropic to join the Gemini team at Google DeepMind?
Yao Shunyu: There were several factors. First, Anthropic grew very rapidly, more than doubling its headcount to nearly two thousand people while I was there. This rapid growth introduced significant cultural dilution and internal friction. I personally dislike cultures where people spend their days on Slack talking about grand, abstract principles rather than doing the hard, meticulous engineering work required to implement ideas. Second, I disagreed with the increasingly extreme and emotional anti-China political stance pushed by Anthropic's leadership. Finally, Anthropic is highly focused on a narrow product set, meaning they do not invest in low-level infrastructure research or multimodal generation. I wanted to learn from a broader set of human minds and have the freedom to explore diverse research directions, and Google DeepMind offers an unparalleled environment for that scale of research freedom.
Xiaojun: Many researchers leave Google because they feel the corporate structure is too slow or restrictive. Why do you have the opposite view?
Yao Shunyu: It depends entirely on what you want. If your goal is to immediately force your specific idea into a consumer product, Google's massive size and complex release processes can be highly frustrating. But if your goal is research freedom and accessing world-class engineering talent across a massive spectrum of technologies, there is no better place. Google's technical reserves are unmatched. Additionally, OpenAI's early success actually saved Google by waking the giant up. It forced Google to reorganize and streamline its decision-making, transforming pre-training into a highly disciplined, predictable engineering process that operates completely within Google's comfort zone.
Why AI Research is Fundamentally Simple
Xiaojun: You have made the controversial statement that AI is fundamentally simple and does not require exceptional intelligence. Can you explain what you mean by that?
Yao Shunyu: In physics, if you do not have experimental data at a specific energy scale, you simply cannot proceed. AI does not face this fundamental constraint. It is a highly empirical field where we are not blocked by a lack of ideas, but rather by the time and compute required to test the thousands of ideas we already have. It is fundamentally simple because you do not need brilliant, paradigm-shifting insights to make progress; you need to be exceptionally reliable, detail-oriented, and systematic in your execution. You must write clean code, eliminate subtle bugs, and carefully analyze your empirical data. Most of the breakthroughs are the result of collective, disciplined engineering rather than individual genius.
Xiaojun: How do you design your interview process to identify reliable, high-quality AI researchers?
Yao Shunyu: I designed an interview task where the candidate is given twenty-four hours to build and train a complete reinforcement learning project from scratch. They must choose the model, the data, and the algorithm, and then discuss their implementation with me for an hour. In the modern AI era, writing the raw code is easy because the candidate can use AI to do it. The real test is whether they truly understand the system they built. If they blindly let the AI generate everything without deep, systematic comprehension, they will immediately fail during our technical discussion. It also tests their dedication and hunger—whether they are willing to pull an all-nighter to deliver an exceptional result under tight constraints.
Engineering Leadership and Managing Large-Scale AI Projects
Xiaojun: How do different labs organize their research teams to manage the trade-offs between pre-training and post-training?
Yao Shunyu: Both Anthropic and Google maintain separate, dedicated teams for pre-training and post-training. OpenAI's structure has historically been more fluid and sometimes chaotic, with their post-training, reinforcement learning, and product teams occasionally overlapping in their responsibilities. Effective organizational structure ultimately relies on having a technical leader with immense credibility. This leader must have the hands-on capability to step onto the front lines and solve critical technical fires when things go wrong, but also have the tolerance to understand and support diverse, exploratory ideas that they might not have conceived themselves. Google has found a strong balance in this regard, with leaders like Koray Kavukcuoglu managing the core Gemini scaling projects while Demis Hassabis focuses on broader scientific applications like drug discovery.
Xiaojun: What is Google's competitive advantage when it comes to custom hardware like TPUs versus the industry-standard NVIDIA GPUs?
Yao Shunyu: From a pure hardware perspective, neither is inherently superior for large-scale deployments. The primary advantage of GPUs is their mature, open-source software ecosystem, whereas TPUs require Google to invest heavily in building its own custom compilers and infrastructure. However, at the scale of hundreds of thousands of chips, this software investment is entirely justified. The architectural design philosophies are quite different. NVIDIA's GPU pods focus on incredibly high-speed NVLink interconnects within a relatively small node of cards. Google's TPUs utilize a 3D Torus network topology, sacrificing some pairwise interconnection speed to link massive numbers of cards directly within a single cluster. If your compilers and data-sharding strategies are written well enough to leverage this topology, you can achieve massive memory capacity and significantly reduce communication bottlenecks during large-scale training.
The Future Outlook for LLM Paradigms and Career Advice for Programmers
Xiaojun: For young people entering the AI field today, is it still a blue ocean of opportunity?
Yao Shunyu: Pure natural language processing and standard LLM scaling are no longer blue oceans; the last train has already left, and the landscape is dominated by massive, consolidated engineering teams. However, AI is an incredibly vast domain. There are massive opportunities in multimodal generation, physical robotics, and using AI to solve fundamental scientific and mathematical problems. The best choice for young researchers is not to chase the loudest trend of today, but to focus on the unsolved, neglected challenges of tomorrow.
Xiaojun: What are your own personal plans for the future?
Yao Shunyu: I will likely not stay at a massive tech corporation forever. I want to continue challenging myself, finding hard problems to solve, and learning new technical domains. Right now, my highest priority is pushing our ML coding and long-horizon training systems into a highly stable, production-ready state. Once that is accomplished, I will look for the next difficult challenge to tackle.
Xiaojun: Every guest on our show recommends a book that has deeply influenced them. What would you recommend to our audience?
Yao Shunyu: I am honestly not very well-read and do not enjoy reading traditional books. Most of what I read is professional research papers. However, I recently read the autobiography of Hideki Yukawa, the Nobel laureate in physics, titled Tabibito, which translates to The Traveler. It is a very authentic and comforting look at a highly successful scientist struggling with self-doubt and challenges during his youth. If you want a great fiction recommendation, I highly recommend the Japanese novel From the New World.
Xiaojun: Thank you, Shunyu, for sharing your incredibly direct and valuable insights with us today.
Yao Shunyu: Thank you. It was a pleasure chatting with you.