State of AI in 2026: LLMs, Coding, Scaling Laws, China, Agents, GPUs, AGI | Lex Fridman Podcast #490

Sebastian Raschka and Nathan Lambert join Lex Fridman for a 4.4-hour deep dive into the state of AI in early 2026, covering architecture, scaling, post-training breakthroughs, coding agents, AGI timelines, and the geopolitical race for open models. The conversation opens with the DeepSeek moment of January 2025, which triggered an avalanche of competitive Chinese open-weight models (Kimi, MiniMax, Z.ai). Raschka emphasizes that no single company holds proprietary ideas—the differentiator is budget and hardware. Lambert notes Anthropic is winning the developer mindshare war through Claude Code, calling it a cultural bet on coding that is paying off, while Gemini 3 fades despite strong benchmarks. On architecture, Raschka makes a striking claim: modern frontier models remain fundamentally the same GPT-2 architecture with incremental tweaks—Mixture of Experts, Group Query Attention, RMSNorm, activation function swaps. You can literally build from GPT-2 to any current model by adding components. The real gains come from three scaling axes: pre-training (data quality and compute), post-training (RLVR on verifiable domains like math and code), and inference-time compute (longer chain-of-thought reasoning). Lambert adds that systems-level improvements (FP8/FP4 training, faster tokens-per-second-per-GPU) enable faster experimentation loops that compound into capability gains invisible at the architecture level. Post-training emerges as the decisive battleground. RLVR (Reinforcement Learning with Verifiable Rewards) replaced RLHF's learned reward model with ground-truth verification on math and code, enabling much longer training runs. DeepSeek R1's "aha moment"—where the model self-corrects mid-reasoning—fell naturally out of this process. Lambert warns that many RLVR papers trained on Qwen base models may be contaminated, inflating reported gains. The compute required for RL post-training now approaches pre-training in wall-clock time, though on different hardware profiles (memory-bound vs compute-bound). On coding with AI, both guests describe a "jagged frontier"—models are superhuman at frontend and traditional ML code but struggle with distributed systems and novel infrastructure. Lambert predicts software engineering will shift toward system design and outcome specification, with agents implementing features over 1-2 day cycles. Raschka compares the trajectory to calculators replacing manual arithmetic: AI will solve coding, but humans will still specify what to build. The spec-driven design pattern emerges as critical—underspecification, not model limitation, is the bottleneck. The AGI timeline discussion is notably sober. Lambert says the AI27 report's prediction of a superhuman coder has been pushed from 2027 to 2031 (mean), and he expects research automation to take even longer. AI capabilities are "jagged"—excellent at some tasks, weak at others—making the "complete" definitions of AGI problematic. Both agree that progress will be amplification of current capabilities rather than paradigm shifts, though breakthrough architectural innovations (text diffusion, state space models) remain possible on longer timescales. Lambert's ADAM Project (American Truly Open Models) frames the geopolitical dimension: Chinese open-weight models now dominate, with zero US-based competitors at DeepSeek caliber as of mid-2025. AI2 received the largest NSF CS grant ever (M over 4 years) to address this. On Meta, both suggest Llama may not have an open-weight successor—internal politics and benchmark-chasing derailed Llama 4, and new leadership under Alexandr Wang deprioritizes open release. Key themes for practitioners: the value of human curation over raw AI output (voice, insight selection, "the 95% implicit requirements"), the permanence of the transformer architecture making implementation knowledge durable, and the emerging pattern of agents managing their own context (Lambert describes training RL where compaction is an action the model controls).