Richard Sutton: Author of the Bitter Lesson Advocates Experiential Learning Over LLM Scaling

Richard Sutton, via his seminal 2019 "Bitter Lesson" essay and a September 2025 Dwarkesh Patel interview, argues that AI progress hinges on general methods leveraging computation through search and learning from raw experience, not human-encoded knowledge like LLMs.[1][2] LLMs mimic human text but lack goals, ground truth feedback, and continual learning from action-reward streams, making them a "dead end" for true intelligence despite short-term gains. This directly challenges Dwarkesh's scaling optimism by predicting experiential RL systems will supersede LLMs, as human priors plateau while compute-leveraging methods scale indefinitely.[2]

• Bitter Lesson (Mar 13, 2019): General computation-based methods outperform knowledge-built ones long-term, as seen in chess, Go, speech, and vision; warns against human knowledge plateaus.[1]
• Dwarkesh interview (Sep 2025): LLMs embed human knowledge, violating Bitter Lesson scalability; true intelligence requires on-the-fly learning like animals, not static pretraining.[2]
• Oak Architecture talk (Aug 2025): Proposes continual RL with meta-learned step-sizes and abstractions for superintelligence from experience alone.[3]

Implications for Competitors: Dwarkesh's LLM+RL scaffold risks lock-in to inefficient human-data dependency; entrants should pivot to raw experiential RL (e.g., robotics sims) for data-efficient scaling post-2030 compute limits.

Andrej Karpathy: Balanced Scaling Plus Agents, AGI in a Decade

Andrej Karpathy, in his October 2025 Dwarkesh interview, supports scaling's empirical success but tempers it: LLMs provide representations via imitation ("ghosts" of humanity), not animal-like evolution, with RL "terrible" due to noise and inefficiency.[4] AGI (~human remote worker) is "a decade away" as agents need multimodality, continual learning, and reliability; progress blends into 2% GDP growth without explosion. Diverges from Dwarkesh's potential short-term takeoff by emphasizing tractable but "difficult" engineering over paradigm shifts.[4]

• AGI Timelines (Oct 17, 2025): "Tractable but difficult" problems like agent cognition yield AGI in ~10 years; rejects 1-5 year hype.[4]
• Scaling/Compute: "Everything plus 20%" across data/hardware/algos; pretraining not dominant, models stay practical sizes amid flops budgets.[4]
• NanoGPT experiments (Jan 2026): Reproduces Chinchilla-optimal scaling (8:1 tokens:params ratio), validating predictable compute-optimal families.[5]

Implications for Competitors: Dwarkesh's explosion odds underestimate agent scaffolding time; focus on "cognitive core" (small, tool-using models) for edge deployment, avoiding frontier compute races.

Yann LeCun: Architectural Paradigm Shift Needed Beyond LLM Scaling

Yann LeCun consistently rejects LLM scaling to AGI (e.g., 2024-2026 talks/papers), arguing autoregressive prediction lacks world models, planning, and common sense; needs joint-embedding predictive architectures (JEPA), energy-based models, and model-predictive control over RL.[6] No specific timelines, but "not in next 2 years... 5-6 years if everything goes well" (Dec 2024); scaling LLMs is inefficient pixel/token prediction vs. latent physics understanding. Implicitly diverges from Dwarkesh by dismissing compute-alone paths, favoring Meta's world-model focus.[7]

• JEPA Path (2024-2026): Abandon generative/contrastive/RL for regularized embeddings predicting abstract states; LLMs "dead end" without world models.[6]
• Timelines (Dec 2024): AGI hard, underestimated historically; not imminent via scaling.[7]

Implications for Competitors: Dwarkesh's bet on LLM extrapolation ignores architecture walls; invest in world models (e.g., robotics data) for post-scaling era.

Dylan Patel: Compute Hardware as the Hard Bottleneck to Scaling

Dylan Patel (SemiAnalysis), in March 2026 Dwarkesh interview/posts, details scaling's physical limits: logic (TSMC/ASML EUV ~200GW cap by 2030), memory (HBM crunch, 30% CapEx), power (solvable). Synthetic data unlocks short-term gains, but supply chains cap compute ~4-5x/year growth.[8] No explicit AGI timelines, but implies US wins short-term (fast scaling), China long-term; aligns with Bitter Lesson via infrastructure enabling compute leverage. Diverges implicitly by quantifying Dwarkesh's "post-2030 algo era" constraints earlier via ASML.[8]

• Bottlenecks (Mar 2026): ASML #1 by 2030 (3.5 tools/GW); memory prices triple; Nvidia dominates N3 wafers.[8]
• Synthetic Data (Dec 2024): Unlocks rapid improvement next 6-12 months.[9]

Implications for Competitors: Dwarkesh's timelines assume smooth scaling; hedge with diversified supply (e.g., older nodes, non-US fabs) or inference optimization.

Category	Sutton (Bitter Lesson Proponent)	Karpathy (Balanced Scaler)	LeCun (Architecture Skeptic)	Patel (Compute Analyst)	Dwarkesh Synthesis (Scaling Optimist)
Scaling Optimism	Low: LLMs plateau; experiential RL scales better[2]	Medium: Continues incrementally ("+20% everything"); obeys Chinchilla-like laws[4]	Low: LLMs doomed; needs JEPA shift[6]	Medium: Synthetic data boosts short-term; hardware caps long-term[8]	High: Predictable till ~2030; 70% AGI by 2040 via scaling+algos[10]
Timeline Confidence	No dates; post-LLM era soon via RL[2]	~10 years to AGI (remote worker)[4]	5-6 years optimistic; hard problem[7]	Implicit: Fast US scaling wins short-term[8]	50% taxes/computer-use AGI by 2028; lognormal, this decade or bust[10]
Architecture Bets	Experiential RL (Oak); no imitation priors[2]	LLM agents + cognitive core; RL poor[4]	JEPA/energy-based/MPC; abandon LLMs/RL[6]	N/A; infra-focused[8]	LLMs + RL/scaffolding; continual learning key bottleneck[10]
Deployment/Econ Gap	Human knowledge locks scalability[2]	Blends into 2% GDP; gradual diffusion[4]	World models enable efficient local AI[6]	ASML/memory/power cap growth; H100 value rises[8]	Explosive if continual learning solved; compute ends ~2030[10]

Strongest Challenges to Dwarkesh:
- Sutton: No goals/ground truth in LLMs blocks world-model RL.[2]
- Karpathy: Agents need decade of engineering; no explosion.[4]
- LeCun: Wrong architecture; scaling predicts tokens, not physics.[6]
- Patel: Hardware (ASML) caps scaling sooner than assumed.[8]

Strongest Agreements:
- All nod to Bitter Lesson/compute leverage; Sutton/Karpathy agree on continual learning need; Patel enables Dwarkesh's scaling short-term.[1][2]

---

Recent Findings Supplement (May 2026)

Richard Sutton: LLMs Lack Experiential Ground Truth for True Intelligence

Richard Sutton, in his September 26, 2025 Dwarkesh Patel podcast, argues LLMs fail the Bitter Lesson by relying on human data rather than scalable experiential learning with intrinsic goals and ground truth feedback.[1] This creates a non-scalable "prior" that plateaus, as LLMs predict human text without verifying outcomes or adapting via surprise—mechanisms essential for animal-like continual learning. No AGI timelines given, but superintelligence inevitable via RL from experience; compute scales methods, but architecture must enable on-the-fly world modeling first.[1]

• Sutton clarifies LLMs are "kinda yes, kinda no" Bitter Lesson: they scale compute but inject human knowledge, which history shows gets superseded (e.g., chess, Go).[1]
• Strongest challenge to Dwarkesh's scaling optimism: LLMs have no "ground truth" (no prediction of real-world response to actions), preventing true world models; building RL atop them repeats past errors where human priors inhibit scalability.[1]
• Agreement: Continual learning is essential for AGI, as humans/animals learn on-the-job without special training phases.[1]

Implications for competitors: Dwarkesh's LLM+RL scaffold risks data exhaustion; pure experiential RL (e.g., Sutton's Oak architecture, presented Aug 2025) offers a compute-efficient path but needs breakthroughs in meta-learning abstractions.[2]

Andrej Karpathy: Agents Need a Decade for Cognitive Fixes Despite Scaling Gains

In his October 17, 2025 Dwarkesh interview, Karpathy predicts AGI (human-level knowledge work) ~10 years out, as current LLMs suffer "cognitive deficits" like no continual learning or reliable computer use—despite scaling across data/algorithms/compute yielding "everything plus 20%" progress.[3] Pre-training bootstraps representations (Bitter Lesson via internet-scale data as "crappy evolution"), but RL is "terrible" (noisy supervision); shift to inference/post-training dominates, with models shrinking for RL speed.[3]

• Timeline from experience: past hype (Atari RL, Universe) failed without priors; agents are "slop" now, maturing over decade via multimodality/memory.[3]
• Strongest challenge: LLMs over-memorize (hazy weights vs. crisp context), needing "cognitive cores" (small, memory-stripped models) for generalization; scaling amplifies bugs like adversarial RL failures.[3]
• Agreement: LLMs analogize human cognition (context=working memory), enabling gradual agentic progress.[3]

Implications for entrants: Bitter Lesson holds (scale general methods), but prioritize post-training/agents over pre-training giants; compute not sole bottleneck—data quality/RL noise demands hybrid human-AI loops.[4]

Yann LeCun: World Models via JEPA Replace LLM Scaling Dead-End

LeCun, post-Meta (late 2025), launched AMI Labs (Jan 2026, $1.03B seed) for JEPA world models, arguing LLMs can't reach human-level AI via scaling: autoregressive prediction lacks causal physics/planning, hitting "dead end" without world models (predict latent states, not pixels/text).[5][6] LeWorldModel paper (Mar 13, 2026) stabilizes JEPA end-to-end from pixels (15M params, single GPU), enabling 48x faster planning vs. giants.[6] No timelines (rejects AGI term; human-level 3-5+ years via paradigmshift); compute wasted on inefficient LLMs.[7]

• Bitter Lesson divergence: scaling LLMs "bullshit" for intelligence; JEPA/model-predictive control scales better for reality.[8]
• Strongest challenge: LLMs can't plan (no world model for "what-if"); JEPA learns causality from video/sensors, obsoleting token prediction.[5]
• Agreement: Scaling compute/data drives progress, but needs architectural pivot (e.g., his 1989 CNN modernized via scale).[3]

Implications for rivals: LLM labs face $200B+ compute sunk cost fallacy; world models (e.g., V-JEPA 2.1) enable efficient robotics/healthcare, but require multimodal data moats.[6]

Dylan Patel: Compute Supply Chains Bottleneck Aggressive Scaling

In his March 13, 2026 Dwarkesh interview, Patel details compute as AI's trilemma: logic (ASML EUV caps ~200GW by 2030), memory (30% of $600B 2026 CapEx, prices 3x), power (scalable).[9] Scaling laws persist (models 10x cheaper/year), favoring early committers (Nvidia/OpenAI lock-ins); H100s appreciate as efficiency rises. No explicit timelines/AGI, but fast progress via RL/smaller models for research speed; China lags but closes if timelines >2035.[9]

• Bitter Lesson alignment: Research flops push Pareto frontier; hardware follows (e.g., Blackwell TMA).
• Strongest challenge: EUV math (~3.5 tools/GW) trumps power hype; older fabs/Taiwan risk limit alternatives.[9]
• Agreement: Aggressive scaling (5-6GW labs by 2026) needed; power no issue.[9]

Implications for builders: Secure forward contracts; inference specialization (context processors) unlocks economics, but memory crunch favors diversified supply.

Comparison Matrix	Scaling Optimism	Timeline Confidence	Architecture Bets	Deployment/Economic Gap
Dwarkesh Synthesis (inferred: LLM+RL scaling)	High: Pretrain priors enable RL	Medium-short (guests vary; his longer ~decade?)[4]	Transformers + agents	Compute scales deployment; GDP blends[3]
Sutton	Low: LLMs plateau; experience scales	Inevitable, no date	RL/experiential (Oak)	Humans data-limited; on-fly learning economical[1]
Karpathy	Medium: Balanced scaling + fixes	~10 years (decade of agents)	Cognitive cores + agents	Inference/RL dominate; gradual GDP 2%[3]
LeCun	Low: LLM dead-end	3-5+ years paradigm shift	JEPA/world models	Robotics needs physics; local efficient[6]
Patel	High: Efficiency laws hold	Short-medium (chip-limited)	RL/smaller for speed	Supply chains cap; forward deals win[9]

No new policy/regulatory updates or stats post-May 2025 beyond compute forecasts; LeWorldModel (Mar 2026) is key publication.[6] Confidence high on interviews; medium on LeCun synthesis (no direct Dwarkesh). Additional X/web for real-time X posts yielded no divergences.