26 January 2026

27 new episodes (19 hours listening) published by the 38 podcasts on the list. 2 highlight clips below (4 mins total).

AI 26 January 2026 · 2 clips

Lex Fridman

Nathan Lambert and Sebastian Raschka

Episode

State of AI in 2026: LLMs, Coding, Scaling Laws, China, Agents, GPUs, AGI | Lex Fridman Podcast #490

Nathan Lambert is post-training lead at the Allen Institute for AI (Ai2) and author of The RLHF Book; the episode draws on Ai2's own OLMo 3 runs, cluster invoices and contamination research. Sebastian Raschka wrote Build a Large Language Model (From Scratch) and Build a Reasoning Model (From Scratch), and reimplements published architectures from GPT-2 upward to check them.

Hosted by

Lex Fridman

31 January 2026 · 4h 25m · 2 clips below

1 of 2 explainer

The Allen Institute for AI spent about two million dollars renting the cluster that trained OLMo 3, a figure that makes building a model the cheap half of owning one.

A thousand-GPU rental runs roughly 100,000 dollars a day, and hundreds of millions of users turn that into billions a year, which is why the largest models are rumoured to have shrunk back below a trillion parameters rather than grown. Labs hit the serving bill long before they hit any limit on whether bigger models keep working.

1m 28s · from 52:10
My observations
- Anyone modelling AI capital spending off training runs is watching the wrong line item. The recurring cost scales with users rather than with releases, so a lab that doubles its subscriber base doubles a bill it cannot amortise over a product. A committee looking at the capex debate is usually shown the smaller of the two numbers.
- Both guests build at small scale, and the two million dollar figure is Ai2's own cluster rental for a 30 billion parameter class model, which makes it a floor for a competent team and not a read on what OpenAI or Anthropic pay.
- OpenAI or Anthropic releasing a model back above a trillion parameters would break this. The rumoured shrinking of the largest models is the whole evidence base underneath it.
2 of 2 contrarian

Qwen's base model answers reworded maths problems with a precision that tool-free generation cannot produce, which two papers this year read as the benchmark having leaked into training.

Keep the wording of a word problem and change only the numbers: a model that reasons degrades, while one that memorised near-identical items keeps returning answers to more decimal places than plain generation allows. Most published reinforcement-learning results are measured on that same benchmark and that same model family, so what reads as a capability gain may be recall.

2m 10s · from 1:44:20
My observations
- A reworded question with the same structure is the whole check, and whether the answer survives settles it. It costs nothing, and almost nobody runs it before a benchmark score reaches a board paper.
- A guest co-authored one of the two contamination papers, so weigh the finding with that in view. The counter-case put in the room is that a base model jumping from 15 to 50% in 50 steps cannot have learned any mathematics, which points the same way.
- Neither guest calls the contamination deliberate, and both land on the same limit: without visibility into the training data there is no controlled experiment to run, and the labs will not disclose it.
- model-capability
- open-models

26 January 2026

AI 26 January 2026 · 2 clips

The Allen Institute for AI spent about two million dollars renting the cluster that trained OLMo 3, a figure that makes building a model the cheap half of owning one.

Qwen's base model answers reworded maths problems with a precision that tool-free generation cannot produce, which two papers this year read as the benchmark having leaked into training.