AI That Can Change Its Mind? (New Architecture) [w/ Sakana CTO]

Episode Overview

• Llion Jones, co-inventor of the Transformer, discusses his departure from Google to start Sakana AI, aiming to explore novel AI architectures beyond the now "oversaturated" Transformer space.
• The podcast critiques current AI models, arguing they rely on brute-force approximation rather than true structural understanding, as illustrated by their failures in extrapolation and generating fine-grained details.
• A new, biologically-inspired model called the "Continuous Thought Machine" (CTM) is introduced, which uses temporal dynamics, neuron synchronization, and an internal "thought dimension" to perform sequential reasoning.
• The CTM exhibits desirable emergent properties, such as adaptive computation (spending more time on harder problems) and near-perfect confidence calibration, without being explicitly trained for them.
• The conversation concludes by proposing a future for AI training that moves beyond static text data to capturing the dynamic "thought traces" of human experts as they solve complex problems.

Key Concepts

• Transformer Saturation: The idea that research on the Transformer architecture has become crowded, limiting opportunities for breakthrough innovation and motivating a shift toward exploring entirely new models.
• Brute Force vs. Inductive Bias: A central theme contrasting the current AI paradigm of scaling models and data (brute force) with the need for architectures that have better built-in assumptions (inductive bias) to understand problems structurally, like the "spiral problem."
• Shallow Understanding: The observation that even large-scale generative models exhibit a superficial grasp of the world, evidenced by errors in structural details (e.g., the number of fingers on a hand), suggesting they approximate rather than truly comprehend.
• Continuous Thought Machine (CTM): A novel, biologically-inspired architecture with three core features:
  • An internal "thought dimension" that allows for sequential, internal computation over time.
  • A representation based on the dynamic synchronization of neurons, rather than static activation states.
  • A reconceptualization of neurons as individual models (Neuron-Level Models or NLMs).
• Adaptive Computation: An emergent property of the CTM where the model naturally allocates more computational steps or "thinking time" to more difficult problems, making it more efficient.
• Emergent Model Properties: The CTM naturally exhibits beneficial traits like near-perfect calibration (its confidence reflects its accuracy) as a byproduct of its design, rather than requiring explicit fine-tuning.
• Training on "Thought Traces": A proposed future direction for AI training that involves capturing the step-by-step reasoning process of human experts, not just their final outputs, to teach models how to think.

Quotes

• At 0:21 - "I'm going to drastically reduce the amount of research that I'm doing specifically on the Transformer because of the feeling that I have that it's an oversaturated space." - Llion Jones explains his core reason for shifting his research focus after co-inventing the dominant AI model.
• At 0:34 - "Do something different. Right, to actually turn up the amount of exploration that I'm doing in my research." - Jones articulates the central mission of his new company, Sakana AI, which is to move beyond the current AI paradigm.
• At 1:03 - "[The model] lets us solve problems in ways that seem more human by being biologically and nature-inspired." - Dr. Darlow describes the unique, nature-inspired approach of the Continuous Thought Machine.
• At 20:17 - "There's no feeling when I look at those... that the ReLU version actually understands that it is a spiral." - The speaker contrasts the brute-force approximation of a ReLU network with a model that would inherently understand the spiral's continuous, generative nature.
• At 22:32 - "Did we fix the problem? Or did we just use more brute force to just, you know, force the neural network to know it's five fingers?" - This quote crystallizes the central debate: whether scaling is true progress or just a more powerful way to hide the same underlying architectural flaws.
• At 30:00 - "The first one is having what we call an internal thought dimension... This is not necessarily something new, it's related conceptually to the ideas of latent reasoning." - Luke begins to explain the first of three key novelties behind the Continuous Thought Machine (CTM) model.
• At 32:40 - "What is the representation for a biological system when it's thinking? Is it just the state of the neurons at any given time? Does that capture a thought?" - Luke questions the static, single-state representation of traditional neural networks and argues for a representation that captures dynamics over time.
• At 45:05 - "I in some sense see chain of thought reasoning as a way of adding more compute to a system." - Luke frames the popular prompting technique as a method for increasing computational depth, a concept their model internalizes.
• At 45:21 - "is have that reasoning component be entirely internal, yet still running in some sort of sequential manner." - Luke describes the core goal of the CTM architecture: to embed the reasoning process within the model itself.
• At 49:30 - "The flavor of this kind of research is such that we didn't actually go out and actually try to create a very well-calibrated model... or a model that was necessarily going to be able to do some kind of adaptive computation time." - E-V explains that their focus was on building a principled architecture, from which useful properties emerged.
• At 52:01 - "can construct path-dependent understanding... because it's completely different to just understanding what the thing is. It's how you got there is very important." - Tim emphasizes the importance of understanding the reasoning process, not just the final answer.
• At 64:23 - "if you wanted AGI, you wouldn't want all the text that humans have ever created. You would actually want the thought traces in their heads as they were creating the text." - E-V suggests that the path to AGI lies in training models on the process of human reasoning, not just the outputs.

Takeaways

• Seek out "blue ocean" research areas; significant progress in AI may come from exploring fundamentally different architectures rather than making incremental improvements in crowded fields.
• Recognize that brute-force scaling is not a panacea; it can mask a model's underlying lack of structural understanding rather than truly solving it.
• Use biology as a source of inspiration for novel computational principles, such as temporal dynamics and synchronization, to build more efficient and capable AI systems.
• Aim to build models that internalize reasoning processes, which could lead to more robust and sample-efficient problem-solving than relying on external techniques like chain-of-thought prompting.
• Focus on creating principled architectures from which desirable behaviors like good calibration and adaptive compute emerge naturally, as this indicates a more robust and sound design.
• Shift evaluation beyond final-answer accuracy to include the coherence of the reasoning process, as understanding how a model arrives at a solution is critical for building true intelligence.
• Consider that the next frontier in AI training data may involve capturing the dynamic "thought traces" of experts, providing a richer learning signal than static text alone.