The Bitter-Er Lesson: Search and AI #
The author of the linked Notion article argues that the effectiveness of search goes hand-in-hand with the quality of the value function. While value functions in chess are well-defined and highly effective, LLMs struggle with this concept.
- Value functions are domain-specific: The author highlights the lack of generalized value functions for LLMs, which restricts their ability to effectively search for solutions in diverse domains.
- Search for AI Research: The author proposes that search could be used for AI research itself, enabling the AI to explore different approaches and iteratively improve. However, this is a bold claim with significant challenges.
Top Quotes
The effectiveness of search goes hand-in-hand with quality of the value function. But today, value functions are incredibly domain-specific, and there is weak or no current evidence (as far as I know) that we can make value functions that generalize well to new domains. -- mxwsn
The "search" process for your brain structure took 13 billion years and 20 orders of magnitude more computation than we will ever harness. -- skulk
The Search Space and LLMs #
The conversation highlights the challenges of implementing search in LLMs, mainly focused around the size and complexity of the search space.
- Branching factor: LLMs have a much larger branching factor compared to chess, making exhaustive search computationally expensive.
- Evaluating branches: Determining the value of a specific branch in an LLM search space is more complex than in chess, where the board evaluation is well-defined.
- Complexity of "AI Research": Unlike chess, the search space for AI research is not well-defined, making it challenging to design effective search algorithms.
The Impact of Search #
The potential benefits of search in LLMs are substantial, but the challenges are significant.
- Enhanced Reasoning: Search could enable LLMs to engage in more complex reasoning, allowing them to "think" about problems more strategically and systematically.
- Self-improvement: Search could be instrumental in creating self-improving AI systems.
- Alignment Issues: The potential for superintelligence through search raises serious concerns about AI alignment, the need for ethical development and control.
The Need for a New Approach #
Many commenters express reservations about the author's claims and encourage a more cautious approach to implementing search in LLMs.
- Human-in-the-Loop: The importance of human oversight and control is emphasized, recognizing the limitations of AI and the potential for errors.
- Real-world Applications: Focus on specific applications where search can be used effectively, such as theorem proving or protein folding.
- Understanding "Search": The definition of "search" in the context of LLMs needs clarity and further elaboration.
Conclusion #
The discussion highlights the potential and challenges of implementing search in LLMs. While promising, it's crucial to approach this concept with caution and focus on addressing the significant technical and ethical questions involved.