Three months ago, OpenAI launched its latest work – OpenAI O1, which quickly became a hot topic in the AI community. The biggest highlight of OpenAI O1 is its ability to reason like humans, meaning it can not only simply match known data patterns when dealing with complex problems, but also execute a process similar to “slow thinking” within the model. This capability allows O1 to perform well in various academic fields such as physics, mathematics, biology, and chemistry. Due to the closed nature of OpenAI’s technology, we do not know the implementation mechanism of O1. Currently, many studies are exploring how OpenAI O1 achieves this breakthrough. This sharing will focus on this theme, led by Zhang Junlin, Chief Scientist of Sina Weibo, to explore the technical principles and possible implementation mechanisms of O1.
The Large Model 2.0 Reading ClubAims to delve into the core technologies and basic principles behind the new paradigm of large model reasoning. The third session of the Large Model 2.0 Reading Club will focus on this theme, led by Zhang Junlin, Chief Scientist of Sina Weibo, to explore the technical principles and possible implementation mechanisms of O1.
Overview of the Sharing Content
This sharing will deeply analyze the technical principles and possible implementation mechanisms of O1. First, it will explain the significant value and meaning of O1, including the migration of Scaling Law from pre-training to Post-Training and Inference phases and its impact on the development of small models; secondly, it will gradually derive the possible training process and model composition of O1; additionally, it will focus on how to combine LLM and AlphaZero’s reinforcement learning to construct a system similar to O1, enabling large models to possess complex logical thinking abilities. Finally, a roundtable discussion will be held.
Outline of the Sharing Content
-
The value and significance of the O1 model
-
Inference time Scaling Law
-
Significance for small models
-
-
Derivation of the possible training process of O1
-
Possible training process of O1
-
Possible model structure of O1
-
How to combine LLM and reinforcement learning to construct O1
-
Introduction to the basics of reinforcement learning
-
Combining LLM and RL to construct a system similar to O1
-
Roundtable Forum (Zhang Junlin, Zhang Jiang, Yue Yutao, Wang Weixun, 30min)
-
How to view Ilya’s statement at NIPS2024: “Pre-training is coming to an end, and the next is superintelligence”?
-
Does Post-Training and Inference Time Scaling Law have sustainable development?
-
Is COT the best way to guide large models for “slow thinking”? Are there any other better methods?
-
Can LLM really do reasoning? Will LLM’s reasoning be different from humans?
-
Will forcing LLM to reason in a human-like way improve interpretability, but constrain performance?
-
…….
Zhang Junlin, Director of the Chinese Society of Chinese Information, PhD from the Institute of Software, Chinese Academy of Sciences, currently serves as Chief Scientist and Head of New Technology R&D at Sina Weibo. Previously, he was a Senior Technical Expert at Alibaba, leading a new technology team. He is the author of technical books such as “This is Search Engine: Core Technology Explained” and “Big Data Daily Record: Architecture and Algorithms.” He has published several related academic papers at international conferences such as ACL, KDD, CIKM, RecSys, and COLING.
His research interests include large models and recommendation systems.
Zhang Jiang, Professor at the School of Systems Science, Beijing Normal University, founder of the Intelligence Club, Intelligence Academy, and Intelligence Research Center. Formerly a special consultant at Tencent Research Institute and Huawei Strategic Research Institute.
His research interests include causal emergence, complex system analysis and modeling, and scale theory.
Yue Yutao, Associate Professor at Hong Kong University of Science and Technology (Guangzhou), founder of the Deep Perception Technology Research Institute, Jiangsu Province Industrial Technology Research Institute. Leader of the innovation research team introduced in Guangdong Province. He has over 300 authorized Chinese invention patents and has published over 60 papers.
His research interests include multimodal perception fusion, mechanisms and interpretable AI of large models, general artificial intelligence, machine consciousness, and applications of large models.
Wang Weixun, Reinforcement Learning Researcher at Alibaba, focuses on exploring and applying cutting-edge technologies in reinforcement learning. Currently engaged in work related to RL for LLM.His research interests include multi-agent systems, deep reinforcement learning, and human feedback-based reinforcement learning.
Personal homepage: http://wwxfromtju.github.io/
1. O1 official information: https://openai.com/o1/
2. OpenAI O1 system card: https://cdn.openai.com/o1-system-card-20241205.pdf
3. AlphaZero paper: Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm
4. Exploration of O1’s Inference-time Scaling (From Hugh Zhang on X: https://x.com/hughbzhang/status/1838288923656941860)
Time:
December 21, 2024 (This Saturday) 19:00-21:00
Participation method:
Scan the code to participate in the Large Model 2.0 Reading Club, join the group chat, gain access to the series of reading club replays, become a seed user of the AI community, and communicate with frontline researchers and practitioners to jointly promote the development of the AI community.
Sign up to become a speaker
Members of the reading club can apply to become speakers during the reading club. Speakers, as members of the reading club, adhere to the content co-creation and sharing mechanism, can receive a refund of the registration fee, and share all content resources generated by this reading club. For details, see: Large Model 2.0 Reading Club: A New Paradigm of Fusion Learning and Reasoning for Large Models!
Launch of the Large Model 2.0 Reading Club
The O1 model represents a new paradigm of fusion learning and reasoning for large language models. The Intelligence Club, together with Professor Zhang Jiang from the School of Systems Science, Beijing Normal University, Research Scientist Feng Xidong from Google DeepMind, Reinforcement Learning Researcher Wang Weixun from Alibaba, and Zhang Jie from the Institute of Computing Technology, Chinese Academy of Sciences, jointly initiated the “Large Model II: A New Paradigm of Fusion Learning and Reasoning for Large Models” reading club. This reading club will focus on the evolution of large model reasoning paradigms, optimization of reasoning based on search and Monte Carlo trees, optimization of large models based on reinforcement learning, methods and internalization mechanisms of thinking chains, self-improvement and reasoning verification. It aims to explore the technical path for the specific implementation of O1, helping us better understand the essence of machine reasoning and artificial intelligence.
Starting from December 7, 2024, it is expected to be held once a week on Saturdays, lasting approximately 6-8 weeks. Friends who are interested are welcome to sign up to participate and spark more ideas!
For details, see: Large Model 2.0 Reading Club: A New Paradigm of Fusion Learning and Reasoning for Large Models!
Click “Read the original text” to sign up for the reading club.
