In-Context Learning

Introduction to ChatGPT Principles and Concepts

by
Introduction to ChatGPT Principles and Concepts

This article introduces four technologies in the ChatGPT model: prompt tuning, instruct tuning, in-context learning, and chain-of-thoughts. By applying these technologies, ChatGPT can interact more intelligently with humans, not only performing natural language understanding and generation but also optimizing and training for specific tasks, thereby achieving a more intelligent and personalized conversational experience. Prompt Tuning … Read more

ACL 2022 Outstanding Paper: Bugs in Prompt Paradigms

by
ACL 2022 Outstanding Paper: Bugs in Prompt Paradigms

By | python The rise of large models like GPT-3 has brought about a new paradigm of in-context learning. In in-context learning, the model does not use gradient descent to adjust parameters based on supervised samples; instead, it connects the inputs and outputs of supervised samples as prompts, guiding the model to generate predictions based … Read more

Using Transformers as Universal Computers with In-Context Learning Algorithms

by
Using Transformers as Universal Computers with In-Context Learning Algorithms

Source: Machine Heart This article is about 4500 words long and is recommended to be read in 5 minutes. What can a 13-layer Transformer do? It can simulate a basic calculator, a basic linear algebra library, and execute an in-context learning algorithm using backpropagation. Transformers have become a popular choice for various machine learning tasks, … Read more