Fine-tuning

The Importance of Refocusing Attention in Fine-Tuning Large Models

by
The Importance of Refocusing Attention in Fine-Tuning Large Models

Click the "Xiaobai Learns Vision" above, select to add "star" or "top" Heavyweight content delivered to you first Author丨Baifeng@Zhihu (Authorized) Source丨https://zhuanlan.zhihu.com/p/632301499 Editor丨Jishi Platform Jishi Guide Surpassing fine-tuning, LoRA, VPT, etc. with only a small number of parameters fine-tuned! Paper link: https://arxiv.org/pdf/2305.15542 GitHub link: https://github.com/bfshi/TOAST We found that when fine-tuning large models on a downstream task, … Read more

Using GPT-4 to Generate Training Data for Fine-tuning GPT-3.5 RAG Pipeline

by
Using GPT-4 to Generate Training Data for Fine-tuning GPT-3.5 RAG Pipeline

Source: DeepHub IMBA This article is about 3200 words long, and it is recommended to read for 6 minutes. This article explores the new integration of LlamaIndex for fine-tuning OpenAI's GPT-3.5 Turbo. OpenAI announced on August 22, 2023, that fine-tuning of GPT-3.5 Turbo is now possible. This means we can customize our own models. Subsequently, … Read more

Understanding BERT: Principles, Code, Models, and Fine-tuning Techniques

by
Understanding BERT: Principles, Code, Models, and Fine-tuning Techniques

In October 2018, the BERT model launched by Google made a stunning impact, sweeping various rankings and even surpassing human baseline scores, achieving a milestone breakthrough in the field of NLP. Today, for NLP algorithm engineers, BERT has become an essential tool. “What if there’s too little data?” — “Just fine-tune BERT!” “What if RNN … Read more

Innovations in the Era of BERT: Comparison of BERT Application Models and More

by
Innovations in the Era of BERT: Comparison of BERT Application Models and More

Author: Dr. Zhang Junlin, Senior Algorithm Expert at Sina Weibo Zhihu Column:Notes on the Frontiers of Deep Learning This article has been authorized, you can click “Read the original” at the end of the article to go directly: https://zhuanlan.zhihu.com/p/65470719 In the past two months, I have been paying close attention to the current application status … Read more

When Bert Meets Keras: The Simplest Way to Use Bert

by
When Bert Meets Keras: The Simplest Way to Use Bert

Author: Su Jianlin Research Direction: NLP, Neural Networks Personal Homepage: kexue.fm Bert is something that probably doesn’t need much introduction. Although I’m not a big fan of Bert, I must say it has indeed caused quite a stir in the NLP community. Nowadays, whether in Chinese or English, there is a plethora of popular science … Read more

Has Prompt Tuning Surpassed Fine Tuning?

by
Has Prompt Tuning Surpassed Fine Tuning?

MLNLP community is a well-known machine learning and natural language processing community both domestically and internationally, covering NLP graduate and doctoral students, university teachers, and corporate researchers. The community’s vision is to promote communication and progress between the academic and industrial sectors of natural language processing and machine learning, especially for beginners. Reprinted from | … Read more

Overview of Prompt Learning and Tuning

by
Overview of Prompt Learning and Tuning

↑ ClickBlue Text Follow the Jishi platform Author丨Jarvis73@Zhihu (Authorized) Source丨https://zhuanlan.zhihu.com/p/601905339 Editor丨Jishi Platform Jishi Guide This article summarizes Prompt Learning/Tuning. >> Join the Jishi CV technology exchange group to stay at the forefront of computer vision Since the advent of Self-Attention and Transformer, they have become the new stars in the field of natural language processing. … Read more

Local Deployment and Fine-Tuning Tutorial for Qwen 2.5 Model

by
Local Deployment and Fine-Tuning Tutorial for Qwen 2.5 Model

“ As a non-professional beginner, my initial interest in large models led me to explore related knowledge. As I read more papers and reports, I always wanted to practice with large models but didn’t know where to start. I believe many students share the same experience as I did back then. This article will guide … Read more

Llama 3.2 Model Fine-Tuning Guide

by
Llama 3.2 Model Fine-Tuning Guide

1. Basic Description 1. The models from ollama are all stripped-down versions. 2. They are relatively weak in distributed environments. 3. ollama is lightweight and suitable for individual users, but not for enterprise users. 4. ollama only performs model inference (used for running models). 5. ollama uses GGUF for quantization. 6. Engineering deployment generally does … Read more

Choosing Between RAG, Fine-Tuning, or RAG + Fine-Tuning

by
Choosing Between RAG, Fine-Tuning, or RAG + Fine-Tuning

1. RAG (Retrieval Augmented Generation) RAG technology is a method that combines retrieval and generation. It typically relies on two core components: a large language model (such as GPT-3) and a retrieval system (such as a vector database). RAG first uses the retrieval system to extract relevant information from a vast amount of data, then … Read more