Lightweight Adaptation Techniques for Multimodal Pre-trained Models

Lightweight Adaptation Techniques for Multimodal Pre-trained Models

This article is approximately 4200 words long, and it is recommended to read it in 8 minutes This article introduces the exploration and sharing of lightweight adaptation techniques for multimodal pre-trained models. Pre-trained language models such as BERT and GPT-3 have been proven to achieve excellent results in the NLP field. With the gradual maturity … Read more

Optimizing Functions and Complete Loss Function Calculation of GANs

Optimizing Functions and Complete Loss Function Calculation of GANs

Click the "Xiaobai Learns Vision" above, select "Add to Favorites" or "Pin" Heavyweight content delivered first time Introduction This article explains in detail how the minimax game and total loss function in GAN optimization functions are derived. It will introduce the meaning and reasoning of the optimization function in the original GAN, as well as … Read more

Uncertainty Planning Method of Zero-Carbon Energy System Based on LSTM Quantile Regression

Uncertainty Planning Method of Zero-Carbon Energy System Based on LSTM Quantile Regression

Cite this article REN Hongbo, WU Qiong, WANG Xiangyu, et al. Uncertainty Plan of a Zero-carbon Energy System Based on LSTM Quantile Regression[J]. Journal of Shanghai University of Electric Power, 2023, 39(2):149-157. Abstract The zero-carbon energy system is an effective way to promote the construction of a clean and low-carbon energy system and achieve carbon … Read more

Performance Optimization of Convolutional Neural Networks

Performance Optimization of Convolutional Neural Networks

Click on the above “Beginner Learning Vision” to choose to add Starred or “Top” Heavy content delivered at the first time Author丨Li Ming Huai Jin Source|https://zhuanlan.zhihu.com/p/80361782 Editor丨Extreme City Platform Extreme City Guide This article first defines the concept of Convolutional Neural Networks, briefly introduces several common optimization methods, and discusses the author’s experiences in this … Read more

Overview of Neural Network Optimization Algorithms

Overview of Neural Network Optimization Algorithms

Click the above “Little White Learns Vision“, choose to add “Starred” or “Top“ Heavyweight content delivered first-hand The optimization of machine learning (objective) can be simply described as: searching for a set of parameters w for the model that can significantly reduce the cost function J(w). This cost function usually includes performance evaluation over the … Read more

Doubling the Efficiency of Large Language Models: A Comprehensive Optimization Guide

Doubling the Efficiency of Large Language Models: A Comprehensive Optimization Guide

Author: Sienna Reviewed by: Los Abstract: Large Language Models (LLMs) have demonstrated exceptional capabilities in numerous language processing tasks; however, the computational intensity and memory consumption required for their deployment have become significant challenges to improving service efficiency. Industry estimates suggest that the processing cost of a single LLM request can be as much as … Read more

Doctoral Review: High-Efficiency Attention Model Architecture Design

Doctoral Review: High-Efficiency Attention Model Architecture Design

Doctoral Innovation Forum Issue Seventy On the morning of March 1, 2024, the seventieth issue of the Doctoral Innovation Forum was held online.PhD student Qin Yubin from Tsinghua University’s School of Integrated Circuits presented an academic report titled “High-Efficiency Attention Model Architecture Design”. The report focuses on the attention-based Transformer model, discussing optimization methods for … Read more

4 Basic Strategies for Optimizing RAG Process

4 Basic Strategies for Optimizing RAG Process

Author: Deephub Imba This article is about 3000 words long, and it is recommended to read it in 7 minutes. This article will introduce four strategies for optimizing Retrieval-Augmented Generation (RAG) using private data. In this article, we will introduce four strategies for optimizing Retrieval-Augmented Generation (RAG) using private data, which can enhance the quality … Read more

Advanced RAG: Enhancing RAG Performance

Advanced RAG: Enhancing RAG Performance

Author: Luv Bansal Translation: wwl Proofreading: Zhang Yiran This article is approximately 4400 words long and is recommended for a reading time of over 10 minutes. This article discusses various techniques for optimizing different parts of the RAG pipeline and enhancing the overall RAG workflow. Image generated by the author using Dalle-3 provided by Bing … Read more

Guide to Optimizing Transformer Memory Usage

Guide to Optimizing Transformer Memory Usage

MLNLP community is a well-known machine learning and natural language processing community in China and abroad, covering NLP master’s and PhD students, university teachers, and corporate researchers. The vision of the community is to promote communication and progress between the academic and industrial fields of natural language processing and machine learning, especially for beginners. Reprinted … Read more