Edge Machine Learning

Amazon: We Extracted an Optimal BERT Subarchitecture, 16% of Bert-Large, 7x CPU Inference Speedup

by
Amazon: We Extracted an Optimal BERT Subarchitecture, 16% of Bert-Large, 7x CPU Inference Speedup

Selected from arXiv Authors: Adrian de Wynter, Daniel J. Perry Translated by Machine Heart Machine Heart Editorial Team Extracting BERT subarchitectures is a highly worthwhile topic, but existing research has shortcomings in subarchitecture accuracy and selection. Recently, researchers from the Amazon Alexa team refined the process of extracting BERT subarchitectures and extracted an optimal subarchitecture … Read more

BERT Implementation in PyTorch: A Comprehensive Guide

by
BERT Implementation in PyTorch: A Comprehensive Guide

Selected from GitHub Author: Junseong Kim Translated by Machine Heart Contributors: Lu Xue, Zhang Qian Recently, Google AI published an NLP paper introducing a new language representation model, BERT, which is considered the strongest pre-trained NLP model, setting new state-of-the-art performance records on 11 NLP tasks. Today, Machine Heart discovered a PyTorch implementation of BERT … Read more

Code Qwen AI Challenge – Algorithm Track Overview

by
Code Qwen AI Challenge - Algorithm Track Overview

Introduction Competition Link: Code Qwen AI Challenge – Algorithm Track_Algorithm Competition_Questions and Data_Tianchi Competition – Data and Questions from Alibaba Cloud Tianchi Code is one of the high-quality languages created by humans, replacing the diverse natural language through high abstraction, ultimately converting to specific programs to complete tasks for humans. It possesses advantages such as … Read more

Analysis of Qwen2.5 Coder Training Process and Data Distribution

by
Analysis of Qwen2.5 Coder Training Process and Data Distribution

I have read some papers and training data on Qwen2.5 Coder and summarized them. Paper link: https://arxiv.org/pdf/2409.12186 1. Introduction The Qwen2.5-Coder series is a major upgrade from its predecessor CodeQwen1.5, aimed at achieving top-notch code task performance across various model sizes. This series includes six models: Qwen2.5-Coder-0.5B Qwen2.5-Coder-1.5B Qwen2.5-Coder-3B Qwen2.5-Coder-7B Qwen2.5-Coder-14B Qwen2.5-Coder-32B The architecture of … Read more

Understanding Qwen1.5 MoE: Efficient Intelligence of Sparse Large Models

by
Understanding Qwen1.5 MoE: Efficient Intelligence of Sparse Large Models

Introduction Official Documentation: Qwen1.5-MoE: Achieving the Performance of 7B Models with 1/3 Activation Parameters | Qwen On March 28, Alibaba announced the open-source MoE technology large model Qwen1.5-MoE-A2.7B for the first time. This model is based on the existing Qwen-1.8B model. The activation parameters of Qwen1.5-MoE-A2.7B are 270 million, but it can achieve the performance … Read more

In-Depth Study of Qwen 2.5 Paper

by
In-Depth Study of Qwen 2.5 Paper

Introduction I must say, Qwen is really impressive. It seems that its foundational capabilities have firmly established it as the leader in open source, and it is not at all inferior compared to most closed sources. Many companies’ foundational teams are likely already being judged on the significance of foundational models. Qwen’s open-source momentum is … Read more

Qwen Technical Report Details Sharing

by
Qwen Technical Report Details Sharing

Introduction Alibaba open-sourced the Qwen-7B model a long time ago, but for some reason, it was taken down. Just yesterday, Alibaba re-open-sourced the Qwen-14B model (the original 7B model was also released), and simultaneously released the technical report on Qwen. Today, I would like to share this with everyone. PS: Now domestic open-source large models … Read more

Interpretation of Qwen2.5 Technical Report

by
Interpretation of Qwen2.5 Technical Report

MLNLP community is a well-known machine learning and natural language processing community both domestically and internationally, covering NLP master’s and doctoral students, university professors, and corporate researchers. The vision of the community is to promote communication and progress between the academic and industrial circles of natural language processing and machine learning, especially for the advancement … Read more

Running GGUF Models with Ollama

by
Running GGUF Models with Ollama

Ollama directly supports many models by default, and you can simply use the ollama run command as shown below: ollama run gemma:2b This allows you to install, start, and use the corresponding model. You can find the models that are directly supported in this way at https://ollama.com/library. There are tens of thousands of models available … Read more