Large Language Models

New Opportunities for Intelligent Film Production: A Review of Multimodal Technology Development at CVPR 2024

by
New Opportunities for Intelligent Film Production: A Review of Multimodal Technology Development at CVPR 2024

This article was published in the “Modern Film Technology” 2024, Issue 7. Expert Commentary Film is an organic combination of visual and auditory arts, presenting an unparalleled audiovisual experience to the audience through the transmission of sight and sound. Multimodal technology synergistically utilizes visual, auditory, textual, and other information to accomplish tasks that are difficult … Read more

Revolutionizing Language Models: The New TTT Architecture Surpasses Transformer

by
Revolutionizing Language Models: The New TTT Architecture Surpasses Transformer

Source: Machine Heart This article is approximately 3200 words long and is recommended for a 5-minute read. This article introduces a brand new large language model (LLM) architecture that is expected to replace the Transformer, which has been dominant in the AI field until now. From 125M to 1.3B large models, performance has improved. Incredible, … Read more

Four Development Directions for Large Models

by

Recently, Zhang Bo, an academician of the Chinese Academy of Sciences and honorary dean of the Institute of Artificial Intelligence at Tsinghua University, stated during his speech at the ISC.AI 2024 12th Internet Security Conference that current artificial intelligence lacks a theoretical foundation and only has developed models and algorithms targeting specific fields. Both software … Read more

Top-Notch: Research Progress of Latest Pre-trained Models from XLNet’s Multi-stream Mechanism

by
Top-Notch: Research Progress of Latest Pre-trained Models from XLNet's Multi-stream Mechanism

Follow the public account “ML_NLP“ Set as “Starred“, heavy content delivered first! Written by | Lao Tao (Researcher from a certain company, hereditary parameter tuning) Translated by | Beautiful person with meticulous thoughts Introduction As the hottest topic in NLP over the past two years, the language pre-training technologies represented by ELMo/BERT are already familiar … Read more

Lightning Attention-2: A New Generation Attention Mechanism

by
Lightning Attention-2: A New Generation Attention Mechanism

Reprinted from: Machine Heart Lightning Attention-2 is a new type of linear attention mechanism that aligns the training and inference costs of long sequences with those of a 1K sequence length. The limitation of sequence length in large language models greatly restricts their applications in the field of artificial intelligence, such as multi-turn dialogue, long … Read more

Adversarial Self-Attention Mechanism for Language Models

by
Adversarial Self-Attention Mechanism for Language Models

Delivering NLP technical insights to you every day! © Author | Zeng Weihao Institution | Beijing University of Posts and Telecommunications Research Direction | Dialogue Summarization Typesetting | PaperWeekly Paper Title: Adversarial Self-Attention For Language Understanding Paper Source: ICLR 2022 Paper Link: https://arxiv.org/pdf/2206.12608.pdf Introduction This paper proposes the Adversarial Self-Attention mechanism (ASA), which reconstructs the … Read more

Lightning Attention-2: Unlimited Sequence Length, Constant Computational Cost, Higher Modeling Accuracy

by
Lightning Attention-2: Unlimited Sequence Length, Constant Computational Cost, Higher Modeling Accuracy

Lightning Attention-2 is a new type of linear attention mechanism that aligns the training and inference costs of long sequences with those of a 1K sequence length. The limitations on sequence length in large language models greatly restrict their applications in the field of artificial intelligence, such as multi-turn dialogue, long text understanding, and the … Read more

Introducing HyperAttention: A New Approximate Attention Mechanism

by
Introducing HyperAttention: A New Approximate Attention Mechanism

Original Source: Machine Heart Edited by: Big Plate Chicken This article introduces a new research on an approximate attention mechanism, HyperAttention, proposed by institutions such as Yale University and Google Research, which accelerates inference time for ChatGLM2 with a context length of 32k by 50%. Transformers have been successfully applied to various learning tasks in … Read more

Possibilities of Artificial Intelligence Development in 2030: Cross-Dimensional Transformation

by
Possibilities of Artificial Intelligence Development in 2030: Cross-Dimensional Transformation

In 2030, will artificial intelligence drive economic growth, create breakthrough medical therapies, and simplify daily life as the data suggests? Or are these predictions overly optimistic: will artificial intelligence gradually fade away, or even make the world worse? Will artificial intelligence replace millions of jobs, replace human relationships, and challenge society with falsehoods? Recently, The … Read more

What Is Auditory? Machine Hearing?

by
What Is Auditory? Machine Hearing?

Auditory Sound waves act on the auditory organs, causing sensory cells to become excited and triggering impulses in the auditory nerve that transmit information to the brain. After analysis by various levels of the auditory centers, this results in the sensation of hearing. External sound waves are transmitted through a medium to the outer ear … Read more