The open-source agent series focuses on introducing currently available open-source agent frameworks in the market, such as CrewAI, AutoGen, LangChain, phidata, Swarm, etc., discussing their advantages, disadvantages, features, effects, and usage. Interested friends can follow the public account “XiaozhiAGI” for continuous updates on cutting-edge AI technologies and products, such as RAG, Agent, Agentic workflow, AGI. … Read more
Click belowCard, follow “AICV and Frontier“ Paper: https://arxiv.org/abs/2412.09262 Code: https://github.com/DAMO-NLP-SG/VideoLLaMA3 01 Introduction A more advanced multimodal foundation model for image and video understanding. The core design philosophy of VideoLLaMA3 is vision-centric: Vision-centric training paradigm Vision-centric framework design. The key point of the vision-centric training paradigm is that high-quality image-text data is crucial for understanding both … Read more
Hello everyone, I’m Kate. Do you remember the English version of the AI creation app I shared yesterday? A user left a message asking if there is a Chinese voice version. Now, it’s finally here! And this time, it’s still open source! In this video, I will take you on a deep dive into the … Read more
Machine Heart reports Editor: Zenan Low-cost devices can run locally. As large models continue to evolve towards larger scales, recent developments have also been made in optimization and deployment. On February 1, Wall Intelligence, in collaboration with Tsinghua NLP Laboratory, officially launched its flagship edge large model “Wall MiniCPM” in Beijing. The new generation large … Read more
mPLUG Team Contribution QbitAI | WeChat Official Account New SOTA in Multimodal Document Understanding! Alibaba’s mPLUG team has released the latest open-source work mPLUG-DocOwl 1.5, proposing a series of solutions to tackle four major challenges: high-resolution image text recognition, general document structure understanding, instruction following, and external knowledge incorporation. Without further ado, let’s take a … Read more
Machine Heart Column Machine Heart Editorial Team In the past few months, with the successive releases of major works like GPT-4V, DALL-E 3, and Gemini, “the next step for AGI”—multimodal generative large models have rapidly become the focus of scholars worldwide. Imagine, AI not only chats but also has “eyes” that can understand images, and … Read more
Multimodal fusion aims to integrate information from various modalities to achieve more accurate predictions. Significant progress has been made in multimodal fusion across a wide range of scenarios including autonomous driving and medical diagnosis. However, the reliability of multimodal fusion in low-quality data environments remains largely unexplored. This paper reviews the common challenges and recent … Read more
Author: Wang Yonggang, Founder/CEO of SeedV Lab, Executive Dean of AI Academy at Innovation Works The advent of ChatGPT/GPT-4 has completely transformed the research landscape in the NLP field and ignited the first spark towards AGI with its multimodal potential. Thus, the era of AI 2.0 has arrived. But where will the technological train of … Read more
Introduction This article shares the research achievements of Ant Group’s multimodal cognitive team in the field of video multimodal retrieval over the past year. The article focuses on how to improve the effectiveness of video-text semantic retrieval and how to efficiently perform video-source retrieval. Main Sections Include: 1. Overview 2. Video-Text Semantic Retrieval 3. Video-Video … Read more
GPT Source: Machine Heart ChatGPT has become the manager of hundreds of models. In recent months, the successive popularity of ChatGPT and GPT-4 has showcased the extraordinary capabilities of large language models (LLMs) in language understanding, generation, interaction, and reasoning, attracting significant attention from both academia and industry, and revealing the potential of LLMs in … Read more