Click the above “Beginner Learning Vision”, select to add Star or Pin

Essential Insights Delivered First Hand

This article is reprinted from: Machine Heart

Agents are one of the most discussed buzzwords in the AI field recently. At the end of 2024, Google emphasized the arrival of the Agentic era with the release of Gemini 2.0. In early 2025, the Google team published a white paper on the topic of Agents, comprehensively introducing the basic concepts of Agents, their differences from LLMs, core architecture, working principles, and practical applications.

Additionally, the report analyzes the differences between Agents and traditional LLMs. It points out that Agents expand the capabilities of LLMs in knowledge acquisition, conversation management, tools, and logical layer implementation, while LLMs are limited to the training data and do not require these capabilities.

Table of Contents

01. What Is an Agent? How Does It Differ from Traditional LLMs?

What is an Agent? Where does it come from? What are the differences from traditional LLMs? What impact does it have on the continuous learning advocated by Richard Sutton?

02. How Does an Agent Work? Why Is the Orchestration Layer Key?

What tools does an Agent use to interact with the external environment? How can we effectively enhance the performance of an Agent?

03. How Does an Agent Interact with Tools and the Environment?

What tools does an Agent use to interact with the external environment?

04. What If the Interaction Environment Is Too Complex? How to Enhance Agent Performance?

How can we more effectively enhance the performance of an Agent?

01 What Is an Agent? How Does It Differ from Traditional LLMs?

The white paper details the concept of an Agent, its core architecture, and working principles. Additionally, the report analyzes the differences between Agents and traditional LLMs. It points out that Agents expand the capabilities of LLMs in knowledge acquisition, conversation management, tools, and logical layer implementation, while LLMs are limited to training data and do not require these capabilities.1. Humans often rely on tools to supplement knowledge and draw conclusions when handling complex tasks, similarly, generative AI models can be trained to use tools to obtain real-time information or perform actual actions. For example, a model can use a database retrieval tool to obtain customer purchase history to generate personalized shopping recommendations or make API calls based on user queries to send emails or complete financial transactions. This combination of reasoning, logic, and access to external information connected with generative AI models introduces the concept of Agents, which are programs that go beyond the independent capabilities of generative AI models.① Agents provide solutions to the limitations of generative AI models in practical applications, allowing models to better handle complex tasks through autonomous planning and execution, as well as interacting with external systems to obtain real-time information, greatly expanding the application range and capability boundaries of models, demonstrating potential immense value in various fields and driving AI technology towards more practical and intelligent directions.2. The working principle of an Agent can be summarized as a cognitive architecture that includes three core components: model, tools, and orchestration layer.① The model is the “brain” of the Agent, responsible for processing information, generating decisions, and producing outputs. It can be one or more language models with instruction-based reasoning and logical framework capabilities.② Tools are the “hands and feet” of the Agent, enabling it to obtain real-time information and perform actual operations.③ The orchestration layer is the “bridge” that connects the model and tools, responsible for managing the flow of information, decision-making processes, and the order of task execution.3. The functionality of an Agent is based on the capabilities of the model, expanding in areas such as knowledge acquisition, conversation management, tools, and logical layer implementation.① The knowledge of the model is limited to its training data, while the Agent connects with external systems through tools, allowing it to expand its knowledge base and obtain real-time information to better meet diverse task demands.② Models usually perform single-instance reasoning based on user queries, and unless specifically requested, they generally do not consider conversation history or continuous context, while Agents can consider conversation history and perform multi-round reasoning based on user queries and orchestration layer decisions, achieving more coherent and contextually relevant interactions.③ Models do not inherently implement tools and logical layers, and users need to construct specific prompts or use reasoning frameworks to guide model predictions; Agents, however, inherently implement tools and cognitive architectures, utilizing reasoning frameworks like CoT, ReAct, or other pre-built agent frameworks to enable more autonomous and intelligent task execution.02 How Does an Agent Work? Why Is the Orchestration Layer Key?In the white paper, the working principles of Agents are detailed, including how to collect, process information, and decide on the next actions. The report particularly mentions that the orchestration layer is the core of the Agent’s cognitive architecture……

Download 1: OpenCV-Contrib Extension Module Chinese Tutorial

Reply "Extension Module Chinese Tutorial" in the "Beginner Learning Vision" public account backend to download the first OpenCV extension module tutorial in Chinese, covering installation of extension modules, SFM algorithms, stereo vision, object tracking, biological vision, super-resolution processing, and more than twenty chapters of content.

Download 2: Python Vision Practical Projects 52 Lectures

Reply "Python Vision Practical Projects" in the "Beginner Learning Vision" public account backend to download 31 practical projects in visual computing, including image segmentation, mask detection, lane line detection, vehicle counting, adding eyeliner, license plate recognition, character recognition, emotion detection, text content extraction, face recognition, etc., to help quickly learn computer vision.

Download 3: OpenCV Practical Projects 20 Lectures

Reply "OpenCV Practical Projects 20 Lectures" in the "Beginner Learning Vision" public account backend to download 20 practical projects based on OpenCV for advanced learning.

Group Chat

Welcome to join the reader group of the public account to communicate with peers. Currently, there are WeChat groups for SLAM, 3D vision, sensors, autonomous driving, computational photography, detection, segmentation, recognition, medical imaging, GAN, algorithm competitions, etc. (will gradually be subdivided in the future). Please scan the WeChat ID below to join the group, and note: "Nickname + School/Company + Research Direction", for example: "Zhang San + Shanghai Jiao Tong University + Vision SLAM". Please follow the format, otherwise, it will not be approved. After successful addition, you will be invited to the relevant WeChat group based on your research direction. Please do not send advertisements in the group, otherwise, you will be removed. Thank you for your understanding~