Initial Trial of Jarvis PC Version – Zhipu CogAgent ‘Niu Niu’

Introduction

Everyone is probably very familiar with Iron Man’s intelligent assistant “Jarvis”. Recently, Zhipu released the PC version of “Jarvis” – the brave “Niu Niu” (official name: GLM-PC, nickname: Niu Niu, which is a cool name).

It is not a dialogue tool, not a coding assistant, and not RPA, but a tool based on visual effects for recognition, analysis, reasoning, and operation through a large model.

Just send commands like in a conversation, and “Niu Niu” will “observe”, “think”, and “operate” like a human, then provide results.

Overview

The following is a direct use of the official introduction for everyone to understand accurately.

GLM-PC is based on Zhipu’s multimodal large model CogAgent, the world’s first computer intelligent agent (agent) available to the public, plug-and-play. It can “observe” and “operate” a computer like a human, assisting users in efficiently completing various computer tasks.

Since the release of GLM-PC v1.0 on November 29, 2024, and the opening of internal testing, we have continuously optimized and upgraded it, recently launching the “Deep Thinking” mode and adding functionality specifically for logical reasoning and code generation. In addition, we also provide support for the Windows system.

• Official Website:https://cogagent.aminer.cn
• User Documentation:https://cogagent.aminer.cn/static/agreement/User_Manual.pdf
• Technical Documentation:https://cogagent.aminer.cn/blog#/articles/cogagent-9b-20241220-technical-report
• Demo Video:https://zhipu-ai.feishu.cn/docx/PVEdd0C6yoZJl5xevsRcupYtnvg

Application

Currently, it adopts an application system, and the process is very simple: fill out the application form, and batch approval will be granted. I applied on the 23rd, and it was activated by the morning of the 24th.

After opening the above official website link, it appears as follows.

Fill out the application form and join the user group to wait for notification.

Installation

Go to the top of the official website for “Internal Test Download”.

Open the download interface.

The installation process is very simple. If you are unsure, you can refer to the official documentation.

https://cogagent.aminer.cn/static/agreement/Installation_Guide.pdf

Try

Send Blessings via WeChat (Fast Mode)

Try sending New Year blessings to friends via WeChat.

Create a New Conversation

Select “Fast Mode” to create a new conversation.

Send Command

Give me an interesting Spring Festival blessing, not too long, to send to WeChat's "Dong Pengfei".

Execution Process

All the following steps are executed automatically by “Niu Niu”, with no human intervention.

1. Automatically call the model to generate a blessing.

Result

The last message is the final effect.

Personal Experience:

1. The overall process is quite good, and I feel that the following points are done quite well:

• Opening WeChat.
• Locating friends.
• The manual confirmation mode for sending confirmation.

• In “Deep Thinking” mode, the WeChat opening state recognition has errors. It indicates that WeChat is not opened when it actually is, and this is not just an isolated case on my end.
• The language model is not sensitive to the current time and cannot automatically infer that the blessing message should be for the Year of the Snake.

Search Content and Summarize (Deep Thinking Mode)

Try searching for specified content and summarizing it through the model.

Create a New Conversation

Select “Deep Thinking” to create a new conversation.

Send Command

Use Baidu to search for information on free highway passage during the Spring Festival 2025 and summarize it into a short paragraph.

Execution Process

Result

Personal Experience:

1. The overall user experience is good, and once matured, its application in searching and organizing materials is expected to significantly improve efficiency.
2. Points that need optimization:

• Sometimes it opens Baidu but looks for other tabs.
• Sometimes it enters search through the homepage and sometimes through the URL.

• Handling of search entry is not very precise:
• The search scenario process is not flexible enough; if the search engine gives aggregated results, the result detail page is actually a list page, and at this time, “Niu Niu” still recognizes it as a text page, leading to errors.

Conclusion

Overall, after the trial, I feel that the design thinking and implementation results have far exceeded expectations, after all, this should be the first case in the country.

In the future, I will share more complex scenarios with everyone. If you are interested, you can try it out as soon as possible.