AI Tools Evaluation for Accountants: DeepSeek, ChatGPT, and Doubao

As of January 2025, there are 302 generative AI services registered nationwide. For many finance professionals facing so many AI tools, it feels like a lavish banquet has been laid out, yet we don’t know how to “pick up the chopsticks”.

On January 20, 2025, the Chinese AI company DeepSeek released the DeepSeek-R1 with deep thinking capabilities, which sparked widespread discussion in the accounting community. So, how well do AI large models respond to accounting-related questions? Can they become effective assistants for accountants? Besides DeepSeek, what other AI tools are worth trying?

Based on the above background, we conducted this evaluation to select the currently most suitable free AI large models for finance and accounting personnel!

PART 1 Major Conclusions from the Evaluation

AI Tools Evaluation for Accountants: DeepSeek, ChatGPT, and Doubao

1. The top three AI tools based on evaluation scores are: DeepSeek, ChatGPT-4o, and Doubao.

2. In this test, 13 AI tools answered 27 questions. The results showed that the lowest score for the test questions was 0 points, the highest was 10 points, the median was no less than 6 points, and the average score was at least 5.6 points. This means that using any of these AI tools, users can generally obtain “acceptable” to “good” answer quality.

3. The quality of AI tool responses has improved rapidly overall.

PART 2 Testing Plan

[Evaluation Rules]

In this test, 20 volunteers raised 27 questions related to the accounting profession, including subjective and objective questions (2 multiple-choice questions each from the intermediate accountant and CPA exams, and 3 calculation questions adapted from CPA multiple-choice questions), and scored the responses of 13 AI tools.

[Evaluation Subjects]

The AI tools selected for this evaluation include ChatGPT-4o and 12 domestic products. The answers from the ChatGPT-4o version were provided by the European evaluator Alohaha2013. The criteria for selecting domestic products were that they should be free to use and have a high recommendation for AI question-and-answer tools. The evaluated products are displayed as follows:

[Scoring Criteria]

PART 3 Test Results Display

This public evaluation took place from February 6 to 11, 2025. The evaluation results indicated that the median scores for the 27 questions answered by the 13 AI tools ranged from 6 to 9 points, with average scores between 5.6 and 8.2 points. Although the highest score reached 10 points, some responses scored extremely low, even as low as 0 points. Overall, AI tools can generally provide “acceptable” or “good” answers, with some questions even yielding “excellent” answers, but there are also cases of extremely poor responses. Therefore, choosing the right AI tool is crucial for obtaining high-quality answers.

PART 4 Summary and Reflections

Regarding this test, this public account has the following summaries and reflections:

1. Based on the crowd test results, domestic users currently recommend DeepSeek for accounting-related inquiries, while Doubao and KimiChat are also good options. DeepSeek’s performance in 27 questions generally reached “good” or even “excellent” levels, providing at least “acceptable” answers; ChatGPT-4o and Doubao’s performance is close to “excellent”, but their quality lower limits are lower; KimiChat’s overall performance is “good”, but some questions scored only 1 point. Other AI tools also have certain potential and are worth trying.

2. AI tools have shown some practicality in accounting-related questions, but the quality of responses varies significantly, and there are even instances of “saying nonsense seriously”, so professional oversight is still needed, and AI should not be completely relied upon.

3. AI tools have significantly improved in answering CPA exam questions and may possess the ability to pass the exams. In the test questions, questions 23 and 24 were real questions from the intermediate accountant exam, and all 13 AI tools answered correctly; questions 25 and 26 were adapted from real CPA exam questions, with 3 and 4 AI tools answering incorrectly, respectively, while the rest were correct, resulting in an overall score rate of 73.54%. Compared to the test results six months ago (August 2024), when models like GPT-4 performed poorly on CPA exam questions, the AI’s performance has likely improved significantly now.

4. AI’s ability to understand and process accounting entry questions is continuously enhancing, showing significant improvement compared to our tests two years ago. Test question 27 required generating relevant accounting entries, and both DeepSeek and Doubao provided correct entries. Below is DeepSeek’s response:

We see that the financial capabilities of AI large models are advancing at an unimaginable speed, while we also feel the confusion and bewilderment of finance personnel in this trend.

Facing the impact of AI, merely feeling anxious will yield no results; utilizing it will bring gains. We hope to reduce information asymmetry through this evaluation, allowing the conveniences of AI large models to reach more finance professionals. We sincerely invite everyone to share and disseminate the results of this evaluation, benefiting more industry colleagues!

AI tools in the accounting field are not limited to answering questions; what experiences do you have in exploring this area? Feel free to leave a message and share.

Appendix

Links to the free domestic large models used in this test

DeepSeek https://chat.deepseek.com/

Yunque Large Model (Doubao) https://www.doubao.com/chat

KimiChat https://kimi.moonshot.cn/

Xunfei Xinghuo https://xinghuo.xfyun.cn/

Baichuan Intelligence https://www.baichuan-ai.com/home

Tian Gong https://www.tiangong.cn/

Tencent Hunyuan https://hunyuan.tencent.com/

Shangliang https://chat.sensetime.com/

Tongyi Qianwen https://tongyi.aliyun.com/qianwen/

Lingyi Wanwu https://platform.lingyiwanwu.com/

Zhipu Qingyan https://chatglm.cn/

Wenxin Yiyan https://yiyan.baidu.com/

Source: Financial Digital Transformation Exploration

Compiled by: Xie Chaoxi

Editor: Li Qian

Content Review: Wang Tao

Media Cooperation: 010-88379072

Disclaimer: Some materials used in this article are sourced from the internet. If there are any copyright issues, please contact us in a timely manner.

Scan the QR code

Financial Management Research

Leave a Comment Cancel reply