RWKV-7-2.9B Model: Mastering Global Languages

Wisemodel.cn Open Source Community

The Wisemodel.cn community is a neutral and open AI open-source community originating from China. We are currently recruiting | New round of open-source co-creation volunteer program, welcome to join us in growing together. The Wisemodel community computing power platform is online, resources like H800/H20 are available at affordable prices, flexible and convenient, supporting online fine-tuning training models, as well as online model experience and exclusive API services, and fully supports ollama online operation.

RWKV-7-2.9B Model: Mastering Global Languages

On February 11, 2025, the RWKV Foundation officially released the RWKV-7-World-2.9B model (hereinafter referred to as RWKV-7-2.9B).

The RWKV-7-2.9B model is trained based on the RWKV World V3 dataset. In both model evaluation and practical experience, RWKV-7-2.9B has surpassed the previous generation RWKV-6-7B model. This model is now online in the Wisemodel AI community, and you are welcome to download and use it.

RWKV-7-2.9B Model: Mastering Global Languages

Model Address

https://wisemodel.cn/models/rwkv4fun/rwkv-7-world/intro

01.

RWKV Performance Improvement

English and Multilingual Evaluation

The English and multilingual capabilities of the RWKV-7-2.9B model significantly outperform all models of the same size, including Llama 3.2 3B, Qwen2.5 3B and other well-known excellent open-source models.

RWKV-7-2.9B Model: Mastering Global Languages
RWKV-7-2.9B-benchmark

MMLU Test

In the multiple-choice format of the MMLU test, the RWKV-7-2.9B model scored 54.56%. In comparison, the previous version RWKV-6-World-3B-V2.1 model scored 32.38% on MMLU.

💡Tips

The performance improvement of the base model RWKV-7-2.9B was achieved entirely through conventional training, without optimization for any specific tests, and no annealing or post-training optimization strategies were employed.

02.

Generation Cases

Here are generation cases of RWKV-7-2.9B (run using RWKV Runner).

Code Tasks

RWKV-7-2.9B Model: Mastering Global Languages
RWKV-7-2.9B-v3-demo1
RWKV-7-2.9B Model: Mastering Global Languages
RWKV-7-2.9B-v3-demo2

Multilingual Tasks

RWKV-7-2.9B writing a leave letter in multiple languages:

RWKV-7-2.9B Model: Mastering Global Languages
RWKV-7-2.9B-v3-demo3

💡Tips

Below are the original text and translation in the image:

Dear Mr. [Name of the Person],
I would like to inform you that I am on my way to the Mars rocket and will be absent for a week starting tomorrow. I made this decision because I want to explore my life.
It has been a great pleasure to be taught by you and to learn so much. I will never forget this knowledge.
I hope we can see each other again soon and thank you for everything!
Best regards,
[Your Name]
尊敬的[先生姓名]先生:
我在此通知您我正在前往火星火箭的途中,我将从明天起缺席一周。做出这个决定是因为我想探索我的生活。
能够接受您的教导并学习很多东西是一种极大的享受。我将永远不会忘记这些知识。
希望我们能很快再次见面并为一切向您致谢!
此致
敬礼
[您的姓名]
--------------------------------------------------------------------------------------
Dear Mr. [Teacher's Name],
I would like to inform you that I am currently heading to the underwater diving ship and will be absent for one day a week. I made this decision because I want to discover the world.
It has been great learning from you and receiving a lot of information. I will always keep this knowledge in my memory.
I hope we meet again and thank you for everything!
Respectfully,
[The Applicant's Name]
尊敬的[老师姓名]先生:
我谨此告知您:我将开始参与深海潜水艇的作业项目(每周将固定缺席一日)。作出这个决定是因为我想借此机会探索未知的世界。
能跟随您学习并收获丰富的知识是我的荣幸,这些宝贵的教导我将永远铭记于心。
期待未来能有重逢之日!衷心感谢您给予的一切!
此致 敬礼
[申请人姓名]

Role Playing

RWKV-7-2.9B performs “Bajie” role-playing, without adding any role-playing prompts or role presets.

RWKV-7-2.9B Model: Mastering Global Languages
RWKV-7-2.9B-v3-demo4

Novel Continuation

RWKV-7-2.9B continues writing a novel (highlighted section is the previous text generated by deepseek-R1):

RWKV-7-2.9B Model: Mastering Global Languages

RWKV-7-2.9B-v3-demo5

03.

Future Plans

The powerful capabilities of the RWKV-7-2.9B model are due to the ingenious improvements in the RWKV-7 architecture. After applying the “dynamic state evolution mechanism”, RWKV-7 has strong in-context-learning capabilities, better learning the relationships of contexts during reasoning, generating content that is more concise and reasonable. RWKV-7-7B is expected to be trained using the new RWKV World V3.1 dataset. The World V3.1 dataset will add a large amount of mathematical, code, and reasoning data based on World V3, further enhancing the model’s code, mathematical, and reasoning abilities.

—– END —–

RWKV-7-2.9B Model: Mastering Global Languages

Related to Wisemodel:1. The Wisemodel AI community is officially launched, aiming to create the Chinese version of “HuggingFace”. 2. The Wisemodel.cn community strives to become the most active AI open-source community in China. 3. Recruiting | New round of open-source co-creation volunteer program, welcome to join us in growing together.

System Upgrades:

4, Upgrade | Wisemodel computing power platform is online, models and computing power are within reach.

5. Upgrade | Open-source large model API services and hosting functions are online, everyone can create their own exclusive API services.

6. Upgrade | Wisemodel edge model area is online, welcome to join the construction of the edge model ecosystem.

Series Models:

7. o1+RAG solves knowledge deficiency, Search-o1 opens new methods for AI inference.

8. The first video “classroom” evaluation benchmark: How does Video-MMMU define AI’s knowledge absorption ability?

9. AI aids mathematical formalization: Natural language generates Lean4 code with one click, greatly lowering the threshold!

10. Tackling challenges! iVideoGPT develops an interactive visual world model through video generation.

11. The Ziyue-o1 inference model is released! The first output step-by-step explanation, can be deployed with consumer-grade graphics cards.

12. Essential for all-modal alignment! Data training evaluation, Peking University align-anything takes care of everything.

RWKV-7-2.9B Model: Mastering Global Languages

More about Wisemodel

RWKV-7-2.9B Model: Mastering Global Languages

1

Welcome to continue to follow and support

RWKV-7-2.9B Model: Mastering Global Languages

Building an open-source community requires long-term persistence and investment, and more importantly, the active participation, contribution, and maintenance of users. We welcome everyone to join the Wisemodel open-source community volunteer program and open-source co-creation program. We look forward to more developers releasing open-source results, including models, datasets, and codes, to the wisemodel.cn community, to build a neutral and open AI open-source community ecosystem. You are welcome to scan the QR code to add Wisemodel WeChat, apply to join the Wisemodel community, and keep following the dynamics of the wisemodel.cn open-source community.

2

Welcome to join the Wisemodel open-source community

Since its launch in September 2023, the Wisemodel AI community has gradually become a neutral and open AI open-source community with increasing influence. To accelerate the company’s development, we are in long-term need of talents in technology, operations, etc. The technology side focuses on AI infrastructure, back-end development, familiar with K8S, model training, and inference technologies, as well as members familiar with developer ecosystem operations. Interested friends are welcome to join, you can add Wisemodel WeChat or send your resume to [email protected]

3

Welcome to submit quality content

We welcome submissions sharing excellent research results related to artificial intelligence. We encourage universities, laboratories, large enterprise research teams, individuals, etc., to share various high-quality content on the Wisemodel platform, which can be interpretations of the latest AI papers, introductions to the latest open-source results, or practices, applications, and summaries of AI technology. Submissions can be emailed to [email protected] or you can scan the code to add Wisemodel WeChat.

4

About the Wisemodel Open Source Community

The Wisemodel.cn open-source community was founded by Liu Daoquan, Deputy Secretary-General of the AI Big Data Special Committee of the Tsinghua Alumni Association, aiming to create and build a neutral and open AI open-source innovation community, and to become the most active AI open-source community besides “HuggingFace”, gathering major AI open-source models, datasets, and codes, etc. We welcome universities, research institutions, large internet companies, innovative startups, individual developers, as well as government departments, associations, alliances, foundations, and investment institutions, technology media, etc., to participate in building the AI open-source innovation ecosystem.

Swipe up to view

More

Leave a Comment