Implementing Intelligent Email Customer Service with Llama3, CrewAI, and Groq

1. Chatting

If there is one industry that feels the most threatened by the arrival of AI, I believe e-commerce customer service should be on the list. Currently, platforms like Taobao and JD.com primarily use AI customer service for replies, only resorting to human agents if customers are dissatisfied, thus achieving cost reduction and efficiency improvement.

This time, we will use Llama3 + CrewAI + Groq to implement a small intelligent email customer service, which is essentially an intelligent reply to emails. It should be noted that this article does not cover the operations of reading and sending emails through code; the focus is primarily on the AI aspect.

2. Technical Stack Analysis

This project mainly involves three core technologies: Llama3, CrewAI, and Groq.

Llama3 is an open-source large model recently released by Meta, which comes in two versions: Llama3-8B and Llama3-70B. For a detailed introduction, I have previously discussed it in my article titled Summary of Llama 3 AI Large Model, so I won’t elaborate further here.

CrewAI is an open-source agent framework developed based on LangChain, aimed at orchestrating and coordinating multiple autonomous AI agents for teamwork.

Groq is a startup that has emerged as a dark horse in the AI chip field. The reason Groq gained popularity is that they launched a new AI chip called LPU (Language Processing Unit), claiming to achieve “the strongest inference on the planet”—the inference speed of running large models on Groq is reported to be ten times faster than NVIDIA GPUs, while the cost is only one-tenth of theirs.

Whether Groq‘s costs can really be that low is uncertain, but its inference speed is indeed fast! My own tests show it can reach about 300 tokens per second, compared to ChatGPT-3.5 which generates only 40 tokens per second, making it a clear winner in speed!

The core process of this project is quite simple: it involves breaking down the email reply into three steps: email classification, information retrieval, and email summary reply, with each step handled by Llama3. Groq primarily serves as an API provider, and finally, CrewAI orchestrates these three steps.

3. Project Practice

3.1 Registering a Groq Account

To implement the intelligent customer service mentioned above, the first step is to register a Groq account. After registration, go to this page^[1] to apply for an API Key.

Note that this step requires magical assistance, and the API Key will only be displayed once, so remember to save it!

Currently, Groq offers a free version, although there are some limitations on usage, but it is sufficient for simple testing.

3.2 Creating an Agent

In CrewAI, an agent is a role we define that can perform tasks, make decisions, and communicate with other agents.

If you have used Coze or Tongyi Qianwen, this concept is similar to Coze‘s AI Bot and the intelligent agents in Tongyi Qianwen.

As mentioned earlier, I have broken down the email reply into three steps: email classification, information retrieval, and email summary reply, which correspond to three different agents.

Why break it down into three agents instead of having one agent handle everything?

In my opinion, the more detailed your steps or questions are, the higher the accuracy of the AI will be. If there’s only one agent, the AI has too much to handle, which may not yield a good answer.

3.2.1 Email Classification Agent

The purpose of classifying emails is to facilitate the AI in providing different responses based on different categories, which will be elaborated on later.

The parameters for the agent are as follows:

• role: The role played by the agent

• goal: The goal of the agent

• backstory: Background information for the agent

• llm: The large model used by the agent

• verbose: Debugging switch; if set to True, it shows detailed information during CrewAI execution

• allow_delegation: Whether to allow delegating tasks to other agents

• max_iter: The maximum number of iterations for the agent to perform the task

• memory: Whether the agent is allowed to have “memory”

• step_callback: Callback function

3.2.2 Information Retrieval Agent

This agent has an additional tool parameter, which specifies the tools that the agent can use.

In the email reply process, some content may involve web content retrieval, and here search_tool is essentially a web search tool that uses the search functionality of the DuckDuckGo browser.

3.2.3 Email Reply Agent

This highlights the benefit of splitting into multiple agents, allowing for different response methods based on different email classifications.

3.3 Creating a Task

In CrewAI, a task is the assignment of a task to the corresponding agent. A task encapsulates all the information needed for the agent to perform the task, which can be simply understood as the prompts we usually input (but more complete).

3.3.1 Email Classification Task

The parameters for the task are as follows:

• description: Description of the task

• expected_output: What output is expected from this task

• output_file: Output to a file

• agent: Which agent will complete the task

Here we emphasize that only one classification should be output to avoid the AI generating too much irrelevant content.

3.3.2 Information Retrieval Task

This task has an additional parameter: context, which refers to the context we commonly talk about. In this stage, the information retrieval task needs to search for specific content based on the previously obtained email classification.

3.3.3 Email Reply Task

3.4 Task Orchestration

After completing the above steps, we can achieve a task and agent orchestration through Crew, and the overall functionality is basically accomplished!

3.5 Actual Results

My email content is as follows:

email = """Hello,
I am writing this email to say that I had a wonderful time at your resort last week.
Thank you very much for everything your staff has done.
Thank you,
AI Tech Wizard
"""

The results after execution are as follows:

For the first task, the Llama3 model will classify the email into customer_feedback, which is still quite accurate overall.

For the second task, we can see that a retrieval was conducted, asking “how to respond to positive customer feedback.”

For the third task, the final reply is as follows:

Overall, the translation is as follows:

Dear AI Tech Wizard,

Thank you very much for taking the time to share your wonderful experience at our resort! We are delighted to hear that you had a great time with us and appreciate your praise for our staff. Your feedback is invaluable to us, and we are glad to have met your expectations.

We are always striving to provide the best experience for our guests, and your feedback will help us continue to improve. If you need any assistance or have any questions in the future, please feel free to contact us.

Thank you again for your feedback and for choosing our resort. We look forward to welcoming you again!

Best regards,
[Your Name] 

Overall, the effect is still acceptable!

3.6 Some Shortcomings

First, this is just a basic example, solely aimed at helping everyone understand how to achieve intelligent replies through AI. It is not a product that can be practically deployed.

Second, it can be observed that the replies are in English because Llama3 is primarily trained on English corpora. If a Chinese reply is needed, a different large model would have to be used.

4. Conclusion

This time, we achieved a small intelligent email customer service using Llama3 + CrewAI + Groq. From the content of the replies, the overall effect is still satisfactory. I hope this provides some inspiration for friends interested in this area.

For the source code of this case, reply “Intelligent Customer Service” in the public account to obtain the complete source code. Lastly, if this article has been helpful to you, I hope for your support through likes!

References