In an era dominated by large language models (LLMs), conversational flows have become the mainstream method for most interactive cloud communications. How can traditional frontend technologies integrate with these powerful models in light of this trend? Besides user interfaces (UI), in what other areas can frontend technology play a role?
At the QCon Global Software Development Conference on December 28-29, we invited Baidu’s senior engineer, Zhou Tingshuai, to demonstrate how to combine traditional interaction methods with conversational flows, including design strategies for context and state transitions. Apart from interaction design itself, they also discussed the PatternPlugin in specific scenarios, which involves applying state machine technology to rogue-like games and event design.
Before the formal presentation, we interviewed Zhou Tingshuai and asked him how to utilize large models for business monitoring and effect evaluation.
InfoQ: What specific challenges did you encounter in integrating AI with traditional frontend technologies, and how did you overcome them?
Zhou Tingshuai: On one hand, frontend technology has strong rendering and interaction capabilities. Under the current common “text conversation model,” we enhance user interaction experience by introducing UI capabilities into the plugin system. On the other hand, the instability of content generated by large models means instability in user experience. We improve quality in this regard through state management and automated monitoring.
InfoQ: Can you share an application case of GPTs in the Wenxin Yiyan App and how it impacts user experience?
Zhou Tingshuai: In the Wenxin Yiyan App, we call it the “Inspiration Center,” which was launched in April and opened to creators in October this year. The Inspiration Center quickly meets users’ demands for specific capabilities, like a well-prepared dish, saving users the learning cost of prompts, such as English speaking teachers and PPT outline generators, which are very popular among users.
InfoQ: What innovations does state machine technology bring to frontend applications? How does it enhance the effectiveness of interaction design?
Zhou Tingshuai: For example, in a text-based game that relies entirely on the context of large models, the game progress can easily be disrupted, and it is impossible to trigger external actions when reaching a certain state. The design of the state machine is intended to solve the problem of accurately holding short-term states. Once the state can be accurately recorded and processed, state-based logical actions can be combined with scenarios outside the large model.
InfoQ: In your experience, what are the best practices for combining conversational flows with traditional interaction methods?
Zhou Tingshuai: A good interaction behavior should meet the standards of efficiency and naturalness. From a technical perspective, supporting UI through plugins or structuring content returned by large models can effectively enhance conversational interaction experiences.
InfoQ: What is the design philosophy of PatternPlugin? How effective is it in real business scenarios?
Zhou Tingshuai: We define it as a pattern plugin aimed at integrating logical orchestration with large models for different scenarios to enhance user experience. It has been validated in cave adventure games and personality tests, showing significantly better results than purely large model-based experiences.
InfoQ: Can you describe some practical applications of AI in frontend technology and how they have changed the development process?
Zhou Tingshuai: On the coding level, the most direct experience is with code completion and single-side generation. Many team members are currently using Baidu’s Comate to improve development efficiency. In the feedback loop among users, developers, and creators, we have created a closed-loop process for input, output, and evaluation through large models.
InfoQ: What are the key elements for implementing performance evaluation and monitoring mechanisms in frontend technology?
Zhou Tingshuai: The core of evaluating and monitoring the quality of creator prompts lies in monitoring the design of prompts. Initially, we tried to set certain standards, but they lacked flexibility. Later, we dynamically generated metrics based on user-created prompts and scored the Q&A pairs based on these metrics, resulting in better evaluative capabilities.
InfoQ: What roles do state machines and memory systems play in business processes, and what specific impacts do they have on business?
Zhou Tingshuai: The design of the state machine addresses the memory issue of PatternPlugin. By refreshing specific state information before and after each interaction with the large model, we ensure the accuracy of key information. This mechanism enables the implementation of dialog systems with logical orchestration, such as cave games, character story branches, and specific event rewards.
InfoQ: As LLM model capabilities improve, how do you think this will affect the processes and architectures of frontend development?
Zhou Tingshuai: With the enhancement of LLM capabilities, large models should significantly assist in the initial version of a page, logical drafts, and data structure design. However, given the complexity of business logic, a lot of manpower is still needed to handle this aspect.
InfoQ: What do you think are the future development directions and new opportunities for frontend technology?
Zhou Tingshuai: On one hand, fully leveraging terminal advantages such as WebNN, GPU, and Wasm to build applications that were previously impossible on the web, like InpaintWeb. On the other hand, AI capabilities have never been as developer-friendly as they are today, not limited to frontend. For example, we use knowledge distillation to help products build characters and assist creators in their work through large model evaluations of prompts.
InfoQ: Recently, many people believe that our attitude towards large models is a bit too optimistic, worrying that we have set expectations too high for these technologies. What is your take on this issue?
Zhou Tingshuai: The development of technology will not change because of individual expectations being too high or too low. Participating with a positive mindset, understanding what can and cannot be done, and finding application scenarios is more meaningful.
Zhou Tingshuai: Senior Engineer of Baidu App Product R&D Department / Frontend Technical Lead
A senior R&D engineer at Baidu, Zhou Tingshuai is the technical lead for the Wenxin Yiyan App’s frontend. He has been dedicated to frontend development since joining Baidu in 2016, responsible for the frontend architecture of various projects, including Baijiahao, Baidu Information Flow, and the Wenxin Yiyan App, with extensive R&D experience.
On December 28-29, the QCon Global Software Development Conference will take place in Shanghai, featuring distinguished guests such as Academician Fan Wenfei, a foreign academician of the Chinese Academy of Sciences and an international database expert, and Dai Jinqian, Global CTO of Intel Big Data Technology, who will share cutting-edge insights in areas such as big data, chips, and architecture.
This conference will primarily discuss the evolution of the comprehensive technical architecture of large models, covering not only inference acceleration, AI Agents, and GenAI related to large models but also architectural evolution ideas, performance optimization, and R&D efficiency improvements represented by intelligent code assistants. Interested friends can scan the QR code below or click “Read Original” to view the detailed agenda of the conference. For ticket inquiries, please contact ticket manager 18514549229.
