Research on the Application Prospects of Natural Language Processing Technology in the Museum Field

Taking ChatGPT as an Example[1]*

Zhou Dingkai, Zhang Fenglin, Ding Zhiguo, Chen Yufei, Mao Ruohan

Introduction

With the continuous development of artificial intelligence technology, more and more institutions and industries are beginning to integrate AI technology to improve business processes and enhance efficiency. Natural language processing (NLP) is an important direction in the fields of computer science and artificial intelligence. It utilizes NLP technology to replace human-computer interaction, providing users with efficient and personalized services[1].

In the museum field, traditional chatbots can only interact according to preset syntax and response patterns, which are highly limited. However, the new generation of NLP technology, represented by ChatGPT, can achieve natural language parsing and reasoning of museum collections, culture, and data, and establish efficient, smooth, and personalized interactions with users[2]. Additionally, the new generation of NLP technology also possesses the ability to learn and optimize independently, continuously improving its language processing capabilities. Therefore, it is imperative to study its application scenarios in the museum field.

Smart museums are an important trend in the current development of museums, and NLP technology is one of the core technologies for achieving the digital transformation of museums, providing efficient information processing and management methods. Studying the application scenarios of NLP technology in the museum field helps promote the intelligent process of museums, enhance the quality and efficiency of cultural dissemination, provide a more user-friendly interactive environment for the public, offer researchers more convenient retrieval conditions, and provide managers with more efficient management tools. Furthermore, the application of NLP technology in the museum field is also one of the important scenarios for the application of NLP technology in the cultural tourism sector. Researching the application prospects of NLP technology in the museum field not only helps promote the digital transformation and intelligent upgrading of museums but also provides reference for the application and popularization of NLP technology in other aspects of the cultural tourism sector.

The technical principles and characteristics of natural language processing technology

Natural language processing (NLP) technology is one of the key technologies for studying human-computer interaction, mainly applied in the processing and understanding of natural language texts by computers. Its core goal is to enable computers to understand, analyze, process, and generate natural language texts like humans, achieving various functions such as semantic analysis, classification, annotation, named entity recognition, machine translation, sentiment analysis, automatic summarization, and speech recognition[3]. The development of NLP technology has made machines more intelligent, capable of handling the complexities brought by human language interactions, significantly enhancing the ability of computers to process text information, and has been widely applied in various fields such as text mining, search engines, intelligent customer service, speech recognition, and chatbots[4].

There are many language models based on NLP technology. For example, BERT developed by Google, DEBERTA developed by Microsoft, and Wenxin Yiyan developed by Baidu, among which the most famous is ChatGPT developed by OpenAI. It is a natural language processing technology based on the Transformer model, demonstrating outstanding performance in natural language generation, capable of generating meaningful, fluent, and natural new text based on the input language text. Additionally, when users ask follow-up questions, it improves its responses by considering the previous context (see Figure 1).

The reason ChatGPT can achieve coherent natural language generation is that its architecture includes multiple encoder and decoder layers, each capable of understanding and generating text. Furthermore, it learns language patterns and hidden information through unsupervised learning techniques while processing a large amount of natural language text, significantly enhancing its ability to generate natural language text. Currently, ChatGPT has been widely applied in text auto-generation, document processing, and efficient productivity tools, becoming an important breakthrough in the NLP field.

Figure 1 ChatGPT can generate and improve responses

ChatGPT is an artificial intelligence chatbot, and its underlying principles and implementation methods are as follows:

ChatGPT utilizes Generative Pre-trained Transformer technology, which involves unsupervised pre-training on a large amount of text data, followed by fine-tuning based on different tasks.

ChatGPT’s model structure is a deep neural network composed of multiple layers of self-attention mechanisms and feed-forward neural networks. It employs Transformer-XL technology, enhancing long-term memory capabilities through relative position encoding and segment recurrence.

The input to ChatGPT is a sequence of text composed of user messages and the chatbot’s responses. Its output is also a sequence of text, which is the next response from the chatbot. It uses an autoregressive method to generate outputs word by word, utilizing masks to avoid seeing future words.

ChatGPT employs cross-entropy as the loss function, minimizing the difference between predicted words and actual words. It uses the Adam optimizer to update model parameters, along with techniques like learning rate decay and gradient clipping to improve training efficiency and stability.

Compared to traditional chatbots, ChatGPT has higher accuracy and fluency in natural language generation and understanding, along with stronger contextual awareness and the ability to understand user intentions[5].

Application Scenarios of Natural Language Processing Technology in the Museum Field

Traditional museum visitor consultation services often require substantial human resources and equipment support, with high economic costs and demands on personnel quality. However, utilizing ChatGPT or similar NLP technologies can automatically handle and respond to various inquiries, providing accurate and rapid answers, reducing labor costs, optimizing user experience, and increasing visitor satisfaction. Furthermore, NLP technology can assist museums in promotional activities, generating content for background research, promotional suggestions, and plans, as well as aiding in online resources, promotional materials, creative copywriting, and language translation.

For instance, by integrating museum-related data, NLP technology can answer user inquiries about basic information (museum address, opening hours, ticket prices, reservation methods), exhibition information (current or upcoming exhibition names, themes, content, and times), collection information (specific collection names, ages, origins, materials, sizes), and background stories (stories behind collections, historical and cultural values).

To evaluate the performance of ChatGPT compared to human customer service, a set of questions was collected for both ChatGPT and human customer service to answer. For example, the following two questions (see Figure 2), the human response was: “Open daily from 9:00 AM to 5:00 PM (last entry at 4:00 PM), closed on Mondays (except national holidays). Visitors with ID can visit for free and can make reservations through the official website or WeChat public account. The advantages of the archaeological museum’s development mainly include: enhancing visitor experience; ensuring site safety; and boosting surrounding industries. The bottlenecks restricting the development of archaeological museums mainly include: constrained by the site’s popularity and influence; constrained by the appeal of the site’s highlights to the public; and constrained by the level of research on the site.”

Figure 2 ChatGPT’s output response to the question

The results show (see Table 1) that compared to human customer service, ChatGPT’s response speed is very fast, and it can accurately answer simple consultation questions, reducing user waiting time. In terms of emotional interaction and semantic expression, it also has certain advantages. However, when dealing with complex, obscure, or sensitive questions, it makes some errors, particularly prone to fabricating facts, leading to user misguidance. Therefore, considering the needs of different scenarios, a combination of NLP technology and human customer service is necessary to achieve better service outcomes.

The super interactivity of NLP technology can greatly expand the knowledge boundaries of museum exhibitions, even to some extent replacing the traditional tour guide positions. Specifically, NLP technology can achieve the following three applications in exhibitions.

1) Personalized Tours.

By using NLP technology, visitor language, emotions, and interests can be analyzed to achieve personalized exhibit recommendations. Visitors can inform the system of their interests and needs through dialogue, and the system will provide corresponding recommended exhibits or routes based on this information, even creating an interactive exploration game for visitors. These personalized tour services can better meet visitor needs and enhance the interactivity of exhibitions.

2) Intelligent Explanations.

By setting up NLP systems in exhibition halls, visitors can not only see text descriptions based on exhibit keywords and historical information but also interact with the exhibits. NLP technology can generate intelligent explanatory audio or virtual characters based on the information provided by the museum, automatically offering deeper and more detailed explanations of cultural relics. Additionally, NLP technology can provide personalized customized explanatory information based on visitor needs and interests, such as introducing the stories behind collections in a serious or humorous tone, thus enhancing the visitor experience.

3) Multilingual Support.

Museum collections involve languages and cultures from different regions, and NLP technology can adapt explanatory devices to different linguistic and cultural backgrounds to meet the needs of visitors from various regions and nationalities, improving the quality and efficiency of interactions. With the support of NLP technology, visitors can gain a more detailed understanding of the collections in the museum while also deeply experiencing the charm brought by cultural diversity.

Let ChatGPT and museum guides with relevant expertise describe the same batch of cultural relics or exhibits.

For example, for the “Turquoise Inlaid Beast Face Bronze Plaque” cultural relic, the standard description by the guide is: “The body is made of bronze, with a main frame that has rounded corners and slightly waist-shaped, with symmetrical ring knots on both sides. Hundreds of pieces of turquoise are inlaid to form a beast face pattern, intricately crafted with no looseness or shedding after three to four thousand years. It was found placed on the chest of the tomb owner, with symmetrical perforated knots on both sides, indicating it was an important medium for communication between heaven, earth, gods, and humans.” ChatGPT’s description is as follows (see Figure 3).

Figure 3 ChatGPT’s written description of the cultural relic

Comparing and evaluating the output results, it is found that ChatGPT’s writing speed is very fast, with fluent and easy-to-understand language, and it can comprehensively introduce relevant information about the cultural relics. However, its professionalism is relatively lacking, and most importantly, there are instances of discrepancies with factual information (see Table 2). Therefore, combining the advantages of both may provide a more effective museum guided tour service.

ChatGPT can serve as a query-based Q&A tool, addressing simple and common inquiries, while the guided tour guide can offer deeper and more comprehensive explanations, providing visitors with a more personalized and customized tour experience. This combination of NLP technology and human service can better enhance the quality and service efficiency of museum guided tours, bringing visitors a richer and deeper cultural experience.

Traditional exhibition arrangement and outline preparation require curators to read and process a large amount of literature, consuming a significant amount of time. NLP technology can quickly and efficiently analyze and extract information relevant to the exhibition theme through text mining, topic extraction, and other techniques, automatically generating exhibition outlines, thereby significantly improving planning efficiency. NLP technology can also participate in the evaluation of museum exhibitions, generating evaluation results based on evaluation models and providing relevant suggestions to assist curators in optimizing exhibitions.

For example, using ChatGPT to describe the curator’s needs, AI will provide exhibition ideas (see Figure 4). Based on the ideas provided by the AI, curators can continually enrich and improve the requirements by providing more precise descriptive terms, allowing the AI to generate exhibition outlines that better align with the curator’s vision.

Figure 4 Exhibition ideas provided by ChatGPT

Although ChatGPT has a high level of intelligence and automation in analyzing and processing exhibition-related information, significantly improving efficiency at the initial stage of curation, it lacks human aesthetic judgment and may not match the curator’s intentions. If the exhibition theme is complex and involves highly specialized content, additional human intervention is still necessary. Curators need to combine their judgment and experience to develop more comprehensive and innovative exhibition plans.

NLP technology can organize clear and accurate cultural relic data types based on the descriptions input by researchers regarding cultural relic names, ages, origins, materials, sizes, etc., through automatic information extraction, data standardization, and classification. For clearly classified cultural relic information, NLP technology can perform intelligent classification, such as classification by different types of cultural relics, different ages, and different materials, to achieve orderly management of cultural relic information. These organized cultural relic photos, textual information, audio, and video materials can provide important material support for the protection of museum collections.

Through comparative experiments between NLP technology and manual processing of cultural relic information, for example, extracting turquoise artifacts from a list of cultural relics. The human response was: turquoise dragon-shaped vessel, turquoise beads, turquoise beast. ChatGPT also listed the turquoise inlaid beast face bronze plaque and dragon-shaped jade as turquoise artifacts, and in fact, the dragon-shaped jade does not have turquoise decorations (see Figure 5).

Figure 5 ChatGPT’s response regarding cultural relic information

The results indicate that due to the complexity and diversity of cultural relic information, ChatGPT currently cannot achieve the accuracy and stability comparable to human performance in handling basic tasks related to cultural relics, let alone complex information processing involving cultural relics. Therefore, human intervention and review are still necessary to ensure the accuracy and reliability of cultural relic information. In the future of collection management, a deep integration of NLP technology and human efforts is essential, requiring continuous optimization and improvement of corresponding systems and workflows to enhance the efficiency and quality of collection management.

NLP technology can assist cultural product developers in quickly processing a large amount of background information, including cultural relics and collection information, historical stories, etc., providing more information support, exploring historical and cultural connotations, and offering reliable basis for cultural product development, enhancing the cultural quality and innovation of cultural products. Simultaneously, by utilizing natural language generation and image recognition technologies, multidimensional presentations of cultural relics and collections can be achieved, providing new inspiration and ideas for the design of museum cultural products, thus creating cultural products with taste and differentiation.

Through comparative experiments between NLP technology and human designers related to the conceptualization of cultural products, for example, designing a lifestyle home product based on the theme of “Turquoise Inlaid Beast Face Bronze Plaque” (see Figure 6).

The design idea of the human designer is: first, to understand the historical and cultural background related to the cultural relics, while combining modern life and consumer needs to create a narrative and meaningful cultural product. Utilizing turquoise and bronze, which both have natural aesthetic appeal, to apply them in various home goods, such as glassware, carpets, storage boxes, etc. By blending modern and traditional elements, creating innovative designs for the cultural product.

Figure 6 ChatGPT’s cultural product design idea

Comparing the two, it can be found that ChatGPT can only interact with users through language input and output, lacking actual design capabilities and manual creativity. In contrast, human designers possess more professional design skills and diverse design ideas, capable of creating more innovative and personalized cultural products based on the characteristics and needs of museum collections.

However, NLP technology can still provide designers with inspiration and suggestions in tasks such as analyzing cultural information, extracting cultural elements, and creative elements. In selecting cultural elements, data filtering, and conceptualization, human involvement is still required to ensure the final product’s quality and cultural value. Thus, the application of NLP technology in cultural product development still needs to combine the wisdom of artificial intelligence and designers to comprehensively enhance the creativity and satisfaction of cultural products.

NLP technology provides an efficient method for information integration and research in cultural heritage studies. NLP technology helps establish semantic relationships and knowledge graphs to study relics, sites, and cultural information more deeply and comprehensively. Through intelligent analysis and semantic association technology, a knowledge graph of information can be constructed, visually presenting the relationships between relevant information more clearly.

Moreover, NLP technology can automatically identify connections between cultural relics, sites, and cultural information and apply them in the field of cultural relic research. NLP technology can automatically integrate the interrelations and influences of the attributes of collections, enhancing researchers’ understanding of collections and enriching relevant knowledge in the field of museum relic research.

Through comparative experiments between NLP technology and human researchers handling literature, for example, regarding the topic of “Similarities between the Erlitou Site and the Yin Ruins” (see Figure 7), the response from a human researcher was: “Both sites have constructed roads, with the widest planned road at the Erlitou site being about 20 meters, equivalent to a modern four-lane highway, and the earliest wheel tracks in China were discovered. From the perspective of palace and road construction, there is a clear inheritance relationship between the two sites.”

The results show that NLP technology can achieve quite good stability, accuracy, and comprehensiveness in tasks such as information analysis and key point extraction, greatly enhancing research efficiency. However, some errors are still inevitable, so human intelligence is needed to process and analyze existing materials to ensure the rigor and innovation of the final research results.

Figure 7 Comparison of literature responses

The future and challenges of natural language processing technology in the museum field

AIGC (AI-generated content) technology refers to the generation of various forms of content, such as text, images, and sound, through algorithms supported by artificial intelligence technologies. Its implementation relies on deep learning, natural language processing, computer vision, audio processing, and other AI technologies[6].

For example, Runway’s Gen-2 can create realistic synthetic videos based on typed descriptions, anything you can imagine can be created. Stability AI’s Stable Diffusion offers services to convert hand-drawn sketches into realistic and exquisite digital images.

In the near future, the combination of NLP technology and other AIGC technologies can bring more new opportunities and transformations to museums, especially in virtual reality (VR) and augmented reality (AR) fields, enabling more realistic and immersive experiences.

For instance, by combining NLP and image generation technologies, natural language text can be converted into descriptive language for images, and then image generation technology can use these descriptions to create images related to the text. This method can be applied in many fields, such as virtual reality experiences of natural scenes, imaginative special effects in movies and games, etc. Importantly, this technology can be applied in exhibition planning and design, playing a crucial role in efficiently implementing the curator’s intentions.

The application of NLP technology in the museum field also faces several technical challenges that may impose limitations on technological advancement and application. The main challenges include the following aspects.

1) High Professionalism.The volume of information in the museum field is enormous, including various literature on collections, relic identification, scientific analysis of relics, relic restoration, archaeological excavations, etc. This information can only be understood and interpreted by professionals. Therefore, when using NLP technology for text processing and information extraction in the museum field, the complexity of domain knowledge itself and the requirements for model training and algorithm validation related to domain professionalism must be considered. It is necessary to combine NLP-related technologies with the specialized knowledge of museums to improve extraction accuracy.

2) High Accuracy Requirements.Museums bear social educational functions, and in the eyes of the public, museums represent authoritative interpretations. Therefore, if NLP technology is applied in the museum field, strict monitoring and validation of the data quality and accuracy generated by NLP must be implemented to ensure that the information and interpretations provided align with historical facts and cultural values, effectively avoiding erroneous information from negatively impacting relics and society. This also requires close collaboration with professionals in the business field to clarify specific goals and requirements for data processing, gradually improving data quality and accuracy to enhance overall reliability.

3) Collection Data Security.Some data involved in the museum field may relate to sensitive confidential information, such as relic information, auction records, restoration records, transfer records, etc. Therefore, during the application of NLP technology, attention must be paid to data security and appropriate security measures must be taken, such as encryption, permission settings, and network protection. In addition, it is essential to comply with relevant laws and regulations to ensure the privacy and security of relic data.

To address these technical challenges, the following solutions may be needed.

1) Build a Knowledge Base in the Museum Field.

A knowledge base is a database that stores structured or semi-structured knowledge, providing rich and accurate domain knowledge for NLP technology. By extracting entities, attributes, and relationships from relevant museum literature, websites, databases, etc., and performing operations such as cleaning, integration, and disambiguation, a knowledge base containing various concepts, instances, and rules in the museum field can be constructed. The knowledge base can assist NLP technology in semantic understanding, reasoning, and generation, enhancing its effectiveness in the museum field.

2) Optimize Model Training and Algorithm Validation Using Deep Learning Methods.

Deep learning is a machine learning method based on multi-layer neural networks that can automatically learn feature representations from large amounts of data, improving model generalization and robustness. Traditional machine learning methods often require manual design of features and prior knowledge, which may be constrained by professionalism and complexity in the museum field. In contrast, deep learning methods can better capture semantic and contextual information in museum-related texts and perform effective classification, extraction, and generation tasks. For example, large language models (LLMs) and fine-tuning techniques can be utilized to train NLP robots for museum applications.

3) Use Multimodal Data Fusion Techniques to Enhance Data Security.

Multimodal data fusion refers to the process of integrating, analyzing, and utilizing different types or sources of data, which can improve data utilization efficiency and value. In the museum field, in addition to text data, there are various types of data such as images, videos, and audio, which can complement, verify, and encrypt each other to enhance data security. For instance, when extracting relic information, textual descriptions, image features, and video content can be used concurrently for cross-validation to prevent single data sources from being tampered with or leaked; when generating relic information, textual descriptions, image features, and video content can be utilized simultaneously for encryption to prevent sensitive information from being stolen or cracked.

Although NLP technology has great potential in the museum field, it also faces some specific limiting factors, mainly including computational resource and performance issues, data management and recommendation effectiveness, misunderstanding and ambiguity issues, among others.

1. Computational Resource and Performance Issues.

NLP systems in museum self-media or website applications need to process large amounts of text and voice information to provide users with intelligent Q&A, recommendations, translations, and other services. These services require high computational resources, and if faced with a large volume of visits, there may be insufficient computational resources, leading to a decline in service quality. To address this issue, distributed computing and low-latency services should be employed while optimizing algorithms and model parameters to improve the processing speed and performance of models, enhancing the scalability and processing efficiency of NLP systems. Additionally, high computing resources and stable data center environments are required during training; otherwise, it may affect training results and model quality.

2. Data Management and Recommendation Effectiveness.

The information on cultural relics in museums involves multiple fields, such as history, culture, art, and science, each with different classifications and attributes. This information constitutes the knowledge base of the museum, which is the foundation for data management and recommendation effectiveness of NLP systems. NLP systems need to effectively organize, store, retrieve, and analyze this information to provide users with the most suitable information. Simultaneously, NLP systems must also be able to offer personalized recommendations based on user interests, preferences, and needs to increase user satisfaction and engagement. For instance, ChatGPT is primarily applied in dialogue generation and Q&A systems in museums, utilizing large-scale dialogue datasets for pre-training, and then fine-tuning for different tasks to generate natural, fluent, and logical dialogues. Selecting appropriate dataset scale, quality, and distribution is key to improving ChatGPT’s performance effectiveness.

3. Misunderstanding and Ambiguity Issues.

Some meanings and interpretations of cultural relic information may have misunderstanding and ambiguity issues due to cultural differences and historical changes. These problems can pose challenges for NLP systems in understanding and expressing cultural relic information. When dealing with these issues, NLP systems require more human supervision and interactive communication to ensure the accuracy of system understanding and expression. Additionally, existing technologies such as polysemy and context analysis should be employed to identify and eliminate ambiguities.

Conclusion and Outlook

The application of natural language processing technology in the museum field has broad prospects and application scenarios, especially the emergence of ChatGPT provides strong tool support for information management, cultural relic explanations, and consultations in museums.

This article analyzes the technical principles and characteristics of ChatGPT and discusses its various application scenarios in museums from a practical application perspective, including cultural relic explanations, automated consultation responses, information management, and research. By comparing the performance differences between ChatGPT and humans in different application scenarios, it can be observed that ChatGPT excels in information processing efficiency and common-sense expression, but it clearly needs improvement in accuracy and factual presentation.

NLP technology has multi-faceted application potential and value in the museum field, potentially combining various museum service roles such as exhibition guides, customer service, curators, collection managers, cultural product developers, and cultural heritage researchers, contributing to enhancing museum service levels, improving social cultural quality, and promoting the protection and inheritance of cultural heritage.

This article also proposes possible future research directions and technical challenges, such as multimodal information processing, open-domain question-answering systems, and personalized recommendation systems. Future research needs to further develop interdisciplinary research and innovation, combining natural language processing technology with other technologies to explore more intelligent technologies suitable for museum applications, such as image processing, artificial intelligence, and big data.

Moreover, security and privacy factors must also be considered, enhancing user information security while improving museum visitor experience and management efficiency. In summary, the application of natural language processing technology in the museum field is a continuously innovating and exploring domain. Through ongoing exploration and improvement, it will provide museums with more intelligent, convenient, and efficient services, offering visitors a more accessible cultural tourism experience.

Source: “Science Education and Museums” published with the author’s permission

Author Introduction: Zhou Dingkai (1990—), male, curator, main research direction: cultural relics and museums.

Zhou Dingkai (Erlitou Summer Capital Site Museum)

Zhang Fenglin (Zhejiang Provincial Museum)

Ding Zhiguo (China Museum Association)

Chen Yufei (Palace Museum)

Mao Ruohan (Zhejiang University)

1* This article is a phase result of the 2023 Ministry of Culture and Tourism’s departmental-level social science research project “Research on the Mechanism and Model of Metaverse Empowering New Business Forms of Museums” (Project Approval Number: 23DY31) ↑