Future Research Directions of NLP in Marketing

• Embeddings and Vector Semantics: Methods like word2vec represent words as vectors and can study semantic relationships, but static embeddings have limitations, and there are also sentence-level embedding methods.

• Deep Learning Architectures: Including RNN, LSTM and CNN, etc., as well as the advantages of the Transformer architecture and different types (autoregressive, autoencoding, sequence-to-sequence) and their applications in marketing.

• Transfer Learning and Pre-trained Models: Can use large-scale pre-trained models for fine-tuning to complete specific tasks and reduce the need for annotated data.

• Concept and Topic Extraction: Including identifying words, phrases or topics, such as using methods like LDA, and its extensions and alternative methods.

• Relationship Extraction: Aims to extract relationships between words and entities, and its application is currently limited.

• Sentiment and Writing Style Extraction: Commonly uses dictionary methods, but has limitations, and advanced machine learning models perform better.

• Language has a dual role of reflecting information about the producer and affecting the recipient.

• In marketing applications, text can be used as an independent variable to predict outcomes, as a dependent variable to be extracted, and also to establish causal relationships.

• Further utilize transfer learning, share data and models.

• Improve the automatic understanding ability of the relationships among words, sentences and concepts.

• Explore the applications of multimodal representation learning in marketing.

• Solve the problems of model interpretability, bias and privacy.

• Provide effective methods for extracting information from text data to support marketing decisions.

• With the development of technology, new methods and applications continue to emerge.

• Limitations

• The interpretability of deep learning models is poor.

• Pre-trained models may have biases, and there are problems with the privacy of training data.

• Validating the constructs extracted from text is challenging.

NLP Applications in Marketing

Traditional Applications

• Concept and Topic Extraction

• Used to identify individual words, n-grams or entire topics in text. For example, brand name monitoring or identifying brand-related concepts in social listening. Common methods include traditional machine learning methods (like Naive Bayes, Support Vector Machines), Stanford Named Entity Recognizer (NER), Latent Semantic Analysis, Latent Dirichlet Allocation (LDA), and Poisson Factorization.

• LDA is a commonly used unsupervised topic model that has been widely applied in various aspects of marketing, such as strategic brand analysis from user-generated content, extracting brand-related information from social tags, etc. There are also some extensions of LDA, such as seed LDA, sentence-based LDA, hierarchical LDA and Poisson Factorization, which have advantages in different aspects.

• Relationship Extraction

• Aims to extract and identify relationships between words and entities. The most basic is to capture relationships through word co-occurrence, such as creating market maps using brand co-occurrence. However, for more complex relationships, such as customer needs or adverse drug reactions on social media, it requires moving beyond co-occurrence and bag-of-words methods. Currently, its application in marketing is limited, possibly due to the complexity of capturing relationships and assessing semantic similarity.

• Sentiment and Writing Style Extraction

• Commonly uses dictionary methods, such as Linguistic Inquiry and Word Count (LIWC), VADER, and Assessment Dictionary 2.0. These dictionaries quantify the information conveyed by documents by comparing the words in the document with the lists of words in the dictionary. LIWC has been widely used to study the impact of writing style on various behavioral outcomes.

• However, dictionaries have some limitations, such as the difficulty of creating high-performance dictionaries for complex structures, and they may not perform as well as traditional machine learning methods in classification accuracy, and their performance is context-dependent. When the required dictionary does not exist, researchers can train their own task-specific writing style classifiers using machine learning methods.

• Sentiment analysis is one of the most popular NLP tasks in marketing, used to identify the positive or negative tendency of text language, and has been applied in various aspects, such as predicting the impact of social media on consumer sentiment metrics, the influence of user-generated content on sales or stock market performance, etc. Although many sentiment analysis applications rely on dictionary methods, advanced machine learning models often perform better.

Emerging Applications

• Improvements of Traditional Tasks with Embedding-based Methods and Pre-trained Models

• Concept and Topic Extraction: Embedded Topic Models (ETM) combine word embeddings with traditional topic models (like LDA) to address the challenges of processing large and heavy-tailed vocabularies, outperforming LDA in topic quality and prediction performance.

• Relationship Extraction: Language models (like BERTopic) can help grasp finer relationships between words and entities compared to co-occurrence analysis, and can be analyzed at the word level (using static embeddings like word2vec or GloVe) or sentence level (using contextual embeddings like SentenceBERT).

• Sentiment and Writing Style Extraction: Advanced language models outperform dictionaries by more than 20 percentage points in sentiment analysis accuracy, capturing linguistic nuances better. For example, SiEBERT can be used to predict binary sentiment.

• Applications of New Tasks

• Text Generation: Large-scale pre-training enables language models to perform automatic text generation at near-human standards. For example, autoregressive models like GPT-3 are suitable for text generation and can be used for marketing applications such as SEO content generation and wine review creation.

• Text Summarization: Automatic text summarization allows advertisers to extract rich knowledge from large-scale text corpora. For instance, sequence-to-sequence models (like BART, T5) can be used for this task, and e-commerce websites and review platforms can use it to summarize product or service content.

• Multimodal Representation Learning: Utilizing methods like CLIP and custom multimodal network architectures to fuse multimedia data such as text, images, and videos. Although there are currently few application examples in marketing, there is significant potential.

Future Research Directions

1. Utilization of Transfer Learning

• The marketing field should better utilize transfer learning. Currently, the NLP and computer science fields are doing well in sharing models, data, and code, but the marketing field has room for development in this regard. By sharing data and training models, learning can be transferred from one application to another, enhancing the accuracy and richness of research. For example, many papers studying consumer reviews could generate new insights if data and models were shared.

• Capturing the true relationships among words, sentences, and concepts is a promising research area. Although embedding-based methods (like word2vec) have achieved many applications, there is still a long way to go in using automated methods to unravel related structures such as similarity, coherence, and relevance. Despite significant improvements in Transformer models, they still struggle with some language patterns that humans easily understand, such as simple negation and sarcasm, requiring further research to address these issues.

• Multimodal representation learning is a promising research direction. Currently, there are few application examples in marketing, and future exploration of utilizing methods like CLIP and custom multimodal network architectures to fuse multimedia data (such as text, images, and videos) in marketing applications can be further pursued.

• Interpretability: The interpretability of deep learning models is a challenge. Better understanding the errors and sensitivities of Transformer models can help build trust in model predictions. Methods like Local Interpretable Model-Agnostic Explanations (LIME) can be used to explain model predictions, or variable selection tools (like regularized regression) and dimensionality reduction tools (like topic modeling, classification dictionaries) can be used to enhance interpretability.

• Bias Handling: Although transfer learning reduces the need for annotated training data in the fine-tuning stage, large-scale pre-training may introduce biases. Language models pre-trained on unfiltered large-scale internet text data may replicate toxic language, amplify implicit biases, perpetuate stereotypes, and may pose privacy threats when trained on sensitive data. It is necessary to debias the models while being cautious to avoid human biases during the data annotation phase.

• Privacy and Validation: NLP methods rely on a large amount of text training data; researchers should be aware of legal and ethical privacy issues when using large-scale text data, such as obtaining data via APIs, and ensuring that any identifiable information is removed from consumer-level text data. At the same time, validation is a core issue when extracting constructs from text, requiring assessment of the accuracy of methods, not only for dictionary methods but also for machine learning classifiers.