How BERT Tokenizes Text

How BERT Tokenizes Text

Follow the official account “ML_NLP“ Set as “Starred“, delivering heavy content promptly! Source | Zhihu Link | https://zhuanlan.zhihu.com/p/132361501 Author | Alan Lee Editor | Machine Learning Algorithms and Natural Language Processing Public Account This article is authorized and reposting is prohibited This article was first published on my personal blog on 2019/10/16 and cannot be … Read more

When Bert Meets Keras: The Simplest Way to Use Bert

When Bert Meets Keras: The Simplest Way to Use Bert

Author: Su Jianlin Research Direction: NLP, Neural Networks Personal Homepage: kexue.fm Bert is something that probably doesn’t need much introduction. Although I’m not a big fan of Bert, I must say it has indeed caused quite a stir in the NLP community. Nowadays, whether in Chinese or English, there is a plethora of popular science … Read more

NLTK: Essential Toolkit for Natural Language Processing

NLTK: Essential Toolkit for Natural Language Processing

NLTK: Essential Toolkit for Natural Language Processing Many people ask me, what tools do I need to master to learn Natural Language Processing (NLP)? In fact, there are many tools for learning NLP, but there is one that you must master, and that is NLTK! NLTK stands for Natural Language Toolkit, which is a natural … Read more

Apache OpenNLP: A Powerful NLP Tool in the Java Ecosystem

Apache OpenNLP: A Powerful NLP Tool in the Java Ecosystem

OpenNLP is a natural language processing toolkit developed by the Apache Foundation, providing a range of machine learning tools for processing natural language text. It supports the most common NLP tasks, such as tokenization, sentence detection, part-of-speech tagging, named entity recognition, and more. Core Advantages Complete Functionality: Covers most basic NLP tasks Easy Integration: Can … Read more

Detailed Explanation of HuggingFace BERT Source Code

Detailed Explanation of HuggingFace BERT Source Code

Follow the official account “ML_NLP“ Set as “Starred“, heavy content delivered first-hand! Reprinted from | PaperWeekly ©PaperWeekly Original · Author | Li Luoqiu School | Master’s Student at Zhejiang University Research Direction | Natural Language Processing, Knowledge Graphs This article records my understanding of the code in the HuggingFace open-source Transformers project. As we all … Read more

Using Large Language Models in LlamaIndex

Using Large Language Models in LlamaIndex

One of the primary steps to consider when building any LLM application based on data is choosing the right LLM. LLMs are a core component of LlamaIndex. They can be used as standalone modules or inserted into other core LlamaIndex modules (indexers, retrievers, query engines). They are generally used during the response synthesis step after … Read more

Beginner’s Guide to NLP: Word2Vec Implementation

Beginner's Guide to NLP: Word2Vec Implementation

Words from Xiaolin For beginners who have never been exposed to NLP or Linux, even simple questions can become obstacles to achieving the final results. So, experts, please refrain from reading. No matter what, keep going; there is always a solution. “ Corpus: In statistical natural language processing, it is actually impossible to observe large-scale … Read more

Getting Started with NLTK: A Python Library for NLP

Getting Started with NLTK: A Python Library for NLP

Hello everyone! Today I want to introduce you to a super interesting Python library — NLTK (Natural Language Toolkit). It is a powerful tool for natural language processing, helping us easily handle human language. Whether you want to analyze text, extract information, or build a chatbot, NLTK can be very useful. Let’s explore this magical … Read more

NLTK: A Treasure Trove for Python Natural Language Processing!

NLTK: A Treasure Trove for Python Natural Language Processing!

▼ Click the card below to follow me ▲ Click the card above to follow me NLTK: A Treasure Trove for Python Natural Language Processing! Hi, Python friends! Today, I am going to take you on an exploration of a super cool Python library – NLTK (Natural Language Toolkit). Imagine being able to easily "understand" … Read more

NLTK: An Elite Python Library for Natural Language Processing

· Click the blue text to follow us Have you often seen those impressive artificial intelligences that can understand and answer human questions, and even write poetry? Actually, this is all thanks to natural language processing technology. Today, let’s talk about a super handy Python natural language processing toolkit – NLTK. PART.1 What is NLTK? … Read more