Milvus is an open-source vector similarity search engine that supports the use of various AI models to vectorize unstructured data and provides search services for vector data. Milvus integrates widely used vector indexing libraries such as Faiss and Annoy, allowing developers to choose different indexing types for different scenarios. Using Milvus, one can develop a minimum viable product at a relatively low cost.

One of the most frequently asked questions in the Milvus community is:

“

I want to build an xx system; can Milvus achieve this? Are there any reference projects?

”

In fact, the project team has open-sourced many projects that use Milvus to accelerate AI applications on zilliz-bootcamp. To facilitate a more direct and straightforward experience with Milvus, we have currently launched 4 online application scenarios on the official website, covering various fields such as CV, NLP, and computer auditory.

🔗https://zilliz.com/solutions

• Intelligent Q&A System
• Image Search System
• Audio Retrieval System
• Video Object Detection System

This article will introduce the basic principles and usage methods of the above application scenarios, so let’s get started!

Application Scenario Introduction

🤖Intelligent Q&A

The Q&A system is a classic application scenario in the field of natural language processing, used to answer questions posed in natural language. Classic applications of intelligent Q&A systems include intelligent voice interaction, online customer service, knowledge acquisition, and emotional chatting. In this application scenario, we combine Google’s open-source model Bert with the vector similarity search engine Milvus to quickly build a semantic understanding-based intelligent Q&A robot.

👉Project Source Code: https://github.com/zilliz-bootcamp/intelligent_question_answering_v2

Usage Method

1. Upload the Q&A dataset in csv format, containing the columns question and answer.
2. Search for the target question to recall similar standard questions.
3. Click on the most similar question to get the answer.

👉Usage Tutorial: Milvus Online Q&A Robot Service v2.0

Implementation Principle

Data Import:

1. Use the Bert model to convert the standard questions in the uploaded dataset into 768-dimensional feature vectors stored in Milvus, which will return the corresponding ID.
2. Store these question IDs and their corresponding answers in PostgreSQL.

When searching for questions:

1. Use the Bert model to convert the user’s question into a feature vector.
2. Perform similarity search on the feature vector in Milvus to obtain the ID of the standard question most similar to the user’s question.
3. Retrieve the corresponding answer from PostgreSQL.

🔎Image Search

You are probably familiar with “image search,” as it is an essential feature in various search engines and shopping platforms. In fact, we can build our own image search system—create an image library and select an image from the library to search, ultimately obtaining several similar images. In this application scenario, Milvus, as a similarity retrieval engine for massive feature vectors, can easily implement the image search function by combining the image feature extraction model VGG.

👉Project Source Code: https://github.com/zilliz-bootcamp/image_search

Usage Method

1. Upload the image dataset in a compressed format, which can only contain jpg images.

2. Upload the image you want to search for to obtain similar images.

👉Usage Tutorial: Image Search System Based on Milvus

Implementation Principle

Data Import:

1. Use the VGG model to convert the images into 512-dimensional feature vectors stored in Milvus, which will return the corresponding ID.

2. Store the IDs of these feature vectors and their corresponding image paths in CacheDB.

When searching for images:

1. Convert the image to be searched into a feature vector using the VGG model.

2. Perform similarity search on the feature vector in Milvus to obtain the ID of the most similar feature vector in the Milvus library.

3. Retrieve the corresponding image from CacheDB.

🎵Audio Retrieval

In daily life, audio is an important type of multimedia data, as we listen to radio programs and enjoy online music. Audio retrieval technology is at the core of intelligent voice systems and has a wide range of application scenarios. Audio retrieval technology can perform real-time retrieval, review, and monitoring of online media to protect copyrights. It plays an important role in audio data classification and statistics. This application scenario utilizes the PANNs (Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition) model based on deep learning networks to extract audio feature vectors and combines them with Milvus to improve the efficiency of retrieving similar audio.

👉Project Source Code: https://github.com/zilliz-bootcamp/audio_search

Usage Method

1. Upload the audio dataset, which should only contain compressed files with wav format.

2. Upload the audio you want to search for to return similar audio.

👉Usage Tutorial: Audio Retrieval System Based on Milvus

Implementation Principle

Data Import:

1. Use the panns-inference pretrained model to convert audio data into feature vectors and import them into Milvus, which will return the corresponding ID.

2. Store the returned ID with related information of the audio data (such as wav_name) in MySQL database.

When performing audio retrieval:

1. Use the panns-inference pretrained model to extract feature vectors of the audio data to be retrieved.

2. Calculate the inner product distance between the feature vector of the audio data to be retrieved and the feature vectors of audio data previously imported into the Milvus library.

3. Retrieve information about similar audio data in MySQL based on the returned ID from the retrieval results.

📹Video Object Detection

With the rapid development of AI technology, significant breakthroughs have been achieved in target detection and image processing technologies in both academia and industry. More and more AI platforms are integrating these technologies and building many practical systems. This application scenario combines Milvus with technologies such as OpenCV, YOLOv3, and ResNet50 to accomplish the task of detecting objects in videos.

👉Project Source Code: https://github.com/zilliz-bootcamp/video_analysis

Usage Method

1. Upload the object dataset, which should be in a compressed image format, with images named after the object names.

2. After data import, upload the video, and the system will analyze the video. Click play to see the detection results for each moment.

👉Usage Tutorial: Video Object Detection System Based on Milvus

Implementation Principle

Data Import:

1. Use the ResNet50 model to extract features of the objects in the dataset and convert them into 2048-dimensional vectors stored in Milvus, which will return the corresponding ID.

2. Store the ID along with the corresponding names and object image paths in MySQL database.

When performing object detection:

1. Use OpenCV to process video frames.

2. Use YOLOv3 network for object detection.

3. Use ResNet50 model to extract features from the detected object images in the video frames and convert them into vectors.

4. Retrieve the corresponding name and image from MySQL based on the feature vector of the object images in Milvus.

Having explored so many application scenarios of Milvus, are you excited? Why not take action and experience it now! If you want to share other interesting Milvus application scenarios with us, feel free to scan the code to join the Milvus technical exchange group~

Welcome to Join the Milvus Community

github.com/milvus-io/milvus | Source Code

milvus.io | Official Website

milvusio.slack.com | Slack Community

zhihu.com/org/zilliz-11| Zhihu

zilliz.blog.csdn.net | CSDN Blog

space.bilibili.com/478166626 | Bilibili