Recently, Milvus released version 2.2.6, which has become increasingly stable after fixing some bugs.
In fact, since upgrading to version 2.X, we have been working hard to improve and optimize, launching new features such as batch importing data from files, disk-based Approximate Nearest Neighbor (ANN) indexing algorithms, non-stop rolling upgrades, Coordinator HA (High Availability), resource groups, and more. We have also improved the performance of metadata storage and batch imports.
This article reviews the changes and upgrades from Milvus 2.2 to Milvus 2.2.6, taking you deeper into a more stable, smooth, and user-friendly Milvus.
01.
Milvus 2.2: Enhancing Stability, Search Speed, and Flexible Scalability of Vector Search
Milvus 2.2 is a major version update that not only introduces new features but also makes many groundbreaking improvements:
-
Batch importing data from files: Milvus provides a series of new APIs to support more efficient batch importing of data from files. It can directly import data from any JSON file stored on Minio or S3 into Milvus, saving time and effort.
-
Pagination of search results: To avoid a single RPC session returning too many search results, Milvus now supports reducing the number of returned search results by setting an offset and keywords.
-
Role-Based Access Control (RBAC): Milvus now supports RBAC. Users can manage users, roles, and permissions to control access to the Milvus database.
-
Quotas and limits: Quotas are a new mechanism used by Milvus to protect the system from memory overflow (OOM) and crashes during traffic spikes. With this mechanism, you can control data insertion, search rates, and memory usage.
-
Collection Time-To-Live (TTL): In previous versions, Milvus only supported setting TTL at the cluster level. Milvus now supports setting TTL when creating or modifying a collection. Data in collections with a set TTL will expire after the TTL ends. This feature allows users to manage data more precisely.
-
Disk-based Approximate Nearest Neighbor Search Index (Beta): Milvus now supports DiskANN for indexing large datasets based on disk. DiskANN is an indexing algorithm that performs approximate nearest neighbor searches on datasets residing on solid-state drives (SSD). This algorithm can save about 10 times the memory usage.
-
Data backup tool (Beta): Milvus 2.2 provides a new data backup tool and supports database backup and recovery via command line and API calls.
In addition to the above new features, Milvus 2.2 has fixed several bugs and made numerous improvements to enhance stability, observability, and performance.
Milvus 2.2 Release Notes: https://milvus.io/docs/release_notes.md#v220
Click to read the article《Milvus 2.2 Version Released!
02.
Milvus 2.2.1 and 2.2.2: Minor Releases Focused on Bug Fixes
Milvus 2.2.1 and 2.2.2 are minor releases that fixed critical issues in previous versions and introduced some new features.
Milvus 2.2.1:
-
Support for Pulsar tenants and authentication
-
Support for Transport Layer Security (TLS) in etcd configuration
-
Search performance improved by over 30%
-
Optimized scheduler and increased task merging probability
-
Fixed multiple vulnerabilities, including: filter failure on indexing scalar fields, crashes caused by index creation failures
-
Milvus 2.2.2:
-
Fixed the issue where Proxy did not update Shard leader cache
-
Fixed the issue of data not being cleaned up in released collections/partitions
-
Fixed the issue where load counts were not cleaned up in time
Milvus 2.2.1 Release Notes:https://milvus.io/docs/release_notes.md#v221
Milvus 2.2.2 Release Notes:https://milvus.io/docs/release_notes.md#v222
03.
Milvus 2.2.3: Secure, Stable, and Upgrade-Friendly
Milvus 2.2.3 primarily enhances the system’s security, stability, and availability, introducing two important features.
-
Non-stop rolling upgrade: Milvus v2.2.3 introduces rolling upgrade capabilities. This means that user queries and search requests will not be interrupted during the upgrade process.
-
Coordinator High Availability (HA): The Milvus coordinator can operate in a master-slave node mode, reducing the risk of single points of failure. Coordinator HA can significantly shorten system recovery time. In the event of a disaster, the system can recover in as little as 30 seconds.
Additionally, Milvus 2.2.3 has made improvements and fixed multiple vulnerabilities, including: significant performance improvements in bulk inserts, optimized metrics information, enhanced query performance, reduced memory usage, and improved metadata storage performance.
Milvus 2.2.3 Release Notes:https://milvus.io/docs/release_notes.md#v223
Click to read the article《Milvus New Version is Here! Chief Engineer Highlights: Secure, Stable, Upgrade-Friendly》
04.
Milvus 2.2.4: Resource Isolation, Multi-Cloud, Efficient
Milvus 2.2.4 is a minor version upgrade of Milvus 2.2. It introduces four new features and enhances performance, reliability, and efficiency while reducing resource consumption.
-
Resource groups: Milvus supports assigning multiple Query Nodes to different resource groups, completely isolating access to physical resources in different resource groups.
-
Renaming collections: A new API for renaming collections has been added, making collection management more flexible.
-
Support for Google Cloud Storage: Milvus now supports Google Cloud Storage as an object storage option, allowing users of Google Cloud to seamlessly integrate with GCP’s object storage.
-
New options for Search and Query APIs: We have added an option to skip Growing data in query parameters, enabling better search performance in insertion scenarios.
Milvus 2.2.4 Release Notes:https://milvus.io/docs/release_notes.md#v224
Click to read the article《Another Version Upgrade! One Sentence Summary of Milvus 2.2.4: Resource Isolation, Multi-Cloud, Efficient》
05.
Milvus v2.2.6: Stability Upgrade
Milvus 2.2.6 addresses numerous stability issues.
-
Performance Enhancements
When latency is at least 5 seconds, slow log performance is added for queries/searches. The default log level for Milvus is info, and since query paths are highly concurrent and performance-sensitive, logs for query paths are at debug level, making it impossible to see these logs in production environments. If users want to track slow queries, they have no way to do so; MySQL’s slow query log serves as a good example, so Milvus has also added slow query records to its logs.
-
Key Issues Resolved
-
Fixed the issue of DataCoord GC failure
-
Fixed the issue where index parameters passed during collection creation would override parameters passed in subsequent create_index operations
-
Resolved the issue of message backlog in RootCoord causing increased system latency
-
Corrected the accuracy of the RootCoord InsertChannelTimeTick metric
-
Resolved the issue where Proxy reported timestamps might stop working in certain cases
-
Fixed the potential panic issue of the Coordinator role during restart
-
Fixed issues caused by etcd restarts leading to abnormal exits of garbage collection goroutines, leaving checkpoints, etc.
Milvus 2.2.6 Release Notes:https://milvus.io/docs/release_notes.md#v226
From Milvus 2.2 to 2.2.6, each version upgrade has been supported by user feedback and suggestions. We are committed to providing users with cutting-edge solutions and supporting applications in various fields. A sneak peek: Milvus 2.2.7 is in the works and will be released soon, stay tuned!
