mirror of
https://github.com/milvus-io/milvus.git
synced 2026-07-21 10:15:43 +00:00
70f144a91cc8ac4305b277c46dccf335d3b9bf0d
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Bingyi SunandGitHub
|
70f144a91c |
enhance: add delete/shard by namespace (#50153)
Shard data by namespace. issue: #50154 --------- Signed-off-by: sunby <sunbingyi1992@gmail.com> |
||
|
congqixiaandGitHub
|
7311d450a0 |
enhance: bump Go dependencies to v3 modules (#49485)
Related to #49398 Bump Go module references from pkg/v2 and milvus-proto/go-api/v2 to pkg/v3 and milvus-proto/go-api/v3 so the client tracks the Milvus 3.x release line. This prepares the repository for the upcoming 3.x.y release by aligning imports, module dependencies, and proto API references with the new major-version module paths. --------- Signed-off-by: Congqi Xia <congqi.xia@zilliz.com> |
||
|
|
645113b043 |
enhance: separate upsert query metrics (#48445)
issue: #48444 Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Bingyi SunandGitHub
|
39606eec74 |
fix: Optimize namespace compaction and query implementation (#46512)
### **User description** issue: https://github.com/milvus-io/milvus/issues/44011 Updated the DescribeCollection functionality to exclude the namespace field from the response schema. This change ensures that these fields are not returned in the collection description. ___ ### **PR Type** Bug fix, Enhancement ___ ### **Description** - Filter namespace field from DescribeCollection response schema - Add support for PartitionKeySortCompaction in compaction task handling - Fix lambda capture issues in C++ expression evaluation code - Add comprehensive test coverage for namespace field filtering ___ ### Diagram Walkthrough ```mermaid flowchart LR A["DescribeCollection Request"] --> B["Filter Fields"] B --> C["Exclude Dynamic Fields"] B --> D["Exclude Namespace Field"] C --> E["Response Schema"] D --> E F["Compaction Task"] --> G["Create/Complete Task"] G --> H["Support PartitionKeySortCompaction"] H --> I["Task Execution"] ``` <details><summary><h3>File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Bug fix</strong></td><td><table> <tr> <td> <details> <summary><strong>service_provider.go</strong><dd><code>Filter namespace field from DescribeCollection response</code> </dd></summary> <hr> internal/proxy/service_provider.go <ul><li>Added import for <code>common</code> package to access <code>NamespaceFieldName</code> constant<br> <li> Modified field filtering logic in <code>DescribeCollection</code> to exclude <br>namespace field alongside dynamic fields<br> <li> Updated filter condition to check both <code>IsDynamic</code> flag and field name <br>equality</ul> </details> </td> <td><a href="https://github.com/milvus-io/milvus/pull/46512/files#diff-e72e29bf5e62a9c5a797c0045e0d6f427d5c49e587c848f68e81ceabaa3e2a0c">+2/-1</a> </td> </tr> <tr> <td> <details> <summary><strong>task.go</strong><dd><code>Exclude namespace field from task execution</code> </dd></summary> <hr> internal/proxy/task.go <ul><li>Updated <code>describeCollectionTask.Execute</code> to filter out namespace field <br>from response schema<br> <li> Modified condition to skip fields that are either dynamic or have <br>namespace field name<br> <li> Ensures namespace field is not included in collection description <br>results</ul> </details> </td> <td><a href="https://github.com/milvus-io/milvus/pull/46512/files#diff-b4aff1bcd223cde92858085e1be028292b0899b08c9664aa590a952f01106e3d">+1/-1</a> </td> </tr> <tr> <td> <details> <summary><strong>Expr.cpp</strong><dd><code>Fix lambda capture and remove duplicate includes</code> </dd></summary> <hr> internal/core/src/exec/expression/Expr.cpp <ul><li>Removed duplicate include of <code>expr/ITypeExpr.h</code><br> <li> Removed unused include of <code>fmt/format.h</code><br> <li> Fixed lambda capture issues in <code>SetNamespaceSkipIndex</code> function by using <br>pointer capture<br> <li> Changed from reference capture <code>[&]</code> to explicit pointer and value <br>captures to avoid dangling references<br> <li> Extracted namespace field ID and value as const variables for safer <br>lambda capture</ul> </details> </td> <td><a href="https://github.com/milvus-io/milvus/pull/46512/files#diff-1051ba05ef883a88cf621aa1ca7ba4f3b433be44f574877c1888f39eed42cb50">+16/-16</a> </td> </tr> </table></td></tr><tr><td><strong>Tests</strong></td><td><table> <tr> <td> <details> <summary><strong>service_provider_test.go</strong><dd><code>Add test for namespace field filtering</code> </dd></summary> <hr> internal/proxy/service_provider_test.go <ul><li>Added imports for <code>schemapb</code> and <code>common</code> packages<br> <li> Created new test <br><code>TestCachedProxyServiceProvider_DescribeCollection_FilterNamespaceField</code><br> <li> Test verifies namespace and dynamic fields are filtered while user <br>fields are preserved<br> <li> Uses mock cache to simulate collection metadata with namespace field</ul> </details> </td> <td><a href="https://github.com/milvus-io/milvus/pull/46512/files#diff-47cdd329529238ca17e2a5c9f66f548cf941e19d332203a099d0fa8a9e67dd47">+65/-0</a> </td> </tr> <tr> <td> <details> <summary><strong>task_test.go</strong><dd><code>Add namespace field filtering test case</code> </dd></summary> <hr> internal/proxy/task_test.go <ul><li>Added comprehensive test <br><code>TestDescribeCollectionTask_FilterNamespaceField</code><br> <li> Test creates schema with namespace field as partition key and dynamic <br>metadata field<br> <li> Verifies both namespace and dynamic fields are excluded from describe <br>result<br> <li> Confirms user fields like id and fvec are properly included</ul> </details> </td> <td><a href="https://github.com/milvus-io/milvus/pull/46512/files#diff-ab14021f2a0170ad1c610628871f98c5cd4386ce95efe7af94b4509bdea4ccf7">+90/-0</a> </td> </tr> </table></td></tr><tr><td><strong>Enhancement</strong></td><td><table> <tr> <td> <details> <summary><strong>compaction_inspector.go</strong><dd><code>Support PartitionKeySortCompaction in task creation</code> </dd></summary> <hr> internal/datacoord/compaction_inspector.go <ul><li>Added <code>datapb.CompactionType_PartitionKeySortCompaction</code> to switch case <br>in <code>createCompactTask</code><br> <li> Now handles PartitionKeySortCompaction alongside MixCompaction and <br>SortCompaction<br> <li> Routes PartitionKeySortCompaction to <code>newMixCompactionTask</code> handler</ul> </details> </td> <td><a href="https://github.com/milvus-io/milvus/pull/46512/files#diff-1c884001f2e84de177fea22b584f3de70a6e73695dbffa34031be9890d17da6d">+1/-1</a> </td> </tr> <tr> <td> <details> <summary><strong>meta.go</strong><dd><code>Handle PartitionKeySortCompaction in mutation completion</code> </dd></summary> <hr> internal/datacoord/meta.go <ul><li>Added <code>datapb.CompactionType_PartitionKeySortCompaction</code> to switch case <br>in <code>CompleteCompactionMutation</code><br> <li> Routes PartitionKeySortCompaction to <code>completeSortCompactionMutation</code> <br>handler<br> <li> Ensures proper mutation completion for partition key sort compaction <br>type</ul> </details> </td> <td><a href="https://github.com/milvus-io/milvus/pull/46512/files#diff-1b1c74d883d233d2457813f0708bd3a3418102555615d6abca5e04b9815e6c37">+1/-1</a> </td> </tr> </table></td></tr></tbody></table> </details> ___ <!-- This is an auto-generated comment: release notes by coderabbit.ai --> - Core invariant: schemapb.CollectionSchema.EnableNamespace is the authoritative per-collection flag for namespace behavior — it is persisted on Collection models, copied into DescribeCollection responses, and used at runtime to derive datapb.SegmentInfo.IsNamespaceSorted and datapb.CompactionSegmentBinlogs/CompactionSegment.IsNamespaceSorted (i.e., per-collection EnableNamespace → per-segment IsNamespaceSorted). - Removed/simplified logic: eliminated property-based namespace toggles and partition-key-sort special-cases (TriggerTypePartitionKeySort / TriggerTypeClusteringPartitionKeySort and IsPartitionKeySortCompactionEnabled) and removed the dual namespace-skip API in expression evaluation (SetNamespaceSkipFunc / SetNamespaceSkipIndex); routing now reuses existing Mix/Sort/Clustering handlers and uniformly treats IsNamespaceSorted alongside IsSorted in compaction, index, and inspection checks. - Why no data loss or behavior regression: wire compatibility preserved (SegmentInfo field number unchanged when renaming is_partition_key_sorted → is_namespace_sorted), binlog/segment IDs, manifest paths and storage versions are unchanged, and compaction requests only add/pass an extra boolean flag — no binlog content, identifiers, or storage layout are modified; DescribeCollection now only changes user-facing filtering of schema fields (internal stored schema unchanged) and unit tests were added for filtering to prevent regressions. - Bug fix (milvus-io/milvus#44011): fixes leakage of internal namespace/meta fields in DescribeCollection by filtering out fields with IsDynamic==true and the Namespace field name (common.NamespaceFieldName) and by propagating EnableNamespace through DescribeCollectionTask.Execute; tests added in internal/proxy/service_provider_test.go and internal/proxy/task_test.go validate the fix. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: sunby <sunbingyi1992@gmail.com> |
||
|
Bingyi SunandGitHub
|
633cae9461 |
enhance: add namespace for query and search request (#44343)
issue: #44011 --------- Signed-off-by: sunby <sunbingyi1992@gmail.com> |