### Summary
Follow-up to #48152 which applied denylist retry to parquet/json/csv
imports but missed two other paths.
- **fix(High)**: `pack_writer.go` `writeLog` now skips retry only for
non-retryable errors (permission denied, bucket not found, invalid
credentials, etc.), matching the denylist strategy in
`retryable_reader.go`.
- **fix(Medium)**: Binlog import's `WithDownloader` callbacks now use
`multiReadWithRetry`, skipping retry only for non-retryable errors.
Previously all transient failures were not retried.
- **fix(Low)**: `IsMilvusError` in `merr/utils.go` switched from
`errors.Cause` (root only) to `errors.As` (full chain traversal).
### Out of Scope
- `pack_writer_v2.go` / `pack_writer_v3.go` — same retry pattern but
different code path (multi-part upload); separate fix.
- `writeDelta` — no retry wrapper; separate concern.
issue: #48153
---------
Signed-off-by: Yihao Dai <yihao.dai@zilliz.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
design doc:
https://github.com/milvus-io/milvus-design-docs/blob/main/design_docs/20260105-external_table.md
issue: https://github.com/milvus-io/milvus/issues/45881
## Summary
- Pre-allocate segment IDs in DataCoord, pass to DataNode for direct
final-path manifest writes (eliminating two-phase ID workflow)
- Add FFI bridges for file exploration (`ExploreFiles`, `GetFileInfo`)
and manifest creation (`CreateManifestForSegment`,
`ReadFragmentsFromManifest`)
- Implement fragment-to-segment balancing with configurable target rows
per segment
- Add `ExternalSpec` parser for external data format configuration
- Extend `UpdateExternalCollectionRequest` proto with schema, storage
config, and pre-allocated segment ID fields
- Add E2E test for external collection refresh with data verification
> **Note**: This PR includes Part3 changes (PR #47303). After Part3 is
merged, this PR will be rebased to only contain Part4-specific changes.
## Test plan
- [x] Unit tests for `task_refresh_external_collection.go` (28 tests)
- [x] Unit tests for `task_update.go` and fragment utilities (40 tests)
- [x] Unit tests for FFI bridges (`exttable_test.go`, 9 tests)
- [x] Unit tests for `ExternalSpec` parser
- [x] Unit tests for paramtable config
- [x] Integration test with real Parquet files
- [x] `make lint-fix` passes
- [ ] E2E test with MinIO backend
---------
Signed-off-by: Jiquan Long <jiquan.long@zilliz.com>
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
## Summary
- Add `.claude/` to `.gitignore` to prevent Claude Code runtime files
(settings.local.json, memory/, projects/) from being accidentally
committed
Signed-off-by: Li Liu <li.liu@zilliz.com>
## Summary
- Add `CLAUDE.md` with project-specific instructions for Claude Code
(architecture, testing, code conventions, commit guidelines)
- Remove `claude.md` from `.gitignore` to allow tracking
Signed-off-by: Li Liu <li.liu@zilliz.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
## Summary
- Enable search by ids tests that were commented out pending fix of
#47065
- Fix syntax error in test: `for i in len()` → `for i in range(len())`
- Update test expectations to match the fixed behavior:
- Search on null vectors now returns empty results instead of error
- Fix assertion to use correct `num_entities_with_not_null_vector` count
## Test plan
- [x] `test_milvus_client_add_nullable_vector_field_search` - verified
passing
- [x] `test_milvus_client_collection_null_vector_field_search` -
verified passing
issue: #47065🤖 Generated with [Claude Code](https://claude.ai/code)
---------
Signed-off-by: Yan Liang <yanliang@zilliz.com>
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
issue: #47285
Fix critical DoS vulnerability where filter expressions containing
division or modulo by zero operations cause Milvus server to crash with
SIGFPE .
Any client with query access could crash the server remotely using
expressions like `"int_field / {d} == 1", params:{"d": 0}`.
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
issue: https://github.com/milvus-io/milvus/issues/27467
>My plan is as follows.
>- [x] M1 Create collection with timestamptz field
>- [x] M2 Insert timestamptz field data
>- [x] M3 Retrieve timestamptz field data
>- [x] M4 Implement handoff
>- [x] M5 Implement compare operator
>- [x] M6 Implement extract operator
>- [x] M8 Support database/collection level default timezone
>- [x] M7 Support STL-SORT index for datatype timestamptz
---
The third PR of issue: https://github.com/milvus-io/milvus/issues/27467,
which completes M5, M6, M7, M8 described above.
## M8 Default Timezone
We will be able to use alter_collection() and alter_database() in a
future Python SDK release to modify the default timezone at the
collection or database level.
For insert requests, the timezone will be resolved using the following
order of precedence: String Literal-> Collection Default -> Database
Default.
For retrieval requests, the timezone will be resolved in this order:
Query Parameters -> Collection Default -> Database Default.
In both cases, the final fallback timezone is UTC.
## M5: Comparison Operators
We can now use the following expression format to filter on the
timestamptz field:
- `timestamptz_field [+/- INTERVAL 'interval_string'] {comparison_op}
ISO 'iso_string' `
- The interval_string follows the ISO 8601 duration format, for example:
P1Y2M3DT1H2M3S.
- The iso_string follows the ISO 8601 timestamp format, for example:
2025-01-03T00:00:00+08:00.
- Example expressions: "tsz + INTERVAL 'P0D' != ISO
'2025-01-03T00:00:00+08:00'" or "tsz != ISO
'2025-01-03T00:00:00+08:00'".
## M6: Extract
We will be able to extract sepecific time filed by kwargs in a future
Python SDK release.
The key is `time_fields`, and value should be one or more of "year,
month, day, hour, minute, second, microsecond", seperated by comma or
space. Then the result of each record would be an array of int64.
## M7: Indexing Support
Expressions without interval arithmetic can be accelerated using an
STL-SORT index. However, expressions that include interval arithmetic
cannot be indexed. This is because the result of an interval calculation
depends on the specific timestamp value. For example, adding one month
to a date in February results in a different number of added days than
adding one month to a date in March.
---
After this PR, the input / output type of timestamptz would be iso
string. Timestampz would be stored as timestamptz data, which is int64_t
finally.
> for more information, see https://en.wikipedia.org/wiki/ISO_8601
---------
Signed-off-by: xtx <xtianx@smail.nju.edu.cn>
issue: #41621
- Deprecate EnableWorkerSQCostMetrics parameter
- Always collect cost metrics from all search and retrieve results
- Update code with comments explaining the changes rationale
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
By recording the active collection lists, The l0 compaction trigger
of view change and idle won't influence each other.
Also this pr replaces the L0View cache with real L0 segments' change.
Save some memory and make L0 compaction triggers more accurate.
See also: #39187
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
issue: #36621
* The front-end code of the management interface has been refactored
using React.
* The interaction and display of the front-end interface have been
optimized.
Signed-off-by: mimo.oyn <ning.ouyang@zilliz.com>
Co-authored-by: zilliz <zilliz@zillizdeMacBook-Pro.local>
issue: #33285
- make message builder and message conversion type safe
- add adaptor type and function to adapt old msgstream msgpack and
message interface
---------
Signed-off-by: chyezh <chyezh@outlook.com>
Now it's able to start multiple datanode, however, they will process the same insert data.
Further changes are needed to truly enable multiple datanode:
While registering, dataservice need to return different insert channels for different datanode.
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
