issue: #50615
## What this PR does
Makes the C++ core build cleanly on **macOS (Apple Silicon) with
Homebrew LLVM/clang 19**.
> [!IMPORTANT]
> **Scope / impact is NOT macOS-only.** Although this PR is framed as
macOS/clang-19 enablement, two of the changes are **top-level
`requires(...)` in `internal/core/conanfile.py`, so they apply to every
platform, including Linux production builds:**
> - **`azure-sdk-for-cpp` `1.11.3@milvus/dev` → `1.16.0@milvus/dev`** (5
minor versions, `force=True`). This is the SDK Arrow uses for the
**Azure Blob remote-storage path**, so this is effectively a major Azure
SDK bump on the **production Azure object-storage path for Linux
deployments** too.
> - **`arrow` recipe `17.0.0@milvus/dev-2.6#c743ea7a…` →
`17.0.0@milvus/dev#f411aa73…`** — same Arrow version, but **rebuilt
against azure 1.16.0** with a patched `FindAzure.cmake`. The Arrow
**`package_id` (recipe revision) DID change**; it is not the same binary
as before.
> - **`libbson/1.30.6@milvus/dev`** added (also global, but new — no
pre-existing behavior to regress).
>
> **Testing blind spot:** CI green proves compile/link/UT/e2e pass, but
the UT and integration suites **do not exercise real Azure Blob
read/write**, so the Azure object-storage path is **not covered** by
this PR's CI. A real Azure Blob remote-storage regression is recommended
before merge (see Verification).
### Why the dependency bumps are required for clang 19
- **azure 1.16.0**: azure 1.11.3 bundles an older `nlohmann/json` that
uses `std::char_traits<unsigned char>`, which was **removed in libc++
19** → 1.11.3 fails to compile on clang 19. 1.16.0 fixes this.
- **arrow rebuilt against azure 1.16.0**: when Arrow is built from
source on clang 19 it must link the same azure 1.16.0; the recipe also
patches `FindAzure.cmake` to the monolithic `Azure` config so
`find_package(Azure)` resolves. Hence a new Arrow recipe revision.
### Toolchain / build enablement
- **`scripts/setenv.sh`**: probe LLVM 19/20/21 first (libc++ 17/18
`chrono operator<<` clashes with Arrow's vendored `date`; libc++ 19
fixes it).
- **`scripts/3rdparty_build.sh`**: arm64 `-march=armv8-a+crypto+crc` so
folly's F14 (SimdAndCrc on Apple Silicon) matches core/knowhere — fixes
`F14LinkCheck` undefined-symbol link errors.
- **`InvertedIndexTantivy.cpp`**: drop spurious `->template` on
non-template members (clang 19
`-Wmissing-template-arg-list-after-template-kw`).
### Drop bsoncxx / libmongoc → libbson only
The BSON layer used the **bsoncxx** C++ driver, whose CMake
unconditionally builds mongo-cxx-driver → **libmongoc** → bundled
**utf8proc**, which fails to configure under newer CMake/clang on macOS.
Milvus only needs BSON (de)serialization.
- New Conan package **`libbson/1.30.6@milvus/dev`** (libbson-only,
`ENABLE_MONGOC=OFF`; recipe published to milvus conanfiles &
production).
- New **`src/common/bson_shim.h`** — a thin `milvus::bson` C++ layer
over libbson's C API.
- Migrated `bson_view.h`, `bson_builder.{h,cpp}`, json_stats consumers
and unit tests off bsoncxx.
- CMake: `find_package(bson-1.0)` + link `mongo::bson_shared`; removed
FetchContent `thirdparty/bsoncxx`.
All three global Conan requires (arrow / libbson / azure) pin an
explicit recipe `#revision` for reproducibility.
## Verification
- Full clean rebuild on macOS arm64 / clang 19: `bin/milvus` builds;
`libmilvus_core` links **only `libbson-1.0.dylib`** (no
libmongoc/mongocxx/utf8proc).
- libbson / arrow / azure resolve from production Conan at the pinned
revisions.
- All 21 BSON unit tests (BsonView / BsonBuilder / DomNode) pass.
- Full ci-v2 suite green on Linux (build / build-ut-cov / ut-cpp / ut-go
/ integration / e2e / go-sdk).
- **TODO before merge:** real Azure Blob remote-storage read/write
regression to cover the azure-1.16.0 / arrow-rebuild blast radius (not
exercised by UT/integration).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Signed-off-by: xiaofanluan <xf@hjjaq.com>
Co-authored-by: xiaofanluan <xf@hjjaq.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
## Summary
- macOS `ld` treats each `-r` flag as a single directory path, unlike
Linux `ld` which accepts colon-separated paths
- Since d69bdd288c added `cmake_build/lib` to `DYLD_LIBRARY_PATH`,
`RPATH=$DYLD_LIBRARY_PATH` produces a colon-separated value like
`path1:path2`
- The Makefile's `-r $RPATH` embeds this as a single invalid path in the
binary, causing `Library not loaded: @rpath/libmilvus-storage.dylib` at
runtime
- Fix: emit two separate `-r` flags so the Makefile expands to `-r path1
-r path2`
## Test plan
- [x] Verified `RPATH` variable output: `path1 -r path2` (correct)
- [x] Verified binary loads correctly after fix (`otool -l` shows two
separate `LC_RPATH` entries)
- [x] Verified Milvus standalone starts and serves requests successfully
issue: #47809
Signed-off-by: sijie-ni-0214 <sijie.ni@zilliz.com>
On Linux, setenv.sh accidentally sets LD_LIBRARY_PATH and RPATH to look
up shared objects in the current working directory first if nothing
explicitly set an LD_LIBRARY_PATH already, because an empty path on
either variable is interpreted as the current working directory. This
results in races during parallel builds (e.g. tools invoked by Conan
finding corrupt shared libraries, because said libraries are currently
compiled and written out by another process in the build). Since the
LD_LIBRARY_PATH is also used as the RPATH, the compiled Milvus binary
also looks up shared objects in the current working directory,
potentially picking up unintended versions of a shared object.
Fix this and similar "empty path" problems for other environment
variables on Linux, macOS, and Windows by placing the path separator
only if there are pre-existing paths.
issue: #47828
Signed-off-by: Markus Boehme <markus.boehme@chainguard.dev>
issue: #41609
- add env `MILVUS_NODE_ID_FOR_TESTING` to set up a node id for milvus
process.
- add env `MILVUS_CONFIG_REFRESH_INTERVAL` to set up the refresh
interval of paramtable.
- Init paramtable when calling `paramtable.Get()`.
- add new multi process framework for integration test.
- change all integration test into multi process.
- merge some test case into one suite to speed up it.
- modify some test, which need to wait for issue #42966, #42685.
- remove the waittssync for delete collection to fix issue: #42989
---------
Signed-off-by: chyezh <chyezh@outlook.com>
