Repository navigation
feat(weaviate): multi-tenancy, search controls, self-hosted connections and quantization - #8324
Duda Nogueira (dudanogueira) wants to merge 4 commits into
Conversation
Strix Security ReviewWarning This pull request has 222 commits after the last Strix review ( No security issues found. Review summaryReviewed the full diff, focusing on the single production code change in The new query path is safe: the only agent-supplied value is Multi-tenancy scoping holds up: Static analysis on the changed file ( Updated for Reviewed by Strix |
59188ac to
a1e723a
Compare
Preview approval requiredThis pull request comes from a fork, so Mintlify won't build a preview until someone with write access approves commit |
Weaviate reads `X-Weaviate-Client-Integration` (since server 1.38.0) to attribute
traffic to the integration that produced it. The tool sent nothing, so CrewAI was
invisible in that telemetry.
Registration goes through `client.integrations.configure` rather than the `headers=`
argument, because that applies the header to the gRPC transport as well as REST --
searches travel over gRPC. It is best-effort: `_IntegrationConfig` is private API, so
a `weaviate-client` without it skips registration instead of breaking the tool.
`MongoDBVectorSearchTool` already identifies CrewAI to its database the same way, via
`DriverInfo(name="CrewAI", version=version("crewai-tools"))`.
Verified against Weaviate 1.39.5: the header arrives on both transports as
`crewai-tools/<version>`.
Adds the first tests for this tool.
Raises the floor from 4.10.2 to the current release. The lock previously resolved two versions (4.16.2 and 4.21.3) depending on the Python marker; it now resolves 4.23.1 everywhere, which also drops the unused grpcio-health-checking. 4.23.1 still ships `weaviate.connect.integrations._IntegrationConfig`, so the integration header registration is unchanged; checked live against Weaviate 1.39.5 (header present on REST and gRPC).
…ns and quantization WeaviateVectorSearchTool could only run a fixed hybrid search against a Weaviate Cloud cluster, and required an OpenAI key even when the collection did not vectorize with OpenAI. - Connection: `weaviate_cluster_url` and `weaviate_api_key` are now optional. Without a cluster URL the tool connects to a self-hosted Weaviate (`weaviate_host`, ports, `weaviate_secure`). `OPENAI_API_KEY` is optional and is merged into `headers` instead of replacing them. - Multi-tenancy: `tenant` scopes every search to that tenant. It is set on the tool, not in the agent's args schema, so an agent cannot pick another tenant. - Search controls: `search_type` (hybrid / near_text / bm25), `target_vector`, `return_properties` and `filters`. - Collection creation: the old create-if-missing branch never ran, because `collections.get` always returns a truthy handle. It is now opt-in with `create_collection_if_missing`, so a typo in the collection name can't silently create an empty collection. `vectorizer` takes `Configure.Vectors.*`, so the quantizer (RQ/SQ/BQ/PQ) and index config apply at creation. Legacy `Configure.Vectorizer.*` still works. A new multi-tenant collection gets its tenant created, because `auto_tenant_creation` only applies on insert. The run that creates a collection returns no results, since Weaviate rejects searching a collection that has no properties yet. - The default vectorizer was `text2vec_openai(model="nomic-embed-text")`, which is not an OpenAI model. It is now `text-embedding-3-small`. - The client is closed even when the search raises. Docs updated in en, ar, ko and pt-BR.
a1e723a to
ad82940
Compare
Related issue
Fixes #8323
Summary
WeaviateVectorSearchToolcould only run a fixed hybrid search against Weaviate Cloud, and it requiredOPENAI_API_KEYeven when the collection didn't vectorize with OpenAI.weaviate_cluster_urlandweaviate_api_keyare now optional. Without a cluster URL, the tool connects to a self-hosted Weaviate (weaviate_host,weaviate_http_port,weaviate_grpc_port,weaviate_secure). The OpenAI key is optional, and it is merged intoheadersinstead of replacing them.tenantscopes every search to that tenant. It is set on the tool and isn't in the agent's args schema, so an agent can't switch to another tenant.search_type(hybrid/near_text/bm25),target_vector,return_propertiesandfilters.collections.getalways returns a truthy handle. It is now opt-in withcreate_collection_if_missing, so a typo incollection_namecan't silently create an empty collection.vectorizertakesConfigure.Vectors.*, so the quantizer (RQ/SQ/BQ/PQ) and vector index config apply at creation. LegacyConfigure.Vectorizer.*objects still work and go tovectorizer_config.auto_tenant_creationonly applies on insert, so without this the first search fails.nomic-embed-text, which isn't an OpenAI model. It is nowtext-embedding-3-small.Behaviour changes for existing users:
OPENAI_API_KEYno longer raises.Things to look at in review:
_vector_config_kwargsuses anisinstancecheck against the private_VectorizerConfigCreateto support legacy configs.Verification
Tests added or updated for the changed behavior
Relevant tests and quality checks pass locally
24 new mocked tests, 39 in the file. They cover connection selection, the optional OpenAI key, tenant scoping and isolation from the args schema, all three search types with their options, and creation: off by default, opt-in, legacy vs
Configure.Vectors, the quantizer reaching the config, and multi-tenant collections with their tenant.The full
crewai-toolssuite passes (1077 passed, 9 skipped).ruffandmypyare clean on the changed files.Checked end to end against Weaviate 1.39.5 (self-hosted, no OpenAI key):
nomic-embed-text+ SQ and an Ollama generative model, the tool created the multi-tenant collection and its tenant.hybrid,near_text,bm25,return_propertiesandfiltersall returned the expected objects.Additional context
Docs updated in en, ar, ko and pt-BR.
tool.specs.jsonis left to the generate-tool-specs workflow.Note
Medium Risk
Behavior changes (optional OpenAI key, no implicit collection creation, tenant scoping) can break existing Weaviate Cloud setups; multi-tenant and auto-create paths touch live database state when enabled.
Overview
WeaviateVectorSearchToolis expanded from hybrid-only Weaviate Cloud searches into a configurable Weaviate client: optional cloud vs self-hosted connections, optionalOPENAI_API_KEY, and best-effortX-Weaviate-Client-Integrationtelemetry (bumpsweaviate-clientto ≥4.23.1).Search behavior adds
search_type(hybrid/near_text/bm25),target_vector,return_properties, andfilters.tenantis fixed on the tool (not agent args) for multi-tenant isolation. Collection setup is opt-in viacreate_collection_if_missing, withConfigure.Vectors(quantization) while legacy vectorizers still map tovectorizer_config; default embeddings switch totext-embedding-3-small.Docs are updated in en/ar/ko/pt-BR, and a large mocked test suite covers connection, tenancy, search modes, creation, and integration registration.
Reviewed by Cursor Bugbot for commit a1e723a. Bugbot is set up for automated code reviews on this repo. Configure here.