-
Notifications
You must be signed in to change notification settings - Fork 4.4k
.Net: [MEVD] Support .Any(x => x.Contains(...)) in filters #13471
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This pull request adds support for the .Any(x => x.Contains(...)) filter pattern in vector store implementations, addressing issue #13152. This enables filtering on array/list properties to check if any element in a column's array matches any value in a provided list.
Changes:
- Adds
TranslateAnymethod to filter translators across all vector store providers (PostgreSQL, SQL Server, Cosmos NoSQL, Azure AI Search, Weaviate, Redis, Qdrant, SQLite) - Implements proper identifier quoting/escaping across SQL-based implementations (from PR #13470)
- Adds storage name validation for providers that don't support identifier escaping (Weaviate, Redis, Azure AI Search)
- Adds comprehensive conformance tests for the new
.Any()functionality
Reviewed changes
Copilot reviewed 26 out of 26 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| FilterTests.cs | Adds base conformance tests for Any with Contains over inline, captured arrays and lists |
| SqliteFilterTests.cs, RedisFilterTests.cs, PineconeFilterTests.cs, MongoFilterTests.cs | Override new tests to expect exceptions for unsupported array operations |
| SqlFilterTranslator.cs | Base implementation of TranslateAny for SQL-based providers, handling inline arrays, captured variables, and MemoryExtensions.Contains |
| PostgresFilterTranslator.cs | Postgres-specific implementation using && operator for array overlap |
| SqlServerFilterTranslator.cs | SQL Server implementation using OPENJSON with EXISTS |
| CosmosNoSqlFilterTranslator.cs | Cosmos NoSQL implementation using EXISTS with subquery |
| AzureAISearchFilterTranslator.cs | Azure AI Search implementation using OData's search.in with any() lambda |
| WeaviateFilterTranslator.cs | Weaviate implementation using ContainsAny operator |
| RedisFilterTranslator.cs | Redis implementation using tag field syntax with pipe delimiters |
| QdrantFilterTranslator.cs | Qdrant implementation using Match with RepeatedStrings/RepeatedIntegers |
| PostgresSqlBuilder.cs | Enhanced with proper identifier quoting using AppendIdentifier |
| SqlServerCommandBuilder.cs | Enhanced with AppendIdentifier for bracket-escaped identifiers |
| SqliteCommandBuilder.cs | Enhanced with AppendIdentifier for double-quote escaped identifiers |
| CosmosNoSqlCollectionQueryBuilder.cs | Enhanced with JSON property name escaping in bracket notation |
| WeaviateModelBuilder.cs, RedisModelBuilder.cs, AzureAISearchModelBuilder.cs | Added storage name validation to ensure valid identifiers for non-escapable contexts |
| Various test files | Updated expectations for properly quoted identifiers in SQL generation |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
NOTE: This PR is based on top of #13470, review 2nd commit only
Closes #13152