ES|QL - Add vector similarity functions to tech preview #139365
Conversation
…S_PUSHDOWN capability and use it on specific functions
carlosdelest
added
Team:Analytics
|
Hi @carlosdelest, I've created a changelog YAML for you. |
| ); | ||
| } | ||
|
|
||
| public void testVCosine() throws IOException { |
There was a problem hiding this comment.
Vector similarity functions tests have been moved to PushVectorSimilarityToLoadIT, as they don't need the check now. We should merge them together again when pushing down is not behind snapshot.
|
|
||
| @Override | ||
| public PushedBlockLoaderExpression tryPushToFieldLoading(SearchStats stats) { | ||
| if (EsqlCapabilities.Cap.BLOCK_LOADER_EXPRESSIONS_PUSHDOWN.isEnabled() == false) { |
There was a problem hiding this comment.
Checks pushdown capability on the pushdown for all the non-vector similarity functions
| def(Magnitude.class, Magnitude::new, "v_magnitude"), | ||
| def(Hamming.class, Hamming::new, "v_hamming") } }; | ||
| // dense vector functions | ||
| def(Magnitude.class, Magnitude::new, "v_magnitude")} }; |
There was a problem hiding this comment.
V_MAGNITUDE is still not pushed down - #137535 tracks that
| new InferIsNotNull(), | ||
| new InferNonNullAggConstraint(), | ||
| new ReplaceDateTruncBucketWithRoundTo(), | ||
| new PushExpressionsToFieldLoad() |
There was a problem hiding this comment.
Add the PushExpressionsToFieldLoad rule by default - now the BlockLoaderExpression.tryPushToFieldLoading method needs to take into account pushdown capabilities
🔍 Preview links for changed docs |
ℹ️ Important: Docs version tagging👋 Thanks for updating the docs! Just a friendly reminder that our docs are now cumulative. This means all 9.x versions are documented on the same page and published off of the main branch, instead of creating separate pages for each minor version. We use applies_to tags to mark version-specific features and changes. Expand for a quick overviewWhen to use applies_to tags:✅ At the page level to indicate which products/deployments the content applies to (mandatory) What NOT to do:❌ Don't remove or replace information that applies to an older version 🤔 Need help?
|
elasticsearchmachine
removed
the
Team:Analytics
|
Pinging @elastic/es-search-relevance (Team:Search Relevance) |
| checkVectorFunctionsNullArgs("v_magnitude(null)"); | ||
| if (EsqlCapabilities.Cap.MAGNITUDE_SCALAR_VECTOR_FUNCTION.isEnabled()) { | ||
| checkVectorFunctionsNullArgs("v_magnitude(null)"); | ||
| } | ||
| if (EsqlCapabilities.Cap.HAMMING_VECTOR_SIMILARITY_FUNCTION.isEnabled()) { |
There was a problem hiding this comment.
I think something is not right here - shouldn't v_magnitude tests still be skipped for non snapshot builds and the v_hamming ones be executed without a cap check?
3 participants
Get the following vector similarity functions into release builds:
V_DOT_PRODUCT
V_COSINE
V_L1_NORM
V_L2_NORM
V_HAMMING
Vector similarity functions use function pushdown to blockloaders - see #137002
Other functions that use pushdown to blockloaders have been set behind a new capability so they will be pushed when all pending pushdown issues are addressed - see #137679 for this.