-
Notifications
You must be signed in to change notification settings - Fork 103
Add Sparse Vector Index Options for Token Pruning to Specification #4761
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
# Conflicts: # output/schema/schema.json
47f8758
to
c3b758e
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall LGTM but one question about pruning config
output/typescript/types.ts
Outdated
@@ -6022,9 +6022,15 @@ export interface MappingSourceField { | |||
|
|||
export type MappingSourceFieldMode = 'disabled' | 'stored' | 'synthetic' | |||
|
|||
export interface MappingSparseVectorIndexOptions { | |||
prune?: boolean | |||
pruning_config?: QueryDslTokenPruningConfig |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we have both QueryDslTokenPruningConfig
and TokenPruningConfig
? Can we consolidate?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't know - this file is one of the ones automatically generated when running make contrib
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmmm... I'm pretty sure it added the QueryDsl*
prefix as it imports the TokenPruningConfig from there ('@_types/query_dsl/TokenPruningConfig')... @kderusso - WDYT about moving the TokenPruningConfig to a more common place? I don't think it would break backwards compatibility (as the final output structure would be the same)...?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, thanks!
The backport to
To backport manually, run these commands in your terminal: # Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-8.19 8.19
# Navigate to the new working tree
cd .worktrees/backport-8.19
# Create a new branch
git switch --create backport-4761-to-8.19
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 0c77ae9561b54c103dd102de6cde777841504596
# Push it to GitHub
git push --set-upstream origin backport-4761-to-8.19
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-8.19 Then, create a pull request where the |
The backport to
To backport manually, run these commands in your terminal: # Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-9.1 9.1
# Navigate to the new working tree
cd .worktrees/backport-9.1
# Create a new branch
git switch --create backport-4761-to-9.1
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 0c77ae9561b54c103dd102de6cde777841504596
# Push it to GitHub
git push --set-upstream origin backport-4761-to-9.1
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-9.1 Then, create a pull request where the |
Index options for token pruning configurations were introduced for sparse vector field types in 8.19/9.1 via this PR. This PR adds the corresponding specification definitions for this.