fix(vertexai-batch): fix vertexai batch file format #13576
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Title
There's a logic inside managed files that replaces the model name inside the JSONL when using batch. This logic returns an
InMemoryFile, which has a filename and content on it. The vertexai transformer (that turns an openai file into a valid vertexai file) only runs the transformation if the content type of the file being received isapplication/jsonl. However, because we are turning the file into anInMemoryFileand the fact thatapplication/jsonlis not an actual mimetype, it was setting the content type as octet-stream, and the vertexai transformer was not modifying the file.Relevant issues
Fixes #11080
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unitType
🐛 Bug Fix
Changes
Set the content type on the InMemoryFile to application/jsonl and handle this type of file inside the extract_data logic so we can set the actual values on it.