[Bug]: ollama_chat erroring for normal image input format

[isaacbmiller]

What happened?

The ollama_chat provider does not accept images in the normal content format where you have arrays of both content: text and content: image.

It expects messages to be passed as a list of:

{
  "role":
  "content":
  "images":
}

objects, which is different from the typical image input format.

Documentation

Initial litellm bug report: #6808
Initial litellm PR with a solution: #6880
DSPy PR with fix: stanfordnlp/dspy#8582

cc @chenmoneygithub

Relevant log output

litellm.llms.ollama.common_utils.OllamaError: {"error":"json: cannot unmarshal array into Go struct field ChatRequest.messages.content of type string"}

Are you a ML Ops Team?

No

What LiteLLM version are you on ?

v1.72.6

Twitter / LinkedIn details

No response