Image to image with gemini-2.0-flash-preview-image-generation

[tpaulshippy]

What this does

Enable image-to-image generation with gemini-2.0-flash-preview-image-generation

Type of change

• New feature

Scope check

• I read the Contributing Guide
• This aligns with RubyLLM's focus on LLM communication
• This isn't application-specific logic that belongs in user code
• This benefits most users, not just my specific use case

Quality check

• I ran overcommit --install and all hooks pass
• I tested my changes thoroughly
• I updated documentation if needed
• I didn't modify auto-generated files manually (models.json, aliases.json)

API changes

• New public methods/classes

Related issues

Screenshots

Here's what the test did.

Input
put this in a ring

Output

Second input
'change the background to blue'

Second output

[tpaulshippy]

Thinking I need to move to content with attachments so the image gets sent properly on the next call.

[crmne]

I know it's a draft and you mentioned it in a comment but you shouldn't add an images attribute to the Message object since we have the Content object for a reason.

[tpaulshippy]

I realize this is a very different approach than the RubyLLM.paint method as it involves generating images within a chat. I do think it has some value however, as it allows for multimodal conversations.

This has similar value as #152 but there is a bit of a clash as this introduces an ImageAttachment that is provider agnostic (although only used in Gemini so far) while that PR has a ImageAttachments class that is OpenAI specific.

I also am not sure exactly how/where to document this in the guides.

@crmne Looking forward to your feedback/thoughts.

[tpaulshippy]

This document describes the two approaches pretty well I think. I could see an implementation of Imagen in RubyLLM that looks more like the #152 approach.

It looks like OpenAI supports conversational image generation through the responses API and a built in tool called "image_generation" - see here.

[tpaulshippy]

I like how OpenAI allows you to reference the previous images via IDs. We really need to get support for these built-in tools via the responses API into RubyLLM. We are already doing it in a fork to get web_search_preview (see diff here) but it's pretty messy.