Add RubyLLM::Chat#with_params to add custom parameters to the underlying API payload

[compumike]

What this does

Implements with_request_options (renamed from with_options due to ActiveRecord conflict -- see conversation) with @crmne's suggestions from comment #130 (review) and tested against all providers.

This allows users to set arbitrary options on the payload before it's sent to the provider's API endpoint. The render_payload takes precedence.

Demo:

chat = RubyLLM
  .chat(model: "qwen3", provider: :ollama)
  .with_request_options(response_format: {type: "json_object"})
  .with_instructions("Answer with a JSON object with the key `result` and a numerical value.")
response = chat.ask("What is the square root of 64?")
response.content
=> "{\n  \"result\": 8\n}"

This is a power-user feature, and is specific to each provider (and model, to a lesser extent). I added a brief section to the docs.

For tests: different providers supported different options, so tests are divided by provider.

(Note that deep_merge is required for Gemini in particular because it relies on a top-level generationConfig object.)

Type of change

• Bug fix
• New feature
• Breaking change
• Documentation
• Performance improvement

Scope check

• I read the Contributing Guide
• This aligns with RubyLLM's focus on LLM communication
• This isn't application-specific logic that belongs in user code
• This benefits most users, not just my specific use case

Quality check

• I ran overcommit --install and all hooks pass
• I tested my changes thoroughly
• I updated documentation if needed
• I didn't modify auto-generated files manually (models.json, aliases.json)

API changes

• Breaking change
• New public methods/classes
• Changed method signatures
• No API changes

Related issues

• Support options on completion calls #130
• Structured Output & JSON mode response support #131
• Passing 'custom' options #221

[elvinaspredkelis]

What an elegant PR!

Quick note - I've been playing around with this. Seems like this would also need to allow disabling the temperature option.

For example, the APIs break once trying to supply the web search options as the gpt-4o-search-preview does not support temperature: Model incompatible request argument supplied: temperature.

This may not be exactly related to this particular PR, but I got to this obstacle when playing around with this.

EDIT

The #with_options interfares with ActiveRecord. Perhaps it'd also make sense to explore other method names? E.g. #with_request_options

[compumike]

Quick note - I've been playing around with this. Seems like this would also need to allow disabling the temperature option.

For example, the APIs break once trying to supply the web search options as the gpt-4o-search-preview does not support temperature: Model incompatible request argument supplied: temperature.

Thanks. Can you try with with_temperature(nil) and see if it works for you? And if not, can you provide a code snipped that caused the Model incompatible request argument supplied error? This quick test worked for me:

chat = RubyLLM
  .chat(model: "gpt-4o-search-preview", provider: :openai)
  .with_temperature(nil)
  .with_options(web_search_options: {search_context_size: "medium"})
chat.ask("<your query here>")

There's already a comment in:

ruby_llm/lib/ruby_llm/providers/openai/chat.rb

Line 21 in a35aa11

# Only include temperature if it's not nil (some models don't accept it)

that the temperature is not added to the payload if you do so. (Possibly this nil handling behavior should be extended to other providers as well?)

I don't think we necessarily want to make this PR any more complicated, but we could do something like this:

Future proposal: fully general block passing

It's possible that in addition to this #with_options, we may want the ability to pass in a block that allows modifying the payload after it's deep_merged but before it's actually sent. For example, this might look like:

chat = RubyLLM
  .chat(model: "gpt-4o-search-preview", provider: :openai)
  .with_options(web_search_options: {search_context_size: "medium"}) do |payload|
    payload.delete(:temperature)
  end
chat.ask(...)

Not necessary in this case, since with_temperature(nil) works. But allows future customization for power users.

But I would personally prefer to merge this PR as-is and discuss the block concept later if a need arises.

[elvinaspredkelis]

Right, my bad!

It's possible to set the temperature to nil with gem "ruby_llm", "1.3.1" and not gem "ruby_llm", "1.3.0".

Just a heads up - it still conflicts with the #with_options that comes with ActiveRecord. So the Rails integration is not as smooth.

[compumike]

Just a heads up - it still conflicts with the #with_options that comes with ActiveRecord. So the Rails integration is not as smooth.

Makes sense. Happy to rename it. #with_request_options has a nice ring to it! 😄

[umairabid]

Can't wait for this to be merged thank you @compumike

[compumike]

I've just pushed a new commit which renames this to #with_request_options to avoid the ActiveRecord conflict that @elvinaspredkelis mentioned!

Please give this a try and let me know if it works for your use case.

Also, if @elvinaspredkelis and @umairabid and others could add a comment specifying what your use case is, and providing a code snippet of how #with_request_options enables that use case, that would be great! ❤️

[umairabid]

hey @compumike tried it here for gemini umairabid/sweaty_wallet@fa8c488, worked flawlessly

[adenta]

Can confirm this works well for my use case.

[bricolage]

Also works for me, this makes RubyLLM versatile for more use cases

[minorbug]

I need to be able to send metadata along with my RubyLLM calls so I can add tracing that makes it to Langfuse

[lucasfcunha]

This is very useful, seems like a no-brainer to support sending custom params to the LLM.

[thedumbtechguy]

+1 on this. A very useful feature that would solve the need for custom params for every thing.

Thank you.

[codecov]

Codecov Report

Attention: Patch coverage is 80.00000% with 3 lines in your changes missing coverage. Please review.

Project coverage is 89.65%. Comparing base (0f60067) to head (3cf7943).
Report is 3 commits behind head on main.

Files with missing lines	Patch %	Lines
lib/ruby_llm/active_record/acts_as.rb	33.33%	2 Missing ⚠️
lib/ruby_llm/provider.rb	85.71%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #265      +/-   ##
==========================================
- Coverage   89.71%   89.65%   -0.07%     
==========================================
  Files          75       75              
  Lines        2811     2823      +12     
  Branches      555      557       +2     
==========================================
+ Hits         2522     2531       +9     
- Misses        289      292       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[crmne]

Hey @compumike really great PR!

Since we have to change from with_options, I would instead call this method with_params and here's why:

1. It's idiomatic: it's ubiquitous in the Rails and wider web community.
2. It's concise: it's a single clear word that's much tighter than with_request_options
3. It's unambiguous: in the context of an API client, with_params clearly signals that we're adding parameters to the outgoing request body.

[compumike]

@crmne Great! I've just pushed a commit that renames the method to #with_params. Thank you for all your work on RubyLLM! ❤️

[tpaulshippy]

Maybe rename the PR?

[compumike]

@tpaulshippy Good catch 😂

[crmne]

Flawless! Amazing work.