Skip to content

Add Cohere Provider support #275

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 23 commits into
base: main
Choose a base branch
from

Conversation

infinityrobot
Copy link
Contributor

@infinityrobot infinityrobot commented Jul 7, 2025

What this does

This PR adds a new Provider for Cohere and adds 21 Cohere models across multiple model families that support chat, embeddings and vision, including:

Implementation overview

Added a new RubyLLM::Providers::Cohere module that follows existing Provider patterns with API key configuration and Capabilities, Chat, Embeddings, Media, Models, Streaming and Tools modules.

All features have full test coverage with VCRs. Relevant documentation has been updated.

Key highlights enabled by adding Cohere as a Provider:

  • Chat completion with streaming, tool and tool support with Command and Aya models
  • Text embeddings with Embed models, including multilingual and image support (to be included in future contribution)
  • Vision capabilities with Aya Vision models
  • Ability to use fine-tuned models using Cohere's platform (leveraging assume_model_exists: true)

Note

I have opted to use Cohere's native API endpoints rather than their OpenAI-compatible interface so we have a more future-proof implementation and the optionality to support Cohere specific features across chat (documents, citations, safety mode, etc.), embeddings (input types, image embeddings, etc.) and other features like reranking and classification.

Usage example

Usage follows the usual RubyLLM::Providers patterns.

# Configure Cohere API key
RubyLLM.configure do |config|
  config.cohere_api_key = ENV['COHERE_API_KEY']
end

# Chat with Command models
chat = RubyLLM.chat(model: "command-a-03-2025")
chat.ask("Why is Ruby an incredible programming language?")

# Generate embeddings
RubyLLM.embed("Ruby", model: "embed-v4.0")

# Vision capabilities
chat = RubyLLM.chat(model: "c4ai-aya-vision-32b")
chat.ask("What is in this image?", with: { image: "spec/fixtures/ruby.png"  })

Other changes / bug fixes

  • Fixed a bug in generating models.json due to Parsera returning models with a nil id.
  • Refactored spec/ruby_llm/embeddings_spec.rb to use an EMBEDDINGS_MODELS constant defined in spec/spec_helper.rb to align specs with chat_spec.rb and handle per-model dimensions. The 768 previously hardcoded is invalid for Cohere models

Future plans

Note – this PR is the first in a series I intend to contribute. In the future I would love to also add support for the below features that I think would be useful to other RubyLLM users if you are open to it @crmne!

Type of change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation
  • Performance improvement

Scope check

  • I read the Contributing Guide
  • This aligns with RubyLLM's focus on LLM communication
  • This isn't application-specific logic that belongs in user code
  • This benefits most users, not just my specific use case

Quality check

  • I ran overcommit --install and all hooks pass
  • I tested my changes thoroughly
  • I updated documentation if needed
  • I didn't modify auto-generated files manually (models.json, aliases.json)

API changes

  • Breaking change
  • New public methods/classes
  • Changed method signatures
  • No API changes

Related issues

No related issues.

@infinityrobot
Copy link
Contributor Author

infinityrobot commented Jul 14, 2025

@crmne, I'd love your thoughts on this when you get a moment. I also have working Provider implementations for Google Vertex and xAI so would love to add those once I know I've got the patterns right.

@crmne crmne added the new provider New provider integration label Jul 16, 2025
@Eric-Guo
Copy link
Contributor

I think as robot, it's much more polite to declare first, you are not human.

@infinityrobot
Copy link
Contributor Author

infinityrobot commented Jul 18, 2025

Hi @Eric-Guo – I've updated the PR to be in line with main and updated a few spots where logic had changed due to #267 and #271 being merged.

Apologies, I'm not sure what you meant by the robot comment. I am a developer based in Sydney and given it was my first contribution to the repo and relatively new to OS contributions I had made a lot of effort to follow the contribution guide and write up what I had done in a lot of detail.

@Eric-Guo
Copy link
Contributor

Sorry, my mistake, a random change in previous comment is very like claude 4 behavies....😂

@infinityrobot
Copy link
Contributor Author

Haha, no worries, Claude definitely does that! I had just added that small fix as I was having the same problem raised in #270 and couldn't run the model, alias or docs rake tasks. Looks like the fix is in main now so have updated to use that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
new provider New provider integration
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants