Feat: Added Flex tier for OpenAI provider - Half the price for effectively same service

[mechanicmuthu]

My first proper submission, please review and pull. Thanks.

Related GitHub Issue #2755

Closes: #2755 Feature request - o4-mini and o3: allow for service_tier parameter #2755
alexzeitgeist started this conversation in Feature Requests

Roo Code Task Context (Optional)

Did not use Roo code to do it.

Description

Added flex tier options in OpenAI provider

Test Procedure

Added mock model info with flex pricing in cost.spec.ts

Pre-Submission Checklist

New parameter Service Tier is added to the API providers OpenAI list in case a compatible model Presently gpt-5, o3, o4-mini are supported models as per Flex documentation

Screenshots / Videos

Documentation Updates

GUI is self explanatory.
No documentation updates are required.

Additional Notes

None.

Get in Touch

None

---

Important

Adds a 'flex' service tier for OpenAI models with discounted pricing, updates API and UI components, and includes tests and localization updates.

• Behavior:
  • Adds flex service tier option to OpenAI provider in openai.ts and openai-native.ts.
  • Updates calculateApiCostOpenAI() in cost.ts to apply flex pricing when selected.
  • Adds serviceTier parameter to API requests in openai.ts and openai-native.ts.
• UI:
  • Adds ServiceTier component in ServiceTier.tsx to select service tier in UI.
  • Updates ApiOptions.tsx to include ServiceTier component.
  • Updates ModelInfoView.tsx to display effective pricing based on selected tier.
• Tests:
  • Adds tests in cost.spec.ts to verify flex pricing logic.
• Localization:
  • Updates localization files to include translations for service tier options.
• Misc:
  • Updates formatPrice() in formatPrice.ts to allow three decimal places.

^{This description was created by for 57ed4e2. You can customize this summary. It will automatically update as commits are pushed.}

[roomote]

Thank you for your contribution! I've reviewed the changes for adding Flex tier support to the OpenAI provider. While the core functionality looks good, there are several critical issues that need to be addressed before merging:

Critical Issues (Must Fix):

1. Debug console.log statements left in production code - Multiple debug console.log statements need to be removed:

   • src/api/providers/openai.ts (multiple locations where serviceTier is set)
   • src/api/providers/openai-native.ts (multiple locations)
   • webview-ui/src/components/settings/ServiceTier.tsx line 33

2. Incorrect model names and dates - The PR references future models like 'gpt-5-2025-08-07' and dates like '14th Aug 2025' which don't exist. These should be corrected to actual OpenAI models that support flex tier (e.g., gpt-4o-mini according to OpenAI docs).

Important Suggestions (Should Consider):

3. Missing error handling - When the service_tier parameter is sent to the API, there's no error handling for cases where the API might reject this parameter for unsupported models.

4. Incomplete test coverage - While tests were added for flex pricing calculation in cost.spec.ts, there are no tests for the ServiceTier UI component or the API integration.

5. Hardcoded model list - The FLEX_COMPATIBLE_MODELS array in ServiceTier.tsx will require manual updates. Consider fetching from configuration or API.

Minor Improvements:

6. formatPrice change scope - Changing maximumFractionDigits from 2 to 3 in formatPrice.ts affects the entire app. Consider if this is intended or if a separate function is needed.

7. Comment formatting - The comment in ServiceTier.tsx line 32 could be improved for clarity.

Positive feedback:

• Good implementation of the flex pricing logic in cost.ts
• Comprehensive localization updates
• Good test coverage for the cost calculation logic

[mechanicmuthu]

Working now.

Please review and accept the PR.

[mrubens]

I like this idea! Do other providers have something similar? If not, I would probably make this option OpenAI specific (like openAIServiceTier or something)

[mechanicmuthu]

Do other providers have something similar?

Other than OpenAI, Anthropic does this service_tier but does not they do not have different rates for standard and priority. They provide priority (less 429 errors) in lieu of minimum spend commitments from customers.

message = client.messages.create( model="claude-opus-4-1-20250805", max_tokens=1024, messages=[{"role": "user", "content": "Hello, Claude!"}], service_tier="auto" # Automatically use Priority Tier when available, fallback to standard )

Groq also exposes service_tier parameter in their api. In this case, they say in their docs:

Flex processing is available for all models to paid customers only with 10x higher rate limits compared to on-demand processing. While in beta, pricing will remain the same as our on-demand tier.

[daniel-lxs]

Thank you for implementing this. I think we can use the existing tiers property in ModelInfo instead of adding a new flexPrice property. For example:

tiers: [{ name: 'flex', inputPrice: 1.5, outputPrice: 7.5 }]

There’s no error handling when setting the service_tier parameter in openai.ts and openai-native.ts. If the API rejects this parameter for unsupported models, it could cause issues. Something like this might be safer:

try {
  (requestOptions as any).service_tier = this.options.serviceTier
} catch (error) {
  // Continue without service_tier
}

The FLEX_COMPATIBLE_MODELS array will need manual updates as new models are added. Instead, this could come from modelInfo.supportsFlexTier. Also, since only OpenAI currently has flex pricing, mrubens’ suggestion about making the field name more specific, like openAiServiceTier, makes sense.

Lastly, the formatPrice change from 2 to 3 decimal places affects all price displays in the app. Was that intentional, or did you only need 3 decimal places for cache pricing?

[mechanicmuthu]

Had to touch one line in the Anthropic and Gemini apis to reuse 'tiers'. (context size is not relevant for flex processing so had to make it optional).

contextWindow: tier.contextWindow. to contextWindow: tier.contextWindow ?? info.contextWindow,

Other suggestions done.