LiteLLM: extra_body with guardrails is not taken into account #1324
Description
Bug description
I am using LiteLLM and set up some input guardrails using the omni-moderation-latest model.
When I set the extra_body parameter in the model settings like this:
extra_body={
"guardrails": [
"openai-moderation-pre"
],
}
it will not be send to LiteLLM and thus will not trigger the guardrail.
Sending a request with curl containing the guardrails key and values in the body does work.
Putting any other values, lists, object into extra_body does work (they are shown in the logs).
Debug information
- Agents SDK version: 0.2.4
- Python version: Python 3.13.5
- LiteLLM version: v1.74.7-stable.patch.2
Repro steps
- Setup litellm with gpt-4o and the following guardrail settings (https://docs.litellm.ai/docs/proxy/guardrails/openai_moderation):
guardrails:
- guardrail_name: "openai-moderation-pre"
litellm_params:
guardrail: openai_moderation
mode: "during_call"
- Create a minimal agent with the extra_body parameter:
agent = Agent(
name="Agent",
instructions="You are a helpful assistant",
tools=[],
model=LitellmModel(model="gpt-4o-mini", base_url="<base_url>", api_key="<key>"),
model_settings=ModelSettings(
extra_body={
"guardrails": [
"openai-moderation-pre"
],
}
)
)
result = asyncio.run(Runner.run(
agent,
input="Hello",
))
- observe the logs in litellm and inspect the request body. It will show something like this:
{
"model": "gpt-4o-mini",
"messages": [
{
"role": "system",
"content": "You are a helpful assistant"
},
{
"role": "user",
"content": "Hello"
}
]
}
as you can see there is no value containing the guardrails.
Expected behavior
The guardrail values in the body should be send to LiteLLM as well