Skip to content

Add CLI options for backend args (like headers and verify) #230

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 29, 2025

Conversation

kdelee
Copy link
Contributor

@kdelee kdelee commented Jul 15, 2025

This PR adds the ability to configure custom request headers and control SSL certificate verification when running benchmarks.

  • The OpenAIHTTPBackend now supports passing custom headers and a verify flag to disable SSL verification.
  • Headers are now merged with the following precedence: CLI arguments (--backend-args), scenario file arguments, environment variables, and then default values.
  • Headers can be removed by setting their value to null in the --backend-args JSON string.
  • The --backend-args help text has been updated with an example of how to use these new features.
  • New documentation has been added for the CLI, configuration options, and supported data formats.
  • Unit tests have been added to verify the new header and SSL verification logic, as well as the CLI argument parsing.

This provides a way to benchmark targets that require custom authentication, other headers, or use self-signed SSL certificates.

@kdelee kdelee marked this pull request as draft July 15, 2025 01:02
Copy link
Collaborator

@sjmonson sjmonson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey, thanks for the contribution. For backend specific arguments, please nest them under --backend_args and handle any parsing within the backend's __init__. Please also replace all occurrences of ssl with cert or some other protocol neutral term.

@kdelee
Copy link
Contributor Author

kdelee commented Jul 16, 2025

Ok thanks for feedback, will look into those changes

@kdelee kdelee changed the title [draft] Add CLI options for auth and ssl verify Add CLI options for backend args (like headers and verify) Jul 16, 2025
@kdelee kdelee marked this pull request as ready for review July 16, 2025 23:37
@kdelee kdelee requested a review from sjmonson July 16, 2025 23:37
@kdelee kdelee force-pushed the ssl_verify_auth_header branch from 037cd42 to e816a81 Compare July 16, 2025 23:39
@kdelee
Copy link
Contributor Author

kdelee commented Jul 17, 2025

@sjmonson pushed some updates, looks like workflow needs approval

Copy link
Collaborator

@sjmonson sjmonson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks pretty good, just a few design nits. To fix markdown linting run python -m mdformat <files>

@kdelee kdelee force-pushed the ssl_verify_auth_header branch from a9f92e1 to d3858ef Compare July 29, 2025 18:39
@kdelee kdelee requested a review from sjmonson July 29, 2025 18:48
Copy link
Collaborator

@sjmonson sjmonson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM but commits need signoff

This commit enhances the OpenAIHTTPBackend to allow for more flexible
configuration via the `--backend-args` CLI option and environment
variables.

Key changes include:
- Support for passing custom request headers. Headers are merged from the
  CLI, scenario files, and environment variables, with the CLI taking
  highest precedence.
- Headers can be removed by setting their value to `null` in the
  `--backend-args` JSON string.
- The `verify` flag for disabling SSL certificate verification is now
  correctly handled.
- New documentation for the CLI, configuration, and data formats.
- New unit tests for the backend logic and CLI argument parsing.

Signed-off-by: Elijah DeLee <[email protected]>
@kdelee kdelee force-pushed the ssl_verify_auth_header branch from d3858ef to 85358d0 Compare July 29, 2025 21:47
@sjmonson sjmonson merged commit bc49ac8 into vllm-project:main Jul 29, 2025
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants