-
-
Notifications
You must be signed in to change notification settings - Fork 190
Added proper handling of streaming error responses across both Faraday V1 and V2 #273
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #273 +/- ##
==========================================
+ Coverage 89.71% 90.34% +0.62%
==========================================
Files 75 75
Lines 2811 2817 +6
Branches 555 559 +4
==========================================
+ Hits 2522 2545 +23
+ Misses 289 272 -17 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These changes to the spec file are quite different than our current style for spec files. We also assume that cassettes can be removed, and in fact rake vcr:record[anthropic]
would remove yours. I'd suggest to mock the messages coming back from the API instead.
It would also be great to test with other providers too.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, I've removed the VCR cassettes and replaced it with stubbed_requests.
I've also added support for other providers (other than Bedrock)
I've verified the error format used in the mocks against Anthropic and Open AI apis, so I'm reasonably confident in that mocking. I don't have access to Bedrock right now, and the error handling seems somewhat different as far as I can tell, so I haven't added support for that in the tests yet. That would ideally be done by someone with access to Bedrock.
…_llm into fix_handle_error_chunk
What this does
When used within our app, streaming error responses were throwing an error and not being properly handled
It looks like the introduction of support for Faraday V1 introduced this error, as the error handling relies on an
env
that is no longer passed. This should provide a fix for both V1 and V2.One thing to note, I had to manually construct the VCR cassettes, I'm not sure of a better way to test an intermittent error response.
I have also only written the tests against
anthropic/claude-3-5-haiku-20241022
- it's possible other models with a different error format may still not be properly handled, but even in that case it won't error for the reasons fixed here.Type of change
Scope check
Quality check
overcommit --install
and all hooks passmodels.json
,aliases.json
)API changes
Related issues