Skip to content

Test gemma models after upgrade to latest transformers #4740

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
sugunav14 opened this issue May 28, 2025 · 0 comments
Open

Test gemma models after upgrade to latest transformers #4740

sugunav14 opened this issue May 28, 2025 · 0 comments
Assignees
@sugunav14
Labels
AutoDeploy

Comments

@sugunav14
Copy link
Collaborator

Gemma3 models fail during export stage due to a .item() call on a meta tensor in modeling_gemma.py file. The latest transformers has a different implementation, but this version of transformers currently break tensorrt_llm. Need to test Gemma models with auto deploy once tensorrt_llm upgrades to the latest transformers.
@sugunav14 sugunav14 self-assigned this May 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sugunav14 sugunav14

Labels
AutoDeploy
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@brb-nv @sugunav14