Skip to content

Issues: NVIDIA/TensorRT-LLM

Beta
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

627 Open 1,958 Closed
627 Open 1,958 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Assigned to nobody Loading
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

[AutoDeploy] Expose logit_softcap in torch attention reference ops AutoDeploy
#4881 opened Jun 3, 2025 by lucaslie
@nvchenghaoz
[AutoDeploy] Expose logit_softcap parameter in flashinfer_attention AutoDeploy
#4880 opened Jun 3, 2025 by lucaslie
@nvchenghaoz
[Profiling] Is there any good way to profile the CPU runtime of trtllm?
#4866 opened Jun 3, 2025 by JuniperHugh
[AutoDeploy] Investigate DemoLLM Token Generation AutoDeploy bug Something isn't working
#4841 opened Jun 2, 2025 by lucaslie
@Fridah-nv
Title: KeyError: 'gemma3' error in GemmaConfig.from_hugging_face when converting Gemma 3 model bug Something isn't working triaged Issue has been triaged by maintainers
#4825 opened Jun 2, 2025 by bebilli
2 of 4 tasks
@brb-nv
1
Driver crash during warmup of DeepSeek-R1-FP4 bug Something isn't working
#4816 opened May 31, 2025 by pathorn
1 of 4 tasks
3
The output of Gemma 3 4B for TensorRT and Transformers is not the same, even when using float32 bug Something isn't working triaged Issue has been triaged by maintainers
#4815 opened May 31, 2025 by Alireza3242
1 of 4 tasks
@brb-nv
1
[Bug] Users need to add cuda_graph_max_batch_size=0 to avoid crash when config from extra-llm-api-config.yml bug Something isn't working
#4811 opened May 30, 2025 by chang-l
4 tasks
@Superjomn
2
Inconsistent output_log_probs with concurrent requests at beam_width and max_batch_size ≥ 2 bug Something isn't working
#4793 opened May 30, 2025 by wonjkim
4 tasks
Gemma-2 Style Attention Pattern Matching with logit softcap AutoDeploy
#4789 opened May 30, 2025 by lucaslie
@lucaslie @nvchenghaoz
1
llmapi usage: how to add callback after each step and embedding table in LLM.generate_async
#4788 opened May 30, 2025 by bnuzhanyu
Feature support: eagle multimodal inputs feature request New feature or request. This includes new model, dtype, functionality support
#4787 opened May 30, 2025 by liyi-xia
@laikhtewari
1
Patch for create_causal_mask() function in transformers masking_utils.py AutoDeploy
#4783 opened May 30, 2025 by sugunav14
1
Priority for stop_token_ids vs. min_tokens
#4776 opened May 29, 2025 by jkbeavers
Retouch ccp executor example cmake to enable or not multi device building
#4770 opened May 29, 2025 by WilliamTambellini
@tongyuantongyu
3
How is the performance of the model with pytorch as the backend Investigating Performance TRTLLM model inference speed, throughput, efficiency. Latency, benchmarks, regressions, opts. triaged Issue has been triaged by maintainers
#4745 opened May 29, 2025 by oppolll
@byshiue @hypdeb
2
Test gemma models after upgrade to latest transformers AutoDeploy
#4740 opened May 28, 2025 by sugunav14
@sugunav14
Phi4-multimodal rotary_scaling is RotarySaclingType.none
#4728 opened May 28, 2025 by jl749
DeepSeek-R1-FP4 crashes when MTP is enabled bug Something isn't working
#4708 opened May 27, 2025 by Shang-Pin
1 of 4 tasks
@lfr-0531
2
[AutoDeploy] Investigate torch.export as a preprocessing step to InferenceOptimizer AutoDeploy
#4704 opened May 27, 2025 by lucaslie
@lucaslie
PluginConfig object has no attribute _paged_kv_cache question Further information is requested
#4701 opened May 27, 2025 by Yoloex
4 tasks
1
Does qwen3 support lora inference?
#4696 opened May 27, 2025 by oppolll
@byshiue @shaharmor98
2
[AutoDeploy] Weight Fusion Revisited AutoDeploy bug Something isn't working
#4674 opened May 27, 2025 by lucaslie
@lucaslie @suyoggupta
1
Error in dispatch_kv_cache_events_thread: 'NoneType' object has no attribute 'new_value' bug Something isn't working
#4666 opened May 26, 2025 by Rb-Ach
4 tasks
Support for Devstral with pytorch backend triaged Issue has been triaged by maintainers
#4653 opened May 26, 2025 by ankitmaurya001
@brb-nv
1
ProTip! Follow long discussions with comments:>50.