Search results

Quantization for LLM models with GQA and bias has not been implemented. Is there any workaround?

feature-request

Status: Open.

#60 In aws-neuron/neuronx-distributed;

· lifelongeeek opened on Oct 2, 2025

Is it possible to run Llama-3.1-8B-Instruct on a trn1.2xlarge instance?

bugSomething isn't workingSomething isn't working

Inference

Trn1

Status: Open.

#53 In aws-neuron/neuronx-distributed;

· dinghongsong opened on Aug 21, 2025

Llama3.1 model requires <code>transformers==4.31.0</code> and current NxD llama3 test case is not compatible with the version.

bugSomething isn't workingSomething isn't working

Training

Trn1

Status: Open.

#40 In aws-neuron/neuronx-distributed;

· KeitaW opened on Jan 26, 2025

Create release tag

enhancementNew feature or requestNew feature or request

Status: Open.

#38 In aws-neuron/neuronx-distributed;

· mhuguesaws opened on Jan 21, 2025

Error in NXD LLama Inference

documentationImprovements or additions to documentationImprovements or additions to documentation

Inference

Status: Open.

#32 In aws-neuron/neuronx-distributed;

· EmilyWebber opened on Oct 4, 2024

Potential bugs for the llama inference example

bugSomething isn't workingSomething isn't working

Inference

Status: Open.

#30 In aws-neuron/neuronx-distributed;

· yinsong1986 opened on Sep 18, 2024

<code>XLA_DISABLE_FUNCTIONALIZATION=0</code> with ZeRO-1 diverges for Mistral on NxD

bugSomething isn't workingSomething isn't working

Training

Status: Open.

#26 In aws-neuron/neuronx-distributed;

· michaelbenayoun opened on Jul 17, 2024

Error: MPMD detected but reload is not supported yet for neuron distributed environment with EAGER DEBUG MODE

bugSomething isn't workingSomething isn't working

Training

Status: Open.

#21 In aws-neuron/neuronx-distributed;

· wenboqian opened on May 4, 2024

Clean up of old checkpoints is crashing

bugSomething isn't workingSomething isn't working

Training

Status: Open.

#20 In aws-neuron/neuronx-distributed;

· evellasques opened on Apr 25, 2024