Skip to content

Compare PiPPy with MegatronLM performace #309

Open
@pbelevich

Description

@pbelevich

Possible scenarios:

  • Compare plain PiPPy with plain Megatron
    • May be hard to implement
  • Compare HF MegatronBERT with PiPPy-wrapped HF BERT
    • Choose a task (ForMaskedLM, ForCausalLM, ForNextSentencePrediction, ForPreTraining, ForSequenceClassification, ForMultipleChoice, ForTokenClassification, ForQuestionAnswering)
    • May depend on Mirror HF examples with PiPPy #301
    • Investigate if it's a fair comparison

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions