Boosting Model performance

• Measure throughput of your model
  • Out-of-the-box throughput metrics from Cerebras Model Zoo
  • Ephemeral throughput spikes after checkpointing
  • Throughput in Weight Streaming execution
• Training with number of tokens loss scaling
  • Background
  • How to enable
• Train with gradient accumulation
  • How to enable
  • Micro-batch size setting in YAML params
  • Known issues and limitations
• Automatic batch exploration
  • Overview
  • Procedure
  • Expected Output
  • Implementation notes

previous

Running Eleuther AI’s Evaluation Harness

next

Measure throughput of your model