logo

Software Documentation (Version 1.7.1)

  • Software Release Notes
  • Documentation Updates

Cerebras Basics

  • How Cerebras Works
  • Types of CS Systems Installations
  • Cerebras Execution Modes
  • ML Workflow on Cerebras
  • Glossary

Getting Started with Cerebras Wafer-Scale Cluster

  • Software Requirements and Dependencies
  • Pytorch: Getting Started
    • Running PyTorch Models
  • TensorFlow: Getting Started
    • Running TensorFlow Models
  • Cerebras job scheduling and monitoring

Getting Started with Original Cerebras Installation

  • Software Requirements and Dependencies
  • PyTorch: Getting Started
    • Running Small to Medium Models (Pipelined Execution)
  • TensorFlow: Getting Started
    • Running Small to Medium Models (Pipelined Execution)

Scripts and Templates (Original Cerebras Installation Only)

  • The run.py Template
  • The csrun_cpu Script
  • The csrun_wse Script

Model Zoo Repository

  • Cerebras Model Zoo
  • GPU Requirements for Running Cerebras ModelZoo

Cerebras Advanced

  • DataLoader Overview
    • PyTorch DataLoader
    • Basics of TFRecords
  • Data Formats
  • Dynamic Loss Scaling
  • Performance Optimization Practices
  • Multi-Replica Data Parallel Training
  • Sparsity
  • Checkpoint Formats

Develop with TensorFlow (Pipelined Mode Only)

  • Workflow for TensorFlow on CS
  • Port TensorFlow to Cerebras
    • Keras Model to CerebrasEstimator
    • Using the CerebrasEstimator
    • The CerebrasEstimator Interface
    • TensorFlow Estimator to CerebrasEstimator
    • Limitations of the CerebrasEstimator
  • Prepare Input
    • Multi-Worker Input Pipeline
    • Sharding For the CS system
    • The CS_AUTOTUNE
    • Optimize Input Function
  • TensorFlow Dynamic Loss Scaling
  • Compile on CPU
  • Train, Eval, and Predict
  • Early Stopping
  • Multiple Models
    • Multi-model Inference
  • TensorFlow Variable Sequence Length
  • Using TensorBoard
  • Best Practices
  • Supported TensorFlow Layers
    • tf package
      • tf.layers.ActivationLayer module
      • tf.layers.AddLayer module
      • tf.layers.AttentionLayer module
      • tf.layers.BaseLayer module
      • tf.layers.Conv2DLayer module
      • tf.layers.Conv2DTransposeLayer module
      • tf.layers.CrossEntropyFromLogitsLayer module
      • tf.layers.DenseLayer module
      • tf.layers.DropoutLayer module
      • tf.layers.EmbeddingLayer module
      • tf.layers.FeedForwardNetwork module
      • tf.layers.FeedForwardNetworkV2 module
      • tf.layers.Input module
      • tf.layers.LayerNormalizationLayer module
      • tf.layers.MaxPool2DLayer module
      • tf.layers.PoolerLayer module
      • tf.layers.PoolerLayerV2 module
      • tf.layers.PositionEmbeddingLayer module
      • tf.layers.PrePostProcessWrapper module
      • tf.layers.ReshapeLayer module
      • tf.layers.SegmentEmbeddingLayer module
      • tf.layers.SharedWeightsDenseLayer module
      • tf.layers.SoftmaxLayer module
      • tf.layers.SquaredErrorLayer module

Develop with PyTorch (Pipelined Mode Only)

  • Workflow for PyTorch on CS
  • Porting PyTorch Model to CS
  • Taking checkpoints off a Cerebras System
  • PyTorch Runners
  • PyTorch Variable Tensor Shape
  • Limitations of PyTorch on Cerebras
  • Cerebras PyTorch Layer API
    • Supported PyTorch Optimizers
    • Supported PyTorch Learning Rate Schedulers
    • modelzoo.common.pytorch.layers.MultiheadAttention
    • modelzoo.common.pytorch.layers.TransformerDecoderLayer
    • modelzoo.common.pytorch.layers.TransformerDecoder
    • modelzoo.common.pytorch.layers.TransformerEncoderLayer
    • modelzoo.common.pytorch.layers.TransformerEncoder

Compiler Reports (Pipelined Mode Only)

  • Input Function Report
  • Compile Report
  • Incremental Compile

Extensions (Original Cerebras Installation Only)

  • Adding Custom Packages To cbcore Container
Theme by the Executable Book Project

Port TensorFlow to Cerebras

Port TensorFlow to Cerebras¶

This section describes how to port your TensorFlow code to Cerebras.

  • Keras Model to CerebrasEstimator
    • Using the KerasModelToCerebrasEstimator
  • Using the CerebrasEstimator
    • Calling the CerebrasEstimator
    • Callback input function
    • Callback model function
    • Setting the runtime configuration
  • The CerebrasEstimator Interface
    • Syntax
    • Arguments
    • Methods
  • TensorFlow Estimator to CerebrasEstimator
    • Step 1: Model function
    • Step 2: Input function
    • Step 3: Use CerebrasEstimator
    • Step 4: Edit RunConfig
    • Step 5: Ensure mixed precision
  • Limitations of the CerebrasEstimator
    • Model function limitations
    • Input function differences
    • Input function limitations
    • Config differences
    • Compilation differences

previous

Workflow for TensorFlow on CS

next

Keras Model to CerebrasEstimator

© Copyright 2022, Cerebras Systems.
Last updated on Mar 20, 2023, 5:45:24 PM.