ML Workflow on Cerebras

ML Workflow on Cerebras

When you are targeting the Cerebras CS system for your neural network jobs, start with the high-level workflow described here.

1

Port your code to CS

2

Prepare input data

3

Compile model

4

Run on the CS system


Familiarize yourself with Cerebras ML workflow:

Port your code to Cerebras-compatible format

For examples of standard models, see our Cerebras Model Zoo. For TensorFlow, see TensorFlow Quick-Start Guide. For PyTorch, see PyTorch Quick-Start Guide.

Prepare input data

Preparing your input data is critical. Due to very high-speed cluster-scale acceleration performed by Cerebras accelerator, your input pipeline must be very fast. You can achieve such high input data throughput by running the input pipeline on multiple CPU nodes simultaneously, all feeding the data to the CS system. This means you must ensure you preprocess the input data by sharding, shuffling, prefetching, interleaving, repeating, batching, etc., in a proper order. Also, make sure to put your input data on a network file system so it can be accessed by all the CPU nodes in the cluster.

Compile model

Compile your model first without running it on the CS system. With this approach, you can optimize your code for your specific CS system early on. Then use the compiled artifacts later when you run this network on the CS system and save time in your workflow.

Run on the CS system

During runtime, the workers in the CPU cluster stream input data to the Cerebras accelerator. The CPU cluster retrieves intermediate and final artifacts from the wafer for tracking progress and review.

> Develop with PyTorch

We recommend you read the Cerebras basics and run quickstart first. When you are ready, start your PyTorch to CS journey here.

> Develop with TensorFlow

We recommend you read the Cerebras basics and run quickstart first. When you are ready, start your TensorFlow to CS journey here.