cerebras.modelzoo.data.common.input_utils.cluster_config#

cerebras.modelzoo.data.common.input_utils.cluster_config()[source]#
Returns (ClusterSpec, TaskSpec). The TaskSpec contains the following fields:

  • rank: the global rank of the current worker

  • local_rank: the rank of the current worker among workers who feed

    the same system as the current worker

  • wse_id: the index of the system that the current worker is

    associated with

The ClusterSpec contains the following fields:

  • tasks: a list of TaskSpecs for each task running on the cluster

  • rank: the rank of the current process’s task in the cluster

  • num_csx: the number of CSX systems in the cluster

  • num_workers_per_csx: the number of worker tasks per CSX

If the current job is running on GPU instead of CS system, then the ranks and world sizes in the returned TaskSpec will be set to the GPU rank and world size.

previous

cerebras.modelzoo.data.common.input_utils.check_sharding_sanity

next

cerebras.modelzoo.data.common.input_utils.get_data_for_task