cerebras.modelzoo.data.common.input_utils.shard_list_of_chunks_contiguous#

cerebras.modelzoo.data.common.input_utils.shard_list_of_chunks_contiguous(input_list_of_chunks, worker_id, num_workers)[source]#

Shards a list of chunks by distributing contiguous segments of each chunk across shards. If the chunk’s length is not divisible by the number of workers, the remainder elements are spread across a subset of the workers such that each worker in the subset receives 1 extra element.

Parameters

  • input_list (list of tuples) – list of chunks to shard. List should be of format [… (chunk_i, length_of_chunk_i), …]

  • worker_id (int) – index of shard to return

  • num_workers (int) – number of shards to create

Returns

a list of the same length as input_list of the format: [… (chunk_i, shard_start_index_i, shard_length_i), …]

Return type

worker_id’s shard

previous

cerebras.modelzoo.data.common.input_utils.shard_list_interleaved

next

cerebras.modelzoo.data.common.input_utils.task_id