cerebras.modelzoo.data_preparation.data_preprocessing.data_preprocessor.get_compression_factor#
- cerebras.modelzoo.data_preparation.data_preprocessing.data_preprocessor.get_compression_factor(filename)[source]#
Calculate and return the compression factor based on a file’s extension.
- Parameters
filename (str) – The name of the file.
- Returns
- Compression factor. Returns 3 for all compressed and parquet formats,
otherwise returns 1 for uncompressed formats.
- Return type
int