cerebras.modelzoo.data_preparation.nlp.hdf5_preprocessing.convert_dataset_to_HDF5.write_hdf5_file#

cerebras.modelzoo.data_preparation.nlp.hdf5_preprocessing.convert_dataset_to_HDF5.write_hdf5_file(file_path, dataset_name, data, n_examples, chunks, dtype='i4', compression='gzip')[source]#

Write data to HDF5 file.

Parameters

  • dataset_name (string) – HDF5 dataset name

  • file_path (string) – HDF5 file path.

  • data (numpy array) – Input features and labels that will be written to HDF5.

  • n_examples (int) – Number of examples that will be written in the file.

  • chunks (tuple or bool) – Chunk shape, or True to enable auto-chunking.

  • dtype (string) – Data type for the HDF5 dataset.

  • compression (string) – Compression strategy.

previous

cerebras.modelzoo.data_preparation.nlp.hdf5_preprocessing.convert_dataset_to_HDF5.convert_dataset_to_HDF5

next

cerebras.modelzoo.data_preparation.nlp.hdf5_preprocessing.create_hdf5_dataset