cerebras.modelzoo.tools.checkpoint_converters.streaming_checkpoints.StreamingCSWriterView#
- class cerebras.modelzoo.tools.checkpoint_converters.streaming_checkpoints.StreamingCSWriterView(checkpoint_file, state, prefix=None)[source]#
Bases:
objectStreamingCSWriterView allows for checkpoints with arbitrarily nested dictionaries/lists to be written in a streaming (incremental) manner by offering a “view” into a StreamingCSWriter. For example, in a checkpoint with the structure {“model”: {<model state>}}, we can obtain a view into the model state via checkpoint[“model”]. This view has state <model state> and prefix [“model”]. The view acts like a dict (offers __getitem__, __setitem__, etc operations) which incrementally saves/loads from an H5 checkpoint under the hood.
- Parameters
checkpoint_file – Path to H5 checkpoint
state – (Sub)state dictionary corresponding to the current view of the checkpoint.
prefix – Chain of keys that were accessed in the checkpoint that yielded the current view
Methods
getitemskeyspopvalues