modelzoo.common.pytorch.run_cstorch_eval_harness.CSEvalHarnessAdapter#

class modelzoo.common.pytorch.run_cstorch_eval_harness.CSEvalHarnessAdapter[source]#

Bases: lm_eval.base.LM

Initializes cstorch components required for executing eval harness on appliance, overriding the loglikelihood method that performs this execution. Subclasses the base LM class that is accepted by the main evaluator.evaluate method of the EEH script.

Methods

`configure_cluster`	Sets up CS cluster config for the run.
`greedy_until`
`init_model`	Initializes the model for the cstorch API.
`loglikelihood`	This is the overriden method of the LM base class in EEH script.
`loglikelihood_rolling`

__call__(*args: Any, **kwargs: Any) → Any#: Call self as a function.

__init__(params, model_fn, input_fn, data_fn)[source]#

static __new__(cls, *args: Any, **kwargs: Any) → Any#

configure_cluster(params)[source]#: Sets up CS cluster config for the run.

init_model(params, model_fn)[source]#: Initializes the model for the cstorch API.

loglikelihood(requests) → List[Tuple[float, bool]][source]#

This is the overriden method of the LM base class in EEH script. This method preprocesses the raw text requests, generates the data samples to be consumed by the GPT2 model, and executes the data on the appliance.

Parameters: requests – list of raw text context, continuation pair of tuples
Returns: list of size len(requests) comprising post-processed results tuple as expected by the EEH script

modelzoo.common.pytorch.run_cstorch_eval_harness.setup_hf_env_vars

modelzoo.common.pytorch.run_cstorch_flow