Profiling the Executor

We provide tools through the DataExecutor to profile its performance during the run.

Currently, the supported activities can be profiled as follows:

Activity	Description
rate	Client side smoothed samples/second of all the samples added since last queried
global_rate	Non-smoothed samples/second since the beginning of when the executor context was entered

These activities can be queried via their names through the DataExecutor’s profiler attribute.

For example,

executor = cstorch.utils.data.DataExecutor(...)
...
print(f"Rate: {executor.profiler.rate()}")
print(f"Rate: {executor.profiler.global_rate()}")

previous

Restartable DataLoaders

next

Evaluation Metrics