ParallelSimRunScenario
Robotics scenarios · Implementation: mlsysim/engine/scenario_definitions/robotics/learning.py.
Construct this scenario with keyword inputs. The fields below show the types and defaults declared by this class.
A GPU-parallel on-policy run: robots environments each stepping steps_per_update times per update for updates updates within training_time, at policy_rate.
Notes
Equation: Transitions=robots*steps_per_update*updates; throughput=transitions/training_time.
Assumptions: Uniform update sizes and a throughput representative of the requested extrapolation.
Excludes: Reset overhead changes, simulator fidelity, and future hardware speedups.
References
Derivation: Each update contains environment count times steps per environment. Multiplying by update count gives transitions; dividing by wall time gives aggregate throughput, while dividing by policy frequency gives aggregate simulated experience.
Examples
The named registry scenario supplies the stated inputs; this is a modeled calculation, not a new measurement.
>>> from mlsysim import Scenarios
>>> scenario = Scenarios.LeggedGymQuadrupedRun
>>> scenario.batch
98304Attributes
| Name | Description |
|---|---|
| batch | Policy transitions per update: environment count times steps per environment. |
| policy_rate | |
| robots | |
| samples | Total policy transitions: batch transitions times update count. |
| simulated_experience | Aggregate environment experience in days: total transitions divided by policy frequency; not wall-clock duration. |
| steps_per_update | |
| throughput | Policy transitions per second; if training_time is an upper bound, this rate is a lower bound. |
| training_time | |
| updates |
Methods
| Name | Description |
|---|---|
| time_for | Estimated wall-clock minutes for a supplied number of policy transitions at this run’s aggregate throughput. |
time_for
engine.scenario_types.ParallelSimRunScenario.time_for(steps)Estimated wall-clock minutes for a supplied number of policy transitions at this run’s aggregate throughput.
Equation: (steps / self.throughput).to(‘minute’)
robots
Type: int.
Declared default: required.
steps_per_update
Type: int.
Declared default: required.
updates
Type: int.
Declared default: required.
training_time
Type: Quantity.
Declared default: required.
policy_rate
Type: Quantity.
Declared default: required.
batch
engine.scenario_types.ParallelSimRunScenario.batch
Policy transitions per update: environment count times steps per environment.
Equation: self.robots * self.steps_per_update
samples
engine.scenario_types.ParallelSimRunScenario.samples
Total policy transitions: batch transitions times update count.
Equation: self.batch * self.updates
throughput
engine.scenario_types.ParallelSimRunScenario.throughput
Policy transitions per second; if training_time is an upper bound, this rate is a lower bound.
Equation: (self.samples / self.training_time).to(‘1/s’)
simulated_experience
engine.scenario_types.ParallelSimRunScenario.simulated_experience
Aggregate environment experience in days: total transitions divided by policy frequency; not wall-clock duration.
Equation: (self.samples / self.policy_rate).to(‘day’)